Export Feature Service with Attachments

[ODAMapping2012]

Hi,

I've got a feature service with attachments enabled (it's a point layer with photos on that can be edited in the field and photos recorded using an ipad) that I'm trying to export out of arcgis online.
Every way that I try to export it doesn't include the attachments, is it possible?

Many thanks,

Andy

[Center_City_DistrictPhiladelph]

I just fixed this bug.

Change the 'lastEditDate' to a capital L like this 'LastEditDate' and update your 'maxRecordCount' and it will save. When you go back to view the layer definition it will change this back to the lower case 'lastEditDate' and the error will continue but the changes in the 'maxRecordCount' will be saved.

[RandyWeaver]

Thanks Rich, this was also an issue I encountered and the fix for the "L" on lastEditDate worked for me as well.

[TrilliumLevine1]

For anyone who is still looking for answers on extracting attachments from a secured ArcGIS online-hosted feature service:  I've been working with Brian Brown (who has been helpful byond the call of duty) on developing a script that I can use to download attachments and replicate the entire hosted feature service locally.

Brian's base code can be found on his github site: https://github.com/bgeomapping/arcgis-toolboxes

I made some minor modifications to it, which are included in the attached code:

1. Functionality to create a directory for attachments and another for the spatial data.
2. Functionality for unzipping and renaming the replicated geodatabase.
3. I need this to run daily and delete the download from 'yesterday', so I added functionality for that.
4. I also wanted to have the global id in the file name, and for it to add a file extension to the attachment even if one wasn't included as part of the the file name -- functionality for that is added.
5. As far as I can tell the base code works only for images, but since attachments can be virtually any format, I modified the code to handle any format.
6. Generate a report showing number of attachments downloaded per layer in feature service.

Other things to note: if the specified url ends in FeatureServer, the entire service will be replicated and all attachments for all layers with attachments in the service will be downloaded.  If the specified url ends in FeatureServer/<layer index>, only that layer will be replicated and only attachments for that layer will be downloaded.  Also, the code creates the following subdirectories for attachments:  Layer > Feature OID > attachments, as well as an 'All attachments' folder -- I don't need the 'All attachments' portion, so it's commented out.  This should work in both Python 2.6 and 2.7 (I've tested in 2.6, Brian in 2.7) and is written so that ArcGIS need not be installed on the machine running the script (in other words, no arcpy import).

I hope this helps some of you!!!

- Trill

[MichaelNesius]

I'm trying to use Trill's script.

It works when I monitor the http traffic with Fiddler, but fails with urllib2.URLError: urlopen error 10061 'no connection could be made because the target machine actively refused it' when I run the script without Fiddler open.

I believe Fiddler makes it work because it is essentially acting as a proxy while it is open. Simply running SETX https_proxy http://user:pw@serverIP:port in the command prompt doesn't seem to cut it for me.

Any suggestions?

thanks,

Mike

*My office uses a wpad.dat for the proxy if that matters

[TrilliumLevine1]

Hi Mike,

When you're running SETX you're entering your specific user information as well as server and port info, right?  Sorry if that's a silly question, but just wanted to make sure.  In lieux of using SETX in the command line to reset the environment variables, you could also include urllib2 proxy handlers directly in the get_response def as shown below -- this is probably a more elegant solution:

def get_response(url, query='', get_json=True):

    opener = urllib2.build_opener(

             urllib2.HTTPHandler(),

             urllib2.HTTPSHandler(),

             urllib2.ProxyHandler(

                {'https': 'http://yourUserName:yourPassword@YourProxyIP:ProxyPort',

                   'http': 'http://yourUserName:yourPassword@YourProxyIP:ProxyPort',

    ))

    urllib2.install_opener(opener)

    encoded = urllib.urlencode(query)

    request = urllib2.Request(url, encoded)

    if get_json:

        return json.loads(urllib2.urlopen(request).read())

    return urllib2.urlopen(request).read()

If you don't want to hardcode your info into the proxy handler, you could use this instead, which would allow you to enter your credentials via the command line:

               {'https': 'http://' + sys.argv[1] + ':' + sys.argv[2] + '@YourProxyIP:ProxyPort',

                 'http': 'http://' + sys.argv[1] + ':' + sys.argv[2] + '@YourProxyIP:ProxyPort'}

If you do it this way, don't forget to import the sys module at the top of your script: import sys

You'd then enter the name of the script and your credentials in the python prompt like this:

>>> nameOfScript yourUserName yourPassword

Keep in mind that if you have special characters in your credentials (@, &, etc.) yourUserName and yourPassword need to be surrounded by double quotes.

Hope this works for you, please let me know if you need any more pointers.

- Trill

[MichaelNesius]

That's it (I'm a complete newbie to working with the REST API)! Thanks for the quick reply!

I started to piece it together late yesterday when I came across these suggested edits for working with proxies in another feature service script on github.

I ended up using pacparser on my office's wpad to determine my server ip and port and hardcoded them

os.environ['http_proxy'] = "http://" + pxy_user + ":" + pxy_pass + "@" + pxy_srvr + ":" + pxy_port

thanks again!

[StephanLe_Roux]

Trillium,

I am trying to use your BackupServiceAttachments.py. I am not a python expert, so I am doing what I can to try to get this working. I need to get this script working for my project, it does exactly what I need done.

I have changed the relevant sections like.

serviceTodayString =

serviceYesterdayString =

When I run the script it creates the folder and then stops with the following error.

{u'message': u'Unable to generate token.', u'code': 400, u'details': [u'Invalid username or password.']}

I am not sure where I should be adding my username and password ? There is a few places in the code where username and password gets mentioned.

Can you please give some direction even if you just give me the line where the username and password should be entered.

Kind regards

[TrilliumLevine1]

Hi Stephen,

You need to enter your credentials in the last if clause at the bottom of the script:

if __name__ == "__main__":

    TOKEN = login("yourUserNameHere", "yourPasswordHere")

Let me know if it works...

-Trill

[StephanLe_Roux]

Trill

Thank you for the response.

I have done as suggested above . Still got the error as below. I must be doing something wrong.

I also tried some of the above mentioned proxy setting. I have tried the SETX and I have tried the def get_response as mentioned in your post above.

[TrilliumLevine1]

Hi Stephan,

Strange...can you post your code so I can take a look?  Unfortunately I'm going to be out of town until next Wednesday, so I likely won't be able to get to it until then.

Trill