using a web service to populate sparse data

[EricEagle]

Hi, I am nothing but a wee baby when it comes to python, know some ultra-simple things to make repetitive tasks faster. But I need some help and googling hasn't gotten me any really usable results yet.  Probably I'm just not using the right vocabulary.

I frequently have to go through piles and piles of sparse data and fill in missing values.  Generally this requires querying over the web using a known value or two.  This query service has a REST endpoint and would be a good candidate for automating, I think.

If anyone has any snippets of code that would:

• Take a value from a particular field in a feature class (say, EVENT ID)
• Send that value as an argument to a base URL (for example, http://www.some_gis_service.com/lookup/<EVENT ID value extracted from field>/details)
• Look through the XML return and find a particular tag (for example, <DATE>2014-01-01T03:00:00</DATE>)
• update a field with that value (or some parsing thereof)

Does this make sense?  I'd probably want to build in some sort of dependency or pause between calls so that I don't overwhelm the server.

Thanks for any help!

Eric

[PaulCrickard]

Python for REST.

My blog explains the code in detail here: ESRI REST and Python: pandas.DataFrame | Architecture and Planning

import urllib, urllib2

import simplejson

param = {‘where':’1=1′,’outFields':’*’,’f':’json’}

url = ‘http://coagisweb.cabq.gov/…/APD_Incidents/MapServer/0/query ?’ + urllib.urlencode(param)

rawreply = urllib2.urlopen(url).read()

reply = simplejson.loads(rawreply)

print reply[“features”][0][“attributes”][“date”]

print reply[“features”][0][“attributes”][“CVINC_TYPE”]

[XanderBakker]

If you want some python examples of how to extract features from a REST end point using python, you can find some in this thread: Re: Extract Features From MapServer/ Rest/ Soap/ etc?  Jake Skinner created a toolbox to do this.

There are also references to some python code I wrote to do this:

the simple way:https://community.esri.com/thread/118781#445348‌

or using a little more advanced:https://community.esri.com/thread/118781#445572

You can find an example of updating a feature service through Python and REST here:

Re: UpdateFeature REST API Python Script Unexpected Error‌

Filip Kral mentions an interesting Python client for ArcGIS for Server and this post (Loading ArcGIS REST Service data into Excel ) by Marc Hoogerwerf is worth reading too.

Kind regards, Xander

[EricEagle]

Thanks Xander!  Since reading this I have a script now that uses a dictionary to store results and then query that first before going out and hitting the service.  Thanks for the links,

Eric

[XanderBakker]

Sounds good! If you are willing to share this with the community it would be great.

Kind regards, Xander