topic Re: How do you store a feature service locally for edits? in ArcGIS REST APIs and Services Questions

How do you store a feature service locally for edits?

stefanlivingston2 — Wed, 17 Dec 2014 20:15:23 GMT

I am creating an app for a local fire department. I have acquired fire hydrant location data from:

http://gisweb.wsscwater.com/ArcGIS/rest/services/FireHydrant/FireBook/MapServer/0

I have imported it as a feature service in ArcGIS Desktop, however I cannot edit the data. I am trying to make a copy (maybe as a shp file) and start with that data to edit it.

Is this possible? Or is there a workaround since I cannot access it (I have asked multiple times but they have not granted as yet)?

Re: How do you store a feature service locally for edits?

TobiasFimpel1 — Thu, 18 Dec 2014 02:18:53 GMT

Stefan, the map service you are referencing does not appear to be editable. You will need to ask the server admin to enable the editing capabilities and assuming the data is stored in an enterprise RDMS and not in a file geodatabase this will spin up a feature service with the URL http://gisweb.wsscwater.com/ArcGIS/rest/services/FireHydrant/FireBook/FeatureServer/0 .‌ But your server admins will likely want to secure the service with a username & password

Re: How do you store a feature service locally for edits?

XanderBakker — Sun, 12 Dec 2021 01:49:09 GMT

If your purpose is to have a local copy of the data to develop against (not updating the service itself) you could use some python code to extract the data of the map service:

import arcpy
fcout = r'C:\Forum\Hydrants\test01.shp'
url = 'http://gisweb.wsscwater.com/ArcGIS/rest/services/FireHydrant/FireBook/MapServer/0/query?text=&geometry=&geometryType=esriGeometryPoint&inSR=&spatialRel=esriSpatialRelIntersects&relationParam=&objectIds=&where=1%3D1&time=&returnCountOnly=false&returnIdsOnly=false&returnGeometry=true&maxAllowableOffset=&outSR=&outFields=*&f=pjson'
fs = arcpy.FeatureSet()
fs.load(url)
arcpy.CopyFeatures_management(fs, fcout)

Since the number of features is more than 1000 and the maps service is configured to return a max of 1000 feature per request, the code should be adapted to use the object id to query with OBJECTID > highest id from previous request. Create the featureclass in the first step and append for the subsequent queries.

Note this will only provide you a local copy of the data. The Map Service has no methods to update the service.

Re: How do you store a feature service locally for edits?

stefanlivingston2 — Thu, 18 Dec 2014 16:49:37 GMT

What would the subsequent code look like? Can you run and provide the SHP file? I am fairly new to the python world.

Re: How do you store a feature service locally for edits?

XanderBakker — Sun, 12 Dec 2021 01:49:11 GMT

In theory you could use something like the code below, but it is throwing me an error (might be due to the proxy):

def main():
  
    import arceditor
    import arcpy
    import urllib, urllib2, json
  
    target = r'D:\Xander\GeoNet\Hydrants\test02.shp'
    service_url = r"http://gisweb.wsscwater.com/ArcGIS/rest/services/FireHydrant/FireBook/MapServer/0"
  
    if arcpy.Exists(target):
        arcpy.Delete_management(target)
  
    bln_copy = True
    exceededTransferLimit = True
    i = 0
    while exceededTransferLimit:
        i += 1
        if i == 1:
            nextoid = 0
        url = '{0}/query?where=OBJECTID>%3D{1}&outFields=*&returnGeometry=true&f=json'.format(service_url, nextoid)
        print url
  
        params = urllib.urlencode({'f': 'json', 'where': "OBJECTID>={0} AND TIPO_DIRECCION = 'R'".format(nextoid), 'outFields': 'OBJECTID', 'returnGeometry': 'false'})
        req = urllib2.Request("{0}/query".format(service_url), params)
        response = urllib2.urlopen(req)
        jsonResult = json.load(response)
        nextoid = getMaxOID(jsonResult) + 1
  
        if "exceededTransferLimit" in jsonResult:
            exceededTransferLimit = jsonResult["exceededTransferLimit"]
  
        fs = arcpy.FeatureSet()
        try:
            fs.load(url)
  
            cnt = arcpy.GetCount_management(fs).getOutput(0)
            print "Count: " + cnt
  
            if cnt != 0:
                if bln_copy:
                    print " - copy"
                    arcpy.CopyFeatures_management(fs, target)
                    bln_copy = False
                else:
                    print " - append"
                    arcpy.Append_management(fs, target)
        except Exception as e:
            print e
  
        if exceededTransferLimit == False:
            break
  
  
def getMaxOID(jsn):
    max_oid = 0
    if "features" in jsn:
        ftrs = jsn["features"]
        for ft in ftrs:
            if "attributes" in ft:
                att = ft["attributes"]
                if "OBJECTID" in att:
                    oid = att["OBJECTID"]
                    if oid > max_oid:
                        max_oid = oid
    return max_oid
  
  
if __name__ == '__main__':
    main()

... so I did it in a little more manual way and attached the shapefile to this post.

Re: How do you store a feature service locally for edits?

stefanlivingston2 — Sat, 20 Dec 2014 04:25:10 GMT

So very helpful but the unforunately didnt inlcude all the data elements in the original feature server. Any idea on how to solve?

Re: How do you store a feature service locally for edits?

XanderBakker — Mon, 22 Dec 2014 23:35:48 GMT

Hi Stefan,

After we had contact through mail I send you a complete shapefile of the 46+K features. For those of you that may have a similar question I'll include an explanation of the steps that I followed.

What I did was:

Through REST a query for all the ID's of the hydrants
Access the list of ObjectIDs
Sort this list of ObjectIDs
Create chunks with less than 1000 elements
For each chunk (list of sorted ID's), determine the min and max
Query the service for ID's in the range of each min and max.

Now at home, where I don't have an eval proxy to fight against, the code posted earlier seems to work after I added the "NO_TEST" parameter to the append tool (the schema of the featureset was not the same as the shapefile, since long field names are truncated in shapefile/DBF format)

So the advice is, simply write to a fgdb featureclass...

Kind regards, Xander

Re: How do you store a feature service locally for edits?

Jay_Gregory — Wed, 14 Jan 2015 18:20:42 GMT

I have an operational (but still in progress) Python class that scrapes Esri REST endpoints and stores the results locally in a geodatabase (it also creates the feature class too). It can be found here. The documentation should be pretty straightforward. Unfortunately, I haven't solved the issue of getting all the records if the number of records is more than the max the service returns. The only workaround I've built in is being able to specify a series of non-overlapping where queries. Good luck - the class has a lot more work to handle other special cases, but I find it works great for a majority or our needs!