Is there a way to find out the maximum permitted batch size for a feature service?

2972
8
02-16-2016 03:56 AM
JamesTreacy_Bagshaw
New Contributor

I have a Map Server with a feature service configured with the MaxRecordCount parameter set to 50000. I want to send modifications (creates, updates or deletes) to this feature service, which I send in batches.

When I send batches of size greater than 100 (even 101!), every request returns with:

{"error":{"code":500,"message":"Unable to complete operation.","details":["Unable to perform updateFeatures operation."]}}

The Map Server logs look like this:

SEVERE: An error occurred.

DEBUG: RequestId: 10808 16/02/2016 10:55:26 45d1eace-fcf7-4d58-9735-2650fddffc8c Error while Performing Edits for GraphicFeatureServer::Update.

SEVERE: An error occurred.

FINE: Update has completed.

SEVERE: An error occurred.

FINE: REST request successfully processed. Response size is 121 characters.

(then my own logging of the response received above).

Batches of size 100 do not encounter this problem (for this particular server).

Is there a way to find out this latent maximum batch size? Or even configure it?

I can handle this case and uses smaller batch sizes in response but ideally, if there is a limit on batch size, I want to query for it rather than sending a request that was never going to work.

I'm using ArcGIS Server 10.3.1.

0 Kudos
8 Replies
JakeSkinner
Esri Esteemed Contributor

Hi James,

How are you sending the batch updates (i.e. python, javascript)?

0 Kudos
JamesTreacy_Bagshaw
New Contributor

Hi Jake,

I'm using a Java HTTP Client to send the updates as JSON.

EDIT: Just realised I hadn't mentioned that I'm using the feature server REST API.

0 Kudos
JakeSkinner
Esri Esteemed Contributor

I'm not sure if there is a limit.  I don't have experience with Java, but I was able to update a feature service with 50,000+ features using Python.  I sent the update in one request and it was success.  Below is the example code I used:

import urllib, urllib2, json, smtplib

username = 'siteadmin'
password = 'siteadmin'

tokenURL = 'http://agsServer.esri.com/arcgis/tokens/'
params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://agsServer.esri.com'}
req = urllib2.Request(tokenURL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
token = data['token']

fsURL = 'http://agsServer.esri.com/arcgis/rest/services/Parcels/FeatureServer/0/query'
params = {'f': 'pjson', 'where': '1=1', 'returnIdsOnly' : 'true'}
req = urllib2.Request(fsURL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
OIDList = sorted(data['objectIds'])

x = 0

while x <= (len(OIDList) -1):
    updatefsURL = 'http://agsServer.esri.com/arcgis/rest/services/Parcels/FeatureServer/0/updateFeatures'
    if x == 0:
        attr = '{"attributes":{"OBJECTID":' + str(OIDList) + ',"PIN":' + str(OIDList) + '}}'
    elif x == (len(OIDList) - 1):        
        attr = attr + ',{"attributes":{"OBJECTID":' + str(OIDList) + ',"PIN":' + str(OIDList) + '}}'
        params = {"features": '[' + attr + ']', 'token': token, 'f': 'json'}
        req = urllib2.Request(updatefsURL, urllib.urlencode(params))
        response = urllib2.urlopen(req)            
    else:
        attr = attr + ',{"attributes":{"OBJECTID":' + str(OIDList) + ',"PIN":' + str(OIDList) + '}}'
    x += 1
JamesTreacy_Bagshaw
New Contributor

Thanks Jake.

I should have clarified this earlier:

I have tried with several other feature servers, and there are no problems with batch sizes of around 200.

It's just this one feature server that seems to behave as if 100 is the hard limit on batch size. (Although I have assumed that was the problem based on what I've seen - seems very likely though.)

My code needs to be able to work for any feature server, so I was hoping there was a way to find out what the limit is for any given feature server and send features in sensible batch sizes accordingly.

0 Kudos
JakeSkinner
Esri Esteemed Contributor

Since this is occurring for a single feature service, I would recommend trying to re-publish the service to see if this remedies the issue. 

0 Kudos
FredSpataro
Occasional Contributor III

Did you ever figure this out?

I'm seeing the same issue... updateFeatures operation.  Fails with the same error messaging if feature array is greater than 100.  ArcGIS Server version 10.6.  

I've created two services off the same table in the database, same result on each.  

The feature class is a "query layer" in MSSQLServer... 

0 Kudos
JamesTreacy_Bagshaw
New Contributor

I don't think we found the root cause.

Instead worked around it by splitting up the updateFeatures batch in response to the error.

0 Kudos
FredSpataro
Occasional Contributor III

thanks.  I've submitted an issue to support.  I can reproduce the setup consistently with a very simple example.  I suspect it's either a bug with query layer sources or a "feature" that's intended to make it harder to work with non-registered geodatabase tables. We'll see.  

0 Kudos