ArcGIS Server custom GP Service error com.esri.arcgis.discovery.json.JSONException

[SrikanthHemadri]

Hi,

I have a Custom GP Service written in JAVA and published as GPService in asynchronous mode in ArcGIS Server 10.7.1. I use it to find version differences for a feature class by providing the global ids of features. When there are huge number of records, I hit the service in batches of 1000 records. This work when the record count is low. But when I hit the service continuously for more than 80k records in batches of 100 (which is total of 80+ different requests continuously), I am getting below error in the ArcGIS Server logs.

com.esri.arcgis.discovery.json.JSONException: Expected a ':' after a key at character 38 of {002FAFF3-E3C1-4D45-848A-2F26BDD58025},{002FB64A-D639-448A-9BA9-BE729FE5122B},....<list of GUIDs continue> at com.esri.arcgis.discovery.json.JSONTokener.syntaxError(JSONTokener.java:454) at com.esri.arcgis.discovery.json.JSONObject.<init>(JSONObject.java:199) at com.esri.arcgis.discovery.json.JSONObject.<init>(JSONObject.java:273) at com.esri.rf.util.RUtil.toJsonValue(RUtil.java:410) at com.esri.rf.util.RUtil.paramsToJson(RUtil.java:368) at com.esri.rf.RRequest.paramsToJson(RRequest.java:1070) at com.esri.client.app.templates.OperationTemplate.gatherParams(OperationTemplate.java:314) at com.esri.client.app.templates.OperationTemplate.execute(OperationTemplate.java:108) at com.esri.client.app.gpserver.GPTaskHandler.submitJob(GPTaskHandler.java:133) at com.esri.client.app.gpserver.GPTaskHandler.service(GPTaskHandler.java:68) at com.esri.rf.RServlet.service(RServlet.java:123) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Unknown Source)

 I can infer from above that there is some issue when the GP Service is trying to parse input or output data to JSON. I need help as to what the reason can be and possible solution.

Thanks,
Srikanth

[SrikanthHemadri1]

Hi All, I am facing this issue again. Need help please.

Below is the stack trace I am getting

Unable to coerce to JSONObject

com.esri.arcgis.discovery.json.JSONException: Expected a ':' after a key at character 38 of {002FAFF3-E3C1-4D45-848A-2F26BDD58025},{002FB64A-D639-448A-9BA9-BE729FE5122B},....<list of GUIDs continue>
at com.esri.arcgis.discovery.json.JSONTokener.syntaxError(JSONTokener.java:454)
at com.esri.arcgis.discovery.json.JSONObject.&lt;init&gt;(JSONObject.java:199)
at com.esri.arcgis.discovery.json.JSONObject.&lt;init&gt;(JSONObject.java:273)
at com.esri.rf.util.RUtil.toJsonValue(RUtil.java:424)
at com.esri.rf.util.RUtil.paramsToJson(RUtil.java:382)
at com.esri.rf.RRequest.paramsToJson(RRequest.java:1083)
at com.esri.client.app.templates.OperationTemplate.gatherParams(OperationTemplate.java:334)
at com.esri.client.app.templates.OperationTemplate.execute(OperationTemplate.java:108)
at com.esri.client.app.gpserver.GPTaskHandler.submitJob(GPTaskHandler.java:133)
at com.esri.client.app.gpserver.GPTaskHandler.service(GPTaskHandler.java:68)
at com.esri.rf.RServlet.service(RServlet.java:123)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Unknown Source)

[ChrisStiwinter1]

We noticed this same error but it seemed to happened when the ESRI sync process ran on our GIS server while staff was trying to download and sync offline map areas for use in ArcGIS FeildMaps.  Not sure if this is related to your issue but the sync is a geoprocessing service running on the server so similar.  After lots of debugging and looking at our setup etc, we determined that the staff devices which were trying to download the map areas were running out of storage space.  Once we corrected that the issue went away and I did not see this error return in our logs...

[ChrisStiwinter1]

We noticed this same error but it seemed to happened when the ESRI sync process ran on our GIS server while staff was trying to download and sync offline map areas for use in ArcGIS FeildMaps.  Not sure if this is related to your issue but the sync is a geoprocessing service running on the server so similar.  After lots of debugging and looking at our setup etc, we determined that the staff devices which were trying to download the map areas were running out of storage space.  Once we corrected that the issue went away and I did not see this error return in our logs... 

[SrikanthHemadri1]

Thankyou very much  @ChrisStiwinter1 for your response. This information gives us a direction to investigate the issue further.