topic Re: Geoprocessing service gets 'Timeout exceeded' after 3 minutes in Python Questions

WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Sat, 11 Dec 2021 06:11:32 GMT

I've published a geoprocessing service and recently noticed that it times out at 3 minutes. I tried updating maxUsageTime but it had no effect. I inspected the service draft xml document and found no timers set to 180 seconds. How can I prevent this from timing out after 3 minutes?

Here is the python code that creates the service draft.

arcpy.CreateGPSDDraft(results,
 dp_ags_gp_rest['sd_draft_fn'],
 dp_ags_gp_rest['title'],
 server_type='FROM_CONNECTION_FILE',
 connection_file_path=AppInfo.ags_connect_connection_file,
 copy_data_to_server=False,
 folder_name=AppInfo.ags_service_folder,
 summary=dp_ags_gp_rest['description'],
 tags=dp['tags'],
 executionType='Synchronous',
 showMessages='Info',
 minInstances=0,
 maxInstances=2,
 maxUsageTime=20*60)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I'm running this on ArcPro 2.4.3 and Python 3.6.8 and Advanced license.

Re: Geoprocessing service gets 'Timeout exceeded' after 3 minutes

DavidPike — Sun, 06 Sep 2020 20:26:33 GMT

Asynchronous processing might be best for longer processes, something to try at-least.

Re: Geoprocessing service gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Sun, 06 Sep 2020 22:28:19 GMT

I actually tried that and it didn't work because the service is being called from the ESRI-supplied geoprocessing widget which apparently does not support asynchronous processing. In any case I would prefer synchronous. What I don't understand is why it is timing out after 3 minutes when I have set the timeout to 10 minutes.

Re: Geoprocessing service gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Mon, 07 Sep 2020 16:21:12 GMT

I'm beginning to think that the timeout is happening on the client (WAB Geoprocessing widget) instead of the the server's geoprocessing REST service. When I run the service directly via an HTTP POST sent to the REST service it does not timeout. It only times out when I run it via the WAB widget. I can't see in the widget code where the timeout value is being set - in any case I'm quite sure I can't do any sort of customization since my web app is being served up in ArcGIS Online. Also, from looking at the Geoprocessing widget code it looks like it should support asynchronous gp services - but I always get a failure pretty much immediately when I try that..... has anybody been able to get that to work?

Re: WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Sat, 11 Dec 2021 06:11:34 GMT

Another update. I found that when switching the GP REST service from synchronous to asynchronous that I had to recreate the WAB geoprocessing widget to get it to realize that the change had taken place - evidently it bakes that in when the widget is created. So now I am able have my WAB geoprocessing widget drive my GP task. In one way it works much better than synchronous because it displays all of the messages output by the tool.

However there is still one problem. After the GP tool completes, the widget hangs up with the "Results" progress bar flashing

In the browser network trace I see that first the job status of "esriJobSucceeded" is received which triggers a "results" request which comes back with:

{"error":{"code":400,"extendedCode":-2147467259,"message":"Unable to complete operation.","details":[]}}

I'm quite sure the tool was successful, so I wasn't not sure why this 400 error was being sent and why the results bar won't go away. The server manager logs do not show the 400 error being sent.

What I eventually found was that the "results" request worked on my tool's 2 input parameters, but failed on the output parameter. The only difference in the parameter definitions in the tool's .pyt file other than the direction was the parameterType. So I changed the parameter type of my output parameter from 'Derived' to 'Optional' and that did the trick. I don't know if this is working as intended - it doesn't seem correct to me, but in the end my WAB widget is working better than ever.

results = arcpy.Parameter(
 displayName="Results",
 name="results",
 datatype="String",
 parameterType="Optional",
 direction="Output")‍‍‍‍‍‍

Re: WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DavidPike — Mon, 07 Sep 2020 19:44:05 GMT

In your code are you setting the parameter e.g. arcpy.SetParameter(your_result, x)?

Re: WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Sat, 11 Dec 2021 06:11:37 GMT

Not exactly like this - I hope what I'm doing is the equivalent:

def execute(self, parameters, messages):
 output = cv_main.clean_and_validate (
 parameters[0].valueAsText, 
 parameters[1].value) 
 parameters[2].value = output 
 return‍‍‍‍‍‍

parameters[2] is the one with direction="Output". It seems to work that way

BTW - I actually made two changes to my parameters to get it to work and I'm not sure which one is the key. In addition to changing the 'parameterType' from 'Derived' to 'Optional', I changed the parameter 'name' from 'results' to 'output'. I don't think I have the energy to spend any more time on this to figure it out.

Re: WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DavidPike — Tue, 08 Sep 2020 00:11:25 GMT

More to see whats under the hood, whats the actual output and tool doing?

Re: WAB Geoprocessing widget gets 'Timeout exceeded' after 3 minutes

DonMorrison1 — Tue, 08 Sep 2020 17:22:26 GMT

The purpose of the WAB widget (hosted on ArcGIS Online) is to perform data cleaning and validation on our geospatial database. So the Geoprocessing widget simply calls the REST service (on our on-premises ArcGIS server) that I created based on code I wrote in a python toolbox. The python code inspects and cleans the various database tables. It logs messages via arcpy.AddMessage. When I ran the service synchronously those messages never got displayed by the widget. But now that I've switched to asynchronous, they do - which I really like.

Once the python code has completed its job, it sets the output message as shown above where I assign the string to "parameters[2]". The widget at that point wipes out the previous messages and displays just the output parameter:

So it is now working very well - although I don't understand totally why. BTW - I had a lot of problems republishing the REST service. For instance:

In ArcPro I could publish only once without restarting. Subsequent attempts did not work or throw an error message
It appeared some aspects of the REST specification where cached and I could not change them with a republish. For instance, changing a parameter name or the type (as defined in the toolbox code). Even if I deleted the existing REST service before the republish, the old values persisted. I had to give my REST service a new name for the proper values to be set. Luckily the WAB widget is the only consumer of the service so I'm not worried about breaking anybody else.
When I switched my REST service from synchronous to asynchronous, the WAB widget did not detect that. I had to delete the widget and create a new one.

Thanks again for putting me on the track of using the asynchronous service - it really does work better than what I was doing before.