Intermittent failure of Custom Print service

Question

Hi Guys, My team administers a number of ArcGIS Server sites with a variety of service types. One particular custom print GP service is showing intermittent failures, seemingly at random; we cannot discern concrete steps to force it to fail every time. This failure occurs when using the service through an OOTB Web AppBuilder widget, or when executing via the rest endpoint. Approximately 50% of the time the job will work, the other half it will fail, with the same input and conditions and in direct succession. There are no messages produced and polling the service endlessly returns esriJobExecuting; i.e., the job doesn't complete, however it is most certainly dead. The manager UI doesn't really give much useful information, returning only the following messages: * Execution of job j9f6c3e5919bd45b4bbbe8b640ce510b0 failed. The service was unable to change the status of the job. * Error executing tool. Export Web Map Job ID: j9f6c3e5919bd45b4bbbe8b640ce510b0 The only other bread crumb trail I have is the messages.xml file in the job folder on the server indicates that the server believes the user cancelled the job (which isn't the case). Message below: 200 0 Cancelled function 200 0 (ExportWebMap) aborted by User. I have set the server logging level to debug and forced the print job to fail but I cannot see anything in the logs that are of interest (although someone with more knowledge than I may see something more, I can supply logs if need be). I cannot see anything of interest in the Windows Event Logs either. This failure occurs only in the production instance of this service, the instance on the test server works fine. Has anyone seen this kind of intermittent failure before? To me it looks like the job has died but the server is not aware of its death status and treats it as executing. I would be very appreciative of any advice on how to fix this or where to go to troubleshoot further for more information on what is going on here? Please let me know if you need any further details, happy to provide what I can. Many thanks in advance. Gino Windows Server 2012 R2 ArcGIS Server 10.7.1 Published using ArcGIS Pro 2.8.0 (I note that this is not the most direct version equivalent to Server 10.7.1, however there are numerous other print services which have been deployed with this version and are functioning fine).

GinoMellino · Accepted Answer

Hi Daniel.

Thanks for sharing and reminding me to update this thread. That is a strange issue.

For us the issue was that one of the servers in the site (this site runs on 2 machines) had a short hostname. We changed it to a fully qualified hostname and the print service hasn't failed since to my knowledge. Unsure why this affected this one particular service and none of the other ones on the site however.

Cheers,

Gino