Select to view content in your preferred language

Intermittently Unable to Create Hosted Feature Services

2627
14
04-17-2024 07:16 AM
Labels (1)
RyanUthoff
MVP Regular Contributor

I'm having an issue where we are intermittently unable to create hosted feature services using ArcGIS Enterprise 11.2 on Windows. This includes creating hosted feature services directly through the Portal UI as well as creating Survey123 forms.

The only error message I receive is: Create Service exception 'java.lang.Exception: Exception: Read timed out'

Survey123 explicitly gives that error message, while Portal gives a generic error message in the web page, but I can see the same error message when viewing the internet traffic. It is a read timed out error, but it usually fails within seconds of attempting to create a hosted feature service.

Esri is not producing any error messages in the Portal, Server, or Data Store logs when this error occurs. This one error message is literally the only error message I'm seeing.

This has happened to multiple people at various times with various hosted feature services/S123 forms, including Esri's template hosted feature services you can create through Portal.

I'm working with Esri support on this one, but does anyone else have any ideas on what this could be? We've already ruled out anti-virus/firewall and the ArcGIS Data Store successfully validates through Server Manager.

0 Kudos
14 Replies
RyanUthoff
MVP Regular Contributor

Thanks for the reply! We've already made sure that all the Esri directories have full permissions to the service account running server, so unfortunately that wasn't the issue.

0 Kudos
AhmadAwada1
Emerging Contributor

Hi Ryan,

i am facing exactly the issue you described. Were you able to figure out a solution? If yes, please help!!

0 Kudos
RyanUthoff
MVP Regular Contributor

Nope. Have an Esri support case open for 7 months and counting. I know at least one other person has an Esri support case open as well, and they haven't had a resolution either.

Right now, we're getting towards the point of having to do a full uninstall and reinstall of the ArcGIS Data Store, and then restore it from a backup, because we're running out of other troubleshooting options.

My recommendation is to open an Esri support case. The more people we have reporting this issue, the better. It might demonstrate that Esri has a legitimate bug on their hands instead of a random one-off issue.

I'm actually hesitant to believe that the Data Store is the problem in the first place because what we found is that when we receive this error message, the requests aren't even making it to the IIS level. A successful publishing of a hosted feature service sends requests through IIS. However, an unsuccessful request does not. So, in order for the Data Store to be able to return an error, a request would need to be made to the Data Store first, which we would see in IIS. But, it's not doing that..... Of course, I don't know what all goes on behind the scenes, but that's my thought process on it.

0 Kudos
Thomas_Puthusserry
Frequent Contributor

@RyanUthoff  Is this anything to do with your GIS server's config store location? I had a similar issue in our TEST env which had the config store / server directories located to Azure storage account. The publication of hosted feature service many times failed (but worked few times). In the end, after moving the config store and server directories to a fileshare VM the issue didn't recur. Just thought to highlight that.

RyanUthoff
MVP Regular Contributor

It's possible, but I don't know for sure. We have two Enterprise environments with the config store in the exact same AWS FSx, but only one environment is having issues. The other is perfectly fine.

In our case, what we notice is that the requests aren't even making it to IIS. When normally creating hosted feature services, we can see requests being sent in the IIS logs. However, when I receive the error, I don't see anything in the IIS logs. Since all internet traffic and requests go through IIS/web adaptors, I'd think that if it was a problem with Portal/Server/Data Store, the request would be logged in IIS. In order for Portal/Server/Data Store to return an error, it needs to receive a request first, but it's not being logged in IIS.

So if my logic is correct, then the problem must be something at the web adaptor or IIS level. But I'm not sure where to look or troubleshoot any further.

0 Kudos