Enterprise Datastore Versions Out of Sync

[CodyPatterson]

Hey All,

Enterprise 11.2

Recently, we've decided to pursue Professional Services regarding some issues with our Workflow Manager as support was unable to resolve the issue after months of work.

Over night I seemingly thought of something that they didn't and decided to change our Server, Portal, and Workflow Manager server logs to DEBUG, and then run through the requests. During this, I found that our Tile Cache was not validating, and right after a timeout occurs, I find that it sends a single status check that returns partial, it then attempts to populate the cache for the database, and stated "Unable to find couchDb for scene server and finally remarks that the job has been completed successfully, but the Workflow is inaccessible.

I looked into our Datastore config, and found this:

Last known status upgrading, which took place December 1st.

Any ideas why this could happen? I assume this is the reason for most of our Workflow issues, as it was working, previous to the December 1st upgrade.

Thanks in advance!

Cody

[Dan_Brumm]

I don't use the Tile Cache DataStore at all, but I am curious. 

1. When you run a describedatastore how does that look?

2. Is the Tile Cache working? 

3. I have run into issue with the Relational DS where it reads lastknownstatus of "stopped" or "transitional". This was never a showstopper. I had to stop the DS service, edit the status field to "stopped" and then restarted the service. 

"datastore.lastknownstatus": "Stopped",
"localstore.lastknownstatus": "Stopped",

[CodyPatterson]

Hey @Dan_Brumm 

Thank you for the reply! When I use describedatastore I get this here:

(I intentionally hid a few pieces of information)

The Tile Cache as far as I'm aware is not working, it's not validating either.

I will give the config file change a shot and see what happens!

Thank you,

Cody

[CodyPatterson]

Hey @Dan_Brumm 

I went ahead and changed the file from Upgrading to Stopped after the service had restarted, once that was completed, I started the service and it changed to Started, but it states that it is not validated as well, should I attempt to change the version?

Thank you,

Cody

[TimoT]

Your describedatastore output only shows relational datastore information and nothing for tilecache. Is your tilecache datastore on the same machine as your relational datastore? If so, it seems the ArcGIS Datastore software doesn't even see itself configured as a tilecache datastore.

Wondering if you might find a useful hint in the ArcGIS Datastore's couchdb logs (off the top of my head it is located at \arcgisdatastore\logs\<machine-name>\database\couchdb\), especially around the time of your Enterprise upgrade in December.

That being said, it sounds like this tilecache datastore has yet to be utilised in your environment. With this in mind, you could opt into removing and reconfiguring a fresh tilecache datastore. I understand that going down this route doesn't give you confidence for any future upgrades to your system though.

You can try:

1. Unregistering tilecache datastore in server manager 
2. Rename tilecache-config.json file
3. Re-configure tilecache datastore to your deployment

This should generate a new tilecache-config.json and you should see a newly generated database name compared to your old json file.

Regarding your query about sample scenes, one of Esri's 3d scene tutorials has BIM data you can download. You'll probably need ArcGIS Pro to publish, or you can use ArcGIS Pro to create an .slpk file from the data.

• https://learn.arcgis.com/en/projects/share-digital-building-models-in-3D-web-scenes/

Esri also have a world_earthquakes slpk in their github (note: I've seen many Portal's not having the required symbology to draw this one)

• https://github.com/Esri/arcgis-python-api/tree/master/samples/05_content_publishers/data

Timo

[CodyPatterson]

Hey @TimoT 

Thank you for the suggestions! I attempted to unregister the Tile Cache and it ended up giving me this error here:

I have an open ticket with support, and I'm hoping they'll be able to check it out with me, but I will be giving the scene things a shot here shortly!

Thank you,

Cody

[ArchitSrivastava]

Hello @CodyPatterson ,

I think the major concerning part here is "localstore.lastknownstatus": "Upgrading" rather than the version. We can run a check to confirm few more things. Could check the following:

• Validate "Tile Cache" Data Store from ArcGIS Server Admin Endpoint 
  • Navigate to ArcGIS Server Admin Endpoint 
  • Home > data > items > nosqlDatabases > AGSDataStore_nosqldb_"Datastore.name" > machines > machine_name > validate
  • Check if this Validates.
• As suggested by @Dan_Brumm we can update the config file as well but I would like to request to run a comparison between "tilecache-config.json.history" and "tilecache-config.json", check if there are any differences.
• Switch the ArcGIS Server logs to DEBUG and Try to publish a Hosted Scene Layer, this would push components to log if there are any errors in config. For more "Publish hosted scene layers "

Additionally, I  will run a check on one of my 11.2 Environment and the share if these information match. Furthermore, there are ways to confirm the Tile Cache DataStore, but let's confirm the above first and then we can look into that if needed.

Hope if helps!

[CodyPatterson]

Hey @ArchitSrivastava 

Thank you for the reply! I attempted to validate from the server manager, and it states that the Tile Cache does not validate. When I went through admin path, the Tile Cache presented this here:

Here is a picture of the current and history file for comparison:

Along with that, I have no hosted scene layers that I could experiment with, I looked online as well and I could not find any at this point, if I'm missing them just let me know and I'll give it a shot!

I will return with more info!

Cody

[CodyPatterson · Accepted solution]

Hey all, just an update, I found what had occurred.

Looks like not deleting these files here could cause a "null" message to occur, but once deleted, I was able to re-register the tile cache easily.

Hope that helps someone in the future!

Cody