ArcGIS Enterprise 11.1-11.2: Restoring tile cache breaks scene layers
Hello,
I am facing an issue since we upgraded from 11.0 to 11.1: when restoring our tile cache datastore using webgisdr on standby environment, scene layers restored are corrupted and broken.
Though the restore is said to be successful, when trying to visualize them, it fails.
All requests querying for tiles description of the service fail:
=> Hosted/Buildings_3D/SceneServer/layers/0/nodes/root?f=json
returns:
{
"error": {
"code": 500,
"message": "Server error unable to process request",
"details": []
}
}
But validation of the datastores from ArcGIS Server Manager or ArcGIS Server admin interface is successful. Webgisdr log say it successful as well.
On ArcGIS Server Manager in debug mode, this 500 errors seems to be related to 404 from CouchDB:
This issue occurs on both ArcGIS Enterprise deployments we are running. The procedure to restore on standby is implemented since several versions and we never faced any issue (except at 11.0 with restoring relational datastore cf. https://community.esri.com/t5/high-availability-and-disaster-recovery-questions/cannot-restore-arcgi... that was fixed later with a patch).
I opened a case 6 months ago after the upgrade with support (case 03338742) and so far no progress...
What we tried:
- manually restoring the tile cache datastore from production tile cache backup: same results
- manually restoring the tile cache datastore from webgisdr backup: same results
- restoring on a datastore content folder stored on C:\ of the VM to exclude drive performance issue: same results
Interesting facts:
- It's not systematic - sometimes all the scene services are broken, sometimes only some of them but never all of them were successfully restored. But most of the time, they are all broken.
- Procedure repeated tens of times with always the same result
- Publishing a new scene layer on the standby datastore with broken restored services works: the tile cache datastore is functional
- Publishing a new scene layer on the production tile cache datastore and restoring it fails as well (it is not restriceted to all scene layers published from previous versions)
Worth to mention, that we could potentially loose all our work as there is currently no way to export scene layers as a backup solution because of "BUG-000143562 - Exporting a multipatch hosted feature layer as a file geodatabase from ArcGIS Online results in a corrupted geodatabase" so there is currently no way to retrieve tile cache datastore content...
I would really appreciate your inputs @ChristopherPawlyszyn and @JonathanQuinn ...
Thanks
