Open Data FGDB not downloading after being updated.

CortDaniel1 · ‎07-02-2019

#We have been experiencing this problem intermittently for several months. Our data is being hosted. After running an update, the file geodatabase does not download. The setting is checked to allow FGDB downloads. Updates are made using ArcMap 10.6. When the download FGDB option is clicked, the download button animates, but the file never downloads.

The only way to fix the problem is to delete the hosted feature service, and re-publish it. The description and custom license then need to be recreated in the administrator. Is anyone else experiencing this issue?

Here is an example of a data set with this problem.

https://gisdata-piercecowa.opendata.arcgis.com/datasets/drainage-network-structure-polygons

RichGwozdz · ‎09-27-2019

Hi Cort –

We received an unspecified error from your server during recaching. When such errors happen, our system waits 24 hours before attempting another cache of the data. The dataset was in this holding period; I’ve forced a retry and the file downloads without issue.

We understand that the messaging of caching/download issues and errors needs improvement and are working to improve it.

Rich Gwozdz | Software Engineer

Esri

rgwozdz@esri.com

CortDaniel1 · ‎09-27-2019

Rich.

Thanks for looking into this issue. When we applied the workaround to get the data downloads to refresh, the FGDB downloads worked.

The workaround is:

Update the data.
Un-share the data.
Wait until the data is no longer available on your Open Data Portal.
Re-share the data.

I assume forces another cache of the data, the same thing you did. We will continue to toggle sharing to allow our users to download file geodatabases. We will monitor the ArcGIS Hub change log for anything that may indicate the caching/download issues and errors have been fixed. If you can, it would be great to let us know on this post, so I get emailed.

Thanks,

Cort

KennyKleewein · ‎09-27-2019

Hello,

Our Open data downloads were incorrect but un-sharing and re-sharing the data to the open data group reset the download cache for us.

Sounds like this is now suppose to happen in the future automatically? Open data is suppose to detect a change in the data and clear the cache?

Thanks for the posts.

Kenny

RichGwozdz · ‎09-30-2019

Hi Kenny -

It depends on if your ArcGIS service exposes the "Last Edit Date" field - it's not the default on all instances. If the service does expose "Last Edit Date", it should happen automatically, but not immediately, We don't currently receive notifications of a changed "Last Edit Date" from services. Instead, we have to poll them periodically to detect whether Last Edit Date has changed, and if it has, we re-cache. So there can be some latency it updating the cache. Re-sharing is an alternative to waiting for the periodic refresh to occur as scheduled.

RichGwozdz · ‎09-30-2019

That last comment sent before I was finished.

If the service doesn't expose "Last Edit Date" we attempt to detect changes by analyzing the statistics of the service periodically. If statistics suggest a change, the data will get re-cached. Since the system only checks periodically, re-caching won't occur immediately following a change. Note that the effectiveness of change detection using service statistics depends on the nature of the change, so if you can configure your service to expose "Last Edit Date", your recaching will be more reliable.

CortDaniel1 · ‎09-30-2019

Rich,

The downloading problem is occurring again. We noticed that it is not only data sets were updated recently. The problem is affecting both shapefiles and FGDB. It is not affecting all data sets. The City of Tacoma's GeoHub site is experiencing the problem too. Here are the affected data sets we found.

Pierce County's

https://gisdata-piercecowa.opendata.arcgis.com/datasets/address-points (updated 9/29/2019 fgdb and shp)
https://gisdata-piercecowa.opendata.arcgis.com/datasets/zoning-and-land-use-designations (updated 8/2/2019 fgdb and shp)
https://gisdata-piercecowa.opendata.arcgis.com/datasets/development-process-areas (updated 7/25/2019 fgdb and shp)

City of Tacoma's

https://geohub.cityoftacoma.org/datasets/wetland-inventory-tacoma-pds (updated 9/24/2019 shp - no fgdb available)
https://geohub.cityoftacoma.org/datasets/biodiversity-corridors-tacoma-pds (updated 9/26/2019 shp - no fgdb available)
https://geohub.cityoftacoma.org/datasets/tacoma-street-network-service (updated 9/15/2019 shp - no fgdb available)

Cort

RichGwozdz · ‎09-30-2019

Hi Cort –

Thanks for flagging this. Please try the Pierce County datasets again. We’ve pushed a bug fix in the last hour. I successfully downloaded all fgdb and shp files for the three datasets you listed.

With respect to the City of Tacoma data, was the “Export Data” feature for these datasets just recently enabled?. Enabling “Export Data” is what triggers the option to download FGDBs. Unfortunately, we don’t receive direct notifications when the state of “Export Data” changes, thus making it difficult to update the download menu properly. As we work to remedy this short-coming, our advice has been to un-share and then reshare the dataset. This will trigger the system to check the status of “Export Data”.

Rich Gwozdz | Software Engineer

Esri

rgwozdz@esri.com

CortDaniel1 · ‎09-30-2019

Rich,

Thanks for promptly looking into this. We will to un-share and then re-share any data we update. There is a new twist on this problem though. We had assumed the download problem only occurred when data was updated. We discovered today that data that was downloading just fine had stopped downloading. Zoning data was updated on 8/2 and Development Process Areas was updated 7/25. Additionally, a site user reported that the Cities in Pierce County shapefile was not downloading. It was updated 8/19 and like the other two, had been downloading fine.

Cort

RichGwozdz · ‎09-30-2019

Ok, thanks for the additional information. I currently don’t find any problems downloading the datasets you listed, so it appears the fix we released earlier may have handled these cases. I’ve also checked the Cities dataset and found it to download as expected.

Rich Gwozdz | Software Engineer

Esri

rgwozdz@esri.com