Open Data downloads getting stuck

MelissaBrooks · ‎09-30-2019

Hi there,

We are experiencing issues with (any type) downloads either

- "Gathering Data" continuously

- Taking a very long time

- Taking a very long time, then the second time "Gathering Data" continuously

It sounds very similar to this post,

What does Gathering Data mean

Can you please check this issue has not come back since the recent updates

Thanks

Melissa

New Plymouth District Council

New Zealand

Info: Our new open data portal is not yet ready for the public, but I can provide information if necessary.

Melissa Brooks
Senior Geospatial Systems Specialist
Tasman District Council (New Zealand)

Anonymous User · ‎10-11-2019

Hi all,

First off, apologies for the download issues over the past week. We observed a significant outage that caused much of the headaches you've experienced.

Over the span of about a week (10/1 – 10/8) the Hub backend experienced substantial delays with dataset indexing and download file preparation. We lost a substantial amount of cached files in our backend, requiring a rebuild across the Hub platform. Rebuilding the cache for all the content across the platform takes significant time. While the system is recovering you may experience unexpected behavior due to indexing failures or download caching failures that occurred during the noted period. We expect these issues to clear themselves in the coming days and will continue to monitor.

We agree that a lack of messaging and error reporting compounded the issues this week, which is why we're investing development energies into better error reporting and response. In addition to adding error display and management features to the application, we will be adding a status page to Hub in the coming weeks. It is our hope that this will help make sure you not only see what's wrong before your end users do, but that you don't waste valuable hours troubleshooting issues in vain.

Again, we apologize for the inconvenience and frustration this caused you. We're working to make sure it doesn't happen again.

Best,

Patrick, Hub Product Engineer

View solution in original post

HemantAngarkar1 · ‎09-30-2019

To troubleshoot initially;

1. Is it happening with particular dataset/s or all of them?

2. If it is particular dataset/s, try dropping cache (Manual Reset). Rebuilding cache may take some time which depends on how big/small corresponding feature service is (number of features).

3. If it is still not working try un-sharing and re-sharing the concerned item/s from Public and Open Data group.

4. If it is still the issue, try adding same Feature Service again to ArcGIS Online using Service URL with a different Name >> Share it with Public and Open data group. If it works then there is probable corruption of the item.

5. If it doesn't work create a new Open Data Group and try sharing services with that group and add that group to your site to see if issue is group specific.

This can go on, but if you can check few things for further troubleshooting;

1. How big are the datasets in question (How many features)?

2. Since when issue occurred? Did anything changed?

Hope this helps!

IvanBrown · ‎10-11-2019

We are also suddenly experiencing stuck downloads (w/ underlying data that hasn't changed).

The Data Manager--where you go to reset download caches--is suddenly missing. I would go there to try fixes like re-indexing the site or resetting caches; it's gone.

MelissaBrooks · ‎10-01-2019

Thanks for this list Hemant Angarkar‌ we will work through this, a logical order.

This issue is inconsistent, one item will work (after a long while) then next time it wont work (continues to try but never finishes) - or vice versa

Polylines tend to be more succeptable than points, but point layers still have issues.

Ranges from very small (4-30) up to med/large (20-40,000 features.)

No more than 20 layers per service as per recommendation.

Began this week, but we think we may have been spot checking last week, it may just be that we needed to focus on some of the more complex layers.

I'll go through this checklist today so I can eliminate any problem with the item and/or open data sharing and provide an update. Thanks again for your feedback

Appreciated!

Melissa

Melissa Brooks
Senior Geospatial Systems Specialist
Tasman District Council (New Zealand)

HemantAngarkar1 · ‎10-02-2019

You are welcome Melissa Brooks‌. Point of all those steps is to find out where issue is. Also I missed in my last post;

1. If services are hosted on ArcGIS for Server then try uploading same data as a service to ArcGIS Online and see if that has issues.

2. If Both Server and AGOL services has issues then I think it may the data that is causing it.Run Check Geometry to see if it has issues and run Repair Geometry to fix those issues. If no geometry issues then it might be having Complex Geometries that is too many vertices per feature (for line and polygon geometries obviously). Run Simplify tools can address the issues but they will slightly modify shape.

3. If it is only ArcGIS for Server services, try setting the Download Cache to "Manual" from Data Manager page for that dataset.

4. If that fixes the issue then I would recommend to check the Machine's (Where ArcGIS for Server is installed) resource utilization (CPU and RAM) from Task Manager while you hit the refresh cache button from Data Manager page for that dataset. If you see spike in Resource utilization then you have to work with your ArcGIS Admin and IT to address that issue(Also involving technical support would be good idea in this case).

After typing all that I am just basing these troubleshooting steps on the basis of my imagination of the issue. I think if you won't get it resolved after working through all the steps, going to technical support will be ideal where they can take a look at the issue in a better way.

Hope that helps!

KarolFallas · ‎10-04-2019

Hi,

We have the same issue. It happen recently.

Thanks

Karol Fallas

Municipalidad de Heredia

Costa Rica

HemantAngarkar1 · ‎10-06-2019

Hello,

Did you try steps mentioned above? Let me know if you need any further explanation.

HemantAngarkar1 · ‎10-06-2019

Please provide link to the dataset with issue if site is public.

KarolFallas · ‎10-07-2019

Hi,

This is an example:

Ubica, Municipalidad de Heredia

Days before it was working.

Thanks

http://ubica-herediasig.opendata.arcgis.com/datasets/zonas-homog%C3%A9neas

HemantAngarkar1 · ‎10-08-2019

Karol Fallas‌

Download worked for me today. I Think it was temporary. Try setting the Download Cache to "Manual" from Data Manager page for that dataset if dataset is not upadted frequently on Server.