ArcGIS Hub site download issues

551
4
09-30-2022 03:25 AM
AndrewLee4
New Contributor II

Hi,

We have received reports from end users that some isolated dataset downloads are hanging in some formats on our ArcGIS Hub site (running for >18 hours).  I have replicated this issue locally.  Please investigate.  We have cleared the browser cache and refreshed the content catalogue with no effect.  Is this a widespread known issue or isolated to our ArcGIS Hub site only?  Please note, not all updated overwritten layers have been affected in this month's update cycle.

Home page: https://naturalengland-defra.opendata.arcgis.com

Screenshots below are in the Chrome browser.  

See examples below - there may be more?

1. Dataset page: https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::ancient-woodland-england/about

AndrewLee4_0-1664532828619.png

2. Dataset page: https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::national-nature-reserves-england/ab... 

AndrewLee4_1-1664532927659.png

3. Dataset page: https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::england-coast-path-route/about

AndrewLee4_2-1664532992194.png

4. Dataset page: https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::england-coast-path-coastal-margin/a...

AndrewLee4_3-1664533402368.png

Kind regards,

Andy Lee

Natural England Open Data Geoportal Service Manager

Natural England, Department for the Environment, Food and Rural Affairs (Defra)

0 Kudos
4 Replies
ThomasHervey1
Esri Contributor

Hello @AndrewLee4,
Thanks for reaching out. I'm sorry to hear that you're seeing a prolonged file generation process for certain formats on certain items. I looked into the examples you've provided and I was able to reproduce the issue but without any obvious causes.

Typically in cases where a large feature server (with records in the 10s of thousands or more and complex geometry) is generating a file, it can take a long time, but not over three days. The cause is usually a misconfigured or older server (which does not seem to be the case), or there is a problem on the Hub system where long running download jobs are not completed. Strategically we're aiming to move away from this system towards a more robust process. I recommend reading our blog post on "hosted downloads" and test out the beta capability on some of your services.

We will investigate this issue further on our side, but in the meantime, when you have low traffic (such as at night or over the weekend), please try the following steps. First, pick one of those example and remove it from the groups shared to the naturalengland-defra site, change the sharing controls to "organization", wait a minute, and then change the sharing controls back to "public" and re-share with a group on the naturalengland-defra site. Then wait a few minutes and open the sidebar to see if the generation message still appears.
Second, can you briefly describe your update process? In isolated cases like this, it's useful to understand how users make updates, when they're scheduled, and when they start noticing issues.

Thanks for your patience

PatrickJankanish
New Contributor II

Hi Thomas,

We've had an Esri support ticket (Esri Case #03111599) open for several weeks regarding the generation of download files on our Open Data site. We've been focusing the troubleshooting on one of our largest datasets, King County parcels (parcel_area). The examples cited by Andy Lee are similar to our experiences with this layer. When we first opened the ticket, choosing to generate a new download file would kick off a process that might never complete or might complete after many hours. Small datasets were not a problem. We did try some similarly large datasets on other Esri Open Data sites, such as the City of Seattle and LA County, and noted that those also exhibited the same behavior, so we thought there was something going wrong on the backend with the system that generates the download files.

If you can access the history on our support ticket, you can see some of the things we've been trying. Over the past week or so we've noted that on some days the download files for parcel_area generate in a matter of minutes for all file formats. On one occasion this week, that was true for some of the file formats, but it took more than 10 times as long for other formats (e.g., 3 minutes vs. 30 minutes). And there have still been instances this week when download file generation appeared to have timed out without producing anything.

Despite some successes, I've been reluctant to agree to close the ticket until we see consistent success.

I don't think the little bit of information I've provided here can help, but I wanted you to know that what Andy Lee has reported is not an isolated situation.

Thanks,
Patrick Jankanish
King County GIS Center

AndrewLee4
New Contributor II

Hi Patrick,

Thanks for noting that this is not an isolated situation and for supporting me with my comments above.  It's good to hear that our site is not the only one impacted across the ecosystem.  I hope Thomas and team are able to get to the root cause soon.  I may try the beta hosted downloads feature, but I would prefer that the existing capabilities continue to work as-is before exploring these further.

Kind regards,

Andy

AndrewLee4
New Contributor II

Hi Thomas,

I have followed your guidance and unshared and re-shared all the latest updated items this month with the site Content group. 

I have also now officially raised a Technical Support ticket with ESRI UK on myesri.com:

#03163302 ArcGIS Hub Download format issues - Natural England Open Data Geoportal

The ticket has been picked up by Tom Cresswell - he has asked:

To gain some insight into your case, please can you provide the following details:

- Confirm where the datasets are being loaded from.
- A workflow detailing how and when the data was overwritten.
- Confirm the formats that appear to be hanging.
Thank you in advance for providing this information.

I will reply to Tom and ask him to pass this over to you now for further investigation and include the link to this page.

Some items completely regenerated their downloads successfully in all five formats, but some did not.

The following succeeded:

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::england-coast-path-route/about

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::ancient-woodland-england/about

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::sites-of-special-scientific-interes...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::sites-of-special-scientific-interes...

The following items have started to regenerate some of the download formats, but not all the formats have become available yet.  They have been running all day (UK time) and appear to be hanging.  Many of these are critical UK national data products published for use by contractors, partners and public users inside the ESRI platform and outside.

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::national-nature-reserves-england/ab...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::england-coast-path-coastal-margin/a...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::countryside-stewardship-scheme-2016...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::countryside-stewardship-scheme-2016...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::environmental-stewardship-scheme-ag...

https://naturalengland-defra.opendata.arcgis.com/datasets/Defra::environmental-stewardship-scheme-op...

To test if this improves download generation performance, I have turned on the Beta capability "Hosted Downloads" option on the Content item pages for each of the above items.

Batch updates are completed using a Python script which was created/adapted for us by ESRI UK Professional Services team. This reads a .CSV file list of the locally stored ArcMap MXD file-paths, parameters for the AGOL item fields and service definition configurations.  This checks if the service names exist, then generates a locally stored service definition .SD file, then pushes it up to AGOL and overwrites the existing services one by one.  Then some manual changes are made on the AGOL item pages to ensure that all previous configurations are retained as-is. e.g. Map Viewer visualisation, pop-ups etc.  There have been no issues at all with the overwrite scripts or AGOL item configuration, only the ArcGIS Hub downloads.

If you would like to view any of the above items in AGOL please click Open in ArcGIS Online from the bottom right of the ArcGIS Hub item About page.

Kind regards,

Andy

0 Kudos