topic Data Downloads from HUB site not working in ArcGIS Hub Questions

Data Downloads from HUB site not working

WillArellano — Fri, 28 Aug 2020 15:17:44 GMT

I just built my first HUB site to make it easier to share data with outside entities. However, I have a big issue. When I go to download certain layers (such as our city limits or ETJ polygons), only certain file types will download successfully. In general, the file geodatabase type and spreadsheet data will download just fine, but if I try to download as a shapefile or KML, the download button just shows that it is downloading forever...it "hangs".

I haven't tested every layer, but it seems that some work fine (like our City Council Districts), and some do not .

Here is the link to the site if you want to try it out yourself: Cibolo GIS Open Data

Any tips?

Re: Data Downloads from HUB site not working

WillArellano — Mon, 31 Aug 2020 15:36:13 GMT

Update: I got some of the layers to work by disabling their M and Z values (not needed in the layers, but were carried over from old CAD files).

Some layers are still being very slow to download to KML format.

Re: Data Downloads from HUB site not working

ThomasHervey1 — Mon, 31 Aug 2020 17:03:17 GMT

Hi Alyssia, do you have a particular dataset that is giving you troubles that I could look into? We are just about to release a new UI that will alert users to the status of download files as well as if there is a fresher version that needs time to create. This can often be the reason you see a 'hanging' download since on our end a new file is being created.

Thanks,

Thomas

Re: Data Downloads from HUB site not working

WillArellano — Mon, 31 Aug 2020 17:46:21 GMT

Yes, right now it seems that the KML format of the Zoning layer is hanging. I waited 20 min the other day and it never downloaded. https://cibologis-cibolotx.hub.arcgis.com/datasets/city-of-cibolo-zoning?geometry=-99.515%2C29.353%2C-96.902%2C29.771&showData=true

Re: Data Downloads from HUB site not working

ThomasHervey1 — Mon, 31 Aug 2020 19:48:11 GMT

Thanks for the link. This looks like a file creation bug that we will look into. Have you had an issue with KML downloads for items that are published initially without a M or Z value?

Re: Data Downloads from HUB site not working

NishantSrivastava — Mon, 31 Aug 2020 19:53:03 GMT

I'm having similar issues with dataset here: 116th Congressional Districts

In the 'Data' tab, the loading of the datatable is erratic, correctly loading about 1 out of every 4 times I refresh. Have been trying since yesterday to download any of the options to no avail, just been seeing an animated arrow on the Downloads button.

Appreciate your insight into why this is happening, but any idea on when it could be remedied?

Re: Data Downloads from HUB site not working

ThomasHervey1 — Mon, 31 Aug 2020 20:39:05 GMT

Thanks for the link Nishant, we'll look into why the dataset table is not loading. This item is a Map Image Layer which can sometimes take a while to generate a download file, also sometimes there are issues with downloads from non-hosted services. Are you the owner of this item?

Re: Data Downloads from HUB site not working

WillArellano — Mon, 31 Aug 2020 20:54:35 GMT

I have had issues with many files, both with and without Z and M values enabled. When I first posted my issue here, I was having issues with 3 of my 10 files on Hub - City Limits, ETJ, and Zoning.

Only City Limits and ETJ had M and Z values enabled. When I disabled them, and re-uploaded, they started working as expected about an hour later.

The City Limits layer initially would only work well for Spreadsheet and FGDB, the KML took at least 10 min, and the Shapefile took 25-30 min to download.

The ETJ layer initially would only work for FGDB, no other formats worked.

My zoning layer is the only one right now that refuses to download the KML format. It is not M or Z enabled, and is less "complicated" then many of my other layers that work fine.

Re: Data Downloads from HUB site not working

ThomasHervey1 — Mon, 31 Aug 2020 21:17:29 GMT

Thanks for the update. I have filed a bug for your KML issue and an engineer is looking into it right now. Regarding long download times, this can be expected because new files need to be created once an update is made. Unfortunately the UI is a bit misleading because before a download can complete, a file needs to be created, which can take a long time in some cases. Be on the lookout for the new download UI within a week or so. It'll be a beta setting under site settings and there will be a blog post reviewing the changes. Also, next time you run into an issue with one of the other datasets, please let me know.

Thanks,

Thomas

Re: Data Downloads from HUB site not working

NishantSrivastava — Mon, 31 Aug 2020 22:25:20 GMT

Hi Thomas, no I'm not the owner of the item. I discovered it yesterday, and left it overnight to attempt to download thinking the same, that a server issue or GET issue was to blame, but it didn't work. Should I reach out to the owners about the issue?

Re: Data Downloads from HUB site not working

ThomasHervey1 — Tue, 01 Sep 2020 14:23:50 GMT

Not necessarily since I am able to reproduce the issue. Content owners have several options to try and fix downloads that don't work, but we will look into this issue. I apologize that I do not have a solution at the moment.

Re: Data Downloads from HUB site not working

BillSchlatter — Wed, 02 Sep 2020 15:56:38 GMT

I've had similar issues with our Open Data Portal. Some of our larger datasets quite simply hang forever when users attempt to download them. I'm pretty confident that it's not an issue with the way the data is formatted or the inclusion of m and z values, because the same data will often work if we break it into smaller pieces. For example, our seven-county building footprints file wouldn't download (observed for up to an hour with no sign of an actual download beginning), so we split it out into seven separate county files, and those worked fine. I had assumed that there was some designed or de facto limit on the size of dataset that the Open Data Portal can functionally work with, but if there's a way to get large datasets to download properly, it would save us a considerable amount of data maintenance work on our end.

Re: Data Downloads from HUB site not working

ThomasHervey1 — Wed, 02 Sep 2020 16:41:08 GMT

Hi Bill,

In the spring we upgraded our download system to handle larger and more complex datasets. File creation time can take up to several hours when there are millions of records or very complex geometry. The new UI we are about to release will give users the option to download a previous version or create a new one, which will take some time. But there may a problem creating the dataset you're mentioning due to other circumstances or the file type. Can you provide a link to large dataset that you've had issues with?

Re: Data Downloads from HUB site not working

BillSchlatter — Wed, 02 Sep 2020 16:55:42 GMT

https://semcog.maps.arcgis.com/home/item.html?id=72152aba68924bc29c9af749172c7918

https://semcog.maps.arcgis.com/home/item.html?id=9e30aad8a0d84fba830fdb3732648173

Those are the newest versions of our building footprints and building points layers. We tried putting them in the Open Data Portal as-is, but they both hung on download. We settled for adding the zipped file geodatabases that were used to upload them into the Open Data Portal, because those downloads worked (at least until recently, I've posted about the issues with them in a separate thread, and I plan to submit a support ticket about an apparent bug related to them soon)

Re: Data Downloads from HUB site not working

ThomasHervey1 — Wed, 02 Sep 2020 21:22:40 GMT

Thanks Bill,

I opening those items in ArcGIS Online's Map Viewer also didn't seem to render quickly, so I imagine the number of records and complexity of the building footprint geometry would reasonably lead to a longer time for creating download files. Feel free to open a support ticket for the items not downloading in Hub and we can see if the issue is because of the size or something else.