AGO Open Data download issues

[ChristopherBoland]

Since the new update this morning, i've been testing my develpment site out and I've come up with a few issues:

1. When trying to download one dataset "Something went wrong with the server.  Please try again or report this issue to the team." (Dataset | FWC GIS & Mapping Data Downloads )
2. While trying to download another large dataset, which I know has been a problem in the past, I only get 1000 of the 2328 records returned in my downloaded dataset (Dataset | FWC GIS & Mapping Data Downloads).  I think this is because I tried to download this dataset before the most recent update and maybe the zipped shapefile was cached on ESRI's server with the max number of features returned by the server, but there's no instruction on how to clear that cached zip file.
3. When downloading a zipped shapefile for a large dataset, the GUI doesn't give you an option of where to save the dataset.  So, if it is a long download like this dataset (Dataset | FWC GIS & Mapping Data Downloads), you have to wait until the large data download is complete before you can save it anywhere, and you might miss the prompt that shows up breifly at the bottom of the browser window.

Any suggestions would be helpful,

Chris Boland

[DanielFenton1]

Hi Chris, let me answer these issues one at a time.

1. I've investigated and it's unclear what's happening here. I'm going to forward this along to our team for analysis.

2. This download is failing due to a known issue with ArcGIS Server. We have a workaround in place in Open Data, it should be deployed early next week.

3. I believe this is a browser specific setting. However, I have an idea for how we could give you the option to right click and save as, when you are retreiving large downloads. I'll talk to the team about getting that implemented.

I'll keep you updated here with regards to the 2 bugs and the enhancement. Let me know if you have any other questions.

Daniel

[ChristopherBoland]

Thanks Daniel.

I think on #3, you're right.  It is probably browser specific.  I am using Firefox 34.0.5.  I do get the save as option with Internet Explorer, and I do not have Chrome installed on my machine.  I also have not tried Safari.

Thanks again,

Chris

[ChristopherBoland]

Daniel,

I spoke with Andrew Stauffer over the phone yesterday and I think on #3 it is because my OpenData site had not been upgraded to the Bootstrap version yet.  I believe once I upgrade my site, this concern will be addressed with the new "My Activity" download menu.

Thanks again for looking into this,

Chris

[DanielFenton1]

Hey Chris,

We have resolved issue number 1. You should be able to download that file without any future problems.

We are still working on issue 2. The extremely high vertex count per feature is presenting some challenges in converting the data into a shapefile or kml extract. If these features were simplified, the issue would not liekely arise. We want people to be able to download original source data so we're going to keep at it and I'll let you know when there are any updates.

[MiguelParedes]

Greetings Daniel,

I have been observing an interesting issue with an Open Data portal, which is none of the issue reported by Christopher, but that is pretty baffling. 

On a particular Open Data portal, I have found a dataset of Planning Applications.  When I check the Table tab, it reports a total of 33,096 features.  However, when I download the data, either as CSV or as a shapefile, the downloaded datasets have double that number of features, or 66192.  Closer examination of the downloaded datasets shows that the features seem to have been duplicated.

Have you heard and / or are you aware of this issue happening?  If so, is there anyway to fix it?

Many thanks in advance.

[ChristopherBoland]

@Miguel Paredes,

This issue has been reported on another geonet discussion (Open data downloads - duplicate records ).  No answer has been provided yet.

Chris

[MiguelParedes]

Greetings Chris,

Many thanks for the link.  As a matter of fact, it was Simon's site and the Planning Applications data set I was referring to. Interesting to find out that there are others, well, at least one more individual having the same issue.

Would be great to have a response from Esri on the issue, and how to resolve it.

Thanks again,

Miguel

[DanielFenton1]

Hi Miguel,

I've fixed the dataset in question. We are having trouble reproducing this issue in our testing environment, but we are working on a fix nonetheless.

Daniel

[MiguelParedes]

Greetings Daniel,

That is brilliant!  Thank you very much for this. 

Questions arise, is this caused by the size of the dataset in question? Or is it something else?

In any case, much appreciate the effort.

Miguel