topic Re: Open Data download issues in ArcGIS Hub Questions

Open Data download issues

JimCourson — Mon, 06 Jul 2015 19:05:40 GMT

I'm having trouble downloading polygon data from Open Data since the latest updates were applied. Three datasets that previously worked now fail to download. In some cases the data gathering completes, then I get "The spreadsheet you requested is currently unavailable". In other cases the data gathering fails with the following message "spreadsheet – An error occurred while processing this dataset". Point data still downloads fine, but polygon data fails. The services work fine outside of Open Data. Any ideas?

Re: Open Data download issues

DanielFenton1 — Mon, 06 Jul 2015 19:07:43 GMT

Hi Jim,

Can you please post the datasets you are having issues with?

Thanks,

Daniel Fenton

Product Engineer | Open Data

Re: Open Data download issues

ChristopherBoland — Mon, 06 Jul 2015 19:13:29 GMT

Daniel,

I'm having the same problem with some of my datasets. In particular, I have had problems with the ones listed below:

If I find others, I'll add to the list.

Thanks,

Chris

Re: Open Data download issues

DanielFenton1 — Mon, 06 Jul 2015 19:14:55 GMT

Have you tried hitting the reset button in your admin data repor?

Re: Open Data download issues

ChristopherBoland — Mon, 06 Jul 2015 19:22:38 GMT

I hadn't before now, but because the data were not reporting any Errors or Warnings, I didn't think I had to. It is a little unclear to me what that button is actually supposed to do.

Re: Open Data download issues

DanielFenton1 — Mon, 06 Jul 2015 19:25:07 GMT

That’s good feedback.

The button will both drop the download cache and reharvest the dataset. It’s the equivalent of jiggling the handle…as in, it usually works.

Re: Open Data download issues

DanielFenton1 — Mon, 06 Jul 2015 19:34:08 GMT

To clarify a bit more:

Sometimes services will fail when requesting data to buid the downloads. This can cause the download process to have issues and never finish generating the cached file. Dropping the download cache via the ‘Resync button’ restarts this process.

We are adding more processing checks to hopefully prevent these intermittent issues from occurring, you’ll see those start to show up very soon. We are also working on alerting you when there are download errors.

Re: Open Data download issues

JimCourson — Mon, 06 Jul 2015 21:36:58 GMT

These are the three I'm having issues with:

http://go.imb.opendata.arcgis.com/datasets/52f55ce1d0024accbe394c8de3c0c9fe_2

http://go.imb.opendata.arcgis.com/datasets/8a82c396f7564bc79e3b45b5ebcd3b6d_1

http://go.imb.opendataarcgis.com/datasets/e1bd36b7cd5c487fa12209f36f3702fe_1

I've repeatedly reset the datasets and downloads and updated the index.

Re: Open Data download issues

DanielFenton1 — Tue, 07 Jul 2015 17:37:56 GMT

Hey Jim,

I took a look at these 3 datasets. The third one appears to be working fine. For the other two, we are seeing failures when we try to request the data from your server. Here are the requests that are being made.

http://api.imb.org/arcgis/rest/services/LIVE/Cultural/MapServer/1/query?outSR=4326&f=json&outFields=*&where=1=1&resultOffset=0&resultRecordCount=1000&geometry=&returnGeometry=true&geometryPrecision=

http://api.imb.org/arcgis/rest/services/LIVE/Cultural/MapServer/2/query?outSR=4326&f=json&outFields=*&where=1=1&resultOffset=0&resultRecordCount=1000&geometry=&returnGeometry=true&geometryPrecision=

I recommend you get in touch with support and show them these two requests.

Daniel

Re: Open Data download issues

DanielFenton1 — Tue, 07 Jul 2015 19:08:44 GMT

Hey Chris,

The fish management dataset seems to be working fine. Are you still having issues?

Also, this dataset no longer exists: http://geodata.myfwc.com/datasets/a78160f5acaf4439b49f9fbef4c100ac_5

My initial analysis is that the complexity of the features in seagrasses and salt marches are giving us the trouble. I'm continuing to debug locally so we can figure out a solution.

Re: Open Data download issues

JimCourson — Wed, 08 Jul 2015 15:42:44 GMT

Thanks Daniel. One service does appears to be working at times, but is still failing on occasion. I've submitted a support request for the other two.

Re: Open Data download issues

DanielFenton1 — Wed, 08 Jul 2015 20:06:53 GMT

We've zeroed in a fix for at least: http://geodata.myfwc.com/datasets/4d1b4e758e704def90773bd49806dd4c_6 and should be testing it soon. I'll keep you posted.

Re: Open Data download issues

JimCourson — Thu, 23 Jul 2015 03:11:24 GMT

Our issues with the services not responding to REST queries in Open Data appear to have been resolved by increasing the App server maximum heap size from 256 to 512 and increasing the SOC maximum heap size from 64 to 256 on each of the ArcGIS Servers in our cluster.

Thanks to Peter in the Charlotte Support Services office for working the problem through to a solution!

Re: Open Data download issues

DanielFenton1 — Mon, 27 Jul 2015 21:42:54 GMT

Hey Chris, it looks like this server is failing to execute queries at all now. Did you make any changes to your service?

http://atoll.floridamarine.org/arcgis/rest/services/FWC_GIS/OpenData_MarineEco/MapServer/6/query?where=1=1&returnIdsOnly…

{

error: {
- code: -2147209212,
- message: "Failed to execute query.",
- details: [ ]
}

}