topic Re: Inconsistent & incorrect data downloaded from Open Data site in ArcGIS Hub Questions

Inconsistent & incorrect data downloaded from Open Data site

C_EHoward — Tue, 24 May 2016 19:20:07 GMT

We just re-introduced our Open Data site and have users citing the following issue-- sometimes the table downloaded is correct and other times it contains data from another layer. For example, I download a '2011' layer (related table, as CSV) and 2009 data appears in the downloaded file....not always but sometimes. This seems to be limited to a few of the year tables I have.

The files are HUGE so I do have them in SDE since I had a problem before where Open Data would not serve my tables despite their being no size limit. I do not like the fact I am seemingly limited in the way I serve my data but I need it work.

Any ideas or explanations? This inconsistency is not something that is acceptable to my users

thanks-

Re: Inconsistent & incorrect data downloaded from Open Data site

DanielFenton1 — Fri, 27 May 2016 14:59:34 GMT

Hi C E,

I'm sorry that you are running into this issue. I recommend you open a ticket with Support right away. Unfortunately, from your description alone there is very little we can use to diagnose the problem. Support will work with you to narrow down the issue and then the Open Data team will be able to resolve it.

In the meantime, could you please point me to the datasets that have become mixed up? Are there any publishing workflows for them that you can share?

Thanks,

Daniel Fenton

Re: Inconsistent & incorrect data downloaded from Open Data site

C_EHoward — Fri, 27 May 2016 15:39:25 GMT

Thanks Daniel,

a support case has been opened and I am awaiting more info after that analyst talks to someone else more familiar with open data.This is what I experienced earlier this week-- I have since reset my download cache and I am trying to see if the problem persists but the downloads are averaging btw 30 min to over an hour. Below are some notes from my user which received bad data downloads:

Error case #1 - Both download methods 1&2 give this file, containing 2010 data: (download from the item list page or via the 'download' button

Hourly_Volumes_2009.csv

link to dataset = Dataset | Regional Transportation Data Clearinghouse

I will say today the data downloaded correct - but it took me several attempts at downloading data earlier in the week to see the error reported by my user

error case #2 -When I click on the 2011 download link, I get the 2009 data table. But, the file name is formatted differently:

2009_Hourly_Volumes_table.csv

Dataset | Regional Transportation Data Clearinghouse

Today the file took over an hour to download but the first download looks like it has correct data

So at a minimum the names of the downloaded files differ depend on where you download-- not sure it matters but wanted to point it out as I have never noticed this before

Publishing workflow is straightforward- MXD with 2 feature classes and several related huge tables. published, content item created, shared with open data and then show up on my page.

An update to this issue from today-- I have a phantom file that is also coming through that was not present when the other weirdness was going on -- the duplicate 2013 file only showed up this morning after resetting my service.

Good layer - Dataset | Regional Transportation Data Clearinghouse

Bad layer - has old metadata refernce and bad sublayer position in the map service

Dataset | Regional Transportation Data Clearinghouse

Also have this warning message

Hourly Volumes 2013Table Updated at 9:17:07 PM Tuesday, May 17, 2016

Warning: This dataset comes from an item that points to more than one layer. For Open Data, we recommend that items reference individual layers.

Warning: The maxRecordCount for this layer is greater than 5,000. This can cause browsers to crash. We recommend lowering this number.

Warning: Query performance on layer is suboptimal and will impact user experience.

Warning: A recent attempt to download this dataset failed. Reset the dataset to resolve this issue or contact support with this information: Request 'http://gis.mwcog.org/wa/rest/services/RTDC/Traffic_Counts_Hourly/MapServer/9?f=json' returned HTTP code 500 and Error 'Unable to get layer metadata: Request for layer information failed: json' for dataset 0ae204341d5d4d588646687b53d485f8_9 at 2016-05-26T19:11:26.031Z.

Any insight is appreciated, thanks

Re: Inconsistent & incorrect data downloaded from Open Data site

C_EHoward — Fri, 27 May 2016 18:37:59 GMT

update-- all seems to be resolved and my user reported success with downloading all of the tables.

I reset everything and unshared the service and then re-added it. I did this earlier in the week and it did not seem to matter, and the 'phantom' layer appeared after all that. I am happy things are fixed but still boggled by the weirdness sharing my pretty basic (albeit huge) tables.

thanks for your assistance--