Open Data and large datsets

Question

We have been having issues with Hub and Open Data related to large data sets. Once we add large data sets to Open Data they fail to download. These would include our parcel and contour data. We have had an incident open with ESRI for almost a year with no resolution.

Has anyone else experienced this?

Thanks!

DaveWatson · Answer

Ha, I was just testing the max count at 250. I will make the change to 100 and run a test. Would there be any benefit in modifying the Max sample size variable?

Anonymous User · Answer

We tested your data on one of our servers and the caching failed when maxRecordCount was set to 250. 100 worked much better and successfully cached in <30min which seems like an improvement.

As others have probably mentioned, this dataset, while large in both a record count and field sense, also seems incredibly complex geometry-wise. Have you considered simplifying the geometry a bit? In most of our tests things slow down ~245k records in, which leads us to think there's one record that may have a large amount of vertices.

DaveWatson · Answer

Okay so I changed the max record count on historical parcels to 250 and changed the max sample size to 50K. I’ll try 2016 parcels again.

I am working on my morning email to Don.

This is what I am thinking. They ‘reset’ our cloud store again.  The largest dataset that we have on open data would be parcels. The rest will have to wait or go to the zip file solution.