API Error Open Data Download

https://geoportal.statistics.gov.uk/datasets/ons::output-areas-december-2021-boundaries-full-clipped-ew-bfc/explore

Hi @JosephBisig
We understand that the "API Error" is generic and can be confusing. Have you enabled the "Local Projections" setting on the Hub site? Hub needs a numeric SRID and will throw an error when trying to query metadata without one. For example, on this item on Hub https://gisdata-piercecowa.opendata.arcgis.com/datasets/pierce-county-council-districts/explore the Hub API returns an error when trying to get metadata. On that item's service

the Spatial Reference is

PROJCS["NAD_1983_HARN_StatePlane_Washington_South_FIPS_4602_Feet",GEOGCS["GCS_North_American_1983_HARN",DATUM["D_North_American_1983_HARN",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["False_Easting",1640416.666666667],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-120.5],PARAMETER["Standard_Parallel_1",45.83333333333334],PARAMETER["Standard_Parallel_2",47.33333333333334],PARAMETER["Latitude_Of_Origin",45.33333333333334],UNIT["Foot_US",0.3048006096012192]],VERTCS["NAVD_1988",VDATUM["North_American_Vertical_Datum_1988"],PARAMETER["Vertical_Shift",0.0],PARAMETER["Direction",1.0],UNIT["Foot_US",0.3048006096012192]]

Screen Shot 2021-11-03 at 6.48.59 PM.png

I believe that you'll either need to make an adjustment to the service's spatial reference or disable local projections, which will serve results in EPSG 4326.

All comments

avonmoos

Can you provide a screenshot of where the local projections setting is? I can't locate it.

Also seeing the "API Error" on some of our datasets on our Open Data site, but not all of them, even though owners haven't been changed, these datasets are different sublayers out of the same ArcGIS Server services, etc. Developer Tools shows that for the erroring datasets, where the request is https://opendata.arcgis.com/api/v3/datasets/whateverhubidthisis/downloads?parameters=1234 the response is errors: message: "Not Found". (The previously mentioned https://od-jcwvgis.opendata.arcgis.com/datasets/8d4df0b309944ccf8b72bf0d279a168b_1/explore?location=39.311357%2C-77.850154%2C11.50 I see also has "Not Found" in Dev Tools for its download endpoint)

Taking a look at https://opendata.arcgis.com/api/v3/datasets, seems like that's where all Open Data datasets are, and at the very bottom it has a "last" page of https://opendata.arcgis.com/api/v3/datasets?page%5Bnumber%5D=1420043&page%5Bsize%5D=10 but going to that last page gets you a:

{"errors":[{"title":"Server Error","status":500,"message":"search_phase_execution_exception: [illegal_argument_exception] Reason: Result window is too large, from + size must be less than or equal to: [10000] but was [14200430]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}],"meta":{}}

With this in mind, I tried searching for my erroring datasets in https://hub.arcgis.com and as you can guess, they're not showing up; my download-OK datasets show up, but not the API error ones.

Esri: I think you can try increasing your [index.max_result_window] index level setting for the Open Data/ArcGIS Hub site to see if this fixes the API Errors that we're seeing? Worth a shot.

Thank you!

N8-CORR

My Hub site starting having this issue out of nowhere last week. Random layers display the error intermittently.

I saw this article posted by Esri: https://support.esri.com/en-us/knowledge-base/error-api-error-next-to-download-options-in-arcgis-hub-000026514 but our layers are not shared via distributed collaboration.

I also tried enabling local projects on the site but no dice. Are there any other causes to this issue @ThomasHervey1?

MichaelJenkins

We've just started to have this issue too. With a single layer in a map service that hosts multiple map layers. All other map layers from this service can be downloaded. https://gis-kingcounty.opendata.arcgis.com/datasets/kingcounty::zipcodes-for-king-county-and-surrounding-area-zipcode-area/explore?location=47.506014%2C-121.485100%2C9.66

MozaAlhemaidi

Also, i have this issue with one of my shared layer in the Hub site.

MattJinman

We are encountering the same issue. This was our biggest release of the year - it worked fine yesterday when I previewed the data. Now on release day I get API errors on all forms of data CSV, GeoJSON, Shapefile and KML.

I've raised a ticket with My Esri and see what they say?

JessicaGormont

I have just discovered that one of my layers is having this issue as well: https://od-jcwvgis.opendata.arcgis.com/datasets/8d4df0b309944ccf8b72bf0d279a168b_1/explore?location=39.311372%2C-77.850154%2C11.50

None of my other layers seem to have this issue. I tried republishing the layer and that didn't fix the problem. Our layer is using the ESRI extent, so it's not a projection issue.

Any suggestions?

JeffCoWV_roads_error.jpg

SimonAllard2

Seems to be working fine Paul! Our site has been acting up quite a bit of late. Sometime it works and sometimes we get the API error. It is very inconsistent.

pjdohertygis

I am also hitting this issue on personal use account (which means I cannot call Esri Support).

@ThomasHervey1 I tried the following with no luck...

1) Recreated the gallery

2) Update the Hub site settings to allow for local projection

Here is my site if you want to test it out https://data-pjdohertymaps.opendata.arcgis.com/

Here is the dataset I am trying to make available via Open Data

DianaKalil

I captured Web traffic with the URLs of the open data-Hub and it took me to: Resource was not found, and it was showing a 404 error.
Based on previous research I verified the item ID in ArcGIS Online and Spatial Reference, and it was 32145, so Spatial reference was not the issue.
I reviewed the REST services for other layers in the same and they worked for downloading. I suggested republishing the dataset since apparently there is no gap in the Spatial Reference or Metadata, and also verifying the export data settings.

AnibalMmartinez

Hola, tengo el mismo error en ARCGIS ENTERPRISE 10.9. con un servicio FeatureAccess: Enabed y compartido como publico ¿Alguna idea?

Saludos,

Anibal

valenj88

From the looks of this, it seems like having standalone tables without spatial component aren't supported for hub download?

JosephBisig

Hi Thomas!

These items now work on our Hub site. It did look like a projection issue that was solved by reloading the data into our map projects and republishing. Always appreciate the help Thomas!

Joey

https://gisdata-piercecowa.opendata.arcgis.com/datasets/pierce-county-council-districts/explore

the Spatial Reference is

Screen Shot 2021-11-03 at 6.48.59 PM.png

I believe that you'll either need to make an adjustment to the service's spatial reference or disable local projections, which will serve results in EPSG 4326.

JosephBisig

We started noticing this on three of our recently published feature services. This is what we see on our Historic Election Precincts 2004.

We are seeing these API errors occur in two other feature classes as well -

https://gisdata-piercecowa.opendata.arcgis.com/datasets/historic-election-precincts-1992/explore

Has this issue popped up again for anyone else?

Thanks,

Joey

Hello
We have just released a follow-up fix. Please check to see if you're still seeing an API Error on your downloads when navigating to content via a header link or a gallery card.

Hello,

I want to give an update on the status of this fix.
We have found a solution but we need time to ensure that automated and manual testing is comprehensive and that no other gallery cards or header links regress. As a result, we will not be able to release the fix today, but we are aiming to release as soon as possible.

When a fix has been released and verified on production, we will update hubstatus.arcgis.com

Hello again,

I apologize for the inconvenience. We have identified the issue and we're aiming to release a hotfix as soon as possible. Thank you for suggesting a work-around @John_Spence .

For future reference, ArcGIS Hub has a service-level status page found at https://hubstatus.arcgis.com/

Todd_Metzler

Good to know there is another workaround. Not practicable for our enterprise. We have hundreds of services and over 30 publishers.

We got past it by republishing the dataset.

Todd_Metzler

Same. First reported to us 2021-09-09. 1st stop was AGOL Health Dashboard . All green still green. We've opened our own case with Esri tech support.

Hi @ThomasHervey1

We seem to be in better shape - all but one of the layers corrected. I ran through the steps of changing sharing levels and removing and then re-adding to the gallery and the hub page but those steps didn't work. I'm going to wait until the next fix that was mentioned yesterday gets pushed through, but if there is no change by Friday morning, I think we are going to just have to remove it and set up that service again.

StanislausCounty

Hi Tom,

Yes, that is the site we are having trouble with. I'll try making them private and then public; with respect to the gallery cards, this was generated via the dynamic gallery layout. I'm assuming that removing it and re-adding it would generate the URLs again so I'll also give that a shot as well. We do appreciate your help!

Sorry to see that you're still having an issue. Is this the site where you're having issues https://open-data-stancounty-gis.hub.arcgis.com/ ?

I have reproduced the issue and I've marked a fix has high priority. In the meantime, I suggest that you try making those items private and then public again. Afterward you may need to remove the link and re-add them within the gallery cards.

The problem comes from the fact that these those gallery cards link out to items referencing services that have multiple layers. The update is still tying to reference the original URL without knowledge that this is a layer. For example, if you click the Annexations gallery card link, you'll route to https://open-data-stancounty-gis.hub.arcgis.com/datasets/d32e342982c849839aa787a1ae513f15_8/explore?layer=8&location=37.546472%2C-120.968143%2C11.62
instead users should be route to a slightly different URL
https://open-data-stancounty-gis.hub.arcgis.com/datasets/d32e342982c849839aa787a1ae513f15_8/explore?location=37.546472%2C-120.968143%2C11.62

We recognize that these bugs are very impactful and that the routing is confusing. Near-term we are adding additional tests to ensure that routing changes do not affect downloads. Mid-term we have plans to simplify the routing logic and remove the confusing _:layerId portion of the URL.

It cleared up for us, but we were also dropped the original feature service and rebuilt it from the ground up. Not a pleasant experience, but it let us get the issue resolved.

StanislausCounty

We are still experiencing the issue. I do see in the changelog that it has been fixed; has it been pushed out yet or is there still some problem?

Thank you for all of your help!

Appreciate the response and look forward to seeing the resolution.

We have verified that this is a bug and we are completing QA now. Will release it tomorrow mid-day EST during our normal weekly Hub release. Please stay tuned to the Hub changelog https://hub.arcgis.com/pages/changelog
Apologies again for the inconvenience and thank you for your patience.

Glad to see it is not just us.

StanislausCounty

Is there an ETA on when this will be fixed? We have quite a few members of the public who are trying to access our open data and are getting this error. I can confirm that the API does work when following @ThomasHervey1 method of adding the _0/explore string to the end of the url.

https://data.bellevuewa.gov/datasets/cobgis::bellevue-permits/explore?layer=0&location=47.589609%2C-122.160741%2C12.97

This issue is also affecting us, but it is not universally an issue on all datasets. Below is one example.

It is not an issue across the board though. The issue may be isolated to specific hives such as EYzEZbDhXZjURPbP.

Thanks @MeghanLeavey it looks like there is a routing bug from the gallery card. We're actively working on a fix and hope to release it as soon as possible.

If you need immediate access to downloads you can update the URL and adjust the layer information. For example, the routing resolution for "Building Footprints" will route to https://gis-monmouthnj.opendata.arcgis.com/datasets/fec0b3d813174cdfb766134315120460/explore?layer=0

You'll still see the view working correctly and downloads will work if you adjust by removing the query and adding "_0"
https://gis-monmouthnj.opendata.arcgis.com/datasets/fec0b3d813174cdfb766134315120460_0/explore

Hi @ThomasHervey1

Yes, our open data can be found here: https://gis-monmouthnj.opendata.arcgis.com/pages/open-data

Thank you!

Thanks for the additional information. Hub should be able to support numeric WKIDs like 27700. However, there may be a bug parsing the WKID that was supplied. We're investigating the issue right now. Can you please give a URL to an example item that is now working?

KevinDown

Thomas,

Thank you for reaching out to me via the ArcGIS Hub Questions thread above. I have since resolved my issue, and it was caused by the Spatial reference used not being a standard WKID. In fact the Spatial reference used was a customized one base on WKID 27700. Because it did not match WKID exactly, Hub could not find the WKID definition required for the API to initialize for the download. When I disabled local reference, the API enabled and +-

Hi,

I'm sorry to hear that you're having an issue with downloads. We're actively working on a bug fix where gallery cards are linking to the wrong dataset view. You may see that the dataset page will load, but downloads will not because it has the wrong layer information, as @MeghanLeavey pointed out. Meghan, can you provide a public URL for content on your Hub site that is having this issue?

After further researching and looking into the network traffic it seems that when the Hub is trying to GET the Request URL for the layers on our open data site, it is not "getting" the entire URL. The layer ID is dropped (e.g. instead of https://opendata.arcgis.com/api/v3/datasets/1bc60d61f55644dc9320624a8fbea521_0 being the Request URL our HUB site is trying to GET https://opendata.arcgis.com/api/v3/datasets/1bc60d61f55644dc9320624a8fbea521 )

Which creates the first resulting in the "404 Not Found" error. The additional error is a "500 Internal Server Error" when you click on the open data download icon and an API Error message instead of being able to download the data.

I have been able to test this with another Open Data site on which our GIS data is accessible. Our site gets the API error but on the other site I can download the same data from the same feature services without issue.

How does this get resolved?

404_error_hub1.JPG

200_OK_hub2.JPG