topic Re: DCAT feed URLs return 404 in ArcGIS Hub Questions

DCAT feed URLs return 404

Anonymous User — Sun, 01 Dec 2019 21:50:05 GMT

The DCAT feed for our Open Data site returns urls which currently throw a 404 error. I've looked at a few sites' DCAT data links and they're also throwing a 404 error. Our site is https://data-transpower.opendata.arcgis.com/ and the DCAT feed lists license urls like http://data-transpower.opendata.arcgis.com/datasets/0e8f361371914a6f95133f58d34ffcb9_0/license.json which are throwing a 404 error. On the DCAT Feed preview page (see attachment) the links point to http://opendata.arcgis.com/... which works e.g. https://opendata.arcgis.com/datasets/0e8f361371914a6f95133f58d34ffcb9_0/license.json

Re: DCAT feed URLs return 404

NickShannon2 — Tue, 03 Dec 2019 03:34:57 GMT

This is a known bug. I submitted the bug report back in August!

BUG-000124770: The DCAT attributes for a ArcGIS Hub site's data.jso..

Perhaps you could escalate this bug with Esri Support to help with expediting a fix.

Re: DCAT feed URLs return 404

Anonymous User — Tue, 03 Dec 2019 03:45:58 GMT

Thanks Nick. I've logged it with our local Esri vendor, who have raised it with Esri. Hopefully it gets some traction!

Re: DCAT feed URLs return 404

NickShannon2 — Wed, 22 Apr 2020 09:35:39 GMT

Esri has responded to the bug report:

BUG-000124770: The DCAT attributes for a ArcGIS Hub site's data.jso..

ArcGIS Hub does not support DCAT for private sites. As user report that this is new behavior, perhaps a bug was recently fixed in which, Hub previously exposed the …/data.json endpoint for private sites when it should not have been doing that in the first place. Also, looking at the DCAT configuration for the public site referred to in the bug report, the email address points to a field that does not exist (in the metadata for the item). If the user wants that to be valid, ensure the metadata for the item actually contains that field. We would be willing to work with user to see what is exactly going on in their particular configuration or scenario.

Private sites?

I have no idea what the technician is talking about. Any ideas?

Re: DCAT feed URLs return 404

Anonymous User — Wed, 22 Apr 2020 12:48:10 GMT

we'll take a look!

Re: DCAT feed URLs return 404

Anonymous User — Wed, 22 Apr 2020 14:38:47 GMT

Hi Nick,

Not sure where the wires got crossed, but the bug you link has to do with formatting changes in the data.json site when shared publicly and I'm not seeing anything about license.json in the ticket. That said, we're seeing that license.json endpoints are 404ing for all sites and are currently working on a fix. Sorry for the confusion on this one.

Best,

Patrick

Re: DCAT feed URLs return 404

NickShannon2 — Thu, 23 Apr 2020 01:39:29 GMT

Thanks Patrick for clearing that up.

Bug-000124770 was marked as 'related' in the My Esri support ticket.

Is this there a Bug case number for the license.json issue?

Re: DCAT feed URLs return 404

Anonymous User — Thu, 23 Apr 2020 05:27:52 GMT

As far as I know there's no bug number for the license.json issue I raised as my case (#02450125) was also attached to BUG-000124770. Based on the provided additional info for BUG-000124770 I contacted Graham Hudgins with the following info:

We’ve got an ArcGIS Hub site https://data-transpower.opendata.arcgis.com/ that has a DCAT feed https://data-transpower.opendata.arcgis.com/data.json which feeds our national Government data site https://catalogue.data.govt.nz/dataset .

Unfortunately the DCAT feed has a couple of problems with it:
Some of the feed items don’t work e.g. our feed contains the following ‘license’ link which throws a 404 error http://data-transpower.opendata.arcgis.com/datasets/e507953ab8934fc3a115b2e79226cbd6_0/license.json
If I change the url to remove ‘data-transpower’ from the start of it i.e. http://opendata.arcgis.com/datasets/e507953ab8934fc3a115b2e79226cbd6_0/license.json it works and the details are correct
All links on the data.json feed (aside from the service URLs) are http instead of https. Our site has 'Enforce https' enabled in case that makes a difference.
The first issue was logged as a case (#02450125) under BUG-000124770 . This bug report has been updated recently (basically closed) but I don’t think our case is actually relevant to that bug (you might be able to advise better on whether it is or is not related?). Our site is public and the DCAT is still an issue. The https issue is under BUG-000116376 . We’ve got an attached case with that bug too, case #02455988.

Am hoping between Patrick and Graham it'll get sorted

Re: DCAT feed URLs return 404

jonathanharte — Thu, 28 May 2020 18:28:56 GMT

Hi … has this been resolved now?

I have a similar issue with HUB

https://www.arcgis.com/home/item.html?id=e8505fbd61e642a088df2eb58deccd7d - My AGOL item

my test HUB item

Bedrock Geology 1:100,000 Ireland ITM Feature Service and this returns a 404 for the license.json. Without this I cannot get CKAN (externally) to harvest with a star rating since it cannot find the licencing reference.

https://gsidcat-dcenr.hub.arcgis.com/datasets/e8505fbd61e642a088df2eb58deccd7d_0/license.json

Any ideas please .. I'm losing direction with this now I have tried so many things.

Regards

Paulg

Re: DCAT feed URLs return 404

Anonymous User — Thu, 28 May 2020 22:58:22 GMT

Unfortunately not, it's still to be fixed.

If you go to Configure DCAT and click the Preview tab, what's the license link it shows there?

John Plunkett‌

Re: DCAT feed URLs return 404

Department_of_Communications__ — Fri, 05 Jun 2020 13:33:03 GMT

Hi Mark, Thanks for the reply, your post prompted me to take a closer look and basically mine was exactly like yours.

Here is the before and after version (post a bit of fiddling)

Before

"@type": "dcat:Catalog",
"conformsTo": "https://project-open-data.cio.gov/v1.1/schema",
"describedBy": "https://project-open-data.cio.gov/v1.1/schema/catalog.json",
"dataset": [
{
"@type": "dcat:Dataset",
"identifier": "http://opendata.arcgis.com/datasets/7e95eba45eb24855aa320c10e5c4b349_0",
"license": "http://opendata.arcgis.com/datasets/7e95eba45eb24855aa320c10e5c4b349_0/license.json",
"landingPage": "http://opendata.arcgis.com/datasets/7e95eba45eb24855aa320c10e5c4b349_0",
"title": "LOADME",
"description": "{{default.description}}",
"keyword": [
"DCAT_TEST"

After

{
"@context": "https://project-open-data.cio.gov/v1.1/schema/catalog.jsonld",
"@type": "dcat:Catalog",
"conformsTo": "https://project-open-data.cio.gov/v1.1/schema",
"describedBy": "https://project-open-data.cio.gov/v1.1/schema/catalog.json",
"dataset": [
{
"@type": "dcat:Dataset",
"identifier": "http://opendata.arcgis.com/datasets/7e95eba45eb24855aa320c10e5c4b349_0",
"license": "https://creativecommons.org/licenses/by/4.0/",
"landingPage": "http://opendata.arcgis.com/datasets/7e95eba45eb24855aa320c10e5c4b349_0",
"title": "LOADME",
"description": "{{default.description}}",
"keyword": [
"DCAT_TEST"
],
"issued": "2020-05-29T10:25:21.000Z",

So I now know I can change the content of the DCAT and for this particular example I just hard coded it with a url as for the moment but I also got it working with the dcat elements that popup on the righthand side when your editing (excuse the terminology I couldn't find and help or direction on this)

I just added one line into the DCAT configuration (bold below) but it can be dynamic using the metadata item

{
"title": "{{default.name}}",
"description": "{{default.description}}",
"keyword": "{{item.tags}}",
"issued": "{{item.created:toISO}}",
"modified": "{{item.modified:toISO}}",
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"source": "{{default.source.source}}"
},
"contactPoint": {
"hasEmail": "{{metadata.metadata.mdContact.rpCntInfo.cntAddress.eMailAdd}}",
"mbox": "info@gsi.ie"
}
}

So for the moment it works ... I just have to better understand the whole configuration aspect. Thanks for the post

Paulg

Re: DCAT feed URLs return 404

Anonymous User — Mon, 08 Jun 2020 00:21:15 GMT

Hi Paulg,

Yes, you can manually configure the DCAT based on DCAT attributes or just hard code it. It's just not so useful when you have mixed license content.

You could try "license": "{{item.licenseInfo}}" instead of hardcoding it. I would change our one to that setting, but the agency that reads our DCAT expects it as a url, pointing to a json file, so it doesn't work for them.

Re: DCAT feed URLs return 404

jonathanharte — Mon, 08 Jun 2020 07:56:46 GMT

Hi Mark,

Yes your correct … I need the license.json response also, my excitement cloudy my judgement 🙂

I feel its time to log a support call.

Thanks for taking the time to respond

Paulg

Re: DCAT feed URLs return 404

ThomasHervey1 — Fri, 19 Jun 2020 13:51:14 GMT

Hi jonathan harte‌ we have work schedule to fix the license.json endpoint. The DCAT editor is about to get an update and so this bug will be investigated during the process.

Re: DCAT feed URLs return 404

Anonymous User — Mon, 10 Aug 2020 20:44:38 GMT

Thomas Hervey any update on this? The Hub change log mentions the DCAT editor has been updated (which I've seen in the product) but the license.json endpoint hasn't changed.

Re: DCAT feed URLs return 404

ThomasHervey1 — Mon, 10 Aug 2020 21:21:40 GMT

Thanks for checking back. This was not fixed during the UI updates that went out. There is work scheduled for this week fix several DCAT issues including correcting 404s for license endpoints and update the links to HTTPS.

Re: DCAT feed URLs return 404

ThomasHervey1 — Mon, 19 Oct 2020 12:25:56 GMT

The 404 on license endpoints has not yet been corrected. I apologize for the delay and inconvenience. We currently do not generate licenses to populate this endpoint, but we're looking forward to work within the next few months to build a license picker, allowing users to set reuse licenses on content. This 404 issue will be corrected then.

For additional context, Hub is also looking to expand and support other catalogs like DCAT-AP. The initial work for during expansion involves a critical look at Hub's DCAT template and will attempt to resolve any other outstanding issues.