topic Re: Delete orphaned dataset in open data site? in ArcGIS Hub Questions

Delete orphaned dataset in open data site?

TomCasady — Mon, 30 Jan 2017 19:22:50 GMT

I have a data set in my open data site, for which the underlying data has been deleted. I can't get the data set listing off the open data site, though. I've tried everything. Any advice?

Re: Delete orphaned dataset in open data site?

CourtneyClaessens — Mon, 30 Jan 2017 21:06:48 GMT

Hi Tom -

Can you please provide the URL of the orphaned dataset?

Thanks

Courtney

Re: Delete orphaned dataset in open data site?

TomCasady — Tue, 31 Jan 2017 19:22:35 GMT

https://lincolnne.maps.arcgis.com/home/item.html?id=e166951c5ef3423f809794d525e19192

Re: Delete orphaned dataset in open data site?

AdamMesser1 — Thu, 02 Feb 2017 23:06:35 GMT

Courtney,

I think we have the same problem on our site. We (bdaigle) are evaluating the discrepancy between the number of items in our agol open data group (n=130), the OD Site Manager summary (126) and data tab (136 datasets 132 items). It appears that this is a continuation of Kevin Dunlop‌ issue of Phantom items in Open Data Beta‌ . Is the current solution to still send you the list of these items for deletion? I assuming the bug is still being worked on by your engineering team?

Re: Delete orphaned dataset in open data site?

TomCasady — Tue, 07 Feb 2017 02:42:36 GMT

I suspect that this resulted when I deleted the underlying data from My Content, without "unsharing" it with the open data group first.

Re: Delete orphaned dataset in open data site?

CourtneyClaessens — Tue, 07 Feb 2017 15:01:14 GMT

Thanks Tom. We've removed your dataset.

Re: Delete orphaned dataset in open data site?

CourtneyClaessens — Tue, 07 Feb 2017 15:05:03 GMT

Hi Adam,

Yes, please send me the list of datasets. We are continuing to fix the bugs as we discover new ways this occurs. Can you please share the workflow you took before you saw this occur?

Courtney

Re: Delete orphaned dataset in open data site?

AdamMesser1 — Thu, 09 Feb 2017 16:21:46 GMT

Courtney,
Our process is scripted so I can describe that it, but likely cannot point you to a specific point when this occurred. We (Bill Daigle LydiaBailey Bill Daigle) maintain feature classes in SDE. When edits occur the data manager will designate the layer be updated in all dependent locations, including AGOL as a hosted feature service. Very large items are loaded as document links that point back to our web server for download. The feature service upload process takes one of two paths, with all tasks being automated via api calls which mostly use github ArcREST api package functionality. The only exception is the full item upload via arcpy.UploadServiceDefinition. If the layer is new or has a schema change, the existing AGOL feature service is deleted and replaced. If the layer only has record level edits, the features on the existing AGOL hosted feature service are deleted and then new records inserted into that item. If errors occur in the second process, we revert back to the first full item replacement option. Items are shared with everyone and our OpenData group which contains all items to be displayed on our OpenData site.

We are aware of issues with item replacement due to the object id being changed. However, this is lower priority since the feature service url does not change, and thus we haven't noted major negative impacts on map and apps pointing to those replaced feature service items.

Following is a summary of our existing data issues related to inconsistencies between our OpenDataGroup and OpenDataSite. Count of items in each location; Items in our ODGroup that should be showing up in the ODSite; Items that are orphaned, not in our ODGroup nor on our AGOL site anymore, and should be removed from the ODSite.

Our evaluations also picked up erroneous duplicate records over the last several days, but those appear to have been cleaned up or fixed as of this morning. In that case, a search for a specific record would show one result on screen, but list 2 results and show duplicates with the same item id when the returned json was evaluated.

Thanks for your help, Adam

________________________
Open Data Site Items:133
Open Data Group Items:132
________________________
In Open Data group, but not on Open Data site
____________
    id: 991f5239cf814ec6b86628b9e490fddd
name: AIS Watercraft Inspection Stations
type: Feature Service
   url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FISH_AIS_INSPECTION_STATIONS/FeatureServer
____________
    id: b938ee0d87a1480aa988109cb963e13b
name: Montana Lakes
type: Document Link
   url: http://gis.fwp.mt.gov/arcgis101/rest/directories/arcgisoutput/webResources/metadata/refrnc/lakes.htm
________________________
On Open Data site, but not in Open Data group
____________
    ITEM ID NOT IN MTFWP ITEMS LIST
      id: 2b095b1f50074ad3803c7535804f50f9
    name: Snowmobile Trails Program PDF Map Index
    type: ItemLayer
     url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FWPLND_WINTER_PDF_MAP_INDEX/FeatureServer/0
____________
    ITEM ID NOT IN MTFWP ITEMS LIST
      id: 3a7ccdc9db804845b126d5bcaaed32fe
    name: Winter Trails
    type: ItemLayer
     url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FWPLND_TRAILS_WINTER/FeatureServer/0
____________
    ITEM ID NOT IN MTFWP ITEMS LIST
      id: bbec615bc12446539758cee15a6868fe
    name: Montana Fish, Wildlife & Parks Warden Districts
    type: ItemLayer
     url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/ADMBND_RESPAREA_WARDEN/FeatureServer/0
________________________

Re: Delete orphaned dataset in open data site?

AdamMesser1 — Mon, 13 Feb 2017 18:06:59 GMT

Courtney Claessens‌ ,

Just a heads up that our GIS manager has logged this issue as a ticket as of last week and we are communicating with ESRI support. The first group above "In Open Data group, but not on Open Data site" has been resolved. The second "On Open Data site, but not in Open Data group" has not and now includes an additional item.

Re: Delete orphaned dataset in open data site?

AdamMesser1 — Tue, 21 Feb 2017 04:22:13 GMT

All of our Open Data orphans have been resolved by ESRI support. All of our data visible on our OD site is through a single Open Data group on our AGOL site. Apparently the following steps removed the orphan layers: From the OD administration pages the OD Group was removed from the site. From AGOL the OD group was changed from Public to Private. These steps were then reversed. From AGOL the OD group was changed from Private to Public, and the group was added back to the OD site. Then on the OD site indexes were updated and the download cache reset.

Re: Delete orphaned dataset in open data site?

GyaneshGounder — Fri, 18 Aug 2017 01:21:33 GMT

Hi I am having the same issue with orphan datasets

CClaessens-esristaff‌ please advise

Re: Delete orphaned dataset in open data site?

CourtneyClaessens — Fri, 18 Aug 2017 14:29:32 GMT

Hi Gyanesh,

Can you please send me the URLs of the datasets that are orphaned?

Courtney

Re: Delete orphaned dataset in open data site?

AdamMesser1 — Fri, 18 Aug 2017 15:31:19 GMT

Courtney, It was nice to meet you at the UC. As we discussed following up on any future issues I'm sending another orphan/duplicate record from OD. The Montana Lakes record is a document link so no specific url for the item on our OD site. It can be located via search on our current our legacy site: Search | Montana Fish, Wildlife & Parks Item id: 41ed181dc6124fa9a2955bcb127b4a9b; You will note the 'Lakes' search returns 7 records in the summary and the page json response, but there are only 6 listed. Testing on our new test OD site, shows this duplicate record in the list. There is only one item on our agol content list. If I unshare this item from our OpenData group, it will remove the valid record from our OD site after a short time but leave the orphan. Let me know if you need any additional information. Thanks again, Adam

Re: Delete orphaned dataset in open data site?

GyaneshGounder — Sun, 20 Aug 2017 20:12:25 GMT

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_1

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_2

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_3

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_4

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_5

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_7

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_8

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_9

http://data-watercare.opendata.arcgis.com/datasets/eefc9d8e461d425b9a44f671ce5bb04a_10

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_1

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_2

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_3

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_4

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_5

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_6

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_7

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_8

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_9

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_11

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_12

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_13

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_14

http://data-watercare.opendata.arcgis.com/datasets/a47b0e20539a41e6b9eff8d47e878a2b_15

Thanks

Re: Delete orphaned dataset in open data site?

TaupoDistrict_Council — Thu, 05 Nov 2020 05:10:42 GMT

Hi Courtney, we are experiencing the same issue with orphaned datasets in our data portal. Are you able to remove them for us if I send the URLs? Thanks

Re: Delete orphaned dataset in open data site?

Anonymous User — Thu, 05 Nov 2020 14:01:51 GMT

sorry to hear about that - go ahead and send them to us so we can take a look!

Re: Delete orphaned dataset in open data site?

TaupoDistrict_Council — Thu, 05 Nov 2020 19:37:03 GMT

Thanks so much Graham! Really appreciate that. The URLs for the orphaned items to be removed are:

https://data-waikatolass.opendata.arcgis.com/datasets/83c9f3306b2147fb85867fa33ba52726_0

https://data-waikatolass.opendata.arcgis.com/datasets/b88a247c220b459bbf085faafe3d85f0_0

https://data-waikatolass.opendata.arcgis.com/datasets/a8c5c009e2504e4a91591ef707d20730_0

https://data-waikatolass.opendata.arcgis.com/datasets/7ca2c17c56c8472db2629ed3ef3b6449_0

https://data-waikatolass.opendata.arcgis.com/datasets/ffe2f3ae85d445adac787b6a1886a8cd_0

Re: Delete orphaned dataset in open data site?

TaupoDistrict_Council — Thu, 12 Nov 2020 21:31:26 GMT

@Anonymous User any update on this? Thanks so much

Re: Delete orphaned dataset in open data site?

Anonymous User — Thu, 19 Nov 2020 16:53:11 GMT

Sorry I was out on holiday. Taking a look now

Re: Delete orphaned dataset in open data site?

ThomasHervey1 — Mon, 30 Nov 2020 13:56:55 GMT

Hi @TaupoDistrict_Council we can delete these orphaned datasets now. We are investigating why you would see this. When did you first notice duplicates? did you republish the layers?