I have a data set in my open data site, for which the underlying data has been deleted. I can't get the data set listing off the open data site, though. I've tried everything. Any advice?
Solved! Go to Solution.
Thanks Tom. We've removed your dataset.
Hi Tom -
Can you please provide the URL of the orphaned dataset?
Thanks
Courtney
Courtney,
I think we have the same problem on our site. We (bdaigle) are evaluating the discrepancy between the number of items in our agol open data group (n=130), the OD Site Manager summary (126) and data tab (136 datasets 132 items). It appears that this is a continuation of Kevin Dunlop issue of Phantom items in Open Data Beta . Is the current solution to still send you the list of these items for deletion? I assuming the bug is still being worked on by your engineering team?
Hi Adam,
Yes, please send me the list of datasets. We are continuing to fix the bugs as we discover new ways this occurs. Can you please share the workflow you took before you saw this occur?
Courtney
Courtney,
Our process is scripted so I can describe that it, but likely cannot point you to a specific point when this occurred. We (Bill DaigleLydiaBaileyBill Daigle) maintain feature classes in SDE. When edits occur the data manager will designate the layer be updated in all dependent locations, including AGOL as a hosted feature service. Very large items are loaded as document links that point back to our web server for download. The feature service upload process takes one of two paths, with all tasks being automated via api calls which mostly use github ArcREST api package functionality. The only exception is the full item upload via arcpy.UploadServiceDefinition. If the layer is new or has a schema change, the existing AGOL feature service is deleted and replaced. If the layer only has record level edits, the features on the existing AGOL hosted feature service are deleted and then new records inserted into that item. If errors occur in the second process, we revert back to the first full item replacement option. Items are shared with everyone and our OpenData group which contains all items to be displayed on our OpenData site.
We are aware of issues with item replacement due to the object id being changed. However, this is lower priority since the feature service url does not change, and thus we haven't noted major negative impacts on map and apps pointing to those replaced feature service items.
Following is a summary of our existing data issues related to inconsistencies between our OpenDataGroup and OpenDataSite. Count of items in each location; Items in our ODGroup that should be showing up in the ODSite; Items that are orphaned, not in our ODGroup nor on our AGOL site anymore, and should be removed from the ODSite.
Our evaluations also picked up erroneous duplicate records over the last several days, but those appear to have been cleaned up or fixed as of this morning. In that case, a search for a specific record would show one result on screen, but list 2 results and show duplicates with the same item id when the returned json was evaluated.
Thanks for your help, Adam
________________________
Open Data Site Items:133
Open Data Group Items:132
________________________
In Open Data group, but not on Open Data site
____________
id: 991f5239cf814ec6b86628b9e490fddd
name: AIS Watercraft Inspection Stations
type: Feature Service
url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FISH_AIS_INSPECTION_STATIONS/Feat...
____________
id: b938ee0d87a1480aa988109cb963e13b
name: Montana Lakes
type: Document Link
url: http://gis.fwp.mt.gov/arcgis101/rest/directories/arcgisoutput/webResources/metadata/refrnc/lakes.htm
________________________
On Open Data site, but not in Open Data group
____________
ITEM ID NOT IN MTFWP ITEMS LIST
id: 2b095b1f50074ad3803c7535804f50f9
name: Snowmobile Trails Program PDF Map Index
type: ItemLayer
url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FWPLND_WINTER_PDF_MAP_INDEX/Featu...
____________
ITEM ID NOT IN MTFWP ITEMS LIST
id: 3a7ccdc9db804845b126d5bcaaed32fe
name: Winter Trails
type: ItemLayer
url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/FWPLND_TRAILS_WINTER/FeatureServe...
____________
ITEM ID NOT IN MTFWP ITEMS LIST
id: bbec615bc12446539758cee15a6868fe
name: Montana Fish, Wildlife & Parks Warden Districts
type: ItemLayer
url: https://services3.arcgis.com/Cdxz8r11hT0MGzg1/arcgis/rest/services/ADMBND_RESPAREA_WARDEN/FeatureSer...
________________________
Just a heads up that our GIS manager has logged this issue as a ticket as of last week and we are communicating with ESRI support. The first group above "In Open Data group, but not on Open Data site" has been resolved. The second "On Open Data site, but not in Open Data group" has not and now includes an additional item.
All of our Open Data orphans have been resolved by ESRI support. All of our data visible on our OD site is through a single Open Data group on our AGOL site. Apparently the following steps removed the orphan layers: From the OD administration pages the OD Group was removed from the site. From AGOL the OD group was changed from Public to Private. These steps were then reversed. From AGOL the OD group was changed from Private to Public, and the group was added back to the OD site. Then on the OD site indexes were updated and the download cache reset.
Courtney, It was nice to meet you at the UC. As we discussed following up on any future issues I'm sending another orphan/duplicate record from OD. The Montana Lakes record is a document link so no specific url for the item on our OD site. It can be located via search on our current our legacy site: Search | Montana Fish, Wildlife & Parks Item id: 41ed181dc6124fa9a2955bcb127b4a9b; You will note the 'Lakes' search returns 7 records in the summary and the page json response, but there are only 6 listed. Testing on our new test OD site, shows this duplicate record in the list. There is only one item on our agol content list. If I unshare this item from our OpenData group, it will remove the valid record from our OD site after a short time but leave the orphan. Let me know if you need any additional information. Thanks again, Adam
Hi I am having the same issue with orphan datasets
CClaessens-esristaff please advise