topic Re: Feature service which has no datasets? in ArcGIS Hub Questions

Feature service which has no datasets?

EricHanson — Mon, 22 Dec 2014 18:25:06 GMT

I have been having some issues with open data lately.

I am using a map service from my own ArcGIS Server. I added items for each layer of the map service as individual items in AGOL as an item from the web with the service url for each layer. I then shared the items (which are public) with my open data group which is Public and designated as available for use in Open Data.

In my open data site (which is currently Private for testing), I then added this group in the Manage Data tab.

A couple of weeks ago, this worked fine and all the data was showing properly. But now when I view the Data Report, it says “feature service which has no datasets” and in the Manage Sites page, it lists that the Open Data site has items but Datasets are n/a even though when I view the items in AGOL or query the layers in the map service, they work fine.

Has anyone else seen this issue lately or know what the problem/solution is? Is this related to the recent update? FYI – AGOL hosted services work just fine.

Thank you!

Re: Feature service which has no datasets?

EricHanson — Tue, 30 Dec 2014 22:14:26 GMT

This seems to be a bug: http://support.esri.com/en/bugs/nimbus/QlVHLTAwMDA4MjQxNQ==

At this time, there is no workaround and no information on when it will be fixed.

One solution would be to have AGOL host the data rather than having it come from your organizations ArcGIS Server, but that does use credits - and depending on how much data you have, that could cost quite a bit of credits every month.

Hopefully ESRI can resolve this soon.

Re: Feature service which has no datasets?

ChristopherBoland — Wed, 14 Jan 2015 13:52:19 GMT

Eric,

I have also found this problem. I've even recalculated the index for the site a couple of time to no avail. I've contacted a member of the ESRI OpenData Development team about this to see where we stand.

Chris

Re: Feature service which has no datasets?

ChristopherBoland — Wed, 14 Jan 2015 14:36:33 GMT

I spoke with the development team this morning and they thought that this issue was fixed already. They said that they'll resolve this ASAP.

Re: Feature service which has no datasets?

DanielFenton1 — Wed, 14 Jan 2015 20:10:13 GMT

Have you guys tried making the private and then public again? That triggers a total wipe and reharvest of the individual item. Can you also share the URL of any of the items you are having issues with?

Sorry for the additional trouble with this.

Re: Feature service which has no datasets?

ChristopherBoland — Wed, 14 Jan 2015 20:26:38 GMT

Hi Daniel,

I had not tried making the items private and then public and shared with OpenData again. This seems to be working for some datasets that were having that issue, but not others.

Currently, this item is being shared with my Open Data group, but in the data report it says "feature service which has no datasets." I will continue retracting my Shares and then reinstating my shares to see if this ultimately fixed many of my problem datasets.

Chris

Re: Feature service which has no datasets?

EricHanson — Wed, 14 Jan 2015 20:59:54 GMT

Hi Daniel,

The site I am working on is currently private and has not been made public yet. We are waiting to have these issues fixed before we make it public.

-Eric

Re: Feature service which has no datasets?

EricHanson — Wed, 14 Jan 2015 21:11:18 GMT

Hi Chris,

The bug still looks like it is active.

I tried to escalate the bug with Esri through the Customer Care Center a couple of weeks ago, but I have not heard anything back yet.

I was told through tech support that it may not get resolved until the next quarterly release, which I would assume is in March.

Re: Feature service which has no datasets?

DanielFenton1 — Wed, 14 Jan 2015 21:33:34 GMT

Hi Eric, we don't actually release every quarter, its more like every 2-3 weeks, so we should have resolution well before March. We are currently experiencing an abnormally high volume of harvesting jobs and we are working through those. Time may solve the issue. Could you please share any of the items you are seeing the problem with?

Re: Feature service which has no datasets?

ChristopherBoland — Thu, 15 Jan 2015 14:57:51 GMT

Hi Daniel,

I checked this morning and it appears that all datasets are now working in open data. It may be that the indexing is just taking about 24 hours or so from when the item was created.

Chris

Re: Feature service which has no datasets?

DanielFenton1 — Thu, 15 Jan 2015 17:59:56 GMT

Glad to hear it, Chris. We're still working through those issues with harvest volume. Once those are resolved things will return to normal speed.

Re: Feature service which has no datasets?

DanielFenton1 — Fri, 16 Jan 2015 15:56:42 GMT

Hey guys, we have dealt with the harvesting backlog. If unsharing and sharing datasets doesn't solve any lingering problems please let me know.

cc Christopher Boland‌

Re: Feature service which has no datasets?

EricHanson — Fri, 16 Jan 2015 16:18:17 GMT

Hi Daniel,

I tried to send you an email through esriDC@esri.com on Wednesday with the item, as I am not ready to share the items yet. I am still having issues. Nothing has changed - still showing no datasets.

-Eric

Re: Feature service which has no datasets?

TobiasFimpel1 — Thu, 22 Jan 2015 03:23:59 GMT

Chris, Daniel, Eric:

Many thanks for haring your information here. I really appreciate it. I am in the same situation and so far making the item private and the public again seems to work - do you have any info on whether this workaround always works, or whether it doesn't work for some cases? Have you found it reliable enough to make your sites public yet?

Also do you know of a place where I can read up on how the "data harvesting" works? What is ArcGIS Online doing under the hood?

Thanks again,

Tobias

Re: Feature service which has no datasets?

EricHanson — Thu, 22 Jan 2015 19:45:04 GMT

Update:

I have been able to get other organizations map services to work with my open data site, but I still cannot get my map services to work. These services are publicly accessible as they do work in public applications, but yet they do not seem to get recognized by open data.

Anyone have any suggestions? Could this be a setting on the ArcGIS Server side when creating the map services or something to do with the ArcGIS Server infrastructure that would not allow the services to be recognized?

I had asked this questions to Esri Customer Service a few weeks ago, but there was limited response to that question (What are the reasons a map service would not be recognized by the open data site?).

Thanks!

Re: Feature service which has no datasets?

DanielFenton1 — Thu, 22 Jan 2015 20:12:37 GMT

Hi Tobias,

This reason this works is because unsharing the ArcGIS item will completely remove it from our database and reharvest the item completely. This is as opposed to when there is a change made to an ArcGIS item which will trigger an update only. It should always work unless there is a problem with the item itself or some temporary issue with our harvesters.

Harvesting is kicked off when:

1. A new public item is shared in an open data group

2. A non-public item in an open data group is made public

3. A change is made to an item that is in an open data group and is public

4. A group that was not previously enabled for open data is enabled for open data

For map services, under the hood our harvesters:

1. fetch the arcgis online item

2. fetch the map service url

3. fetch all the child layers

4. conduct statistics on all the fields of each layer

5. add the datasets to the open data index

The time it takes to conduct step 4 is a function of [the number of layers, the size of each layer (i.e. feature count), the number of attributes of each later]. And that can cause delays in items showing up in the index.

We have literally hundreds of customers who have found harvesting stable enough to launch their sites. You can check out over 100 of them at dc.esri.com/showcase .

I hope you find this information useful. Please let me know if you have any other questions.

Re: Feature service which has no datasets?

DanielFenton1 — Thu, 22 Jan 2015 20:13:16 GMT

Eric, would you mind sharing the ArcGIS online item as well as the URL to the map service? I'll take a look and diagnose where the problem lies.

Re: Feature service which has no datasets?

TobiasFimpel1 — Thu, 22 Jan 2015 20:29:17 GMT

Extremely helpful information, Daniel. Thank you for taking the time. I have a follow up question (?how to move ti to a new thread?):

In the documentation that I have read so far I couldn't find any best practices for configuring ArcGIS Server map/feature services for optimum performance with ArcGIS Online Open Data. Do you have any recommendations, or does it really not matter as long as the querying capability is enabled? (Min/max instances, high/low isolation, max. # features returned, service timeout, etc....)

Thanks again,

Tobias

Re: Feature service which has no datasets?

EricHanson — Thu, 22 Jan 2015 20:41:09 GMT

Hi Daniel.

Here is an item I have tried and the corresponding URL.

Thanks for your help!

Re: Feature service which has no datasets?

DanielFenton1 — Thu, 22 Jan 2015 20:48:30 GMT

Eric, can you please make sure that item is public? I'm unable to view it.