Select to view content in your preferred language

Hosted feature layer size same for subset of data

899
4
Jump to solution
10-19-2022 10:34 AM
julian_svcs
Frequent Contributor

I created a hosted feature layer with all its features (294162 features) and the size is 199.156MB. I then added a definition query in ArcGIS Pro to show only the first 20 features. When I published the subset of data as a hosted feature layer, the size is 199.18MB. The subset layer shows 20 features in the attribute table.

Why would the size of the subset layer with 20 features be the same (or actually slightly larger) than the original layer with 294162 features?

Original layer details:

julian_svcs_0-1666200751153.png

julian_svcs_1-1666200776196.png

 

Subset layer details:

julian_svcs_2-1666200820189.png

julian_svcs_3-1666200841073.png

 

 

0 Kudos
1 Solution

Accepted Solutions
julian_svcs
Frequent Contributor

Seems like this is by design, which I think is not ideal. I would have thought that if you have a definition query on the layer, you only want those to be published and not everything.

This also has other issues associated with it:

  • Performance of effecting the definition query in the service
  • AGOL credit usage
  • Security - might have records published in AGOL which could potentially be exposed if someone was able to remove the definition query (unlikely, but possible)

Here is an excerpt from this site:

Note:

If the layers in the ArcMap or ArcGIS Pro map have definition queries applied to them, ArcGIS applies the definition queries to the published hosted feature layer. However, all data is uploaded to ArcGIS Online. If you want only a subset of the data to be published to and stored in ArcGIS Online, you must export the subset of the data to another feature class and publish that.

If you intend to overwrite the hosted feature layer to update it, be sure to choose the option to Preserve layer IDs when you publish the feature layer from ArcGIS Pro. This option is not available in ArcMap.

View solution in original post

0 Kudos
4 Replies
jcarlson
MVP Esteemed Contributor

I believe that size stat comes from the service definition file that is uploaded, not the service itself. I've noticed that my layers don't change their size even when all existing records or truncated, or also when the number of records more than doubles.

- Josh Carlson
Kendall County GIS
0 Kudos
julian_svcs
Frequent Contributor

Seems like this is by design, which I think is not ideal. I would have thought that if you have a definition query on the layer, you only want those to be published and not everything.

This also has other issues associated with it:

  • Performance of effecting the definition query in the service
  • AGOL credit usage
  • Security - might have records published in AGOL which could potentially be exposed if someone was able to remove the definition query (unlikely, but possible)

Here is an excerpt from this site:

Note:

If the layers in the ArcMap or ArcGIS Pro map have definition queries applied to them, ArcGIS applies the definition queries to the published hosted feature layer. However, all data is uploaded to ArcGIS Online. If you want only a subset of the data to be published to and stored in ArcGIS Online, you must export the subset of the data to another feature class and publish that.

If you intend to overwrite the hosted feature layer to update it, be sure to choose the option to Preserve layer IDs when you publish the feature layer from ArcGIS Pro. This option is not available in ArcMap.

0 Kudos
AlfredBaldenweck
MVP Regular Contributor

That's so weird. Are we able to remove the definition query once it's been uploaded?

0 Kudos
julian_svcs
Frequent Contributor

Yes I can. If I go into the REST's admin endpoint and I can edit the layer's definition in the updateDefinition. If I remove the 'lastEditDate' and remove the 'definitionQuery', I get the full set of features in my hosted feature layer.

I know it is really unlikely for someone to be able to access the admin endpoint, but could this still be seen as a data exposure risk? The data is still uploaded to AGOL instead of just the subset (as most people would expect) when using a definition query when publishing the service.