AGOL hosted feature layer mysteriously changed to hosted table

456
8
04-27-2021 04:29 PM
MichaelRobinson2
New Contributor II

Has anyone experienced an ArcGIS Online hosted feature layer mysteriously changing to a hosted table? This has happened to me twice now (first time on April 2, 2021). I called Esri Tech Support the first time this happened and they didn't have a record of it happening before. I ended up deleting the hosted table and related Excel file and uploading a new version, but had to rebuild pop ups, etc. in my map. I plan to do the same again, but wondering if anyone else has experienced this and/or has any ideas why this might happen?

There are around 2,900 records in the table and I manually overwrite the hosted feature layer in AGOL every 1-3 days. The first hosted feature layer was stable for over a year before it popped. I built this new layer (with a new Excel file) on April 2, 2021 and it was fine until today.

0 Kudos
8 Replies
jcarlson
MVP Notable Contributor

I've never experienced this myself. What does the Excel file look like, and what is the process you use to overwrite the layer? I can imagine scenarios where the feature layer to table outcome actually makes sense.

Is there a reason you can't append to the layer rather than overwrite the whole thing?

- Josh Carlson
Kendall County GIS
0 Kudos
MichaelRobinson2
New Contributor II

Hi Josh,

Thanks for the reply. Very standard Excel (XLSX) file -- currently ~2,900 rows (grows by ~5 rows every day or two) and 20 columns of text and numbers, no gaps or spaces (except some blank cells [no data] here and there), no summations or other formulas at the ends of rows or columns, short field names. 

The process I use to overwrite is to Update Data → Overwrite Entire Layer from the hosted feature layer's Overview page in AGOL. I overwrite rather than append the layer because it is possible for data throughout the table to change occasionally.

I deleted the XLSX and hosted table in AGOL yesterday, created a CSV from the XLSX on my local computer, and uploaded the CSV to create a new hosted feature layer. The only thing I could think of is that the problem may have had something to do with overwriting an XLSX?! I've had this happen twice now and it was with an XLSX file both times -- never had this happen when the hosted feature layer was created from, and updated with, a CSV file.

I'd be really curious to hear some scenarios where the feature layer to table outcome makes sense.

Thanks again!

-Michael

0 Kudos
jcarlson
MVP Notable Contributor

That makes sense with the updates.

So where does the shape data come from in the excel file? Is it just lat/lon columns, or a location string?

Also, I don't use that same process w/ tabular data like Excel much, so I am wondering: does it prompt you to define the location fields each time?

I should say, "makes sense" doesn't necessarily mean that it's the intended outcome. I've run into lots of issues with the content on an Excel file being incorrectly inferred (string as integer, and vice versa) upon being published, where it changes a field type upon overwrite.

I could see a situation in which the contents of the spatial column in the uploaded file have changed, AGOL doesn't properly infer its spatial type, defaults it to some other column. Then, lacking a spatial component, keeps the resulting service as a table. Like I said, "makes sense" ≠ "the right result". Maybe "understandable" is a better word.

Since you're overwriting the service entirely, it's possible for basically everything about the layer to change. I've done overwrites going from points to polygons and back again, massively changing the schema each time.

- Josh Carlson
Kendall County GIS
MichaelRobinson2
New Contributor II

Interesting. Yes, I'm using columns of lat/long coords to locate features. I can see what you mean re: AGOL incorrectly inferring a field's data type, changing a decimal/double to, for example, a string, not being able to locate features, and then converting the feature layer to a table. Weird, but I can envision that occurring.

I've been experimenting with a variety of Python scripts to overwrite and append layers in AGOL since it is something we do very frequently. I seem to recall somewhere that we can hard code data types for the fields we import -- I'll look into adjusting the code to enforce importing the lat/long coord fields as decimals/double and maybe that will prevent this issue? Also, the CSV file type may prevent the "incorrect field type inference" issue since CSVs are field type agnostic. Either of these should work as long as there are no accidental text characters in the fields?

Thanks for the input -- really appreciated!

-Michael

0 Kudos
jcarlson
MVP Notable Contributor

No problem!

If you're looking at going the Python route, you might consider using a Spatially Enabled DataFrame. If you read the file into a pandas dataframe first, you can use GeoAccessor.from_xy to convert that to a SEDF.

A benefit of a SEDF is that you can directly convert that to a FeatureSet for appending / updating existing services. It also gives you more control over the data types prior to publishing / editing.

We actually use them extensively for comparing a published service to another data source and syncing changes between the two.

df = pandas.read_csv('your-file.csv')

sdf = arcgis.features.GeoAccessor.from_xy(df, 'lon-column', 'lat-column', 'spatial-reference')

agol_df = gis.content.get('item-id').layers[0].query(as_df=True)

 

- Josh Carlson
Kendall County GIS
MichaelRobinson2
New Contributor II

Very cool. I'll look into this. Thanks so much, Josh!

-Michael

0 Kudos
MichaelRobinson2
New Contributor II

Based on the ".layers[0]" in the code, I'm assuming the 'item-id' refers to the hosted feature layer and not the csv file that is getting overwritten/appended?

0 Kudos
jcarlson
MVP Notable Contributor

That's right. Doing it this way would actually replace the "overwrite layer" process entirely.

Another user was doing something similar, and I posted about my process there. You can see it here: https://community.esri.com/t5/arcgis-online-questions/what-would-happen-to-a-layer-view-if-its-sourc...

 

- Josh Carlson
Kendall County GIS