topic Re: SpatialDataFrame has been deprecated in ArcGIS API for Python Questions

SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 12:32:21 GMT

I receive following warning after this statement is run: sdf = features.SpatialDataframe(df)

opt/anaconda3/lib/python3.7/site-packages/arcgis/features/_data/geodataset/geodataframe.py:221: UserWarning: SpatialDataFrame has been deprecated.  Please switch to the GeoAccessor/GeoSeriesAccessor.   warnings.warn("SpatialDataFrame has been deprecated.  Please switch to the GeoAccessor/GeoSeriesAccessor.")

What is the correct way to create an 'sdf' now?

Re: SpatialDataFrame has been deprecated

Egge-Jan_Pollé — Tue, 31 Mar 2020 12:48:56 GMT

Hi art zaifman,

The answer to your question might be found here: Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers

New at version 1.5, the Spatially Enabled DataFrame is an evolution of the SpatialDataFrame object that you may be familiar with. While the SDF object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.

HTH,

Egge-Jan

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 15:06:45 GMT

It does help and thank you for the rapid reply, Egge-Jan!

Short follow-up question: is there a description of the method in which GIS().content.import_data(panda data frame containing lat, lon columns) populates the SHAPE column? Main question is why are the 'x, y' values in the SHAPE column so different from the original 'latitude, longitude' values

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 15:26:47 GMT

Are these UTM coordinates?

Thanks.

Re: SpatialDataFrame has been deprecated

Egge-Jan_Pollé — Tue, 31 Mar 2020 15:30:25 GMT

Hi art zaifman,

Always happy to help 🙂 If you consider your initial question being answered you might consider to 'close the call' by marking my answer as correct.

Regarding your follow-up question: I guess this has something to do with the use of different coordinate systems being used. It looks like your input latitude and longitude are in WGS84 (EPSG:4326) whereas the coordinates in the SHAPE column are - most probably - in EPSG:3857. (EPSG:3857 is a Spherical Mercator projection coordinate system popularized by web services such as Google and later OpenStreetMap, and also being used by Esri, with WKID:102100).

I guess the points do appear in the right location when plotted on the map, don't they?

BR,

Egge-Jan

Re: SpatialDataFrame has been deprecated

DanPatterson_Retired — Tue, 31 Mar 2020 16:09:51 GMT

webby stuff is usually in a Web Mercator projection

Re: SpatialDataFrame has been deprecated

Egge-Jan_Pollé — Tue, 31 Mar 2020 16:12:46 GMT

... so that would be this WKID:102100/EPSG:3857 thing, as I mentioned above.

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 16:20:56 GMT

The points are mapped correctly and thank you again!

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 17:06:00 GMT

from pyproj import Proj, transform

print(transform(Proj('epsg:4326'), Proj('epsg:3857'), 45.810, -122.691)) # longitude first, latitude second.

(-13657899.64491753, 5749953.689939638)

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 18:37:58 GMT

BTW, here's an excellent reference for how to code bulk transformations using pyproj: High-performance bulk transformation of coordinates

Re: SpatialDataFrame has been deprecated

artzaifman — Tue, 31 Mar 2020 22:13:54 GMT

Related to the issues above, is it possible to use sdf.spatial.plot(map_widget=...) on an SEDF derived from a regular panda DF where the SEDF was NOT created with import_data() method? The SEDF was instead created as follows: sdf = GeoAccessor.from_xy(df, 'x', 'y')

Re: SpatialDataFrame has been deprecated

artzaifman — Wed, 01 Apr 2020 01:06:44 GMT

Found reason:

SEDF which works =

{'x': -13657899.64491753,  'y': 5749953.6899396395,  'spatialReference': {'wkid': 102100, 'latestWkid': 3857}}

SEDF which doesn't =

{'x': -13657899.64491753,  'y': 5749953.689939638,  'spatialReference': {'wkid': 4326}}

Re: SpatialDataFrame has been deprecated

artzaifman — Wed, 01 Apr 2020 15:31:28 GMT

Continuing to add to this thread since it may help other newbies to arcGIS Pro like like me!

Discovered that sdf.spatial.plot() will fail to render (without generation an error) if it encounters NaN values.

Here's snippet to remove NaNs from SEDF:

sdf = GeoAccessor.from_xy(df, 'x', 'y', sr=102100) # 102100 is necessary to render using sdf.spatial.plot() method

...

sdf[~sdf['SHAPE'].apply(lambda r: numpy.isnan(r['x']) or numpy.isnan(r['y']))] # drop NaNs which cause plot to silently fail