topic Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer in ArcGIS API for Python Questions

NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

BenjaminSperry1 — Mon, 11 Feb 2019 18:58:45 GMT

Hello,

I am trying to convert a FeatureLayer object into a SpatialDataFrame object but am running into a strange error:

NameError: name 'OLD_PANDAS' is not defined

Any ideas what is wrong or how I can fix this?

I have imported the requisite packages I believe;

Many thanks

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

JoshuaBixby — Mon, 11 Feb 2019 19:42:18 GMT

Given that the ArcGIS API for Python is still fairly new, there are some big changes that can occur between releases, so it is important to say specifically which version you are working with.

Looking at Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers:

New at version 1.5, the Spatially Enabled DataFrame is an evolution of the SpatialDataFrame object that you may be familiar with. While the SDF object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.

I would focus your effort on learning the new data frame rather than troubleshooting the older spatial data frame.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

DanPatterson_Retired — Tue, 12 Feb 2019 02:00:23 GMT

/blogs/dan_patterson/2016/08/14/script-formatting

and you can't go back and edit it once you have pasted... you have to edit when you are there the first time

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

BenjaminSperry1 — Tue, 12 Feb 2019 13:15:26 GMT

Thanks again Dan!

That was immensely helpful.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

BenjaminSperry1 — Tue, 12 Feb 2019 13:16:14 GMT

I am going to move this error to a different thread as it is not exactly related. Thank you again for the input.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

Anonymous User — Thu, 28 Feb 2019 03:26:33 GMT

Try downgrading to pandas version "0.23.4" (I'm not sure on the exact version where it breaks, but this worked for me). There was a change in a pandas module that the ArcGIS API for Python is not accounting for yet.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

DanPatterson_Retired — Thu, 28 Feb 2019 03:33:16 GMT

for ArcGIS Pro 2.3.1 installed today, ArcGIS and pandas are listed below

(arcgispro-py3) C:\arc_pro\bin\Python\envs\arcgispro-py3>conda list
# packages in environment at C:\arc_pro\bin\Python\envs\arcgispro-py3:
#
...
arcgis                    1.5.3            py36h66efe7d_1    esri
arcgispro                 2.3                  h35c310c_0    esri
...
pandas                    0.24.1           py36ha925a31_

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

DavidWasserman — Mon, 11 Mar 2019 22:40:09 GMT

I ran into this issue, and while you did not have the correct answer - I literally said to my self "If I search Dan Patterson I might find good forum to check this".

Bingo...

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

TeodoraEnache — Thu, 21 Mar 2019 11:50:58 GMT

I had the exact same problem with the data frame conversion and downgrading to pandas 0.23.4 worked! Thank you.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

JillKelly — Mon, 29 Apr 2019 16:38:30 GMT

I downgraded to 0.23.4, but it did not solve the problem. pandas.DataFrame.spatial does not seem to have a method .from_layer, at least not according to tab-complete. (using arcgis 1.6)

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

MaazaMekuria — Tue, 02 Jul 2019 21:55:48 GMT

Exactly the same error message yesterday when I added GeoPandas to Python using Conda over the ArcPro 2.3 version. I then upgraded to ArcPro2.4 which removed GeoPandas and now the Python Script needed to be upgraded to the latest version and the script is running. It seems to me that GeoPandas and its dependencies conflict with ArcPy and ESRI's other installed dependencies.

But we maynot need GeoPandas implementation to manipulate teh AGOL features, so fo rthe time being I will forgo that install.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

DavidWasserman — Mon, 15 Jul 2019 14:38:15 GMT

Did you install Geopandas in a separate environment? I have had success with Geopandas and Arcpy coexisting but I do so in separate environments. I can share a requirements file if it would help.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

MaazaMekuria — Tue, 16 Jul 2019 20:38:50 GMT

Hello, David:

No, I did not. I wanted to run both in the same space. That how the previous libraries were replaced with new ones that ArcGIS was not compatible with. I find what most of what GeoPandas would do is already implemented in AGOL so I will just use that for AGOL purposes.

Thank you for the input.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

DavidWasserman — Tue, 06 Aug 2019 00:37:59 GMT

That makes sense. Generally, I would recommend sticking to one or the other. It can be really difficult to get them to play nice.

Re: NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

BenjaminSperry1 — Sun, 12 Dec 2021 10:46:17 GMT

Thank you very much for the response. I hadn't realized the change.

I am using the arcgis v 1.5.2

pandas v 0.24.0

I am not familiar with the spatially enabled data frame object but after following the guide I am getting the following error:

I tried it with several different feature layers including a very simple one with no dates and it always gives me the same error. Am I missing something?

Here is the full error message in case it helps:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
   1966             from arcgis.features.geo._io.serviceops import from_layer
-> 1967             return from_layer(layer=layer)
   1968         except ImportError:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_io\serviceops.py in from_layer(layer, query)
    122     else:
--> 123         return layer.query(where=query, as_df=True)#.sdf
    124     return res

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in query(self, where, out_fields, time_filter, geometry_filter, return_geometry, return_count_only, return_ids_only, return_distinct_values, return_extent_only, group_by_fields_for_statistics, statistic_filter, result_offset, result_record_count, object_ids, distance, units, max_allowable_offset, out_sr, geometry_precision, gdb_version, order_by_fields, out_statistics, return_z, return_m, multipatch_option, quantization_parameters, return_centroid, return_all_records, result_type, historic_moment, sql_format, return_true_curves, return_exceeded_limit_features, as_df, **kwargs)
    709                 import pandas as pd
--> 710                 df = self._query_df(url, params)
    711                 dt_fields = [fld['name'] for fld in self.properties.fields \

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in _query_df(self, url, params)
   1452                     df.loc[q, fld['name']] = 0
-> 1453         df = df.astype(dtypes, False)
   1454         if 'geometryType' in featureset_dict:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
   5667                 if col_name in dtype:
-> 5668                     results.append(col.astype(dtype[col_name], copy=copy))
   5669                 else:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
   5680             new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors,
-> 5681                                          **kwargs)
   5682             return self._constructor(new_data).__finalize__(self)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in astype(self, dtype, **kwargs)
    530     def astype(self, dtype, **kwargs):
--> 531         return self.apply('astype', dtype=dtype, **kwargs)
    532 

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
    394 
--> 395             applied = getattr(b, f)(**kwargs)
    396             result_blocks = _extend_blocks(applied, result_blocks)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in astype(self, dtype, copy, errors, values, **kwargs)
    533         return self._astype(dtype, copy=copy, errors=errors, values=values,
--> 534                             **kwargs)
    535 

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in _astype(self, dtype, copy, errors, values, **kwargs)
    594         # convert dtypes if needed
--> 595         dtype = pandas_dtype(dtype)
    596         # astype processing

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\dtypes\common.py in pandas_dtype(dtype)
   2028     elif npdtype.kind == 'O':
-> 2029         raise TypeError("dtype '{}' not understood".format(dtype))
   2030 

TypeError: dtype '<class 'datetime.datetime'>' not understood

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-5-8e6be27d7403> in <module>
----> 1 spd = pd.DataFrame.spatial.from_layer(feature_layer)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
   1971             raise Exception("Malformed response from server, could not load the dataset: %s" % str(je))
   1972         except Exception as e:
-> 1973             raise Exception("Could not load the dataset: %s" % str(e))
   1974     #----------------------------------------------------------------------
   1975     @staticmethod

Exception: Could not load the dataset: dtype '<class 'datetime.datetime'>' not understood‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍