NameError: OLD_PANDAS Spatial Data Fram from FeatureLayer

4273
14
Jump to solution
02-11-2019 10:58 AM
BenjaminSperry1
Occasional Contributor

Hello,

I am trying to convert a FeatureLayer object into a SpatialDataFrame object but am running into a strange error:

NameError: name 'OLD_PANDAS' is not defined

Any ideas what is wrong or how I can fix this?

I have imported the requisite packages I believe;

Many thanks

1 Solution

Accepted Solutions
JoshuaBixby
MVP Esteemed Contributor

Given that the ArcGIS API for Python is still fairly new, there are some big changes that can occur between releases, so it is important to say specifically which version you are working with.

Looking at Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers:

New at version 1.5, the Spatially Enabled DataFrame is an evolution of the SpatialDataFrame object that you may be familiar with. While the SDF object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.

I would focus your effort on learning the new data frame rather than troubleshooting the older spatial data frame.

View solution in original post

14 Replies
JoshuaBixby
MVP Esteemed Contributor

Given that the ArcGIS API for Python is still fairly new, there are some big changes that can occur between releases, so it is important to say specifically which version you are working with.

Looking at Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers:

New at version 1.5, the Spatially Enabled DataFrame is an evolution of the SpatialDataFrame object that you may be familiar with. While the SDF object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.

I would focus your effort on learning the new data frame rather than troubleshooting the older spatial data frame.

BenjaminSperry1
Occasional Contributor

Thank you very much for the response. I hadn't realized the change.

I am using the arcgis v 1.5.2

pandas v 0.24.0

I am not familiar with the spatially enabled data frame object but after following the guide I am getting the following error:

I tried it with several different feature layers including a very simple one with no dates and it always gives me the same error. Am I missing something?

Here is the full error message in case it helps:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
   1966             from arcgis.features.geo._io.serviceops import from_layer
-> 1967             return from_layer(layer=layer)
   1968         except ImportError:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_io\serviceops.py in from_layer(layer, query)
    122     else:
--> 123         return layer.query(where=query, as_df=True)#.sdf
    124     return res

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in query(self, where, out_fields, time_filter, geometry_filter, return_geometry, return_count_only, return_ids_only, return_distinct_values, return_extent_only, group_by_fields_for_statistics, statistic_filter, result_offset, result_record_count, object_ids, distance, units, max_allowable_offset, out_sr, geometry_precision, gdb_version, order_by_fields, out_statistics, return_z, return_m, multipatch_option, quantization_parameters, return_centroid, return_all_records, result_type, historic_moment, sql_format, return_true_curves, return_exceeded_limit_features, as_df, **kwargs)
    709                 import pandas as pd
--> 710                 df = self._query_df(url, params)
    711                 dt_fields = [fld['name'] for fld in self.properties.fields \

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in _query_df(self, url, params)
   1452                     df.loc[q, fld['name']] = 0
-> 1453         df = df.astype(dtypes, False)
   1454         if 'geometryType' in featureset_dict:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
   5667                 if col_name in dtype:
-> 5668                     results.append(col.astype(dtype[col_name], copy=copy))
   5669                 else:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
   5680             new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors,
-> 5681                                          **kwargs)
   5682             return self._constructor(new_data).__finalize__(self)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in astype(self, dtype, **kwargs)
    530     def astype(self, dtype, **kwargs):
--> 531         return self.apply('astype', dtype=dtype, **kwargs)
    532 

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
    394 
--> 395             applied = getattr(b, f)(**kwargs)
    396             result_blocks = _extend_blocks(applied, result_blocks)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in astype(self, dtype, copy, errors, values, **kwargs)
    533         return self._astype(dtype, copy=copy, errors=errors, values=values,
--> 534                             **kwargs)
    535 

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in _astype(self, dtype, copy, errors, values, **kwargs)
    594         # convert dtypes if needed
--> 595         dtype = pandas_dtype(dtype)
    596         # astype processing

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\dtypes\common.py in pandas_dtype(dtype)
   2028     elif npdtype.kind == 'O':
-> 2029         raise TypeError("dtype '{}' not understood".format(dtype))
   2030 

TypeError: dtype '<class 'datetime.datetime'>' not understood

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-5-8e6be27d7403> in <module>
----> 1 spd = pd.DataFrame.spatial.from_layer(feature_layer)

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
   1971             raise Exception("Malformed response from server, could not load the dataset: %s" % str(je))
   1972         except Exception as e:
-> 1973             raise Exception("Could not load the dataset: %s" % str(e))
   1974     #----------------------------------------------------------------------
   1975     @staticmethod

Exception: Could not load the dataset: dtype '<class 'datetime.datetime'>' not understood
0 Kudos
DanPatterson_Retired
MVP Emeritus

/blogs/dan_patterson/2016/08/14/script-formatting 

and you can't go back and edit it once you have pasted... you have to edit when you are there the first time

BenjaminSperry1
Occasional Contributor

Thanks again Dan!

That was immensely helpful.

0 Kudos
DavidWasserman
Occasional Contributor III

I ran into this issue, and while you did not have the correct answer - I literally said to my self "If I search Dan Patterson I might find good forum to check this". 

Bingo...

David Wasserman, AICP
0 Kudos
BenjaminSperry1
Occasional Contributor

I am going to move this error to a different thread as it is not exactly related.  Thank you again for the input.

0 Kudos
MaazaMekuria
Occasional Contributor

Exactly the same error message yesterday when I added GeoPandas to Python using Conda over the ArcPro 2.3 version.  I then upgraded to ArcPro2.4 which removed GeoPandas and now the Python Script needed to be upgraded to the latest version and the script is running.  It seems to me that GeoPandas and its dependencies conflict with ArcPy and ESRI's other installed dependencies.  

But we maynot need GeoPandas implementation to manipulate teh AGOL features, so fo rthe time being I will forgo that install.

0 Kudos
DavidWasserman
Occasional Contributor III

Did you install Geopandas in a separate environment? I have had success with Geopandas and Arcpy coexisting but I do so in separate environments. I can share a requirements file if it would help. 

David Wasserman, AICP
0 Kudos
MaazaMekuria
Occasional Contributor

Hello, David:

No, I did not.  I wanted to run both in the same space.  That how the previous libraries were replaced with new ones that ArcGIS was not compatible with. I find what most of what GeoPandas would do is already implemented in AGOL so I will just use that for AGOL purposes. 

Thank you for the input. 

0 Kudos