Hello,
I am trying to convert a FeatureLayer object into a SpatialDataFrame object but am running into a strange error:
NameError: name 'OLD_PANDAS' is not defined
Any ideas what is wrong or how I can fix this?
I have imported the requisite packages I believe;
Many thanks
Solved! Go to Solution.
Given that the ArcGIS API for Python is still fairly new, there are some big changes that can occur between releases, so it is important to say specifically which version you are working with.
Looking at Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers:
New at version 1.5, the Spatially Enabled DataFrame is an evolution of the
SpatialDataFrame
object that you may be familiar with. While theSDF
object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.
I would focus your effort on learning the new data frame rather than troubleshooting the older spatial data frame.
Given that the ArcGIS API for Python is still fairly new, there are some big changes that can occur between releases, so it is important to say specifically which version you are working with.
Looking at Introduction to the Spatially Enabled DataFrame | ArcGIS for Developers:
New at version 1.5, the Spatially Enabled DataFrame is an evolution of the
SpatialDataFrame
object that you may be familiar with. While theSDF
object is still avialable for use, the team has stopped active development of it and is promoting the use of this new Spatially Enabled DataFrame pattern. The SEDF provides you better memory management, ability to handle larger datasets and is the pattern that Pandas advocates as the path forward.
I would focus your effort on learning the new data frame rather than troubleshooting the older spatial data frame.
Thank you very much for the response. I hadn't realized the change.
I am using the arcgis v 1.5.2
pandas v 0.24.0
I am not familiar with the spatially enabled data frame object but after following the guide I am getting the following error:
I tried it with several different feature layers including a very simple one with no dates and it always gives me the same error. Am I missing something?
Here is the full error message in case it helps:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
1966 from arcgis.features.geo._io.serviceops import from_layer
-> 1967 return from_layer(layer=layer)
1968 except ImportError:
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_io\serviceops.py in from_layer(layer, query)
122 else:
--> 123 return layer.query(where=query, as_df=True)#.sdf
124 return res
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in query(self, where, out_fields, time_filter, geometry_filter, return_geometry, return_count_only, return_ids_only, return_distinct_values, return_extent_only, group_by_fields_for_statistics, statistic_filter, result_offset, result_record_count, object_ids, distance, units, max_allowable_offset, out_sr, geometry_precision, gdb_version, order_by_fields, out_statistics, return_z, return_m, multipatch_option, quantization_parameters, return_centroid, return_all_records, result_type, historic_moment, sql_format, return_true_curves, return_exceeded_limit_features, as_df, **kwargs)
709 import pandas as pd
--> 710 df = self._query_df(url, params)
711 dt_fields = [fld['name'] for fld in self.properties.fields \
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in _query_df(self, url, params)
1452 df.loc[q, fld['name']] = 0
-> 1453 df = df.astype(dtypes, False)
1454 if 'geometryType' in featureset_dict:
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
5667 if col_name in dtype:
-> 5668 results.append(col.astype(dtype[col_name], copy=copy))
5669 else:
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\generic.py in astype(self, dtype, copy, errors, **kwargs)
5680 new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors,
-> 5681 **kwargs)
5682 return self._constructor(new_data).__finalize__(self)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in astype(self, dtype, **kwargs)
530 def astype(self, dtype, **kwargs):
--> 531 return self.apply('astype', dtype=dtype, **kwargs)
532
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\managers.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
394
--> 395 applied = getattr(b, f)(**kwargs)
396 result_blocks = _extend_blocks(applied, result_blocks)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in astype(self, dtype, copy, errors, values, **kwargs)
533 return self._astype(dtype, copy=copy, errors=errors, values=values,
--> 534 **kwargs)
535
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\internals\blocks.py in _astype(self, dtype, copy, errors, values, **kwargs)
594 # convert dtypes if needed
--> 595 dtype = pandas_dtype(dtype)
596 # astype processing
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\pandas\core\dtypes\common.py in pandas_dtype(dtype)
2028 elif npdtype.kind == 'O':
-> 2029 raise TypeError("dtype '{}' not understood".format(dtype))
2030
TypeError: dtype '<class 'datetime.datetime'>' not understood
During handling of the above exception, another exception occurred:
Exception Traceback (most recent call last)
<ipython-input-5-8e6be27d7403> in <module>
----> 1 spd = pd.DataFrame.spatial.from_layer(feature_layer)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo\_accessor.py in from_layer(layer)
1971 raise Exception("Malformed response from server, could not load the dataset: %s" % str(je))
1972 except Exception as e:
-> 1973 raise Exception("Could not load the dataset: %s" % str(e))
1974 #----------------------------------------------------------------------
1975 @staticmethod
Exception: Could not load the dataset: dtype '<class 'datetime.datetime'>' not understood
/blogs/dan_patterson/2016/08/14/script-formatting
and you can't go back and edit it once you have pasted... you have to edit when you are there the first time
Thanks again Dan!
That was immensely helpful.
I ran into this issue, and while you did not have the correct answer - I literally said to my self "If I search Dan Patterson I might find good forum to check this".
Bingo...
I am going to move this error to a different thread as it is not exactly related. Thank you again for the input.
Exactly the same error message yesterday when I added GeoPandas to Python using Conda over the ArcPro 2.3 version. I then upgraded to ArcPro2.4 which removed GeoPandas and now the Python Script needed to be upgraded to the latest version and the script is running. It seems to me that GeoPandas and its dependencies conflict with ArcPy and ESRI's other installed dependencies.
But we maynot need GeoPandas implementation to manipulate teh AGOL features, so fo rthe time being I will forgo that install.
Did you install Geopandas in a separate environment? I have had success with Geopandas and Arcpy coexisting but I do so in separate environments. I can share a requirements file if it would help.
Hello, David:
No, I did not. I wanted to run both in the same space. That how the previous libraries were replaced with new ones that ArcGIS was not compatible with. I find what most of what GeoPandas would do is already implemented in AGOL so I will just use that for AGOL purposes.
Thank you for the input.