topic Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable in ArcGIS API for Python Questions

Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

Zeke — Wed, 24 Feb 2021 21:26:29 GMT

I upgraded to Spyder 4.2.1 from 3.6, and now when I run my script that creates a spatially enabled dataframe, the dataframe 'sdf' won't open in Variable Explorer. Instead it gives a message stating 'The variable is not picklable'. The console pane shows a long traceback ending in '
_pickle.PicklingError: Can't pickle 'geoprocessing server result object' object: <geoprocessing server result object object at 0x000001F686A82A80>'.

This was working before the upgrade, and I'm thinking after it for a while as well (might be wrong there). running things like sdf.head() works fine. Code below.

Running Python 3.7.9 64-bit | Qt 5.9.6 | PyQt 5.9.2 | Windows 10 | Spyder 4.2.1

import time as t import os import sys import pandas as pd import arcpy from arcpy import env from arcgis.features import GeoAccessor, GeoSeriesAccessor gdb = r"C:\Users\Owner\Desktop\Tract_Aggregation\Tract_aggregation.gdb" fds = 'Detroit_tracts' # fc = 'All_Detroit_Tracts_TL_Data_lt_4200' fc = 'Tract_Agg_1_021621_New' census_tracts = os.path.join(gdb, fds, fc) mem_tracts = arcpy.management.MakeFeatureLayer(census_tracts, r"memory\tracts") env.overwriteOutput = True env.workspace = os.path.join(gdb, fds) cutoff = 4200 used_gids = [] def create_sdf(): """ Returns ------- df : Pandas Dataframe Census tracts. """ print('\tCreating Spatial DataFrame...') drop_flds = ['OBJECTID', 'objectid_1', 'object_id'] try: df = pd.DataFrame.spatial.from_featureclass(mem_tracts) df.dropna(subset=['geoid'], inplace=True) df.drop(drop_flds, axis=1, inplace=True) df['under125pe'] = df['under125pe'] * 1.0 df['under200pe'] = df['under200pe'] * 1.0 df['under125pe'].fillna(0.0) df['under200pe'].fillna(0.0) df.set_index('geoid', inplace=True, drop=False) df.rename_axis('gid', inplace=True) df.sort_values(by='total_hh', axis=0, ascending=False, inplace=True) print('\tSpatial DataFrame created\n') return df except Exception as e: print(sys.exc_value)

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

DanPatterson — Wed, 24 Feb 2021 22:39:58 GMT

What version of arcgis?

There were recent issues with pandas dataframes

Spatially Enabled Data Frame not working with pandas datetime · Issue #874 · Esri/arcgis-python-api (github.com)

and

conda upgrade --no-pin arcgis

this was the solution

Not a spyder issue that I could find on their Issues link

Issues · spyder-ide/spyder (github.com)

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

Zeke — Fri, 26 Feb 2021 17:58:36 GMT

Thanks @DanPatterson. Using ArcGIS Pro 2.7.1, cloned Python environment from ArcGIS default to which I've added Spyder.

Default Spyder install via this method is Spyder 3.6. This runs but does nothing when I try to view the spatial dataframe in Variable Explorer. Upgrading Spyder to 4.2.1 also runs, but gives error message the pickling error message in original post. Using the upgrade arcgis method you posted tells me that it's already installed. This is really baffling me. Thanks!

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

DanPatterson — Fri, 26 Feb 2021 18:47:24 GMT

I suspect you won't be able to since it isn't a true Pandas df. I doubt the spyder crew will have much interest in providing support unless there is a widespread need. Numpy and Pandas work fine

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

Zeke — Sun, 28 Feb 2021 18:03:01 GMT

Thanks @DanPatterson. Maybe so, but prior to a few days ago it would display it. Oh well.

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

DanPatterson — Sun, 28 Feb 2021 18:34:21 GMT

was the arcgis module updated during the process? Lots of Issues on their github site with the transition from 1.8.2 to 1.8.3 to 1.8.4

Re: Can't open spatial dataframe in Spyder 4.2.1 Variable Explorer - not picklable

Zeke — Sun, 28 Feb 2021 22:27:54 GMT

That may be it. I did get the latest (afaik) arcgis module.