topic Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2 in ArcGIS Pro Questions

Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2

KateNewell1 — Mon, 11 Dec 2023 20:33:10 GMT

I have a script that worked perfectly last week...before I upgraded from ArcGIS Pro 2.9.5 to 3.2 (see below script).

The script is something that I recreated using help from this post (thank you @jcarlson). The script is run within the python window in Pro.

When I re-ran this script this week, after updating to Pro 3.2, the script no longer works. I was able to test it on a workstation that was still on 2.9.5, and it works! So that is pretty clear to me that something has changed from the python/panda libraries between 2.9.5 and 3.2.

The error I am getting happens on the last line of the script when I try to write the dataframe to a table. Yes, I know the table does not have any geometry, but this worked in the past and created a fgdb table perfectly. The error I get is: "TypeError: 'field_names' must be string or non empty list or tuple of strings"

I am a self-proclaimed hack at python/pandas, so any help is greatly appreciated to get this up and running again.

@jcarlson

@DanPatterson_Retired

import arcpy arcpy.env.workspace = "N:\ArcShared\Planning\CompPlanUpdate_2025\Data\BellinghamProfile\BhamDataBook_2022.gdb" arcpy.env.overwriteOutput = True # Import modules from arcgis import GIS import pandas as pd import requests import json #------------------------------------------------------------------------------------- #------------------------------------------------------------------------------------- #TOTAL POPULATION, SEX, AGE & RACE data for Bellingham City # Submit request to API response = requests.get('https://api.census.gov/data/2022/acs/acs5/profile?get=NAME,DP05_0001E,DP05_0002E,DP05_0003E,DP05_0005E,DP05_0006E,DP05_0007E,DP05_0008E,DP05_0009E,DP05_0010E,DP05_0011E,DP05_0012E,DP05_0013E,DP05_0014E,DP05_0015E,DP05_0016E,DP05_0017E,DP05_0018E,DP05_0073E,DP05_0079E,DP05_0080E,DP05_0081E,DP05_0082E,DP05_0083E,DP05_0084E,DP05_0085E,DP05_0019E,DP05_0024E&for=place:05280&in=state:53') # Read response as JSON into dataframe # First row is column names, skipped for data import df = pd.DataFrame(response.json()[1:]) # Apply column names from first row cols = response.json()[0] column_dict = {} for c in cols: column_dict.update({cols.index(c):c}) df.rename(columns=column_dict, inplace=True) df = df.astype({"NAME": str, "DP05_0001E": int, "DP05_0002E": int, "DP05_0003E": int, "DP05_0005E": int, "DP05_0006E": int, "DP05_0007E": int, "DP05_0008E": int, "DP05_0009E": int, "DP05_0010E": int, "DP05_0011E": int, "DP05_0012E": int, "DP05_0013E": int, "DP05_0014E": int, "DP05_0015E": int, "DP05_0016E": int, "DP05_0017E": int, "DP05_0018E": float,"DP05_0073E": int,"DP05_0079E": int,"DP05_0080E": int,"DP05_0081E": int,"DP05_0082E": int,"DP05_0083E": int,"DP05_0084E": int,"DP05_0085E": int,"DP05_0019E":int,"DP05_0024E":int}) # Export to layer processed_table = df.spatial.to_table('DP05_Pop_Age_Sex_Race')

Re: Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2

jcarlson — Mon, 11 Dec 2023 20:54:28 GMT

Odd. "field_names" isn't a parameter you get to define in that function...

If you have a script that you need to work consistently, it's a good idea to keep a stable Python env for it. Pro has a built-in environment manager, or honestly you could just install Mamba to manage your envs outside of Pro. If you prefer a full GUI setup, you can check out Anaconda Navigator, too.

It doesn't look like your script actually needs arcpy at all, so you really don't need to run it from Pro. You could invoke the script from the terminal, a Jupyter notebook, or wherever else.

Re: Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2

DanPatterson — Mon, 11 Dec 2023 21:31:50 GMT

you might want to raise an issue on

Esri/arcgis-python-api: Documentation and samples for ArcGIS API for Python (github.com)

since the issue isn't arcpy but the arcgis and pandas modules were updated during the 3.2 migration

Re: Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2

KateNewell1 — Wed, 13 Dec 2023 19:43:40 GMT

Posted this as an issue on the ArcGIS API for Python GitHub site. Recommended change was going from processed_table = df.spatial.to_table('DP05_Pop_Age_Sex_Race') to processed_table = df.spatial.to_table(os.path.join(arcpy.env.workspace, 'DP05_Pop_Age_Sex_Race'))

This resulted in the "to_table" functioning again, but all the field names were "sanitized", so needed to set that parameter to False:

processed_table = df.spatial.to_table(os.path.join(arcpy.env.workspace, 'DP05_Pop_Age_Sex_Race',),overwrite=True, sanitize_columns=False)

arcgis.features module | ArcGIS API for Python