Pandas dataframe to Esri table fails after ArcGIS Pro update to 3.2

[KateNewell1]

I have a script that worked perfectly last week...before I upgraded from ArcGIS Pro 2.9.5 to 3.2 (see below script).

The script is something that I recreated using help from this post  (thank you @jcarlson). The script is run within the python window in Pro.

When I re-ran this script this week, after updating to Pro 3.2, the script no longer works. I was able to test it on a workstation that was still on 2.9.5, and it works! So that is pretty clear to me that something has changed from the python/panda libraries between 2.9.5 and 3.2. 

The error I am getting happens on the last line of the script when I try to write the dataframe to a table. Yes, I know the table does not have any geometry, but this worked in the past and created a fgdb table perfectly. The error I get is: "TypeError: 'field_names' must be string or non empty list or tuple of strings"

I am a self-proclaimed hack at python/pandas, so any help is greatly appreciated to get this up and running again.

@jcarlson 

@DanPatterson_Retired 

import arcpy
arcpy.env.workspace = "N:\ArcShared\Planning\CompPlanUpdate_2025\Data\BellinghamProfile\BhamDataBook_2022.gdb" 
arcpy.env.overwriteOutput = True

# Import modules
from arcgis import GIS
import pandas as pd
import requests
import json

#-------------------------------------------------------------------------------------

#-------------------------------------------------------------------------------------
#TOTAL POPULATION, SEX, AGE & RACE data for Bellingham City

# Submit request to API
response = requests.get('https://api.census.gov/data/2022/acs/acs5/profile?get=NAME,DP05_0001E,DP05_0002E,DP05_0003E,DP05_0005E,DP05_0006E,DP05_0007E,DP05_0008E,DP05_0009E,DP05_0010E,DP05_0011E,DP05_0012E,DP05_0013E,DP05_0014E,DP05_0015E,DP05_0016E,DP05_0017E,DP05_0018E,DP05_0073E,DP05_0079E,DP05_0080E,DP05_0081E,DP05_0082E,DP05_0083E,DP05_0084E,DP05_0085E,DP05_0019E,DP05_0024E&for=place:05280&in=state:53')


# Read response as JSON into dataframe
# First row is column names, skipped for data import
df = pd.DataFrame(response.json()[1:])

# Apply column names from first row
cols = response.json()[0]
column_dict = {}
for c in cols:
    column_dict.update({cols.index(c):c})
    
df.rename(columns=column_dict, inplace=True)
df = df.astype({"NAME": str, "DP05_0001E": int, "DP05_0002E": int, "DP05_0003E": int, "DP05_0005E": int, "DP05_0006E": int, "DP05_0007E": int, "DP05_0008E": int, "DP05_0009E": int, "DP05_0010E": int,	"DP05_0011E": int, "DP05_0012E": int, "DP05_0013E": int, "DP05_0014E": int, "DP05_0015E": int, "DP05_0016E": int, "DP05_0017E": int, "DP05_0018E": float,"DP05_0073E": int,"DP05_0079E": int,"DP05_0080E": int,"DP05_0081E": int,"DP05_0082E": int,"DP05_0083E": int,"DP05_0084E": int,"DP05_0085E": int,"DP05_0019E":int,"DP05_0024E":int})

        
# Export to layer
processed_table = df.spatial.to_table('DP05_Pop_Age_Sex_Race')

[jcarlson]

Odd. "field_names" isn't a parameter you get to define in that function...

If you have a script that you need to work consistently, it's a good idea to keep a stable Python env for it. Pro has a built-in environment manager, or honestly you could just install Mamba to manage your envs outside of Pro. If you prefer a full GUI setup, you can check out Anaconda Navigator, too.

It doesn't look like your script actually needs arcpy at all, so you really don't need to run it from Pro. You could invoke the script from the terminal, a Jupyter notebook, or wherever else.

[DanPatterson]

you might want to raise an issue on

Esri/arcgis-python-api: Documentation and samples for ArcGIS API for Python (github.com)

since the issue isn't arcpy but the arcgis and pandas modules were updated during the 3.2 migration

[KateNewell1 · Accepted solution]

Posted this as an issue on the ArcGIS API for Python GitHub site. Recommended change was going from processed_table = df.spatial.to_table('DP05_Pop_Age_Sex_Race') to processed_table = df.spatial.to_table(os.path.join(arcpy.env.workspace, 'DP05_Pop_Age_Sex_Race'))

This resulted in the "to_table" functioning again, but all the field names were "sanitized", so needed to set that parameter to False:

processed_table = df.spatial.to_table(os.path.join(arcpy.env.workspace, 'DP05_Pop_Age_Sex_Race',),overwrite=True, sanitize_columns=False)

arcgis.features module | ArcGIS API for Python