Max Number of Records When Exporting Pandas Dataframe to Geodatabase Table?

Joshua_Flickinger

I'm using the arcgis .to_table() method to convert a pandas dataframe to a GIS table.  However, I'm losing records from my table in the process.  My dataframe has 51,970 records.  My table in GIS only has 45,314 records.  If I export the exact same dataframe to csv, the records are there - so I can use the csv if necessary to construct my table in ArcGIS.  But, I'm curious to find out what's going on.  I already ran a function to convert the object data types to strings, and I tried resetting the index right before export.  I end up with the same result.  So, I'm wondering if there's a maximum number of records that the .to_table() method can handle?  If not, I must be missing something in the data that I can investigate more fully.

 

 

danretired

to_table ultimately is a call to

C:\... your install folder ... \bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\features\geo\_io\fileops.py

lines 335-366

It in essence uses arcpy.da.ExtendTable and an arcpy.da.InsertCursor  

during the insertcursor operation, any row that fails to be converted to a proper list will skip that row (lines 451-456)

the to_csv method doesnt do that checking, so perhaps you have nulls/blanks etc in rows that the to_table doesn't explicitly handle.

In short,

• there is no imposed limit on the number of rows by to_table
• use to_csv since, for whatever reason, it doesn't really care what is in the rows.