pd.DataFrame.spatial.from_table() is loading column labels but not data?

3942
10
Jump to solution
11-10-2020 07:19 AM
SamuelStroebel
New Contributor

as the title says, trying to access a local file in a GDB, a dataframe is created with the correct number of columns but no data? 

Code:

import pandas as pd
from arcgis.features import GeoAccessor, GeoSeriesAccessor
from arcgis import GIS
import arcpy

df = pd.DataFrame.spatial.from_table(r'some_file_path_to_a_table_in_a_gdb')

df.shape

output: 0 x number of cols in input table

Tags (1)
0 Kudos
1 Solution

Accepted Solutions
JoshuaBixby
MVP Esteemed Contributor

I see what you are running into, clearly a defect.  To work around it, specify the fields you want in a field list, and it will work.  It appears the wildcard for all fields isnt' working.

View solution in original post

10 Replies
JoshuaBixby
MVP Esteemed Contributor

It always helps to show your code.

0 Kudos
SamuelStroebel
New Contributor

thanks, updated

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

What version of the ArcGIS API for Python are you using?

0 Kudos
SamuelStroebel
New Contributor

1.8.2, in a cloned environment 

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

I see what you are running into, clearly a defect.  To work around it, specify the fields you want in a field list, and it will work.  It appears the wildcard for all fields isnt' working.

SamuelStroebel
New Contributor

Your work around worked perfectly, thanks!

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

Please mark my response as Correct to close out the thread.

Also, I opened in issue on GitHub:  GeoAccessor.from_table Returns Empty Table When Using All Fields Wildcard · Issue #829 · Esri/arcgis... 

Hudi
by
New Contributor

I'm not entirely sure the solution response is the entire picture. It appears to me to be a failure of spatial.from_table to deal with null values.

The default behavior looks like it will cut down the rows it actually reads into the dataframe such that it can have at least one complete row of non-null data that includes values from all specified fields. So as you add more fields to the list the chances of an entire row containing non-null values diminishes.

Therefore specifying fields probably means also excluding many fields you don't care about. That explains why specifying fields works. If you used arcpy.ListFields() and built a list of all field names it should behave the same as default where no fields are entered, which is the same as fields='*'.

So to handle this behavior you can add something like null_value='-9999' and then in the dataframe switch that value back to NaN if that is desired.

 

gallgher55
New Contributor II

I've confirmed this is the actual issue. I supplied my fields via a list from arcpy.ListFields() to no avail. Using the null_value is what actually made it work for me.

0 Kudos