i'm trying to export a feature layer with several fields of different datatypes but 5 of the fields are datetime data types but when i use the below i get this error
nparr = arcpy.da.FeatureClassToNumPyArray('mob', fields,null_value=-9)
numpy.savetxt('H:\mobcsv2.csv', nparr, delimiter=",", fmt="%s")
how do i get around this?
Stu
It seems like the feature class contains fields with geometry or datetime data types which connot be converted to a numpy array.
So you can try to exclude them. Untested
import arcpy
import numpy as np
# Set workspace
arcpy.env.workspace = 'Geodatabase_Path' # Replace with your actual geodatabase path
# Define feature class and fields
feature_class = 'mob'
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]
# Check excluded fields
print(f"All fields: {all_fields}")
print(f"Selected fields: {fields}")
# Create NumPy array excluding geometry and datetime fields
try:
nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9)
np.savetxt('H:\\mobcsv2.csv', nparr, delimiter=",", fmt="%s")
print("Array saved successfully.")
except Exception as e:
print(f"Error: {e}")
If you need them you can try something like this. Untested
import arcpy
import numpy as np
# Set workspace
arcpy.env.workspace = 'Geodatabase_Path' # Replace with your actual geodatabase path
# Define feature class
feature_class = 'mob'
# List all fields
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]
# Include geometry field to extract coordinates
fields_with_geometry = fields + ['SHAPE@XY']
# Include datetime fields separately for conversion
datetime_fields = [f.name for f in arcpy.ListFields(feature_class) if f.type == 'Date']
# Function to convert datetime fields to string
def convert_datetime_to_string(array, datetime_fields):
for field in datetime_fields:
array[field] = array[field].astype(str)
return array
try:
# Create NumPy array including geometry and datetime fields
nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields_with_geometry + datetime_fields, null_value=-9)
# Convert datetime fields to strings
nparr = convert_datetime_to_string(nparr, datetime_fields)
# Add geometry as separate columns
nparr_with_geometry = np.array(
[(item.tolist() + list(coord)) for item, coord in zip(nparr, nparr['SHAPE@XY'])],
dtype=[(name, nparr.dtype[name]) for name in nparr.dtype.names if name != 'SHAPE@XY'] + [('X', float), ('Y', float)]
)
# Save to CSV
np.savetxt('H:\\mobcsv2_combined.csv', nparr_with_geometry, delimiter=",", fmt="%s")
print("Array with geometry and dates saved successfully.")
except Exception as e:
print(f"Error: {e}")
thanks @TonyAlmeida really appreciate the time, i tried it but get the same error about the datetime, i did remove the geometry and got the same also
It's hard to determine the issue, if we don' t see the entire code. Also what print out when the code got to lines 13 & 14 of the first code? I am guessing it's not finding the Date field, is the name of the actual field called "Date"?
thanks @TonyAlmeida the first code does print all the field names and the next only the fields that are not geomentry or date but when it tried to export i get this error:
when trying to run the second piece of code in steps i get this error with the code, but the print(datetime_fields) does return the 5 date fields
nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields_with_geometry + datetime_fields, null_value=-9)
The array memory error typically occurs when the script tries to create an array that's too large for the available memory, how big is the data?
Try this,
import arcpy
import numpy as np
# Set workspace
arcpy.env.workspace = 'Geodatabase_Path' # Replace with your actual geodatabase path
# Define feature class
feature_class = 'mob'
# List all fields
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]
# Include geometry field to extract coordinates
geometry_field = 'SHAPE@XY'
# Include datetime fields separately for conversion
datetime_fields = [f.name for f in arcpy.ListFields(feature_class) if f.type == 'Date']
# Function to convert datetime fields to string
def convert_datetime_to_string(datetime_values):
return [val.strftime('%Y-%m-%d %H:%M:%S') if val else '' for val in datetime_values]
# Extract non-geometry and non-datetime fields
try:
# Create NumPy array for non-geometry, non-datetime fields
nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9)
# Convert datetime fields to strings using a cursor
datetime_data = []
with arcpy.da.SearchCursor(feature_class, datetime_fields) as cursor:
for row in cursor:
datetime_data.append(row)
datetime_data = np.array(convert_datetime_to_string([row[0] for row in datetime_data]), dtype=str)
# Extract geometry
geometry_data = []
with arcpy.da.SearchCursor(feature_class, [geometry_field]) as cursor:
for row in cursor:
geometry_data.append(row[0])
# Add geometry as separate columns
nparr_with_geometry = np.array(
[(item.tolist() + list(coord)) for item, coord in zip(nparr, geometry_data)],
dtype=[(name, nparr.dtype[name]) for name in nparr.dtype.names] + [('X', float), ('Y', float)]
)
# Combine datetime data
datetime_array = np.array(datetime_data.tolist(), dtype=str)
# Create final array including datetime fields
final_dtype = [(name, nparr_with_geometry.dtype[name]) for name in nparr_with_geometry.dtype.names] + \
[(f'DATE_{i}', str) for i in range(len(datetime_fields))]
final_array = np.array(
[(list(nparr_with_geometry[i]) + list(datetime_array[i])) for i in range(len(nparr_with_geometry))],
dtype=final_dtype
)
# Save to CSV
np.savetxt('H:\\mobcsv2_combined.csv', final_array, delimiter=",", fmt="%s", header=",".join(final_array.dtype.names), comments='')
print("Array with geometry and dates saved successfully.")
except Exception as e:
print(f"Error: {e}")
thanks @TonyAlmeida i got the same error at the same point unfortunately, in terms of data there are around 350000 records and 95 fields.
i think i'm going to use a search cursor and just write it out as a csv file
thanks for the help
Stu