Select to view content in your preferred language

export feature to csv in arcmap

382
6
08-01-2024 07:05 AM
StuartMoore
Frequent Contributor

i'm trying to export a feature layer with several fields of different datatypes but 5 of the fields are datetime data types but when i use the below i get this error

nparr = arcpy.da.FeatureClassToNumPyArray('mob', fields,null_value=-9)
numpy.savetxt('H:\mobcsv2.csv', nparr, delimiter=",", fmt="%s")

StuartMoore_0-1722521045610.png

how do i get around this?

Stu

 

0 Kudos
6 Replies
TonyAlmeida
Frequent Contributor

It seems like the feature class contains fields with geometry or datetime data types which connot be converted to a numpy array.

So you can try to exclude them. Untested

 

import arcpy
import numpy as np

# Set workspace
arcpy.env.workspace = 'Geodatabase_Path'  # Replace with your actual geodatabase path

# Define feature class and fields
feature_class = 'mob'
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]

# Check excluded fields
print(f"All fields: {all_fields}")
print(f"Selected fields: {fields}")

# Create NumPy array excluding geometry and datetime fields
try:
    nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9)
    np.savetxt('H:\\mobcsv2.csv', nparr, delimiter=",", fmt="%s")
    print("Array saved successfully.")
except Exception as e:
    print(f"Error: {e}")

 

 

 If you need them you can try something like this. Untested

 

import arcpy
import numpy as np

# Set workspace
arcpy.env.workspace = 'Geodatabase_Path'  # Replace with your actual geodatabase path

# Define feature class
feature_class = 'mob'

# List all fields
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]

# Include geometry field to extract coordinates
fields_with_geometry = fields + ['SHAPE@XY']

# Include datetime fields separately for conversion
datetime_fields = [f.name for f in arcpy.ListFields(feature_class) if f.type == 'Date']

# Function to convert datetime fields to string
def convert_datetime_to_string(array, datetime_fields):
    for field in datetime_fields:
        array[field] = array[field].astype(str)
    return array

try:
    # Create NumPy array including geometry and datetime fields
    nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields_with_geometry + datetime_fields, null_value=-9)
    
    # Convert datetime fields to strings
    nparr = convert_datetime_to_string(nparr, datetime_fields)
    
    # Add geometry as separate columns
    nparr_with_geometry = np.array(
        [(item.tolist() + list(coord)) for item, coord in zip(nparr, nparr['SHAPE@XY'])],
        dtype=[(name, nparr.dtype[name]) for name in nparr.dtype.names if name != 'SHAPE@XY'] + [('X', float), ('Y', float)]
    )
    
    # Save to CSV
    np.savetxt('H:\\mobcsv2_combined.csv', nparr_with_geometry, delimiter=",", fmt="%s")
    print("Array with geometry and dates saved successfully.")
except Exception as e:
    print(f"Error: {e}")

 

 

StuartMoore
Frequent Contributor

thanks @TonyAlmeida really appreciate the time, i tried it but get the same error about the datetime, i did remove the geometry and got the same also

StuartMoore_0-1722526648367.png

 

0 Kudos
TonyAlmeida
Frequent Contributor

It's hard to determine the issue, if we don' t see the entire code. Also what print out when the code got to lines 13 & 14 of the first code? I am guessing it's not finding the Date field, is the name of the actual field called "Date"?

0 Kudos
StuartMoore
Frequent Contributor

thanks @TonyAlmeida the first code does print all the field names and the next only the fields that are not geomentry or date but when it tried to export i get this error:

StuartMoore_0-1722603302532.png

when trying to run the second piece of code in steps i get this error with the code, but the print(datetime_fields) does return the 5 date fields

nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields_with_geometry + datetime_fields, null_value=-9)



StuartMoore_1-1722603433502.png

 

0 Kudos
TonyAlmeida
Frequent Contributor

The array memory error typically occurs when the script tries to create an array that's too large for the available memory, how big is the data?

 

Try this,

 

import arcpy
import numpy as np

# Set workspace
arcpy.env.workspace = 'Geodatabase_Path'  # Replace with your actual geodatabase path

# Define feature class
feature_class = 'mob'

# List all fields
all_fields = [f.name for f in arcpy.ListFields(feature_class)]
fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']]

# Include geometry field to extract coordinates
geometry_field = 'SHAPE@XY'

# Include datetime fields separately for conversion
datetime_fields = [f.name for f in arcpy.ListFields(feature_class) if f.type == 'Date']

# Function to convert datetime fields to string
def convert_datetime_to_string(datetime_values):
    return [val.strftime('%Y-%m-%d %H:%M:%S') if val else '' for val in datetime_values]

# Extract non-geometry and non-datetime fields
try:
    # Create NumPy array for non-geometry, non-datetime fields
    nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9)

    # Convert datetime fields to strings using a cursor
    datetime_data = []
    with arcpy.da.SearchCursor(feature_class, datetime_fields) as cursor:
        for row in cursor:
            datetime_data.append(row)
    
    datetime_data = np.array(convert_datetime_to_string([row[0] for row in datetime_data]), dtype=str)
    
    # Extract geometry
    geometry_data = []
    with arcpy.da.SearchCursor(feature_class, [geometry_field]) as cursor:
        for row in cursor:
            geometry_data.append(row[0])
    
    # Add geometry as separate columns
    nparr_with_geometry = np.array(
        [(item.tolist() + list(coord)) for item, coord in zip(nparr, geometry_data)],
        dtype=[(name, nparr.dtype[name]) for name in nparr.dtype.names] + [('X', float), ('Y', float)]
    )
    
    # Combine datetime data
    datetime_array = np.array(datetime_data.tolist(), dtype=str)
    
    # Create final array including datetime fields
    final_dtype = [(name, nparr_with_geometry.dtype[name]) for name in nparr_with_geometry.dtype.names] + \
                  [(f'DATE_{i}', str) for i in range(len(datetime_fields))]
    
    final_array = np.array(
        [(list(nparr_with_geometry[i]) + list(datetime_array[i])) for i in range(len(nparr_with_geometry))],
        dtype=final_dtype
    )
    
    # Save to CSV
    np.savetxt('H:\\mobcsv2_combined.csv', final_array, delimiter=",", fmt="%s", header=",".join(final_array.dtype.names), comments='')
    print("Array with geometry and dates saved successfully.")
except Exception as e:
    print(f"Error: {e}")

 

StuartMoore
Frequent Contributor

thanks @TonyAlmeida i got the same error at the same point unfortunately, in terms of data there are around 350000 records and 95 fields.

i think i'm going to use a search cursor and just write it out as a csv file

thanks for the help

Stu

0 Kudos