topic export feature to csv in arcmap in Python Questions

export feature to csv in arcmap

StuartMoore — Thu, 01 Aug 2024 14:05:10 GMT

i'm trying to export a feature layer with several fields of different datatypes but 5 of the fields are datetime data types but when i use the below i get this error

nparr = arcpy.da.FeatureClassToNumPyArray('mob', fields,null_value=-9) numpy.savetxt('H:\mobcsv2.csv', nparr, delimiter=",", fmt="%s")

how do i get around this?

Stu

Re: export feature to csv in arcmap

TonyAlmeida — Thu, 01 Aug 2024 14:29:15 GMT

It seems like the feature class contains fields with geometry or datetime data types which connot be converted to a numpy array.

So you can try to exclude them. Untested

import arcpy import numpy as np # Set workspace arcpy.env.workspace = 'Geodatabase_Path' # Replace with your actual geodatabase path # Define feature class and fields feature_class = 'mob' all_fields = [f.name for f in arcpy.ListFields(feature_class)] fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']] # Check excluded fields print(f"All fields: {all_fields}") print(f"Selected fields: {fields}") # Create NumPy array excluding geometry and datetime fields try: nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9) np.savetxt('H:\\mobcsv2.csv', nparr, delimiter=",", fmt="%s") print("Array saved successfully.") except Exception as e: print(f"Error: {e}")

If you need them you can try something like this. Untested

Re: export feature to csv in arcmap

StuartMoore — Thu, 01 Aug 2024 15:38:14 GMT

thanks @TonyAlmeida really appreciate the time, i tried it but get the same error about the datetime, i did remove the geometry and got the same also

Re: export feature to csv in arcmap

TonyAlmeida — Thu, 01 Aug 2024 16:28:09 GMT

It's hard to determine the issue, if we don' t see the entire code. Also what print out when the code got to lines 13 & 14 of the first code? I am guessing it's not finding the Date field, is the name of the actual field called "Date"?

Re: export feature to csv in arcmap

StuartMoore — Fri, 02 Aug 2024 12:58:57 GMT

thanks @TonyAlmeida the first code does print all the field names and the next only the fields that are not geomentry or date but when it tried to export i get this error:

when trying to run the second piece of code in steps i get this error with the code, but the print(datetime_fields) does return the 5 date fields

nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields_with_geometry + datetime_fields, null_value=-9)

Re: export feature to csv in arcmap

TonyAlmeida — Fri, 02 Aug 2024 22:34:02 GMT

The array memory error typically occurs when the script tries to create an array that's too large for the available memory, how big is the data?

Try this,

import arcpy import numpy as np # Set workspace arcpy.env.workspace = 'Geodatabase_Path' # Replace with your actual geodatabase path # Define feature class feature_class = 'mob' # List all fields all_fields = [f.name for f in arcpy.ListFields(feature_class)] fields = [f for f in all_fields if arcpy.ListFields(feature_class, f)[0].type not in ['Geometry', 'Date']] # Include geometry field to extract coordinates geometry_field = 'SHAPE@XY' # Include datetime fields separately for conversion datetime_fields = [f.name for f in arcpy.ListFields(feature_class) if f.type == 'Date'] # Function to convert datetime fields to string def convert_datetime_to_string(datetime_values): return [val.strftime('%Y-%m-%d %H:%M:%S') if val else '' for val in datetime_values] # Extract non-geometry and non-datetime fields try: # Create NumPy array for non-geometry, non-datetime fields nparr = arcpy.da.FeatureClassToNumPyArray(feature_class, fields, null_value=-9) # Convert datetime fields to strings using a cursor datetime_data = [] with arcpy.da.SearchCursor(feature_class, datetime_fields) as cursor: for row in cursor: datetime_data.append(row) datetime_data = np.array(convert_datetime_to_string([row[0] for row in datetime_data]), dtype=str) # Extract geometry geometry_data = [] with arcpy.da.SearchCursor(feature_class, [geometry_field]) as cursor: for row in cursor: geometry_data.append(row[0]) # Add geometry as separate columns nparr_with_geometry = np.array( [(item.tolist() + list(coord)) for item, coord in zip(nparr, geometry_data)], dtype=[(name, nparr.dtype[name]) for name in nparr.dtype.names] + [('X', float), ('Y', float)] ) # Combine datetime data datetime_array = np.array(datetime_data.tolist(), dtype=str) # Create final array including datetime fields final_dtype = [(name, nparr_with_geometry.dtype[name]) for name in nparr_with_geometry.dtype.names] + \ [(f'DATE_{i}', str) for i in range(len(datetime_fields))] final_array = np.array( [(list(nparr_with_geometry[i]) + list(datetime_array[i])) for i in range(len(nparr_with_geometry))], dtype=final_dtype ) # Save to CSV np.savetxt('H:\\mobcsv2_combined.csv', final_array, delimiter=",", fmt="%s", header=",".join(final_array.dtype.names), comments='') print("Array with geometry and dates saved successfully.") except Exception as e: print(f"Error: {e}")

Re: export feature to csv in arcmap

StuartMoore — Wed, 07 Aug 2024 13:32:45 GMT

thanks @TonyAlmeida i got the same error at the same point unfortunately, in terms of data there are around 350000 records and 95 fields.

i think i'm going to use a search cursor and just write it out as a csv file

thanks for the help

Stu