nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

ArcPy fails due to Unicode encoding error

rbd

Hi All,

I'm automating exports of some GDB feature class attribute tables to CSV using Python. One FC works beautifully and the other produces this error:

UnicodeEncodeError                        Traceback (most recent call last)
In  [27]:
Line 9:     csvwriter.writerow(row)

File C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\encodings\cp1252.py, in encode:
Line 19:    return codecs.charmap_encode(input,self.errors,encoding_table)[0]

UnicodeEncodeError: 'charmap' codec can't encode character '\u0374' in position 149: character maps to <undefined>

Does anyone know what I should do to fix this?

-r

Find more posts tagged with

Python

Accepted answers

DavidSolari

You open the file on line 27, try adding

encoding="utf-8"

as parameter and see if that fixes things.

All comments

DavidSolari

It might be worth a quick trip to the official Python docs for file objects. In short, the "open" function returns a file object with various settings from the function, including the file's encoding. The "writer" object from the "csv" module is just a wrapper around the file object that translates raw CSV data to and from Python data types; you can write a CSV file without the wrapper but it makes it easier.

rbd

Success @DavidSolari ! I had tried that piece last week in the row below, where other parameters seem to be listed. Do you know the reason it should go in Line 27 rather than Line 28? I'm trying to understand the inner workings of the code so I can improve. Thanks again!

DavidSolari

You open the file on line 27, try adding

encoding="utf-8"

as parameter and see if that fixes things.

rbd

Thanks David. I've looked at a number of resources for troubleshooting this, and have read that utf-8 is used, but I have not been successful at placing that specification in my code in a way that works. Where in the code should this be specified? Below is my code, up to the point of failure:

# import the modules, including ones that hopefully troubleshoot the unicode encoding error
import arcpy
from arcpy import da
import csv
import codecs
import os
import fileinput
import sys
# set the workspace environments
# file location
arcpy.env.workspace = r"C:workspace"
# overwrite files of the same name in the output location
arcpy.env.overwriteOutput = True
# transfer the domains and descriptions to shapefile headers and attributes
arcpy.env.transferDomains = True
#define the input feature classes output file names and types as variables
in_gdb = r"C:In.gdb"
in_sites = f"{in_gdb}/WSPRC_CulturalResources/ResourcePolyRecordedUpdated"
sites_shp = f"{arcpy.env.workspace}/ResourcePolyRecordedUpdated.shp"
out_sites_temp = f"{arcpy.env.workspace}/DoNotModify_CurrentSitesListTemp.csv"
out_sites = f"{arcpy.env.workspace}/DoNotModify_CurrentSitesList.csv"
## export the feature classes to shapefiles
arcpy.conversion.FeatureClassToShapefile(in_sites, arcpy.env.workspace)
# delete the unnecessary fields from the exported shapefiles
arcpy.management.DeleteField(sites_shp, ["Shape_Leng", "Shape_Area"])
#write the csv
with open(out_sites_temp, "w") as csvfile:
    csvwriter = csv.writer(csvfile, delimiter=',', lineterminator='\n')
    ## Write field name header line
    fields = ['ProjName', 'Date', 'Acreage', 'Notes']
    csvwriter.writerow(fields), 
    ## Write data rows
    with arcpy.da.SearchCursor(sites_shp, fields) as s_cursor:
        for row in s_cursor:
            csvwriter.writerow(row)

DavidSolari

Based on the encoding the CSV writer pulled in you're trying to write the data to a file that isn't Unicode compatible. Specify a suitable encoding when you open the file ("utf-8" works in virtually every case) and you should make more progress.

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.