Select to view content in your preferred language

Re: UnicodeDecodeError: utf8 codec can't decode byte invalid continuation byte

71276
20
05-17-2015 02:57 AM
Yaron_YosefCohen
Frequent Contributor

From: SearchCursor directory and subdirectories using python

I run the code and py fined layers with YEUD=20

but i also get en error:

Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\geonet.PY", line 11, in <module>
    for row in rows:
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 23: invalid continuation byte
>>> 

i use files and folders names Right to left- how can i solve this issue please?

20 Replies
curtvprice
MVP Alum

This line is a replacement for line 08. I was hoping someone that knows more about international characters will chime in!

0 Kudos
Yaron_YosefCohen
Frequent Contributor

i get an error:

>>>


Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\python.py", line 9, in <module>
    shp.decode('iso-8859-1').encode('utf-8')
NameError: name 'shp' is not defined
>>>
0 Kudos
curtvprice
MVP Alum

Sorry:

shp = (os.path.join(root, filename)).decode('iso-8859-1').encode("utf-8")

Yaron_YosefCohen
Frequent Contributor

an error:

IOError: <unprintable IOError object>

i think now there no unicode error -but there problem else

0 Kudos
curtvprice
MVP Alum

I think you're going to have to find someone that has a better handle on this unicode stuff. I added some tags that maybe will attract some help.

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

YYC,

I'm not familiar with unicode, but I did test you code with a some of my own shape files against a field ACRES (format DOUBLE) and it worded without an error.  What is the format of your "YEUD" field?  The code I used (changed to match my data):

import os  
import fnmatch  
import arcpy  
rootPath = r"D:\Project\layers"   
pattern = 'lake*.shp'   
for root, dirs, files in os.walk(rootPath):   
    for filename in fnmatch.filter(files, pattern):   
        shp = os.path.join(root, filename)  
        if arcpy.ListFields(shp, "ACRES"):  
            print("{} has ACRES field".format(shp))   
            with arcpy.da.SearchCursor(shp, ["ACRES"]) as rows:  
                for row in rows:  
                    if row[0] > 300000:  
                        print("{} has a record with ACRES > 3000000".format(shp))  
                        break  

My results were:

D:\Project\layers\lakes.shp has ACRES field

D:\Project\layers\lakes.shp has a record with ACRES > 3000000

D:\Project\layers\lake_10.shp has ACRES field

D:\Project\layers\lake_11.shp has ACRES field

0 Kudos
curtvprice
MVP Alum

YYC's issue is not my scripting -- he's working with international characters in the path names, which are way out of my expertise!!  The YEUD field is an integer (in the international language of IEEE representations) so there is not a problem there.

YYC, I'm afraid think you may need to ask for tech support from your local Esri affiliate or user groups on this one.

0 Kudos
Yaron_YosefCohen
Frequent Contributor

Rebecca,

"YEUD" field type is Double

0 Kudos
curtvprice
MVP Alum

As I said YEUD is not the issue, it's the feature class pathname (shp) using those funky right-to-left characters.

0 Kudos
Yaron_YosefCohen
Frequent Contributor

i think you right

0 Kudos