Re: UnicodeDecodeError: utf8 codec can't decode byte invalid continuation byte

68440
20
05-17-2015 02:57 AM
Yaron_YosefCohen
Occasional Contributor II

From: SearchCursor directory and subdirectories using python

I run the code and py fined layers with YEUD=20

but i also get en error:

Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\geonet.PY", line 11, in <module>
    for row in rows:
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 23: invalid continuation byte
>>> 

i use files and folders names Right to left- how can i solve this issue please?

20 Replies
curtvprice
MVP Esteemed Contributor

This line is a replacement for line 08. I was hoping someone that knows more about international characters will chime in!

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

i get an error:

>>>


Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\python.py", line 9, in <module>
    shp.decode('iso-8859-1').encode('utf-8')
NameError: name 'shp' is not defined
>>>
0 Kudos
curtvprice
MVP Esteemed Contributor

Sorry:

shp = (os.path.join(root, filename)).decode('iso-8859-1').encode("utf-8")

Yaron_YosefCohen
Occasional Contributor II

an error:

IOError: <unprintable IOError object>

i think now there no unicode error -but there problem else

0 Kudos
curtvprice
MVP Esteemed Contributor

I think you're going to have to find someone that has a better handle on this unicode stuff. I added some tags that maybe will attract some help.

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

YYC,

I'm not familiar with unicode, but I did test you code with a some of my own shape files against a field ACRES (format DOUBLE) and it worded without an error.  What is the format of your "YEUD" field?  The code I used (changed to match my data):

import os  
import fnmatch  
import arcpy  
rootPath = r"D:\Project\layers"   
pattern = 'lake*.shp'   
for root, dirs, files in os.walk(rootPath):   
    for filename in fnmatch.filter(files, pattern):   
        shp = os.path.join(root, filename)  
        if arcpy.ListFields(shp, "ACRES"):  
            print("{} has ACRES field".format(shp))   
            with arcpy.da.SearchCursor(shp, ["ACRES"]) as rows:  
                for row in rows:  
                    if row[0] > 300000:  
                        print("{} has a record with ACRES > 3000000".format(shp))  
                        break  

My results were:

D:\Project\layers\lakes.shp has ACRES field

D:\Project\layers\lakes.shp has a record with ACRES > 3000000

D:\Project\layers\lake_10.shp has ACRES field

D:\Project\layers\lake_11.shp has ACRES field

0 Kudos
curtvprice
MVP Esteemed Contributor

YYC's issue is not my scripting -- he's working with international characters in the path names, which are way out of my expertise!!  The YEUD field is an integer (in the international language of IEEE representations) so there is not a problem there.

YYC, I'm afraid think you may need to ask for tech support from your local Esri affiliate or user groups on this one.

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

Rebecca,

"YEUD" field type is Double

0 Kudos
curtvprice
MVP Esteemed Contributor

As I said YEUD is not the issue, it's the feature class pathname (shp) using those funky right-to-left characters.

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

i think you right

0 Kudos