Re: UnicodeDecodeError: utf8 codec can't decode byte invalid continuation byte

68216
20
05-17-2015 02:57 AM
Yaron_YosefCohen
Occasional Contributor II

From: SearchCursor directory and subdirectories using python

I run the code and py fined layers with YEUD=20

but i also get en error:

Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\geonet.PY", line 11, in <module>
    for row in rows:
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 23: invalid continuation byte
>>> 

i use files and folders names Right to left- how can i solve this issue please?

20 Replies
curtvprice
MVP Esteemed Contributor

I started a new thread on this issue.

These links may help, but I have no experience with international codesets -- perhaps you need to encode your path strings before parsing them, or do something with locale - or somehow change your codec to Latin-1?

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 6: ordinal not in range(128)

python - UnicodeDecodeError, invalid continuation byte - Stack Overflow

RebeccaStrauch__GISP
MVP Emeritus

Check to make sure your indentations are correct and : are placed where they should be, etc.  I've received similar errors  when I've had just a typo several lines up.  i know this question was spun off another with coding suggestions., so maybe  something was missed when merging with your code.  Errors can be deceiving sometimes, so worth a syntax check.

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

Hi Rebecca,

i run the code and there where no identations error-else python would not  run the code

0 Kudos
RebeccaStrauch__GISP
MVP Emeritus

YYC - It may be helpful for others if you post the full code where you are getting an error.  The post this was spun off of shows Curtis' suggestion, but not you final code.

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

Hi Rebecca,

this is the code:

import os
import fnmatch
import arcpy
rootPath = r"C:\Project\layers" 
pattern = 'mig*.shp' 
for root, dirs, files in os.walk(rootPath): 
    for filename in fnmatch.filter(files, pattern): 
        shp = os.path.join(root, filename)
        if arcpy.ListFields(shp, "YEUD"):
            print("{} has YEUD field".format(shp)) 
            with arcpy.da.SearchCursor(shp, ["YEUD"]) as rows:
                for row in rows:
                    if row[0] == 20:
                        print("{} has a record with YEUD = 20".format(shp))
                        break

0 Kudos
curtvprice
MVP Esteemed Contributor

Based on this:

character encoding - Python: Converting from ISO-8859-1/latin1 to UTF-8 - Stack Overflow

I'm wondering if you may be able to fix this by doing a quick conversion to utf-8:

shp = (os.path.join(root, filename)).encode("utf-8")

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

i get an error :

>>> 
Traceback (most recent call last):
  File "C:\Users\yaron.KAYAMOT\Desktop\python.py", line 9, in <module>
    shp = (os.path.join(root, filename)).encode("utf-8")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 23: ordinal not in range(128)
>>> 
0 Kudos
curtvprice
MVP Esteemed Contributor

Sorry, I didn't read the post I linked carefully enough. In this example iso-8859-1 is the codes your paths are in.

shp.decode('iso-8859-1').encode('utf-8')

I'm going to link the post again because it has lots of really useful information about unicode, latin1, utf-8...

python - UnicodeDecodeError, invalid continuation byte - Stack Overflow

0 Kudos
Yaron_YosefCohen
Occasional Contributor II

where to  insert this route?

0 Kudos