Python script file not encoded correctly with ANSI or utf8

[JoeBorgione]

We've been struggling to get a python script running that originally came from ESRI: it's a maintenance script for a vector tile basemaps.  The script hasn't worked for us and the subject line above is the response we got from ESRI tech support.

Below are identical lines of code:  Line one is what we see in IDLE while Line 2 is what we see in Spyder. 

tpk =  r'‪C:\Users\jaburton\Documents\ArcGIS\Projects\VectorBasemap\VectorTilePackage.vtpk

tpk =  r'‪C:\Users\jaburton\Documents\ArcGIS\Projects\VectorBasemap\VectorTilePackage.vtpk'‍‍‍

When we run it as a stand-alone script in Spyder it bails out with this error:

So its' not seeing the path where the variable tpk points to. The source of the raw text string is a copy and paste from a Windows Explorer window: pretty standard practice.

How do we convince Spyder all is good?

[DanPatterson_Retired · Accepted solution]

# -*- coding: utf-8 -*-

at the very top of all scripts

[JoeBorgione]

awesome.  Thanks!

edited moments later:

Dan- it already appears at the top of the script; Spyder does it for you...

[DanPatterson_Retired]

One of the 2 to 3 things you need to remember... strings are no more... strings are now Unicode and way more character sets are now supported.

[JoeBorgione]

Just ran it again with # -*- coding: utf-8 -*- as line 1.  Errors out with the same problem....

[DanPatterson_Retired]

even worse... you didn't raw encode or use forward slashes..

Our class motto, when it comes to file paths

"C:\Users\Is\For\Losers"
 

File "<ipython-input-1-d259788efdf0>", line 1
    "C:\Users\Is\For\Losers"
                            ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Don't use 'Users'

[JoeBorgione]

It's working now, not sure what the hicup was....

[Luke_Pinner]

Quick hack to strip out any non-ascii characters:

input = 'C:/temp/test.py'
output = 'C:/temp/test_edited.py'

input = open(input).read()
with open(output, 'w') as outf:
    outf.write(input.encode('ascii', 'ignore').decode())

print(repr(input))
print(repr(open(output).read()))
‍‍‍‍‍‍‍‍‍

Output:

"tpk =  r'‪C:\\Users\\jaburton\\Documents\\ArcGIS\\Projects\\VectorBasemap\\VectorTilePackage.vtpk'\n"
"tpk =  r'C:\\Users\\jaburton\\Documents\\ArcGIS\\Projects\\VectorBasemap\\VectorTilePackage.vtpk'\n"
‍‍