topic Re: Script works as stand alone, fails when ran as script tool with ascii encoding er in Python Questions

Script works as stand alone, fails when ran as script tool with ascii encoding errors

Anonymous User — Sun, 12 Dec 2021 16:48:35 GMT

I am encountering a strange problem with a script I am trying to run. It updates a Site confgiruation .xml file for setting up Silverlight based REST sites. The code below is a function that is imported into another larger script. This code works perfectly when ran as a stand alone script but it fails when ran as a script tool.

Here is the testing code:

import os, arcpy, codecs, shutil, glob, sys
import xml.etree.ElementTree as ET

def Message(msg):
    print str(msg)
    arcpy.AddMessage(str(msg))
    

def AdjustSite(Site_Folder, name, user, image):
    _dir = os.path.dirname(sys.argv[0])
    site_dir = os.path.join(_dir, 'Sites')
    temp_dir =  os.path.join(_dir, 'OtherTemplates')
    ws = os.path.join(site_dir, Site_Folder)
    tags = ['Permissions','PrintTemplates','Security','Viewers','Workflows']
    site = os.path.join(ws, 'Site.xml')
    shutil.copy(site, site[:-4] + '_Copy.xml')
    trees = ET.parse(site)
    doc = trees.getroot()
    elms = [elm.tag for elm in doc]
    with codecs.open(site, 'r', encoding='utf-8') as f:
        orig = f.readlines()[:-1]

    # Iterate through tags and add if necessary  
    for xtag in tags:
        if xtag not in elms:
            with codecs.open(os.path.join(temp_dir, '%s.xml'%xtag),'r', encoding='utf-8') as rd:
                for l in rd.readlines():
                    orig.append(l.replace('VIEWER_XXX',name).replace('USER_XXX',user).replace('ZZZZ',Site_Folder[:4]))
            Message('Added %s tag' %xtag)

    # Copy Reports
    reports = os.path.join(temp_dir, 'Reports')
    #--------------------------------------------------------------------------------------------------
    # Temporary:  Delete Reports if exists  <----------- Remove this after testing
    rep_fold = os.path.join(ws, 'Reports')
    if os.path.exists(rep_fold):
        arcpy.Delete_management(rep_fold)
    #--------------------------------------------------------------------------------------------------
    shutil.copytree(reports, os.path.join(ws, 'Reports'))
    shutil.copy(image, os.path.join(ws, 'Reports',os.path.basename(image)))
    Message('Copied Reports')

    # Rename Print templates
    for rpx in glob.glob(os.path.join(ws,'Reports','*PrintTemplate*.rpx')):
        os.rename(rpx, rpx.replace('XXXX',Site_Folder[:4]))

    # Write out new Site.xml
    orig.append('</Site>')
    with codecs.open(site, 'w', encoding='utf-8') as wr:
        wr.writelines(orig)
    Message('Updated Site.xml for %s' %Site_Folder)
    return                

if __name__ == '__main__':

    # Test Run
    # Stand alone params - comment out for script tool run
    Site_Folder = 'ConfigTest'
    name = 'ConfigViewer'
    user = 'calebma'
    image =  os.path.join(os.getcwd(), r'Logo\Zimmerman.png')

    AdjustSite(Site_Folder, name, user, image)

##    # Get args
##    # script tool params - un-comment this to run as script tool
##    argv = tuple(arcpy.GetParameterAsText(i) for i in range(arcpy.GetArgumentCount()))
##
##    AdjustSite(*argv)

And here is the error message I am getting from the script tool:

<type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xae in position 14: ordinal not in range(128)

Essentially, all this is doing is taking an existing xml file and adding some stuff from template xml files (while doing a find/replace) at the end and copying an image to the reports folder. The .rpx files I am looping through are the just xml files.

My guess is there is something going on 'under that hood' where ArcGIS is overwriting the 'utf-8' encoding my script is setting up using the codecs module and encoding it back to ascii (probably to standardize all input params for a script tool?). Has anyone ever encountered anything like this before? I don't know much about unicode/ascii so I don't know what is going on.

I have attached the sample data.zip. Everything is relative pathed so to test between stand alone and script tool, the testing variables need to be commented out and un-comment the script tool params. Also, before the script alters the "Site.xml", it creates a backup called "Site_Copy.xml". So after each run, the modified "Site.xml" needs to be deleted and the copy needs to be renamed back to "Site.xml" (these files will be located inside the Sites/ConfigTest folder).

For now, I have a janketty workaround that calls a .bat file to run this script as a python subprocess with the input params so it is ran completely outside of ArcGIS as stand alone. This is obviously not ideal and it seems the issue is occurring from being ran from an ArcGIS Script tool.

Re: Script works as stand alone, fails when ran as script tool with ascii encoding er

T__WayneWhitley — Tue, 10 Dec 2013 20:19:58 GMT

hmmmm, I ran as both standalone and script tool (with an unrelated minor change for the image pathname) and both ran fine --- could be because I'm on a different os (32 bit)....or I've made a mistake shuffling the files around.

I'll keep looking at it. Out of curiosity can you try running it on a different system - maybe hop over to a 10.1 or .2 system? Maybe invoke the 32-bit (if you have the 64-bit installed as well). I'm just trying to understand this better.

Thanks,
Wayne

Re: Script works as stand alone, fails when ran as script tool with ascii encoding er

T__WayneWhitley — Tue, 10 Dec 2013 20:36:15 GMT

Bingo, got the error!
Strange indeed... sorry, Caleb, I simply wasn't paying attention to your script comments!

Let me see here.......try to isolate this further.

Re: Script works as stand alone, fails when ran as script tool with ascii encoding er

T__WayneWhitley — Tue, 10 Dec 2013 20:45:36 GMT

Here you go Caleb -
It's a dumb pet trick: I printed your tuple argv and it turns out that is what it's tripping on in passing from the ArcGIS tool interface to python (if I stated that correctly) --- at any rate, the 'encoding' was lost in translation (or not translated, more exactly), so to speak.

I did this, and it 'corrected' itself:

argv = tuple(str(arcpy.GetParameterAsText(i)) for i in range(arcpy.GetArgumentCount()))

Re: Script works as stand alone, fails when ran as script tool with ascii encoding er

Anonymous User — Tue, 10 Dec 2013 20:53:21 GMT

Here you go Caleb -
It's a dumb pet trick: I printed your tuple argv and it turns out that is what it's tripping on in passing from the ArcGIS tool interface to python (if I stated that correctly) --- at any rate, the 'encoding' was lost in translation (or not translated, more exactly), so to speak.

I did this, and it 'corrected' itself:

argv = tuple(str(arcpy.GetParameterAsText(i)) for i in range(arcpy.GetArgumentCount()))

Wow.......You have got to be kidding me...Well I feel like an idiot. I never printed out the tuple of args to see what that looked like for the script tool. That's what I get for trying to use shortcuts. Thanks Wayne!

Re: Script works as stand alone, fails when ran as script tool with ascii encoding er

T__WayneWhitley — Tue, 10 Dec 2013 21:14:41 GMT

Actually, I'm surprised too - I just assumed Unicode was fully supported. ArcGIS does offer at least 'partial' support - I know that because I was quite surprised to find you could use unicode characters for degrees, minutes, seconds in the field calculator....it is still something I haven't wrapped my head around. I guess then the rule of thumb here is to test at every possible point of failure, as silly as it may seem.

I'm a little surprised ESRI didn't get back to you on this. (And I apologize for not getting back to you sooner myself.)

Ah, well, so much to learn and so little time...keep up the good work! By the way, interesting script.

Enjoy,
Wayne