topic Re: Help with Logging in Python Questions

Help with Logging

StevenTaylor — Mon, 20 May 2013 17:24:34 GMT

I am having some issues actually getting my python script to write out to a log file. It is running to the point of the script that says "with open(rapLogFile, 'r') as f:" then I am getting a traceback error of "Traceback (most recent call last):
File "C:\Python26\ArcGIS10.0\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 312, in RunScript
exec codeObject in __main__.__dict__
File "C:\Documents and Settings\taylorsb\Desktop\Rec_Post_Log_Test.py", line 145, in <module>
thisLogger.error(' ' + e.args[0])
TypeError: cannot concatenate 'str' and 'int' objects". When I comment out that part of the traceback it is still telling me it does not like something about line 90. Any help would be appreciated! Here is the code:

import sys, arcpy, os, smtplib, logging, datetime, time
from logging.handlers import TimedRotatingFileHandler

fullPath = r'\\10.1.64.36\Indiana\Steve_WIP\Nightly_Log_Files'
##thisPath = os.path.dirname(fullPath)
##thisFile = os.path.basename(fullPath)
thisName = r'\\10.1.64.36\Indiana\Steve_WIP\Nightly_Log_Files'
##cfgFile = thisName + '.cfg'
logFileInfo = thisName + '-Info.log'
logFileDebug = thisName + '-Debug.log'

chFormatter = logging.Formatter('%(funcName)-24s: %(levelname)-8s %(message)s')
fhFormatter = logging.Formatter('%(asctime)-2s %(funcName)-24s %(levelname)-8s %(message)s', '%m/%d/%Y %I:%M:%S %p')

ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(chFormatter)

fhInfo = logging.FileHandler(logFileInfo, 'w')
fhInfo.setLevel(logging.INFO)
fhInfo.setFormatter(fhFormatter)

fhDebug = TimedRotatingFileHandler(logFileDebug, when='D', interval=30, backupCount=13)
fhDebug.setLevel(logging.DEBUG)
fhDebug.setFormatter(fhFormatter)

global thisLogger
thisLogger = logging.getLogger('ThisLogger')
thisLogger.setLevel(logging.DEBUG)
thisLogger.addHandler(ch)
thisLogger.addHandler(fhInfo)
thisLogger.addHandler(fhDebug)

thisLogger.info('* * *  B E G I N  ' + fullPath + '  * * *')
thisLogger.info('Begin function\n')

rapLogFile = thisName + '-ReconcileAndPost.log'
rapLogFile = fullPath + '\\' + rapLogFile
rLogFile = thisName + '-Reconcile.log'
rLogFile = fullPath + '\\' + rLogFile


West_Versions=arcpy.ListVersions("Database Connections\\INAW_West_GISADMIN_10.sde")
i=2
targetVersion = 'West_Versions'

      

try:
    # Get start time
    start = datetime.datetime.now()
        
    thisLogger.info('  Begin function')
    
    # Get a list of the versions that match the versionFilter
    thisLogger.debug('  Getting a list of the versions to reconcile and post...')
    versionList = []
    ##for version in arcpy.ListVersions("Database Connections\\INAW_West_GISADMIN_10.sde"):
        ##if version.name.upper().split('.')[1] == versionFilter:
            ##versionList.append(version.name)
            ##thisLogger.debug('  - ' + version.name)
        ##thisLogger.info('  Found ' + str(len(versionList)) + ' versions to reconcile and post.')
                
    # Remove the log files (from previous runs) if they exist
    if os.path.isfile(rapLogFile):
        os.remove(rapLogFile)
    if os.path.isfile(rLogFile):
        os.remove(rLogFile)
                
    # Execute the ReconcileVersions tool.       
    while i <=len(West_Versions)-1:
        thisLogger.debug('  Reconciling and posting ' + West_Versions + '...') 
        print "Reconciling: " + West_Versions
        arcpy.ReconcileVersion_management("Database Connections\\INAW_West_GISADMIN_10.sde", West_Versions, West_Versions[1], "BY_OBJECT", "FAVOR_TARGET_VERSION", "LOCK_AQUIRED", "ABORT_CONFLICTS", "POST")
        i=i+1
        print "Reconcile and Post Production Version Complete"
    print "Reconciling: " + West_Versions[1]
    arcpy.ReconcileVersion_management("Database Connections\\INAW_West_GISADMIN_10.sde", West_Versions[1], West_Versions[0], "BY_OBJECT", "FAVOR_TARGET_VERSION", "LOCK_AQUIRED", "ABORT_CONFLICTS", "POST")
    print "Reconcile and Post QC Version Complete"
    
    # Open the log file and write the contents to thisLogger
    thisLogger.info('  Results of Reconcile and Post:')
    thisLogger.info(' ')
    with open(rapLogFile, 'r') as f:
        lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + line)
    thisLogger.info(' ')
        
    # Execute the ReconcileVersions tool.
    # Reconcile all the child versions with the parent to pull all the edits down to the child versions
    thisLogger.debug('  Reconciling changes in ' + targetVersion + ' to the posted versions...')
    print "Starting Reconciling Only Operation"
    print "Reconciling: " + West_Versions[1]
    arcpy.ReconcileVersion_management("Database Connections\\INAW_West_GISADMIN_10.sde", West_Versions[1], West_Versions[0], "BY_OBJECT", "FAVOR_TARGET_VERSION", "LOCK_AQUIRED", "ABORT_CONFLICTS", "NO_POST")
    print "Reconcile Only QC Version Complete"
    i=2
    while i <=len(West_Versions)-1:
        thisLogger.debug('  Reconciling changes in ' + West_Versions[1] + ' to ' + West_Versions)
        print "Reconciling: " + West_Versions
        arcpy.ReconcileVersion_management("Database Connections\\INAW_West_GISADMIN_10.sde", West_Versions, West_Versions[1], "BY_OBJECT", "FAVOR_TARGET_VERSION", "LOCK_AQUIRED", "ABORT_CONFLICTS", "NO_POST")
        i=i+1
        print "Reconcile Only Production Versions Complete"
    
    
    # Open the log file and write the contents to thisLogger
    thisLogger.info('  Results of Reconcile:')
    thisLogger.info(' ')
    with open(rLogFile, 'r') as f:
        lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + line)
    thisLogger.info(' ')

    # Append the contents of logFile2 to logFile1
    thisLogger.debug('  Appending the Reconcile log file to the Reconcile and Post log file...')
    with open(rapLogFile, 'a') as f1:
        with open(rLogFile, 'r') as f2:
            f1.write('\n' + f2.read())

    # Email a log to the rapDestination users:
    ##thisLogger.debug('  Sending email...')
    ##sendEmail(mailServer, logFile1, sender, destination, subject)
    ##thisLogger.info('  Sent "' + subject + '" email.')
        
    # Get end time and calculate the elapsed time
    elapsedTime = getElapsedTime(start)

    thisLogger.info('  --[Function Time: ' + elapsedTime + '] End function normally\n')

        
except Exception as e:
    # If an error occurred, print line number and error message
    import traceback, sys
    tb = sys.exc_info()[2]
    thisLogger.error('    Line %i' % tb.tb_lineno)
    thisLogger.error('  ' + e.args[0])
    thisLogger.info('  End function with ERROR\n')

Re: Help with Logging

ThaiTruong — Sat, 11 Dec 2021 16:25:12 GMT

Your error could be from this line:

targetVersion = 'West_Versions'

Try to modify it to (drop the quotes):

targetVersion = West_Versions

Re: Help with Logging

StevenTaylor — Mon, 20 May 2013 18:36:04 GMT

I eliminated the quotes and I am still getting the same error (See picture attached).

Re: Help with Logging

ThaiTruong — Sat, 11 Dec 2021 16:25:15 GMT

I found another error here:

    # Get end time and calculate the elapsed time
    elapsedTime = getElapsedTime(start)

    thisLogger.info('  --[Function Time: ' + elapsedTime + '] End function normally\n')

need to modify to:

    # Get end time and calculate the elapsed time
    elapsedTime = getElapsedTime(start)

    thisLogger.info('  --[Function Time: ' + str(elapsedTime) + '] End function normally\n')

You need to debug your code line by line to see any where in the code you trying to add an integer to a string (which raised the error).

Re: Help with Logging

JamesCrandall — Sat, 11 Dec 2021 16:25:18 GMT

The error is telling you exactly what is wrong: you are trying to concatenate an Integer and a string. Likely values in "line" are coming across as integers


with open(rapLogFile, 'r') as f:
    lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + line)
    thisLogger.info(' ')

Change to:


with open(rapLogFile, 'r') as f:
    lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + str(line))
    thisLogger.info(' ')

Re: Help with Logging

StevenTaylor — Sat, 11 Dec 2021 16:25:21 GMT

Thanks for that, I also found a part in the traceback that had that issue:

 thisLogger.error('    Line %i' % tb.tb_lineno)
    thisLogger.error('  ' + str(e.args[0]))

I would still consider myself a novice at Python, but you only get better through trying, learning, and asking right. This is my first time attempting logging as well.

My traceback error is now gone, but here is what I am getting now (see attached image).

When I look at the Python in Notepad ++ line 90 is this section:

with open(rapLogFile, 'r') as f:
        lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + line)
    thisLogger.info(' ')

Re: Help with Logging

StevenTaylor — Sat, 11 Dec 2021 16:25:23 GMT

I am thinking it has something to do with the way I wrote this:

#\\10.1.64.36\Indiana\Steve_WIP\Nightly_Log_Files
fullPath = r'\\10.1.64.36\Indiana\Steve_WIP\Nightly_Log_Files'
##thisPath = os.path.dirname(fullPath)
##thisFile = os.path.basename(fullPath)
thisName = r'\\10.1.64.36\Indiana\Steve_WIP\Nightly_Log_Files'
##cfgFile = thisName + '.cfg'
logFileInfo = thisName + '-Info.log'
logFileDebug = thisName + '-Debug.log'

I looked up error 2 and I have found some results saying the file cannot be found. Maybe I am reading the output error wrong, but I it also could very well be with how I wrote the above. Stepping outside my comfort zone, I appreciate all your guys input on this!

Re: Help with Logging

JamesCrandall — Sat, 11 Dec 2021 16:25:26 GMT

Thanks for that, I also found a part in the traceback that had that issue:
 thisLogger.error('    Line %i' % tb.tb_lineno)
    thisLogger.error('  ' + str(e.args[0]))
I would still consider myself a novice at Python, but you only get better through trying, learning, and asking right. This is my first time attempting logging as well.

My traceback error is now gone, but here is what I am getting now (see attached image).

When I look at the Python in Notepad ++ line 90 is this section:
with open(rapLogFile, 'r') as f:
        lines = f.read().splitlines()
    f.close
    for line in lines:
        thisLogger.info('    ' + line)
    thisLogger.info(' ')

Did you attempt to set line to string values?

thisLogger.info('    ' + str(line))