topic Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog. in Python Questions

Python script executes in pyscripter but fails when trying to run as script in arccatalog.

Anonymous User — Mon, 26 Oct 2020 15:58:07 GMT

I've got a script that will execute fine if I run it in PyScripter.

However when I import the script into a toolbox in ArcCatalog it will not execute.

I'm a novice with Python and I'm confused about is causing the issue.

The script is using pyPDF2 library and I've checked the Python environments in both C:\Python27\ArcGIS10x64 and ArcGIS10.5 to confirm that pyPDF2 package is installed. It is.

PyScripter is using the 10x64 environment and I'm curious how to confirm which one ArcCatalog is using.

Help is appreciated.

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

JoshuaBixby — Mon, 26 Oct 2020 16:10:35 GMT

Instead of guessing what may or may not be working, what do the error messages say when you try to run the script as a script tool? That is, if you check the messages from the tool, what are they?

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

Anonymous User — Mon, 26 Oct 2020 16:47:56 GMT

Good point.

I just double checked the script and it will successfully execute once and then fail on successive attempts unless I restart arccatalog. When I initially posted this question I had forgotten about that.

I have no idea what this error means.

'NoneType' object has no attribute 'modules'
<attribute 'message' of 'exceptions.BaseException' objects>

I can post the script if you're interested.. It's relatively simple but I'm having a relatively hard time!

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

JoshuaBixby — Mon, 26 Oct 2020 17:16:56 GMT

If sharing the script is possible, that always helps. The message you are seeing is common when Python None is returned by a function/method/properties, and subsequent code is expecting something different, so the code tries to treat the None like the object it expects and it fails.

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

Anonymous User — Sat, 11 Dec 2021 14:32:04 GMT

I'm using TIKA and PyPDF2

I've spent a good amount of time on this since I'm a newbie... my goal is simple, just to get the count of deaths from the PDF report for our county and this is the only method I've been able to get working.

I'm assuming that Catalog is locking the file after the first run but I don't know how to release it once the process has completed.

it's been failing here:

pdf_reader = PyPDF2.PdfFileReader(pdf_content)

import urllib, io
import requests
import PyPDF2
import os
import shutil
import datetime
import arcpy

#import tika
from datetime import date
from PyPDF2 import PdfFileReader, PdfFileWriter

today = datetime.date.today()
yesterday = today - datetime.timedelta(days=1)

os.chdir("C:\\Test") #Change directory to location to save the file
dt = date.today().strftime("%#m-%#d-%Y")

#to check for and use the current date uncomment the next line:
strUrl = "https://www.health.pa.gov/topics/Documents/Diseases%20and%20Conditions/COVID-19%20County%20Data/County%20Case%20Counts_" + dt + ".pdf"
print (strUrl)

#If the daily report isn't yet available, use this one for testing
#strUrl = "https://www.health.pa.gov/topics/Documents/Diseases%20and%20Conditions/COVID-19%20County%20Data/County%20Case%20Counts_10-21-2020.pdf"
response = urllib.urlopen(strUrl)

PDF = response.read()
#Search for a File Not Found error in return. If found, it hasn't been published yet
FileNotFoundStatus = PDF.find("File Not Found")

if (FileNotFoundStatus == -1) :
 # We didn't get a -1| 'file not found' error, the PDF Document is available and can be processed
 try:
arcpy.AddMessage ("The Daily Report is available to be processed")
 pdf_content = io.BytesIO(PDF)
 pdf_reader = PyPDF2.PdfFileReader(pdf_content)
 arcpy.AddMessage ("Read PDF")
#pageObj = pdf_reader.getPage(0)
 if pdf_reader.isEncrypted:
 print ("PDF Was encrypted")
 pdf_reader.decrypt("")
 else:
 #PDF Is not encrypted
 import urllib2
 response = urllib2.urlopen(strUrl)
 #####TEMPORARY FOR TESTING###
 #####COMMENT OUT WHEN CURRENT DAY IS AVAILABLE
 #dt = "10-21-2020"
 #############################
 file = open(dt + ".pdf", 'wb')
 file.write(response.read())
 print(file.name)
 from tika import parser
 parsed_pdf = parser.from_file(file.name)
 txtDOHData = parsed_pdf["content"]
 LancLoc = txtDOHData.find("LANCASTER SE ")
 DOHCases = txtDOHData[LancLoc+13:LancLoc + 17]
 arcpy.AddMessage("Department of Health cases reported for " + dt + " : " + DOHCases)
 #print("Department of Health cases reported for " + dt + " : " + DOHCases)
 file.close()
 TimelineTbl = "\\OutputData\\DEVTLOutputData.gdb\\TLOutputTbl"
 TLSumOutputtbl = "\\OutputData\\DEVTLOutputData.gdb\\TLSumOutputTbl"

except Exception as e:
 arcpy.AddMessage("Error Occured")
 arcpy.AddMessage(e)
else:
 print ("The Daily Report from Department of Health has not been published yet for: " + date.today().strftime("%#m-%d-%Y"))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

Anonymous User — Sat, 11 Dec 2021 14:32:07 GMT

Thanks again for your feedback.. I've got a different question related to this script.. It's actually a general python scripting question and scope of variables.

I'm referring to the section of the code where I do a selection on records and then use a search cursor.

#Set variable for yesterday's cumulative cases
 with arcpy.da.SearchCursor("Timelinetv", ("New_Cases", "Total_Cases"), SQLYesterday) as cursor:
 for row in cursor:
 ydaycase = (row[1])
 #print "Yesterday's cumulative case number:" + str(ydaycase)
 #arcpy.AddMessage("Yesterday's cumulative case number:" + str(ydaycase))
with arcpy.da.SearchCursor("Timelinetv", ("New_Cases", "Total_Cases"), SQLToday) as cursor:
 for row in cursor:
 todaycase = (row[1])
 #print "Today's cumulative case number:" + str(todaycase)
 #arcpy.AddMessage("Today's cumulative case number:" + str(todaycase))
 newcase = todaycase - ydaycase
 print "Today's new case number:" + str(newcase)
 #arcpy.AddMessage("Today's new case number:" + str(newcase))
print (todaycase)
 print (ydaycase)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

when I execute this I get a todaycase not defined error..

Can you explain why todaycase isn't in scope?

I tried setting todaycase = 0 before I go into the first for loop but todaycase stays at 0 when I do that.

What do I need to do to be able to assign the value inside the for loop and then access it later in the script?

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

HannesZiegler — Tue, 27 Oct 2020 15:51:28 GMT

Hi,

Could you edit your posts to show the code syntax highlighted? This will preserve indentations of your code.

While editing/posting, click Expand Toolbar (... symbol) >> More dropdown >> Syntax Highlighter, then choose Python and paste the code into the text area directly from your editor.

This will help us read the code, thanks!

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

JoeBorgione — Sat, 11 Dec 2021 14:32:10 GMT

Your search cursors may be clobbering each other. I think I see what you are doing with two different SQL queries, but perhaps you could combine them into 1? That might tidy things up a bit. It would be really helpful to:

Show both of your queries

Use the syntax highlighter as suggested below.

Here is what your code looks like when doing so:

#Set variable for yesterday's cumulative cases
with arcpy.da.SearchCursor("Timelinetv", ("New_Cases", "Total_Cases"), SQLYesterday) as cursor:
    for row in cursor:
        ydaycase = (row[1])
        #print "Yesterday's cumulative case number:" + str(ydaycase)
        #arcpy.AddMessage("Yesterday's cumulative case number:" + str(ydaycase))

with arcpy.da.SearchCursor("Timelinetv", ("New_Cases", "Total_Cases"), SQLToday) as cursor:
    for row in cursor:
        todaycase = (row[1])
        #print "Today's cumulative case number:" + str(todaycase)
        #arcpy.AddMessage("Today's cumulative case number:" + str(todaycase))
        newcase = todaycase - ydaycase
        print("Today's new case number: {}".format(newcase))
        #arcpy.AddMessage("Today's new case number:" + str(newcase))

print (todaycase)
print (ydaycase)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

At least that's what I think it should look like...

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

HannesZiegler — Tue, 27 Oct 2020 16:45:42 GMT

It's not an issue with scope, likely todaycase is never assigned because SQLToday query returns empty (also explains why todaycase remains 0 when you assigned outside of the for loop), try adding some logic to deal with the case where the query does not return anything to iterate over.

Re: Python script executes in pyscripter but fails when trying to run as script in arccatalog.

Anonymous User — Mon, 14 Dec 2020 15:54:23 GMT

I appreciate all the responses..

I just realized I can get the results I'm looking for my removing the search cursor for SQLToday alltogether since since I already have that value in the earlier part of the script (DOHCases) (Line 58).

I just realized there's no need to query for that.