topic Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool in Python Questions

Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

cmwrenn3 — Thu, 04 Feb 2021 18:48:06 GMT

Hey, everyone.

Has anyone installing Pro 2.7.1 (Python 3.7.9) run into an issue using scripts written in Python 3.6.10 (Pro 2.6.3), specifically Geoprocessing Tools?

We have a script, written in 3.6.10 that calculates three fields, and when run from a machine with Pro 2.6.3, the calculations are correct. When run from a machine with 3.7.9/Pro 2.7.1, one of the fields calculates correctly (current date & time), while the others don't (copy the number of COVID cases from a joined table; divide the number of cases by population, then multiply by 100,000). The first one places a Null into the COVID cases field which causes the second to become Null.

Thanks,

Charlie

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

DanPatterson — Thu, 04 Feb 2021 21:35:28 GMT

The script would help. There is nothing in python that would have caused and error, so that leaves something in arcpy or calls to tools that would have changed.

If it is the Calculate Field (Data Management)—ArcGIS Pro | Documentation

tool that you are using, you can see if there are syntax changes between Pro 2.6 and 2.7 in the toggle beneath the title

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

cmwrenn3 — Sat, 06 Feb 2021 16:23:42 GMT

Here's the script (slightly reformatted for display here.

I previously noticed the syntax changes between 2.6.3 and 2.7 (though Esri's sample scripts are the same for both versions), and have tried both with the same results. Again, the first one fails with a null result; the second may or may not be failing (since the first provides a null value that the second references), and the third (date) works on both python versions.

# Update COVID19 Zip Code data def email_results(results): """Email the completion results.""" msg = EmailMessage() msg.set_content(results) msg['Subject'] = 'COVID19 ZipCode Update Completed' msg['From'] = 'cmwrenn@co.pg.md.us' msg['To'] = 'cmwrenn@co.pg.md.us' server = smtplib.SMTP('smtp.co.pg.md.us', 25) server.starttls() server.send_message(msg) server.quit() return import arcpy from arcpy import mp import smtplib from email.message import EmailMessage aprx = mp.ArcGISProject(r'\\ ... \ZipCodeRate.aprx') # Production #aprx = mp.ArcGISProject(r'C:\Users\ ... \ZipCodeRate.aprx') # Testing m = aprx.listMaps('Layers')[0] zipcovid = m.listLayers('*COVID_STATEZIP')[0] zipmd = m.listLayers('ZIPCodes_MD')[0] #arcpy.AddMessage('Calculating CASES') print('Calculating CASES') ## Flow Trace # Pro 2.6.3/Py 3.6.10 arcpy.CalculateField_management(zipcovid, 'CASES', '!L0ZIPCODES_MD.ProtectedCount!') # Pro 2.7.1/Py 3.7.9 #arcpy.management.CalculateField(zipcovid, # 'CASES', # '!L0ZIPCODES_MD.ProtectedCount!') #arcpy.AddMessage('Calculating RATE') print('Calculating RATE') ## Flow Trace # Pro 2.6.3/Py 3.6.10 arcpy.CalculateField_management(zipcovid, 'RATE', '!CASES! / !POP100! * 100000') # Pro 2.7.1/Py 3.7.9 #arcpy.management.CalculateField(zipcovid, # 'RATE', # '!CASES! / !POP100! * 100000') print('Calculating RIMPRT_DATE') ## Flow Trace # Pro 2.6.3/Py 3.6.10 arcpy.CalculateField_management(zipcovid, 'IMPRT_DATE', 'time.strftime("%m/%d/%Y")') # Pro 2.7.1/Py 3.7.9 #arcpy.management.CalculateField(zipcovid, # 'IMPRT_DATE', # 'time.strftime("%m/%d/%Y")') print('Saving project file') ## Flow Trace aprx.save() results = 'The ZipCodes have been updated with the current Cases and Population per 100,000 cases.' email_results(results) print('Completed') ## Flow Trace

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

Anonymous User — Sat, 06 Feb 2021 17:44:23 GMT

Could you reference the layer in its location without having to open the aprx and step through the project elements?

Maybe it is the ArcGISProject module is causing the issue. (There are a few other topics posted recently that are using the ArcGISProject module and they too seem to be having behavior issues)

If you update the underlying featureclass/shapefile, it should reflect those changes in the project's map when its opened and loaded.

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

cmwrenn3 — Mon, 08 Feb 2021 11:59:45 GMT

Thanks, I'll see what I can do - I'm not sure how to access a map service via arcpy, but I'll try to find out.

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

Anonymous User — Mon, 08 Feb 2021 14:23:31 GMT

Knowing that you are doing this on a hosted layer now, I would look at using the arcgis for python api as a possible alternative. The fact it worked in 3.6, this might be overkill if it is something that could get fixed in the next patch. But, for example as an alternative: (This could very well not work right out of the box- I just pulled what I think would be the process from the editing features part of the API docs)

import arcpy from arcgis.gis import GIS import smtplib from email.message import EmailMessage gis = GIS(username="someuser", password="secret1234") # Update COVID19 Zip Code data def email_results(results): """Email the completion results.""" msg = EmailMessage() msg.set_content(results) msg['Subject'] = 'COVID19 ZipCode Update Completed' msg['From'] = 'cmwrenn@co.pg.md.us' msg['To'] = 'cmwrenn@co.pg.md.us' server = smtplib.SMTP('smtp.co.pg.md.us', 25) server.starttls() server.send_message(msg) server.quit() def getLayersFromAGOL(): search_result = gis.content.search('title:COVID_STATEZIP') #? covidzip_item = search_result[0] covidCnt_layer = covidzip_item.layers covidCnt_fset = covidCnt_layer[0].query() covidZip_features = covidCnt_fset.features return covidZip_features def getCounts(): search_res = gis.content.search('title:ZIPCodes_MD') #? covidTot_item = search_res[0] covidTot_layer = covidTot_item.layers covidTot_fset = covidTot_layer[0].query() covidTotCnt_features = covidTot_fset.features return [f for f in covidTotCnt_features if f.attributes['ProtectedCount'][0]] def updateZipcodeCounts(protectCnts, zipCodeCntLyr): return arcpy.CalculateFields_management(zipCodeCntLyr, "PYTHON3", [ ["Cases", protectCnts], ['RATE', '!CASES! / !POP100! * 100000'], ['IMPRT_DATE', 'time.strftime("%m/%d/%Y")']]) if __name__ == '__main__': covidCntsLyr = getLayersFromAGOL() updtedLyr = updateZipcodeCounts(getCounts(), covidCntsLyr) covidCntsLyr.edit_features(updates=[updtedLyr]) email_results('The ZipCodes have been updated with the current Cases and Population per 100,000 cases.') print('Completed') ## Flow Trace

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

cmwrenn3 — Mon, 08 Feb 2021 14:47:52 GMT

I think, for now, we can run the script (via a DOS/Windows batch file) from a machine that has 2.6.3. I'll contact Esri service in the mean time.

I will try the AGOL API angle, but with the interactive authentication (Auth2?); I'm not comfortable placing passwords as plain text inside a script.

Thanks.

Re: Change from ArcGIS Pro 2.6.3 to 2.7.1 causes error in arcpy GP Tool

Anonymous User — Mon, 08 Feb 2021 15:29:48 GMT

That is understandable and going the API route with hiding the sensitive info will certainly add some code as a solution. Hopefully tech support can help.

To get around storing the username/password as text in scripts, one method we use is keeping them in a separate file that is located elsewhere on the server and is retrieved using a script that encrypts and decrypts the values. Not 100% secure procedure, but it keeps the sensitive info from being in plain view within the script. I can send you the encrypt/decrypt script if interested.