topic Re: ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS in Python Snippets Questions

ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS

ChrisHarrison7 — Thu, 16 Apr 2020 20:52:01 GMT

To whom it may concern

A recent update for ArcGIS Pro 2.5.0 and ArcPY will cause serious issues to features and shapefiles.

While testing some code I wrote between myself and our GIS analyst, I found something very odd occurring with the following sample code snipet.

arcpy.management.CalculateField(currentFeature, "DesiredFieldName", "'New Text'", "PYTHON3", '')

In ArcGIS 2.4.2 this code snippet fails if "DesiredFieldName" does not exist in the table of currentFeature. This is normal and should occur ensuring that currentFeature is indeed properly formated and that "DesiredFieldName" is indeed within the feature.

However, in ArcGIS Pro 2.5.0 this code adds the field "DesiredFieldName" if it does not exist and fills it with "'New Text'"... this is very very VERY dangerous.

I would much rather code fail to require a coding change than have code "succeed" by damaging my features / shapefiles if I run the following snipet:

arcpy.management.CalculateField(currentFeature, "DesiredFieldName_SPELLED_WRONG", "'New Text'", "PYTHON3", '')

Note: I am assuming this is a ArcGIS Pro version issue as this was the only difference between myself and our GIS analyst.

Re: ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS

KoryKramer — Thu, 16 Apr 2020 21:50:47 GMT

You're right that this changed at 2.5.

Documented here: Data Management toolbox history—Data Management toolbox | Documentation

The details are explained in the tool's help.

Calculate Field—Data Management toolbox | Documentation

I don't really work with Python much, but knowing how the tool now works with the capability to add a new field if it doesn't exist and specify the field_type, would you be able to adjust the script to check for existing fields and have the calculate field line only accept an existing field?

Re: ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS

ChrisHarrison7 — Thu, 16 Apr 2020 22:47:08 GMT

Hey Kory!

Thanks for replying!

Can I reprogram such that the code checks for the existence of the field prior to calculating?

Yes. I can go through my thousands of lines of code to fix. Tis the nature of coding.

I may even come to love a geoprocessing tool that says one thing in its name, "CalculateField" but is actually "CalculateField_or_AddFieldIfItDoesNotExist"

Perhaps ESRI is trying to phase out "AddField"?

But as it stands, this is just plain dangerous.

In testing (or even processing) I could mistakenly point at feature B instead of A, and add a series of fields to my feature without knowing it.

At best, ESRI is forcing testers to go back and delete the fields. A typical day for us coders.

At worst, mistakes are compounding and going unnoticed in a geodatabase based on "CalculateField" actually being "CalculateField_or_AddFieldIfItDoesNotExist"

Re: ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS

JoeBorgione — Sun, 12 Dec 2021 09:14:20 GMT

I don't see it as dangerous, per se; the worst you end up with is a new, misspelled field, right? Drop that field and re-run the calculation. If you goof up your spelling in a stand alone script and then continue to work on the correctly spelled field, thinking it's been updated, then yes, that's not good thing.

Perhaps you could elaborate on what you stand to lose or corrupt; there may be a work around for you. One thing I like to do for field names in a stand alone script is to first do this is in the console:

for field in arcpy.ListFields(your_data):
    print(field.name)‍‍

Then it's a copy and paste of the field name(s) you want to work with....

Re: ArcGIS Pro 2.5.0 - ArcPy issue - VERY DANGEROUS

ChrisHarrison7 — Fri, 17 Apr 2020 01:52:02 GMT

Hey Joe!

Thanks for replying.

"Drop the field and re-run", yes, write checking code, yes.

I can even see the value in '"CalculateField_or_AddFieldIfItDoesNotExist"

But here is the timeline of events for my use of the geoprocessing tool that is called "CalculateField"

I have consistent fields through multiple features in multiple projects. One field is called "SRCSTATUS", a text field.

I have more or less hardcoded it as it is consistently added to multiple features and used regularly to pass information from office to field and back.

I upgraded to ArcGIS Pro. 2.5.0.

I have been writing code to go from CSV to GDB to Online.

To save time I created a dummy CSV that I forgot to put in "SRCSTATUS" as a field.

Because "CalculateField" in 2.5.0 is actually "CalculateField_or_AddFieldIfItDoesNotExist", I experienced no issues. In fact, the update to the tool FIXED my data. This happened twice in my code for the field "SRCSTATUS" and another field "MultiTag" (a short field).

However! I gave the code to my analyst to test, and it promptly failed at "CalculateField" on the test data. She lacks experience as to what to do and would have been stuck for hours had I not been online at the time. It took me a head scratch and a check of ArcGIS Pro versions (mine 2.5.0 hers 2.4.1) to realize something had changed.

I see a point where automated code will run and instead of errors stopping at mistakes (as it should), the mistakes keep going and going and going. I could perpetuate a misspelling or "mis-fielding" on ten's to hundreds to thousands of features.

Wait till I tell you what the update to "CaclulateField" did to a JOIN with a wrong field once I can understand what the heck went wrong there.