Field Calculator '*.cal' file encoding ... not '*.py'

[DanPatterson_Retired]

The ability to save and reuse python snippets for using in the Calculate Field tools is useful (aka... the field calculator for non-ArcGIS Pro users).  A couple of questions then...

• UTF-16-LE .... the encoding used for some reason rather than utf-8 for instance

• UC2-2 LE BOM ... is this good enough if you use a Notepad++ file?
• -*- coding: <encoding-name> -*-   as the first line using either the UTF or UC2 version... will that solve encoding problems with the *.cal files?

And the biggie....Why can't you use Python *.py files?

It is more out of curiosity and... I have to teach... so this is what I have to tell my students...

• the default language used in Pro is Python...
• Python files have a *.py file extension ...
• except in the calculate field tool for Pro, 
• *.cal files is all you will see,
• save your 'defs' as cal and py files if you want to use them everywhere
• no more questions now... I have a headache

Some illumination would be useful so I can expand my bullet list.  Others that teach out there, may wish to jump in with addition missives should they have encountered any

Curtis Price‌ Timothy Hales‌ perhaps, or an FYI##

[JoshuaBixby]

Great question Dan.  I would like, or even hope, someone knowledgeable from Esri responds but we will have to wait and see.

I foresee a "compatibility" argument being made, which will be frustrating because I think Esri talks out both side of their mouth when it comes to compatibility between ArcMap and ArcGIS Pro.  I ran into an issue the other day where the parameter order of an ArcPy function changed between ArcMap and Pro, and the argument was made that Pro is a new application not everyone will work example the same.  Yet, somehow the CAL file survived.

[DanPatterson_Retired]

Thanks Joshua... as for the parameter out of order issue... while we wait... I use the named parameter method generally rather than relying on position of the parameter... makes the code a bit longer to write the first time, but is manageable should someone pull a fast shifty on us.

Lets pull in some of the others... to add to the list since  they might have an insider perspective or could raise this issue during the plenary session coming up at UC  

xander_bakker‌ vangelo-esristaff‌ jayanta.poddar‌

[curtvprice]

the parameter order of an ArcPy function changed

Please do tell which function!  I have found this pretty unusual - I have a pretty complex toolbox written in 9.3 that still works.

[DanPatterson_Retired]

quit getting off track .... focus on the main question and we might tell you 

EDIT

PS... I have a short list of 10+ in the evolution of Pro since 1. to 1.4.1... and that doesn't include enhancements but additions/changes to the tool parameters.  I have counted groups of changes as 1 since there  a bunch of changes in a category of tools that affected all tools within it.

But I digress from the pressing issue

[ModyBuchbinder]

Hi Dan

This is a good question and I had these problems too.

My workaround is to replace the calc tool with simple python script that run an update cursor. Then I can run a "real" python script. The update cursor itself is just a few lines around the calc expression.

Mody

[DanPatterson_Retired]

Thanks for that Mody, but I am using the simple field calculator defs as an introduction to using full-fledged scripts in my teaching... so chicken/egg scenario unfortunately.

[XanderBakker]

I wonder if using the Field Calculator is the easiest way to explain Python. It does allow you create a simple function and if you are lucky you may write it without errors. But I have seen cases that the function was simple and valid but still didn't work in the Field Calculator. When you store the .cal file you get this beautiful "__esri_field_calculator_splitter__" to separate the function from the expression, but it will not tell you the Parser used for the cal file. Since in some cases I have mixed using cal files with VBA and Python parsers just for being lazy, this can be the cause of problems. In the long run, I believe creating a standalone script is better than using the .cal files. It makes more sense to me. 

[DanPatterson_Retired]

Thanks Xander, my experience in class has been the opposite.  You can create some really simple field calculator expressions as defs then save them out... and as you point out the splitter does a fine job for reuse.  I use those as a base to introduce the Calculate Field from a standalone script, then searchcursors.  I should note... most of the 2nd year class doesn't have any experience in programming, coming from a social science background.

The question remains... why the restriction to a *.cal

[VinceAngelo]

It's my understanding that there are two ways to invoke Python; for lack of better terms, I'll call them "script" and "immediate". The field calculator submits a string to the parser in "immediate" mode, but because it's not a file, you get more generic errors (and sometimes it's just "error occurred"). 

Python toolboxes have a .pyt suffix, so .cal isn't the only quirky suffix for Python-like code.

I expect the UTF-16 encoding is due to the default character width in the Windows app, and avoiding the need to translate to UTF-8 before submitting to the parser.

I've been programming since I was 12, so I'll stay out of the debate on how to teach non-programmers scripting languages.

- V