topic Re: Field Calculator '*.cal' file encoding in ArcGIS Pro Questions

Field Calculator '.cal' file encoding ... not '.py'

DanPatterson_Retired — Wed, 21 Jun 2017 12:01:04 GMT

The ability to save and reuse python snippets for using in the Calculate Field tools is useful (aka... the field calculator for non-ArcGIS Pro users). A couple of questions then...

UTF-16-LE .... the encoding used for some reason rather than utf-8 for instance
UC2-2 LE BOM ... is this good enough if you use a Notepad++ file?
-*- coding: <encoding-name> -*- as the first line using either the UTF or UC2 version... will that solve encoding problems with the *.cal files?

And the biggie....Why can't you use Python *.py files?

It is more out of curiosity and... I have to teach... so this is what I have to tell my students...

the default language used in Pro is Python...
Python files have a *.py file extension ...
except in the calculate field tool for Pro,
*.cal files is all you will see,
save your 'defs' as cal and py files if you want to use them everywhere
no more questions now... I have a headache

Some illumination would be useful so I can expand my bullet list. Others that teach out there, may wish to jump in with addition missives should they have encountered any

Curtis Price‌ Timothy Hales‌ perhaps, or an FYI##

Re: Field Calculator 'cal' encoding

JoshuaBixby — Wed, 21 Jun 2017 14:19:23 GMT

Great question Dan. I would like, or even hope, someone knowledgeable from Esri responds but we will have to wait and see.

I foresee a "compatibility" argument being made, which will be frustrating because I think Esri talks out both side of their mouth when it comes to compatibility between ArcMap and ArcGIS Pro. I ran into an issue the other day where the parameter order of an ArcPy function changed between ArcMap and Pro, and the argument was made that Pro is a new application not everyone will work example the same. Yet, somehow the CAL file survived.

Re: Field Calculator 'cal' encoding

DanPatterson_Retired — Wed, 21 Jun 2017 14:56:20 GMT

Thanks Joshua... as for the parameter out of order issue... while we wait... I use the named parameter method generally rather than relying on position of the parameter... makes the code a bit longer to write the first time, but is manageable should someone pull a fast shifty on us.

Lets pull in some of the others... to add to the list since they might have an insider perspective or could raise this issue during the plenary session coming up at UC

xander_bakker‌ vangelo-esristaff‌ jayanta.poddar‌

Re: Field Calculator 'cal' encoding

curtvprice — Wed, 21 Jun 2017 21:51:43 GMT

the parameter order of an ArcPy function changed

Please do tell which function! I have found this pretty unusual - I have a pretty complex toolbox written in 9.3 that still works.

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Wed, 21 Jun 2017 23:45:03 GMT

quit getting off track .... focus on the main question and we might tell you

EDIT

PS... I have a short list of 10+ in the evolution of Pro since 1. to 1.4.1... and that doesn't include enhancements but additions/changes to the tool parameters. I have counted groups of changes as 1 since there a bunch of changes in a category of tools that affected all tools within it.

But I digress from the pressing issue

Re: Field Calculator '*.cal' file encoding

ModyBuchbinder — Thu, 22 Jun 2017 05:03:34 GMT

Hi Dan

This is a good question and I had these problems too.

My workaround is to replace the calc tool with simple python script that run an update cursor. Then I can run a "real" python script. The update cursor itself is just a few lines around the calc expression.

Mody

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Thu, 22 Jun 2017 05:10:34 GMT

Thanks for that Mody, but I am using the simple field calculator defs as an introduction to using full-fledged scripts in my teaching... so chicken/egg scenario unfortunately.

Re: Field Calculator '*.cal' file encoding

XanderBakker — Thu, 22 Jun 2017 12:49:09 GMT

I wonder if using the Field Calculator is the easiest way to explain Python. It does allow you create a simple function and if you are lucky you may write it without errors. But I have seen cases that the function was simple and valid but still didn't work in the Field Calculator. When you store the .cal file you get this beautiful "__esri_field_calculator_splitter__" to separate the function from the expression, but it will not tell you the Parser used for the cal file. Since in some cases I have mixed using cal files with VBA and Python parsers just for being lazy, this can be the cause of problems. In the long run, I believe creating a standalone script is better than using the .cal files. It makes more sense to me.

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Thu, 22 Jun 2017 14:00:19 GMT

Thanks Xander, my experience in class has been the opposite. You can create some really simple field calculator expressions as defs then save them out... and as you point out the splitter does a fine job for reuse. I use those as a base to introduce the Calculate Field from a standalone script, then searchcursors. I should note... most of the 2nd year class doesn't have any experience in programming, coming from a social science background.

The question remains... why the restriction to a *.cal

Re: Field Calculator '*.cal' file encoding

VinceAngelo — Thu, 22 Jun 2017 14:53:50 GMT

It's my understanding that there are two ways to invoke Python; for lack of better terms, I'll call them "script" and "immediate". The field calculator submits a string to the parser in "immediate" mode, but because it's not a file, you get more generic errors (and sometimes it's just "error occurred").

Python toolboxes have a .pyt suffix, so .cal isn't the only quirky suffix for Python-like code.

I expect the UTF-16 encoding is due to the default character width in the Windows app, and avoiding the need to translate to UTF-8 before submitting to the parser.

I've been programming since I was 12, so I'll stay out of the debate on how to teach non-programmers scripting languages.

- V

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Thu, 22 Jun 2017 15:02:57 GMT

'68 for me, with cards... thanks Vince

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Tue, 27 Jun 2017 05:42:17 GMT

Fresh ones arrive... with 10.5.1 more parameters in slope and aspect check geoprocessing section and Pro 2.0 is live

Re: Field Calculator '*.cal' file encoding

curtvprice — Tue, 27 Jun 2017 12:26:39 GMT

The help for 10.5.1 and Pro 2.0 is up, but no software yet for me on my.esri. Probably very soon though.

This is cool: GPU processing with Spatial Analyst—Help | ArcGIS Desktop

Re: Field Calculator '*.cal' file encoding

DanPatterson_Retired — Tue, 04 Jul 2017 23:21:02 GMT

Wow... 29 viewers, hi to the esri staff that I haven't heard of. 325 emails? must have generated some interest.

BUT

I will close this off as a discussion since it appears that the status quo will remain for some reason (legacy purposes?).

Workarounds abound ... I don't need them, but thank you. I am more interested in finding a suitable explanations for my students without using

'.... because, that's the way it is... '

in an explanation of why python is widely adopted, except when saving in the field calculator.

Anyone at UC... ask around...

Dan

Re: Field Calculator '.cal' file encoding ... not '.py'

ScottDrzyzga — Thu, 25 Jan 2018 16:07:46 GMT

Thanks for posting this, for you helped me answer an unrelated question. I had a student that used a text editor to edit a *.cal file, but on [Load], all the characters had "changed" to Chinese. We kept trying to fix it by saving to UTF-8 or ANSI, but ... UC2-2 LE BOM. Thanks.

While I'm here, has the big question been answered yet? Not here, obviously, but elsewhere?

"And the biggie....Why can't you use Python *.py files?"

Re: Field Calculator '.cal' file encoding ... not '.py'

DanPatterson_Retired — Thu, 25 Jan 2018 18:22:20 GMT

It hasn't.... and there is probably some reason beyond 'inertia', that I am not privy to.

In any event, I just keep two exact same versions, or if I just forget, I change the file extension and comment out the last 2 lines that the *cal version magically adds

__esri_field_calculator_splitter__
angle_between(!Shape!, True)

Those two lines are what populates the Expression box so that the *cal does all the work for you.

topic Re: Field Calculator '*.cal' file encoding in ArcGIS Pro Questions

Field Calculator '*.cal' file encoding ... not '*.py'

Re: Field Calculator 'cal' encoding

Re: Field Calculator 'cal' encoding

Re: Field Calculator 'cal' encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding

Re: Field Calculator '*.cal' file encoding ... not '*.py'

Re: Field Calculator '*.cal' file encoding ... not '*.py'

Field Calculator '.cal' file encoding ... not '.py'

Re: Field Calculator '.cal' file encoding ... not '.py'

Re: Field Calculator '.cal' file encoding ... not '.py'