topic Re: inconsistent Arcpy.Mapping behaviour in Python Questions

inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 31 Jul 2014 17:55:56 GMT

I've written a script using Arcpy.mapping module to automate map making. It works as a one-off for use in ArcMap if I use the ArcToolbox to call the script. But if I call it from python (eclipse) I get weird error messages ("ExecuteError: ERROR 000358: Invalid expression") pointing towards the "Select by Layer" tool. But it executes fine if run from ArcToolbox, no errors... same script.

Part 2: I gave up running it from Eclipse and ran it from ArcToolbox within ArcMap. Now I'm trying to run a batch - multiple maps. It used to work until this morning. this morning I made the most minor of changes... to check to see if a variable.upper() == 'YES'

again, this works in the "one-off" scenario within ArcMap called from ArcToolbox. But I'm trying to run my batch script (also called by ArcToolbox) and I get that same query error as above. Why is that? it's the exact same script. I don't want to interactively run this script in ArcMap for all 350+ maps. Why is arcpy.mapping so flaky? Is there a way to fix this???

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 31 Jul 2014 20:00:29 GMT

ps for the one-off my mxd is set to "CURRENT" but for the batch I did link it to the actual mxd file location "C:\blah\blah\awesome.mxd"

The batch worked this morning until I added the "upper()" part.

if you want to see the scripts you can get them at: ftp://ftp.geobc.gov.bc.ca/pub/outgoing/GeoBC_Regional/Coast/tmp_mapRobot/

thx

Re: inconsistent Arcpy.Mapping behaviour

JasonScheirer — Thu, 31 Jul 2014 20:25:23 GMT

Can you post a full traceback? There's a lot that could be going wrong here. arcpy.mapping should work the same in all environments, I suspect something else is at play here.

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 31 Jul 2014 20:39:28 GMT

what is a traceback?

I'm not a real programmer - just a GIS guy thrown into the possibilities of python. A lot of my script had to be fixed by a real python guy but he stopped when he said that the errors that are ESRI based are out of his "scope".

If you mean the error messages it looks like this now:

Executing: batchMap4PMV

Start Time: Thu Jul 31 13:38:22 2014

Running script batchMap4PMV...

checking out inset map placement: None

no inset map

beginning layer filter of data frameLayers

CROWN_LANDS_FILE = '2411100'

Tantalis Files

defined the lyr for Tantalis Files?

ERROR 000358: Invalid expression

Failed to execute (SelectLayerByAttribute).

Traceback (most recent call last):

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 80, in <module>

runMapRobotScript(filenum, outfile)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 42, in runMapRobotScript

obj.dataFrameFilter(obj.mainDF, obj.mainDefQry, inset_map)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\map_robot_2014_PMV_v2_4batch.py", line 161, in dataFrameFilter

arcpy.SelectLayerByAttribute_management(lyr,"NEW_SELECTION", defQry)

File "e:\sw_nt\arcgis\desktop10.1\arcpy\arcpy\management.py", line 6498, in SelectLayerByAttribute

raise e

ExecuteError: ERROR 000358: Invalid expression

Failed to execute (SelectLayerByAttribute).

Failed to execute (batchMap4PMV).

Failed at Thu Jul 31 13:38:29 2014 (Elapsed Time: 7.00 seconds)

Oh and I've tried the quotes around the field names. It works OK without the quotes in the ArcMap interactive world.

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Wed, 06 Aug 2014 17:25:05 GMT

no answers? no similar experiences?

I'll try rebuilding my mxd from scratch... maybe it's another corrupted MXD????

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 16:01:18 GMT

P.S. with the new forum format, where is this post supposed to go?

is this the wrong forum? where do the arcpy.mapping experts hang out?

Re: inconsistent Arcpy.Mapping behaviour

IanMurray — Thu, 07 Aug 2014 16:31:59 GMT

Probably this should go in the python space. Python‌

Timothy Hales‌, can you get this moved?

Xander Bakker‌, you mind taking a look at this code?

Re: inconsistent Arcpy.Mapping behaviour

TimothyHales — Thu, 07 Aug 2014 16:36:30 GMT

I've moved it to Python‌. Thanks for the heads up!

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 18:12:20 GMT

so I haven't touched this for a week. I copied my old mxd and re-ran the program with no changes to the script. Now the error message is different:

Traceback (most recent call last):

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 80, in <module>

runMapRobotScript(filenum, outfile)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 46, in runMapRobotScript

obj.doDimensions(obj.mainDF, tantlyr)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\map_robot_2014_PMV_v2_4batch.py", line 230, in doDimensions

arcpy.mapping.UpdateLayer(dFrame, upLyr, srcLyr, False)

File "e:\sw_nt\arcgis\desktop10.1\arcpy\arcpy\utils.py", line 181, in fn_

return fn(*args, **kw)

File "e:\sw_nt\arcgis\desktop10.1\arcpy\arcpy\mapping.py", line 1869, in UpdateLayer

assert isinstance(update_layer, Layer)

AssertionError

Failed to execute (batchMap4PMV).

I placed a whole bunch of add message statements to see where exactly it was bombing and it's on the UpdateLayer line.

Again, this script runs without error when I'm doing it one at a time in ArcMap via ArcToolbox. But when trying to run as a batch - namely have a script call the original script with a bunch of arguments - it bombs. The error message doesn't make much sense to me and so far every online answer to similar queries point to people sending a text string to the UpdateLayer function instead of a map document object or a layer object. If you check my script (it's still on the FTP site listed above) I'm fairly certain I have sent it map document object and layer object.

Any help much appreciated.

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 18:16:12 GMT

p.s. I've uploaded the most recent version of the batch script and the other one.

minor changes included checking for upper case value for a text variable, changing a UNC path to a drive letter mapping, and adding a bunch of Add Message to see where it's bombing.

search for "check 1" "check 2" etc and you'll see... it doesn't get to check 6.

Re: inconsistent Arcpy.Mapping behaviour

ToddUlery — Thu, 07 Aug 2014 18:42:28 GMT

How come your using this OO formatted python script?

Seems like a few added steps, for what may seem to cut time.

Re: inconsistent Arcpy.Mapping behaviour

ThomasStanley-Jones — Thu, 07 Aug 2014 19:01:22 GMT

I think "ListLayers" returns a list. I'm assuming that you are only expecting one layer called "Dimensions" to be returned. Try changing this:

arcpy.mapping.UpdateLayer(dFrame, upLyr, srcLyr, False)

to this:

arcpy.mapping.UpdateLayer(dFrame, upLyr[0], srcLyr, False)

Re: inconsistent Arcpy.Mapping behaviour

IanMurray — Thu, 07 Aug 2014 19:11:05 GMT

yea, you are missing an 0 index on a listlayers by check 3

arcpy.AddMessage("check 3")

#upLyr = arcpy.mapping.ListLayers(self.mxd, "Dimensions", dFrame)[0]

upLyr = arcpy.mapping.ListLayers(self.mxd, "Dimensions", dFrame)

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 21:57:09 GMT

i've tried both 0 and not 0 (i've taken turns commenting out one or the other one)

same result... bombs before check 6

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 22:05:28 GMT

if I leave the [0] in there I just got an error saying "list index out of range".

But I do have a layer called "Dimensions" in my mxd... I just double checked

I decided to close out ArcMap - re open ArcMap and now the error message has reverted back to the

ERROR 000358: Invalid expression

Failed to execute (SelectLayerByAttribute).

very weird because other than re-activating the line upLyr = arcpy.mapping.ListLayers(self.mxd, "Dimensions", dFrame)[0]

I didn't do anything different. Seems so random.

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 22:10:26 GMT

@ToddUlery

the idea was to utilize python to be able to send a batch of maps - 350+

it included lots of customization like truncating text fields, creating text files where it was truncated, adding text if it meets certain conditions, filtering out a layer, zooming to it, etc. Not all of those functions are readily available in Data Driven Pages yet.

as for OO formatting I'm still learning this stuff. I was told that programming best practices in python (or anything else) was to use modules, classes, etc

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Thu, 07 Aug 2014 22:57:33 GMT

I closed ArcMap again, re opened and now I'm back at the List Index out of range error for the updateLayer

Re: inconsistent Arcpy.Mapping behaviour

XanderBakker — Fri, 08 Aug 2014 02:45:56 GMT

Just had a "short" look at the code, and from the error message I see you're using version 10.1 of ArcGIS. Any service packs installed? At 10.1 you can use lot's of the goodies provided like the data access module, but you're using the old and slow cursors.

Code normally does the same thing each time over and over. If it doesn't then normally something of the environment it runs in (ArcGIS, underlying data, state of elements in your MXD, etc) changed.

Just to respond to the discussion on whether to use quotes around the field name or field values not. Don't think about this to much, since arcpy can do the work for you. As you know the way you have to specify the field with single quotes, square brackets or just without, depends on the data source. Arcpy comes with a tool ("AddFieldDelimiters") that will do the work for you. Have a look at the snippet below:

fc = r"C:\some\path\to\a\folder\or\fgdb.gdb\myFeatureClass"
fld_name = "myField"
a_value = 123456

qdef = "{0} = {1}".format(arcpy.AddFieldDelimiters(fc, fld_name), a_value)

When you want to append text into one string try to avoid this:

"CROWN_LANDS_FILE = '" + str(fileNum) + "'"

... rather use the string format syntax like:

"CROWN_LANDS_FILE = '{0}'".format(fileNum)

This makes it easier to construct complex strings.

If you want to zoom to a feature, you can also use the extent of the feature geometry, as I have explained in this thread:

https://community.esri.com/message/393455#393455‌

Then there's this thing about classes; the fact that you want to run a piece of code 350 times, for me isn't a reason to create a class for it. Sure you can, but the idea of classes (IMHO) is that they should contain reusable generic code and not all kinds of specifics as is the case in your code. It makes it very hard to read your code.

Kind regards, Xander

Re: inconsistent Arcpy.Mapping behaviour

JoshuaChan — Fri, 08 Aug 2014 15:21:47 GMT

I asked our server administrator and here's what patches we have:

ArcGIS 10.1 for Desktop
ArcGIS 10.1 SP1 for Desktop
ArcGIS 10.1 SP1 for Desktop Set Bing Key Patch
ArcGIS 10.1 SP1 for Desktop Quality Improvement Patch

I'm not sure how this affects this strange bug where my program works one at a time but not in a batch. I've changed the query format as you suggested.

So after using the string format syntax example, instead of the update layer error (as per yesterday) it is giving me the older query format error again:

Executing: batchMap4PMV

Start Time: Fri Aug 08 08:04:35 2014

Running script batchMap4PMV...

checking out inset map placement: None

no inset map

beginning layer filter of data frameLayers

CROWN_LANDS_FILE = '2411100'

Tantalis Files

defined the lyr for Tantalis Files?

ERROR 000358: Invalid expression

Failed to execute (SelectLayerByAttribute).

Traceback (most recent call last):

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 80, in <module>

runMapRobotScript(filenum, outfile)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\batchMapRobot.py", line 42, in runMapRobotScript

obj.dataFrameFilter(obj.mainDF, obj.mainDefQry, inset_map)

File "W:\srm\sry\Local\projlib\2014-15\Auth_Lands_PMV\data\scripts\map_robot_2014_PMV_v2_4batch.py", line 163, in dataFrameFilter

arcpy.SelectLayerByAttribute_management(lyr,"NEW_SELECTION", defQry)

File "e:\sw_nt\arcgis\desktop10.1\arcpy\arcpy\management.py", line 6498, in SelectLayerByAttribute

raise e

ExecuteError: ERROR 000358: Invalid expression

Failed to execute (SelectLayerByAttribute).

Failed to execute (batchMap4PMV).

Failed at Fri Aug 08 08:04:43 2014 (Elapsed Time: 8.00 seconds)

Re: inconsistent Arcpy.Mapping behaviour

XanderBakker — Fri, 08 Aug 2014 16:15:56 GMT

I suppose that if you manually apply the definition query CROWN_LANDS_FILE = '2411100' to the layer, it works without any problems, right? If so, then the definition query is not the problem. It might be the layer itself or other aspects of the environment you're working in. Those are hard to solve without access to the data and mxd you're using.

I would probably step away from using of classes and rewrite the code using the da cursors and set the extent using the geometry rather than selecting and zooming to the extent of the selection.