topic Re: Still having problems stepping through a list inside of a tool in Python Questions

Still having problems stepping through a list inside of a tool

JohnNunnali — Thu, 05 Apr 2012 19:14:35 GMT

I had started a thread about a month ago on this same topic but want to start again at the beginning.

I am attempting to iterate through a list inside of the SelectLayerByAttributes tool in ArcPy. My goal is to be able to select attributes from a field (column), export the current map view with the selected attributes as a jpeg, clear the selection, and then start the process again for a different field. I am currently trying to run all three steps within a loop. My issue is that I can not figure out how to make the SelectLayerByAttributes tool step to the next field name after the first completes. Currently the loop runs but just keeps selecting the values from the initial field and not changing to the next in the list. I have attached an example of how the data is setup.

I am trying to select for values greater than zero for each column, then export the results, clear the selection, and them move to next column.

The code currently looks like this:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest"
import arcpy.sa
#generate a list
speciesList =("Achillea_millefolium", "Acroptilon_repens", "Actinea_osterhoutii")

#Create Loop and List to run processing for
for species in speciesList:

  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", "'Achillea_milefolium' > '0'")

  #Export selected to jpeg
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\Achillea_milefolium.jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

Where I need help is in the where clause of the arcpy.SelectLayerByAttribute_management function and the arcpy.mapping.ExportToJPEG function.

Any suggestions are greatly appreciated.

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Sat, 11 Dec 2021 14:04:44 GMT

You need to feed in the variable (species) value.

for species in speciesList:

  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", " ' " + species + " ' > '0'")

and...

arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\" + species + ".jpg")

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:04:47 GMT

Looks like that may be a step in the right direction. However when I run the script with the recommended changes it is now selecting all features and also giving me the following error:

Runtime error <type 'exceptions.AttributeError'>: PageLayoutObject: Error in executing ExportToJPEG

With the suggested changes here is how the code looks:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest"
import arcpy.sa
#generate a list
speciesList =("Achillea_millefolium", "Acroptilon_repens", "Actinea_osterhoutii")

#Create Loop and List to run processing for
for species in speciesList:

  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", " ' " + species + " ' > '0'")

  #Export selected to jpeg
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\" + species " + ".jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

I added another double quote to the ExportToJPEG function because I received a parsing error when testing the script.

Any thought on why it is selecting all features and not just the desired features?

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Sat, 11 Dec 2021 14:04:49 GMT

Numbers don't get quotes in SQL. And we had the quotes wrong anyways: field name gets double quotes, the whole thing is in single quotes.

arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", ' " ' + species + ' " > 0')

I'm pretty sure this is right (you have to have an even number of matching quotes):

arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\" + species + ".jpg")

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:04:52 GMT

This time I received the following error:

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute).

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest"
import arcpy.sa
#generate a list
speciesList =("Achillea_millefolium", "Acroptilon_repens", "Actinea_osterhoutii")

#Create Loop and List to run processing for
for species in speciesList:

  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", ' " ' + species + ' " > 0')

  #Export selected to jpeg
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\' + species + '.jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

Should I be using

arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", ' " ' + speciesList + ' " > 0')

instead?

Never mind. Can't concatenate str and list or str and tuple objects.

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Thu, 05 Apr 2012 20:30:47 GMT

What kind of file are you trying to select from?

Your workspace is set to: "P:\CoWetlandTools\maps\distribution_maps_Johns_pytest"
The file is: "counties_edited_datamergetest"
...which amounts to a file called "P:\CoWetlandTools\maps\distribution_maps_Johns_pytest\counties_edited_datamergetest"
...which is not any sort of feature class I know of.

If it's inside a geodatabase, that should be in your workspace (e.g. "P:\CoWetlandTools\maps\distribution_maps_Johns_pytest.gdb" if that's what it's called).
If it's a shapefile, that should be in the file name (e.g. "counties_edited_datamergetest.shp").

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Thu, 05 Apr 2012 20:43:29 GMT

I'm selecting from a shapefile in an mxd.

I tried adding .shp to the file name but get the following runtime error which I did not encounter before.

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000732: Layer Name or Table View: Dataset counties_edited_datamergetest.shp does not exist or is not supported

I think the function is written to locate a layer name or table view as it appears in the environment that you set as the workspace. When testing the process for a single species I did not encounter any problems with data paths.

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Thu, 05 Apr 2012 20:53:52 GMT

Are the fields actually numeric (integer, float, double) or text? You can check in the field properties. They should be a numeric data type.

Also, try putting "print species" (no quotes) as the first line in your loop just to make sure it's bringing in the right name.

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:04:55 GMT

The field types are "double".

I added print species as the first line of the loop and it returned

Achillea_millefolim

so it appears it is bringing them in correctly. I am still getting the following error message:

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute).

My current code looks like this:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest.mxd"
import arcpy.sa
#generate a list
speciesList =("Achillea_millefolium", "Acroptilon_repens", "Actinea_osterhoutii")
species = ' ' in speciesList

#Create Loop and List to run processing for
for species in speciesList:
  print species   
  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", ' " ' + species + ' " > 0')

  #Export selected to jpeg
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\' + speciesList + '.jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Sat, 11 Dec 2021 14:04:57 GMT

I'm almost positive your workspace can't be an mxd.

Also, this line needs to have matching quotes, and just "species", like so:

arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\" + species + ".jpg")

Re: Still having problems stepping through a list inside of a tool

KevinHibma — Sat, 11 Dec 2021 14:05:00 GMT

Try this for your select statement:

arcpy.SelectLayerByAttribute_management("counties_edited_datamergetest", "NEW_SELECTION",  "'" + str(species) + "' > 0  "  )

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Thu, 05 Apr 2012 21:38:21 GMT

Hmmmm.....I was able to make it work for a single species. I need to have the workspace as an mxd because I am exporting the "current" view with the attributes selected to a jpeg.

Any suggestions?

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:05:03 GMT

@Kevin

I tried your code line and still received the following error:

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute).

This is my current code:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest.mxd"
import arcpy.sa
#generate a list
speciesList =("Achillea_millefolium", "Acroptilon_repens", "Actinea_osterhoutii")


#Create Loop and List to run processing for
for species in speciesList:
  print species   
  #select counties each species is known to occur  
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION",  '"' + str(species) + '"  > 0')

  #Export selected to jpeg
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\"'" + str(species) + "'.jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

Re: Still having problems stepping through a list inside of a tool

BruceNielsen — Fri, 06 Apr 2012 15:14:39 GMT

Try using this:

arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION", "\"%s\" > 0" % species)

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:05:06 GMT

@ Bruce

I tried your suggestion but received the same error I received with the other formats:

Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute).

I have re-worked my code some and I am now using arcpy.ListFields to define the fields I want to make the selections in. I just can't seem to figure out the reason why the SQL expression in the SelectLayerByAttribute function is invalid. I know that the field list is generating properly because when testing I was able to successfully print the desired field names.

Any thoughts or suggestion are much appreciated.

Here is my current code:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
arcpy.env.workspace = r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest.mxd"
import arcpy.sa


#list object that is the list of fields in our table
#List fields and generate a list

fields = arcpy.ListFields(r"Y:\John_N\Wetland_field_guide_distribution_maps\counties_edited_datamergetest_export.shp", None , "Double")


#select fieldname1 > 0

#Create Loop and List to run processing for
for field in fields:
  
  #select counties each field has attribute values > 0
  #select field1 > 0
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "NEW_SELECTION",  "'" + str(field) + "'" ' > 0')

  # export current to jpg such that the name of the file is field1
  mxd = arcpy.mapping.MapDocument(r"CURRENT")
  
  arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\"'" + str(field) + "'.jpg")

  #clear selected attributes
  arcpy.SelectLayerByAttribute_management ("counties_edited_datamergetest", "CLEAR_SELECTION")

#finished
print "Finished"

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Wed, 11 Apr 2012 15:44:59 GMT

I have made a little progress. I used pythonWin to debug the program and finally have more info on why the script is failing at the SelectLayerByAttribute step. The error is as follows:

Traceback (most recent call last):
File "C:\ESRI\Python26\ArcGIS10.0\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 309, in RunScript
    debugger.run(codeObject, __main__.__dict__, start_stepping=0)
File "C:\ESRI\Python26\ArcGIS10.0\Lib\site-packages\pythonwin\pywin\debugger\__init__.py", line 60, in run
    _GetCurrentDebugger().run(cmd, globals,locals, start_stepping)
File "C:\ESRI\Python26\ArcGIS10.0\Lib\site-packages\pythonwin\pywin\debugger\debugger.py", line 624, in run
    exec cmd in globals, locals
File "Y:\John_N\From_desktop\python_training_scripts\dist_map_test1.py", line 22, in <module>
    arcpy.SelectLayerByAttribute_management (r"Y:\John_N\Wetland_field_guide_distribution_maps\counties_datamergetest_export2.lyr", "NEW_SELECTION", '"' + str(field) + '" ' > 0)
File "C:\ESRI\ArcGIS\Desktop10.0\arcpy\arcpy\management.py", line 4259, in SelectLayerByAttribute
    raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000825: The value is not a layer or table view
ERROR 000840: The value is not a Raster Layer.
ERROR 000840: The value is not a Mosaic Layer.
Failed to execute (SelectLayerByAttribute).

I saved the shapefile I was using as a layer file and tried to use that path but receive the same error. Am I missing something or incorrectly saving as a layer file?

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Wed, 11 Apr 2012 15:53:03 GMT

Your where statement must be entirely enclosed in single quotes. Your field name must be enclosed in double quotes.

' " ' + str(field) + ' " > 0 '

Also, reference the layer in your map ("counties_edited_datamergetest"), not the .lyr file.

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Sat, 11 Dec 2021 14:05:09 GMT

OK, this problem was frustrating me so I actually tried it. Here's a copy that works for my mxd named "junk" and a file named "file". Change as appropriate.

import arcpy
mxd = arcpy.mapping.MapDocument(r"H:\GIS_Data\junk.mxd")
fields = arcpy.ListFields(r"H:\GIS_Data\TEMP.gdb\file", None , "SmallInteger")
for field in fields:
    for lyr in arcpy.mapping.ListLayers(mxd, "file", arcpy.mapping.ListDataFrames(mxd)[0]):
        arcpy.SelectLayerByAttribute_management (lyr, "NEW_SELECTION",  '"' + field.name + '" > 0')
        arcpy.mapping.ExportToJPEG(mxd, r"H:\GIS_Data\map_" + field.name + ".jpg")
        arcpy.SelectLayerByAttribute_management (lyr, "CLEAR_SELECTION")

Re: Still having problems stepping through a list inside of a tool

JohnNunnali — Sat, 11 Dec 2021 14:05:12 GMT

The last suggestion appears to solve the invalid expression issue I was having in the SelectLayerByAttribute function. The script now returns no error messages when running it through the PythonWin debugger. However when executing it no records are selected. It appears it successfully reads all lines in the code because it prints "finished" however it seems to exit the loop without selecting any features. The current code is:

#Script to create distribution maps for wetland species
#Import arcpy and set workspace
import arcpy
import arcpy.mapping
mxd = arcpy.mapping.MapDocument (r"P:\CoWetlandTools\maps\distribution_maps_Johns_pytest.mxd")

#list object that is the list of fields in our table
#List fields and generate a list

fields = arcpy.ListFields(r"Y:\John_N\Wetland_field_guide_distribution_maps\counties_datamerge.gdb\counties_edited_datamergetest" , None , "Double")

#select fieldname1 > 0

#Create Loop and List to run processing for
for field in fields:
  for lyr in arcpy.mapping.ListLayers (mxd, "file" , arcpy.mapping.ListDataFrames (mxd) [0]):
    #select counties each field has attribute values > 0
    #select field1 > 0
    arcpy.SelectLayerByAttribute_management (lyr, "NEW_SELECTION", '"' + field.name + '"  > 0')

    # export current to jpg such that the name of the file is field1
    arcpy.mapping.ExportToJPEG(mxd, r"P:\CoWetlandTools\maps\map_ " + field.name + ".jpg")

    #clear selected attributes
    arcpy.SelectLayerByAttribute_management (lyr, "CLEAR_SELECTION")

#finished
print "Finished"

I tried with the data in a shapefile and a personal geodatabase with similar results.

All suggestions are greatly appreciated.

Re: Still having problems stepping through a list inside of a tool

DarrenWiens2 — Sat, 11 Dec 2021 14:05:14 GMT

for lyr in arcpy.mapping.ListLayers (mxd, "file" , arcpy.mapping.ListDataFrames (mxd) [0]):

The above line means, "list all of the layers named 'file' from the first data frame in the map, 'mxd' ". Change 'file' to be the name of your layer in the map.