topic Re: Python Script: loop through selected features and and copy to geodatabase in Python Questions

Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Sun, 13 Apr 2014 19:35:18 GMT

I am working with surface types that I derived from LIDAR data. These surface types are in one feature class (polygon) -
associated with the feature class is a column called "grid_code"; the grid_code column has values from 0 to 145. These are elevation values and will be selected out to classify three surface types (ground surface, shrubs and trees).

I want to write a script that selects each surface type and creates a feature class for each of them. My output would have three
feature classes: ground, shrub and tree.

Here is the code I am working with but I am getting errors:

Thanks!

import arcpy, os

aWS = r"C:\Thesis_Research\GIS\Prov_classification.gdb"

arcpy.env.workspace = aWS

arcpy.env.overwriteOutput = True

try:

    # Create query statements for each land cover type
    LIDAR_tree = '"grid_code" > \'3\''
    LIDAR_shrub = '"grid_code" = \'2\' OR "grid_code" = \'3\''
    LIDAR_ground = '"grid_code" <= \'1\''

    aListLIDAR = [LIDAR_tree, LIDAR_shrub, LIDAR_ground]

    for item in aListLIDAR:

        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", P303_layerL)

        arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", aListLIDAR)

        arcpy.CopyFeatures_management(P303_layerL, "P303_" + str(item))

    del aListLIDAR
    del item
    del P303_layerL


except:

    # Code to run when an error occured
    print "An ERROR Occured!"
    print "\n" + arcpy.GetMessages()

else:

    # Message when there was no error
    print "\nNo Error occurred"

# Script end message
print "\nEnd of the script!"

Re: Python Script: loop through selected features and and copy to geodatabase

DanPatterson_Retired — Mon, 14 Apr 2014 05:50:13 GMT

what are the errors?
enclose your script in code blocks so that they can be read

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Fri, 10 Dec 2021 23:07:59 GMT

I put the script in a code block - It is not telling me the error.

I am new to python, and I just threw this script together by reading other python scripts. Let me know if you need more info

# Developed by: David Bailey
# Description: Providence Creek Watershed: land cover processing


# import arcpy site-package and os module
import arcpy, os

# Set geoprocessing environment:
# a) set the workspace environment 
aWS = r"C:\Thesis_Research\GIS\Prov_classification.gdb"
arcpy.env.workspace = aWS

# Set geoprocessing environment:
# b) the overwriteOutput parameter controls whether tools will
#    automatically overwrite any existing output 
arcpy.env.overwriteOutput = True

try:
   
    # Create query statements for each land cover type
    LIDAR_tree = '"grid_code" > \'3\''
    LIDAR_shrub = '"grid_code" = \'2\' OR "grid_code" = \'3\''
    LIDAR_ground = '"grid_code" <= \'1\''

    aListLIDAR = [LIDAR_tree, LIDAR_shrub, LIDAR_ground]
    

    for item in aListLIDAR:
        
        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", P303_layerL)

        arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", aListLIDAR)
        
        arcpy.CopyFeatures_management(P303_layerL, "P303_" + str(item))

    del aListLIDAR
    del item
    del P303_layerL

    
except:
    
    # Code to run when an error occured
    print "An ERROR Occured!"
    print "\n" + arcpy.GetMessages()

else:
    
    # Message when there was no error
    print "\nNo Error occurred"
    
# Script end message
print "\nEnd of the script!"

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Mon, 14 Apr 2014 06:44:58 GMT

I am assuming the "for loop" is not correctly structured?

Re: Python Script: loop through selected features and and copy to geodatabase

IanMurray — Fri, 10 Dec 2021 23:08:02 GMT

I put the script in a code block - It is not telling me the error.

I am new to python, and I just threw this script together by reading other python scripts. Let me know if you need more info

# Developed by: David Bailey
# Description: Providence Creek Watershed: land cover processing


# import arcpy site-package and os module
import arcpy, os

# Set geoprocessing environment:
# a) set the workspace environment 
aWS = r"C:\Thesis_Research\GIS\Prov_classification.gdb"
arcpy.env.workspace = aWS

# Set geoprocessing environment:
# b) the overwriteOutput parameter controls whether tools will
#    automatically overwrite any existing output 
arcpy.env.overwriteOutput = True

try:
   
    # Create query statements for each land cover type
    LIDAR_tree = '"grid_code" > \'3\''
    LIDAR_shrub = '"grid_code" = \'2\' OR "grid_code" = \'3\''
    LIDAR_ground = '"grid_code" <= \'1\''

    aListLIDAR = [LIDAR_tree, LIDAR_shrub, LIDAR_ground]
    

    for item in aListLIDAR:
        
        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", P303_layerL)

        arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", aListLIDAR)
        
        arcpy.CopyFeatures_management(P303_layerL, "P303_" + str(item))

    del aListLIDAR
    del item
    del P303_layerL

    
except:
    
    # Code to run when an error occured
    print "An ERROR Occured!"
    print "\n" + arcpy.GetMessages()

else:
    
    # Message when there was no error
    print "\nNo Error occurred"
    
# Script end message
print "\nEnd of the script!"

A few questions and observations. First is "P303_LIDAR_Poly" a feature class that is directly within your geodatabase, or is it in a feature class within the feature dataset? If its in a feature dataset within the geodatabase, your environment for that part needs to be within that feature dataset it resides.

Second, "arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", aListLIDAR)" is wrong. it should be "arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", item)". Right now it is using the whole list for your SQL expression, which is a list. You want it to run through each expression individually, so use item.

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Mon, 14 Apr 2014 16:24:57 GMT

Thanks for the response.

"P303_LIDAR_Poly" is a feature class directly in my geodatabase - I did not create a feature dataset.

I changed the "arcpy.SelectLayerByAttribute_management" tool to (arcpy.SelectLayerByAttribute_management(P303_layerL, "NEW_SELECTION", item)" but I still had an error.

I have my code written where it will tell me what the error was - but for some reason it is not giving me the messages.

Re: Python Script: loop through selected features and and copy to geodatabase

MattEiben — Fri, 10 Dec 2021 23:08:05 GMT

Well there seems to be a couple issues here.

In your arcpy.MakeFeatureLayer_management method, your outlayer is being written to a variable "P303_layerL" that was never declared anywhere, that would certainly throw an error.

I made some changes to your script and posted them below here.

Some of the changes I made in your for loop:

[INDENT]I integrated the select by attributes method right into the Make Feature Layer, since it has an optional where clause where you put your query.
Make Feature Layer Documentation

I changed your naming in the copy feature layer to reflect which layer you're copying by vegetation type.

I added in a delete step at the end to remove the temporary feature layer before you create another one on the next loop. Not deleting the feature layer before the next step may cause issues.[/INDENT]

Hopefully this code works for you!
I'm happy to help you tweak it further if you're still having some issues.

Matt

# Developed by: David Bailey
# Description: Providence Creek Watershed: land cover processing


# import arcpy site-package and os module
import arcpy, os

# Set geoprocessing environment:
# a) set the workspace environment 
aWS = r"C:\Thesis_Research\GIS\Prov_classification.gdb"
arcpy.env.workspace = aWS

# Set geoprocessing environment:
# b) the overwriteOutput parameter controls whether tools will
#    automatically overwrite any existing output 
arcpy.env.overwriteOutput = True

try:
   
    # Create query statements for each land cover type
    LIDAR_tree = '"grid_code" > \'3\''
    LIDAR_shrub = '"grid_code" = \'2\' OR "grid_code" = \'3\''
    LIDAR_ground = '"grid_code" <= \'1\''

    aListLIDAR = [LIDAR_tree, LIDAR_shrub, LIDAR_ground]
    layerName = ["tree","shrub","ground"]

    n = 0
    for query in aListLIDAR:

        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", "temp_layer", where_clause=query)
        arcpy.CopyFeatures_management("temp_layer", "P303_{0}_layer".format(layerName))
        arcpy.Delete_management("temp_layer")
        n += 1

except:
    
    # Code to run when an error occured
    print "An ERROR Occured!"
    print "\n" + arcpy.GetMessages()
    
# Script end message
print "\nEnd of the script!"

Re: Python Script: loop through selected features and and copy to geodatabase

benberman — Tue, 15 Apr 2014 12:13:26 GMT

The arcpy.Delete_management("temp_layer") should be outside of the for loop, yet on the same indentation. You want the temp layer to be deleted after you have processed the items in the list.

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Wed, 16 Apr 2014 06:21:34 GMT

Thanks for the responses. The new script looks good and it ran without any errors - however when I opened my three new land
surface types (P303_ground_layer, P303_shrub_layer, and P303_tree_layer) they had the same attributes as "P303_LIDAR_poly".
It seems as if it did not process the query statements.

Re: Python Script: loop through selected features and and copy to geodatabase

IanMurray — Wed, 16 Apr 2014 13:09:49 GMT

arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", "temp_layer", where_clause=query)

should probably be

arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", "temp_layer", query)

As I recall, you don't have to put the word where_clause in the parameters, its understood since its the 3rd parameter for make feature layer.

Also Matt

In your arcpy.MakeFeatureLayer_management method, your outlayer is being written to a variable "P303_layerL" that was never declared anywhere, that would certainly throw an error.

its not being written to a variable, its a string, and since its an output parameter, it doesn't need to exist already, you are just naming your output layer name, it no different then what you changed it to. "temp_layer" has no special meaning, its just a name you are giving it to store in memory.

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Fri, 10 Dec 2021 23:08:07 GMT

This is what my code looks like now (It works - but my outputs still have the same attributes as the P303_LIDAR_Poly):

Any ideas?

# Developed by: David Bailey
# Description: Providence Creek Watershed: land cover processing


# import arcpy site-package and os module
import arcpy, os

# Set geoprocessing environment:
# a) set the workspace environment 
aWS = r"C:\Thesis_Research\GIS\Prov_classification.gdb"
arcpy.env.workspace = aWS

# Set geoprocessing environment:
# b) the overwriteOutput parameter controls whether tools will
#    automatically overwrite any existing output 
arcpy.env.overwriteOutput = True

try:
   
    # Create query statements for each land cover type
    LIDAR_tree = '"grid_code" > \'3\''
    LIDAR_shrub = '"grid_code" = \'2\' OR "grid_code" = \'3\''
    LIDAR_ground = '"grid_code" <= \'1\''

    aListLIDAR = [LIDAR_tree, LIDAR_shrub, LIDAR_ground]
    layerName = ["tree","shrub","ground"]

    n = 0
    for query in aListLIDAR:

        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", "temp_layer", query)
        arcpy.CopyFeatures_management("temp_layer", "P303_{0}_layer".format(layerName))
    arcpy.Delete_management("temp_layer")
    n += 1

    
except:
    
    # Code to run when an error occured
    print "An ERROR Occured!"
    print "\n" + arcpy.GetMessages()

else:
    
    # Message when there was no error
    print "\nNo Error occurred"
    
# Script end message
print "\nEnd of the script!"

Re: Python Script: loop through selected features and and copy to geodatabase

MattEiben — Fri, 10 Dec 2021 23:08:10 GMT

Hmmmm,

It sounds like the query isn't working when the feature layer is being made. To figure out what's going on, my suggestion would be to make your feature layer and then run a select by attribute. I put a code snippet below. Try replace your for loop with that code. I also added in an arcpy.AddMessage that should echo out the Query you're using for each layer. This isn't really required, just included that for error checking.

    n = 0
    for query in aListLIDAR:

        arcpy.MakeFeatureLayer_management("P303_LIDAR_Poly", "temp_layer")

        # You can use "print" instead of "arcpy.AddMessage" if you're directly in the console.
        arcpy.AddMessage("Query For: {0}: {1}".format(layerName,query))
        arcpy.SelectLayerByAttribute_management ("temp_layer", "NEW_SELECTION",query)

        arcpy.CopyFeatures_management("temp_layer", "P303_{0}_layer".format(layerName))
        arcpy.Delete_management("temp_layer")
    n += 1

Are there three layers being made with the names
P303_tree_layer,
P303_shrub_layer,
P303_ground_layer
currently?

Hope this helps!

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Sat, 19 Apr 2014 21:27:53 GMT

Yes, with my current script I am getting three different layers with the names: P303_tree_layer, P303_shrub_layer, P303_ground_layer - however they are all the same. They should be different according to the query that was created.

I tried the code you gave me and it came back with this error:

Executing: SelectLayerByAttribute temp_layer NEW_SELECTION ""grid_code" > '3'"
Start Time: Sat Apr 19 15:13:12 2014
ERROR 000358: Invalid expression
An invalid SQL statement was used.
An invalid SQL statement was used. [P303_LIDAR_Poly]
Failed to execute (SelectLayerByAttribute).
Failed at Sat Apr 19 15:13:12 2014 (Elapsed Time: 0.00 seconds)

End of the script!

It appears that my SQL statements are not written correctly? I don't see where

Re: Python Script: loop through selected features and and copy to geodatabase

MattEiben — Sun, 20 Apr 2014 21:45:09 GMT

Make sure to null out the quotes inside your expression using backslashes like this on all your SQL queries:

"\"grid_code\" > '3'"

If you don't do that, Python will see those other quotes as symbols to close and open your string instead of as literal parts of your string.

Re: Python Script: loop through selected features and and copy to geodatabase

DavidBailey — Wed, 23 Apr 2014 05:29:57 GMT

I did this and it still gives me this error:

An ERROR Occured!

Executing: SelectLayerByAttribute temp_layer NEW_SELECTION ""grid_code" > '3'"
Start Time: Tue Apr 22 23:29:17 2014
ERROR 000358: Invalid expression
An invalid SQL statement was used.
An invalid SQL statement was used. [P303_LIDAR_Poly]
Failed to execute (SelectLayerByAttribute).
Failed at Tue Apr 22 23:29:17 2014 (Elapsed Time: 0.00 seconds)

End of the script!