Error on Make Feature Layer for looping over selected attributes

[IanMurray]

I've ran into a bit of a unique problem, which for the life of me I cannot get debugged.

I have a two geodatabase feature classes, one with customer regions, and one with zip codes.  I'm iterating through each region and selecting each zip code within and writing the region with zip codes to a csv.  This works flawlessly the first loop through, but I run into an error on the Make Featuer Layer on subsequent iterations.  I am entering in the code through the iterative python window, and both fcs are in the TOC, and neither Feature Layer exists prior to running the code. 

 import arcpy from arcpy import env env.overwriteOutput = 1  arcpy.env.overwriteOutput = True #env.workspace = r"C:\Users\iamurray\Desktop\Ray\Zip Code\ZipCodes.gdb" output = open(r"C:\Users\iamurray\Desktop\Output.csv", "w")  count = 1  for item in range(1,163,1): # there are 162 features to be selected,      input1 = "TVA_Distributors"     input2 = "TVA_PSA_Zips"     outlayer1 = "TVA_Distributors_Layer"     outlayer2 = "TVA_PSA_Zips_Layer"     print count     #arcpy.SelectLayerByAttribute_management(input1, "NEW_SELECTION", [OBJECTID] =  + str(count))     arcpy.MakeFeatureLayer_management(input1, outlayer1, 'OBJECTID = ' + str(count)) #works on count = 1 but not on count = 2 and subsequent           cursor = arcpy.da.SearchCursor(outlayer1, ["DISTRIBU_1"])     for row in cursor:         output.write(str(row[0])+ "\n")         print row[0]     del cursor       # Process: Select Layer By Location     arcpy.SelectLayerByLocation_management(input2, "INTERSECT", outlayer1, "", "NEW_SELECTION")  # Process: Make Feature Layer (2)     arcpy.MakeFeatureLayer_management(input2, outlayer2)     field = "ZIP"     cursor = arcpy.SearchCursor(outlayer2)     for row in cursor:         output.write(str(row.getValue(field))+ "\n")     count += 1     print "The Current number of Distributors Processed is " + str(count - 1)     arcpy.Delete_management(outlayer1)     arcpy.Delete_management(outlayer2)     del row     del cursor output.close() 

These are the results I am getting
1
Warren RECC
The Current number of Distributors Processed is 1
2
Runtime error  Traceback (most recent call last):   File "<string>", line 20, in <module>   File "c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py", line 5748, in MakeFeatureLayer     raise e ExecuteError: ERROR 000622: Failed to execute (Make Feature Layer). Parameters are not valid. ERROR 000628: Cannot set input into parameter in_features.

It works fine the first time, because it reaches the print count on the second run through successfully.  At first I thought it was an issue with the overwrite, but I've set both of those to allow overwrite and deleted the intermediate layers prior to the next MakeFeatureLayer.  The input and outputs should not change just the expression, which should still be valid each time, since each has a unique OBJECTID number.  I simply can't see what is wrong with the tool parameters that would cause this to crash on the subsequent loops. 

If anyone has some insight into this, I would be most appreciative, I'm using ArcGIS 10.1 SP1.

EDIT: There are no syntax errors if you find any, its a result of me copying the code over.

[AdamCox1]

Hi there, one thing that looks fishy is the syntax around the where clauses you have.  I always follow this format.
For text fields

'"fieldname" = \'' + value + "'" #make sure value is a string

For integer fields (like OBJECTID)

'"fieldname" = ' + value #make sure value is a string (though a digit, of course)

What could be happening is that the first where clause is creating an empty feature layer, which in turn is an invalid input.  Just guessing though...

[IanMurray]

Adding the extra set of quotation marks made it crash on the first run through.  The code does actually select OBJECTID =  1 the first time and performs the subsequent steps correctly.  It is merely a problem of not accepting the input parameters for the make feature layer on subsequent iterations.

Thank you for the feedback though

[AdamCox1]

May be worth adding a

if arcpy.Exists(outlayer1):
    arcpy.management.Delete(outlayer1)

just before you make the feature layer.  I know you have set the overwrite to true, but it's worth a shot in case something is going weird with that.

[IanMurray]

Tried it, but it should have already been deleted from the delete statement near the end of the script. 

Again, thank you for your input. 

I'm thinking this is a problem with something stored in memory, or with a workspace or something.  I just have an utter lack of experience with handling something like that.

[T__WayneWhitley]

Haven't tested this, but think I see the source of your problem...you said:

"I am entering in the code through the iterative python window, and both fcs are in the TOC, and neither Feature Layer exists prior to running the code."

I'm sure you meant 'interactive python window'- regardless of that, you also said 'both fcs are in the TOC'.  I think that's your problem.  You would achieve more effective results using the fc reference to your workspace.  Note that what you refer to as fcs in your TOC is actually what I've seen called a 'map layers' or at any rate are really just feature layers you can reference from the map.

So your fix then is to use the statement you currently have commented out (if that gdb is actually where your input is):
#env.workspace = r"C:\Users\iamurray\Desktop\Ray\Zip Code\ZipCodes.gdb"

You can use the fc name (not the map layer name) directly in the Make Feature Layer line...so, in brief, something like this (I'll leave the testing to you - the following is simply test code to show successful execution for a 2nd iteration, and so on.):

import arcpy
from arcpy import env
env.overwriteOutput = True
env.addOutputsToMap = True
env.workspace = r"C:\Users\iamurray\Desktop\Ray\Zip Code\ZipCodes.gdb"

# set your input var to a sample input fc name in your source, ZipCodes.gdb
input1 = 'your gdb fc name here'

# sample of 2 outputs, added to the map only for testing purposes
outlayer1 = "TVA_Distributors_Layer"
outlayer2 = "TVA_PSA_Zips_Layer"

for i in range(2):
     arcpy.MakeFeatureLayer_management(input1, outlayer + str(i + 1), 'OBJECTID = ' + str(count))
     print '...check the map for {0}'.format(outlayer + str(i + 1))

print 'done.'

Just test code, so let me know if something doesn't work - don't forget to set the input1 variable to an fc name in your gdb (this is just a test, so initially it doesn't matter what it is) -- should result in 2 added outputs reflecting feature layers restricted (or in other words, filtered) by the OBJECTIDs 1 and 2.  If you get both added to your map okay, then you should be able to extend this to your specific example, thereby fixing your error.

Wayne

[IanMurray]

Okay so not sure why this fixed it but it did.

Originally I was using feature classes in a gdb, and I got this error on time second time through the for-loop.

Runtime error Traceback (most recent call last): File "<string>", line 20, in <module> File "c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py", line 5748, in MakeFeatureLayer raise e ExecuteError: ERROR 000622: Failed to execute (Make Feature Layer). Parameters are not valid. ERROR 000628: Cannot set input into parameter in_features.

I merely converted them to shapefiles and brought them into arcmap instead of the feature classes from a GDB and it worked after switching the selecting field name. 

There must be something about the gdb that puts some sort of lock it on for subsequent iterations where it does not for shapefiles.

this is the code that ended up working

import arcpy
from arcpy import env
env.overwriteOutput = 1
 
arcpy.env.overwriteOutput = True

output = open(r"C:\Users\iamurray\Desktop\Output.csv", "w")
input1 = "TVA_Distributors"
input2 = "TVA_PSA_Zips"
outlayer2 = "TVA_PSA_Zips_Layer"
   
for item in range(0,162,1):
    print item
    arcpy.MakeFeatureLayer_management(input1, "TVA_Distributors" + str(item), "\"FID\" = " + str(item))

    cursor = arcpy.da.SearchCursor("TVA_Distributors" + str(item), ["DISTRIBU_1"])
    for row in cursor:
        output.write(str(row[0])+ "\n")
        print row[0]
    del cursor
     
    arcpy.SelectLayerByLocation_management(input2, "INTERSECT", "TVA_Distributors" + str(item), "", "NEW_SELECTION")    
   # Process: Make Feature Layer (2)
    arcpy.MakeFeatureLayer_management(input2, outlayer2)
    field = "ZIP"
    cursor = arcpy.SearchCursor(outlayer2)
    for row in cursor:
        output.write(str(row.getValue(field))+ "\n")
    print "The Current number of Distributors Processed is " + str(item)
    arcpy.Delete_management("TVA_Distributors" + str(item))
    arcpy.Delete_management(outlayer2)
    del row
    del cursor
output.close()

This is the code I used for the gdb feature classes which failed, the only changes made were changing FID to ObjectID and the range from 0,162,1 to 1,163,1

import arcpy
from arcpy import env
env.overwriteOutput = 1
 
arcpy.env.overwriteOutput = True

output = open(r"C:\Users\iamurray\Desktop\Output.csv", "w")
input1 = "TVA_Distributors"
input2 = "TVA_PSA_Zips"
outlayer2 = "TVA_PSA_Zips_Layer"
  
for item in range(1,163,1):

    print item
    arcpy.MakeFeatureLayer_management(input1, "TVA_Distributors" + str(item), "\"OBJECTID\" = " + str(item))
    
    cursor = arcpy.da.SearchCursor("TVA_Distributors" + str(item), ["DISTRIBU_1"])
    for row in cursor:
        output.write(str(row[0])+ "\n")
        print row[0]
    del cursor
      
  # Process: Select Layer By Location
    arcpy.SelectLayerByLocation_management(input2, "INTERSECT", "TVA_Distributors" + str(item), "", "NEW_SELECTION")
 # Process: Make Feature Layer (2)
    arcpy.MakeFeatureLayer_management(input2, outlayer2)
    field = "ZIP"
    cursor = arcpy.SearchCursor(outlayer2)
    for row in cursor:
        output.write(str(row.getValue(field))+ "\n")
    print "The Current number of Distributors Processed is " + str(item)
    arcpy.Delete_management("TVA_Distributors" + str(item))
    arcpy.Delete_management(outlayer2)
    del row
    del cursor
 output.close()

And this was the error I was getting

Runtime error Traceback (most recent call last): File "<string>", line 20, in <module> File "c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py", line 5748, in MakeFeatureLayer raise e ExecuteError: ERROR 000622: Failed to execute (Make Feature Layer). Parameters are not valid. ERROR 000628: Cannot set input into parameter in_features.


I'm really curious why using the interactive python window that shapefiles would work and gdb feature classes would not.  Both had the same names.  The syntax had to be correct, because for the feature class code it would run the first iteration, but throw that error on the subsequent ones.  With the shapefiles it ran through without a hiccup.  In both instances the files were the only items in the TOC.

When I attempt to run it outside of ArcGIS with env.workspace set to the gdb, or the file path hardcoded, it crashes with ERROR 000386.



Traceback (most recent call last):
  File "C:\Users\iamurray\Desktop\Zipcode.py", line 31, in <module>
    arcpy.SelectLayerByLocation_management(input2, "INTERSECT", "TVA_Distributors" + str(item), "", "NEW_SELECTION")
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 6559, in SelectLayerByLocation
    raise e
ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000368: Invalid input data.
Failed to execute (SelectLayerByLocation).

>>>

Similarly when I run it in the interactive window with env.workspace set or the file paths hard-coded it crashes with same error

If I run it with just the file names with both feature classes brought into the dataframe it runs through the first time successfully and crashes with the error code from my first post.

Runtime error Traceback (most recent call last): File "<string>", line 20, in <module> File "c:\program files (x86)\arcgis\desktop10.1\arcpy\arcpy\management.py", line 5748, in MakeFeatureLayer raise e ExecuteError: ERROR 000622: Failed to execute (Make Feature Layer). Parameters are not valid. ERROR 000628: Cannot set input into parameter in_features.

When I try to run it with the shapefiles using env.workspace set to their folder location it crashes, since the input must be a feature layer or feature class.


So I'm really curious how my only solution to this was to bring the shapefiles into the TOC as feature layers and run the program in the interactive python window using only that workspace.

Wayne and Adam,
thank you for your attempts to help on this, it did get my brain thinking in different ways and eventually got me to a solution.

[T__WayneWhitley · Accepted solution]

I was listening to your further comments on this thread and decided to revisit this later in the day to test; I tested with 10.2.1 and was surprised to duplicate your error -- I used an updated script I'll include below; unfortunately there was an error with my line in the previous script (previous post above) attempting to reference a variable that hasn't been defined yet (outlayer, in the MakeFeatureLayer line):

outlayer + str(i + 1)

It now works for gdb fcs and was correct in directing you to try using fully qualified fc pathnames:

import arcpy, os from arcpy import env env.overwriteOutput = True env.addOutputsToMap = True env.workspace = r"C:\Users\whitley-wayne\Documents\ArcGIS\Default.gdb"  # the following line results in 000622 and 000628 failures on the 2nd iteration... #input1 = 'Export_Output'  # so instead set your input var to a fully qualified pathname to your gdb source input1 = os.path.join(env.workspace, 'Export_Output') print '\nusing a fully-qualified pathname to the fc:\n{0}\n'.format(input1)  # sample of 2 outputs, added to the map only for testing purposes outlayer1 = "TVA_Distributors_Layer" outlayer2 = "TVA_PSA_Zips_Layer"  # if querying OID, make sure you're querying the correct one objID = arcpy.Describe(input1).OIDFieldName print 'using the true OID field name, {0}\n'.format(objID)  for i in range(2):      outlayer = 'outlayer{0}'.format(str(i + 1))      qry = '{0} = {1}'.format(objID, str(i + 1))      print 'this is the query:  {0}'.format(qry)      arcpy.MakeFeatureLayer_management(input1, outlayer, qry)      print '...check the map for {0}\n'.format(outlayer)  print 'done.'

I think you were at least partially correct about the 'in-memory locking' issue (or however you described it), because if I use the name with the layer in the TOC, somehow Make Feature Layer doesn't seem to 'look' for the fc directly in the gdb -- what I mean by that is I think the in-memory pointer to the layer creates a 'lock' or at least blocks the 2nd iteration.  I'll show you what I mean - if the line in the code is used as follows:

input1 = 'Export_Output'

...then, the run (from the same code above with the fully qualified line of code commented out) prints feedback as follows - I get hit with the both the dreaded 000628 and 000622 errors:

using the true OID field name, OBJECTID_1  this is the query:  OBJECTID_1 = 1 ...check the map for outlayer1  this is the query:  OBJECTID_1 = 2  Runtime error  Traceback (most recent call last): File "<string>", line 26, in <module>  File "C:\Program Files (x86)\ArcGIS\Desktop10.2\arcpy\arcpy\management.py", line 6043, in MakeFeatureLayer  raise e ExecuteError: ERROR 000622: Failed to execute (Make Feature Layer). Parameters are not valid. ERROR 000628: Cannot set input into parameter in_features. 

...alternately (this is the fix), if I run with the fully qualified pathname line (os.path.join etc etc), the execution is seamless, successfully completing through the 2nd iteration (as intended) with the printed feedback:

using a fully-qualified pathname to the fc: C:\Users\whitley-wayne\Documents\ArcGIS\Default.gdb\Export_Output  using the true OID field name, OBJECTID_1  this is the query:  OBJECTID_1 = 1 ...check the map for outlayer1  this is the query:  OBJECTID_1 = 2 ...check the map for outlayer2  done.

Hope that's clear...

Wayne


PS - intriguing, your post #7, thank you for that feedback.

[IanMurray]

Rather ironically, I the work I was doing was at someone elses request, and they decided to have the list exported the other way, using the second layer as the selector. 

This made the script that much more annoying, since I have 1100+ features to go through instead of ~150.  ArcGIS crashed out while running the script while using the interactive python window while going through the larger number of features, so this has been most useful in getting the script to work outside of ArcGIS, which is probably about 10x faster to boot.  I'd rather have done it this way from the start, but that nagging error kept getting me.

I would like an explanation though why a fully qualified file path is necessary, and why just connecting to the data with env.workspace caused an issue.

Thank you Wayne for your help.  I've been meaning to get back on this sooner if nothing else just to discuss this interesting problem, but I was out of town a few days and now your help has paid off big time.

[T__WayneWhitley]

Not sure that I can explain to your complete satisfaction, particularly since I'm not a systems level programmer.  I'll try though, and as I said, it is at least related to what you've already surmised is a 'locking' issue, could even be a bug, just not sure -- I just cannot describe where or why exactly the lock is occurring.  But to at least entertain that this could be a 'design feature', I'll defend that in saying it could very well serve good purpose in stopping the overwrite or otherwise unintended modification of a map layer.

I experimented further with the map layers and there is some in-memory locking and such going on -- I don't think I'll post it here, seems it would only serve at this point to confuse matters.  I will say the behavior was consistent on specifying a string that matched a layer name in the TOC executed fine on a '1st iteration'.  Not sure if a temporary in-memory reference was made to the the layer in the map, then couldn't 'find' that reference (or clear it for availability) on a '2nd iteration' or what's going on.  But I think the overriding principle here is that you can rely on the 'hard-coded' pathname reference to your gdb or possibly even make a specific in-memory copy you can reference again...I wouldn't think it necessary to make an in-memory copy if a perfectly good copy is in a highly available gdb, but then again it depends on how processing intensive you have to get in the steps to reach your final output.

Enough said about that I suppose -- I think from here it is worth mentioning there are some very useful arcpy.mapping module techniques you can use to manipulate and refresh the layers in your map after you've finished processing stuff in-memory (or in the default gdb, or python data structure, whatever you have convenient to you).

Incidentally, to clarify a statement you said about env.workspace -- I have in my code 3 env statements together.  The workspace environment variable did not have to be set there, just a matter of preference, but I just like to keep them together (so they're easy to find and change as needed).  Since I set the pathname to the workspace there (env.workspace), then I could make certain I'm specifying the pathname to my layer's datasource itself using os.path.join(<the gdb>, <the fc by name in the gdb>), so that 1st param is simply env.workspace.... and like I said, if you wanted to change that gdb source, you'd only need to change it in one place, where you set the env.workspace var earlier in the script.


One more note and I'll close -- the fooling around I did with the layer names wasn't necessary, the layers added to the map did not pass the text, "TVA_Distributors_Layer" or "TVA_PSA_Zips_Layer", because in the loop I wasn't referencing the variables outlayer1 and outlayer2, but the text 'outlayer1' and 'outlayer2' and so that text became the layer names.  A better way to pull this off is to use the name property of the layer object in the mapping module, see this:

Layer (arcpy.mapping)
Desktop » Geoprocessing » ArcPy » Mapping module
http://resources.arcgis.com/en/help/main/10.2/index.html#//00s300000008000000


Hope that helps.

Wayne