topic Re: Select by attribute use cursor save off as feature class in Python Questions

Select by attribute use cursor save off as feature class

MikePowell — Tue, 18 Feb 2014 18:39:59 GMT

I feel this is a very basic problem but for the life of me I can't seem to figure out what is wrong, which is driving mad. I am assuming it is my logic. I have a feature class with hundreds of polygons. I figured I could make a layer file, use a search cursor to select each polygon and save each one off as it's own feature class. I seem to always get an error that the dataset xxxxx does not exist or is not supported when trying to do Copy Feature. Here is what I have:

import sys, string, os, cx_Oracle, arcpy, datetime

arcpy.env.workspace = "c:\\work\\scripting\\test\\testdata.gdb"
arcpy.MakeFeatureLayer_management("rsids","rsid_view")
rows = arcpy.SearchCursor("rsid_view","","","NUM; EXTENSION","NUM A")
row = rows.next()
while row:
    RSIDNum = str(row.getValue("NUM"))
    Ext = str(row.getValue("EXTENSION"))
    if Ext == "None":
        Ext = ""
    rsid = RSIDNum+Ext
    RSID = "RSID"+RSIDNum+Ext
    arcpy.CopyFeatures_management(RSID, "c:\\work\\scripting\\test\\testdata.gdb\\"+RSID)
    row - rows.next()
del row
del rows
if arcpy.Exists("rsid_view"):
   arcpy.Delete_management("rsid_view")
   print "View deleted"

I have tried using Select by attribute first (before the search cursor) but that almost seems redundant, and I get the same problem anyway. Any help will be appreciated. Thanks in advance

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Fri, 10 Dec 2021 23:03:48 GMT

Hello Mike,

A few notes:

First off, you have change the row - rows.next() to row = rows.next() (in the while loop).

Second, check your field name "NUM A" can't be a field name since it has a space.

Third, I would actually change the while loop to a for loop (and get rid of both row = rows.next() lines). This would negate the first comment. I find for loops generally easier to grasp and less problematic.

Forth, I would use the Feature Class To Feature Class tool. It has a built in where clause that would allow you to only export one record at a time.

I would use something like this:

import arcpy
arcpy.env.workspace = "C:\\work\\scripting\\test\\testdata.gdb"
fc="rsid"
outLocation="C:\\work\\scripting\\test\\output"
rows = arcpy.SearchCursor(fc,"","","NUM; EXTENSION","NUM A")
for row in rows:
    where='"FID" = '+str(row.NUM) #Note that the given field must be unique, also assumes field is a number type. We'll have to make a new where statement if NUM is a string/text
    arcpy.FeatureClassToFeatureClass_conversion(fc, outLocation, str(row.NUM), where)
del row
del rows

Let me know how it goes, we'll figure it out!

Re: Select by attribute use cursor save off as feature class

MikePowell — Fri, 10 Dec 2021 23:03:51 GMT

Thanks for the quick reply. I will just go off your steps:

1. Yep, that was a type-o.
2. NUM A is not field name. It is an attribute of the SearchCursor where I am sorting on the field name NUM. NUM and EXTENSION are the two field names.
3. I have tried the "for loop" as well. with no success.
4. I like the use of the Feature Class to Feature Class. I tried that once but I couldn't get it to work.

I ran what you provided, this is what I got:

ExecuteError: ERROR 000361: The name starts with an invalid character
Failed to execute (FeatureClassToFeatureClass).

I know that NUM is set up as a string but they are all numbers and isn't that an issue when naming a feature class. That is why I had that other code in there to merge the field names and add "RSID" in front of it. I went ahead and tried this:

arcpy.FeatureClassToFeatureClass_conversion(fc, outLocation, "r"+str(row.NUM), where)

that seemed to work until it got to a duplicate. That is why I have a Select by Layer to eliminate those duplicates:

arcpy.SelectLayerByAttribute_management("rsid_view", "NEW_SELECTION", """"EXTENSION" not like 'X%' AND "EXTENSION" <> 'x'""")
arcpy.SelectLayerByAttribute_management("rsid_view", "ADD_TO_SELECTION", """"NUM" = '783'""")

I guess I just answered my own thoughts about it being redundant with the SearchCursor. Any thoughts? Can you use FeatureClassToFeatureClass with a MakeFeatureLayer? I also noticed that the feature classes that it did make, were named correctly but the attributes were not correct on any of them and some didn't have any. Appreciate any other thoughts you might have. Thanks again in advance

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Fri, 10 Dec 2021 23:03:53 GMT

Hello Mike,

Regarding item 2: my apologies, you're right. Also, nice fix on the error!

Regarding the where clause:
If NUM is actually set up as a string, you'll need to change the where expression to:

where='"NUM" = \''+str(row.NUM)+'\''

However, I guess if you got it working then you should leave it as is!

Regarding the duplicate issue, it depends on how you want to handle them. I assume you sorted the records and you only want to take the first unique record.

Assuming your dataset isn't absurdly large, I would create an array of all NUM's you've processed and then only create a new FC if the current NUM isn't in the processed NUM array. Like this:

import arcpy
arcpy.env.workspace = "C:\\work\\scripting\\test\\testdata.gdb"
fc="rsid"
outLocation="C:\\work\\scripting\\test\\output"
rows = arcpy.SearchCursor(fc,"","","NUM; EXTENSION","NUM A")
processedNUMs=[]
for row in rows:
    num=str(row.NUM)
    if num in processedNUMs:
        continue
    where='"NUM" = '+num #if NUM is a number
    #where='"NUM" = \''+num+'\'' #if NUM is a string/text
    if row.EXTENSION == "None":
        ext=""
    else:
        ext=str(row.EXTENSION)
    newFCName="RSID"+num+ext
    arcpy.FeatureClassToFeatureClass_conversion(fc, outLocation, newFCName, where)
    processedNUMs.append(num)
del row
del rows

Re: Select by attribute use cursor save off as feature class

MikePowell — Fri, 10 Dec 2021 23:03:56 GMT

Nice! here is the final code that worked:

import arcpy
arcpy.env.workspace = "C:\\work\\scripting\\test\\testdata.gdb"
fc="rsids"
outLocation="C:\\work\\scripting\\test\\results.gdb"
rows = arcpy.SearchCursor(fc,"","","NUM; EXTENSION","NUM A")
processedNUMs=[]
for row in rows:
    num=str(row.NUM)
    if num in processedNUMs:
        continue
    #where='"NUM" = '+num #if NUM is a number
    where='"NUM" = \''+num+'\'' #if NUM is a string/text
    if row.EXTENSION == "None":
        ext=""
    else:
        ext=str(row.EXTENSION)
    newFCName="RSID"+num+ext
    arcpy.FeatureClassToFeatureClass_conversion(fc, outLocation, newFCName, where)
    processedNUMs.append(num)
del row
del rows

I had to make a couple of minor changes but it worked. I wasn't thinking array. Dictionary came up but wasn't sure about that. The only problem now, which I just found out after running this, is that in the attribute table, it isn't consist in the order how things were put in. If you notice in my Select Layer by Attributes, I was eliminating those with the EXTENSION = X (because those are duplicates of another with the same NUM but different EXTENSION but I don't want the ones with EXTENSION = X). After running your script, I found that some of those were ahead of the ones I wanted so it was doing those instead of the ones I wanted. Thank you so much for the help. If you have any ideas of having them skip the ones with EXTENSION = X that would be great. I will be working on that as well. I am thinking some kind of "if" statement running through the array? I'm just happy that it is working. Thanks again

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Fri, 10 Dec 2021 23:03:58 GMT

Hello Mike,

I'm happy to hear we're on the right track. As I understand it (please correct me if I'm wrong), you want to skip over any records that has an extension starting with the character 'x', or if the extension is simple 'x'. I'll also assume you want to skip the record regardless of case ('x' vs 'X').

If this is all correct, add these lines (just before the line newFCName="RSID"+num+ext😞

if ext.lower().startswith('x'):
    continue

Note this will also skip records if the extension is simple 'x' or 'X'. If you want to be case sensitive, remove .lower() and change 'x' to 'X' if desired.

Let me know how it goes!

Re: Select by attribute use cursor save off as feature class

MikePowell — Wed, 19 Feb 2014 14:17:42 GMT

That was the ticket. Thanks for all the help. There is a lot more to this script that I am working on, but I will be able to incorporate this into it and get that much closer to finishing it. It is possible that I will be on the forum again soon but I try to use it as a last resort and try to figure it out first on my own. I feel I understand more than the basics but I have a problem putting it all together in script form. I can read scripts a lot better than I can write them. Thanks again

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Wed, 19 Feb 2014 14:39:56 GMT

Good. I'm happy we fixed this part of your script. Your coding looks pretty good, you just need to keep learning the small tricks. I hope you get your master script working, but i'll keep an eye out (on this thread or a new one) in case you hit any brick walls.

Good luck, have fun!

Re: Select by attribute use cursor save off as feature class

MikePowell — Wed, 19 Feb 2014 17:10:29 GMT

Thanks, I appreciate it.

There is one last thing that has come up. There is one, where EXTENSION = <Null>. Everything works but it is still naming the feature class "RSID"+num+ext where ext is None (ex. RSID783None). There is a catch for it but it doesn't seem to be blanking it out. My guess is it isn't understanding the <Null>. I would like it to look like this, RSID783. Any thoughts?

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Wed, 19 Feb 2014 18:07:30 GMT

In python null values are stored as None (which is different form a string set to "None"). None is a keyword in python. You can type x=None into python. When you convert None to a string (str(None)) it prints "None" which is kind of confusing. If it printed an empty string ("") it might be more accurate.

Change this line if row.EXTENSION == "None": to:

if row.EXTENSION == "None" or row.EXTENSION==None:

This would also work, but it's kind of confusing, so I would not use it:

if str(row.EXTENSION) == "None":

Let me know if it gives you any problems.

Re: Select by attribute use cursor save off as feature class

MikePowell — Wed, 19 Feb 2014 19:14:04 GMT

Again thanks. That worked great. I tried a couple of different ways to make it blank but I never thought to not have quotes around None. Thanks for all the insight. The rest of the script is coming together great. Just cleaning up little things to make it all tie together. There will be one more section that I will have to write that will probably rack my brains a little but hopefully, I will be able to figure it out. If not, I will back on here. Thanks again

Re: Select by attribute use cursor save off as feature class

MikePowell — Wed, 11 May 2016 22:39:15 GMT

Joshua, I was wondering if you were still out there. I am having a problem with the above script again (I know 2 yrs later). The FeatureClassTo FeatureClass is not working. I get the dreaded ERROR 9999999. I can't figure out why. Some stuff has changed in the database that I would think make thing easier but I just can't seem to get it past that point. If you are available for a little bit I was hoping to ask for a little help, since you were the one that was so kind to helped me before. If you can help I will fill you in on what has changed in the database and what I thought would work. Thanks again.

Re: Select by attribute use cursor save off as feature class

DanPatterson_Retired — Wed, 11 May 2016 22:44:24 GMT

Mike... I sent Josh a message, hopefully you will hear from him.

Re: Select by attribute use cursor save off as feature class

MikePowell — Wed, 11 May 2016 22:47:12 GMT

I really appreciate it. I hope it’s not much of a problem

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Thu, 12 May 2016 14:03:00 GMT

Hello Mike,

I'm still kicking around. Feel free to ask away!

I'll see if I can help (and it looks like we may get help from guru Dan Patterson)

Re: Select by attribute use cursor save off as feature class

MikePowell — Thu, 12 May 2016 15:01:04 GMT

Again, thanks in advance for any help, and time. I hope this isn’t too hard to read. Basically, I am trying to do the same thing as before. I have a feature class (RSID)with a few hundred features. I would like to select each on and make it its own feature class and do stuff to it. So, I have everything done that I need to do to each RSID but I can’t seem to get the FeatureClassToFeatureClass to work. Now before, when we worked on it, there was a column NUM and an EXTENSION column that I needed and that is how we did what we did before. Now there is a NAME column that has what I need (both name and extension and when I dissolve on NAME I figure I don’t need a lot of what we had in there before. I thought that I could just search cursor through the NAME column, add “RSID” to the front of the name as I know that a feature class can’t start with a number (the NAME column looks like this ex. 789M, 801M, 541L) and make the name feature class. I am getting the ERROR 999999. I have what I did down below, and I have commented out the old code that we were using. To explain the script, I am making sure that that I have copies of feature classes and dissolving on NAME. I am than setting up some directories and feature datasets to put everything and making sure everything is correct with the slashes, than I do the searchcursor. It all works until it gets to the FeatureClassToFeatureClass. Any thoughts? I guess I haven't figured out how to copy and paste the script. The first attempt was pretty bad. The second was better but I decided just to attach it. Let me know if it doesn't work

Re: Select by attribute use cursor save off as feature class

JoshuaChisholm — Thu, 12 May 2016 15:55:31 GMT

Hello Mike, Thanks for the clear post.

Nice looking script, I have a few suggestions of the get-go.

the line arcpy.FeatureClassToFeatureClass_conversion(nfc, NewoutLocation, newFCName) has no where clause (the optional 4th parameter). I think this means that even if the line weren't erroring out, it would create a full copy of every row in the "nfc" feature class X times (where X is the number of rows in the "nfc" feature class).
I think you will want to add a where clause so it only exports the current row each time. Something like this might work:
```
arcpy.FeatureClassToFeatureClass_conversion(nfc, NewoutLocation, newFCName, "NAME = '"+row.NAME+"'")
```
In addition to the query, you might want to try exporting to the GDB, not the Dataset within the GDB. I've run into problems before when exporting to a Dataset. Sometimes inconsistent projections or precisions can cause problems. Try outputting to "outLocation" for now, and exporting to a Dataset once we have everything else working. Your new line (including item 1 above, should look something like this:
```
arcpy.FeatureClassToFeatureClass_conversion(nfc, outLocation, newFCName, "NAME = '"+row.NAME+"'")
```

Let me know if those 2 changes do anything. Also,any chance you can post a small data set for testing?

Good luck!

Re: Select by attribute use cursor save off as feature class

MikePowell — Thu, 12 May 2016 17:53:16 GMT

Cool, that seemed to work. I was under the impression that the where clause was optional and that you didn’t have to have it as a parameter. This is working now. It is bombing at another point in the script where it is doing other things but is using FeatureClassToFeatureClass_conversion without a where clause. I will have to look into that a little more to determine the where clause that I am going to need. I just haven’t reached that point before now. Thanks again for the help. I am glad to know that my thinking was correct in that I didn’t need all that other code now, but it makes me wonder why I didn’t dissolve it in the first place. Oh well, live and learn. I will keep in mind the Dataset issue as well. I will let you know if something else comes up. Again, appreciate the help