counting search cursor results

[MikeTischler]

Hi,
I'm using the arcpy.SearchCursor code to query a standalone table.  All I really need is the count of the results - not any of the actual data.  The searchcursor itself finishes rather quickly, but iterating through the search cursor takes a surprisingly long time. 

Here's the method I'm using.  Is there a faster way to get what I need?

 rows = arcpy.SearchCursor(table,searchstring)

                #count the number of rows in output   
                count = 0
               
                for row in rows:
                    count += 1

[KevinHibma]

For sure, "Get Count":
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//0017000000n7000000.htm

If you need a selection/search on it, you can do a "make table view" then "select by attribute" Adding these couple steps might slow it down a little, however if you just want the number of rows in a table, featureclass, etc - Get Count should be the fastest.

[MikeTischler]

Thanks!

I tried making a table view and then running GetCount on that, and it didn't seem any faster than what I was doing before.  And unfortunately, I can't run GetCount on the SearchCursor object. 

thanks,
mike

[ChrisSnyder]

Just use the SQL paramater of either the MakeFeatureLayer or MakeTableView tool, then the GetCount tool.

It should take like 1 second. Some v9.3 code for example:

gp.MakeFeatureLayer_management(indxTileFC, indxTileFL, "FID_local_know_nw > -1")
if int(gp.GetCount_management(indxTileFL).getoutput(0)) > 0:
   gp.Blah_managment(this, that, theOther)

[MikeTischler]

That method does work, but it isn't any faster (at least with my tables) than using a search cursor and iterating through it.

[ChrisSnyder]

How many rows are in your table?

[DaleHoneycutt]

This is a mystery -- Get Count (using the mechanics that Chris showed) should be WAY faster than cycling through the records with a cursor.  There's something about your data or process that we're missing -- any other clues (besides # of records, like Chris asks?)

[MikeTischler]

Sorry for the late response - my email filter sent the reply notices to my spam folder.  Go figure.

There are a substantial amount of records for some tables....3-4 million in a few cases.

A test with a 2,130,000 records took 19.07 seconds using the following code:

from arcpy import *
from stopwatch import clockit
@clockit
def maketv_count(table):
    
    arcpy.MakeTableView_management(table,"mytv5k","NEAR_DIST < 1000")
    arcpy.GetCount_management("mytv5k")

using a search cursor and iterating through took 19.43 seconds using this code:

def maketv_cursor(table):
 
    rows = arcpy.SearchCursor(table,"NEAR_DIST < 1000")
    count = 0
    for row in rows:
        count += 1

Granted, 2 million records is a LOT...but I didn't expect the methods to be nearly identical in their speed.

Thanks
mike

[ChrisSnyder]

Well.... That's a lot of records - seems 20 seconds would be a reasoable time to expect to return the number of selected records in 2.1 million record table! However:

1. If you add an attribute index, the performance may increase (probably for both the cursor and getcount methods): http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//00170000005z000000.htm and http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//005s00000011000000.htm

2. This probably isn't any faster (could be for some weird reason though?), but there is a nifty geoprocessing scripting function that will return the OIDs of a selected set of records within a feature layer. From that you can gleen the selected count. For example:

gp.MakeFeatureLayer_management(fc, "fl")
gp.SelectLayerByAttribute_management("fl", "NEW_SELECTION", "WIDGETS > 30")
fidSet = gp.describe("fl").fidset #this returns a big honking text string - can be millions of charatcters long in my experience!
selectedOidList = []
for fid in fidSet.split(";"):
   selectedOidList.append(fid)
print len(selectedOidList)

[ChrisSnyder]

Also, I should mention that if you have to find the selected record count repeated (like in some sort of loop), I would suggest investigating a Python dictionary object. Basically you have to take an initial time hit by loading the table's records into the dictionary, but once it's there.... Holy crap it's fast (light speed and not magnetic speed)!

See http://forums.arcgis.com/threads/8428-Setting-a-Single-Record-to-Selected-with-Python?p=25930&viewfu...