Select to view content in your preferred language

arcpy.da.SearchCursor - sqlprefix

10177
24
Jump to solution
09-08-2014 02:20 PM
ManikMohandas
New Contributor II

Does the sql prefix - DISTINCT work well on SDE tables.

http://resources.arcgis.com/en/help/main/10.1/index.html#//018w00000011000000

Syntax SearchCursor (in_table, field_names, {where_clause}, {spatial_reference}, {explode_to_points}, {sql_clause})

An SQL prefix clause is positioned in the first position and will be inserted between the SELECT keyword and the SELECT COLUMN LIST. The SQL prefix clause is most commonly used for clauses such as DISTINCT or ALL.

My following example is resulting in all rows, not distinct rows.  rs = arcpy.da.SearchCursor(sTableName, (sSelectField1, sSelectField2), sSqlExp, None, None, ("DISTINCT", sSqlPostfix))

  1. What elese could I do to get truely distinct records only.
  2. I store the values in a list and are they any function to get unique values in a list. Example [1,2,2,3,3,3,5,6,7] to [1,2,3,5,6,7]. Also set(xyz) won't work because I'm using a list, whereas set(xyz) works on tuples.
Tags (1)
0 Kudos
24 Replies
ManikMohandas
New Contributor II

Thanks James and Robert.

0 Kudos
ManikMohandas
New Contributor II

@Xander Bakker

What does this do: set("{0}_{1}".format(r[0], r[1]) . Specifically, the curly brackets {} and the need for format(). Is format() mandatory?


Is it possible to breakdown the following into more "bite sized", multi-line code:

unique_list = list(set("{0}_{1}".format(r[0], r[1]) for r in arcpy.da.SearchCursor(FC_or_TBL, (fld_name1, fld_name2, fld_search), where)))


Also, what is the difference between set() and list() do. Can't you do an app

0 Kudos
XanderBakker
Esri Esteemed Contributor

Hi Manny,

My explanation was a little short, sorry for that... and creating these type of "one-liners" doesn't allow for easy interpretation.

Let's break it down into pieces:

# let's say we have a list with duplicate values

list1 = [1, 3, 5, 7, 3, 2, 5, 6, 8, 7, 5, 4, 3, 2, 6, 2]

# the set() is used to convert it to a set of unique values

set1 = set(list1)

print set1

# returns: set([1, 2, 3, 4, 5, 6, 7, 8])

# if you want to convert it to a list again (to loop over it)

list2 = list(set1)

print list2

# returns [1, 2, 3, 4, 5, 6, 7, 8]

A set is very powerful to get a unique list of values and combining sets allows you to obtain values that are in one list but not in the other and info like this. More on this in my post: Some Python Snippets (scroll down to comparing lists)

To create the list some list comprehensions are used (check out this document I posted: Using Python List Comprehensions ).

Let's break down the code:

unique_list = list(set("{0}_{1}".format(r[0], r[1]) for r in arcpy.da.SearchCursor(FC_or_TBL, (fld_name1, fld_name2, fld_search), where)))

The set() and list() at the beginning of the line have been explained above.

Within the brackets of the set() command a lot is going on. Let's not start from left to right, but with the SearchCursor. It takes a table or featureclass, a tuple (or list) of field names and in this case a where clause. This is pretty standard and I don't think needs much explanation. Normally when you use a search cursor you obtain a cursor object which you use to loop through each row (or feature) in the cursor. Next you probably access the fields in each row and do something with it like this:

with arcpy.da.SearchCursor(FC_or_TBL, (fld_name1, fld_name2, fld_search), where) as curs:

    for row in curs:

        val_fld1 = row[0]

        # etc

In case of the list comprehension this is done on one line. It retrieves each row "r" in the SeachCursor and returns this:

"{0}_{1}".format(r[0], r[1])

string.format() allows you to create a string and pass in variables. It is a pretty need way to combine values and string in a certain format

r[0] refers to the value of fld_name1

r[1] refers to the value of fld_name2

{0} refers to the first parameter specified in the format command

{1} refers to the second parameter specified in the format command

... so it the value for fld_name1 is "abc" and the value in fld_name2 would be 1, this would result in the string "abc_1".

I used something similar for creating the where clause:

where = "{0} = '{1}'".format(arcpy.AddFieldDelimiters(FC_or_TBL, fld_search), search_value)

It starts with a string "{0} = '{1}'". The {0} is replaced with the first argument of the format function:

arcpy.AddFieldDelimiters(FC_or_TBL, fld_search)

This will put quotes or brackets or nothing around the field name based on the type of workspace (that can be useful if you don't know if your input is a shapefiles, personal, file or enterprise geodatabase...

The {1} is replace with the value in the variable search_value. Please note that this where clause will fail if your field is not of type TEXT (string). It is possible to extend this functionality and detect the columns type, but for simplicity reasons (read I was to lazy) I did not do that.

If you have any other doubt, please let me know...

Kind regards,

Xander

0 Kudos
ManikMohandas
New Contributor II

Xander Bakker‌. Thank you so very much. That was a good primer on data access, lists, set and string functions. Yes, I was able to work around the limitations of SQL prefix like DISTINCT not working on multiple fields through arcpy.da.SearchCursor. Wish ESRI would further improve "DA".

0 Kudos
JamesCrandall
MVP Frequent Contributor

So... are you confirming that what I mentioned is not possible with Make Query Table?

There's some confusion that this is possible with multiple fields returned.  However, I just don't see how so because a correct SQL statement that returns DISTINCT values on multiple fields simply must have a GROUP BY clause in it.

0 Kudos