Select to view content in your preferred language

Script to contatenate the same field from several records

2254
14
09-26-2014 05:37 AM
JoseSanchez
Frequent Contributor

Hi all,

I am looking for a script, preferably a Python script that reads all the records from a file and concatenates them in a result variable.

I am using ArcMap to "Select By Attributes" and I would like to have a way to format a list of IDs I am importing form a text file, or Ms Excell list. To build a Definition Query in the Query Builder window.

Input: (from xxx.txt)

341440

341444

341521

341534

341580

341664

341783

Output: (to zzz.txt)

ID=’341440’ OR ID= ‘341444’ OR ID= ‘341521’ OR ID= ‘341534’ OR ID= ‘341580’ OR ID= ‘341664’ OR ID= ‘341783’

Thanks

0 Kudos
14 Replies
curtvprice
MVP Alum

Here's how I do it. I have a function as this something I find myself doing a lot! This function works in 10.0 and 10.1. It reads a table, not a text file.  (If you add a field name to the top of your text file, and name the file with a .csv or .txt file extension you can read it directly as a table in ArcGIS.)

def ListUnique(inTable, Field):

    """Create a list of unique values from a table/tableview.

    arguments

    inTable  Table or table view

    Field    Field name

    """

    try:

        try:

            # this will only work for 10.1

            import arcpy.da

            lstValues = [r[0] for r in

                    arcpy.da.SearchCursor(inTable, Field)]

        except ImportError:

            # 10.0 version

            try:

                Rows = arcpy.SearchCursor(inTable, "", "", Field, Field)

                Row = Rows.next()

                lstValues = []

                while Row:

                    lstValues.append(Row.getValue(Field))

                    Row = Rows.next()

            except:

                raise

            finally:

                del Row, Rows

        # unique-ize and sort the list

        lstValues = sorted(set(lstValues))

        return lstValues

    except Exception as msg:

        raise Exception("ListUnique failed\n" + str(msg))

Once you have the list of values, you can make this into a SQL expression from a list like this:

numlist = ",".join(str(L))

strlist = ",".join(["'{}'".format(vv) for vv in L]) # need quotes for string values

where = '"{}" IN ({})'.format(field, numlist) # "FIELD" IN (1,2,3)

James wanted me to use the FIELD = 1 OR FIELD = 2 syntax, so here's a way to do that (this replaces line 03 above):

where = " OR ".join(['"{}" = {}'.format(field, k) for k in strlist])

JamesCrandall
MVP Alum

Keep in mind that the "IN" statement does have limits (depending upon database).  I think Oracle limit is 1000?  Not exactly sure but I recently had to break up my list of values that built my IN portion of a WHERE clause.

curtvprice
MVP Alum

I am not very familiar with "real" RDBMS's so I'm wondering what string length fails if you use the more basic form

FIELD = 1 OR FIELD = 2 or FIELD = 3 ...

For 1000 fields that is at least 14K of text!

I would guess that RDBMS would not support  indexes when you use the IN operator - but in theory that wouldn't be a problem with many in the list because IN would parallelize the comparisons (ie all comparisons would be done on each record, in one pass).

0 Kudos
JamesCrandall
MVP Alum

Curtis,

In RDBMS-landia, if I have >1000 values I'd probably insert them into a temporary table of the values for my IN operator and setup a JOIN to the main table I am interested in querying.  I know that doesn't specifically address your post but is likely the best solution.  Or if I were in SQL Server I'd likely opt for the Table variable instead of a Temp Table.

curtvprice
MVP Alum

Yeah - I've heard this type of join to select rows called a "filter".

There used be an awesome ARCPLOT command  that I still miss. (Those were the days.)

reselect <dataset> <dstype> keyfile <table> <field> keyitem <key_field>

I've tried to reproduce this functionality using layers and the Add Join tool but found it slow and unreliable.

Maybe the Make Query tool... or a python solution using sets....