Hi all,
I am looking for a script, preferably a Python script that reads all the records from a file and concatenates them in a result variable.
I am using ArcMap to "Select By Attributes" and I would like to have a way to format a list of IDs I am importing form a text file, or Ms Excell list. To build a Definition Query in the Query Builder window.
Input: (from xxx.txt)
341440
341444
341521
341534
341580
341664
341783
Output: (to zzz.txt)
ID=’341440’ OR ID= ‘341444’ OR ID= ‘341521’ OR ID= ‘341534’ OR ID= ‘341580’ OR ID= ‘341664’ OR ID= ‘341783’
Thanks
Here's how I do it. I have a function as this something I find myself doing a lot! This function works in 10.0 and 10.1. It reads a table, not a text file. (If you add a field name to the top of your text file, and name the file with a .csv or .txt file extension you can read it directly as a table in ArcGIS.)
def ListUnique(inTable, Field):
"""Create a list of unique values from a table/tableview.
arguments
inTable Table or table view
Field Field name
"""
try:
try:
# this will only work for 10.1
import arcpy.da
lstValues = [r[0] for r in
arcpy.da.SearchCursor(inTable, Field)]
except ImportError:
# 10.0 version
try:
Rows = arcpy.SearchCursor(inTable, "", "", Field, Field)
Row = Rows.next()
lstValues = []
while Row:
lstValues.append(Row.getValue(Field))
Row = Rows.next()
except:
raise
finally:
del Row, Rows
# unique-ize and sort the list
lstValues = sorted(set(lstValues))
return lstValues
except Exception as msg:
raise Exception("ListUnique failed\n" + str(msg))
Once you have the list of values, you can make this into a SQL expression from a list like this:
numlist = ",".join(str(L))
strlist = ",".join(["'{}'".format(vv) for vv in L]) # need quotes for string values
where = '"{}" IN ({})'.format(field, numlist) # "FIELD" IN (1,2,3)
James wanted me to use the FIELD = 1 OR FIELD = 2 syntax, so here's a way to do that (this replaces line 03 above):
where = " OR ".join(['"{}" = {}'.format(field, k) for k in strlist])
Keep in mind that the "IN" statement does have limits (depending upon database). I think Oracle limit is 1000? Not exactly sure but I recently had to break up my list of values that built my IN portion of a WHERE clause.
I am not very familiar with "real" RDBMS's so I'm wondering what string length fails if you use the more basic form
FIELD = 1 OR FIELD = 2 or FIELD = 3 ...
For 1000 fields that is at least 14K of text!
I would guess that RDBMS would not support indexes when you use the IN operator - but in theory that wouldn't be a problem with many in the list because IN would parallelize the comparisons (ie all comparisons would be done on each record, in one pass).
Curtis,
In RDBMS-landia, if I have >1000 values I'd probably insert them into a temporary table of the values for my IN operator and setup a JOIN to the main table I am interested in querying. I know that doesn't specifically address your post but is likely the best solution. Or if I were in SQL Server I'd likely opt for the Table variable instead of a Temp Table.
Yeah - I've heard this type of join to select rows called a "filter".
There used be an awesome ARCPLOT command that I still miss. (Those were the days.)
reselect <dataset> <dstype> keyfile <table> <field> keyitem <key_field>
I've tried to reproduce this functionality using layers and the Add Join tool but found it slow and unreliable.
Maybe the Make Query tool... or a python solution using sets....