Latest Contributions by MarcNakleh

Hello Anthony, This might be because you are calling many modules that must be loaded the first time you run a Python script (as discussed in Section 6.1.3 on this page from Python.org.) The longer load-up on the first run is actually generating a compiled PYC file to make it easier and faster to load up data the second time. I ran the following code on my computer, to demonstrate: import arcpy import timeit arcpy.env.overwriteOutput = True rrn = r'C:\xxx\RRN_Line.shp' osm = r'C:\xxx\OSM_Line.shp' rrn_layer, osm_layer = 'temp1', 'temp2' def main():     arcpy.MakeFeatureLayer_management(rrn, rrn_layer)     arcpy.MakeFeatureLayer_management(osm, osm_layer)     arcpy.SelectLayerByLocation_management(rrn_layer, 'INTERSECT', osm_layer) t = timeit.Timer("main()", "from __main__ import main") print t.timeit(1) and got the following results after running it a few times: 2.9045302362 1.4424658839 1.45464152827 The thing is, this happens regardless of the content of your code, anytime you need to import modules. import os import timeit def main():     x = os.getcwd() t = timeit.Timer("main()", "from __main__ import main") print t.timeit(1) gives results like this: 8.8015191422e-06 6.2867993873e-06 6.30113935691e-06 You should notice that same time discrepancy the first time you run any Python application in a newly-loaded environment, after the addition of a module import, or in any instance where the PYC file for your current session hasn't been created. Hope this helps! Marc ... View more

Hello Johan, I think that the main issue here is FID's field type: it is technically of type 'Object ID' which is queried like an integer field. So a query referencing it would look like: '"FID" = 3' without any additional quotes (in contrast to string fields, where you would need those quotes.) The examples on ArcGIS' Select Analysis page seem to all be referencing string fields, unfortunately. The Building SQL Queries page has more info on how to structure queries, and features a section dealing with searching for numbers. I hope this helps! Marc ... View more

Hello there, I don't know if this is just a rendering thing, but your code seems to have unicode quotes (�??�?�, a.k.a. \201C, \201D), as opposed to regular quotes (""), which may cause some issues when using arcpy. I've had some minor issues in the past using these in larger scripts and tools.... In either case, would it be possible to see the exact code you are using? It's sometimes hard to know whether there's something hiding behind metacode. A description of how to use arcpy.MakeQueryTable_management (though with aliases) can be found on ESRI's usage page for Make Query Table A big issue to note is that your WHERE clause is being used to filter values but also to define the JOIN parameters between your two tables. One of the examples on ESRI's page has a WHERE clause that looks like: whereClause = "vtest.COUNTIES.FIPS = vtest.CODEMOG.Fips and vtest.COUNTIES.STATE_NAME = 'California'" that both joins on COUNTIES.FIPS = CODEMOG.Fips, but also filters based on STATE_NAME = 'California' So, unless I'm mistaken, your WHERE would need to look roughly something like this whereClause = "A.number = B.number and A.subnumber = B.subnumber and B.number IS NULL" which should return the rows where there is no B.number matching that particular A.number (i.e. A's value is not present in B.) However, that's assuming it acts as a Left or Full Outer Join. If not, could compare the values that are still present in A.number with your original values in A.number using a Search Cursor, or some other means. Also worth checking: this older thread on the same topic. I hope this helps! Marc ... View more

Interestingly, they also take about the same time! (I used a folder with a lot of subfolders and a lot of PDF's) def main1():     x = []        for pdf in filter(lambda x: x.endswith('.pdf'), DirectoryWalker(search_dir)):         x.append(pdf) def main2():     y = []     for pdf in rglob(search_dir,'*.pdf'):         y.append(pdf) from timeit import Timer t1 = Timer("main1()", "from __main__ import main1") t2 = Timer("main2()", "from __main__ import main2") print("DirectoryWalker: " + str(t1.timeit(100))) print("rglob: "                  + str(t2.timeit(100))) DirectoryWalker: 155.908681642 rglob:                 162.25917093 ... View more

That.... is a thing of beauty!!! Thank you so much for sharing that, Luke! I just want to say that in the interest of giving the poster a smaller bit of code to dissect, one could make a smaller version using a filter and the exact example you were inspired by on the OS example page. import os class DirectoryWalker:       '''Callously stolen (with attribution!) from os-path-walk-example-3.py       Copyright © 1995-2010 by Fredrik Lundh       http://effbot.org/librarybook/os-path.htm'''           def __init__(self, directory):         self.stack = [directory]         self.files = []         self.index = 0     def __getitem__(self, index):         while 1:             try:                 file = self.files[self.index]                 self.index = self.index + 1             except IndexError:                 # pop next directory from stack                 self.directory = self.stack.pop()                 self.files = os.listdir(self.directory)                 self.index = 0             else:                 # got a filename                 fullname = os.path.join(self.directory, file)                 if os.path.isdir(fullname) and not os.path.islink(fullname):                     self.stack.append(fullname)                 return fullname import shutil search_dir = r'C:\Project' out_dir = r'C:\Workspace' # This filter takes all the results from DirectoryWalker and only iterates # through those that match the condition 'file.endswith('.jpg') for jpg in filter(lambda x: x.endswith('.jpg'), DirectoryWalker(search_dir)):     print('Copying: ' + jpg)     shutil.copy(jpg,out_dir) Diana: If you need any help understanding either of these examples, feel free to ask. Cheers, Marc ... View more

Hi Chris, No problem at all. Python's website is chock-full of examples you can use as references for good coding (that's how I figured a lot of stuff out!) When it comes to your issue of disparate errors, I'd need a bit more information: could you give a specific example of the errors? When you say you are "programming python" (vs. "using IDLE"), what programming environment are you programming in (i.e. what is the program sending you errors?) Is it PythonWin or some other editor? Cheers, Marc ... View more

Hi there, The files within the Python Standard Library have some pretty good examples of how to lay out code (especially code metadata) in a module. If not, I've always enjoyed this page's take on structuring code. I try and set up my own code as follows: #!/usr/bin/env python ############################################################################### '''TITLE    ------    Description''' ############################################################################### import os import sys import arcpy __author__       = "Marc Nakleh" __copyright__   = "TrakMaps 2011" __version__      = "1.0" #----------# def main(): #----------#     # Code goes here     pass ################## if __name__ == '__main__':     main() I have an unhealthy fixation on formatting, though! I like your idea of a local set-up. I'd just mention that there are a lot of reasons to be running a module locally, testing (which would require the parameters you are querying the user for) being a good one. ... View more

Hi Henry, Agreed on all points. To be honest, I'm not 100% sure whether the os or arcpy modules could do much of what you're wondering about. My money is on the need to use more in-depth analysis modules (or the associated programs themselves) to find out this kind of metadata. I don't even know if these would be part of Python's standard library, though I can't say for sure. Your best bet would be to look around online for GIS modules developed in Python for Image Analysis or File management. Cheers, Marc ... View more

Hi Darren, No problem at all! The only thing I would add is that the Field Calculator works a bit differently from the other environments you mentioned. The Field Calculator itself actually functions as an Update Cursor, so there's no need to scroll through values one row at a time: Field Calculator will do that for you! In fact, the code would look much simpler (though there is usually a need to separate the bulk of the code into a function under Code Block so as not to clutter the one-line Expression field) ######## # GPIN2 = # ######## compile_GPIN(!XCentroid!, !YCentroid!) ######################## # Code Block / Pre-Logic Script Code# ######################## def compile_GPIN(XCentroid, YCentroid):        strX = str(XCentroid)     strY = str(YCentroid)     GPIN = (strX[2] + strY[1] + strX[3] + strY[2] + "-" +                  strX[4] + strY[3] + "-" +                  strX[5] + strY[4] + strX[6] + strY[5])     return GPIN The !XCentroid! and !YCentroid! passed to the compile_GPIN function keep sending the appropriate values for those fields, one row at a time. And, for each row, the 'return' sends the GPIN value back to the field to be written, one row at a time. Best of luck in the rest of your program! Marc ... View more

Asteriskes can also be used as a wildcard in arcpy's List methods (ex. ListFields, ListFeatureClasses, ListDatasets) to expand your searches For example, arcpy.ListFields('line*') will return any field that starts by "line". Also useful if you're not too sure whether you're in a feature class or a shapefile. For example, you can use arcpy.ListFeatureClasses('Hospital*') to return a feature class named 'Hospital' but also 'Hospital.shp'. Chris: Thanks for the code! I'm actually thinking of implementing something similar as a kind of value printer for tables. A couple of quick thoughts: 1) Python usually encourages programmers to use very targeted try...except calls. Read up on exception handling on this page. For example, your main try...except encapsulates the entire program, and will catch (and hide) all errors, even errors you might not have expected in your code itself! You could wrap only the sections that might explode (like the Create Table) in exception handlers, and even try to catch specific errors (ESRI favours catching arcpy.ExecuteError and printing them out with arcpy.GetMessages()) 2) You might also want to look at finding ways to represent repetitive code by some of Python's built-in data structures, like dictionaries and lists (explained more here.) For example, you could replace the following:     fieldName1="field_name"     fieldName2="aliasName_"     fieldName3="baseName_"     fieldName4="domain_"     fieldName5="isNullabl_"     fieldName6="precision_"     fieldName7="required_"     fieldName8="scale_"     fieldName9="type_"     fieldName10="length_"     fieldName11="editable_"     fieldName12="Table_Name"     fieldLength=255     # Execute AddField new fields. All fields are set as text even if they contain numbers.     arcpy.AddField_management(inFeatures, fieldName12, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName1, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName2, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName3, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName4, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName5, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName6, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName7, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName8, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName9, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName10, "TEXT", "", "", fieldLength)     arcpy.AddField_management(inFeatures, fieldName11, "TEXT", "", "", fieldLength) by the following:     fieldLength=255     field_names = ["field_name", "aliasName_", "baseName_", "domain_", "isNullabl_",                             "precision_", "required_", "scale_", "type_", "length_", "editable_"]     for field_name in field_names:         arcpy.AddField_management(inFeatures, field_name, 'TEXT', '', '', field_length) Though they won't work everywhere, lists are a great way of consolidating your code and often improving legibility. Cheers! Marc ... View more

hahaha this might be because I gave you sloppy code. Apologies! Here is the proper code that I have just retested (but with different variable values.) The changes I made are indicated by comments import arcpy import os #set local variables inFeatures = 'Feature Class' XCentroid = 'xCentroid' YCentroid = 'yCentroid' tableField = 'GPIN2' #Load values into Variables scurfc = arcpy.UpdateCursor(inFeatures)                                        # UpdateCursor needs to be called, not SearchCursor for sf in scurfc:     strX = repr(sf.getValue(XCentroid))     strY = repr(sf.getValue(YCentroid))     GPIN = (strX[2] + strY[1] + strX[3] + strY[2] + "-" +                  strX[4] + strY[3] + "-" +                  strX[5] + strY[4] + strX[6] + strY[5])     #Update values and delete variables     sf.setValue(tableField, GPIN)                                                      # setValue and updateRow need to be called IN the loop     scurfc.updateRow(sf) del tableField, scurfc, strX, strY, XCentroid, YCentroid, GPIN          # X and Y no longer exist, so there shouldn't be an attempt to delete them. I tend not to use repr() as it's a bit less common and it has some funky properties in limit cases (like returning hex values for non-Unicode strings) but you are right: it does do the job here. In answer to your questions: setValue is called inline of the loop cycling through the rows (which is the for sf in scurfc line.) The process is usually as follows for updating values based on other values in a table: 1) Cycle through rows, one by one 2) Get input from the value of a given column on the current row (like the sf.getValue(XCentroid) line in the code) 3) Do calculations to get the output value for the current row (like your GPIN = expression line) 4) Write value to the output field of the current row (this is the sf.setValue(tableField, GPIN) line) 5) Ask the table to update the current row so that the values are written to the actual table (this is the scurfc.updateRow(sf) line) 6) Rinse, repeat As you can see, these steps need to be done for each row, which is why everything is inside the "for sf in scurfc" loop. There's no need to loop a second time, or write the values to an array, as the values are updated directly into the table, one row at a time. Cheers, Marc ... View more

Hi Stacy, Thanks for the correction! ... View more

hmmm... that's an interesting question! I've wondered the same thing and have so far not had any luck when it comes to finding a built-in function in arcpy to get the size of a feature class or shapefile. If you were cycling through the various shapes in a path, I would have gone with something using list comprehensions like: for shp in arcpy.ListFeatureClasses():     total_size = sum([os.path.getsize(x) for x in glob.glob(os.path.join(path, shp[:-4]) + '*')]) or, cleaning it up a bit: for shp in arcpy.ListFeatureClasses():     mask = os.path.join(path, shp[:-4]) + '*'     total_size = sum([os.path.getsize(x) for x in glob.glob(mask)]) But it still feels sloppy to me, and it doesn't work for feature classes or datasets (only shapes.) Still, I hope this helps! ... View more

Hi there, There are a couple of ways to do this, though Python's built-in way would be: os.path.dirname(path_to_dataset) import os pathX = r'C:/Hello/HelloWorld' print os.path.dirname(pathX) Result: C:/Hello which returns the parent directory of the file or folder under consideration. So, if you feed it a file, it will give the directory the file is in. If you give it a folder, it will return the folder hosting it. ... View more