topic Re: Loading multiple features from shapefile before processing in Python Questions

Loading multiple features from shapefile before processing

AustinMilt — Sun, 14 Oct 2012 18:36:31 GMT

I want to load multiple features from a shapefile into a list before processing them; processing must come after. However, when I try to do this, I end up with the features all referencing the same object rather than having separate objects. Some code:

# define the input file shapefile = '_myfile.shp'  # import arcpy import arcpy arcpy.overwriteOutput = True  # set the cursor shpName = arcpy.Describe(shapefile).shapeFieldName cursor = arcpy.SearchCursor(shapefile)  # get the features BEFORE doing processing # WHERE THINGS GET FUNKY features = [r.getValue(shpName) for r in cursor]  cursor = None r = None  # to illustrate these are all the same, save them results = [] for i in xrange(len(features)):     fName = '_myfeat%i.shp' % i     results.append(arcpy.CopyFeatures_management(features, fName)

Well, I just tried running my own code and got even worse results. This time, instead of the features all being the same, I had one feature that was completely out of whack, with an infinite inside-out rectangle extending to the left, and all other saved features being empty (but with the same extent?).

I then ran it two more times on the same file, and ArcMap crashed when I tried to load the individual shapefiles into the program.

I then ran it again on another shapefile and got similar behavior, where one feature from the shapefile was present, and all the others are empty.

Any clues as to how to amend this? The major problem is that I am processing the features after loading all of them into Python. If I copy the features to their own shapefiles as soon as I load them, then the problem goes away. However, I'd like to avoid making a separate shapefile for each and every feature due to I/O time and file sizes.

Re: Loading multiple features from shapefile before processing

T__WayneWhitley — Sun, 21 Oct 2012 00:43:58 GMT

I think I too have seen this before and it isn't so intuitive from the online samples - and I may not explain it well, but I'll give it a shot.
The cursor row object is a reference, a 'pointer' if you will...so when you go to the next row, it's actually referencing the next geometry. (At least that's how I understand it.)

Methods have changed for 10.1 and I'm still at 10, so I cannot test this - however, I think it's a little more straightforward at 10.1 so I think it's easier to show that 1st. I think you need the object reference like this for the list you seem to be after (this should only store a list of the geometry objects):

features = [r[0] for r in arcpy.da.SearchCursor(shapefile, ("SHAPE@"))]

The 10.1 documentation actually states a geometry object is returned using the '@' token. Also notice the use of the da (data access module), supposedly a performance enhancement, although I am not sure it is efficient to load the geometry as you wish into a list - better test it out.

For 10.0, which I am a little more familiar with, but haven't tested this at length enough to say it'll work, I think you have to 'load' the geometry into a separate object you create, as in the script excerpt shown (this is not what I wrote, but a webhelp sample):

########################################
# Create an empty Geometry object
#
g = arcpy.Geometry()

# Run the CopyFeatures tool, setting the output to the geometry object. GeometryList
# is returned as a list of geometry objects.
#
geometryList = arcpy.CopyFeatures_management("c:/temp/outlines.shp", g)
########################################

I didn't know CopyFeatures could be used directly - that a cursor is not necessary to do exactly what you want (although I suppose you'll be opening a cursor to then write further output). This short script in its entirety can be found here (at the bottom of the page):

Using geometry objects with geoprocessing tools
Resource Center » Professional Library » Geoprocessing » Geoprocessing with Python » Accessing geographic data in Python
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Using_geometry_objects_with_geoprocessing_tools/002z0000001z000000/

Re: Loading multiple features from shapefile before processing

MikeHunter — Fri, 10 Dec 2021 23:18:38 GMT

Try this:

features = [r.Shape for r in cursor]

I tried this and it works just fine. However, I am using 10.1 (but I didn't use da.searchcursor). I suspect it will work on 10.0.

Also, you can use the in_memory workspace to make a temporary featureclass instead of writing out a shapefile.

Mike

Re: Loading multiple features from shapefile before processing

AustinMilt — Tue, 30 Oct 2012 13:52:52 GMT

I think I too have seen this before and it isn't so intuitive from the online samples - and I may not explain it well, but I'll give it a shot.
The cursor row object is a reference, a 'pointer' if you will...so when you go to the next row, it's actually referencing the next geometry. (At least that's how I understand it.)

Methods have changed for 10.1 and I'm still at 10, so I cannot test this - however, I think it's a little more straightforward at 10.1 so I think it's easier to show that 1st. I think you need the object reference like this for the list you seem to be after (this should only store a list of the geometry objects):

features = [r[0] for r in arcpy.da.SearchCursor(shapefile, ("SHAPE@"))]

The 10.1 documentation actually states a geometry object is returned using the '@' token. Also notice the use of the da (data access module), supposedly a performance enhancement, although I am not sure it is efficient to load the geometry as you wish into a list - better test it out.

For 10.0, which I am a little more familiar with, but haven't tested this at length enough to say it'll work, I think you have to 'load' the geometry into a separate object you create, as in the script excerpt shown (this is not what I wrote, but a webhelp sample):

########################################
# Create an empty Geometry object
#
g = arcpy.Geometry()

# Run the CopyFeatures tool, setting the output to the geometry object. GeometryList
# is returned as a list of geometry objects.
#
geometryList = arcpy.CopyFeatures_management("c:/temp/outlines.shp", g)
########################################

I didn't know CopyFeatures could be used directly - that a cursor is not necessary to do exactly what you want (although I suppose you'll be opening a cursor to then write further output). This short script in its entirety can be found here (at the bottom of the page):

Using geometry objects with geoprocessing tools
Resource Center » Professional Library » Geoprocessing » Geoprocessing with Python » Accessing geographic data in Python
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Using_geometry_objects_with_geoprocessing_tools/002z0000001z000000/

That worked! I wont claim to understand why, but I greatly appreciate the help.

It's great to know you can have arcpy write to in-memory variables using a predefined Geometry object. I will be using that in the near future.

Re: Loading multiple features from shapefile before processing

AustinMilt — Fri, 10 Dec 2021 23:18:41 GMT

Try this:
features = [r.Shape for r in cursor]
I tried this and it works just fine. However, I am using 10.1 (but I didn't use da.searchcursor). I suspect it will work on 10.0.

Also, you can use the in_memory workspace to make a temporary featureclass instead of writing out a shapefile.

Mike

Your solution didnt work. I am using ArcMap 10.0, which may be the problem. I will have to look into the in_memory workspace, which I didnt know existed. Thank you.