How to create summary statistics for each iterated geoprocess in Modelbuilder?

[AlejandroBlei]

In a nutshell, the summary statistics table generated by my combination of the Feature Selection Iterator and the Select Layer by Location should have as many rows as there are features (in my case 25). Unfortunately, my resulting table only has one row. Please help me diagnose where I have gone wrong! I am using the ModelBuilder in ArcMap 10.1. The image below sets the stage:

I have a feature layer of 25 buffers, which are the light blue circles you see. Let’s call each buffer a community. Many buffers overlap. Each buffer has a unique ID.

I also have a feature layer of points (centroids). Some of the centroids fall within certain buffers, some fall outside. Every centroid has columns of information associated with it, two of which are important to me: Origins and Destinations.

WHAT I WOULD LIKE TO DO:

(1) I want to iterate through each feature in the buffer layer (loop through the buffers one-by-one) and ask: which centroids are completely within this buffer? (2) Then, I want to ask: of the centroids that fall within the buffer, what is the sum of the origins and the sum of the destinations? (3) Last, I want to combine the results of (1) and (2) in a table with three columns of information: The buffer ID, the sum of origins in that buffer, and the sum of destinations in that buffer.

MY SETUP and MY PROBLEM

SETUP

• I am using the Feature Selection Iterator in ModelBuilder to loop through the buffer file. I have chosen to group by the BufferID field.
• I use the Select Layer by Location tool to select the centroids that are ‘completely within’ each iterated feature in the buffer file. (The buffer is the selecting feature, the centroids are the input feature layer)
  • The output of this geoprocess, labeled Centroids(2) in the image below, is, I believe, the collection of centroids ‘completely within’ a particular buffer.

• I use the Summary Statistics tool to calculate the sum of the Origins and Destinations associated with Centroids(2). In the Summary Statistics tool interface, I select Origins and Destinations as statistics fields and I calculate their sum. The output of this tool is a table. I select the location of this output table.

PROBLEM

• This is where I am stuck. If I run the model above, I get this:

• This is interesting, because I see the tools turning to red as many times as there are buffers – so I believe that the model is iterating through each feature in the buffer layer, selecting centroids by location, and creating summary statistics.
• Clearly, though, I have not set up the model properly to get the desired output because the resulting table has only one row. There should be as many rows as there are buffers, which in this case is 25.  Moreover, I would like the OBJECTID* in the above image to be the BufferID from the buffer layer. When I grouped the Feature Selection Iterator by BufferID I thought this might carry over to the Summary Statistics; it did not.
• Although the model is iterating 25 times (as evidenced by the tools turning red 25 times when I run the model) the output it generates is for one iteration only. What do I have to change in order to get: (A) a table with 25 rows, and (B) an ID that corresponds to the BufferID?

My sincere thanks for your help and suggestions. 

[RiyasDeen · Accepted solution]

Hi Alejandro,

You don't have to use iterator for this. Below steps should work.

1. Intersect your buffer FC with your centroid FC with join attributes all and output type point. ArcGIS Help 10.1
2. Do summary statistics with your buffer ID as case field and origins and destinations as destination field with statistics type sum ArcGIS Desktop

This will give you sum of origin and destination for all your buffer IDs.

[AlejandroBlei]

Wow - Riyas I tried your solution and it does the trick! Thank you! I really appreciate that your solution helps me me think about geoprocessing problems in new ways.

One more question, if I may. The above question dealt with points inside circles. What if the features in question were hundreds of lines of different lengths scattered about, and I wanted to determine the lines that were completely within individual buffers? This is my next problem. See below:

I tried variations of your solution, but I only want lines that are completely within buffers. Again, the goal is to sum information associated with these lines (in this case, a flow value). If a line crosses two buffers, the intersect solution will tell me that this line belongs to two buffers. In reality, for the purpose of this analysis, a line that crosses two buffers is not useful, I only want to know and sum the information for lines completely within individual buffers.

Is there a way to do this without the feature selection iterator? This problem is similar to the one you answered, only slightly different.

Again, thanks very much!

[RiyasDeen]

Hi Alejandro,

In this case you can use spatial join. ArcGIS Desktop

1. Target feature will be your line features, join features will be your buffer feature, join operation 1 to many, join type keep common, match option COMPLETELY_WITHIN

2. From the output run summary statistics on buffer ID and sum of origin and destinations.

[RichardFairhurst]

Based on the way you have described what you need, virtually every line crosses the boundary of another buffer given the amount of overlap you are showing for the buffers.  Riyas last solution works if you are only concerned with lines being completely within a single buffer and do not deal at all with lines crossing the boundary of another buffer.

If boundary crossing between separate buffers is a real concern then I would calculate the original line length into a double field and then perform an Intersect.  Only lines that end up with a final length equal to their original length in the double field were completely within only one buffer and did not cross the boundary of any buffer.  Performing Summary Statistics should confirm that only that set of lines meets the criteria you described.

Riyas solution is the one you should use if you did not mean to say "a line that crosses two buffers is not useful" and instead meant to say "a line that cross a buffer's boundary is not useful".

[AlejandroBlei]

Riyas and Richard, many thanks for your replies. Re: Richard, indeed, what I meant to say was "a line that crosses a buffer's boundary is not useful/important."

Therefore Riyas's solution should work. Unfortunately I receive an error message that "arcgis has encountered a serious application error" when I try the spatial join (lines = join features; buffers = target features, join operation = one-to-many, match option = completely contains). I've tried switching 'background processing' (Geoprocessing > Geoprocessing options) on and off, but that doesn't help, I always receive an error message and ArcMap crashes.

It is worth mentioning that he visual examples I've provided above are small pieces of the larger dataset I'm dealing with. I don't think the size of the data I'm working with should cause the computer to crash: in all, there are 599 buffers and 2,895 lines. I'm a bit confused about the error message and plan to call ESRI tech support to see if they can be of any help.

Since the spatial join does not work (at least for the moment, until I can resolve the error message issue) I've gone back to the Modelbuilder exercise at the very top. I got it to work, although I must say it is a rather inefficient method to get the information I want. To the output of the Summary Statics GP tool I added an 'add field' tool and named it BufferID, then I added a 'calculate field' and drew an arrow from the light blue 'Value' oval and made this the expression. Then I added the 'append' tool where the target dataset was a Table I had to create separately.

For the 599 buffers and 2,895 lines, this model took 34 minutes 45 seconds. In other words, about 3.48 seconds on average per iterated feature (3.48 seconds * 599 iterated features = 34.75 minutes). If I could do a spatial join, I am sure the process would run in under 2 minutes! Frustrating...

I may try Richard's solution, which I find quite inventive, to cut down on the processing time.

Thanks again!

[RichardFairhurst]

Given that you only meant that you wanted to determine if the lines were completely within a single buffer, my solution is only partially what you need to do.  In addition to the steps I outlined you would need to complete the process by performing a Dissolve of the Intersected line features using the BufferID as the Dissolve case field.  The calculated length double field would be a summary field and you would just preserve the original value using FIrst, Last, Min or Max (not Sum).  If the final dissolved lines have the same length as the calculated FIRST_LENGTH field value, then it was completely contained inside that BufferID.  You may need to allow for a very small fractional difference in lengths, because by cutting the lines up some minor adjustments to the line geometry will have occurred based on your tolerance and resolution settings.  I typically use an SQL formula like the following for inexact value numeric tolerances:

ABS(SHAPE_LENGTH - FIRST_LENGTH) <= .001

The Field Calculation, Intersect and Dissolve should certainly all process in less than 5 minutes and perhaps less than 2 minutes.