nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Find features around a pipeline for every 1 Km segment, within a buffer

ZeeshanAhmad2

I need to perform following operations.

Draw a buffer around a pipeline (Network Data)
For each 1 Km pipeline segment, split the buffer polygon
For every buffer polygon segment, run query on some layer and find how many features are present in that area

Problem is with the 2nd operation. I am not sure how to achieve this. Please see a screenshot of illustration of what I want. End result will be to show for each segment of pipeline, how many features are present on both sides of pipeline within the buffer.

What tools/commands can be used for this. I eventually want this functionality as a GP service.

buffer_pipeline.png

dipper hi.jpg

Trouver plus de publications taguées avec

Accepted answers

RichardFairhurst

Now that I am trying it I see I guessed a few settings wrong and missed a few steps.

The Excel table should be exported to a file geodatabase to create ObjectIDs and to make it possible to recalculate values. All other exports should be to a file geodatabase.

I applied the RIGHT offset direction option to the Make Route Event Layer tool to make the side offsets relative to the digitized direction of the line. Also, the angles created by the Make Route Event Layer tool are Arithmetic angles, meaning they increase in a counterclockwise direction that start at the east, while the Bearing Distance to Line tool uses Geographic angles which increase in a clockwise direction and start at the north. To correct for that difference, calculate a new angle field called NORTH_ANGLE with this VB Script equation (use the Script codeblock check box):

Parser: VB Script

Use codeblock: checked

Pre-logic Script codeblock:

North_Angle = ABS([LOC_ANGLE] - 360) + 90 If North_Angle >= 360 Then   North_Angle =  North_Angle - 360 End If

NORTH_ANGLE (the field name for the output expression): North_Angle

Export the event points to a real point layer to make calculation faster. Add two double fields to the points for the X and Y Coordinate of each side offset point and calculate those fields with a Python calculation using !Shape.Centroid.X! or !Shape.Centroid.Y!

That point feature class should be the input to the Bearing Distance to Line tool and the NORTH_ANGLE field should supply the bearing.

I have attached a picture of results I was able to create.

BufferCut.jpg

All comments

RichardFairhurst

I would make the pipelines LR Routes with M values based on length in km. (LR is ideal for all things line.)

Create a table in Excel with 4 columns:
1) The pipeline Route ID which will be the same value for every row generated for the next column.
2) A sequential number column for km along the line beginning at 1 and using as many rows that it takes to reach the highest km value for your longest line.
3) A column for side distance offset values that are about the negative double of the length of your buffer width. This column value is the same for every row generated for column 2.
4) A column of values equal to double the absolute value of column 3 to use as a distance to cut across the buffer polygon. This column value is the same for every row generated for column 2.

Use the spreadsheet with the Make Route Event Layer tool to make an LR point event table with the options that apply a side offset and that create an autogenerated angle field that is normal to the pipeline. You should also generate an Event error field so that you can select the set of point that actually fell on the line, which will have a value of 'NO ERROR'. The points will fall outside the buffer so that the line you generate next will be sure to fully cut across the buffer. The table is a template that would just need to change the route ID to work on any given pipeline. You should be able to generate a complete table that appends every pipeline Route ID to the last 3 columns using the Make Query Table tool to join the pipeline routes and the template table.

Use that layer as the input to the Bearing Distance to Line tool to create the lines that cut across the pipeline buffer. Because the offset was negative the autogenerated normal angle will work as a bearing that will cut across the buffered line intersecting the pipeline at every 1 km position on the line.

Finally split the buffer with the lines created by the Bearing Distance to Line tool using the Feature to Polygon tool. This needs to be processed with an iterator that selects/filters only the related set of cut lines to just one pipeline Route ID at a time to avoid unwanted buffer and cut intersections. The iterator would also process each separate cut up pipeline buffer one at a time to perform the intersect against the other layers you want to use in your process to avoid excessive polygon creation due to intersections with other pipeline buffers.

ZeeshanAhmad2

Create a table in Excel with 4 columns:
1) The pipeline Route ID which will be the same value for every row generated for the next column.
2) A sequential number column for km along the line beginning at 1 and using as many rows that it takes to reach the highest km value for your longest line.
3) A column for side distance offset values that are about the negative double of the length of your buffer width. This column value is the same for every row generated for column 2.
4) A column of values equal to double the absolute value of column 3 to use as a distance to cut across the buffer polygon. This column value is the same for every row generated for column 2.

Thank you Richard for your help. Can you please share a sample Excel file?

RichardFairhurst

Here is one. The Route_ID, Side_Dist and Dist_Along are an arbitrary values in this case. They should change to be a a real pipeline Route_ID with distances relative to the buffer width you use.

Template.xlsx

RichardFairhurst

North_Angle = ABS([LOC_ANGLE] - 360) + 90 If North_Angle >= 360 Then   North_Angle =  North_Angle - 360 End If

BufferCut.jpg

ZeeshanAhmad2

Thanks alot Richard. That was really helpful. Really appreciate it.
I am able to generate the buffer segments at my end. Now I am working on automating it through model/script.

RichardFairhurst

Use the attached template table instead. I adds two sets of From and To fields for line segments. One set is the full length of the segment between points on the line. The second if 1 meter short of that length at both ends. Use the Make Route Event Layer with the FROM_M_2 and TO_M_2 fields and then Spatially Join those segments to your cut up buffers. Because the lines will not touch any segment but the one they are within the Spatial Join should assign the attributes of only one segment to the buffer portion. Then you will know what linear portion of the pipeline each buffer segment represents.

The Spatial Join must occur after the buffer is segmented and before you intersect the buffer segments with any other layers.

RichardFairhurst

Sorry. Here is the attached template spreadsheet.

Template.xlsx

ZeeshanAhmad2

Ok. Thanks. I will try this template.
Currently I have added all the processes in a model but the problem I am facing is that Bearing Distance To Line tool generates only one line whereas when I run this tool manually on same output from Make Route Events Layer it works fine. Can you suggest what is the issue here?

RichardFairhurst

Ok. Thanks. I will try this template.
Currently I have added all the processes in a model but the problem I am facing is that Bearing Distance To Line tool generates only one line whereas when I run this tool manually on same output from Make Route Events Layer it works fine. Can you suggest what is the issue here?

I don't know why that would occur. You could try clearing any layer selection or copying the Make Route Event Layer to a feature class using the Feature Class to Feature Class tool and using that output. Anyway, I would have to see the model to have any other ideas of what is occurring.

Screen Shot 2021-07-28 at 8.43.10 am.png

ZeeshanAhmad2

I converted output from make route event layer tool to feature class from model and ran the tool manually instead of running it in model. It gave me correct results but when I use model, it draws only one line feature.I will share with you the model by tomorrow as I am away from my workstation right now.Really appreciate your help. Thanks

ZeeshanAhmad2

I finally got that to work. There was some issue with model which got resolved.
I have one question. In case of the curves or sharp turns, the resulting segments are not consistent. I understand this is because we are drawing lines perpendicular to pipeline so this is expected to happen in case of curves/sharp turns.
Is there a way we can recitfy this. I have attached a screenshot of my output. Please take a look.

test.png

RichardFairhurst

I finally got that to work. There was some issue with model which got resolved.
I have one question. In case of the curves or sharp turns, the resulting segments are not consistent. I understand this is because we are drawing lines perpendicular to pipeline so this is expected to happen in case of curves/sharp turns.
Is there a way we can recitfy this. I have attached a screenshot of my output. Please take a look.

For my taste, I like the way the cuts on the curves look and would want those bending polygon cuts, just not the sharp turns, i.e., as long as the cut lines don't cross other cut lines that best follow the overall buffer direction I would leave the cut lines you showed alone. So for the case you showed, I would only design additional steps to just correct 2 crossing lines. (Basically I don't want to force real world bending lines into being straight and am more concerned about maintaining equal lengths along the real line than equal sized buffers for most of my problems).

However, it sounds like you don't want to use the real pipeline shape at all, just a two point best fit trend line that represents the overall buffer and use the normal bearing to that imaginary generalization of the pipeline to cut up the buffer regardless of how the pipeline actually bends. Is that what you want? That could result in cuts that cross the real pipeline in more than one location, depending on the sharpness and proximity of the angle to any generalized cut line. The length of actual pipeline within each buffer segment will vary with this approach, but the length along the generalized trend line within each buffer segment would not vary.

Both outcomes involve summaries that find a mean bearing for the pipeline and some amount of generalization of the pipeline shape to create the cut lines that will segment the buffer. But I need to know how much generalization you actually envision that you want. You should look at the Generalization and Cartography toolsets and tell me if you see any tool that seem to get rid of the kinks at the level you envision.

In my road networks I have roads that range from straight lines, to 90 degree bent roads, to half circles, to hairpin turns, to full circles, so no one method would work. I would have to factor the amount of line sinuosity into the model and select sinuosity ranges for separate generalization subroutines. See this post about sinuosity. Do you need to do that, or would you say that a two point best fit trend line will work for all of your pipelines?

You also may need to look at the Minimum Bounding Geometry tool. In particular the rectangle by width option for finding the trend of the line or buffer and an average cut bearing angle.

ZeeshanAhmad2

Currently I am talking to the actual user of this tool and then I'll get confirmation if they are flexible in this scenario. The initial requirement was to keep the distance along pipeline same i.e. 1 Km. However the lines cutting other lines are unwanted and we need to correct them. How do we go about it?

RichardFairhurst

Currently I am talking to the actual user of this tool and then I'll get confirmation if they are flexible in this scenario. The initial requirement was to keep the distance along pipeline same i.e. 1 Km. However the lines cutting other lines are unwanted and we need to correct them. How do we go about it?

Off the top of my head this may add a fair amount of steps to only correct the two lines that intersect the other lines. My concepts are not optimized and there may be several better approaches, but I will give a possible approach. Test it manually first on your sample line before trying to model it.

I would start by testing for intersections. I would use the Intersect tool with just the cut lines as input and the Point output option. If no points are generated from self intersections then the line is fine. If points are generated then do the subroutine to find and fix the offending lines.

To identify the offending lines I would use the Summary Statistics on all of the cut lines whether or not they intersected other lines using the BufferID as the unique case field and get the Min, Max and Mean of the bearings. The Mean bearing should be closer to one end or the other in most cases and a simple SQL statement should be able to divide the records into 2 groups where the Mean is closer to the Min or closer to the Max. The Minimum Bounding Geometry could be an alternative to confirm the trend line orientation to get an average normal for the line.

Select by location the lines that touch the intersection points you created to isolate just the lines that intersect. Then select from the current selection one of the two Mean groups using a join to the Summary. So the SQL would look something like this for the group where the Mean is closer to the min and the line is closer to the Max:

ABS(Summary.Mean_Bearing - Summary.Min_Bearing) < ABS(Summary.Mean_Bearing - Summary.Max_Bearing) AND Line.Bearing > Summary.Mean_Bearing

These lines could have their bearing replaced by the Mean_Bearing using the field calculator. (Probably rather than doing this as a select from selection operation it would be better as a field calculation that keeps the existing bearing for lines that are closer to the mean after just doing the Select by Location only)

Now you would have to replace all of the cut lines with a set of lines that used the Mean_Bearing from a point on the line to each side of the line, rather than from an LR offset point. So now you would need a new field for all of the cut lines that is the 180 degree opposite of the original normal bearing. Use a field calculation similar to the one I gave you that makes a new bearing field that adds 180 to the main bearing and then corrects for bearings that go above 360.

Now instead of building one Distance and Bearing line, build two separate line outputs using these two bearing and use the point on the line as the point location. (You may need to run the Flip tool on one of the two line outputs). Append the lines together and Dissolve where the line ends touch each other only. This should create a new set of cut lines at the cut locations on the line with bearings that are much more similar to each other for those lines that originally crossed other lines.

My assumption is that in most cases the new cut lines should correct the problem. However, this output should be retested for self intersections. If any line requires another pass I would need to look at the new set of cut lines to find out what level of complexity is causing the self intersections and determine if this approach is going to lead to a solution with a second pass or if the complexity will defy a solution with this method. Some set of lines may be so complex that they will require manual intervention to fix them, since it is possible no clear set of simple rules and operations may be adequate to reach a solution and a human may be faster and better at reaching the best solution.

Sometimes it is not worth modeling beyond the 90% group that follows simple rules and deal with the oddball complex 10% using selection logic to identify and isolate them and then doing manual fixes. Often a 90% model makes the remainder that requires manual fixes a manageable size. Doing manual fixes of the 10% may identify another simple pattern that would define and fix 80%-90% of the problem remainder that you can model so that the new remainder requiring manual fixes is even smaller. But there are diminishing returns as you have to tackle smaller and smaller sets of problem features using a blind model to the point it simply is not worth modeling.

ZeeshanAhmad2

Thank you Richard, for your kind help. Really appreciate it. I will try this method.

RichardFairhurst

Thank you Richard, for your kind help. Really appreciate it. I will try this method.

Keep in mind that not every step is necessarily listed. This is basically brainstorming that is trying to identify the key aspects of an approach that should work. Some creativity will be required on your part to fill in any gaps I missed.

For example, the event table would have to be altered to add the bearing fields and I did not necessarily list the best stage to do that so that you can create a new point layer on the lines with the bearings needed for the Distance Bearing tool. You may also need a new distance field that is positive that equals half the length of the original bearing distance field, since you don't want lines that are much longer than the cut lines you already created. Also, the unique case field should actually be the RouteID and not the BufferID so that the summary can be joined to the event table. You also have to probably join the cut lines back to the event table to fill in the initial bearing or export the cut line attributes to a Standalone table so that its attributes can act as an event table.

Hopefully it is enough to guide you to a solution for cases like the one you showed me.

Rejoindre la conversation

Les membres connectés peuvent publier, suivre les mises à jour, et plus encore. Nouveau ici ? Inscrivez-vous gratuitement.

Liens rapides

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.