Filtering similar point elevations

AlexanderMa

1. Created multiple polylines that I identified with the field "Poly_ID"
2. Densified the vertices to a 10-foot spacing
3. Interpolated the polylines on a DEM raster (3-foot cell resolution)
4. Converted the polylines to points
5. Created fields "Poly_ID", "X", "Y", and "Z"
6. Calculated values for "X", "Y", and "Z"

Goal: Let's say the average number of points for each unique "Poly_ID" is above 1000 but I need to trim it to 500 points.

Question: Aside from starting over with the generalized set of polylines, what process(es) should I utilize that helps me trim the number of points down such that if I were to plot the points before trimming and after trimming on a profile, the differences would be minimal?

An example of this in HEC-RAS, a non-GIS environment, where you can copy and paste cross-section stationing and elevation, go into tool called "Cross-Section Point Filter" and have the program maintain the integrity of your profile while reducing relatively redundant points.

DanPatterson_Retired

you could filter by sequential differences in the values.  I can give you an example using python/numpy to illustrate what I mean.

Filter the values in a sequential list of data, retaining only those values where the sequential difference is >= 0.2

<SPAN class="operator token">>></SPAN><SPAN class="operator token">></SPAN> a
array<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">[</SPAN> <SPAN class="number token">1.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">1.1</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">2.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">2.1</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">2.2</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">3.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">5.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">10.0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="operator token">>></SPAN><SPAN class="operator token">></SPAN> np<SPAN class="punctuation token">.</SPAN>ediff1d<SPAN class="punctuation token">(</SPAN>a<SPAN class="punctuation token">,</SPAN> to_begin<SPAN class="operator token">=</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">></SPAN> <SPAN class="number token">0.2</SPAN>
array<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">1</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">1</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">1</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> dtype<SPAN class="operator token">=</SPAN>bool<SPAN class="punctuation token">)</SPAN>
<SPAN class="operator token">>></SPAN><SPAN class="operator token">></SPAN> a<SPAN class="punctuation token">[</SPAN>np<SPAN class="punctuation token">.</SPAN>ediff1d<SPAN class="punctuation token">(</SPAN>a<SPAN class="punctuation token">,</SPAN> to_begin<SPAN class="operator token">=</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">></SPAN> <SPAN class="number token">0.2</SPAN><SPAN class="punctuation token">]</SPAN>
array<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">[</SPAN> <SPAN class="number token">1.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">2.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">3.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">5.0</SPAN><SPAN class="punctuation token">,</SPAN>&nbsp; <SPAN class="number token">10.0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

In the above example, a list of values is expressed as an array, a

the sequential 1D differences in the array are determined and compared to a threshold value, 0.2

in the above example, I wanted to retain the first point so I set the to_begin to equal 1 (True in boolean world).

In line 5, the values are extracted (sliced) from the input array, wherever, the difference is True (aka... 1) yielding what you see on line 6.

This is easily implemented in any tools that you have available. The only issue would be whether it is really important to retain the start position of the inputs... changing this might have some impact on the resultant and may have to be investigated.

You could also look at a first order or second order difference to decide on what type of differences you are interested.  This can be done easily and is built-in to many languages (ie np.gradient in numpy)

Just some food for thought... might be better than just taking every 2nd, 3rd, 'x'rd point which is standard go-to