Relative depth to absolute depth- segments don't line up because of raster DEM

[ErinKennedy]

Hello,

I am having a problem trying to generate a 3D visualization of my polylines (pipes data). I only have relative depths (1 meter underground, 1.2 meters underground, etc) and DEM data. I want to make an absolute depth (based on sea level) with my DEM, but then I get pipes that don't line up because of the "jumps" at raster cell edges.

Any idea how to fix this? I'm thinking there is either a way to correct the non connecting pipes or a way to interpolate continuous DEM values so my pipes don't offset by up to a meter at a time.

Any suggestions are appreciated!

[TimBarnes]

What method have you tried so far? If you use the tool 'Interpolate shape' in the 3D Analyst toolbox and choose the 'Interpolate Vertices Only' then this will avoid the problems you describe.

Ideally, you should also fix the DEM data by interpolating tiles with an overlap and then either clipping them to a tile boundary or merging them using average values.

[ErinKennedy]

Hi Tim,

Thanks for your response. Sorry I wasn't so clear in my post: to determine the absolute "height" from the relative heights that I have, I added the average DEM per line segment into the attribute table (using the tool "add surface information"). Then, I subtracted the depth of the pipes to get the absolute depth. Now, I have the absolute heights (or depths) of the pipes in the attribute table.  After I had these absolute values, I used the tool 'features to 3D by attribute' and chose the absolute height value to determine the z values on.

Unfortunately, the tool 'interpolate shape' won't work, because the pipes are at different depths- I don't want to interpolate to the surface of the DEM.

My problem with the DEM is that if two segments from the same line are both at 1 meter depth, but sit on two different raster cells where there is a half meter difference, they will be a half meter apart in the vertical direction. I think no matter what I do with the DEM raster, I will have problems with the pipes lining up in 3D since the absolute depth per line segment is determined based on an average of the DEM along that line. So, the most likely solution is something like a snap or extend line tool (or similar workaround), but then for only the horizontal direction.

I would be happy if you or anyone else has another suggestion!!

[XanderBakker]

Hi Erin,

Do the pipelines with relative heights correctly connect at the start and end points?  If not then adding the DEM information will not correct anything. Adding an average height for a line will create these kind of errors.

What you need is to distinguish between "from" and "to" height (DEM elevation) for each polyline and use the "Feature To 3D By Attribute (3D Analyst)". To do this you could follow these steps:

Create points at From and End point with the tool "Feature Vertices To Points (Data Management)" using the point_location = "BOTH_ENDS". This requires an Advanced (Arc/Info) license.

If you don't have Arc/Info you could use some Python code like this:

import arcpy, os

# featureclasses
polylineFC = r'C:\Path\To\Your\filegeodatabase.gdb\Pipelines' # edit this!
fldID = "NameOfYourUniqueIDfield"

# outputs
outWS = r'C:\Path\To\Your\filegeodatabase.gdb' # output workspace
outFromNodes = 'FromNodes' # output names
outToNodes = 'ToNodes' # output names

# Set local variables
geometry_type = "POINT"
has_m = "ENABLED"
has_z = "ENABLED"

# Use Describe to get a SpatialReference object
spatial_reference = arcpy.Describe(polylineFC).spatialReference

# Execute CreateFeatureclass
arcpy.CreateFeatureclass_management(outWS, outFromNodes, geometry_type, polylineFC, has_m, has_z, spatial_reference)
arcpy.CreateFeatureclass_management(outWS, outToNodes, geometry_type, polylineFC, has_m, has_z, spatial_reference)

# make nodeDict polylines
featuresFrom = []
featuresTo = []
lstIDs = []
curF = arcpy.da.InsertCursor(outWS + os.sep + outFromNodes,("SHAPE@", fldID))
curT = arcpy.da.InsertCursor(outWS + os.sep + outToNodes,("SHAPE@", fldID))
with arcpy.da.SearchCursor(polylineFC, ("SHAPE@", fldID)) as cursor:
    for row in cursor:
        polyline = row[0]
        uniqueID = row[1]
        pntF = polyline.firstPoint
        pntT = polyline.lastPoint
        curF.insertRow((pntF,uniqueID))
        curT.insertRow((pntT,uniqueID))

del row
del curT
del curF

Use the output feature classes to extract the DEM values to the points.

Create in the pipes featureclass the fields "FromHeight" and "ToHeight". Join both the point featureclasses to the pipes featureclass based on the unique ID field. Fill the "FromHeight" and "ToHeight" fields with the field calculator. Apply the relative height to obtain the absolute height for from and to points of the pipes.

Now you can use "Feature To 3D By Attribute (3D Analyst)" using the "FromHeight" and "ToHeight". If pipes share a from/to point the height will be the same at that point IF the relative height for the pipes is the same.

Kind regards,

Xander

[TimBarnes]

Hmm... time for a 'reality check' here I think

If the whole pipe line segments have the depth value, then not lining up at the start and end points is actually valid for co-incident line segments with different depths (i.e. in the real world these may 'join up' in a man hole or riser where pipes enter a junction at different depths). In this case the pipe will mimic any DEM value minus the depth and a simple 'Interpolate Shape' is also valid. If visualising this in 3D, you may need to add a extruded polygon at the coincident end/start points to represent the man hole etc.

[ErinKennedy]

Thanks Xander,

I think the feature vertices to points tool will be very useful- then I can determine the relative depth at each end point (using add surface information to attribute table) instead of the average along the line, which will mean that I can eventually get the lines to line up.

However, I am not sure that feature to 3D by attribute will be the right tool to use, or at least I am unsure of what steps to take in between.

Now I have a point file with the absolute depths of the end points of the pipes in one field of the attribute table and a seperate line file of the pipes. Feature to 3d by attribute has a "to" and "from" option, but I would need to use the lines as input features and add the to and from values from the points to the attribute table of the lines (and I'm not sure how to approach that problem). However, it seems like there *should* be a tool to create a 3D layer from the lines using the depths at the locations of the points (kind of like how you would derive the z values for surface features from a DEM).

Any ideas?

[XanderBakker]

Now I have a point file with the absolute depths of the end points of the pipes in one field of the attribute table and a seperate line file of the pipes. Feature to 3d by attribute has a "to" and "from" option, but I would need to use the lines as input features and add the to and from values from the points to the attribute table of the lines (and I'm not sure how to approach that problem). However, it seems like there *should* be a tool to create a 3D layer from the lines using the depths at the locations of the points (kind of like how you would derive the z values for surface features from a DEM).

Any ideas?

Hi Erin,

What method did you use to obtain the height at the end points? Do you also have the height at the "from" points? If feature vertices to points was use, did you use the BOTH_ENDS setting?

The idea is to have a point feature class with the location of both the from and the to point, or have two feature classes (one with the from and the other with the to points). Extract the raster values using the point feature class(es). Next step would be to join the point feature class through a common field back to the lines. That way you will have the from and to height of each pipe. Use this information to use the tool "Feature to 3d by attribute" with the "to" and "from" option.

Kind regards,

Xander

[ErinKennedy]

Xander,

Sorry, I somehow missed the second part of your post after the code. When you use the setting "both ends", you get two points for each line, which if added to the line file, I don't know how to use field calculator to add "every other" value to the end or start point field. So, I will instead do the operation "feature vertices to points" twice, once for start points and one for end points and join both of them to the lines.

Thanks for all your help!

[ErinKennedy]

So, now I have a different problem. I am trying to use 'feature vertices to points' for another feature and I have 24,755 lines, but it gives me both 93,540 begin and end points. I took a look at the situation by selecting generated startpoints with the same objected and the related line, so I can see what the situation is (see attached screenshot), but I don't know how to deal with this. If I determine the DEM at these points, how would I again couple it to a line to create a 3D model by attributes?

Ideas?

[XanderBakker]

So, now I have a different problem. I am trying to use 'feature vertices to points' for another feature and I have 24,755 lines, but it gives me both 93,540 begin and end points. I took a look at the situation by selecting generated startpoints with the same objected and the related line, so I can see what the situation is (see attached screenshot), but I don't know how to deal with this. If I determine the DEM at these points, how would I again couple it to a line to create a 3D model by attributes?

Ideas?

Hi Erin,

It looks like your lines may be multiparts. I have seen this at a client who manages 3D pipelines.

In some causes this can be caused by individual segments dissolved together, but due to opposite directions of individual segments, a multipart is created. You could verify this by going into Edit mode, select the feature, choose Edit Vertices and open the Edit Sketch Properties window. Another alternative is to create new field in the line featureclass and calculate (using Python syntax):  !Shape!.partCount

This can (sometimes) be corrected by using the "Dissolve (Data Management)" tool, with the UNSPLIT_LINES option switched on. (Lines are only dissolved when two lines have an end vertex in common). The create multipart features option should be off.

Kind regards,

Xander