topic Re: Feature extraction from LiDAR and derived rasters in Imagery and Remote Sensing Questions

Feature extraction from LiDAR and derived rasters

RobertFord1 — Tue, 21 Jun 2016 12:13:06 GMT

Good morning all!

I am trying to obtain heights of man made structures and also update / create building footprints using LiDAR and the rasters I have created from them. I have gone through the this tutorial that explains how to create a DSM and DEM from LiDAR data (so I now have a DSM and DEM) and also this tutorial that explains how to get heights of trees (I just used the minus tool for the entire area and have heights for the entire area).

The issues I am running into for heights are the outliers in the LiDAR data. I have tried using the Locate Outliers tool, but it just keeps failing saying it can't locate my output table. These outliers skew data greatly when it comes to calculating average heights for buildings (I tested it on an existing polygon). I would like to reclassify these outliers as noise somehow if possible.

As for feature extraction, I really am a new guy when it comes to intense LiDAR analysis in ArcGIS and just am not sure where to begin. I know there are all kinds of paid for 3rd party extensions for LiDAR that include feature extraction, but alas we do not have the budget for such things....

Any help is much appreciated!

Rob

Re: Feature extraction from LiDAR and derived rasters

JakeSkinner — Tue, 21 Jun 2016 12:20:41 GMT

Hi Robert,

You can use the LAS Dataset Profile View window to reclassify the outliers. See the following help document for more information:

LAS dataset Profile View—Help | ArcGIS for Desktop

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Tue, 21 Jun 2016 12:42:27 GMT

Thanks Jake! I am looking for a way to locate outliers on a mass scale. That's why the locate outliers tool looked like the tool I needed. My study area is county wide so going through one by one is not really an option. Although I will definitely use this for looking at buidlings in question later.

Re: Feature extraction from LiDAR and derived rasters

AdrianWelsh — Tue, 21 Jun 2016 14:31:57 GMT

Rob,

Try converting your LAS Dataset into a TIN

LAS Dataset To TIN—Help | ArcGIS for Desktop

Then you can run the Locate Outliers Tool.

Though, from looking at the tool, it looks like it will work on an LAS Dataset as well:

Locate Outliers—Help | ArcGIS for Desktop

"Summary

Identifies anomalous elevation measurements from terrain, TIN, or LAS datasets that exceed a defined range of elevation values or have slope characteristics that are inconsistent with the surrounding surface."

Have you created an LAS Dataset yet from your LAS data?

Create LAS Dataset—Help | ArcGIS for Desktop

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Tue, 21 Jun 2016 16:39:04 GMT

Adrian,

I have been using my 13,000 .las files in a LAS Dataset then attempting to feed that into the Locat Outliers tool input. I have discovered that it completes on a small LAS Dataset but not the full one. Does anyone know a of a faster way to break up my enormous folder of .las files into small datasets and iterate through them while using locate outliers?

Thanks again guys!

Re: Feature extraction from LiDAR and derived rasters

JakeSkinner — Sat, 11 Dec 2021 16:22:03 GMT

You can use python to iterate through your LAS files. Below is an example. It first obtains a spatial reference from an existing las dataset. The code then iterates through all las files in a directory, creates a LAS Dataset, and then runs the Locate Outliers tool with a min/max of 80/325, and an outlier cap of 5,000,000. It outputs the feature classes to a single geodatabase that you can then merge together.

import arcpy, os
from arcpy import env
env.overwriteOutput = 1
arcpy.CheckOutExtension('3D')
env.workspace = r"D:\Temp\Python\UserData"

dataset = "las1.lasd"
spatial_ref = arcpy.Describe(dataset).spatialReference

x = 1
for lasFile in arcpy.ListFiles("*.las"):
    arcpy.CreateLasDataset_management(lasFile, "lasDataset.lasd", "NO_RECURSION", "", spatial_ref)
    outlierFC = env.workspace + os.sep + "Data.gdb" + os.sep + "LocateOutliers_" + str(x)
    arcpy.LocateOutliers_3d("lasDataset.lasd", outlierFC, "APPLY_HARD_LIMIT", "80", "325", "APPLY_COMPARISON_FILTER", "0", "150", "0.5", "5000000")
    x += 1

arcpy.CheckInExtension('3D')

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Wed, 22 Jun 2016 14:15:21 GMT

Thank you 1000 times Jake! It's cranking away at it now. While it's doing that, how would I go about extracting features from the resulting rasters? Once the outliers are removed, I will then follow the steps I mentioned in my intial post to get heights (using minus tool). So now I have to extract man made features from this namely buildings.

Thanks again!

Re: Feature extraction from LiDAR and derived rasters

ArthurCrawford — Wed, 22 Jun 2016 15:24:50 GMT

Hello Jake,

The below works, but I have a new tools set, demo data and training material for an improved process. Here's the link to it, it requires ArcGIS Pro 2.0, Spatial Analyst and 3D Anaylst:

http://esriurl.com/3DMappingWithLidar

First question is what is the spacing of your lidar? If it's less than 3ft, you should be able to extract buildings from it. They will not be perfect. If you buildings are classified with a class code of 6 for buildings, I would suggest this process assuming your data is in feet. Some of the below is from Clayton Crawford on the 3D Team.

1. LASPointStatisticsAsRaster (LAS layer filter set to class code 6, building,
points)

2. RasterToPolygon (no simplification)

3. RegularizeBuildingFootprint:

a. Run Regularize Building Footprint as Circle with a tolerance of 8 and minimum
radius of 0.1/max radius 100000 and export those with a status of 0 and area
over 275 sq. meters to Large Circles.

b. Run Regularize Building Footprint as Circle with a tolerance of 3 and minimum
radius of 0.1/max radius 100000 and export those with a status of 0 and area
over 25 sq. meters and under 275 sq meters to Small Circles.

c. Combine those circles and use select by location to remove those polygons from
processing.

d. The rest divide into three groups: Large, Medium and Small buildings.

Large: "Shape_Area" >= 2300 Medium: "Shape_Area" >= 464 AND "Shape_Area" < 2300 Small: "Shape_Area"< 464

Large run Regularize Building Footprint as Right Angles and Diagonals with tolerance 4, densification 4,
Precision 0.25, Diagonal Penalty of 1.5.

Medium run Regularize Building Footprint as Right Angles and Diagonals with tolerance 3, densification 3, Precision 0.25, Diagonal Penalty of 1.5.

Small run Regularize Building Footprint as Right Angles with tolerance 2, densification 2, Precision 0.25

One approach with buildings not classified:

Classify LAS Ground

ClassifyLASBuilding (Pro 1.3) or the 3D Samples tools for Classify Rooftop if you don't have access to 1.3 (might want to wait).

LAS Point Statistics As Raster (LAS layer filter set to class code 6, building, points)

RasterToPolygon (no simplification)

Regularize Building Footprint (see above)

If you have parcels, you can do a union on them with the output polygons and select out those with a -1 for fid for the parcel before running Regularize Building Footprint. This will help with dividing lidar buildings to individual buildings in downtown areas and divide joined buildings (vegetation) between properties where the buildings are close. I have also found doing a negative buffer, mulitpart to singlepart and then a positive buffer to help get rid of trees next to buildings that get captured. Sometime I run a statistics function on the output of the LASPointStatisticsAsRaster and then a eliminate polygon part after the Raster to Polygon to get rid of interior parts that are not desired.

Thanks, hope this helps.

Arthur Crawford - Content Team - Living Atlas

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Wed, 22 Jun 2016 16:11:48 GMT

Wow thank you so much Arthur! I will try this out and report back. The last step seems out of my reach as I do not see the Regularize Building Footprint tool in my toolbox....I have the 3D analyst extension and an ArcInfo license on ArcGIS 10.3. Can I download this somewhere? Is this only for 10.4 possibly?

Thank you again

Robert

Re: Feature extraction from LiDAR and derived rasters

ArthurCrawford — Wed, 22 Jun 2016 16:21:54 GMT

Regularize Building Footprint is in 10.4 or 1.2 ArcGIS Pro. I highly recommend upgrading if you are going to do building extraction cleanup. You may use Simplify Building, but the results are better with Regularize Building Footprint. There are other methodologies I use depending on the quality of the data. If you have 1ft spacing lidar, the results can be fairly good. Below are some buildings before manual cleanup from 2ft resolution lidar using another process for extraction, but using the Regularize Building Footprint process above that I processed this morning to add to the World Topo Map. You can see a tree made it in that needs to be deleted in the North West corner.

Arthur Crawford - Content Team/Professional Services

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Thu, 23 Jun 2016 12:54:07 GMT

Arthur,

My LiDAR has 1 ft spacing. I starting the raster to polygon tool yesterday afternoon and it is still running today. Is this normal? I know my raster is pretty large. I'll just let is crank away I suppose.

Also, I ran into this article that outlines a different methodology for feature extraction. It was written by David Rohde, City of Edmonds. He goes through the same process of creating a DEM and DSM and using the Minus tool. However, to obtain stories for buildings, he uses the raster calculator to divide the resulting Minus raster by 8, converts raster to polygon, adds an area field, then selects all polygons with an area > 300 sq ft. It's a bit more of a dirty method but it looks like it would work. I am still going with what Arthur suggested as it sounds like the better methodology.

Re: Feature extraction from LiDAR and derived rasters

ArthurCrawford — Thu, 23 Jun 2016 15:50:42 GMT

Hello Robert,

I would start with a small area to test with and get your parameters correct for your data. I 'm doing my processing with a model using an iterator with a grid over the area to do small areas and then merge together. One question is if your data is classified for buildings? Did your output raster show buildings with just one value and no data in the other areas? Also, was there holes in the buildings in the Raster. If so, run a Majority on it before converting to polygons.

On this article, it's uses Simplify Buildings instead of Regularize Building Footprints. He also says there is quite a bit of manual clean up, something I avoid as much as possible. I also suggest getting polygons over a certain size, I usually use 500 square feet as my input. There are many ways to get buildings from Lidar that have been developed, I sharing what has worked for me.

We have some additional tools for extraction we will be releasing on a website soon in prototype form that are alternatives to the process above.

Thanks,

Arthur Crawford

Re: Feature extraction from LiDAR and derived rasters

SteveSnow — Thu, 23 Jun 2016 16:09:10 GMT

Hi Robert, we have a workshop on this workflow with step by step instructions using two methods 1) from Point Clouds 2) from rasters at the Esri Imaging and Mapping Forum this week in San Diego. Unfortunately you will have to attend to learn how. #IMF2016 #EsriUC Lineup of Speakers & #Technology http://dld.bz/egRY4 #Drones #Imagery #Lidar #mapping #GIS

We will give 2 workshops Drone2map 2D and 3D mapping, and Mapping from Point Clouds. Hope to see you there!

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Thu, 23 Jun 2016 18:17:46 GMT

Arthur,

My data is classified and your method using LAS Points Stats as Raster worked out. The resulting raster does have holes. I tried the majority filter and it still has holes:

I converted it to polygons and got the following with holes still:

Would I use the integrate or dissolve tool to somehow fill these in? I'm looking what I can on the simplify building tool now.

Re: Feature extraction from LiDAR and derived rasters

ArthurCrawford — Thu, 23 Jun 2016 18:26:45 GMT

Hi Robert,

I would suggest next time remapping/reclassify the image areas with data to one value. For where you are at now, run a dissolve on the polygons, then a positive buffer of 2 ft and then a negative buffer of -2ft. This should get rid of small holes and merge the multiple polygons your have. Then a multipart to singlepart. Then continue with the instruction for the Regularize Building Footprint.

Arthur Crawford - Content Team/Professional Services

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Thu, 30 Jun 2016 14:22:35 GMT

Arthur,

I now have my buidling footprints layer. I realize there is still a good bit of clean up (some footprints are merged into one large one, looks like some concrete structures that aren't buildings made it into class 6).

Thank you for your help!

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Thu, 30 Jun 2016 14:32:56 GMT

Unfortunately, I never was able to get the locate outliers to work correctly....ESRI doesn't have any good examples on how this tool should be properly used. I am attempting to filter out any LiDAR points that have more than 100 foot difference from the surrounding points. I set my comparison filter in the Locate Outliers tool as such:

The output is not what I expect to see as I know where many outliers are but the tool is not flagging them.The following picture is a screenshot of the Minus tool output. The circled point says it is 1782 feet tall. Obviously this is not correct. The locate outliers tool does not flag any points in my LAS dataset near this point.

Now that I have my building footprints layer, I want to calculate the max z value / elevation for each polygon to get the building height using the Minus tool output. This outlier, if it were within a building footprint polygon, would then tell me my building is 1782 feet tall!

Am I misunderstanding how the locate outliers tool should be used? Is there a way to just locate local outliers from the Minus tool output raster?

Thank you again guys!

Re: Feature extraction from LiDAR and derived rasters

RobertFord1 — Thu, 30 Jun 2016 16:24:07 GMT

I've created a separate thread for this question that is awaiting approval to be posted:

https://community.esri.com/message/618436

Re: Feature extraction from LiDAR and derived rasters

ArthurCrawford — Mon, 11 Jul 2016 20:13:55 GMT

Hi Robert,

I know you saw this, but wanted others to see it. There are two blogs on extracting features out there on Geonet.

Mine:

Arthur's Feature Extraction from LiDAR, DEMs and Imagery

Geoff's:

3D Workshop Feature Extraction Tools

Thanks,

Arthur

Re: Feature extraction from LiDAR and derived rasters

LeoLadefian5 — Fri, 17 Nov 2017 14:35:16 GMT

Hi all, thanks a bunch for the tools, I'm getting alot of tree noise using these tools:
any ideas about how to curb this?