Is 2.8 billion lidar points in 700 .las files too much for LASPointStatisticsAsRaster?

4062
4
Jump to solution
08-13-2015 08:37 AM
Labels (1)
TheodoreRakel
Occasional Contributor

I have over 700 .las files in a lasd dataset.  These files have around 2.8 billion LiDAR points in them with point spacings from 0.3 to 10.  The geoprocessing tool LAS Points Statistics As Raster which I use to create a canopy height is taking over 24 hours to run.  I know I can change the cell size to speed things along and I will do that.  Right now cell size is 10 and that's worked well before with data from this vendor, but fewer files with around 400 million points.  Does anyone know if combining the 700 files into several big files would speed performance?  Has anyone processed this many points using LAS Points Statistics As Raster? 

Tags (1)
1 Solution

Accepted Solutions
DanPatterson_Retired
MVP Emeritus

what software? If ArcMap, version, memory etc also

http://desktop.arcgis.com/en/desktop/latest/manage-data/las-dataset/creating-a-las-dataset.htm

Along with improving overall performance, computing statistics for the LAS files will increase the optimal file-size threshold for LAS files referenced by a LAS dataset from 200 MB to 500 MB.

  and what reading did you do in th las links?

http://desktop.arcgis.com/en/desktop/latest/manage-data/las-dataset/what-is-a-las-dataset-.htm

View solution in original post

4 Replies
DanPatterson_Retired
MVP Emeritus

what software? If ArcMap, version, memory etc also

http://desktop.arcgis.com/en/desktop/latest/manage-data/las-dataset/creating-a-las-dataset.htm

Along with improving overall performance, computing statistics for the LAS files will increase the optimal file-size threshold for LAS files referenced by a LAS dataset from 200 MB to 500 MB.

  and what reading did you do in th las links?

http://desktop.arcgis.com/en/desktop/latest/manage-data/las-dataset/what-is-a-las-dataset-.htm

TheodoreRakel
Occasional Contributor

Thanks Dan, I'll compute las dataset statistics and see if that helps.  I'm using ArcMap 10.2.1 on a Windows 7 desktop with 4 GB RAM. 

0 Kudos
CodyBenkelman
Esri Regular Contributor

Theodore

Let me look into this - I'll give you a "half answer" right now.  We appreciate that for very large datasets, some geoprocessing tools
need improvements for scalability.  There is a tool called "LAS Dataset to Tiled Rasters" available here http://links.esri.com/3dSamples that
can take in a large collection of LAS (via LASD) and output raster *tiles* rather than one large raster.  You can let the tool define the tiles, or
input your own feature class to define tile boundaries, names, other attributes.

Having said that, I realize you asked about a different GP tool (LAS Points Statistics As Raster) and we don't currently have that
one implemented with a "tiled raster" mode, but I've asked the engineer who built LAS Dataset to Tiled Rasters and it may not be difficult to create something similar, either in Python or with an iterator in ModelBuilder.

Wherever possible, I recommend *against* creating very large single rasters, but instead create more manageable smaller rasters, then manage them as a single mosaic using the Mosaic Dataset.  Note If you're seeking to build canopy height, rather than use Z_range in LAS Points Statistics As Raster my recommended workflow would be as follows:

  1. Run LAS Dataset to Tiled Rasters twice, once filtered for ground points only, 2nd time for First Returns, to
    create a bare earth DTM and first return DSM (unless you already have them?) in tiles
  2. Build separate Mosaic Datasets for your two raster collections (DSM and DTM), then create a Referenced Mosaic Dataset based on the
    DSM and apply the Arithmetic Function to subtract the DTM.  This Referenced Mosaic Dataset will be your canopy height (without taking up any disk space to store it).
  3. If you really need the canopy height as a separate raster, you can export it from the Referenced Mosaic Datase

(This may sound like more work - and I suppose it is - but since you have lidar at different point densities, I would suggest that the ideal processing would be to build DSM and DTM at different resolutions as limited by the point density - specifically, process each lidar collection separately as "projects" and then combine them as recommended here http://esriurl.com/LidarGuidebook  Your resulting canopy height map would have variable resolution based on which lidar collection it came from, but the mosaic dataset can easily handle that, and you could then summarize at different resolutions as necessary. )

Last note re: processing speed, I haven’t looked closely at your figures to estimate if there is something wrong or if that’s an appropriate amount of time to expect.  Even if you were to iterate through either GP tool to create multiple output tiles, it may not run any faster - the total processing
may still take many hours - but the results should be more maintainable.

Let me know if this is helpful

Cody B.

TheodoreRakel
Occasional Contributor

This is very helpful.  I'll try this on the 2.8 billion point collection soon.  Thanks again

0 Kudos