I haven't posted this until now - perhaps because it's not a problem.
Over the past year we've compiled a gob of data into our countywide Terrain Dataset:
- photogrammetric 1/4800 (10-ft) and 1/1200 (2-ft) contours and breaklines
- FEMA map modernization LiDAR bare-earth filtered points and breaklines
- NCALM high-res LiDAR along select seismic hazards
For best use with Terrain Dataset, the contours were vertex densified to 5-foot (horizontal) intervals along each contour line; several iterations of QA have repaired problem contours and stream-following water line features. The FEMA LiDAR points were retained in original cloud positions, while we had to consume the NCALM data in their gridded product form, to reset Z reference from an up-to-the minute, research-grade ellipsoid into WGS84-NAVD88 (Geoid 2003). Once into the geoid, the grid was converted to point-Z features and poured into the Terrain Dataset.
By the time we were ready to try out Beta 1, we had 960 million point-Z features in about 30 feature classes, water polyline-Z and hard break polyline-Z features, and a clipping polygon all set to go. Under 9.3.1 we'd actually ground out some countywide 30cm grids, but it took more than 20 tiles to get the job done without crashing ArcMap.
Almost on a whim, we re-imaged an existing GIS workstation with Windows Server 2003 OS and chose to install 9.4 Beta 1 for its version of ArcGIS. The OS change was to accomodate larger-sized seamless mosaics with ERDAS Imagine, where mosaics larger than about 70 GB tend to fail in non-server flavors of Windows.
About three weeks ago, we set up the Beta 1 system with a torture test: build the terrain, and then grind a single seamless countywide 40cm gridded surface, at pyramid level 0, from the Terrain Dataset that was approaching a giga-point of content. The first reflexive attempt to run a 30-cm gridding produced a scratch output file of 133 GB, but we were not confident that ArcMap was running properly, so we revised the grid to 40cm, and 77 GB of output, and watched just over one Xeon core of processor loading running, and running, and running...
=> after 135 CPU hours and 7.5 calendar days, it finished. The Beta 1 did hang when trying to create pyramids, but the output ".img" grid was serviceable in ERDAS, where we calculated statistics and pyramids.
As far as I can tell, the 3D analyst "Terrain to Raster" function has really grown some teeth since 9.3.1; thank you, Beta crew! The 40cm DSM product has been used already for some hydrology applications.
