CPU speed and Memory requirements for ArcHydro and Geoprocessing

[MarkBoucher]

This is a duplicate post from the Hydro Forum: http://forums.arcgis.com/threads/55351-CPU-speed-and-Memory-requirements-for-ArcHydro-and-Geoprocess...

I'm wondering what others are using to operate ArcHydro (performs geoprocessing for hydrology purposes).

I'm getting more and more suspicious that most of the current problems I'm having stem from memory issues.

I've got:

• Windows 7, ArcGIS 10, the latest version of ArcHydro.


• 64-bit


• 4GB RAM


• 300 GHz Duo Core Intel processor


• I'm currently working with a 10' grid DEM 10.2 square miles mask extracted rawdem of 24.41 MB.

Seems like plenty, but is this enough?

This is the same configuration they have given everyone in the Department. I'd need "special dispensation" to get more.

[MarkBoucher]

Maybe I'm not telling anything new

Fernando

This is new to me.

Thanks for replying to the post and for reference material! I found it at http://support.esri.com/en/downloads/datamodel/detail/15.

Cheers!

Mark

[ChrisSnyder]

An update: I have been doing some testing of a machine with some newer 6.0 Gb/s SSDs (solid state hard drives). WOW!!! Huge performance increase over the 10k RPM SAS drives I had been using. Depends on the operation, but typically the hydrologic tools and associated raster-based tools (such as zonal statistics) are running 2-3 times faster with the SSDs. Raster stuff tends to be very disk I/O intensive, so I guess this makes sense. Anyway - in addition to CPU and memory... think about upgrading to SSDs also.

[MarkBoucher]

Chris,

Thanks for the reply. I'll look into getting a SSD.

I tried to find the HKEY_LOCAL_MACHINE\SOFTWARE\ESRI\Raster\Preferences\grid.max_table_range in Registry Editor, but it was not there. The document "Comprehensive Terrain Preprocessing Using Arc Hydro Tools" was written for 9.3. Is there a different registry setting for 10.0?

I'm using Windows 7, ArcGIS 10, the latest version of ArcHydro, 64-bit machine.

[ChrisSnyder]

Don't edit the registry directly... Instead, use the "AdvancedArcMapSettings.exe" registry editor - typically stored in the C:\Program Files\ArcGIS\Desktop10.0\Utilities folder.

[MarkBoucher]

Thanks so much!

[MarkBoucher]

Would a large capacity thumb drive work for this?

[ChrisSnyder]

Negative on the thumbdrive idea... The transfer speed is too slow and the drive capacity too small. Although some of their comoponents are similar, thumb drives are not the same thing as a SSD.

Check out:
http://www.amazon.com/Intel-Series-2-5-Inch-Solid-State-Drive/dp/B006VCP9G6/ref=sr_1_1?s=electronics...
or
http://www.amazon.com/Samsung-830-Series-MZ-7PC256N-Internal/dp/B005T3GPXY

Things change rapidly, but these are probably among the best SSDs you can get in May, 2012.

[MarkBoucher]

Don't edit the registry directly... Instead, use the "AdvancedArcMapSettings.exe" registry editor - typically stored in the C:\Program Files\ArcGIS\Desktop10.0\Utilities folder.

I followed your instructions yesterday (quoted above). Today I got back on a large rawdem project using ArcHydro. Still had the stream definition and stream link "fail".**  I ran  AdvancedArcMapSettings.exe and found that the setting was back to 65536 instead of 1000000 like I had set it. Does this need to be reset every time?

By the way, I've been running the same tools in Model Builder on a smaller watershed and its been working like a charm.


**(The model stops after the str and strlnk grids, but they are created without projections. When I add projections and put them in the MXD, then put them in the model and link them to the next steps, i.e. two manual tasks, the rest of model finished to adjoincatchments).

[JeffTang]

Greetings Mark!

I recently switched over (several months ago) from an Intel Core 2 Duo 2.4GHz processor with 4GB to a Intel i7 3.4GHz with 8GB.  Prior to the switch, I was running ArcGIS V9.3 under Windows 7-64; I'm presently running ArcGIS 10.  I don't think I necessarily needed to switch, but at the time, I was running into errors trying to process a large DEM and thought it might be due to the amount of memory.  The errors were more likely due to size limitation of ArcGIS than my computer components.  I've since found a workaround for the DEM where it's no longer a 1+ GB file (although my coworker recently informed me that it IS possible to process a DEM greater than 1GB).  I would imagine your setup is quite adequate for what you're doing.  For comparison purposes, the coworker that recently discovered it was possible to process a DEM greater than 1GB, is using a laptop that is more than several years old, which I would guess has a similar setup as your desktop.

[EricRice]

The errors were more likely due to size limitation of ArcGIS than my computer components.  I've since found a workaround for the DEM where it's no longer a 1+ GB file (although my coworker recently informed me that it IS possible to process a DEM greater than 1GB).  I would imagine your setup is quite adequate for what you're doing.  For comparison purposes, the coworker that recently discovered it was possible to process a DEM greater than 1GB, is using a laptop that is more than several years old, which I would guess has a similar setup as your desktop.

I'm not entirely sure how you came to the conclusion that there is some size limitation imposed by ArcGIS, but there is not.  Certain formats have size limitations. Tiff comes to mind, but even the old 4gb limit has been overcome with the advent of BigTiff format which ArcGIS is capable of writing.  I've been working on some hydro things since the beginning of the year, specifically processing NHDPlus v1 data for the lower 48 states.  I routinely process data with ArcHydro that is in excess of 8-10GB.  Outside of my hydro work, I do a lot of testing on NED data, which is several hundred GB's of data.  1GB of data is really nothing much these days.

My test machine specs:

• Windows Server 2008 R2 64 bit, ArcGIS 10.1


• 6GB RAM


• Intel Xeon W3550 @ 3.07GHz

Best Regards,
Eric