Set a processing folder for all ArcGIS non-RAM / file based temporary processing to allow for much faster geoprocessing on USB3 SSD's

Idea created by mboeringa2010 on May 12, 2012
    • mboeringa2010
    • BruceLang
    • cjdusa
    • dknudsen
    Hi all,

    I know there is the "Scratch Workspace" Environment setting used in Models to store intermediate files, but I have noticed ArcGIS actually writes for many tools quite extensively, and often, temporary files in a users Windows profile Temp directory (e.g. C:\Documents and Settings\YOUR USERNAME\Local Settings\Temp"

    This means extensive hard disk activity during some processes, and a serious slow down of some processes compared to in-RAM processing, not to mention the risk of prematurely wearing out the hard disk. Processes that really shouldn't being going on a hard disk at all.

    Now with the new USB3 SSD based storage solutions becoming slowly mainstream, and new desktop computers featuring high speed USB3 with a theoretical limit of up to 5Gbps (625 MB per second) transfer rate, using an external disk to seriously augment fast ArcGIS's "RAM" type memory processing, is becoming a real possibility.

    It means you can have 64/128GB or whatever amount of Gigabytes of pseudo/USB3-SSD based "RAM". No more being bogged down by lack of RAM memory! You essentially have limitless processing space and RAM.

    The beauty of this idea of allowing a user to set ArcGIS's temporary processing space, is that it allows users to be in full control themselves as to how much "RAM" they have available, even in highly restricted corporate network environments where ArcGIS users don't have any say as to with how much RAM or disk space their computer is equipped. As long as you are allowed to plug in an USB drive, you should be fine.

    I tested this idea by equipping an older - not yet USB3 - machine with an external USB3 SSD. Through some not-for-the-faint-hearted finicking using NTFS "junctions" with Windows, I was able to move my ENTIRE Windows profile (so NOT JUST "My Documents" etc.) onto the external USB drive. Even though on just USB2, ArcMap starts up more than twice as fast, and my hard disk is entirely silent during any (geo-)processing done in ArcGIS. Great! The results should be 10x better on USB3.