try: deleteme = Raster(arcpy.gp.ExtractByMask_sa(WSILayerRas, AOIshape)) except: continue
Solved! Go to Solution.
import time time.sleep(1.1) #insurance newTempDir = r"C:\temp\gptmpenvr_" + time.strftime('%Y%m%d%H%M%S') os.mkdir(newTempDir) os.environ["TEMP"] = newTempDir os.environ["TMP"] = newTempDir
def doCalc(year): yearstr = str(year) print('Starting doCalc: ' + yearstr) defaultGDB = "D:\\GIS\projects\\year" + yearstr + ".gdb" #Setting environmental variables arcpy.env.workspace = defaultGDB arcpy.env.scratchWorkspace = defaultGDB
import time time.sleep(1.1) #insurance newTempDir = r"C:\temp\gptmpenvr_" + time.strftime('%Y%m%d%H%M%S') os.mkdir(newTempDir) os.environ["TEMP"] = newTempDir os.environ["TMP"] = newTempDir
def doCalc(year): yearstr = str(year) import time time.sleep(1.1) newTempDir = r"C:\temp\gptmpenvr_" + time.strftime('%Y%m%d%H%M%S') + yearstr os.mkdir(newTempDir) os.environ["TEMP"] = newTempDir os.environ["TMP"] = newTempDir
Does the os.environ command set the temp directory for just the current process?
time.sleep(1.1) #sleep for 1.1 seconds
Hello,
I have been setting os.environ["TMP"] and os.environ["TEMP"] to unique folders while multiprocessing for some time now and it worked until now. Now I have some code which keeps using the TMP as defined in "User defined variables" in computer properties. The script is too long and too secret to be presented here but I've been through it many times and could not find anything that should cause a problem.
I wrote a function to dump the os environ variables, workspace, scratch workspace, scratch folder, and scratch gdb to a file and I used this function in several places in the script. Each dump indicates that these variables are set as I intended, i.e. to the unique folders or geodatabases. When I run the script however, many things are written into the TMP folder defined in "User defined variables" and not to the unique folders.
I tried to troubleshoot it for several days but to this time I am not entirely sure why is this happening. However, based on what I have seen, I believe that setting os.environ["TMP"] and os.environ["TEMP"], workspace and scratch workspace to unique folders or geodatabases is necessary, but it is not always enough (in many cases workspace and scratch workspace can point to the same folder or geodatabase).
It seems that the crashing I've experienced is caused by spatial analyst tools that write temporary data to represent Raster objects into the temp folder as ESRI Grid (as mentioned in some previous posts). Unless the Raster objects are explicitly saved to a unique workspace or with a unique name, ArcGIS will likely crash. Therefore, it really depends how the functions your master scripts calls and any deeper functions are written. What I ended up doing in my scripts is something like:
...
shrunk_raster = arcpy.sa.Shrink(a_raster)
shrunk_raster.save(os.path.join(wd, 'shrunk')) # wd is a unique workspace for the process, consider using arcpy.CreateScratchName
...
The problem is that now I have to pass the unique workspace wd down to every function I call and some function may not allow that kind of parameter.
The bottom line is that the crashing would not occur if the deeper functions respected the TMP and TEMP variables in os.environ. Do you have any idea why the os.environ is not honoured?
Here is what remained in the TMP folder defined in computer properties once the whole process was over. The CitrixLogs folder is probably not from ArcGIS.
Filip.
I use a pattern of subprocesses and independent workspaces, here:
http://www.arcgis.com/home/item.html?id=b3c7c6273ef54e91aa57a073aa873eca