ArcGIS server Cluster

Question

I run a gp service on a cluster. The message showed "Executing: CrossTab 2001 2011 C:\Users\ArcGIS\AppData\Local\Temp\corsstab_gpserver\j51ff72dacca....\scratch\scratch.gdb\output", in which crossTab is my gp service name, two inputs are 2001 and 2011, the output is "C:\Users\ArcGIS\AppData\......"

It is quite strange, why the output is not in the folder like "C:\arcgisserver\directories\arcgisjobs\corsstab_gpserver\j51ff72dacca....\scratch\scratch.gdb\output"?

ScottFierro2 · Answer

Looked at some of our servers and yes I'd say that is correct. Seems that we only have 1 GP that's utilizing the space and all the rest I've done the published service modifications to, etc. It appears though that while it does use the space it's only partially cleaning things out or has a very long period it retains the temp job folders and contents.

springzhang · Answer

Hi Scott, Thanks so much for the reply. Yes, we have the cluster configuration as you mentioned above I saw a relate post on ESRI website. Please see below. "The local temp folder is a directory that the GIS Server writes intermediate output to when the server participates in a cluster with more than one machine, or the directories are referenced using a UNC path. The temp folder location is unique to the user account on which ArcGIS Server runs. This account is set when ArcGIS Server is installed. On Windows, you may see C:\Users\\AppData\Local\Temp in the execution messages of your service. To change the location of this folder you will have to log on to the computer with the account running ArcGIS Server. The temporary directory is modified by changing the path of the TEMP variable within the Environment Variables settings." Based on the paragraph, it seems that in the cluster environment the output folder is indeed in C:\Users\\AppData\Local\Temp...... Is it?

ScottFierro2 · Answer

See this with all of our GP's and based off what you have said I can make some observations.

1) You configured your servers to use the basic service account named ArcGIS during installation and when running the GP it's through anonymous authentication so the service account is being used, hence C:\Users\ArcGIS. Also curious, for your cluster, did you configure your config-store and directory folders to use a folder on the C:\ of the first server you installed ArcServer onto and then made that an available share folder? If so, this might present issues because unless you replicate the same local pathway other servers in the cluster can't find the folder (doubt you did this) OR if you used the shared path (UNC) \<server_name>\arcgisserver\... then you are creating load strain on any work done to that original server (more likely what most people do). It's best to have network storage or at least a server not used within the cluster to establish the shared location for the config-store and directories.

2) Within your GP you are using the inline variable call for %scratchWorkspace% or %scratchGDB% and while this works great from desktop it seems something doesn't quite convert over during the "packaging" of the services. Doesn't seem to be from server not recognizing the inline variables because the services will publish and run. I've dug into the published services from within the directory folders and ensured the variables are not be miss translated during publishing. In cases I have just gone in and built out a FGDB within the published service called it Scratch.gdb then modified the published service by hardcoding in the new Scratch.gdb. Far less than ideal especially with regards to cleanup, etc. but ensures certain GP's don't inadvertently overload any local server drives and built a simple python script scheduled via Windows Task Scheduler to nightly go through and clean out the FGDB.

Sorry I can't give you a clearer or better answer just what we've encountered thus far but it will be interesting to see if any ESRI staff chimes in with better details.