running an EXE from a geoprocessing service

[AlanStewart]

This is a complex setup, but I'm hoping someone has ideas...

I have a Python geoprocessing service that makes calls into a custom C# .NET assembly which makes calls into a custom 64-bit C++ COM DLL containing ArcObjects code. Everything publishes to Server and works fine up until the point where the COM DLL runs a custom 32-bit C++ EXE in a new process. The EXE does not use ArcObjects. It reads a file, writes a file and exits. The process itself seems to be created successfully, but the EXE is never executed. GetExitCodeProcess() returns C0000135, DLL cannot be found.

Some pertinent facts:
1) I am building all components on the ArcGIS Server 10.1 box (Windows 7 Ultimate).
2) The EXE is in its own folder which also contains all the DLLs it is dependent upon, other than ones that delivered with Windows and VC runtimes.
3) This folder is registered as a data store with ArcGIS Server 10.1.
4) The ArcGIS Server user has execute permission for the EXE.
5) I have unit tests for the 64-bit C++ COM DLL that succeed in running this EXE. Those tests run using an ArcGIS Server 10.1 license.

It seems like the problem has to be environmental. I have captured the path for the parent process when the EXE process is created. It matches exactly with the path present in a Windows command shell, an environment in which the EXE runs successfully.

TIA for any input.

[RichardWatson]

So you are trying to run a 32 bit process (written in C++) and it is returning DLL not found.  Right?

One of things that it a common problem occurs when the C++ code is compiled in debug and then you try to run it on a non-development machine.  In this case the issue is that the debug C Run-Time (CRT) libraries is not found on production machines.  You can see this if you use something like the old Depends utility (Google it) which shows you the DLLs that are referenced by a executable image.

Run Depends on the machine on which you get the above error.  Does it find all the DLLs?

Another issue is that old school C++ DLLs are found using the PATH environment variable.  That can vary depending on who the user is.  I don't know which user GP tasks run as but you should be able to figure that out.  Once you so then you can use a utility such as:

http://technet.microsoft.com/en-us/sysinternals/cc300361.aspx

In order to run your program as same user as the GP task is running as.  Your goal is to try and simulate the same execution environment as the GP task.

[AlanStewart]

So you are trying to run a 32 bit process (written in C++) and it is returning DLL not found.  Right?

That is correct.

One of things that it a common problem occurs when the C++ code is compiled in debug and then you try to run it on a non-development machine.  In this case the issue is that the debug C Run-Time (CRT) libraries is not found on production machines.  You can see this if you use something like the old Depends utility (Google it) which shows you the DLLs that are referenced by a executable image.

Run Depends on the machine on which you get the above error.  Does it find all the DLLs?

I'm building everything on the same machine that I am running ArcGIS Server, so all needed runtimes are certainly there. Also, I am building in release, not debug. It works fine in debug! That should be a clue, but so far I don't understand it. The EXE also runs fine when executed from a command shell (with the working directory set to something other than the folder containing the EXE) and when executed from the Desktop version of our product (same mechanism as in Server, just 32-bit COM DLLs instead of 64-bit), whether directly or from a foreground GP task.

Another issue is that old school C++ DLLs are found using the PATH environment variable.  That can vary depending on who the user is.  I don't know which user GP tasks run as but you should be able to figure that out.  Once you so then you can use a utility such as:

http://technet.microsoft.com/en-us/sysinternals/cc300361.aspx

In order to run your program as same user as the GP task is running as.  Your goal is to try and simulate the same execution environment as the GP task.

At this point I've inserted a number of printf statements into the code to log various items, including the value of PATH at the time CreateProcess() is called. It is identical to the value of the same in a command shell, where the EXE runs successfully. CreateProcess() itself succeeds, the error is returned by the process that is created. Earlier CreateProcess() was the culprit, but ensuring that the ArcGIS Server user has execute privilege and registering the folder containing it as a data store seems to have resolved that error.

I'm in agreement that this has to be a runtime environment issue. I'm fairly certain the user is the ArcGIS Server user that Server processes run as, but I'll put in a runtime check to be sure, and see if using RunAs reveals anything.

Thanks for the advice.

[AlanStewart]

In order to run your program as same user as the GP task is running as.  Your goal is to try and simulate the same execution environment as the GP task.

I've confirmed that the user is the ArcGIS Server user and experimented with using 'runas' to launch the EXE as that user. This works just fine.

Is it possible this has something to do with sandboxing by IIS or ArcGIS Server?

[RichardWatson · Accepted solution]

Run the Microsoft SysInternals Process Monitor utility and then make your progam fail.

The volume of output from the utility is overwhelming so the key to being productive is filtering.

[AlanStewart]

Run the Microsoft SysInternals Process Monitor utility and then make your progam fail.

Thanks, I haven't used 'procmon' in many a year. I'd quite forgotten about it. It was the DLLs in the same folder with the EXE that needed read/execute permission for the ArcGIS user. The easy way to do this is to grant that privilege for the entire folder. I also discovered that registering the folder as a data store with Server is not necessary.