Calling ArcGIS Pro Python from ArcGIS Server 10.6.1

Question

We have enterprise 10.6.1 single machine deployment (windows). ArcGIS Pro 2.7 is installed on same EC2 instance Portal/Server are installed on. I am attempting to do the following: * GP Initiated through REST service call * GP calls a batch file through python SubProcess (import subprocess and popen method) like so: p = Popen([matrix_run_bat], stdout=PIPE, stderr=PIPE) p.wait() output, errors = p.communicate() print([p.returncode, errors, output]) arcpy.AddMessage([p.returncode, errors, output]) if p.returncode or errors: print('something went wrong...') arcpy.AddMessage('something went wrong...{0}, {1}'.format(errors, output)) 3. The batch file triggers another python file using ArcGIS Pro Python like : @echo Starting Telr Matrix call "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3-clone-telr-matrix\python" "D:\2nform\TELR_GRID\Pat_Dev elr-matrix\main.py" "run_telr_matrix" 10 2021 @echo Finished The python env in first argument of the bat file is a clone in my ArcGIS Pro env. The error I get when running gp service is that a module (that I know has been installed in the bat specified python env) in main.py is missing: esriJobMessageTypeInformative: something went wrong...b'Traceback (most recent call last): File "D:\2nform\TELR_GRID\Pat_Dev\telr-matrix\main.py", line 3, in from telr_matrix.run_telr_matrix import RunTelrMatrix File "D:\2nform\TELR_GRID\Pat_Dev\telr-matrix\telr_matrix\run_telr_matrix.py", line 4, in from telr_matrix.cbmp.cbmp import CBMP File "D:\2nform\TELR_GRID\Pat_Dev\telr-matrix\telr_matrix\cbmp\cbmp.py", line 5, in import geopandas as gpd ModuleNotFoundError: No module named \'geopandas\' ', b'Starting Telr Matrix C:\Program Files\ArcGIS\Server\framework\runtime\jre\bin>call "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3-clone-telr-matrix\python.exe" "D:\2nform\TELR_GRID\Pat_Dev\telr-matrix\main.py" "run_telr_matrix" 10 2021 Finished ' It looks as if it is still attempting to run the python script using the 10.6.1 Server python version instead of the ArcGIS Pro python specified in bat file. When I run subprocess locally from a variety of default python env, it will use the correct env specified in the bat call. And just FYI, the reason I want to use the ArcGIS Pro env and not update the default Server 10.6.1 env for the gp task is because of this ominous bit about updating module in default server env at 10.6.1: You will be modifying the defaultArcGIS Serverconda environment. In the event of environment corruption, you will need to re-install yourArcGIS Server. If you are usingArcGIS Server10.7 or later, a safer process to deploy your Python packages is available. So, should it be possible to call ArcGIS Pro Python from ArcGIS Server 10.6.1 python gp service using a subprocess that runs a bat script that specifies to use ArcGIS Pro Python? Thanks!

IhabHassan · Answer

Hi @PatSmyth 
If the role of the GP service is just to kickstart the python code, then it might be easier to do so using another RESTful service technology , like NodeJS or DotNet.
I don't have a workable code sample, but something like https://www.npmjs.com/package/python-shell can be a good start, which will allow you to run python from a specific python environment.

PatSmyth · Answer

Thanks for the help. I know for certain the Enterprise python and Pro python are out of sync. This is part of the reason I would like to use the subprocess + bat script to call my python program using the Pro python runtime rather than the difficult/risky to update Server python runtime.

cdickerson · Answer

Hi,

I don't have an exact answer for you but your Enterprise deployment and your Pro version are out of sync. There is a chance that if you are running it through Server it might be running on the older ArcMap runtime vs the new runtime that would be associated with Pro. (pardon if my terminology is a little off but this might help with your search for a solution)