Error when consolidating Python Web Tool. Not all the reference code is consolidated

[Cristian_Galindo]

In version ArcGIS Pro 2.9,3.0 and 3.1 the following structure of a web tool work as intended:

Updm2018
└── TestService.atbx
├  __init__.py
├  MyTestService.py
├  foo.py
└──bar
    ├ __init__.py
    ├ real_bar.py
    ├ file_required_by_baz.json
    └─ baz.py

The definition was staged  using the StageService function and the .sd file will include the required files.

Note: I also tried using the Save As Offline Service Definition tool:

It is worth to mention that the file MyTestService.py include the imports:

…
import foo
from bar import realbar
from bar import baz
…

Execution of the tool StageService in ArcGIS Pro 3.3.1 generates an .sd file with the following structure (.sd file is just a zip file):

│   manifest.xml
│   serviceconfiguration.json
│   tilingservice.xml
│
├───cd
│   └───updm2018
│           foo.py
│           MyTestService.py
│           TestService.atbx
│
├───esriinfo
│   │   iteminfo.xml
│   │
│   └───metadata
│           metadata.xml
│
└───p30
        MyTestService.msd
        MyTestService.rltx
        MyTestService.tbx

There are missing files required by the tool to properly run:

real_bar.py
file_required_by_baz.json
baz.py

The attached file contains the structure folder and Web Tool to reproduce the issue.

Following the indications of the documentation , it should be possible to do it. As a matter of fact it has been working with out any issue, until this version.

[Cristian_Galindo]

Hello people from the future, in my mirrored post in gis.stackexchange one of the users said:

"I chatted with the Esri Geoprocessing Services Team -- they confirmed this is a bug and have entered a ticket into their system. I can't make any promises on their behalf, but they've set a target to fix in ArcGIS Pro 3.4"

if that is true, I can't afford such delay, therefore I had to figure a solution:

1. In root level, I created a file called modules_importer.py:

Updm2018
└── TestService.atbx
├  __init__.py
├  MyTestService.py
├  modules_importer.py
├  foo.py
└──bar
    ├ __init__.py
    ├ real_bar.py
    ├ file_required_by_baz.json
    └─ baz.py

 2. Inside the file I put the following code:

import arcpy
import sys
from os.path import exists


def set_path():
    bar_module = r".\bar"
        
    if exists(arcpy.env.packageWorkspace + '\\..\\cd\\'):
        sys.path.append(arcpy.env.packageWorkspace + '\\..\\cd\\')

3. Modified the imports of the file MyTestService.py:

...
import arcpy
import foo

arcpy.AddMessage("Loading own modules")
import modules_importer
modules_importer.set_path()
arcpy.AddMessage("AfterLoading own modules")

import bar.real_bar as real_bar
import bar.baz as baz
....

4. When the stage definition is created, is very important  to select the Copy all data option (in ArcGIS Pro):

or its equivalent  in python: setting True the property copyDataToServer in the GeoprocessingSharingDraft class.

when all this is set, the creation of the .sd file will consolidate, and the file structure will be:

│   manifest.xml
│   serviceconfiguration.json
│   tilingservice.xml
│
├───cd
│   ├───updm2018
│   │        foo.py
│   │       MyTestService.py
│   │       TestService.atbx
│   └───bar
│           real_bar.py
│           file_required_by_baz.json
│            baz.py
│
├───esriinfo
│   │   iteminfo.xml
│   │
│   └───metadata
│           metadata.xml
│
└───p30
        MyTestService.msd
        MyTestService.rltx
        MyTestService.tbx

and the file modules_importer.py will be consolidated as:

# Esri start of added imports
import sys, os, arcpy
# Esri end of added imports

# Esri start of added variables
g_ESRI_variable_1 = os.path.join(arcpy.env.packageWorkspace,'..\\cd\\bar')
# Esri end of added variables

import arcpy
import sys
from os.path import exists

def set_path():    
    bar_module = g_ESRI_variable_1
    # pythonModules = r".\\" # this line make a recursive call to the current folder
    arcpy.AddMessage("Call set_path")
    arcpy.AddMessage(arcpy.env.packageWorkspace + '\\..\\cd\\')
    if exists(arcpy.env.packageWorkspace + '\\..\\cd\\'):
        sys.path.append(arcpy.env.packageWorkspace + '\\..\\cd\\')