Problems sharing complex geoprocessing services to portal

[eiriksd]

I have a case where I would like to use the BatchTrace tool from the UtilityDataManagementSupport (UDMS) toolbox through the ArcGIS Rest API on my ArcGIS Enterprise server. I therefore would like to create a service-definition that I share as a geoprocessing service. I am going to use arcgis rest api and arcpy in production, but have also tried with the "Share as web tool" function in ArcGIS Pro with the exact same problem.

My ideal solution would be to create a light wrapper .pyt file that imports the UDMS-toolbox and hardcodes some of the values that will be equal always for my case, making the usage simpler for the end users. It turns out that both this path and trying to share the BatchTrace tool directly encounters the same error, and I'm at my wits end trying to work out how to fix it:

I keep getting error "00068 Script Batch Trace contains broken project data source: String". This is after the tool has run successfully, both the light wrapper-tool AND the one from the .atbx-toolbox (where the whole toolbox was uploaded to the project in arcgis pro). I am pretty sure it is the dependencies that does not get packaged correctly but I have found no fixes or way to troubleshoot further.

I have also tried extracting all the logic and reimplementing the batch trace functionality but it is so complex and full of pitfalls I hope to not have to go that route...

Can anyone help me find out how to correctly package and share complex geoprocessing services and make it available in our portal? Either the BatchTrace tool inside udms or my lightweight wrapper (see code bellow)

For reference, the light wrapper tool code is (udms atbx-file is stored in the same folder): 

def execute(self, parameters, messages) -> None:
        in_utility_network = parameters[0].valueAsText
        trace_locations = parameters[1].valueAsText
        expression = parameters[2].valueAsText or ""
        calc_on_start = parameters[3].valueAsText

        arcpy.ImportToolbox(os.path.join(os.path.dirname(__file__), "UtilityDataManagementSupport.atbx"))

        arcpy.udms.BatchTrace(
           in_utility_network=in_utility_network,
           trace_locations=trace_locations,
           result_types="CALCULATE",
           trace_config="Trace Config:: BT LV Downstream Ex NS", # Hardcoded trace configuration for LV Downstream
           expression=expression,
           output_folder=None,
           key_field=None,
           summary_store_field=None,
           fields=self._FIELDS,
           calc_on_start=calc_on_start,
           history_field=None,
           default_terminal_id=None,
           code_block="",
        )

        parameters[4].value = str("SUCCESS")
        messages.addMessage("Batch Trace completed successfully.")

    def postExecute(self, parameters) -> None:
        return

[MikeMillerGIS]

If I recall correctly, there is a current bug(BUG-000160584)/issue with publishing the style of atbx UDMS is.  We build the ATBX with embedded python modules.  When ArcGIS Pro packages the ATBX for publishing, the consolidation process removes these embedded modules.  

I am not sure there is an easy way to publish this ATBX.  

[MikeMillerGIS · Accepted solution]

My only thought is to extract the atbx(it is a compressed file), and install the UDMS into the ArcGIS Servers and Pros python environment.  I am just not sure of the ramifications of this.  You can call the underlying batch trace function like this

results = udms.logic.batch_trace(
                utility_network=str(target_network),
                trace_locations=str(sp),
                result_types=result_type,
                output_folder=str(out),
                summary_store_field=None,
                field_mapping=[["FlagType", filter_class, "labeltext", "FIRST"]],
                key_field="GroupBy",
                expression=None,
                trace_config=f"Trace Config:: {added_tc}",
                calc_on_start=False,
                history_field="historydate",
            )

[eiriksd]

Alright, thanks for the reply. It is good to know that there is an actual bug and that my suspicion was correct. I see that the bug is set as "In product plan" but last change was in may 2025. Is there any chance it is around the corner or should I just check the bug periodically?

In the meantime I'll check out the possibility of adding UDMS as a python package rather than toolbox, and either do that or reimplement the main BatchTrace logic without dependencies in a standalone .pyt file.