SetParameterAsText is not including subdirectory of arcpy.env.scratchFolder

[jslinksy]

This is in ArcGIS Enterprise 10.9.1

So I have a geoprocessing service that creates an html file as a report. It creates this html file using the following environment/path. Here is the core of the code that I have an issue with: 

workspace = arcpy.env.scratchFolder
output_folder = create_ouput_folder(workspace) # creates datetime subdirectory inside the workspace
html_report = create_html_report_file(output_folder) # creates file in output folder

# Format file with data ...
# ...
# ...
# Done formatting file
arcpy.AddMessage(html_report.name) # prints correct path: <path_to_scratch_folder_with_jobID>/scratch/<datetime>/html_report.html

# This next line is the issue. Once deployed as a geoprocessing service, 
# the path to the file that is returned is: <path_to_scratch_folder_with_jobID>/scratch/html_report.html
arcpy.SetParameterAsText(1, html_report.name) 

Here is an example of the output which is almost correct: 

{
 "paramName": "outputFile",
 "dataType": "GPDataFile",
 "value": {"url": "<url_to_scratch_folder_with_jobID>/scratch/html_report.html"}
}

The problem is that the SetParameterAsText function does not return the full path to the html report. For some reason it does not include the datetime subdirectory of the scratch directory. I have verified that on our servers, the file is being created in the correct location which is: <path_to_scratch_folder_with_jobID>/scratch/<datetime>/html_report.html. But instead of returning that directory, the output is: <path_to_scratch_folder_with_jobID>/scratch/html_report.html And I can verify that the script IS printing the correct path (WITH the datetime subdirectory) right before the SetParameterAsText() call. I have also verified that the options are setup correctly for the output parameter (data type is File, type is Derived, direction is Output). Why is SetParameterAsText not including the datetime subdirectory in the result path?

[jslinksy]

Here is a small python script that you can upload as a geoprocessing service to recreate the issue. You will see that the output file url does not include the datetime directory. However, if you grab the datetime from the log info message and modify the url to include the datetime directory, you will get the file. 

import arcpy
import os

workspace = arcpy.env.scratchFolder
# create output file date
now = datetime.datetime.now()
file_date = now.strftime("%Y%m%d_%H%M%S")
# create output folder
output_folder = os.path.join(workspace, file_date)
os.mkdir(output_folder)
result_file_path = os.path.join(output_folder, 'result_file.txt')
result_file = open(result_file_path, 'w')
result_file.write('Hello World!')
result_file.close()
arcpy.AddMessage("Result File: " + result_file.name)
arcpy.SetParameterAsText(0, result_file.name)