How to publish features as a hosted image layer using ArcGIS API for Python

[JYI]

Hi, 

I am looking for the ArcGIS Python API to publish a hosted map image layer from a web feature layer, https://pro.arcgis.com/en/pro-app/latest/help/sharing/overview/publish-a-map-image-layer-from-a-host... 

The reason is that the client found the hosted line feature layers have serious performance issues with huge number of polylines, in their dashboards. The hosted feature layers are published from Databricks using ArcGIS Python API. They are looking for a way to publish the features to a hosted image layer instead, using ArcGIS API for Python. The data is located in data lake, i.e., there is no way to use relational database like PostgreSQL.  If a direct way exists, that is ideal. Otherwise, a workaround would work. 

Thanks.

Update: the final solution is publishing a map service to a standalone ArcGIS Server, instead of the hosted map service in Portal. Code. 

[JYI]

I would publish a geoprocessing service with the tool https://pro.arcgis.com/en/pro-app/latest/help/sharing/overview/publish-a-map-image-layer-from-a-host...

Then in Databricks, call the rest service to publish the hosted feature layer as an image layer.

Update: it turns out that there is no this arcpy function available for this tool at this moment. 

[JYI · Accepted solution]

This is the final solution - after gone through a lot of pains, and gains of course. 

1. In ArcGIS Pro 2.9, create a Script tool, to publish .geojson file to a standalone ArcGIS Server, or Portal.

 Lessons learned: 

a. Cannot use ArcGIS Pro 3.0. The versions between ArcGIS Pro and enterprise Portal much match. Thanks to ESRI Tech support for this point.  Reference.

b. Must have  a standalone ArcGIS Server deployed in the enterprise Portal platform. Reference. 

c. Must use .zip files as input files, as .geojson file, .ags file, .lyrx, .aprx files are not recognized when running the web tool published to Portal. However, if you publish a Geoprocessing Service to the standalone server, then this is unnecessary, after you add MIME type to the web.config. 

d. Be careful with the env. settings. The script runs inside ArcGIS Pro, does not guarantee that it runs in the Portal web tool too. Usually the error is caused by the file path that inaccessible. Reference. 

e. For the map service symbology settings, the final solution is here, a workaround solution, due to Bug-000117445, 

f. The parameter Data types used is "File" for the zipped files, from the Script -> Tool Properties in ArcGIS Pro. 

2. Once the Geoprocessing service is successfully published to Portal, or to standalone server, call the services from Databricks, or a regular Python env. outside of ArcGIS env. 

Lessons learned:

a. File upload procedure is not included in the Geoprocessing service. This, however is not stated anywhere. When using arcgis.geoprocessing.DataFile, it always complains that the required  DataFile parameter "does not exist or is not supported". Thanks to ESRI Support, that this is because the required files are not on the standalone server yet. 

b. Do not want to call the "uploads ArcGIS REST API" to upload the 60MB files, before running the Geoprocessing service, as there will be performance issue. Use self hosted Integration Runtime to hoop up the data lake data with the Portal web server VM, such that user can access the data through a url. References: 1, and 2. 

c. Calling the Geoprocessing service is straight forward. References: 1 and 2. 

d. This Creating and Using Web Tools and Geoprocessing Services video is very helpful.