topic Upload local Image file to ArcGIS Image for AGOL using API in Imagery in ArcGIS Online Questions

Upload local Image file to ArcGIS Image for AGOL using API

MichaelWen — Fri, 18 Feb 2022 20:57:23 GMT

Is there a way to programmatically upload image files to ArcGIS Image for ArcGIS Online? There doesn't seem to be an Upload operation in ImageServer and I couldn't find anything applicable in ArcGIS REST API.

If there is no way to upload images to Image Services then are there other way to share image files programmatically with AGOL?

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Thu, 24 Feb 2022 22:53:50 GMT

Hey Michael,

You can add images through the Python API to ArcGIS Image for ArcGIS Online. You can create and analyze the imagery layer in ArcGIS Online through the API.

arcgis.raster.module

Take a look there and see if that is what you were looking for.

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Thu, 16 Feb 2023 14:26:29 GMT

Hello @JeffreySwain , @MichaelWen, sorry for reviving this. We have been trying to implement our workflows on several rasters we use mainly for visualization on our arcgis online organization. We use "copy_raster" from raster.analytics module to create mosaics of images (as described here), no issues there. But we are not able then to edit or substitute our tiled imagery layers, only to create new ones with new serviceItemIDs (which breaks all our apps). We need at least to be able to "substitute" our raster mosaic with new data every few hours, while keeping the same ID linked to our maps and dashboards. Is there anything we are missing??? Thanks in advance.

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Thu, 16 Feb 2023 19:53:13 GMT

Hey @GilMastrapa ,

Do you know if you are using dynamic or tiled imagery layers? The update will only work on dynamic image collections. So if you are not using the image collections, the update is not supported. As a question about your workflow, how large are the files that you are changing out every few hours?

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Thu, 16 Feb 2023 23:28:08 GMT

Thanks for the quick response.

We checked our code and we are using “copy_raster” default parameters, so after changing to ‘tiles_only’=False, we should be able to create dynamic imagery layers. Is this enough or only by using the “create_image_collection” function we will be able later to edit and update??

About your question regarding size, we are using as input a single-parameter netcdf file with several timeslots(~36Mb). We hope to do a similar procedure for several products on different intervals. Size in all of them should be similar, we do hope to rotate and only keep the current data hosted.

We did tried out the “create_image_collection” function, but errors arised, even when we were using the same input file as with "copy_raster". We tested both from an IDE and from an ArcGIS Notebook. Here is a snippet of our code:

from arcgis.raster.analytics import copy_raster
from arcgis.raster.analytics import create_image_collection
from arcgis.gis import GIS
gis = GIS( )

dataset = '/arcgis/home/accum_inmediata.nc'

def arcgis_save_raster(path_out, image_collection_name):

image_collection_item = create_image_collection(image_collection=image_collection_name,

input_rasters=path_out,

raster_type_name="NetCDF",

folder='Testing',

gis=gis)
arcgis_save_raster(dataset, 'test')

Sorry for the long post! Thanks for the help!

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Fri, 17 Feb 2023 14:50:14 GMT

Hey @GilMastrapa ,

You will need the image collection layer configuration in order to update it. So you will need to use that configuration. Sorry I wasn't precise before.

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Fri, 17 Feb 2023 15:19:29 GMT

Thanks @JeffreySwain , got it.

This creates another problem, we haven't been able to use that specific "create_image_collection" function. Is there any license requirement or limitation that applies only to it and not to other functions in the raster.analytics module (e.g. copy_raster)? Any other advice on what may be causing our errors? Thanks again.

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Fri, 17 Feb 2023 16:47:59 GMT

So you can create dynamic imagery layers, but not image collection varieties? If you can create the dynamic layers, then you should be able to create them. What is the specific error you are getting?

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Fri, 17 Feb 2023 19:51:02 GMT

Yes, i just redid the tests now, i'm able to create dynamic Imagery layers using copy_raster with 'tiles_only=False' flag.

On the other hand, when i run the create_image_collection function, i get this:

Exception                                 Traceback (most recent call last)
Input In [10], in <cell line: 13>()
      7 def arcgis_save_raster(path_out, image_collection_name):
      8     image_collection_item = create_image_collection(image_collection=image_collection_name,
      9                                                     input_rasters=path_out,
     10                                                     raster_type_name="NetCDF",
     11                                                     folder='Testing',
     12                                                     gis=gis)
---> 13 arcgis_save_raster(dataset, 'test4')

Input In [10], in arcgis_save_raster(path_out, image_collection_name)
      7 def arcgis_save_raster(path_out, image_collection_name):
----> 8     image_collection_item = create_image_collection(image_collection=image_collection_name,
      9                                                     input_rasters=path_out,
     10                                                     raster_type_name="NetCDF",
     11                                                     folder='Testing',
     12                                                     gis=gis)

File /opt/conda/lib/python3.9/site-packages/arcgis/raster/analytics.py:3879, in create_image_collection(image_collection, input_rasters, raster_type_name, raster_type_params, out_sr, context, source_mosaic_dataset, gis, future, **kwargs)
   3877 gis = _arcgis.env.active_gis if gis is None else gis
   3878 # url = gis.properties.helperServices.rasterAnalytics.url
-> 3879 return gis._tools.rasteranalysis.create_image_collection(
   3880     image_collection=image_collection,
   3881     input_rasters=input_rasters,
   3882     raster_type_name=raster_type_name,
   3883     raster_type_params=raster_type_params,
   3884     out_sr=out_sr,
   3885     context=context,
   3886     future=future,
   3887     md_to_upload=source_mosaic_dataset,
   3888     **kwargs,
   3889 )

File /opt/conda/lib/python3.9/site-packages/arcgis/_impl/tools.py:10558, in _RasterAnalysisTools.create_image_collection(self, image_collection, input_rasters, raster_type_name, raster_type_params, out_sr, context, future, md_to_upload, **kwargs)
  10555 if future:
  10557     return RAJob(gpjob, output_service)
> 10558 return RAJob(gpjob, output_service).result()

File /opt/conda/lib/python3.9/site-packages/arcgis/geoprocessing/_job.py:666, in RAJob.result(self)
    664 elif isinstance(self._item, (tuple, list)):
    665     [i.delete() for i in self._item if isinstance(i, Item)]
--> 666 raise e

File /opt/conda/lib/python3.9/site-packages/arcgis/geoprocessing/_job.py:658, in RAJob.result(self)
    651 """
    652 Return the value returned by the call. If the call hasn't yet completed
    653 then this method will wait.
    654 
    655 :return: object
    656 """
    657 try:
--> 658     return self._gpjob.result()
    659 except Exception as e:
    660     from arcgis.gis import Item

File /opt/conda/lib/python3.9/site-packages/arcgis/geoprocessing/_job.py:222, in GPJob.result(self)
    220     return self._process_fa(self._future.result())
    221 elif self._is_ra:
--> 222     return self._process_ra(self._future.result())
    223 elif self._is_ortho:
    224     return self._process_ortho(self._future.result())

File /opt/conda/lib/python3.9/concurrent/futures/_base.py:446, in Future.result(self, timeout)
    444     raise CancelledError()
    445 elif self._state == FINISHED:
--> 446     return self.__get_result()
    447 else:
    448     raise TimeoutError()

File /opt/conda/lib/python3.9/concurrent/futures/_base.py:391, in Future.__get_result(self)
    389 if self._exception:
    390     try:
--> 391         raise self._exception
    392     finally:
    393         # Break a reference cycle with the exception in self._exception
    394         self = None

File /opt/conda/lib/python3.9/concurrent/futures/thread.py:58, in _WorkItem.run(self)
     55     return
     57 try:
---> 58     result = self.fn(*self.args, **self.kwargs)
     59 except BaseException as exc:
     60     self.future.set_exception(exc)

File /opt/conda/lib/python3.9/site-packages/arcgis/geoprocessing/_support.py:322, in _future_op(gptool, task_url, job_info, job_id, param_db, return_values, return_messages)
    318 def _future_op(
    319     gptool, task_url, job_info, job_id, param_db, return_values, return_messages
    320 ):
--> 322     job_info = _analysis_job_status(gptool, task_url, job_info)
    323     resp = _analysis_job_results(gptool, task_url, job_info, job_id)
    325     # ---------------------async-out---------------------#

File /opt/conda/lib/python3.9/site-packages/arcgis/geoprocessing/_support.py:247, in _analysis_job_status(gptool, task_url, job_info)
    244     num_messages = num
    246 if job_response.get("jobStatus") == "esriJobFailed":
--> 247     raise Exception("Job failed.")
    248 elif job_response.get("jobStatus") == "esriJobCancelled":
    249     raise Exception("Job cancelled.")

Exception: Job failed.

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Fri, 17 Feb 2023 20:01:52 GMT

Can you confirm if you can create an image collection dynamic imagery layer through the web app? If you can, then it is something else going on. Please let me know.

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Fri, 17 Feb 2023 20:31:56 GMT

Hello again. I ran the create imagery layer routine on the web app. When choosing dynamic Imagery layers, i'm able to create image mosaics, BUT NOT image collections. Images below:

Re: Upload local Image file to ArcGIS Image for AGOL using API

JeffreySwain — Fri, 17 Feb 2023 21:04:14 GMT

Ok so there is a problem with the org with image collections. Also it appears you are using netCDFs can you tell me which version of netCDF you are using?

Re: Upload local Image file to ArcGIS Image for AGOL using API

GilMastrapa — Fri, 17 Feb 2023 21:15:50 GMT

Yes, it seems to be a problem with our org and image collections. We are using NetCDF4, latest version. It works without issues on copy_raster function. Is there any checklist that we may review?

Re: Upload local Image file to ArcGIS Image for AGOL using API

kaander08 — Wed, 29 May 2024 19:30:13 GMT

I am looking to create a similar workflow. Did you have any progress with this?