Select to view content in your preferred language

How to check for valid urls of layers in a webmap at ArcGIS Python api version 2.4

374
8
Jump to solution
2 weeks ago
JoeGuziStarkCountyOH
Frequent Contributor

Greetings,

I am working on updating my code to the new 2.4 version. I have a jupyter notebook that will loop through all of the web maps of an organization, it will list all of the layers within each webmap, and it will check the urls of the layers to check if they are valid. I am attempting to update my code to utilize the new Map module and the content property. When I run  the code the behavior is different than it used to be. When it reaches a layer with an invalid url the code just errors out and does not continue to loop through the layers, or the other web maps. Here is a link to the notebook i am attempting to update: ArcGIS Online Web Map Service Validation Audit .

I imagine I could read the json of the webmap and check the layer urls using that method, but I am trying to learn/understand the new updates in the 2.4 version. I saw a blog that mentioned that the help documentation was updated, but I was unable to find examples on how the map module and content property works. I'm just kinda lost. Thank you in advance for any assistance you can provide. Thank you for everything and have a delightful day.

Sincerely,

Joe Guzi

 

Old Way:

 

...
print("Begin Execute Code Section.") 
Audit = gis.content.search(query="", item_type="Web Map", max_items = 10000)
s = ""
for Item in Audit:
    WebMapItem = WebMap(webmapitem = Item)
    Layers = WebMapItem.layers
    for Layer in Layers:
        URL = Layer.get('url')
        if URL != None:
            try:
                request = requests.get(URL)
                if request.status_code == 200:
                    print('Web site exists')
                else:
                    ItemID = str(Item.get("id"))
...

 

 

New Way

 

...
User = gis.users.me.username
#print(User)
OwnerQuery = "owner:" + User
#print(OwnerQuery)
Audit = gis.content.search(query=OwnerQuery, item_type="Web Map", max_items = 10000)

s = ""
for Item in Audit:
    ItemID = str(Item.get("id"))
    print("itemid")
    print(ItemID)
    WebMapItem = Map(item = Item)
    print("webmap")
    print(WebMapItem)
    Layers = WebMapItem.content.layers #breaking here
    print("layers")
    print(str(Layers))
    for Layer in Layers:
        print(Layer.url)
        URL = Layer.url
        if URL != None:
            try:
                request = requests.get(URL)
                if request.status_code == 200:
                    print('Web site exists')
                else:
                    ItemID = str(Item.get("id"))
...

 

 

Error:

 

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18855, in _GISResource._hydrate(self)
  18854 try:
> 18855     self._refresh()
  18857 except HTTPError as httperror:  # service maybe down

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18829, in _GISResource._refresh(self)
  18828         else:
> 18829             raise e
  18831 self._lazy_properties = _mixins.PropertyMap(dictdata)

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18818, in _GISResource._refresh(self)
  18817     else:
> 18818         dictdata = self._con.post(
  18819             self.url, params
  18820         )  # , token=self._lazy_token)
  18821 except Exception as e:

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1504, in Connection.post(self, path, params, files, **kwargs)
   1503     return resp
-> 1504 return self._handle_response(
   1505     resp=resp,
   1506     out_path=out_path,
   1507     file_name=file_name,
   1508     try_json=try_json,
   1509     force_bytes=kwargs.pop("force_bytes", False),
   1510 )

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:979, in Connection._handle_response(self, resp, file_name, out_path, try_json, force_bytes, ignore_error_key)
    978     errorcode = data["error"]["code"] if "code" in data["error"] else 0
--> 979     self._handle_json_error(data["error"], errorcode)
    980 return data

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1002, in Connection._handle_json_error(self, error, errorcode)
   1001 errormessage = errormessage + "\n(Error Code: " + str(errorcode) + ")"
-> 1002 raise Exception(errormessage)

Exception: Service SampleWorldCities/MapServer not started 
(Error Code: 500)

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
Cell In[2], line 15
     13 print("webmap")
     14 print(WebMapItem)
---> 15 Layers = WebMapItem.content.layers #breaking here
     16 print("layers")
     17 print(str(Layers))

File /opt/conda/lib/python3.11/site-packages/arcgis/map/map_widget.py:506, in Map.content(self)
    500 """
    501 Returns a MapContent object that can be used to access the layers and tables
    502 in the map. This is useful for adding, updating, getting, and removing content
    503 from the Map.
    504 """
    505 if self._content is None:
--> 506     self._content = MapContent(self)
    507 return self._content

File /opt/conda/lib/python3.11/site-packages/arcgis/map/map_widget.py:1604, in MapContent.__init__(self, webmap)
   1600 self._helper = webmap._helper
   1602 # Initialize layer list
   1603 # list to be added/removed to when performing operations
-> 1604 self.layers = self._helper._layers
   1606 # Initialize tables list
   1607 # list to be added/removed to when performing operations
   1608 self.tables = self._helper._tables

File /opt/conda/lib/python3.11/site-packages/arcgis/map/_utils.py:307, in _HelperMethods._layers(self)
    304 operational_layers = self.pydantic_class.operational_layers or []
    305 for index, layer in enumerate(operational_layers):
    306     # index needed when dealing with group layers
--> 307     l = self._infer_layer(layer, index)
    308     if l:
    309         layers.append(l)

File /opt/conda/lib/python3.11/site-packages/arcgis/map/_utils.py:340, in _HelperMethods._infer_layer(self, layer, index)
    337     return layer_class(layer_url, gis=self._source._gis)
    339 if layer_url:
--> 340     return arcgis_layers.Service(layer_url, server=self._source._gis)
    341 if item_id:
    342     try:

File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_service_factory/_layerfactory.py:376, in ServiceFactory.__call__(cls, url_or_item, server, initialize)
    374             server = ServicesDirectory(url=site_url)
    375     return cls._get_layer_instance(layer_type, url, server, connection)
--> 376 return cls._get_layer_instance(layer_type, url, server)

File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_service_factory/_layerfactory.py:324, in ServiceFactory._get_layer_instance(layer_type, url, server, connection)
    322 elif layer_type == _arcgis.geocoding._functions.Geocoder:
    323     return layer_type(location=url, gis=server)
--> 324 return layer_type(url=url, gis=server)

File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_msl/map_layers.py:2287, in MapImageLayer.__init__(self, url, gis)
   2277 """
   2278 .. Creates a map image layer given a URL. The URL will typically look like the following.
   2279 
   (...)
   2283 :param gis: the GIS to which this layer belongs
   2284 """
   2285 super(MapImageLayer, self).__init__(url, gis)
-> 2287 self._populate_layers()
   2288 self._admin = None
   2289 self._session = gis.session

File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_msl/map_layers.py:2345, in MapImageLayer._populate_layers(self)
   2343 layers = []
   2344 tables = []
-> 2345 if "layers" in self.properties and self.properties.layers:
   2346     for lyr in self.properties.layers:
   2347         if "subLayerIds" in lyr and lyr.subLayerIds is not None:  # Group Layer

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18841, in _GISResource.properties(self)
  18839     return self._lazy_properties
  18840 else:
> 18841     self._hydrate()
  18842     return self._lazy_properties

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18878, in _GISResource._hydrate(self)
  18875 try:
  18876     # try as a public server
  18877     self._lazy_token = None
> 18878     self._refresh()
  18880 except HTTPError as httperror:
  18881     _log.error(httperror)

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18829, in _GISResource._refresh(self)
  18827             dictdata = self._con.get(self.url, params)
  18828         else:
> 18829             raise e
  18831 self._lazy_properties = _mixins.PropertyMap(dictdata)

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18818, in _GISResource._refresh(self)
  18812         dictdata = self._con.post(
  18813             self.url,
  18814             params,
  18815             timeout=None,  # token=self._lazy_token,
  18816         )
  18817     else:
> 18818         dictdata = self._con.post(
  18819             self.url, params
  18820         )  # , token=self._lazy_token)
  18821 except Exception as e:
  18822     if hasattr(e, "msg") and e.msg == "Method Not Allowed":

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1504, in Connection.post(self, path, params, files, **kwargs)
   1502 if return_raw_response:
   1503     return resp
-> 1504 return self._handle_response(
   1505     resp=resp,
   1506     out_path=out_path,
   1507     file_name=file_name,
   1508     try_json=try_json,
   1509     force_bytes=kwargs.pop("force_bytes", False),
   1510 )

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:979, in Connection._handle_response(self, resp, file_name, out_path, try_json, force_bytes, ignore_error_key)
    977             return data
    978         errorcode = data["error"]["code"] if "code" in data["error"] else 0
--> 979         self._handle_json_error(data["error"], errorcode)
    980     return data
    981 else:

File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1002, in Connection._handle_json_error(self, error, errorcode)
    999                 # _log.error(errordetail)
   1001 errormessage = errormessage + "\n(Error Code: " + str(errorcode) + ")"
-> 1002 raise Exception(errormessage)

Exception: Service SampleWorldCities/MapServer not started 
(Error Code: 500)

 

For the record, I have that service stopped on purpose because I am actively trying to test this code. The old way, the code wouldn't stop when it reached a layer that was invalid. It would gather information about the layer and keep going. That is what I am trying to debug in the 2.4 version.

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
JoeGuziStarkCountyOH
Frequent Contributor

Update:

Esri Support did submit a BUG-000173532... It was explained in the bug:

Status: As Designed
Additional Information: The `arcgis.map` module is a new module and although we carried over some functionality and workflows from the `arcgis.mapping` module, they are not one to one.

Alternate Solution:

Use the web map's JSON property to check the layers' URLs.

 

My thoughts: I don't totally find this to be an acceptable answer, but the JSON Solution is easy enough that it is only mildly inconvenient. At a minimum the documentation about the new arcgis.map.Map module needs to be improved so that we know exactly what it can and cannot do, especially if it cannot do what the old arcgis.mapping modules can do. 

View solution in original post

8 Replies
JakeSkinner
Esri Esteemed Contributor

Hi @JoeGuziStarkCountyOH,

There have been some issues reported by other users using the new Map module.  See the following thread.  I would recommend using the Web Map's JSON (see my first comment in the aforementioned thread) until these issues are resolved.

0 Kudos
JoeGuziStarkCountyOH
Frequent Contributor

Greetings @JakeSkinner 

Thank you for your quick response. I had a feeling the JSON would be the work around. It's just a bummer because I have several scripts that follow a similar workflow, and now I have to rewrite them. It's sad when updates break everything. Time to get to work. Thanks again. 

JakeSkinner
Esri Esteemed Contributor

@JoeGuziStarkCountyOH I would recommend logging a tech support case as well.  Not sure if the user from the previous post did, or not, but they will verify any bugs and log them.  This will notify the proper people on the development team to implement a fix.

0 Kudos
JoeGuziStarkCountyOH
Frequent Contributor

that is less fun then rewriting the code lol...

JoeGuziStarkCountyOH
Frequent Contributor

I kid, I will find some time and do that. 

Tom_Laue
Regular Contributor

Can you set an Except statement to print the URL that cannot be reached so the script will continue instead of stop?

0 Kudos
JoeGuziStarkCountyOH
Frequent Contributor

Greetings @Tom_Laue ,

Thank you for your suggestion. Unfortunately no, because the code breaks before it even gets to the part to retrieve the url. That is why I believe it is an issue with the new Map Module and content property. I like your train of thought though. The JSON work around does work. I have some more things to update, but that is the best way forward until the Map Module and content property gets fixed. Thank you for everything and have a delightful day.

Sincerely,

Joe Guzi

0 Kudos
JoeGuziStarkCountyOH
Frequent Contributor

Update:

Esri Support did submit a BUG-000173532... It was explained in the bug:

Status: As Designed
Additional Information: The `arcgis.map` module is a new module and although we carried over some functionality and workflows from the `arcgis.mapping` module, they are not one to one.

Alternate Solution:

Use the web map's JSON property to check the layers' URLs.

 

My thoughts: I don't totally find this to be an acceptable answer, but the JSON Solution is easy enough that it is only mildly inconvenient. At a minimum the documentation about the new arcgis.map.Map module needs to be improved so that we know exactly what it can and cannot do, especially if it cannot do what the old arcgis.mapping modules can do.