Greetings,
I am working on updating my code to the new 2.4 version. I have a jupyter notebook that will loop through all of the web maps of an organization, it will list all of the layers within each webmap, and it will check the urls of the layers to check if they are valid. I am attempting to update my code to utilize the new Map module and the content property. When I run the code the behavior is different than it used to be. When it reaches a layer with an invalid url the code just errors out and does not continue to loop through the layers, or the other web maps. Here is a link to the notebook i am attempting to update: ArcGIS Online Web Map Service Validation Audit .
I imagine I could read the json of the webmap and check the layer urls using that method, but I am trying to learn/understand the new updates in the 2.4 version. I saw a blog that mentioned that the help documentation was updated, but I was unable to find examples on how the map module and content property works. I'm just kinda lost. Thank you in advance for any assistance you can provide. Thank you for everything and have a delightful day.
Sincerely,
Joe Guzi
Old Way:
...
print("Begin Execute Code Section.")
Audit = gis.content.search(query="", item_type="Web Map", max_items = 10000)
s = ""
for Item in Audit:
WebMapItem = WebMap(webmapitem = Item)
Layers = WebMapItem.layers
for Layer in Layers:
URL = Layer.get('url')
if URL != None:
try:
request = requests.get(URL)
if request.status_code == 200:
print('Web site exists')
else:
ItemID = str(Item.get("id"))
...
New Way
...
User = gis.users.me.username
#print(User)
OwnerQuery = "owner:" + User
#print(OwnerQuery)
Audit = gis.content.search(query=OwnerQuery, item_type="Web Map", max_items = 10000)
s = ""
for Item in Audit:
ItemID = str(Item.get("id"))
print("itemid")
print(ItemID)
WebMapItem = Map(item = Item)
print("webmap")
print(WebMapItem)
Layers = WebMapItem.content.layers #breaking here
print("layers")
print(str(Layers))
for Layer in Layers:
print(Layer.url)
URL = Layer.url
if URL != None:
try:
request = requests.get(URL)
if request.status_code == 200:
print('Web site exists')
else:
ItemID = str(Item.get("id"))
...
Error:
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18855, in _GISResource._hydrate(self)
18854 try:
> 18855 self._refresh()
18857 except HTTPError as httperror: # service maybe down
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18829, in _GISResource._refresh(self)
18828 else:
> 18829 raise e
18831 self._lazy_properties = _mixins.PropertyMap(dictdata)
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18818, in _GISResource._refresh(self)
18817 else:
> 18818 dictdata = self._con.post(
18819 self.url, params
18820 ) # , token=self._lazy_token)
18821 except Exception as e:
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1504, in Connection.post(self, path, params, files, **kwargs)
1503 return resp
-> 1504 return self._handle_response(
1505 resp=resp,
1506 out_path=out_path,
1507 file_name=file_name,
1508 try_json=try_json,
1509 force_bytes=kwargs.pop("force_bytes", False),
1510 )
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:979, in Connection._handle_response(self, resp, file_name, out_path, try_json, force_bytes, ignore_error_key)
978 errorcode = data["error"]["code"] if "code" in data["error"] else 0
--> 979 self._handle_json_error(data["error"], errorcode)
980 return data
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1002, in Connection._handle_json_error(self, error, errorcode)
1001 errormessage = errormessage + "\n(Error Code: " + str(errorcode) + ")"
-> 1002 raise Exception(errormessage)
Exception: Service SampleWorldCities/MapServer not started
(Error Code: 500)
During handling of the above exception, another exception occurred:
Exception Traceback (most recent call last)
Cell In[2], line 15
13 print("webmap")
14 print(WebMapItem)
---> 15 Layers = WebMapItem.content.layers #breaking here
16 print("layers")
17 print(str(Layers))
File /opt/conda/lib/python3.11/site-packages/arcgis/map/map_widget.py:506, in Map.content(self)
500 """
501 Returns a MapContent object that can be used to access the layers and tables
502 in the map. This is useful for adding, updating, getting, and removing content
503 from the Map.
504 """
505 if self._content is None:
--> 506 self._content = MapContent(self)
507 return self._content
File /opt/conda/lib/python3.11/site-packages/arcgis/map/map_widget.py:1604, in MapContent.__init__(self, webmap)
1600 self._helper = webmap._helper
1602 # Initialize layer list
1603 # list to be added/removed to when performing operations
-> 1604 self.layers = self._helper._layers
1606 # Initialize tables list
1607 # list to be added/removed to when performing operations
1608 self.tables = self._helper._tables
File /opt/conda/lib/python3.11/site-packages/arcgis/map/_utils.py:307, in _HelperMethods._layers(self)
304 operational_layers = self.pydantic_class.operational_layers or []
305 for index, layer in enumerate(operational_layers):
306 # index needed when dealing with group layers
--> 307 l = self._infer_layer(layer, index)
308 if l:
309 layers.append(l)
File /opt/conda/lib/python3.11/site-packages/arcgis/map/_utils.py:340, in _HelperMethods._infer_layer(self, layer, index)
337 return layer_class(layer_url, gis=self._source._gis)
339 if layer_url:
--> 340 return arcgis_layers.Service(layer_url, server=self._source._gis)
341 if item_id:
342 try:
File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_service_factory/_layerfactory.py:376, in ServiceFactory.__call__(cls, url_or_item, server, initialize)
374 server = ServicesDirectory(url=site_url)
375 return cls._get_layer_instance(layer_type, url, server, connection)
--> 376 return cls._get_layer_instance(layer_type, url, server)
File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_service_factory/_layerfactory.py:324, in ServiceFactory._get_layer_instance(layer_type, url, server, connection)
322 elif layer_type == _arcgis.geocoding._functions.Geocoder:
323 return layer_type(location=url, gis=server)
--> 324 return layer_type(url=url, gis=server)
File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_msl/map_layers.py:2287, in MapImageLayer.__init__(self, url, gis)
2277 """
2278 .. Creates a map image layer given a URL. The URL will typically look like the following.
2279
(...)
2283 :param gis: the GIS to which this layer belongs
2284 """
2285 super(MapImageLayer, self).__init__(url, gis)
-> 2287 self._populate_layers()
2288 self._admin = None
2289 self._session = gis.session
File /opt/conda/lib/python3.11/site-packages/arcgis/layers/_msl/map_layers.py:2345, in MapImageLayer._populate_layers(self)
2343 layers = []
2344 tables = []
-> 2345 if "layers" in self.properties and self.properties.layers:
2346 for lyr in self.properties.layers:
2347 if "subLayerIds" in lyr and lyr.subLayerIds is not None: # Group Layer
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18841, in _GISResource.properties(self)
18839 return self._lazy_properties
18840 else:
> 18841 self._hydrate()
18842 return self._lazy_properties
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18878, in _GISResource._hydrate(self)
18875 try:
18876 # try as a public server
18877 self._lazy_token = None
> 18878 self._refresh()
18880 except HTTPError as httperror:
18881 _log.error(httperror)
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18829, in _GISResource._refresh(self)
18827 dictdata = self._con.get(self.url, params)
18828 else:
> 18829 raise e
18831 self._lazy_properties = _mixins.PropertyMap(dictdata)
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/__init__.py:18818, in _GISResource._refresh(self)
18812 dictdata = self._con.post(
18813 self.url,
18814 params,
18815 timeout=None, # token=self._lazy_token,
18816 )
18817 else:
> 18818 dictdata = self._con.post(
18819 self.url, params
18820 ) # , token=self._lazy_token)
18821 except Exception as e:
18822 if hasattr(e, "msg") and e.msg == "Method Not Allowed":
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1504, in Connection.post(self, path, params, files, **kwargs)
1502 if return_raw_response:
1503 return resp
-> 1504 return self._handle_response(
1505 resp=resp,
1506 out_path=out_path,
1507 file_name=file_name,
1508 try_json=try_json,
1509 force_bytes=kwargs.pop("force_bytes", False),
1510 )
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:979, in Connection._handle_response(self, resp, file_name, out_path, try_json, force_bytes, ignore_error_key)
977 return data
978 errorcode = data["error"]["code"] if "code" in data["error"] else 0
--> 979 self._handle_json_error(data["error"], errorcode)
980 return data
981 else:
File /opt/conda/lib/python3.11/site-packages/arcgis/gis/_impl/_con/_connection.py:1002, in Connection._handle_json_error(self, error, errorcode)
999 # _log.error(errordetail)
1001 errormessage = errormessage + "\n(Error Code: " + str(errorcode) + ")"
-> 1002 raise Exception(errormessage)
Exception: Service SampleWorldCities/MapServer not started
(Error Code: 500)
For the record, I have that service stopped on purpose because I am actively trying to test this code. The old way, the code wouldn't stop when it reached a layer that was invalid. It would gather information about the layer and keep going. That is what I am trying to debug in the 2.4 version.
Solved! Go to Solution.
Update:
Esri Support did submit a BUG-000173532... It was explained in the bug:
Status: As Designed
Additional Information: The `arcgis.map` module is a new module and although we carried over some functionality and workflows from the `arcgis.mapping` module, they are not one to one.
Alternate Solution:
Use the web map's JSON property to check the layers' URLs.
My thoughts: I don't totally find this to be an acceptable answer, but the JSON Solution is easy enough that it is only mildly inconvenient. At a minimum the documentation about the new arcgis.map.Map module needs to be improved so that we know exactly what it can and cannot do, especially if it cannot do what the old arcgis.mapping modules can do.
There have been some issues reported by other users using the new Map module. See the following thread. I would recommend using the Web Map's JSON (see my first comment in the aforementioned thread) until these issues are resolved.
Greetings @JakeSkinner
Thank you for your quick response. I had a feeling the JSON would be the work around. It's just a bummer because I have several scripts that follow a similar workflow, and now I have to rewrite them. It's sad when updates break everything. Time to get to work. Thanks again.
@JoeGuziStarkCountyOH I would recommend logging a tech support case as well. Not sure if the user from the previous post did, or not, but they will verify any bugs and log them. This will notify the proper people on the development team to implement a fix.
that is less fun then rewriting the code lol...
I kid, I will find some time and do that.
Can you set an Except statement to print the URL that cannot be reached so the script will continue instead of stop?
Greetings @Tom_Laue ,
Thank you for your suggestion. Unfortunately no, because the code breaks before it even gets to the part to retrieve the url. That is why I believe it is an issue with the new Map Module and content property. I like your train of thought though. The JSON work around does work. I have some more things to update, but that is the best way forward until the Map Module and content property gets fixed. Thank you for everything and have a delightful day.
Sincerely,
Joe Guzi
Update:
Esri Support did submit a BUG-000173532... It was explained in the bug:
Status: As Designed
Additional Information: The `arcgis.map` module is a new module and although we carried over some functionality and workflows from the `arcgis.mapping` module, they are not one to one.
Alternate Solution:
Use the web map's JSON property to check the layers' URLs.
My thoughts: I don't totally find this to be an acceptable answer, but the JSON Solution is easy enough that it is only mildly inconvenient. At a minimum the documentation about the new arcgis.map.Map module needs to be improved so that we know exactly what it can and cannot do, especially if it cannot do what the old arcgis.mapping modules can do.