topic Re: Jupyter Notebook for Broken Links in Python Questions

Jupyter Notebook for Broken Links

ArielLow2146 — Mon, 29 Jan 2024 18:15:42 GMT

I'm hoping to create a Jupyter Notebook that I can schedule to notify me if there are any broken feature service links in my web maps.

Originally I was going to combine two Esri sample scripts, one to detect the broken links and one to send an email notification.

Unfortunately the Esri sample script to detect broken links doesn't work because the error message if a layer number has changed or if the service has been deleted isn't in the response header. Curious whether anyone has already written a script to do something similar and look at the body of the response to determine if the feature service is working.

Re: Jupyter Notebook for Broken Links

AlexanderDanielPratama — Fri, 23 Feb 2024 12:54:30 GMT

I guess you need to list first all the url in the web app that has been used. Then, you can use urllib3 library. Usually broken link is 404, then you use smtp library to send to your email.

Hopefully, it helps you

Cheers

import urllib3 from http.client import responses http = urllib3.PoolManager() request = http.request('GET', 'http://server.arcgis.com/service/rest/to/your/service') http_status = request.status http_status_description = responses[http_status]

source: python - urllib3 how to find code and message of Http error - Stack Overflow

Re: Jupyter Notebook for Broken Links

ArielLow2146 — Tue, 27 Feb 2024 17:18:30 GMT

I ended up doing the following:

broken_urls = {}

for item in web_map_items:
    web_map = WebMap(item)
    print("-" * 79)
    print(f"Checking web map '{item.title}' (id: {item.id})...")
    layers = [layer for layer in web_map.layers + web_map.basemap.baseMapLayers if hasattr(layer, "url")]
    for layer in layers:
        try:
            response = requests.get(
                url=layer.url,
                params={"f": "json"}
            )
            response = json.loads(response.text)
            try:
                print(f"[✗] {layer.title}: layer URL returned {response['error']}. {layer.url}")
                broken_urls.setdefault(item.title + " - " + item.id, []).append(f"{layer.title}: layer URL returned {response['error']}. {layer.url}")
            except:
                print(f"[✓] {layer.title}: layer URL is working.")
        except requests.ConnectionError as e:
            print(f"[✗] {layer.url}: unable to reach layer due to '{e}'")
            broken_urls.setdefault(item.title + " - " + item.id, []).append(f"{layer.title}: unable to reach layer due to '{e}'. {layer.url}")

Looking at the response header doesn't work because layers return a 200 regardless of whether the layer url is functional in a web map. The actual error is in the json.

For the email component I used the arcgis.gis.Group.notify function.