ArcGIS API for Python and Story Map maintenance

[kmsmikrud]

Hi,

At the moment I'm working to learn python and also creating Story Maps. I'm trying to figure out more on the ArcGIS API for Python and the functionality related to Story Maps both creation and maintenance.

For instance, is there a way to use the API to look at the locations within a Shortlist Story Map and download the text information included to see all the urls such as links to nearby parks and find out which ones are broken? It would also be great if there was away just to view a table of all the info to make sure that links are updated to be pointing to the correct url.

I'm interested to know how folks are using the new API with Story Maps/web maps.

Thanks in advance,

Kathy

[RebeccaStrauch__GISP]

Hi Kathy, have you looked at ArcGIS Online Assistant  yet?  You can see and change some things like the URLs using that 

Keep in mind that Python and the API for Python are two different things.  Give me a call and I can set you up with some more tools that might help (internally).

[kmsmikrud]

Thanks Becky. I will check out the ArcGIS Online Assistant. 

I didn't write my question very well though, what I'm trying to get after is the urls that I have in the Shortlist description text boxes for the actual location within the Story Map. Is there a way to get at these urls. I also will need to check the URLS of the web map services so ArcGIS Online or the API seems useful for checking multiple maps.

Story Maps have a lot of urls in the text description themselves and it seems like it would be VERY useful to be able to check/see all these in the maintenance of the Story Maps. In fact it was a recent criticism I received in that it is such a manual process to update and/or just check these urls to be sure they are still valid. Does anyone have a workflow for this?

Thanks!!!

[SethLewis1]

It might be possible using the Python API to look at a given Story Map's webmap for the operational layers array and the description string that's contained as a part of each record. You'd need to find a way to parse out the URL values contained within the description text, possibly through a regular expression and then check the HTTP response codes of each URL. I don't have a workflow for this but for more general maintenance purposes the following script lets you check the HTTP status of RESTful services in a given webmap.

#import modules
from arcgis.gis import *
from IPython.display import display

#declare a connection to ArcGIS Online with credentials
gis = GIS("portal", "user", "pass")
print("Successfully logged in as: " + gis.properties.user.username)

import urllib

itemIdInput = input('Paste the webmap id')

#search for a specific web map

webmaps = gis.content.search(itemIdInput, 'Web Map')

#return the RESTful endpoints in the searched for webmap along with the HTTP response codes
for webmap in webmaps:
    webmapJSON = webmap.get_data(try_json=True)
    for layer in webmapJSON['operationalLayers']:
        if 'url' in layer:
            url = layer['url']
            try:
                response_code = urllib.request.urlopen(url).getcode()
            except IOError:
                response_code = 404
            success = (response_code == 200)
            print(layer['url'] + ' (' + str(success) + ')')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[kmsmikrud]

Hi Seth,

I read your response awhile back and it gave me hope. That said I haven't been focusing too much on this lately but need to start looking at it again. I really appreciate the code example above and ideas! If you've thought of anything since then please let me know. Thanks kindly for your help!

[EmmaHatcher]

Hi Kathy,

Building on Seth's comment, here's a way you could scan the story map webpages in case you're using them as the primary section content. But if you're just embedding the URLs in the story map text, you could use the same idea to scan a list of URLs (see comments in the code), and schedule this script a couple times a week, receiving an email summary. I left what I have so far with the email component commented out-- I think I'm missing a credentials step to actually make it work, and I'm evidently not providing the correct server name. I currently get an error associated with server = smtplib.SMTP(srvr), which says

gaierror: [Errno 11001] getaddrinfo failed

But if you don't need an email, this will print the URL messages to the python command space.

import arcgis
from arcgis.gis import GIS
from arcgis.apps import storymap
import urllib, smtplib
from getpass import getpass

username='DCRAOpenData'

## Create a connection to your portal for publishing (enter your ArcGIS Online 
##  password in the textbox that appears, then hit 'Enter' on your keyboard)
gis = GIS("https://www.arcgis.com", username, getpass())

## Find the storymaps to verify URLS (you could also just provide a list of URLs
find_storymaps = gis.content.search("title: story map name/naming convention", item_type="Web Mapping Application", max_items=1, outside_org=False)

#srvr = 'server.domain.com'
#from_email = 'Name <email@domain.com>'
#to_email = 'Name <email@domain.com>'
#subject = 'URLs in story maps, script results'

msgs = []

for f in find_storymaps: ## Replace with code at bottom if just scanning list of URLS
    print(f.title, f.id)
    csmap = storymap.JournalStoryMap(gis.content.get(str(f.id)))
    sections = csmap.properties["values"]["story"]["sections"]
    for s in sections:
        if s["media"]["type"] == "webpage":
            url = s["media"]["webpage"]["url"]
            try:
                response_code = urllib.request.urlopen(url).getcode()
                print(str(url) + ' in the ' + str(s["title"]) + ' section is current and accessible!')
            except IOError:
                print('---- URL not funtional: ' + str(url) + '----')
                msgs.append('---- URL not funtional: ' + str(url) + '----')

## If no errors were found, configure messages to say so
if msgs == []:
    msgs.append('All URLs are functional!')

##Set up the email of the final messages and send it                    
#final_msg = " ".join(msgs)
#email = """\
#From: %s
#To: %s
#Subject: %s

#%s
#""" % (from_email, to_email, subject, final_msg)


#server = smtplib.SMTP(srvr)
#server.sendmail(from_email, to_email, final_msg)
#server.quit()

### IF SCANNING A LIST OF URLS ###
#for f in find_storymaps:
#    try:
#        response_code = urllib.request.urlopen(url).getcode()
#    except IOError:
#        print('---- URL not funtional: ' + str(url) + '----')
#        msgs.append('---- URL not funtional: ' + str(url) + '----')
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍