I have been trying to add (or move) a SwissTopo WMS layer to an existing arcgis Online map using the arcgis API for python.
Using the API, I can add a set a selected set of SwissTopo layers using WMS as a new map to my arcgis Online portal. (I am happy to share code for this if anyone is interested.)
I cannot add (or move) the WMS layer to another map.
I cannot find any API documentation addressing this, nor a useful answer after web searching.,
I can do the above manually on arcgis Online, but my requirements are to script this using python.
Are there examples or tutorials available for handling SwissTopo WMS layers with the arcgis API for python? Perhaps a Jupyter Notebook?
Thanks in advance for replies.
You should register the WMS (layers) to your portal first. From there it's possible to search for the registered layers and add them to any map you want.
This code should help you registering the WMS layers, the script has a lot of options, and I'm not sure whether they all work for your WMS.
from arcgis import GIS
import requests
from xml.etree import ElementTree
gis = GIS("pro") #using the very easy arcgis pro login, run this script from arcgis pro python console and make sure you're logged in
#inputs
wmsurl = 'https://the.wms.server/geoserver/namespace/wms?' #should end with ?
mapurl = 'https://the.wms.server/geoserver/namespace/ows?' #should end with ?
featureinfourl = 'https://the.wms.server/geoserver/namespace/ows?' #should end with ?
capabilitiesurl = wmsurl + 'SERVICE=WMS&request=getcapabilities' #should end with ?
xmlfile = r'path/to/yourcapabilitiesdocumentlocation\wms.xml'
portalproxy = None #r'https://your.portal.server/portal/sharing/proxy?' change this to you portal proxy when you want to want to proxy all requests through portal,
#layerdefautls and prefixes and postfixes
thumbnailfile = r'path/to/your/logofile'
usethumbnailfromWMS = True
downloadcapabilities = True # set this to False and save the getcapabilities xml to specified location yourself. could be helpful if your struggling with a corporate proxy
layerdescriptionpostfix = 'generic text about this layer'
defaultaccessInformation = 'CopyrightHolder'
snippetprefix = 'text to insert before layertitle in layer summary'
snippetpostfix = 'text to insert after layertitle in layer summary'
layerlicenseinfo = 'licenseinfo and use constraints go here'
defaulttags = 'Tags,Go,here'
useWMSkeywordsAsTags = True #set this to False if you only want the default tags
folder = None #'yourfolder' change to a string with existing folder name in your portal content
if downloadcapabilities:
resp = requests.get(wmsurl + 'service=wms&request=getcapabilities&version=1.3.0')
with open(xmlfile, 'w') as f:
f.write(resp.text)
if portalproxy is not None:
wmsurl = portalproxy + wmsurl
mapurl = portalproxy + mapurl
featureinfourl = portalproxy + featureinfourl
tree = ElementTree.parse(xmlfile)
root = tree.getroot()
for capability in root.findall('{http://www.opengis.net/wms}Capability'):
for rootlayer in capability.findall('{http://www.opengis.net/wms}Layer'): #root layer
#here we get all supported crs
crsresult = []
for crs in rootlayer.findall('{http://www.opengis.net/wms}CRS'):
crstext = crs.text
if crstext.startswith("EPSG:"):
crsresult.append(crstext.lstrip('EPSG:'))
for layer in rootlayer.findall('{http://www.opengis.net/wms}Layer'):
name = layer.find('{http://www.opengis.net/wms}Name').text
title = layer.find('{http://www.opengis.net/wms}Title').text
abstract = layer.find('{http://www.opengis.net/wms}Abstract').text
if abstract is None:
abstract = ''
bboxelement = layer.find('{http://www.opengis.net/wms}EX_GeographicBoundingBox')
xmin = bboxelement.find('{http://www.opengis.net/wms}westBoundLongitude').text
xmax = bboxelement.find('{http://www.opengis.net/wms}eastBoundLongitude').text
ymin = bboxelement.find('{http://www.opengis.net/wms}southBoundLatitude').text
ymax = bboxelement.find('{http://www.opengis.net/wms}northBoundLatitude').text
bbox = '{},{},{},{}'.format(xmin,ymin,xmax,ymax)
layertags = defaulttags
keywordlist = layer.find('{http://www.opengis.net/wms}KeywordList')
if keywordlist is not None:
for keyword in keywordlist.findall('{http://www.opengis.net/wms}Keyword'):
layertags = layertags+"," + keyword.text
attributionelement = layer.find('{http://www.opengis.net/wms}Attribution')
accessInformation = defaultaccessInformation
if attributionelement is not None:
accessInformation = defaultaccessInformation + ',' + attributionelement.find('{http://www.opengis.net/wms}Title').text
properties['title'] = title
properties['type'] = 'WMS'
properties['typeKeywords'] = 'Data, Service, Web Map Service, OGC'
properties['tags'] = layertags
properties['url'] = wmsurl
properties['description'] = abstract + layerdescriptionpostfix
properties['accessInformation'] = accessInformation
properties['snippet'] = snippetprefix + title + snippetpostfix
properties['licenseInfo'] = layerlicenseinfo
properties['text'] = '{"title":"WMS","url":"'+wmsurl+'","mapUrl":"'+mapurl+'","version":"1.3.0","layers":[{"name":"'+ name+'","title":"'+title+'","legendURL":"'+ mapurl + 'service=WMS&request=GetLegendGraphic&format=image%2Fpng&width=20&height=20&layer='+name+'","queryable":true}],"copyright":"none","maxHeight":5000,"maxWidth":5000,"spatialReferences":['+','.join(crsresult)+'],"format":null,"featureInfoUrl":"' +featureinfourl+'","featureInfoFormat":"text/html"}'
properties['extent'] = bbox
properties['thumbnailURL'] = mapurl + 'SERVICE=WMS&REQUEST=GetMap&FORMAT=image/png&TRANSPARENT=TRUE&STYLES=&VERSION=1.3.0&LAYERS='+name+'&WIDTH=800&HEIGHT=532&CRS=EPSG:4326&BBOX=50.720978888973335,3.238626318098767,53.59715706128808,7.262998577472529'
if usethumbnailfromWMS:
thumbnailfile = properties['thumbnailURL']
gis.content.add(properties,thumbnail = thumbnailfile, folder=folder)
Thanks a lot for your reply.
The gis.content.add() method is what I have been using. This puts the retrieved WMS layer(s) into a new map, which has the name given by the item title.
Now if I do this several times, I end up with a set of "WMS" maps, each of which has one set of retrieved WMS layers.
What I want is one map in arcgis online that has all the retrieved WMS layer sets as layers. This can be done manually. I want to script it.
As far as I can see, your method is a very nice elaboration of what I am doing (and I will probably use it to harden my procedure assuming that is ok with you). However, it doesn't solve my issues as described above.
Any more ideas?
Thanks again :)
Just for clarification:
> You should register the WMS (layers) to your portal first.
Yes, this is what I am doing. This works for me. The WMS layers are on my portal.
>From there it's possible to search for the registered layers and add them to any >map you want.
This is where I am having difficulty with the arcgis API. Any hints would be appreciated.
this should probably work, haven't tested it though
first I get the Portal registered WMS, I retrieve the url and create a new WMSlayer with that url and then I add the first layer.
from arcgis.gis import GIS
from arcgis.mapping.ogc import WMSLayer
from arcgis.mapping import WebMap
wmslayers = gis.content.search("DataCentraWarmte", item_type="WMS", max_items=5)
print (wmslayers[0].itemid)
layer = WMSLayer(wmslayers[0].url)
wm = WebMap()
wm.add_layer(layer.layers[0])
Would be much easier when you could just add the item to the map and let the Python api find out what kind of layer the item contains
Once more, many thanks.
Unfortunately using this procedure still doesn't achieve what I need, as described above.
First I fixed a typo in your code:
wmslayers = gis.content.search("DataCentraWarmte", item_type="WMS", max_items=5)
print (wmslayers[0].itemid)
layer = WMSLayer(wmslayers[0].url)
wm = WebMap()
wm.add_layer(layer) # FIXED
Then I see that the output does not match the WMS 'layers' that I select for gis.content.add(). What happens (for me, might be different for your WMS) is that WMSLayer(wmslayers[0].url just gets the SwissTopo WMS URL with no layers selected, resulting in my case with hundreds of layers being put into the map.
So still no joy.
I have been going through various methods systematically and all the methods I have tried have issues with a WMS layer :(
I am starting to think this may be an ESRI feature ;)
Hopefully I am wrong.
(and I will probably use it to harden my procedure assuming that is ok with you)
Yes, please steal my code :)
My last code example wasn't my best one. I knew I had to do something with the layers object within the WMS layer, but not that way.
If you want a subset of all the layers in the WMS you should use this. This time I tested my code:
This WMS had three layers. I only want the first one visible in my map
using this line of code,I overwrite the layer list in my WMS layer (wm.layers[0]) with a new list containing only the first layer
wm.layers[0].layers = [wm.layers[0].layers[0]]
Which gives me this:
Now, when I run:
wm.update()
The map is saved and only the remaining layer is visible in the Map viewer
Hope this helps
Thanks again for your tips. Using WMSlayer.layers property is a potential workaround, but apparently there are layer indexes in webmap that also need to be set (?). WMSlayer however does not have any methods associated with it.
I have now done a fair amount of testing for handling WMS layers with the arcgis API for Python for arcgis Online. My conclusions so far are:
- WMS layers can be created as content
- a practical way to add WMS layers to a webmap was not found
- the documentation, API methods and exception handling for WMS layers are rudimentary.
ESRI developers present in this forum (hopefully) are welcome to respond.
A code snippet of my efforts follows.
# test retrieving WMS layers and putting these onto a webmap on arcgis online portal
from arcgis.gis import GIS
from arcgis.mapping import WebMap
from arcgis.mapping.ogc import WMSLayer
import json
gis = GIS("https://poyry.maps.arcgis.com/", USERNAME, PASSWD)
# retrieve WMS layers
spatialReferences = [2056, 2400, 3006, 3007, 3008, 3009, 3010, 3011, 3012, 3013, 3014, 3015, 3016, 3017, 3018,
3021, 3031, 3032, 3033, 3035, 3785, 3857, 4326, 30061, 30062, 900913, 3152, 30063, 30064]
# above list works -> need to research exactly what is needed
layers= [
{
"name": "ch.swisstopo.geologie-eiszeit-lgm",
"title": "Letzteiszeitl. Max. (Vektor) 500",
"queryable": True
},
{
"name": "ch.swisstopo-karto.skitouren",
"title": "Skirouten",
"queryable": True
}
]
text_layers = {
"title": "SwissTopo",
"url": "http://wms.geo.admin.ch/",
"version": "1.3.0",
"layers": layers,
"spatialReferences": spatialReferences ,
"format": None,
"featureInfoFormat": "text/html"
}
item={
'title':'SwissTopo_WMS_testing',
'description':'SwissTopo_testing2',
'tags':'wms',
'type':'WMS',
'url':'http://wms.geo.admin.ch/',
'extent': {'xmin': 5.83581, 'ymin': 45.65916, 'xmax': 10.97931, 'ymax': 47.86991}, # not working
'text': json.dumps(text_layers)
}
swisstopo_testing_wms = gis.content.add(item_properties=item)
print("swisstopo_testing_wms: ", swisstopo_testing_wms)
display(swisstopo_testing_wms) # results in 2 WMS layers grouped under a single webmap layer, OK
# put WMS layers onto webmap
# attempt #1 using .add_layer() -> NG since swisstopo_testing_wms.url is general to all SwissTopo layers
layer = WMSLayer(swisstopo_testing_wms.url)
wm = WebMap()
wm.add_layer(layer)
wm_item_properties = {'title':'WMS_test',
'snippet':'Map created using Python API',
'tags':['python'],
}
wm.save(wm_item_properties) # puts all 705 SwissTopo layers (!) onto a webamp on portal
# attempt #2 using .publish() -> NG as content.publish() method does not wor for WMS
#swisstopo_htesting_feaure_layer_item = swisstopo_testing_wms.publish(
# publish_parameters={'file_type':"WMS"})
# ValueError: A file_type must be provide, data format not recognized
# attempt #3 using WMSlayer.layers property -> NG as wrong layer is displayed in Map Viewer
wm.layers[0].layers = wm.layers[0].layers[501] # replace 705 SwissTopo layers with a single layer
#wm.update()
# RuntimeError: Item object missing, you should use `save()` method if you are creating a new web map item ??
wm.save(wm_item_properties) # results in a single webmap layer entry, which is correctly shown in Overview,
# however the Map Viewer displays the wrong layer !
I totally agree with your conclusions. Maybe somebody from Esri can take a look at them? @AndrewChapkowski
I'll add it to the backlog of issues to get it on our board.
Thank you for all the code and the description of the problem. The ArcGIS API team will take a look at this Post 1.8.5 release, which will occur next month.