Latest Contributions by Egge-Jan_Pollé

Do you recognize this? "Over time our ArcGIS Online organization has grown. We have quite a few users now and a considerable number of groups has been created. Can you please provide me with a quick overview of who is member of which group?" "Yes, I can." "OK - and please note: not only quick, but also in Excel. Is that possible?" "Eh... yeah, that is also possible." We are going to use the ArcGIS API for Python with groups.search() and users.search() to get an overview of our organization. Next we compile a matrix - users along the y-axis and groups along the x-axis - with a 1 or a 0 to indicate wether the user is a member of the group or not. This matrix is being written to a CSV file. And then the boss should be able to import this CSV into a colorful and manageable spreadsheet. With or without a little help. Does the script below work for you? Just 'like' or 'share' if it does. And if you want to, you can just open your Jupyter Notebook to try it line by line. ## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ## ArcGIS Online Management Information ## Script: agol_group_membership.py ## Goal: to create an overview of group membership in your ArcGIS Online organisation ## Author: Egge-Jan Polle - Tensing GIS Consultancy ## Date: August 3, 2018 ## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # # This script should be run within a specific ArcGIS/Python environment using the batch file below # (This batch file comes with the installation of ArcGIS Pro) # "C:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" agol_group_membership.py # import csv,os, sys from arcgis.gis import GIS from provide_credentials import provide_credentials print('===================') print('The script that is running: ' + __file__) print('First you have to log in to ArcGIS Online') # Log in username, password = provide_credentials() my_agol = GIS("https://www.arcgis.com", username, password) print ("Start: "+datetime.datetime.today().strftime('%c')) ## Get all groups my_groups = my_agol.groups.search() ## Optionally: have a look at all groups #my_groups ## Optionally: count the number of groups #len(my_groups) ## Get all users my_users = my_agol.users.search(max_users = 350) # The default of max_users = 100, so increase it if you have more ## Optionally: have a look at all users #my_users ## Optionally: count the number of users #len(my_users) ## Create a list with all group titles my_group_titles = [] for my_group in my_groups: my_group_titles.append(my_group.title) ## Create a list with field names fieldnames = [] fieldnames = ['USERNAME','EMAIL'] for title in my_group_titles: fieldnames.append(title) ## Optionally: have a look at the field names #fieldnames ## Create a CSV file with a matrix of the groups with their members today = datetime.datetime.today().strftime('%Y%m%d') fname = 'AGOL_Group_Membership'+today+'.csv' try: os.remove(fname) except OSError: pass outfile = open(fname, 'a') writer = csv.DictWriter(outfile, delimiter = ';', lineterminator='\n', fieldnames=fieldnames) writer.writeheader() ## Add for each user the full name and email and for each group a 1 or 0, depending on group membership for user in my_users: membership = [] try: thisUser = {} thisUser['USERNAME'] = user.fullName thisUser['EMAIL'] = user.email try: # Group membership outside the organisation will raise an error ("You do not have permissions to access this resource or perform this operation.") for group in user.groups: membership.append(group.title) except: pass for title in my_group_titles: if title in membership: thisUser[title] = 1 else: thisUser[title] = 0 except: print("PLEASE NOTE: no information can be retrieved about user "+user.fullName+".") pass writer.writerow(thisUser) outfile.close() print ("Ready: "+datetime.datetime.today().strftime('%c')) print() print('===================') print('The CSV file can be found here:') print(os.path.abspath(fname)) print('===================')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ And here is the provide_credentials script which is used above to login to AGOL: import json, os from getpass import getpass #to accept passwords in an interactive fashion def provide_credentials(): file_with_credentials = 'my_credentials.json' username = '' password = '' if os.path.exists(file_with_credentials): with open(file_with_credentials) as f: data = json.load(f) username = data['username'] password = data['password'] if not username or username == 'USERNAME' or not password or password == 'PASSWORD': username = input('Please enter your username: ') password = getpass('Please enter your password (this will remain invisible): ') return username, password‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ With the input file my_credentials.json: { "username":"USERNAME", "password":"PASSWORD" }‍‍‍‍ Happy coding! Egge-Jan ... View more

Hi all, My question is: why is it that domain values are not implemented to the fields in a layer in a FeatureServer when you publish Feature Classes from an FGDB containing domain values? This is the workflow I have followed (Python files to replicate this workflow can be found in the attached zip file): I have a Python (ArcPY) script to create an FGDB - with Editor Tracking and Attachments enabled, and wit coded domain values applied to multiple fields. After creation, I have (manually) zipped the FGDB Next I use a Python script to upload the FGDB to ArcGIS Online, and once uploaded the Feature Classes from this FGDB are published to a FeatureServer /Hosted Feature Layer. This script also takes care of all kinds of adminstrative tasks, like adding a Snippet, a Description, the Terms of Use, Tags, Credits and a Thumbnail. Everything looks fine after publication, except for the Coded Value Domains which where present in the FGDB: these are not applied to the fields in the layers in the FeatureServer... These fields just remain free fields with no domains attached. And that's a pity, isn't it? Can anyone advise? Pythons files used in this process (see attached zip file) - please feel free to have a look at the code; it creates just a simple FGBD for demonstration purposes: Step 1 above: arcpy_prepare_fgdb_example.py together with domain_definitions.py and feature_class_fields.csv Step 3 above: agol_publish_fc_from_fgdb_to_featureserver_layer.py together with provide_credentials.py and my_credentials.json Of course I can add the domains afterwards, manually (using the ArcGIS Online UI) or using the ArcGIS API for Python, but I just wonder why the domain values are not implemented in the first place. Thanks in advance, Egge-Jan ... View more

Leaflet is een JavaScript bibliotheek voor het maken van interactieve web maps. Als je in een Leaflet web map gebruik wil maken van ArcGIS services, dan moet je de Esri Leaflet plugin toevoegen. De standaardprojectie voor webkaarten is Web Mercator, en dat is ook de default voor Leaflet. Esri Nederland biedt echter heel erg mooie basiskaarten aan in de Nederlandse projectie (RD_New/EPSG: 28992). Is het ook mogelijk om deze kaarten als ondergrond te gebruiken? Ja, dat is mogelijk! Voor het gebruik van kaarten in een andere projectie dan Web Mercator is er de Proj4Leaflet plugin, en deze kan ook gebruikt worden in combinatie met Esri content. Natuurlijk is hier enige kennis van projecties, ruimtelijke referentiesystemen en tiling voor nodig. Het Esri Leaflet team geeft op hun GitHub pagina zelf een voorbeeld met een Engelse kaart (EPSG:27700). En deze web map laat zien dat je ook de basiskaarten van Esri Nederland in een Leaflet kaart kunt ontsluiten. Zie hiervoor de broncode van deze webpagina: TWIAV - Tips & trucs - Basiskaarten van Esri Nederland (in EPSG:28992) in Leaflet ... View more

Hoi, Wie gaat er ook naar de Esri GIS Tech 2019 op dinsdag 9 april 2019 in Rotterdam? Esri Nederland heeft een interessant programma in elkaar gezet. En natuurlijk is er ook weer de DevDay, met een middagprogramma in het Mees Auditorium, speciaal voor gewone ontwikkelaars. Tot ziens, over twee weken, in Rotterdam. Met vriendelijke groet, Best regards,   Egge-Jan Pollé GIS Consultant E: [email protected]    Tensing GIS Consultancy B.V. Achterweg 38 4181 AE Waardenburg The Netherlands KvK: 61927449 www.tensing.com ... View more

For all the railway buffs and trainspotters out there: recently a very nice data set has been published with high resolution aerial imagery of the Dutch railway network. This data set is available as a WMS OGC Web Service via this URL. The imagery has been flown in 2017/2018 on behalf of ProRail, the organization responsible for the maintenance of the public railway infrastructure in the Netherlands. The pictures have been taken from a helicopter (!!) from a height of only 190 meters on average. To appreciate this imagery in your ArcGIS Online Map Viewer: Go to Add > Add Layer from Web Select A WMS OGC Web Service Check Use as Basemap - this will allow you to zoom in to a level where you can spot individual railroad ties Click GET LAYERS to check the layer 'meest_recent' (i.e. most recent) - this will show the imagery from 2017/2018. (The images taken in 2014, 2015 and 2016 do have a considerable lower resolution.) Cool, isn't it? You might also want to check the ProRail imagery in this web map: TWIAV - Tips & trucs - Luchtfoto's ProRail (in EPSG:28992) ... View more

In this post we will demonstrate how you - as a Spatial Data Scientist - can use R to access an ArcGIS REST Service, by answering the following question: "Are there any abandoned railroads in Florida?". The ArcGIS Living Atlas of the World offers a nice data set with railroads in the United States: USA Railroads. Following the Service URL we can get to the query page of this Feature Layer. The R script below shows how you can quickly create an interactive map to answer our question. Please scroll down for a little explanation... library(httr) library(sf) library(tmap) url <- parse_url("https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services") url$path <- paste(url$path, "USA_Railroads_1/FeatureServer/0/query", sep = "/") url$query <- list(where = "STATE = 'FL'", outFields = "*", returnGeometry = "true", f = "geojson") request <- build_url(url) Florida_Railroads <- st_read(request) tmap_mode(mode = "view") tm_shape(Florida_Railroads)+tm_lines(col="NET_DESC", palette = "Set1", lwd = 5)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ In RStudio we start by loading the 3 following R packages: httr - to help us build the query URL sf - to read the spatial data tmap - to create the interactive map, based on the Leaflet JavaScript library We parse the URL to create a list where we can add all the parameters of our query - where, outFields, returnGeometry and f - with their appropriate values. As soon as the list is populated we use the function build_url() to create a properly encoded request. This request is the passed to the function st_read() to populate Florida_Railroads. This Florida_Railroads becomes both a simple features object and a data frame, i.e. it does contain both the geometry and the attribute values of the railroads. In the screen dump below we can see that the total number of records returned is 3305, which is well beyond the maxRecordCount (1000) for this Feature Layer. The GDAL library, which is used by the st_read() function in the sf package, provides automatic paging. Nice, isn't it? With the last two lines in the script we create the interactive map in the Viewer in RStudio. So, the answer to our question is: "Yes, there are quite a few abandoned railroads in Florida". These abandoned lines are marked in red in the map in the screen dump below. ... View more

Hi, I just wondered: is there a limitation on the number of views you can create on a Hosted Feature Layer? In ArcGIS Online and/or in Portal for ArcGIS? On this page (Create hosted feature layer views—Portal for ArcGIS | ArcGIS Enterprise) it is stated that You can create a maximum of 20 views from the same hosted feature layer. whereas on this page (Create hosted feature layer views—ArcGIS Online Help | ArcGIS) this limitation seems absent (although there is a limit of 20 to the number of categories you can assign to the item....) And this limitation of 20 views on a Hosted Feature Layer in Portal for ArcGIS Enterprise, is that just a limitation of the UI which can be circumvented using a Python script, as kimberly peter states on GitHub (Add information and a sample for creating hosted feature layer views using Python · Issue #340 · Esri/arcgis-python-api … )? With the Python script below I can create 25 views on a Hosted Feature Layer in ArcGIS Online. Would a similar approach be possible in Portal for ArcGIS Enterprise? TIA, Egge-Jan from arcgis.gis import GIS from arcgis.features import FeatureLayerCollection from provide_credentials import provide_credentials username, password = provide_credentials() my_agol = GIS("https://www.arcgis.com", username, password) service = my_agol.content.get("<serviceItemId_of_your_Hosted_Feature_Layer>") flc = FeatureLayerCollection.fromitem(service) clients = ['ONE','TWO','THREE','FOUR','FIVE','SIX', 'SEVEN','EIGHT','NINE','TEN','ELEVEN','TWELVE', 'THIRTEEN','FOURTEEN','FIFTEEN','SIXTEEN','SEVENTEEN','EIGHTEEN', 'NINETEEN','TWENTY','TWENTY-ONE','TWENTY-TWO','TWENTY-THREE','TWENTY-FOUR', 'TWENTY-FIVE'] for client in clients: view_name = service.name + "_View_" + client print(view_name) view = flc.manager.create_view(name=view_name, spatial_reference=None, extent=None, allow_schema_changes=True, updateable=True, capabilities='Query, Update', view_layers=None) for lyr in view.layers: query = "Description = '%s'"%(client) print(query) lyr.manager.update_definition({'viewDefinitionQuery': query})‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ ... View more