BLOG
|
Do you recognize this? "Over time our ArcGIS Online organization has grown. We have quite a few users now and a considerable number of groups has been created. Can you please provide me with a quick overview of who is member of which group?" "Yes, I can." "OK - and please note: not only quick, but also in Excel. Is that possible?" "Eh... yeah, that is also possible." We are going to use the ArcGIS API for Python with groups.search() and users.search() to get an overview of our organization. Next we compile a matrix - users along the y-axis and groups along the x-axis - with a 1 or a 0 to indicate wether the user is a member of the group or not. This matrix is being written to a CSV file. And then the boss should be able to import this CSV into a colorful and manageable spreadsheet. With or without a little help. Does the script below work for you? Just 'like' or 'share' if it does. And if you want to, you can just open your Jupyter Notebook to try it line by line. ## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
## ArcGIS Online Management Information
## Script: agol_group_membership.py
## Goal: to create an overview of group membership in your ArcGIS Online organisation
## Author: Egge-Jan Polle - Tensing GIS Consultancy
## Date: August 3, 2018
## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# This script should be run within a specific ArcGIS/Python environment using the batch file below
# (This batch file comes with the installation of ArcGIS Pro)
# "C:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" agol_group_membership.py
#
import csv,os, sys
from arcgis.gis import GIS
from provide_credentials import provide_credentials
print('===================')
print('The script that is running: ' + __file__)
print('First you have to log in to ArcGIS Online')
# Log in
username, password = provide_credentials()
my_agol = GIS("https://www.arcgis.com", username, password)
print ("Start: "+datetime.datetime.today().strftime('%c'))
## Get all groups
my_groups = my_agol.groups.search()
## Optionally: have a look at all groups
#my_groups
## Optionally: count the number of groups
#len(my_groups)
## Get all users
my_users = my_agol.users.search(max_users = 350) # The default of max_users = 100, so increase it if you have more
## Optionally: have a look at all users
#my_users
## Optionally: count the number of users
#len(my_users)
## Create a list with all group titles
my_group_titles = []
for my_group in my_groups:
my_group_titles.append(my_group.title)
## Create a list with field names
fieldnames = []
fieldnames = ['USERNAME','EMAIL']
for title in my_group_titles:
fieldnames.append(title)
## Optionally: have a look at the field names
#fieldnames
## Create a CSV file with a matrix of the groups with their members
today = datetime.datetime.today().strftime('%Y%m%d')
fname = 'AGOL_Group_Membership'+today+'.csv'
try:
os.remove(fname)
except OSError:
pass
outfile = open(fname, 'a')
writer = csv.DictWriter(outfile, delimiter = ';', lineterminator='\n', fieldnames=fieldnames)
writer.writeheader()
## Add for each user the full name and email and for each group a 1 or 0, depending on group membership
for user in my_users:
membership = []
try:
thisUser = {}
thisUser['USERNAME'] = user.fullName
thisUser['EMAIL'] = user.email
try: # Group membership outside the organisation will raise an error ("You do not have permissions to access this resource or perform this operation.")
for group in user.groups:
membership.append(group.title)
except:
pass
for title in my_group_titles:
if title in membership:
thisUser[title] = 1
else:
thisUser[title] = 0
except:
print("PLEASE NOTE: no information can be retrieved about user "+user.fullName+".")
pass
writer.writerow(thisUser)
outfile.close()
print ("Ready: "+datetime.datetime.today().strftime('%c'))
print()
print('===================')
print('The CSV file can be found here:')
print(os.path.abspath(fname))
print('===================') And here is the provide_credentials script which is used above to login to AGOL: import json, os
from getpass import getpass #to accept passwords in an interactive fashion
def provide_credentials():
file_with_credentials = 'my_credentials.json'
username = ''
password = ''
if os.path.exists(file_with_credentials):
with open(file_with_credentials) as f:
data = json.load(f)
username = data['username']
password = data['password']
if not username or username == 'USERNAME' or not password or password == 'PASSWORD':
username = input('Please enter your username: ')
password = getpass('Please enter your password (this will remain invisible): ')
return username, password With the input file my_credentials.json: {
"username":"USERNAME",
"password":"PASSWORD"
} Happy coding! Egge-Jan
... View more
03-21-2019
02:35 PM
|
4
|
10
|
2441
|
POST
|
Hi Kelly, Cool! Didn't know about this ArcGIS Online Activity Log yet: Exploring the Organization Activity Log CSV BTW - do you happen to know how many questions in the new ArcGIS Online Administration Technical Certification Exam will be dealing with this Activity Log? 🙂 Cheers, Egge-Jan
... View more
03-21-2019
09:43 AM
|
0
|
1
|
735
|
POST
|
I had to make a quick screendump to be able to read the full error message 🙂 BTW - please let us know if and when the issues have been solved (e.g. by marking this question as being answered)
... View more
03-21-2019
08:54 AM
|
1
|
0
|
735
|
POST
|
It looks like there is an issue with the layers DWR_Habitat_Layers_Service, Geology 100k (30X60 Quads), PLSS_Service and Sage Grouse Management Area (SGMA) as they cannot be added to the map. You say that everything worked fine until two days ago. So, it seems that two days ago something has happened that caused these issues. What could that be? By the way: two days ago, on the 19th of March 2019, ArcGIS Online has been updated: What's new in ArcGIS Online (March 2019) Might this have caused the problem? This is the error message that appears on my screen just before the systems asks me to login to Utah DNR Online Maps: HTH, Egge-Jan
... View more
03-21-2019
08:46 AM
|
1
|
5
|
735
|
POST
|
Hi all, My question is: why is it that domain values are not implemented to the fields in a layer in a FeatureServer when you publish Feature Classes from an FGDB containing domain values? This is the workflow I have followed (Python files to replicate this workflow can be found in the attached zip file): I have a Python (ArcPY) script to create an FGDB - with Editor Tracking and Attachments enabled, and wit coded domain values applied to multiple fields. After creation, I have (manually) zipped the FGDB Next I use a Python script to upload the FGDB to ArcGIS Online, and once uploaded the Feature Classes from this FGDB are published to a FeatureServer /Hosted Feature Layer. This script also takes care of all kinds of adminstrative tasks, like adding a Snippet, a Description, the Terms of Use, Tags, Credits and a Thumbnail. Everything looks fine after publication, except for the Coded Value Domains which where present in the FGDB: these are not applied to the fields in the layers in the FeatureServer... These fields just remain free fields with no domains attached. And that's a pity, isn't it? Can anyone advise? Pythons files used in this process (see attached zip file) - please feel free to have a look at the code; it creates just a simple FGBD for demonstration purposes: Step 1 above: arcpy_prepare_fgdb_example.py together with domain_definitions.py and feature_class_fields.csv Step 3 above: agol_publish_fc_from_fgdb_to_featureserver_layer.py together with provide_credentials.py and my_credentials.json Of course I can add the domains afterwards, manually (using the ArcGIS Online UI) or using the ArcGIS API for Python, but I just wonder why the domain values are not implemented in the first place. Thanks in advance, Egge-Jan
... View more
03-21-2019
03:44 AM
|
0
|
1
|
678
|
POST
|
Yeah, this modulo operation (%) is cool! See: https://en.wikipedia.org/wiki/Modulo_operation And, by the way, the formula works both ways, so both from Arithmetic to Geographic and from Geographic to Arithmetic, just like a switch: # e.g. Geographic North (12 o'clock) = 0
(450 - 0) % 360 # returns 90 (Arithmetic)
# e.g. Arithmetic South (6 o'clock) = 270
(450 - 270) % 360 # returns 180 (Geographic)
... View more
03-20-2019
03:19 PM
|
1
|
0
|
3551
|
POST
|
Probably the quickest way to get an answer to your question would be to contact the provider of the attribute table. They will (or at least should) be able to explain the content of the different fields. Egge-Jan
... View more
03-20-2019
12:09 PM
|
0
|
0
|
627
|
BLOG
|
Leaflet is een JavaScript bibliotheek voor het maken van interactieve web maps. Als je in een Leaflet web map gebruik wil maken van ArcGIS services, dan moet je de Esri Leaflet plugin toevoegen. De standaardprojectie voor webkaarten is Web Mercator, en dat is ook de default voor Leaflet. Esri Nederland biedt echter heel erg mooie basiskaarten aan in de Nederlandse projectie (RD_New/EPSG: 28992). Is het ook mogelijk om deze kaarten als ondergrond te gebruiken? Ja, dat is mogelijk! Voor het gebruik van kaarten in een andere projectie dan Web Mercator is er de Proj4Leaflet plugin, en deze kan ook gebruikt worden in combinatie met Esri content. Natuurlijk is hier enige kennis van projecties, ruimtelijke referentiesystemen en tiling voor nodig. Het Esri Leaflet team geeft op hun GitHub pagina zelf een voorbeeld met een Engelse kaart (EPSG:27700). En deze web map laat zien dat je ook de basiskaarten van Esri Nederland in een Leaflet kaart kunt ontsluiten. Zie hiervoor de broncode van deze webpagina: TWIAV - Tips & trucs - Basiskaarten van Esri Nederland (in EPSG:28992) in Leaflet
... View more
03-20-2019
10:27 AM
|
1
|
0
|
611
|
POST
|
Hoi, Wie gaat er ook naar de Esri GIS Tech 2019 op dinsdag 9 april 2019 in Rotterdam? Esri Nederland heeft een interessant programma in elkaar gezet. En natuurlijk is er ook weer de DevDay, met een middagprogramma in het Mees Auditorium, speciaal voor gewone ontwikkelaars. Tot ziens, over twee weken, in Rotterdam. Met vriendelijke groet, Best regards, Egge-Jan Pollé GIS Consultant E: ejpolle@tensing.com Tensing GIS Consultancy B.V. Achterweg 38 4181 AE Waardenburg The Netherlands KvK: 61927449 www.tensing.com
... View more
03-19-2019
07:24 AM
|
3
|
0
|
857
|
BLOG
|
For all the railway buffs and trainspotters out there: recently a very nice data set has been published with high resolution aerial imagery of the Dutch railway network. This data set is available as a WMS OGC Web Service via this URL. The imagery has been flown in 2017/2018 on behalf of ProRail, the organization responsible for the maintenance of the public railway infrastructure in the Netherlands. The pictures have been taken from a helicopter (!!) from a height of only 190 meters on average. To appreciate this imagery in your ArcGIS Online Map Viewer: Go to Add > Add Layer from Web Select A WMS OGC Web Service Check Use as Basemap - this will allow you to zoom in to a level where you can spot individual railroad ties Click GET LAYERS to check the layer 'meest_recent' (i.e. most recent) - this will show the imagery from 2017/2018. (The images taken in 2014, 2015 and 2016 do have a considerable lower resolution.) Cool, isn't it? You might also want to check the ProRail imagery in this web map: TWIAV - Tips & trucs - Luchtfoto's ProRail (in EPSG:28992)
... View more
03-18-2019
04:31 AM
|
1
|
0
|
533
|
BLOG
|
In this post we will demonstrate how you - as a Spatial Data Scientist - can use R to access an ArcGIS REST Service, by answering the following question: "Are there any abandoned railroads in Florida?". The ArcGIS Living Atlas of the World offers a nice data set with railroads in the United States: USA Railroads. Following the Service URL we can get to the query page of this Feature Layer. The R script below shows how you can quickly create an interactive map to answer our question. Please scroll down for a little explanation... library(httr)
library(sf)
library(tmap)
url <- parse_url("https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services")
url$path <- paste(url$path, "USA_Railroads_1/FeatureServer/0/query", sep = "/")
url$query <- list(where = "STATE = 'FL'",
outFields = "*",
returnGeometry = "true",
f = "geojson")
request <- build_url(url)
Florida_Railroads <- st_read(request)
tmap_mode(mode = "view")
tm_shape(Florida_Railroads)+tm_lines(col="NET_DESC", palette = "Set1", lwd = 5) In RStudio we start by loading the 3 following R packages: httr - to help us build the query URL sf - to read the spatial data tmap - to create the interactive map, based on the Leaflet JavaScript library We parse the URL to create a list where we can add all the parameters of our query - where, outFields, returnGeometry and f - with their appropriate values. As soon as the list is populated we use the function build_url() to create a properly encoded request. This request is the passed to the function st_read() to populate Florida_Railroads. This Florida_Railroads becomes both a simple features object and a data frame, i.e. it does contain both the geometry and the attribute values of the railroads. In the screen dump below we can see that the total number of records returned is 3305, which is well beyond the maxRecordCount (1000) for this Feature Layer. The GDAL library, which is used by the st_read() function in the sf package, provides automatic paging. Nice, isn't it? With the last two lines in the script we create the interactive map in the Viewer in RStudio. So, the answer to our question is: "Yes, there are quite a few abandoned railroads in Florida". These abandoned lines are marked in red in the map in the screen dump below.
... View more
03-17-2019
09:34 AM
|
3
|
3
|
5994
|
POST
|
OK - Tnx. Will investigate and get back to you to mark this as the right answer as soon as I have confirmed this for myself 🙂
... View more
03-14-2019
04:38 PM
|
0
|
0
|
2054
|
POST
|
Hi, I just wondered: is there a limitation on the number of views you can create on a Hosted Feature Layer? In ArcGIS Online and/or in Portal for ArcGIS? On this page (Create hosted feature layer views—Portal for ArcGIS | ArcGIS Enterprise) it is stated that You can create a maximum of 20 views from the same hosted feature layer. whereas on this page (Create hosted feature layer views—ArcGIS Online Help | ArcGIS) this limitation seems absent (although there is a limit of 20 to the number of categories you can assign to the item....) And this limitation of 20 views on a Hosted Feature Layer in Portal for ArcGIS Enterprise, is that just a limitation of the UI which can be circumvented using a Python script, as kimberly peter states on GitHub (Add information and a sample for creating hosted feature layer views using Python · Issue #340 · Esri/arcgis-python-api … )? With the Python script below I can create 25 views on a Hosted Feature Layer in ArcGIS Online. Would a similar approach be possible in Portal for ArcGIS Enterprise? TIA, Egge-Jan from arcgis.gis import GIS
from arcgis.features import FeatureLayerCollection
from provide_credentials import provide_credentials
username, password = provide_credentials()
my_agol = GIS("https://www.arcgis.com", username, password)
service = my_agol.content.get("<serviceItemId_of_your_Hosted_Feature_Layer>")
flc = FeatureLayerCollection.fromitem(service)
clients = ['ONE','TWO','THREE','FOUR','FIVE','SIX',
'SEVEN','EIGHT','NINE','TEN','ELEVEN','TWELVE',
'THIRTEEN','FOURTEEN','FIFTEEN','SIXTEEN','SEVENTEEN','EIGHTEEN',
'NINETEEN','TWENTY','TWENTY-ONE','TWENTY-TWO','TWENTY-THREE','TWENTY-FOUR',
'TWENTY-FIVE']
for client in clients:
view_name = service.name + "_View_" + client
print(view_name)
view = flc.manager.create_view(name=view_name, spatial_reference=None, extent=None, allow_schema_changes=True, updateable=True, capabilities='Query, Update', view_layers=None)
for lyr in view.layers:
query = "Description = '%s'"%(client)
print(query)
lyr.manager.update_definition({'viewDefinitionQuery': query})
... View more
03-14-2019
02:36 PM
|
3
|
4
|
2509
|
POST
|
Extending the example above with some other parameters in the update_dict: The geometry is now a proper polygon now (instead of a rectangular envelope) To make sure we only see objects that lie entirely within the Area of Interest I use the operater esriSpatialRelContains update_dict = {"viewLayerDefinition":{"filter":
{"operator":"esriSpatialRelContains","value":
{"geometryType":"esriGeometryPolygon","geometry":
{"rings": [ [ [ 599538.094340413692407, 6828149.832508580759168 ],
[ 599619.427406286238693, 6828068.295202748849988 ],
[ 599782.09063097089529, 6828068.294832927174866 ],
[ 599863.424065723433159, 6828149.831768924370408 ],
[ 599863.42953326983843, 6828394.4517279881984 ],
[ 599538.091433217399754, 6828312.911664362065494 ],
[ 599538.094340413692407, 6828149.832508580759168 ] ] ],
"spatialReference":{"wkid":102100,"latestWkid":3857}}}}}}
... View more
03-14-2019
09:54 AM
|
0
|
0
|
1813
|
POST
|
Hi Earl, Thanks for your prompt reply. Very useful 🙂 Now I know I was on the right track, but it was your sample code that really helped me out. I did extent your example a little bit to apply the Area of Interest to each and every layer in the Hosted Feature Layer View. And yes, it works - see the red rectangle in the screendump below. Now I can go and test with more complex geometries for the Area of Interest, and with another spatial operator. (There is already one test result I can share: the Spatial Reference system of my dataset is 28992 (not 102100/3857). And yes it is posssible to define the Area of Interest in another Spatial Reference system. But if you do so, this area will not be visualized on the Visualization tab. So, if this visualization is important to you, you should stick to 102100/3857.) Thanks again, Egge-Jan ## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
## ArcGIS Online Tools
## Script: agol_create_view_area_of_interest.py
## Goal: to create a Hosted Feature Layer View (i.e. a View on a Hosted Feature Layer) with an Area of Interest
## Author: Egge-Jan Polle - Tensing GIS Consultancy
## Date: March 14, 2019
## ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# This script should be run within a specific ArcGIS/Python environment using the batch file below
# (This batch file comes with the installation of ArcGIS Pro)
# "C:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" agol_create_view_area_of_interest.py
from arcgis.gis import GIS
from arcgis.features import FeatureLayerCollection
from provide_credentials import provide_credentials
print('===================')
print('The script that is running: ' + __file__)
print('First you have to log in to ArcGIS Online')
# Log in
username, password = provide_credentials()
my_agol = GIS("https://www.arcgis.com", username, password)
print ("Start: "+datetime.datetime.today().strftime('%c'))
service = my_agol.content.get("<serviceItemId_of_your_Hosted_Feature_Layer>")
flc = FeatureLayerCollection.fromitem(service)
xmin = 599538.1
ymin = 6828068
xmax = 599863.4
ymax = 6828394
update_dict = {"viewLayerDefinition":{"filter":
{"operator":"esriSpatialRelIntersects","value":
{"geometryType":"esriGeometryEnvelope","geometry":
{"xmin":xmin,"ymin":ymin,
"xmax":xmax,"ymax":ymax,
"spatialReference":{"wkid":102100,"latestWkid":3857}}}}}}
view_name = service.name + "_view"
print(view_name)
view = flc.manager.create_view(name=view_name, spatial_reference=None, extent=None,
allow_schema_changes=True, updateable=True, capabilities='Query, Update', view_layers=None)
for lyr in view.layers:
lyr.manager.update_definition(update_dict)
print ("End: "+datetime.datetime.today().strftime('%c'))
... View more
03-14-2019
07:55 AM
|
2
|
1
|
1813
|
Title | Kudos | Posted |
---|---|---|
1 | 2 weeks ago | |
2 | 03-25-2024 02:06 PM | |
1 | 01-17-2024 10:19 AM | |
1 | 03-30-2023 05:57 AM | |
2 | 02-02-2023 02:47 PM |
Online Status |
Offline
|
Date Last Visited |
4 hours ago
|