Skip navigation
All Places > GeoDev Germany > Blog > Authors buglukas

GeoDev Germany

3 Posts authored by: buglukas

Introduction

Date extraction from OpenStreetMap (OSM) can be performed in various ways. The OSM project features a website, where data can be extracted in the OSM-invented .pbf-format. External platforms provide already preprocessed and verified data from the websites. The Overpass API allows users to download small extents of data (approximately on town and suburb level) via a web request. This API acts like a database server and can be queried with a special database request language, Overpass QL.

 

The free OSM2ArcGIS Python script, available on GitHub, is based on data requested via the Overpass QL using an adapted version of Jeffrey Scarmazzi’s osm_runner python module. The requested data is processed and converted to the ArcGIS SpatialDataframe format. After data processing in ArcGIS Online / ArcGIS Enterprise, a new feature service is created inside the user’s portal.

 

Within the feature service, the data is organized in at least three different layers for the three available geometry types point, polyline and polygon (more layers for more configurations, will be explained later). The additional features of OSM2ArcGIS are explained in detail within the following sections of this blog entry.

 

Requirements

Python

Following Python version and packages must be installed. The installation of Anaconda helps to keep existing python installations working by creating indepedent python environments. Users benefit from an Anaconda integrated package installation and administration interface.

 

How to set up the Python Environment in Anaconda:

Download and install the 3.x Version of Anaconda. Then open the Anaconda Navigator and create a new Python Environment. The Python version must be 3.5 or later. Then install these packages in the environment:

Additional Requirement

An ArcGIS Online Account with user type “Creator” (New user types) or an ArcGIS Enterprise account with user type “Level 2” (Named user, for version 10.6. and below) and at least a “Publisher role(An ArcGIS Online Developer Account also meets these requirements)

 

First steps

  1. Setup your python environment as described in the section “Requirements”.
  2. Download the files from the GitHub repository.
  3. Extract the files from the downloaded OSM2ArcGIS zip archive to a location of your choice.
  4. Adapt the content of the file runScript.bat to match the paths of your Anaconda installation, the python-environment and the location of the OSM2ArcGIS folder, where the project files have been extracted to in step 2. E.g.:
    set path=C:\your_anaconda_install_path\Anaconda3\Library\bin\;%PATH%
    call conda activate OSM2ArcGIS
    call "C:\Users\your_username\AppData\Local\conda\conda\envs\your_python_environment"\python.exe "C:\your_project_extraction_path"\MainModule.py
  5. Adapt the content of the ArcGIS Online / Portal for ArcGIS configuration file agolconfig.json to match your portal URL, your username and password.
  6. Optionally adapt the content of the OpenStreetMap configuration file osmconfig.json to the desired region, add configurations or remove the sample configuration.
  7. Execute the file runScript.bat from your file-browser
  8. Go to your ArcGIS Online portal and look at the results
  9. Click on your most recently created feature service and use the pane “Visualization” to show the results. If you did not modify osmconfig.json, the result should be an output like this:
     

Figure 1: Output data in the portal with the OSM2ArcGIS sample configuration

 

 

The data

The geodata contributed to the OSM project is not as standardized as data provided by governmental organizations or companies. There are guidelines inside the OSM wiki, but users operate various clients on various devices and are not forced to generate geometries in a specific way or provide detailed attribute values in exactly defined attribute fields.


This may lead to problems when parsing the data and trusting on specific data types. Also using OpenStreetMap for a roadmap with exact speeds on road sections is not always a good idea when there is no exact unit specified in all road sections. That’s the reason we decided not to use integer or float data types in attribute data, all attribute data except the timestamp for every OSM object are stored as strings.


Another challenge using OSM is related to the geometric data structure with way, node and relation objects. Nodes can be easily transferred to ArcGIS point objects, way objects not having the same start and end can also be easily transferred to ArcGIS polyline objects. More complicated output geometries are polygons, as they can be formed by way objects with the same start and end or relations, which connect multiple way objects to form a “multipolygon”.

 

Relations can also connect multiple point objects or point and way objects. To take only relation objects into account, that form “multipolygons” by connecting way elements, the polygon-connecting relations feature the attribute “multipolygon” with the value “yes” (see figure below). Relations not forming “multipolygon” features are not regarded in OSM2ArcGIS.

 

 

Figure 2: Multipolygon relation element in OpenStreetMap


The configuration

The configuration of OSM2ArcGIS can be defined by the user in two JSON-based configuration files, agolconfig.json and osmconfig.json. The first one enables the tool to establish connections to the ArcGIS Enterprise / Online portal.  The latter is used to setup configurations for requests to the OpenStreetMap server. Explanation of all JSON attributes and examples for configurations are listed in the table below.

 

ParameterUsageExample
"categories"Controls the export of elements from OpenStreetMap, for every new configuration with another geometry or OSM key a new category must be created within the following 5 properties: 

- The desired OSM key for "categoryName" property. Multiple values not allowed here. 

- The desired OSM tags for "categoryValue" property. Multiple values in square brackets. 

- The excluded fields from service on ArcGIS Online for the "attributeFieldsToExclude" property. Multiple values in square brackets. 

- The geometry type, valid types are "line", "point" or "polygon". Multiple values not allowed here. 

- Set the "isEnabled" property to "yes" to activate or to "no" to deactivate a configuration. Currently unneeded configurations retainable in configuration file.
"categories" : 

     { 
         "categoryName" : "public_transport", 
         "categoryValues" :["station", "platform"], 
         "attributeFieldsToExclude" : ["bus", "tram"], 
         "geometryType" : "polygon", 
         "isEnabled" : "yes" 
     }, 
     { 
         "categoryName" : "public_transport", 
         "categoryValues" : ["station", "platform"], 
         "attributeFieldsToExclude" : ["bus", "tram"], 
         "geometryType" : "point", 
         "isEnabled" : "yes" 
     }, 
     { 
         "categoryName" : "public_transport", 
         "categoryValues" : ["platform", "network"], 
         "attributeFieldsToExclude" : ["bus", "tram"], 
         "geometryType" : "line", 
         "isEnabled" : "yes" 
     } 
]
"boundingBox"Bounding box for the data to be loaded. Multiple bounding boxes not allowed here."boundingBox" : 

     "minLatInit" : "48.0503", 
     "minLonInit" : "11.2723", 
     "maxLatInit" : "48.2597", 
     "maxLonInit" : "11.8113" 
}


Valid OSM keys are listed here: https://taginfo.openstreetmap.org/api/4/projects/keys

Valid OSM tags are listed here: https://taginfo.openstreetmap.org/api/4/projects/tagshttps://taginfo.openstreetmap.org/api/4/projects/tags


The modules

OSM2ArcGIS was originally developed by Simon Geigenberger. The tool consists of seven modules, which are loaded, when they are needed at the current position in the program. The MainModule is loaded as the core part of OSM2ArcGIS and invokes the modules AGOLConfigHelper, OSMConfigHelper, OSMHelper and AGOLHelper.

 

AGOLConfigHelper

AGOLConfigHelper validates the configfile agolconfig.json, the module checks the availability of ArcGIS API for Python and then continues to check the configfile. A quick connection check with the given credentials in the configfile is also performed within this module. The module terminates the whole program if invalid values were specified within the configuration file.

 

OSMConfigHelper

OSMConfigHelper validates the configfile osmconfig.json, the module checks the availability of ArcGIS API for Python and then continues to check the configfile. Categories and Tags available in OSM are downloaded from an OSM database and the values set in the configfile are compared with the values in the database. The module terminates the whole program if the set values or tags in the configfile are not available in the OSM database of valid tags/values, an invalid boundingbox or other invalid values are set within the configfile.

OSMHelper

OSMHelper module is designed to setup and control the requests sent to the OpenStreetMap server using the extended osm_runner module. The module is configured to optimize processing speed by creating threads, to simultaneously download data from the OSM server for the configurations in the configfile.

Osm_runner

Osm_runner module was originally developed by Jeffrey Scarmazzi and has been extended to process the relation entities with the attribute “multipolygon” and the value “yes”.

Osm_runner_utils

Osm_runner_utils provides configuration settings used with the osm_runner module. Details on the OSM server request format can be set, as well as the output format for the server’s answer and additional filters can be changed by modifying this module.

AGOLHelper

AGOLHelper expects an Esri SpatialDataFrame as input object and performs the creation of a feature service inside the user’s portal in ArcGIS Enterprise / Online and the upload of data for the defined configurations to at least one layer for every geometry type set in osmconfig.json. Uploads to the portal are performed using multiple chunks of data to prevent server errors on large datasets.

 

Conclusion

OSM2ArcGIS makes it easy to transfer data from OSM to an ArcGIS Enterprise / Online portal. All you need to do is to setup your python installation, configure the tool via the JSON-based configfiles and to make sure, that you were granted at least a publisher role by your portal administrator.

If you have any questions or comments please feel free to contact me or leave a comment in the comment section below. 

 

The script is available for download from GitHub


Happy data processing, Lukas 

(E-Mail: lukas.bug@aol.de)

Introduction

Publishing one feature layer in ArcGIS Online or ArcGIS Enterprise in one feature service is a simple procedure, there is a native function for uploading data inside a Spatial Dataframe to one hosted feature layer inside a feature service available in ArcGIS API for Python.

However, publishing multiple layers with different geometries inside one feature service requires more effort. To make this process less complex and more convenient for all users we created a 10 step workflow. We attached a Jupyter Notebook to the end of this blog, which can be simply uploaded to Esri Jupyter Hub to test the workflow.

 

To simplify the written language of this article, we describe the workflow mentioning ArcGIS Online only. All steps performed within this article are also applicable to ArcGIS Enterprise.

 

The Data

Data is available from multiple sources on the web in various geospatial formats. The ArcGIS Spatial Dataframe is a pythonic data structure, that allows to import and include several other geospatial formats on the fly. The table like structure is merely based on a Pandas DataFrame, but incorporates several extensions for geodata elements.

We use the ArcGIS Spatial Dataframe object to store tabular and geometric data. Pandas DataFrames are also used, to store tabular data.

 

Step 1: Import of the needed modules and login to ArcGIS Online with your subscription

We will have to load a few modules to setup the publishing process, the getpass module allows us to interactively enter our password. Using this module we have a secure way to do the login process, as we do not need to type our password as plaintext. The second argument to the GIS function should be changed from "username" to your username for ArcGIS Online. We suggest the installation of the needed python modules including the python environment and the Jupyter Notebooks by using Anaconda Distribution. It features python environments, which are separated from the system wide python installations and therefore reduces complexity and enhances compatibility.

from arcgis import GIS
from arcgis import geometry
from arcgis import features as fs
from getpass import getpass as pwd
import pandas as pd
pwd = pwd()
gis = GIS("https://www.arcgis.com/home/index.html", "username", pwd)

Step 2: Create the features and setup two geometry types

To demonstrate this workflow, we will now create some hard coded sample features with basic point and polyline geometries. To define the two features, we will also create some field values to visualize how field creation and setting attributes to features works in ArcGIS Online.

2.1 Creation of sample field values for features

We create two dictionaries for each geometry and two lists to contain the dictionaries. The list of dictionaries of each geometric element is then added to a Pandas DataFrame. If we had to create attributes for multiple features of one geometry, we would have to add as many dictionaries to the list of dictionaries as geometric features are available.

dictPointAttr = {}
dictPointAttrList = []
dictPointAttr["someint1"] = 123456789101112
dictPointAttr["sometext1"] = "Text"
dictPointAttr["somefloat1"] = 1.25
dictPointAttrList.append(dictPointAttr)
dfpnt = pd.DataFrame.from_dict(dictPointAttrList)

dictPolyLineAttr = {}
dictPolyLineAttrList = []
dictPolyLineAttr["someint2"] = 121110987654321
dictPolyLineAttr["sometext2"] = "Txet"
dictPolyLineAttr["somefloat2"] = 5.21
dictPolyLineAttrList.append(dictPolyLineAttr)
dfpll = pd.DataFrame.from_dict(dictPolyLineAttrList)

2.2 Creation of a simple point geometry

To setup a simple point geometry we will need to create a dictionary containing the two point coordinates “x” and “y” as keys, as well as a sub dictionary with “spatialReference” as key, containing the spatial reference defined by EPSG codes. The dictionary object is then converted to an ArcGIS point geometry and appended to a list of geometries. To store the feature data, an ArcGIS Spatial Dataframe is used. It is capable of storing lists of geometries as well as Pandas DataFrames containing the attribute data for every feature.

listGeometryPoint = []
pnt = {"x": 11.0419520737521,"y": 48.4391156813199, "spatialReference" : {"wkid" : 4326,"latestWkid" : 4326}}
point = geometry.Point(pnt)
listGeometryPoint.append(point)
spatialDataFramePoint = fs.SpatialDataFrame(data=dfpnt, geometry=listGeometryPoint)

2.3 Creation of a simple polyline geometry

The workflow of creating a simple polyline geometry is more or less the same as the creation of a simple point geometry, it differs in requirement of another syntax to define the vertices of the line.

listGeometryPolyLine = []
plyln = { "paths": [[[11.0419520737521,48.4391156813199],[11.0416097302404,48.4390815262157], [11.0414886646127,48.4392674452629]]], "spatialReference" : {"wkid" : 4326, "latestWkid" : 4326}}
polyline = geometry.Polyline(plyln)
listGeometryPolyLine.append(polyline)
spatialDataFramePolyLine = fs.SpatialDataFrame(data=dfpll, geometry=listGeometryPolyLine)

Step 3: Creation of the feature service and a feature layer collection

In this step we will setup a feature service and create a feature layer collection from a new (empty) service. The feature layer collection is used to store the definition of the feature service as well as the layer objects, which will be created in the following steps. To create the service we call the function “gis.content.create_service” from ArcGIS API for Python.

Feature services must have a unique name. If you like to repeat this workflow, please ensure to rename your feature service.

 

Provided function parameters are:

 

  • A name for the resulting feature service via name=
  • The spatial reference via wkid=
  • The spatial extent via item_properties=

 

The extent setting is necessary to ensure, that layers inside the feature service are displayed correctly in maps on ArcGIS Online. Without the extent setting, the wrong initial area might be displayed and users need to zoom to the appropriate part of the map by using “zoom to layer”. In this example we use the extent of the whole world. The items inside the dictUpdate dictionary can be set to user defined values. We recommend setting the "objectIdField" key to a custom value. Therefore, we ensure:

 

  • Every new feature object is provided with a new feature ID
  • No conflicts with the default setting of "objectIdField" occur (default setting is "objectId")

 

The key "capabilities" is needed to define what users with access to the feature service item can do with the service. Information on capabilities is provided in the ArcGIS REST API documentation.

newService = gis.content.create_service(name="featureservice", wkid=4326, item_properties={"extent":[-180.0,-90.0,180.0,90.0]})

newFtLC = fs.FeatureLayerCollection.fromitem(newService)

dictUpdate = {"copyrightText": "copyrightText",
              "objectIdField" : "FID",
              "globalIdField" : "GlobalID",
              "maxRecordCount": 25000,
              "serviceDescription": "description",
              "capabilities": 'Editing,Query,Update,Uploads,Delete,Sync,Extract',
              "spatialReference": {"wkid": 4326,"latestWkid": 4326},
              "initialExtent":{
                   "xmax": 180.0,"ymin": -90.0,"xmin": -180.0,"ymax": 90.0,
                   "spatialReference": {"wkid" : 4326, "latestWkid" : 4326}
                  },
              "fullExtent":{
                   "xmax": 180.0,"ymin": -90.0,"xmin": -180.0,"ymax": 90.0,
                  "spatialReference": {"wkid" : 4326, "latestWkid" : 4326}
                  }
    }
newFtLC.manager.update_definition(dictUpdate)

Step 4: Setup of required and optional fields inside the feature layers

The fields required to describe a valid feature layer are the "objectIdField" and the "globalIdField", which we already defined by their name in the previous step. All fields can be provided with an alias if the field name is too short to provide a meaningful description of its content. The length attribute should be set to appropriate values to add the needed count of features to the layers. The field attributes ["nullable"] and ["editable"] should be set to False for both fields "objectIdField" and "globalIdField".

# General Fields
dictFieldOBJECTID = {}
dictFieldOBJECTID["alias"] = "FID"
dictFieldOBJECTID["name"] = "FID"
dictFieldOBJECTID["type"] = "esriFieldTypeOID"
dictFieldOBJECTID["sqlType"] = "sqlTypeOther"
dictFieldOBJECTID["length"] = 10
dictFieldOBJECTID["nullable"] = False
dictFieldOBJECTID["editable"] = False
dictFieldOBJECTID["domain"] = None
dictFieldOBJECTID["defaultValue"] = None

dictFieldGlobalID = {}
dictFieldGlobalID["alias"] = "GlobalID"
dictFieldGlobalID["name"] = "GlobalID"
dictFieldGlobalID["type"] = "esriFieldTypeGlobalID"
dictFieldGlobalID["sqlType"] = "sqlTypeOther"
dictFieldGlobalID["length"] = 38
dictFieldGlobalID["nullable"] = False
dictFieldGlobalID["editable"] = False
dictFieldGlobalID["domain"] = None
dictFieldGlobalID["defaultValue"] = None

The optional fields are defined to ensure all values we defined in step 2.1 are transferred to the appropriate fields in the layer.

For integer fields we need to pay attention to the required field lengths. If we have 64 bit numbers, we need to define a data type for the key ["sqlType"] named "sqlTypeBigInt" and set the key ["type"] to "esriFieldTypeInteger". By ignoring this fact we may cause data corruption in our layers in ArcGIS Online, the values which do not fit into a standard integer field are sometimes written on into the next field, overwriting values in the next field.

 

FYI: If you use the Spatial Dataframe workflow as mentioned in the introduction, you are not able to use 64 bit numbers in integer fields, they are automatically converted into string fields / strings or stored as 32 bit integers causing data corruption in the hosted feature layer.

Fields containing float values can be defined by setting the key ["type"] to "esriFieldTypeDouble" and the key ["sqlType"] to "sqlTypeOther".

Fields containing strings can be defined by setting the key ["sqlType"] to "sqlTypeOther", the key ["type"] to "esriFieldTypeString" and defining a field length by setting the key ["length"] to a numeric integer value like the number 10.

# Fields for point layer
dictFieldInt1 = {}
dictFieldInt1["alias"] = "someint1"
dictFieldInt1["name"] = "someint1"
dictFieldInt1["sqlType"] = "sqlTypeBigInt"
dictFieldInt1["type"] = "esriFieldTypeInteger"
dictFieldInt1["nullable"] = "yes"
dictFieldInt1["editable"] = "yes"
dictFieldInt1["visible"] = "yes"

dictFieldString1 = {}
dictFieldString1["alias"] = "sometext1"
dictFieldString1["name"] = "sometext1"
dictFieldString1["sqlType"] = "sqlTypeOther"
dictFieldString1["type"] = "esriFieldTypeString"
dictFieldString1["length"] = 10

dictFieldFloat1 = {}
dictFieldFloat1["alias"] = "somefloat1"
dictFieldFloat1["name"] = "somefloat1"
dictFieldFloat1["sqlType"] = "sqlTypeOther"
dictFieldFloat1["type"] = "esriFieldTypeDouble"

 

# Fields for polyline layer
dictFieldInt2 = {}
dictFieldInt2["alias"] = "someint2"
dictFieldInt2["name"] = "someint2"
dictFieldInt2["sqlType"] = "sqlTypeBigInt"
dictFieldInt2["type"] = "esriFieldTypeInteger"
dictFieldInt2["nullable"] = "yes"
dictFieldInt2["editable"] = "yes"
dictFieldInt2["visible"] = "yes"

dictFieldString2 = {}
dictFieldString2["alias"] = "sometext2"
dictFieldString2["name"] = "sometext2"
dictFieldString2["sqlType"] = "sqlTypeOther"
dictFieldString2["type"] = "esriFieldTypeString"
dictFieldString2["length"] = 10

dictFieldFloat2 = {}
dictFieldFloat2["alias"] = "somefloat2"
dictFieldFloat2["name"] = "somefloat2"
dictFieldFloat2["sqlType"] = "sqlTypeOther"
dictFieldFloat2["type"] = "esriFieldTypeDouble"

 

Step 5: Create list of fields for each layer element and append field definitions

To store our field definitions in the following layer definitions, we create two lists, which store the dictionaries for every field. We need two lists to store the fields because we are not allowed to put the two different geometric objects, point and polyline, into the same layer.

The python list.append operation is applied for every field dictionary.

 

# Lists containing the fields
listFieldsPoint = []
listFieldsPolyLine = []

# Append fields to list of fields for points
listFieldsPoint.append(dictFieldOBJECTID)
listFieldsPoint.append(dictFieldGlobalID)
listFieldsPoint.append(dictFieldInt1)
listFieldsPoint.append(dictFieldString1)
listFieldsPoint.append(dictFieldFloat1)

# Append fields to list of fields for polylines
listFieldsPolyLine.append(dictFieldOBJECTID)
listFieldsPolyLine.append(dictFieldGlobalID)
listFieldsPolyLine.append(dictFieldInt2)
listFieldsPolyLine.append(dictFieldString2)
listFieldsPolyLine.append(dictFieldFloat2)

Step 6: Create layer definitions for two layers

We need to define the layer definitions to provide information on what type of features we would like to add to the feature service and how the layers are named inside the service. As we defined fields via dictionary elements, we need to define the layer definition using dictionaries as well. The key ["geometryType"] defines the geometries, one layer contains. The layer definition also contains information, we already set in step 3, the definition of the ["objectIdField"] and the ["globalIdField"]. For every layer a separate extent, which matches the extent of all features in the layer and a separate name can be defined. Fields are defined by setting the key ["fields"] to the corresponding list of fields for the layer.

dictPolyLineLayer = {}
dictPolyLineLayer["geometryType"] = 'esriGeometryPolyline'
dictPolyLineLayer["globalIdField"] = 'GlobalID'
dictPolyLineLayer["objectIdField"] = 'FID'
dictPolyLineLayer["extent"] = {
         "xmax": 180.0,"ymin": -90.0,"xmin": -180.0,"ymax": 90.0,
        "spatialReference": {"wkid": 4326, "latestWkid": 4326}
        }
dictPolyLineLayer["name"] = 'PolyLineLayer'
dictPolyLineLayer["fields"] = listFieldsPolyLine
dictPointLayer = {}
dictPointLayer["geometryType"] = 'esriGeometryPoint'
dictPointLayer["globalIdField"] = 'GlobalID'
dictPointLayer["objectIdField"] = 'FID'
dictPointLayer["extent"] = {
         "xmax": 180.0,"ymin": -90.0,"xmin": -180.0,"ymax": 90.0,
        "spatialReference": {"wkid": 4326, "latestWkid": 4326}
        }
dictPointLayer["name"] = 'PointLayer'
dictPointLayer["fields"] = listFieldsPoint

Step 7: Create a layer list, append layer definitions and update feature layer collection

This step will update the feature layer collection, by adding the layer definitions. They are added to a list and this list is then added to a dictionary as value for the key "layers". The feature layer collection is then updated by the dictionary.

layersList = []
layersList.append(dictPointLayer)
layersList.append(dictPolyLineLayer)
dictUpdate = {"layers": layersList }
newFtLC.manager.add_to_definition(dictUpdate)

Step 8: Create FeatureSets from Spatial Dataframes and append FeatureSets to list

The Spatial Dataframes are converted to FeatureSet elements and then added to a list. The list is used to store the values for the next step, the creation of features using a feature layer collection.

dflist = []
featureSetPoint = spatialDataFramePoint.to_featureset()
featureSetPolyLine = spatialDataFramePolyLine.to_featureset()
dflist.append(featureSetPoint)
dflist.append(featureSetPolyLine)

Step 9: Create features and add them to feature layer collection

The actual feature creation is performed in this step. We need to loop through our feature layer collection using a counter to add new features.

To improve readability and structure we create the function add_features, which performs the additions of features for each layer. The input values for the function are the current layer object and the current set of features. We define a variable result, which we set to the output of the edit function layer.edit_features of the given layer. As argument for the function layer.edit_features, we set (adds = featlist). The variable featlist matches our current set of features. We then return the result of our editing. The function add_features is called inside the for loop

for i in range(len(newFtLC.layers)):


were we loop through the feature layer collection by counter. The output of the function call is set to the variable status, which is then printed to the console.

The console output shows us, if our layer operation was successful. If this is the case, we will see two times an output containing: 'success': True.

def add_features(layer, featlist):
     result = layer.edit_features(adds = featlist)
     return result

for i in range(len(newFtLC.layers)):
     status = add_features(newFtLC.layers[i],
     dflist[i])
     print (status)

Step 10: Inspect results on ArcGIS Online

We created a sample feature service containing two different layers with two different layer types. To check if everything was created and all of the items work, we open another web browser window or tab and go to our portal. There should be an item in the list of items named “featureservice” if we click on the “content”  tab. We can now click on the item and inspect it, there should be values in the attribute table and one point in the point layer as well as one polyline in the polyline layer when regarding the features inside the layers. When we add the feature service to a map we should see a result like this:

The results

Conclusion

Setting up a feature service with multiple layers can be performed fast and efficiently using ArcGIS API for Python. There is no need to publish services from ArcGIS for Desktop, datasets can be easily published to ArcGIS Enterprise or ArcGIS Online using a few lines of python code. Though it is cool to publish a feature service with multiple layers in ten steps, it would be great if there was a function available, which makes the effort to publish multiple layers in one single feature service smaller and allows setting up a service and the creating layers and features within only a few steps. If you have any questions or comments please feel free to contact me or leave a comment in the comment section below.

 

Happy coding, Lukas

Introduction

Esri Deutschland developed a python tool based on ArcGIS API for Python to manage licenses and Named User in ArcGIS  and ArcGIS Enterprise. The tool is named ArcGIS Python package for user administration (ArcGIS admin toolbox). Information on this tool can be found in a blog on GIS IQ (in german language). While optimizing, several pitfalls and common reasons for major errors in user administration tasks and license assignment could be identified. Problems occur, when programmers do not notice, that some items do not exist at a certain point or for certain users or some items are assigned to users, which then prevents functions from completion. The most common identified pitfalls in administration tasks and license assignment are presented below.

 

1. Deleting User

 

Deleting Named User fails if a user has one or more licenses assigned, to ensure deleting works without any problem, programmers are reminded to always revoke licenses before deleting users. Exceptions should always be handled correctly, exceptions invisible to the user may also prevent scripts from working correctly. Sometimes missing attributes like biography-description-texts can lead to malfunctioning of scripts, it is always suggested to use if-else-queries to prevent accessing attributes, which are missing.

Missing biography-description-text causes delete to fail

 

Deleting Named User from ArcGIS  can be safely performed if the user:

 

  • Has no licenses assigned
  • Missing properties (e.g. biography-description-text) for users are properly handled
  • Unimportant exceptions are properly handled

 

2. Assigning Licenses

 

Assigning licenses to Named User fails if users are not entitled to use the requested license. If licenses are not available in the user’s organization, but are hard-coded and therefore required for every user, a exception may be raised if licenses are not available in some organizations. Invalid licenses as well as a missing count of licenses also prevent programs from running correctly. If-else-queries help to avoid running into problems and notify the user of invalid arguments. Exceptions should always be handled correctly, exceptions invisible to the user may also prevent scripts from working correctly.

If-queries to inform users on invalid arguments or properly handle list and string objects

 

Assigning Licenses to users in ArcGIS  works flawlessly if:

 

  • The user has the rights and is entitled for the requested license
  • The requested license is available for the user’s organization
  • The requested license is a valid license
  • Enough licenses are available
  • Unimportant exceptions are properly handled

 

3. Revoking Licenses

 

Misspelling or non-existence of licenses or Named User is also a common cause for errors. Revoking licenses will fail if a requested user, whose license should be revoked is misspelled or does not exist. License being revoked from users must be previously assigned to the user. 

 

Revoking Licenses from users in ArcGIS  works flawlessly if:

 

  • The requested user exists
  • The requested license to revoke has been assigned to the user
  • The requested license to revoke is a valid license
  • Unimportant exceptions are properly handled

 

Conclusion

 

ArcGIS API for Python provides several functions for user administration, license assignment and revoking of licenses. There are several ways of accessing these functions and use them in scripts. It is up to the programmer to implement tasks and workflows properly. Beginners should note, that the API requires at least basic know-how on programming tasks and procedures and does not check if users that should be deleted, have licenses assigned. The programmer should be aware, that non-revoked licenses may cause deleting tasks to fail. Though there are a few things to notice, it is great to have many opportunities to do user administration tasks in python with the ArcGIS API for Python. If you have any questions or comments please feel free to contact me or leave a comment in the comment section below.

Filter Blog

By date: By tag: