Latest Contributions by Jan-Tschada

Introduction Spatial data science offers insights into various domains, but also uses computing resources and emits greenhouse gases. We honestly believe, that if we focus on carbon effective spatial data science, we become part of the climate solution. We are focusing on reducing the negative impacts of spatial data science calculations on our climate by reducing the carbon emissions. We introduce some common practices doing spatial data science in a carbon effective way. If you are interested in the common green software engineering approach. You should have a look at the Green Software Foundation | GSF. Setup your spatial data science environment Our spatial data science environment is based upon Python and pip/conda. So that we are using a lightweight Python module estimating the amount of carbon dioxide produced by the spatial data science compute workflow. If you are using conda you can install CodeCarbon using the condaforge channel. Installing CodeCarbon into a conda environment       # author: Jan Tschada # SPDX-License-Identifer: Apache-2.0 conda install -c codecarbon -c conda-forge codecarbon=2.2       CodeCarbon Motivation  CodeCarbon Methodology  CodeCarbon Installation  Methods The spatial data science workflows represents the most common use cases we experienced during our daily work. ArcGIS Notebooks provide a cloud-native Software-as-a-Service solution optimized for spatial data science. Every notebook starts a dedicated instance running in the cloud. So that we can easily extend this Python environment using the CodeCarbon module. Live Traffic The city of Bonn provides real-time traffic information on the three Rhine bridges and the most important inner-city main roads of Bonn. Every 5 minutes the traffic information is updated. We want to collect the real-time traffic information using a dedicated feature service running in our ArcGIS Online instance. The estimated carbon emissions need to be preprocessed and serialized into a feature service. The emission tracker uses a location API detecting in which cloud region the Python process is running. ArcGIS utility functions for collecting and estimating       # author: Jan Tschada # SPDX-License-Identifer: Apache-2.0 from arcgis.gis import GIS from arcgis.features import FeatureSet, GeoAccessor from codecarbon import EmissionsTracker import logging import pandas as pd import requests import sys def query_traffic(): url = 'http://stadtplan.bonn.de/geojson?Thema=19584' response = requests.get(url) response.raise_for_status() return response.json() def prepare_emissions(tracker): emissions_data = tracker.final_emissions_data emissions_df = pd.DataFrame.from_records([dict(emissions_data.values)]) emissions_df['timestamp'] = pd.to_datetime(emissions_df['timestamp']) emissions_sdf = GeoAccessor.from_xy(emissions_df, 'longitude', 'latitude') emissions_sdf.rename(columns={ 'project_name': 'project', 'emissions_rate': 'rate', 'energy_consumed': 'consumed', 'country_name': 'country', 'country_iso_code': 'iso_code', 'cloud_provider': 'provider', 'cloud_region': 'cloud', 'codecarbon_version': 'codecarbon', 'python_version': 'python', 'ram_total_size': 'ram_total', 'tracking_mode': 'tracking' }, inplace=True) return emissions_sdf def prepare_traffic(traffic_featureset): traffic_sdf = traffic_featureset.sdf[['strecke_id', 'auswertezeit', 'geschwindigkeit', 'verkehrsstatus', 'SHAPE']] traffic_sdf['auswertezeit'] = pd.to_datetime(traffic_sdf['auswertezeit']) traffic_sdf['geschwindigkeit'] = pd.to_numeric(traffic_sdf['geschwindigkeit']) traffic_sdf.rename(columns={ 'strecke_id': 'seg_id', 'auswertezeit': 'time', 'geschwindigkeit': 'speed', 'verkehrsstatus': 'traffic' }, inplace=True) return traffic_sdf def publish_emissions(tracker): emissions_sdf = prepare_emissions(tracker) return emissions_sdf.spatial.to_featurelayer(title='Carbon Emissions', folder='Stadt Bonn', tags=['Open Data', 'Carbon', 'Digital Twin']) def publish_traffic(traffic_featureset): traffic_sdf = prepare_traffic(traffic_featureset) return traffic_sdf.spatial.to_featurelayer(title='Stadt Bonn - Aktuelle Straßenverkehrslage', folder='Stadt Bonn', tags=['Open Data', 'Traffic', 'Digital Twin']) def add_emissions(emissions_featurelayer, tracker): emissions_sdf = prepare_emissions(tracker) new_features = emissions_sdf.spatial.to_featureset().features edit_result = emissions_featurelayer.edit_features(adds=new_features) return edit_result def add_traffic(traffic_featurelayer, traffic_featureset): traffic_sdf = prepare_traffic(traffic_featureset) new_features = traffic_sdf.spatial.to_featureset().features edit_result = traffic_featurelayer.edit_features(adds=new_features) return edit_result def find_emissions(gis): portal_items = gis.content.search(query='title:Carbon Emissions AND tags:"Open Data"', item_type='Feature Layer') if 0 < len(portal_items): first_portal_item = portal_items[0] if 0 < len(first_portal_item.layers): return first_portal_item.layers[0] return None def find_traffic(gis): portal_items = gis.content.search(query='title:Stadt Bonn - Aktuelle Straßenverkehrslage AND tags:"Open Data"', item_type='Feature Layer') if 0 < len(portal_items): first_portal_item = portal_items[0] if 0 < len(first_portal_item.layers): return first_portal_item.layers[0] return None       The emission tracker estimates the carbon emissions. The following snippet represents the live traffic implementation. It validates whether or not a dedicated feature layer was already published. If not a new feature service hosting the carbon emissions and the traffic information is created. Snippet for collecting and estimating       # author: Jan Tschada # SPDX-License-Identifer: Apache-2.0 gis = GIS("home") root = logging.getLogger() root.setLevel(logging.INFO) handler = logging.StreamHandler(sys.stdout) handler.setLevel(logging.INFO) root.addHandler(handler) tracker = EmissionsTracker(project_name='Open Data Bonn', output_dir='/arcgis/home/') tracker.start() traffic_geojson = query_traffic() traffic_featureset = FeatureSet.from_geojson(traffic_geojson) traffic_featurelayer = find_traffic(gis) if None is traffic_featurelayer: publish_result = publish_traffic(traffic_featureset) logging.info(publish_result) else: delete_result = traffic_featurelayer.delete_features(where='1=1') logging.info(delete_result) add_result = add_traffic(traffic_featurelayer, traffic_featureset) logging.info(add_result) emissions = tracker.stop() emissions_featurelayer = find_emissions(gis) if None is emissions_featurelayer: publish_result = publish_emissions(tracker) logging.info(publish_result) else: add_result = add_emissions(emissions_featurelayer, tracker) logging.info(add_result) emissions       Results The Green Spatial Engineering Dashboard shows the estimated carbon equivalents in kilograms for our spatial data science workflows. The first use case represents the carbon footprint for querying and collecting the real-time traffic information from the city of Bonn every 15 minutes. Links: Green Spatial Engineering Repository            ... View more

Der einfachste Weg direkten Zugriff auf die Location Services von ArcGIS Platform zu erhalten, ist die Verwendung eines API Keys. Beim Erstellen eines Qt Creator Projektes mit der ArcGIS Maps SDKs for Qt  - Vorlage wird ein Platzhalter für den entsprechenden API Key im Quellcode erzeugt. Man muss lediglich sich in das eigene Developer Dashboard bewegen, einen entsprechenden API Key anlegen und kann dann den Zugriff auf die Location Services verwalten. Der API Key wird einfach im Quellcode ersetzt, oder über einen Konfigurationsmechanismus eingelesen - z.B. über eine entsprechende Umgebungsvariable.   #include "ArcGISRuntimeEnvironment.h" #include <QProcessEnvironment> using namespace Esri::ArcGISRuntime; /// /// \brief readApiKeyFromEnvironment /// Reads the API key directly accessing the location services of ArcGIS Platform /// using the current process environment. /// The environment must contain a variable named 'arcgisruntime_api_key'. /// static void readApiKeyFromEnvironment() { QString apiKeyName = "arcgisruntime_api_key"; QProcessEnvironment systemEnvironment = QProcessEnvironment::systemEnvironment(); if (systemEnvironment.contains(apiKeyName)) { QString apiKey = systemEnvironment.value(apiKeyName); ArcGISRuntimeEnvironment::setApiKey(apiKey); } else { qWarning() << "Use of Esri location services, including basemaps, requires " "you to authenticate with an ArcGIS identity or set the API Key property."; } }   ArcGIS Developers API Keys  ... View more

Hin- und wieder möchte man wissen, welche Tools und Modelle sich in einer Toolbox befinden. Eine Python Toolbox verhält sich dabei wie ein Python Modul. D.h. man kann z.B. mit inspect die einzelnen Klassen, Funktionen und sämtliche Attribute auflisten. Das geht aber auch einfacher, indem man per __all__ die enthaltenen Tools und Modelle abfragt. import arcpy def list_tools(toolbox_path: str)->list[str]: """ Returns the tool/model names of the specified toolbox. :toolbox_path: path-like-object representing the full-qualified *.tbx file """ toolbox = arcpy.ImportToolbox(toolbox_path) return toolbox.__all__ list_tools(r"..\ArcGIS\Projects\AIS\AIS.tbx")   Was befindet sich in euerer Lieblings-Toolbox? ... View more

Wenn man direkt die ArcGIS REST API beim Zugriff auf Feature Services verwendet, bemerkt man dass die Datumswerte von Features als Ganzzahl abgebildet werden. Diese repräsentiert das Unix Timstamp in Millisekunden. Die folgende Funktion erwartet ein FeatureSet als JSON-Objekt und konvertiert die gewünschten Felder in korrekte Datumswerte bzw. in eine ISO-konforme Zeichenkettendarstellung. Ein FeatureSet kann einfach per json.loads(<Esri FeatureSet String>) erzeugt werden. Für den Test haben wir einfach die Ladesäulen in Deutschland - Übersicht (arcgis.com) abgefragt, in der Abfrageansicht einfach (Where: '1=1', Out Fields: Inbetriebnahmedatum, Format: JSON). from datetime import datetime def convert_timestamps(feature_set: FeatureSet, field_names, use_iso=True): """Converts timestamps (UNIX epoch in [ms]) into datetime objects by modifying the feature set directly.""" for feature in feature_set.features: for attribute_name in feature.attributes: if attribute_name in field_names: if use_iso: feature.attributes[attribute_name] = datetime.utcfromtimestamp(feature.attributes[attribute_name] * 1e-3).isoformat() else: feature.attributes[attribute_name] = datetime.utcfromtimestamp(feature.attributes[attribute_name] * 1e-3)   Wenn ihr ISO verwendet, was kommt bei euch als Suffix '+00:00' oder 'Z' (steht nicht für 'Zero' sondern für 'Zulu'), oder etwas ganz anderes? ... View more

In einer ArcGIS Pro Python-Umgebung kann direkt auf die aktuell verbundene ArcGIS Online bzw. ArcGIS Enterprise Instanz zugegriffen werden. Dafür verwendet man einfach das Schlüsselwort 'home'. Die ArcGIS API for Python verwendet in einer ArcGIS Pro Python-Umgebung arcpy. Wenn man allerdings eine Python-Umgebung ohne arcpy verwendet, wird man mit einem Stacktrace belohnt. from arcgis.gis import GIS # Uses the current login of the ArcGIS Pro environment # e.g. ArcGIS Pro authorized ArcGIS identiy, needs arcpy gis = GIS('home') gis.users.me   In einem Jupyter notebook erscheint mein Profil und bei euch? ... View more

We have some customers and partners still relying on the File Geodatabase API. Currently, they try to migrate or at least estimate the migration effort to the new .NET 6 long-term support version. Related Github issue  Would love to receive some feedback from the maintainers. ... View more