nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Python-Skript zum Einlesen von Waze-Daten in ArcGIS Online

Anonymous User

<\/HEAD>

Im Rahmen des Waze Connected Citizen Program (CCP)<\/A> erhalten Partner Zugang zu Echtzeit-Verkehrsstaus und Vorfällen, die über die Waze-App gemeldet werden (im XML- und JSON-Format). Es gibt bereits Techniken und Dokumentationen<\/A>, wie diese Daten in eine ArcGIS-Plattform mit GeoEvent integriert werden können.<\/P>

<\/P>

In diesem Blog werde ich eine Technik beschreiben, um diese Daten über Python und den Windows Task Scheduler in ArcGIS Online zu integrieren. Das

ursprüngliche Python-Skript<\/A> wurde von Joel Meier<\/A> entwickelt, welches ich angepasst habe, um den vorgeschlagenen Workflow zu erfüllen. Ein besonderer Dank geht auch an meine Kollegen Edmund Hall<\/A> und Louis Tian<\/A>, die mir beim Debuggen geholfen haben. Und nicht zuletzt möchte ich Andrew Stauffer<\/A>03danken, der uns als Partner unterstützt hat und das Beste aus dem Waze CCP<\/A> herausholt.<\/P>

<\/P>

Erstellen von ArcGIS Online Feature-Layern zur Speicherung von Verkehrsinformationen<\/H2>Melden Sie sich bei ArcGIS Online > Meine Inhalte an, klicken Sie auf Erstellen > Feature-Layer, wählen Sie Von URL aus<\/EM> und geben Sie die genaue URL der Punkt-Feature-Layer-Vorlage ein. Wiederholen Sie anschließend denselben Vorgang für den Linien-Feature-Layer. Dadurch werden neue leere Feature-Layer mit dem erforderlichen Schema erstellt (Feldnamen, Typen und räumliche Referenz)<\/EM>.<\/P>Punkt-Feature-Layer-Vorlage<\/STRONG><\/SPAN><\/P>https:\/\/services1.arcgis.com\/E5n4f1VY84i0xSjy\/arcgis\/rest\/services\/WazeTrafficAlertsEmpty\/FeatureServer<\/A><\/SPAN><\/P>
Linien-Feature-Layer-Vorlage<\/STRONG><\/SPAN><\/P>
https:\/\/services1.arcgis.com\/E5n4f1VY84i0xSjy\/arcgis\/rest/services/WazeTrafficJamsEmpty/FeatureServer<\/_a>Alternativ besuchen Sie https://developers.arcgis.com/layers/new, um Punkt- und Linien-Geometrie-Feature-Layer (WKID = 4326) mit den folgenden Feldern zu erstellen.Punkt - Feature LayerOBJECTID (Typ: esriFieldTypeOID, Alias: OBJECTID, SQL-Typ: sqlTypeOther, Länge: 0, nullable: false, bearbeitbar: false)DateReported (Typ: esriFieldTypeDate, Alias: DateReported, SQL-Typ: sqlTypeTimestamp2, Länge: 0, nullable: true, bearbeitbar: true)Title (Typ: esriFieldTypeString, Alias: Title, SQL-Typ: sqlTypeOther, Länge: 25, nullable: true, bearbeitbar: true)ReportType (Typ: esriFieldTypeString, Alias: ReportType, SQL-Typ: sqlTypeOther, Länge: 25, nullable: true, bearbeitbar: true)ReportSubtype (Typ: esriFieldTypeString, Alias: ReportSubtype, SQL-Typ: sqlTypeOther, Länge: 50, nullable: true, bearbeitbar: true)Street (Typ: esriFieldTypeString, Alias: Street, SQL-Typ: sqlTypeOther, Länge: 256, nullable: true, bearbeitbar: true)City (Typ: esriFieldTypeString, Alias: City, SQL-Typ: sqlTypeOther, Länge: 50, nullable: true, bearbeitbar: true)Description (Typ: esriFieldTypeString, Alias: Beschreibung, SQL-Typ: sqlTypeOther, Länge: 1024, nullable: true, bearbeitbar: true)<\/EM><\/SPAN><\/LI>UUID (Typ: esriFieldTypeDouble, Alias: UUID, SQL-Typ: sqlTypeOther, nullable: true, bearbeitbar: true)<\/EM><\/SPAN><\/LI><\/UL>ShapeLength (Typ: esriFieldTypeDouble, Alias: ShapeLength, SQL-Typ: sqlTypeDouble, nullable: true, nicht bearbeitbar)<\/STRONG><\/EM><\/SPAN><\/LI>CreationDate (Typ: esriFieldTypeDate, Alias: CreationDate, SQL-Typ: sqlTypeOther, Länge: 8, nullable: true, nicht bearbeitbar)<\/STRONG><\/EM><\/SPAN><\/LI>Creator (Typ: esriFieldTypeString, Alias: Creator, SQL-Typ: sqlTypeOther, Länge: 128, nullable: true, nicht bearbeitbar)<\/STRONG><\/EM><\/SPAN><\/LI>EditDate (Typ: esriFieldTypeDate, Alias: EditDate, SQL-Typ: sqlTypeOther, Länge: 8, nullable: true, nicht bearbeitbar)<\/STRONG><\/EM><\/SPAN><\/LI>Editor (Typ: esriFieldTypeString, Alias: Editor, SQL-Typ: sqlTypeOther, Länge: 128, nullable: true, nicht bearbeitbar)<\/STRONG><\/EM><\/SPAN><\/LI><\/UL>*<\/STRONG>vorausgefüllte Felder<\/EM><\/SPAN><\/P>Bitte stellen Sie sicher, dass Sie die genauen Feldnamen und Typen<\/EM><\/STRONG> verwenden (alternativ müssten Sie das Skript anpassen).<\/SPAN><\/P>Sobald Feature-Layer konfiguriert sind, bitte Aktivieren Sie die Bearbeitung<\/STRONG> und setzen Sie sie auf PRIVATE<\/STRONG>.<\/SPAN><\/P>Bitte beziehen Sie sich auf "Waze Feed Technical Specifications [Get Waze Data]"<\/EM><\/STRONG> im Partner's Portal<\/A> für genaue Definitionen zu Feldbeschreibungen.<\/P><\/BLOCKQUOTE>
<\/P>

Konfiguration des Python-Skripts<\/H4>Ersetzen Sie die Kommentarabschnitte (Zeilen 6, 9 und 10)<\/SPAN><\/EM> mit Ihrem eigenen XML-Feed und URLs für Ihre Punkt- und Linien-Feature-Layer.<\/P>import<\/SPAN> xml.<\/SPAN>etree.<\/SPAN>ElementTree as<\/SPAN> ET import<\/SPAN> urllib.<\/span>request import<\/span> arcpy from<\/span> datetime import<\/span> datetime namespaces ={'georss': 'http://www.georss.org/georss', 'linqmap': 'http://www.linqmap.com'} tree = ET.ElementTree(file=urllib.request.urlopen('##URL for XML waze connection##')) root = tree.getroot() channel_head = [] fcp = "##URL to Point - Feature Layer##" fcl = "##URL to Line - Feature Layer##" for member in root.findall('channel/item'): channel = [] Title = member.find('title').text.upper() try: point = member.find('georss:point', namespaces) line = member.find('georss:line', namespaces) except: pass print('Fehler Stufe 0') if point != None: try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find token">.<\/SPAN>find(<\/SPAN>'linqmap:subtype'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text ist<\/SPAN> nicht<\/SPAN> None:<\/SPAN>\n Subtype =<\/SPAN> member.<\/SPAN>find(<\/SPAN>'linqmap:subtype'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text.<\/SPAN>replace(<\/SPAN>'_'<\/SPAN>,<\/SPAN> ' '<\/SPAN>)<\/SPAN>.<\/SPAN>capitalize(<\/SPAN>)<\/SPAN>\n sonst<\/SPAN>:<\/SPAN>\n Subtype =<\/SPAN> ''<\/Span>\n\n UUID =<\/SPAN> member.<\/Span>find(<\/Span>'linqmap:uuid'<\/Span>,<\/Span> namespaces)<\/Span>.<\/Span>text\n Reliability =<\/Span> member.<\/Span>find(<\/Span>'linqmap:reliability'<\/Span>,<\/Span> namespaces)<\/Span>.<\/span>text\n\n wenn len(member.findall('linqmap:reportDescription', namespaces)) 3e 30): Description = member.find('linqmap:reportDescription', namespaces).text sonst: Description = '' Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(' ') row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]), float(point[0]) cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype", "ReportType", "Title", "DateReported", "City", "Street", "UUID", "Reliability", "Description", "SHAPE@X", "SHAPE@Y"]) cursor.insertRow(row) del cursor print('stage 1') except: pass print('fail Stage 1') elif line != None: if Title != 'IRREGULARITY': try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(<\/SPAN>member.<\/SPAN>findall(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>)<\/SPAN> ><\/SPAN> 0<\/SPAN>:<\/SPAN> EndNode =<\/SPAN> member.<\/SPAN>find(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text else<\/SPAN>:<\/SPAN> EndNode =<\/SPAN> ''<\/SPAN> if<\/SPAN> len(<\/SPAN>member.<\/SPAN>findall(<\/SPAN>'linqmap:reportDescription'<\/Span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Beschreibung =<\/span> member.<\/span>find(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>.<\/span>text else<\/span>:<\/span> Beschreibung =<\/span> ''<\/span> if<\/span> len(<\/span>member.<\/span>findall(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Straße =<\/span> member.<\/span>find(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>.<\/span>text else<\/span>:<\/span> Straße =<\/span> ''<\/span> UUID < spanclass= "operatortoken" > = < \/ span > member < spanclass= " punctuationtoken " > . < \/ span > find < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > 'linqmap:uuid' < \/ span > , namespaces < spanclass= " punctuationtoken " > ) < \/ span > . text Typ < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . < \/ span > find < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > 'linqmap:type' < \/ span > , namespaces < spanclass= " punctuationtoken " > ) < \/ span > . text . replace ( '_' , ' ' ) . capitalize ( ) Länge < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . find ( 'linqmap:length' , namespaces ) . text Ebene < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . find ( 'linqmap:level' , namespaces ) . text Linie < spanclass= " operatortoken " > = < \/ span > member . find ( 'georss:line' , namespaces ) . text . replace ( '\n' , '' ) . replace ( ' ' , '],[' ) . replace ( '],[-' , ', -' ) . split ( ', ' ) Geschwindigkeit < spanclass= " operatortoken " > = < \/ span > member . find ( 'linqmap:speed' , namespaces ) . text Verzögerung < spanclass= " operatortoken " > = < \/ span > member . find ( 'linqmap:delay' , namespaces ) . text cur = arcpy.da.InsertCursor(fcl, ["Speed", "Delay", "Title", "ReportType", "DateReported", "City", "Street", "Length", "ReportLevel", "UUID", "StartNode", "EndNode", "Description", "SHAPE@"]) array = arcpy.Array() für coords in Linie: xy = coords.split('],[') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Geschwindigkeit, Verzögerung, Titel, Typ, DatumGemeldet, Stadt, Straße, Länge, Ebene, UUID, StartNode, EndNode, Beschreibung, arcpy.Polyline(array)]) del cur print( token">'Stufe 2'<\/SPAN>)<\/SPAN> außerhalb von Ausnahme<\/SPAN>:<\/SPAN> passiere<\/SPAN> drucke<\/SPAN>(<\/SPAN>'Stufe 2 fehlgeschlagen'<\/SPAN>)<\/SPAN> sonst<\/SPAN>:<\/SPAN> versuche<\/SPAN>:<\/SPAN> Datum =<\/SPAN> mitglied.<\/SPAN>finde(<\/SPAN>'updateDate'<\/SPAN>)<\/SPAN>.<\/SPAN>text DatumGemeldet =<\/SPAN> datetime.<\/SPAN>strptime(<\/SPAN>Datum, <\/SPAN> '%a %b %d %H:%M:%S %z %Y'<\/SPAN>)<\/SPAN> wenn<\/SPAN> len(<\/SPAN>mitglied.<\/SPAN>findealle(<\/Span>'linqmap:city'<\/Span>, <\/Span> namespaces)<\/Span> 0: StartKnoten = mitglied.find('linqmap:startNode', namespaces).text sonst: StartKnoten = '' wenn len(mitglied.findall('linqmap:endNode', namespaces)) > 0: EndKnoten = mitglied.find('linqmap:endNode', namespaces).text sonst: EndKnoten = '' wenn len(mitglied.findall('linqmap:reportDescription', namespaces)) > 0: Beschreibung = mitglied.find('linqmap:reportDescription', namespaces).text sonst: Beschreibung = '' wenn len(mitglied.findall('linqmap:street', namespaces)) > 0: Straße = mitglied.find('linqmap:street', namespaces).text sonst: Straße = '' UUID = mitglied.find('linqmap:id', namespaces).text Typ = mitglied.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() Länge = mitglied.find('linqmap:length', namespaces).text Ebene = mitglied.find('linqmap:trend', namespaces).text linie = mitglied.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace'],[-', ', -').split(', ') Geschwindigkeit = mitglied.find('linqmap:speed', namespaces).text Verzögerung = mitglied.find('linqmap:delaySeconds', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed", "Delay", "Title", "ReportType"] token">,"DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() für coords in line: xy = coords.split('],[') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)]) del cur print('Stufe 3'‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍ Planung des Skripts

Sobald das Python-Skript läuft, verwenden Sie den Windows Aufgabenplanung, um das Skript in bestimmten Zeitintervallen auszuführen, wodurch neue Features zu den ArcGIS Online Feature-Layern hinzugefügt werden (diese Technik hält eine Historie aller Verkehrsinformationen seit dem ersten Ausführen des Skripts).

Screenshot der Stau-Feature-Layer

Anzeige von Live-Verkehrsinformationen

Um Live-Verkehrsinformationen anzuzeigen - erstellen Sie Feature-Layer-Ansichten und definieren Sie deren Features, um nur Features anzuzeigen, die in den letzten x Minuten erstellt wurden, wobei x die Häufigkeit ist, mit der Sie das Python-Skript eingerichtet haben.

Ich habe das Python-Skript so eingerichtet, dass es alle 4 Minuten ausgeführt wird - entsprechend die Feature-Layer-Ansichten, um Features anzuzeigen, die in den letzten 4 Minuten erstellt wurden.

Definieren Sie Features zur Anzeige von Live-Verkehrsinformationen

Sobald eingerichtet, können Sie diese Ansichten öffentlich teilen und in öffentlich zugänglichen Anwendungen anzeigen.

Unten finden Sie Links zu unseren Feature-Layer-Ansichten und eine Karte, die diese Features zusammen mit Straßen-/Gehwegssperrungen anzeigt.

Karte der Straßen-/Gehwegssperrungen in Canberra - einschließlich Live-Verkehr*

*Klicken Sie hier, um mehr über unseren Antragsprozess für Straßen-/Gehwegarbeiten zu erfahren

E-Mail an bestimmte Personen senden, wenn ein Unfall gemeldet wird

Ich habe die folgende Methode verwendet, alternativ könnten Sie ähnliche Python-Skripte nutzen oder Ihr eigenes Python-Skript erstellen, um eine E-Mail auszulösen, wenn neue Features zu einem Feature-Layer hinzugefügt werden.

Führen Sie das E-Mail-Skript Ihrer Wahl auf einer neuen Feature-Layer-Ansicht (Geometrietyp = Punkt), die nur Unfälle (ReportType = "Accident"), bei denen DateReported in den letzten x Minuten lag, anzeigt.

Definieren Sie Features, um nur Unfälle für ein bestimmtes Zeitintervall anzuzeigen

Und schließlich richten Sie den Windows Aufgabenplaner ein, um alle neuen Features (Unfälle) zu erfassen, die in den letzten x Minuten aufgetreten sind.

Ich hoffe, Sie können diese Workflows nutzen, um sie an Ihre eigenen Geschäftsanforderungen anzupassen. Und bitte zögern Sie nicht, Ihre Gedanken, Kommentare oder Fragen im Kommentarbereich unten zu teilen.

Viele Grüße! - Gee Fernando

Finde mehr Beiträge mit dem Tag

Kommentare

TrafficOpsGIS

Would anyone be willing to post their most up to date python script. Have not been able to run mine successfully for a couple months now. Thank you

TrafficOpsGIS

@daniel_marincounty Nice work, thank you for sharing.

TrafficOpsGIS

@JasonSimpson Waze changed their Feed URL format a few months back. Here you may find details on new format which you will need to upload your new format version on the Waze Partner Hub.

Please log in to the Partner Hub and use the new links for your feeds under “Get data from Waze” > “Waze data feed” (see image below)
The old URLs will be deprecated by end of May 2023

Screenshot 2023-03-14 at 13.23.532470×1604 226 KB

If you’re not sure, here are examples of how the URLs look like:

Old URLs:

New URL:

https://www.waze.com/partnerhub-api/partners/11697892328/waze-feeds/58c6486b-430e-4835-83a2-8a6e12593d96?format=1 14

JasonSimpson

Can someone help me? I've had a waze feed running consistently for a couple years now, but the feed suddenly stopped in july. After researching the issue, Ive noticed that a URL feed has changed. After making the proper changes, I am still receiving an error when running the script in PRO. Below is the script.

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/1103703####/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4###?format=1'))

root = tree.getroot()
channel_head = []
fcp = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Points/FeatureServer/0"
fcl = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Lines/FeatureServer/0"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')

And this is the error I am receiving when ran in PRO

Script2
=====================
Parameters

=====================
Messages

Start Time: Monday, August 28, 2023 9:46:17 AM
Traceback (most recent call last):
File "C:\Users\jsimpson\Desktop\WazeAug23.py", line 7, in <module>
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/11037037395/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4e04b?format=1'))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 540, in __init__
self.parse(file)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 580, in parse
self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

Failed script Script2...
Failed to execute (Script2).
Failed at Monday, August 28, 2023 9:46:18 AM (Elapsed Time: 0.91 seconds)

Any help is appreciated.

daniel_marincounty

I was able to copy @TrafficOpsGIS script into my own notebook and create the layers using the manual directions. However, it doesn't work when I run it in a notebook for some reason. If I run it as a .py it works 🙂 feel free to copy

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('##URL'))

root = tree.getroot()
channel_head = []
fcp = "##URL"
fcl = "##URL"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')

daniel_marincounty

The links to create the feature layers no longer work. Are there new ones?

TrafficOpsGIS

Final Final pt3 pt3 pt2 pt2 pt1 pt1

@RosyChawla2 , @MichelleNino Here is the script I use. See images 1-4 for complete script. Hope you are able to replicate. I run it directly in ArcPro Notebook. Please let me know if you have any questions on.

RosyChawla2

Hi All,

This script is exactly what we need at our county to be able to display data we get from Waze Connected Citizen Program. After proper indentation I tried running it with ArcPy 3.0 as well as ArcPy 2.7 Unfortunately I was not able to get it going.

Will appreciate any help.

Thanks,

-Rosy

MichelleNino

@TrafficOpsGIS Hello! I would like to know if you've tried to run this script with python3.0? Or have you modified this code according to your needs?

Thank you!

Michelle

TrafficOpsGIS

Can confirm that with some formatting for Python3.0 that this workflow does indeed still work! Thank you all for all your shared information!

TrafficOpsGIS

Does this workflow still work? Thank you

MarkCandland

I used this awesome article to get a nice dashboard up and running but after it running a couple months (and data piling up) I started running into performance issues. I thought it was the amount of records, so I created a new feed to test. That feed with far fewer records is having the same issue.

Anyone else experiencing some draw speed issues? it seems like the data is updating quickly (in my lists) but the map updates painfully slow.

Thanks

Mark

EmergencyManagement

Hello Geethaka Fernando We followed your instructions with your provided python link. Had to fix some formatting errors as when it was pasted it didn't indent correctly. I'm running into the issue that this script, when ran in python versions 2.7.12 & 3.8.2 both can not find the library / module urllib.requests. Is there a specific version of python I need to download to get this to work? Thanks

PaulRobinson1

Geethaka,

I'm Executive Director of a non-profit organization that exists to accelerate innovation and business growth around coastal resilience solutions.We're offering up to $500,000 to create a next generation traffic app that offers drivers real-time flooding and re-routing information.

If you're interested in learning more, visit riseresilience.org for more details, or email paulrobinson@riseresilience.org

Regards,

Paul Robinson

ErnestoCarreras2

Geethaka Fernando‌ thanks for this amazing script!!! I have a question regarding the DateReported and Length data. In the Waze documentation, it says the data is collected in UTC and kmh. Is the processed data in this format? I am asking because if that is the case, I would like to convert the data to my local time zone, as well as kmh to mph. Thanks!

ZairaRosas

Hey! Can anyone help me?

I'm getting no error running the script, but it's kind of strange because I live in northen hemisphere but my XML data has the "world-georss" on the url, so I ran both possible scripts (the script on the post and the script with solution that Geethaka Fernando gave to Eric Shreve ) and both worked... I even tried with and witout the "/0" at the end of the layer url and BOTH worked hahaha... What I mean by saying that all of this worked is that I got no errors running the script (I run it from the ArcGIS Pro IDLE) but no data is displaying on the "Data" tab from My Content / Waze Layers into ArcGIS Online... I don't know what else to do! Please, any one can help me???

I would really appreciate it!

PatriceCase1

This is a fantastic tutorial, thank you for sharing this!

GeneSipes1

Does anyone know of a way to modify the script so that it doesn't copy over duplicate rows every time the script is ran? I have two feature layers, one that holds all of the data, and is accumulated over time as the script is ran every 5 minutes, and another with a filter showing only the events that have been reported in the last 10 minutes. I am noticing that events are getting copied over multiple times and the data set is growing very fast. Thanks!

GeneSipes1

I was able to execute the python script, but I am getting the below error. Can you elaborate on how the script is supposed to access and edit the feature layer?

fail Stage 1
cannot open 'https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer'

Ok, I figured this one out. I needed to use the individual feature service with its index number at the end. Like this:

https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer/0

Anonymous User

Hello Gene, when I originally wrote this script it was for python 2.7 and it looks like from that error Noorhaizarinie Zaini gets is from trying to run it in python3. The parse function in python3 is different in some regard. you could try changing

to

tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.fromstring</SPAN><SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">##URL for XML waze connection##'))</SPAN>tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>ElementTree<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit;">##URL for XML waze connection##'))

</SPAN>

I am at the UC so I dont have a test environment to see if this works but I'm fairly certian it has to do with that line of code reading the xml from the url.

GeneSipes1

Hello Gee, I love this write up, thanks for sharing? Are you at the UC this year? If so I would love to meet up and see if you can help me quickly set this up. I have made my feature layers and altered the script, but I get the same error that Noorhaizarinie Zaini gets when executing it from Pro.

Thanks!

GeneSipes1

I got the same error. Did you get it solved?

NoorhaizarinieZaini1

Thanks Gee.

But could you please help me on this. I've run this script using ArcGIS Pro, but there are errors like the one below.

How to fix it?

Geethaka Fernando

https://community.esri.com/groups/departments-of-transportation/blog/2018/10/31/python-script-to-ingest-waze-data-to-arcgis-online

Error when run with ArcGIS Pro

NuozhouFu

Thanks for sharing the code! BTW, is it the only way to use Waze data by becoming a CCP member?

Anonymous User

Thanks Jonathan Pollack‌,

Have a look at this post - https://community.esri.com/groups/departments-of-transportation/blog/2019/02/04/traffic-study-reports-utilising-waze-data-and-arcgis-insights

Sometime in the near future, I'll update the script to ONLY ingest data into AGOL, when something changes in the XML feed.

Gee

deleted-user-8TVt6oc0ewsX

Great job on this! Has anyone done anything to deal with duplicate records getting created as the data gets consumed and loaded into AGOL? Maybe filter by date or some other field when ingesting the data? Any ideas?

Anonymous User

All good Joel.

In terms of error handling, the only thing I changed was - ability to handle missing elements and self closing tags.

For Example

<SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>member<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">></SPAN> <SPAN class="number token">0</SPAN> <SPAN class="operator token">and</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; Subtype <SPAN class="operator token">=</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'_'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">' '</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>capitalize<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; Subtype <SPAN class="operator token">=</SPAN> <SPAN class="string token">''</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

And, I haven't come across any errors since changing this

Gee

Anonymous User

Seeing people use this makes me wish I put some actual error handling in here

Anonymous User

You need to have ArcGIS Pro or ArcMap installed to use the module arcpy

And if you intend on using ArcMap, you'll have to amend the syntax slightly to cater for Python 2.x

Alternatively, you could look into using the ArcGIS API for Python

Gee

OriGudes3

Thanks, does it have to be ArcGIS Pro?

Nehmen Sie an der Diskussion teil

Angemeldete Mitglieder können Beiträge verfassen, Updates folgen und mehr. Neu hier? Registrieren Sie ein kostenloses Konto.

Schnellzugriffe

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.