nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Script Python pour ingérer les données Waze dans ArcGIS Online

Anonymous User

<\/HEAD>

Dans le cadre du Programme Waze Connected Citizen (CCP)<\/A>, les partenaires ont accès aux congestions de trafic en temps réel et aux incidents signalés via l'application Waze (aux formats XML et JSON). Il existe déjà des techniques et une documentation<\/A> sur la manière de consommer ces données dans une plateforme ArcGIS en utilisant GeoEvent.<\/P>

<\/P>

Dans ce blog, je décrirai une technique pour consommer ces données dans ArcGIS Online via Python et le Planificateur de tâches Windows. Le

script Python initial<\/A> a été développé par Joel Meier<\/A> que j'ai modifié pour correspondre au flux de travail proposé. Une mention spéciale va également à mes collègues Edmund Hall<\/A> et Louis Tian<\/A> qui m'ont aidé à déboguer. Et enfin, je tiens à remercier Andrew Stauffer<\/A> pour nous avoir aidés à collaborer et à tirer le meilleur parti du Waze CCP<\/A>.<\/P>

<\/P>

Création de couches d'entités ArcGIS Online pour stocker les informations de trafic<\/H2>Connectez-vous à ArcGIS Online > page Mon contenu, cliquez sur Créer > Couche d'entités, choisissez Depuis URL<\/EM> et saisissez l'URL exacte du modèle de couche d'entités ponctuelles. Ensuite, répétez la même procédure pour la couche d'entités linéaires. Cela créera de nouvelles couches d'entités vides avec le schéma requis (noms des champs, types et référence spatiale)<\/EM>.<\/P>Modèle de couche d'entités ponctuelles<\/STRONG><\/SPAN><\/P>https:\/\/services1.arcgis.com\/E5n4f1VY84i0xSjy\/arcgis\/rest\/services\/WazeTrafficAlertsEmpty\/FeatureServer<\/A><\/SPAN><\/P>
Modèle de couche d'entités linéaires<\/STRONG><\/SPAN><\/P>
https:\/\/services1.arcgis.com\/E5n4f1VY84i0xSjy\/arcgis\/rest\/services\/WazeTrafficJamsEmpty\/FeatureServer<\/A><\/SPAN><\/P><\BLOCKQUOTE>
Alternativement, visitez
https:\/\developers.arcgis.com/layers/new<\ /A> pour créer des couches d'entités Point et Ligne (WKID = 4326) avec les champs suivants.<\ /SPAN><\ /P><\ P ><\ /SPAN ><\ /P >

Point - Couche d'entités<\ /SPAN ><\ /H4 >OBJECTID (type : esriFieldTypeOID, alias : OBJECTID, type SQL : sqlTypeOther, longueur : 0, nullable : false, modifiable : false)<\ /STRONG ><\ /EM ><\ /SPAN ><\ /LI >DateReported (type : esriFieldTypeDate, alias : DateReported, type SQL : sqlTypeTimestamp2, longueur : 0, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >Title (type : esriFieldTypeString, alias : Title, type SQL : sqlTypeOther, longueur : 25, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >ReportType (type : esriFieldTypeString, alias : ReportType, type SQL : sqlTypeOther, longueur : 25, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >ReportSubtype (type : esriFieldTypeString, alias : ReportSubtype, type SQL : sqlTypeOther, longueur : 50, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >Street (type : esriFieldTypeString, alias : Street, type SQL : sqlTypeOther, longueur : 256, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >City (type : esriFieldTypeString, alias : City, type SQL : sqlTypeOther, longueur : 50, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >Description (type : esriFieldTypeString, alias : Description, type SQL : sqlTypeOther, longueur : 1024, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >Reliability (type : esriFieldTypeInteger, alias : Reliability, type SQL : sqlTypeOther, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI >UUID (type : esriFieldTypeString, alias : UUID, type SQL : sqlTypeOther, longueur : 100, nullable : true, modifiable : true)<\ /EM ><\ /SPAN ><\ /LI ><\ /UL >CreationDate (type : esriFieldTypeDate , alias : CreationDate , type SQL : sqlTypeOther , longueur : 8 , nullable : true , non modifiable : false )< \ /STRONG > < \ / EM > < \ / SPAN > < \ LI > < LI style = Description, Type SQL : sqlTypeOther, longueur : 1024, nullable : true, modifiable : true)<\/EM><\/SPAN><\/LI>UUID (type : esriFieldTypeDouble, alias : UUID, Type SQL : sqlTypeOther, nullable : true, modifiable : true)<\/EM><\/SPAN><\/LI><\/UL>ShapeLength (type : esriFieldTypeDouble, alias : ShapeLength, Type SQL : sqlTypeDouble, nullable : true, non modifiable)<\/STRONG><\/EM><\/SPAN><\/LI>CreationDate (type : esriFieldTypeDate, alias : CreationDate, Type SQL : sqlTypeOther, longueur : 8, nullable : true, non modifiable)<\/STRONG><\/EM><\/SPAN><\/LI>Creator (type : esriFieldTypeString, alias : Creator, Type SQL : sqlTypeOther, longueur : 128, nullable : true, non modifiable)<\/STRONG><\/EM><\/SPAN><\/LI>EditDate (type : esriFieldTypeDate, alias : EditDate, Type SQL : sqlTypeOther, longueur : 8, nullable : true, non modifiable)<\/STRONG><\/EM><\/SPAN><\/LI>Editor (type : esriFieldTypeString, alias : Editor, Type SQL : sqlTypeOther, longueur : 128, nullable : true, non modifiable)<\/STRONG><\/EM><\/SPAN><\/LI><\/UL><\/STRONG>champs pré-remplis<\/EM><\/SPAN><\/P>Veuillez vous assurer d'utiliser les noms de champs exacts et les types<\/EM><\/STRONG>, (sinon vous devrez ajuster le script).<\/SPAN><\/P>Une fois les couches d'entités configurées, veuillez activer l'édition<\/STRONG> et les définir en PRIVÉ<\/STRONG>.<\/SPAN><\/P>Veuillez vous référer aux "Waze Feed Technical Specifications [Get Waze Data]"<\/EM><\/STRONG> du Portail Partenaire<\/A> pour les définitions exactes des descriptions de champs.<\/P><\/BLOCKQUOTE>
<\/P>

Configuration du script Python<\/H4>Remplacez les sections de commentaires (lignes 6, 9 et 10)<\/SPAN><\/EM> par votre propre flux XML et les URL de vos couches d'entités points et lignes.<\/P>import<\/SPAN> xml.<\/SPAN>etree.<\/SPAN>ElementTree as<\/SPAN> ET import<\/SPAN> urllib.<\/SPAN>request import<\/SPAN> arcpy from<\/SPAN> datetime import<\/SPAN> datetime namespaces =<\/SPAN> {<\/SPAN>'georss'<\/SPAN> :<\/SPAN> '<\/'><A class="jive-link-external-small" href="https:\/\/community.esri.com\/external-link.jspa?url=http%3A%2F%2Fwww.georss.org%2Fgeorss" target="_blank">http:\/\/.georss.org/georss</A>'<\/'>,<\/'> 'linqmap'<\/'>: '<a class='jive-link-external-small' href='https://community.esri.com/external-link.jspa?url=http%3A%2F%2Fwww.linqmap.com' target='_blank'>http://www.linqmap.com</a>' } tree = ET<ElementTree>(file=urllib.request.urlopen('##URL for XML waze connection##')) root = tree.getroot() channel_head = [] fcp = "##URL to Point - Feature Layer##" fcl = "##URL to Line - Feature Layer##" for member in root.findall('channel/item'): channel = [] Title = member.find('title').text.upper() try: point = member.find('georss:point', namespaces) line = member.find('georss:line', namespaces) except: pass print('Fail Stage 0') if point != None: try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' if len(member.findall('linqmap:subtype', namespaces)) > 0 and member<punctuation token">.<\/SPAN>find(<\/SPAN>'linqmap:subtype'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text n'est pas<\/SPAN> None:<\/SPAN> Sous-type =<\/SPAN> member.<\/SPAN>find(<\/SPAN>'linqmap:subtype'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text.<\/SPAN>replace(<\/SPAN>'_'<\/SPAN>,<\/SPAN> ' '<\/SPAN>)<\/SPAN>.<\/SPAN>capitalize(<\/SPAN>)<\/SPAN> sinon<\/SPAN>:<\/SPAN> Sous-type =<\/SPAN> ''<\/Span> UUID =<\/span> member.<\/span>find(<\/span>'linqmap:uuid'<\/span>,<\/span> namespaces)<\/span>.<\/span>text Fiabilité =<\/span> member.<\/span>find(<\/span>'linqmap:reliability'<\/span>,<\/span> namespaces)<\/span>.<\/span>text si<\/span> len(<\/span>member.<\/span>findall(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>)<\/span> 3E<\/span> 0<\/span>:<\/span> Description =<\/span> member.<\/span>find(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>.<\/span>text sinon<\/span>:<\/span> Description =<\/span> ''<\/Span> Type = < \/ Span > Member . < \/ Span > Find ( < \/ Span > 'linqmap:type' < \/ Span > , < \/ Span > Namespaces ) < \/ Span > . Text . Replace < \/ Span > ( '_' < \/ Span > , ' ' < \/ Span > ) . Capitalize ( ) point =< \/Span > member . < \/Span > find(<'georss:point'><\ /Span>, namespaces<\ /Span>).text.replace(' -',',-').split(' ') ligne =< \/Span > Sous-type, Type, Titre, DateSignalée, Ville, Rue, UUID, Fiabilité, Description, float(point[1]), float(point[0]) curseur = arcpy.da.InsertCursor(fcp, ["ReportSubtype", "ReportType", "Title", "DateReported", "City", "Street", "UUID", "Reliability", "Description", "SHAPE@X", "SHAPE@Y"]) curseur.insertRow(ligne) del curseur print('étape 1') except: pass print('échec Étape 1') elif ligne != None: if Titre != 'IRREGULARITY': try: Date = member.find('pubDate').text DateSignalée = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: Ville = member.find('linqmap:city', namespaces).text else: Ville = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(<\/SPAN>member.<\/SPAN>findall(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>)<\/SPAN> ><\/SPAN> 0<\/SPAN>:<\/SPAN>\n EndNode =<\/SPAN> member.<\/SPAN>find(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>text\n else<\/SPAN>:<\/SPAN>\n EndNode =<\/SPAN> ''<\/SPAN>\n\n if<\/SPAN> len(<\/SPAN>member.<\/SPAN>findall(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span>\n Description =<\/span> member.<\/span>find(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>.<\/span>text\n else<\/span>:<\/span>\n Description =<\/span> ''<\/span>\n\n if<\/span> len(<\/span>member.<\/span>findall(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span>\n Street =<\/span> member.<\/span>find(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>.<\/span>text\n else<\/span>:<\/span>\n Street =<\/span> ''<\/span>\n\n UUID =<\/span> member.<\/span>find(<\/span>'linqmap:uuid'<\/span>,<\/span> namespaces)<\/span>.<\/span>text\n Type =<\/span> member.<\/span>find<(</pan><panclass=stringtoken>'linqmap:type'</pan>,namespaces).text.replace('_',' ').capitalize()\n Length <panclass=operatortoken>=</pan> member.find('linqmap:length',namespaces).text\n Level <panclass=operatortoken>=</pan> member.find('linqmap:level',namespaces).text\n line <panclass=operatortoken>=</pan> member.find('georss:line',namespaces).text.replace('\n','').replace(' ','],[').replace'],[-',' , -').split(', ')\n Speed <panclass=operatortoken>=</pan> member.find('linqmap:speed',namespaces).text\n Delay <panclass=operatortoken>=</pan> member.find('linqmap:delay',namespaces).text\n cur <panclass=operatortoken>=</pan> arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"] )\n array <panclass=operatortoken>=</pan> arcpy.Array()\n for coords in line:\n xy = coords.split('],[')\n array.add(arcpy.Point(xy[1], xy[0]))\n cur.insertRow([Speed, Delay, Title, Type, DateReported, City, Street, Length, Level, UUID, StartNode, EndNode, Description, arcpy.Polyline(array)])\n del cur\n print( token">'étape 2'<\/SPAN>)<\/SPAN> sauf<\/SPAN>:<\/SPAN> passer<\/SPAN> imprimer<\/SPAN>(<\/SPAN>'échec Étape 2'<\/SPAN>)<\/SPAN> sinon<\/SPAN>:<\/SPAN> essayer<\/SPAN>:<\/SPAN> Date =<\/SPAN> membre.<\/SPAN>trouver(<\/SPAN>'updateDate'<\/SPAN>)<\/SPAN>.<\/SPAN>texte DateReported =<\/SPAN> datetime.<\/SPAN>strptime(<\/SPAN>Date, <\/SPAN> '%a %b %d %H:%M:%S %z %Y'<\/SPAN>)<\/SPAN> si<\/SPAN> len(<\/SPAN>membre.<\/SPAN>findall(<\/SPAN>'linqmap:city'<\/Span>, namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Ville =<\/span> membre.<\/span>trouver(<\/span>'linqmap:city'<\/span>, namespaces)<\/span>.<\/span>texte sinon<\/span>:<\/span> Ville =<\/span> ''<\/span> si<\/span> len(<\/span>membre.<\/span>findall(<\/span>'linqmap:startNode'<\/span>, namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> NoeudDépart =<\/span> membre.<\/span>trouver(<\/span>'linqmap:startNode'<\/span>, namespaces)<\/span>.<\/span>texte sinon<\/span>:<\/span> NoeudDépart =<\/span> ''<\/span> si<\/span> len(<\/span>membre.<\/span>findall(<\/span>'linqmap:endNode'<\/span>, namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> NoeudFin < spanclass = \ "operatortoken \ "> = membre<spanclass = \ " punctuationtoken \ ">. trouver<spanclass = \ " punctuationtoken \ "> ( < spanclass = \ " stringtoken \ ">'linqmap:endNode' ,namespaces<spanclass = \ " punctuationtoken \ ">) .texte < spanclass = \ " keywordtoken \ "> sinon < spanclass = \ " punctuationtoken \ "> : NoeudFin < spanclass = \ " operatortoken \ "> = < spanclass = \ " stringtoken \ ">' ' < spanclass = \ " keywordtoken \ "> si len<spanclass = \ " punctuationtoken \ ">( membre<spanclass = \ " punctuationtoken \ ">. findall<spanclass = \ " punctuationtoken \ ">( < spanclass = \ " stringtoken \ ">'linqmap:reportDescription' ,namespaces<spanclass = \ " punctuationtoken \ ">) ) > 0 : Description < spanclass = \ " operatortoken \ "> = membre<spanclass = \ " punctuationtoken \ ">. trouver<spanclass = \ " punctuationtoken \ ">( < spanclass = \ " stringtoken \ ">'linqmap:reportDescription' ,namespaces<spanclass = \ " punctuationtoken \ ">) .texte < spanclass = \ " keywordtoken \ "> sinon < spanclass = \ " punctuationtoken \ "> : Description < spanclass = \ " operatortoken \ "> = '' < spanclass = \ " keywordtoken \ "> si len<spanclass = \ " punctuationtoken \ ">( membre<spanclass = \ " punctuationtoken \ ">. findall<spanclass = \ " punctuationtoken \ ">( 'linqmap:street', namespaces) ) > 0 : Rue < spanclass = \ " operatortoken \ "> = membre.find('linqmap:street', namespaces).texte sinon : Rue = '' UUID = membre.find('linqmap:id', namespaces).texte Type = membre.find('linqmap:type', namespaces).texte.replace('_', ' ').capitalize() Longueur = membre.find('linqmap:length', namespaces).texte Niveau = membre.find('linqmap:trend', namespaces).texte ligne = membre.find('georss:line', namespaces).texte.replace('\n', '').replace(' ', '],[').replace('],[-', ', -').split(', ') Vitesse = membre.find('linqmap:speed', namespaces).texte Délai = membre.find('linqmap:delaySeconds', namespaces).texte cur = arcpy.da.InsertCursor(fcl, ["Speed", "Delay", "Title", "ReportType"] token">,"DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() for coords in line: xy = coords.split('],[') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Titre,Type,DateReported,Ville,Rue,Longueur,Niveau,UUID,NoeudDépart,NoeudFin,Description,arcpy.Polyline(array)]) del cur print('étape 3' ) except: pass print('échec Étape 3'<spanclass= " punctuationtoken ">) < spanclass = " keywordtoken ">else: < spanclass = " keywordtoken ">print( 'Erreur'< spanclass = " punctuationtoken ">)< < EMOJI_0 > >‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍<\/SPAN>‍

Planification du script

Une fois le script Python en cours d'exécution, utilisez le Planificateur de tâches Windows pour exécuter le script à des intervalles spécifiques, ce qui ajoutera de nouvelles entités aux couches d'entités ArcGIS Online (cette technique conservera un historique de toutes les informations sur le trafic depuis la première exécution du script).

Capture d'écran de la couche d'entités des embouteillages

Affichage des informations de trafic en direct

Pour afficher les informations de trafic en direct - créez des vues de couches d'entités et définissez ses entités pour n'afficher que les entités créées dans les x dernières minutes - x étant la fréquence à laquelle vous avez configuré l'exécution du script Python..

J'ai configuré le script Python pour s'exécuter toutes les 4 minutes - par conséquent, les vues de couches d'entités affichent les entités créées dans les 4 dernières minutes.

Définir les entités pour afficher les informations de trafic en direct

Une fois configuré, vous pouvez partager ces vues publiquement et les afficher sur des applications destinées au public.

Voici des liens vers nos vues de couches d'entités, et une carte affichant ces entités ainsi que les fermetures de routes/trottoirs.

Carte des fermetures de routes/trottoirs de Canberra - y compris le trafic en direct*

*Cliquez ici pour en savoir plus sur notre processus de demande de travaux sur route/trottoir

Envoyer un e-mail à des personnes spécifiques lorsqu'un accident est signalé

J'ai utilisé la méthode suivante, alternativement vous pouvez utiliser des scripts Python similaires ou créer votre propre script Python pour déclencher un e-mail lorsqu'une nouvelle entité est ajoutée à une couche d'entités.

Exécutez le script e-mail de votre choix sur une nouvelle vue de couche d'entités hébergée (type de géométrie = point) qui n'affiche que les accidents (ReportType = "Accident"), où DateReported était dans les x dernières minutes.

Définir les entités pour n'afficher que les accidents pour un intervalle de temps spécifique

Et enfin, configurez le planificateur de tâches Windows pour capturer toutes nouvelles entités (accidents) qui se produisent dans les x dernières minutes.

J'espère que vous pourrez tirer parti de ces flux de travail pour répondre à vos propres besoins professionnels. Et n'hésitez pas à partager vos réflexions, commentaires ou questions dans la section commentaires ci-dessous.

Cordialement ! - Gee Fernando

Trouver plus de publications taguées avec

Réponses

TrafficOpsGIS

Would anyone be willing to post their most up to date python script. Have not been able to run mine successfully for a couple months now. Thank you

TrafficOpsGIS

@daniel_marincounty Nice work, thank you for sharing.

TrafficOpsGIS

@JasonSimpson Waze changed their Feed URL format a few months back. Here you may find details on new format which you will need to upload your new format version on the Waze Partner Hub.

Please log in to the Partner Hub and use the new links for your feeds under “Get data from Waze” > “Waze data feed” (see image below)
The old URLs will be deprecated by end of May 2023

Screenshot 2023-03-14 at 13.23.532470×1604 226 KB

If you’re not sure, here are examples of how the URLs look like:

Old URLs:

New URL:

https://www.waze.com/partnerhub-api/partners/11697892328/waze-feeds/58c6486b-430e-4835-83a2-8a6e12593d96?format=1 14

JasonSimpson

Can someone help me? I've had a waze feed running consistently for a couple years now, but the feed suddenly stopped in july. After researching the issue, Ive noticed that a URL feed has changed. After making the proper changes, I am still receiving an error when running the script in PRO. Below is the script.

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/1103703####/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4###?format=1'))

root = tree.getroot()
channel_head = []
fcp = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Points/FeatureServer/0"
fcl = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Lines/FeatureServer/0"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')

And this is the error I am receiving when ran in PRO

Script2
=====================
Parameters

=====================
Messages

Start Time: Monday, August 28, 2023 9:46:17 AM
Traceback (most recent call last):
File "C:\Users\jsimpson\Desktop\WazeAug23.py", line 7, in <module>
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/11037037395/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4e04b?format=1'))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 540, in __init__
self.parse(file)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 580, in parse
self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

Failed script Script2...
Failed to execute (Script2).
Failed at Monday, August 28, 2023 9:46:18 AM (Elapsed Time: 0.91 seconds)

Any help is appreciated.

daniel_marincounty

I was able to copy @TrafficOpsGIS script into my own notebook and create the layers using the manual directions. However, it doesn't work when I run it in a notebook for some reason. If I run it as a .py it works 🙂 feel free to copy

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('##URL'))

root = tree.getroot()
channel_head = []
fcp = "##URL"
fcl = "##URL"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')

daniel_marincounty

The links to create the feature layers no longer work. Are there new ones?

TrafficOpsGIS

Final Final pt3 pt3 pt2 pt2 pt1 pt1

@RosyChawla2 , @MichelleNino Here is the script I use. See images 1-4 for complete script. Hope you are able to replicate. I run it directly in ArcPro Notebook. Please let me know if you have any questions on.

RosyChawla2

Hi All,

This script is exactly what we need at our county to be able to display data we get from Waze Connected Citizen Program. After proper indentation I tried running it with ArcPy 3.0 as well as ArcPy 2.7 Unfortunately I was not able to get it going.

Will appreciate any help.

Thanks,

-Rosy

MichelleNino

@TrafficOpsGIS Hello! I would like to know if you've tried to run this script with python3.0? Or have you modified this code according to your needs?

Thank you!

Michelle

TrafficOpsGIS

Can confirm that with some formatting for Python3.0 that this workflow does indeed still work! Thank you all for all your shared information!

TrafficOpsGIS

Does this workflow still work? Thank you

MarkCandland

I used this awesome article to get a nice dashboard up and running but after it running a couple months (and data piling up) I started running into performance issues. I thought it was the amount of records, so I created a new feed to test. That feed with far fewer records is having the same issue.

Anyone else experiencing some draw speed issues? it seems like the data is updating quickly (in my lists) but the map updates painfully slow.

Thanks

Mark

EmergencyManagement

Hello Geethaka Fernando We followed your instructions with your provided python link. Had to fix some formatting errors as when it was pasted it didn't indent correctly. I'm running into the issue that this script, when ran in python versions 2.7.12 & 3.8.2 both can not find the library / module urllib.requests. Is there a specific version of python I need to download to get this to work? Thanks

PaulRobinson1

Geethaka,

I'm Executive Director of a non-profit organization that exists to accelerate innovation and business growth around coastal resilience solutions.We're offering up to $500,000 to create a next generation traffic app that offers drivers real-time flooding and re-routing information.

If you're interested in learning more, visit riseresilience.org for more details, or email paulrobinson@riseresilience.org

Regards,

Paul Robinson

ErnestoCarreras2

Geethaka Fernando‌ thanks for this amazing script!!! I have a question regarding the DateReported and Length data. In the Waze documentation, it says the data is collected in UTC and kmh. Is the processed data in this format? I am asking because if that is the case, I would like to convert the data to my local time zone, as well as kmh to mph. Thanks!

ZairaRosas

Hey! Can anyone help me?

I'm getting no error running the script, but it's kind of strange because I live in northen hemisphere but my XML data has the "world-georss" on the url, so I ran both possible scripts (the script on the post and the script with solution that Geethaka Fernando gave to Eric Shreve ) and both worked... I even tried with and witout the "/0" at the end of the layer url and BOTH worked hahaha... What I mean by saying that all of this worked is that I got no errors running the script (I run it from the ArcGIS Pro IDLE) but no data is displaying on the "Data" tab from My Content / Waze Layers into ArcGIS Online... I don't know what else to do! Please, any one can help me???

I would really appreciate it!

PatriceCase1

This is a fantastic tutorial, thank you for sharing this!

GeneSipes1

Does anyone know of a way to modify the script so that it doesn't copy over duplicate rows every time the script is ran? I have two feature layers, one that holds all of the data, and is accumulated over time as the script is ran every 5 minutes, and another with a filter showing only the events that have been reported in the last 10 minutes. I am noticing that events are getting copied over multiple times and the data set is growing very fast. Thanks!

GeneSipes1

I was able to execute the python script, but I am getting the below error. Can you elaborate on how the script is supposed to access and edit the feature layer?

fail Stage 1
cannot open 'https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer'

Ok, I figured this one out. I needed to use the individual feature service with its index number at the end. Like this:

https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer/0

Anonymous User

Hello Gene, when I originally wrote this script it was for python 2.7 and it looks like from that error Noorhaizarinie Zaini gets is from trying to run it in python3. The parse function in python3 is different in some regard. you could try changing

to

tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.fromstring</SPAN><SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">##URL for XML waze connection##'))</SPAN>tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>ElementTree<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit;">##URL for XML waze connection##'))

</SPAN>

I am at the UC so I dont have a test environment to see if this works but I'm fairly certian it has to do with that line of code reading the xml from the url.

GeneSipes1

Hello Gee, I love this write up, thanks for sharing? Are you at the UC this year? If so I would love to meet up and see if you can help me quickly set this up. I have made my feature layers and altered the script, but I get the same error that Noorhaizarinie Zaini gets when executing it from Pro.

Thanks!

GeneSipes1

I got the same error. Did you get it solved?

NoorhaizarinieZaini1

Thanks Gee.

But could you please help me on this. I've run this script using ArcGIS Pro, but there are errors like the one below.

How to fix it?

Geethaka Fernando

https://community.esri.com/groups/departments-of-transportation/blog/2018/10/31/python-script-to-ingest-waze-data-to-arcgis-online

Error when run with ArcGIS Pro

NuozhouFu

Thanks for sharing the code! BTW, is it the only way to use Waze data by becoming a CCP member?

Anonymous User

Thanks Jonathan Pollack‌,

Have a look at this post - https://community.esri.com/groups/departments-of-transportation/blog/2019/02/04/traffic-study-reports-utilising-waze-data-and-arcgis-insights

Sometime in the near future, I'll update the script to ONLY ingest data into AGOL, when something changes in the XML feed.

Gee

deleted-user-8TVt6oc0ewsX

Great job on this! Has anyone done anything to deal with duplicate records getting created as the data gets consumed and loaded into AGOL? Maybe filter by date or some other field when ingesting the data? Any ideas?

Anonymous User

All good Joel.

In terms of error handling, the only thing I changed was - ability to handle missing elements and self closing tags.

For Example

<SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>member<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">></SPAN> <SPAN class="number token">0</SPAN> <SPAN class="operator token">and</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; Subtype <SPAN class="operator token">=</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'_'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">' '</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>capitalize<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; Subtype <SPAN class="operator token">=</SPAN> <SPAN class="string token">''</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

And, I haven't come across any errors since changing this

Gee

Anonymous User

Seeing people use this makes me wish I put some actual error handling in here

Anonymous User

You need to have ArcGIS Pro or ArcMap installed to use the module arcpy

And if you intend on using ArcMap, you'll have to amend the syntax slightly to cater for Python 2.x

Alternatively, you could look into using the ArcGIS API for Python

Gee

OriGudes3

Thanks, does it have to be ArcGIS Pro?

Rejoignez la conversation

Les membres connectés peuvent publier, suivre les mises à jour, et plus encore. Nouveau ici ? Inscrivez-vous gratuitement.

Liens rapides

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.