Als onderdeel van het Waze Connected Citizen Program (CCP)<\/A>, krijgen partners toegang tot realtime verkeersopstoppingen en incidenten die via de Waze-app worden gemeld (in XML- en JSON-formaten). Er zijn al technieken en documentatie<\/A> over hoe deze gegevens in een ArcGIS-platform kunnen worden gebruikt met GeoEvent.<\/P><\/P>In deze blog zal ik een techniek beschrijven om deze gegevens te gebruiken in ArcGIS Online via Python en Windows Taakplanner. Het initiële Python-script<\/A> is ontwikkeld door Joel Meier<\/A> die ik heb aangepast om te passen bij de voorgestelde workflow. Ook een speciale vermelding voor mijn collega’s Edmund Hall<\/A> en Louis Tian<\/A> die mij hebben geholpen met debuggen. En last but not least wil ik Andrew Stauffer<\/A> bedanken voor het helpen van ons partner en het maximale uit het Waze CCP<\/A> te halen.<\/P><\/P>
<\/P>
In deze blog zal ik een techniek beschrijven om deze gegevens te gebruiken in ArcGIS Online via Python en Windows Taakplanner. Het
Lijn Feature Layer Sjabloon<\/STRONG><\/SPAN><\/P>
import<\/SPAN> xml.<\/SPAN>etree.<\/SPAN>ElementTree as<\/SPAN> ETimport<\/SPAN> urllib.<\/SPAN>requestimport<\/SPAN> arcpyfrom<\/SPAN> datetime import<\/SPAN> datetimenamespaces =<\/SPAN> {<\/SPAN>'georss'<\/SPAN> :<\/SPAN> '<\/'>http:\/\/www.georss.orggeorss'<\/'>,<\/'> 'linqmap'<\/'>:'http://www.linqmap.com' }tree = ET.ElementTree(file=urllib.request.urlopen('##URL for XML waze connection##'))root = tree.getroot()channel_head = []fcp = "##URL to Point - Feature Layer##"fcl = "##URL to Line - Feature Layer##"for member in root.findall('channel/item'): channel =[] Title = member.find('title').text.upper()try: point = member.find('georss:point', namespaces)) line = member.find('georss:line', namespaces))except: pass print('Fail Stage 0') if point != None: try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' if len(member.findall('linqmap:subtype', namespaces)) > 0 and member token">.<\/SPAN>vind(<\/SPAN>'linqmap:subtype'<\/SPAN>, <\/SPAN>namespaces)<\/SPAN>.<\/SPAN>tekst is<\/SPAN> niet<\/SPAN> None:<\/SPAN> Subtype =<\/SPAN> lid.<\/SPAN>vind(<\/SPAN>'linqmap:subtype'<\/SPAN>, <\/SPAN>namespaces)<\/SPAN>.<\/SPAN>tekst.<\/SPAN>vervang(<\/SPAN>'_'<\/SPAN>, <\/SPAN>' '<\/SPAN>)<\/SPAN>.<\/SPAN>capitalizeer(<\/SPAN>)<\/span>anders<\/span>:<\/span> Subtype =<\/span> ''<\/span> UUID =<\/span> lid.<\/span>vind(<\/span>'linqmap:uuid'<\/span>, <\/span>namespaces)<\/span>.<\/span>tekst Betrouwbaarheid =<\/span> lid.<\/span>vind(<\/span>'linqmap:reliability'<\/span>, <\/span>namespaces)<\/span>.<\/span>tekstals<\/span> len(<\/span>lid.<\/span>vindalles(<\/span>'linqmap:reportDescription'<\/span>, <\/span>namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Beschrijving =<\/span> lid.<\/span>vind(<\/span>'linqmap:reportDescription'<\/span>, <\/span>namespaces)<\/span>.<\/span>tekstanders<\/span>:<\/span> Beschrijving =<\/span> ''<\/span> Type < spanclass= "operatortoken" > = < \/ span > lid < spanclass= " punctuationtoken " > . < \/ span > vind < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > 'linqmap:type' < \/ span > , namespaces < spanclass= " punctuationtoken " > ) < \/ span > . tekst < spanclass= " punctuationtoken " > . vervang < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > '_' < \/ span > , < spanclass= " stringtoken " > ' ' < \/ span > ) . capitalizeer ( ) punt < spanclass= " operatortoken " > = lid . vind ( 'georss:point', namespaces ) . tekst . vervang ( ' -', ',-' ) . split ( ' ' ) rij = Subtype, Type, Titel, DatumGerapporteerd, Stad, Straat, UUID, Betrouwbaarheid, Beschrijving, float(punt[1]), float(punt[0]) cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype", "ReportType", "Title", "DateReported", "City", "Street", "UUID", "Reliability", "Description", "SHAPE@X", "SHAPE@Y"]) cursor.insertRow(rij) verwijder cursor print('fase 1') behalve: passeren print('mislukte Fase 1') elif lijn != None: als Titel != 'IRREGULARITY': probeer: Datum = lid.vind('pubDate').tekst DatumGerapporteerd = datetime.strptime(Datum, '%a %b %d %H:%M:%S %z %Y') als len(lid.vindalles('linqmap:city', namespaces)) > 0: Stad = lid.vind('linqmap:city', namespaces).tekst anders: Stad = '' als len(lid.vindalles('linqmap:startNode', namespaces)) > 0: StartNode = lid.vind('linqmap:startNode', namespaces).tekst anders: StartNode = '' als len(<\/SPAN>member.<\/SPAN>findall(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>)<\/SPAN> ><\/SPAN> 0<\/SPAN>:<\/SPAN> EndNode =<\/SPAN> member.<\/SPAN>find(<\/SPAN>'linqmap:endNode'<\/SPAN>,<\/SPAN> namespaces)<\/SPAN>.<\/SPAN>textanders<\/SPAN>:<\/SPAN> EndNode =<\/SPAN> ''<\/SPAN>als<\/SPAN> len(<\/SPAN>member.<\/span>findall(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Description =<\/span> member.<\/span>find(<\/span>'linqmap:reportDescription'<\/span>,<\/span> namespaces)<\/span>.<\/span>textanders<\/span>:<\/span> Description =<\/span> ''<\/span>als<\/span> len(<\/span>member.<\/span>findall(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>)<\/span> ><\/span> 0<\/span>:<\/span> Street =<\/span> member.<\/span>find(<\/span>'linqmap:street'<\/span>,<\/span> namespaces)<\/span>.<\/span>textanders<\/span>:<\/span> Street =<\/span> ''<\/span> UUID < spanclass= "operatortoken" > = < \/ span > member < spanclass= " punctuationtoken " > . < \/ span > find < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > 'linqmap:uuid' < \/ span > , namespaces < spanclass= " punctuationtoken " > ) < \/ span > . text Type < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . < \/ span > find < spanclass= " punctuationtoken " > ( < \/ span > < spanclass= " stringtoken " > 'linqmap:type' < \/ span > , namespaces < spanclass= " punctuationtoken " > ) < \/ span > . text . replace ( '_' , ' ' ) . capitalize ( ) Length < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . find ( 'linqmap:length' , namespaces ) . text Level < spanclass= " operatortoken " > = < \/ span > member < spanclass= " punctuationtoken " > . find ( 'linqmap:level' , namespaces ) . text line < spanclass= " operatortoken " > = < \/ span > member . find ( 'georss:line' , namespaces ) . text . replace ( '\n' , '' ) . replace ( ' ' , '],[' ) . replace ( '],[-' , ', -' ) . split ( ', ' ) Speed = member.find('linqmap:speed', namespaces).text Delay = member.find('linqmap:delay', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() voor coords in line: xy = coords.split('],[') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed, Delay, Title, Type, DateReported, City, Street, Length, Level, UUID, StartNode, EndNode, Description, arcpy.Polyline(array)]) verwijder cur print( token">'fase 2'<\/SPAN>)<\/SPAN>behalve<\/SPAN>:<\/SPAN>passeren<\/SPAN>printen<\/SPAN>(<\/SPAN>'mislukte Fase 2'<\/SPAN>)<\/SPAN>anders<\/SPAN>:<\/SPAN>proberen<\/SPAN>:<\/SPAN> Datum =<\/SPAN> lid.<\/SPAN>vinden(<\/SPAN>'updateDate'<\/SPAN>)<\/SPAN>.<\/SPAN>tekst DatumGerapporteerd =<\/SPAN> datetime.<\/SPAN>strptime(<\/SPAN>Datum, <\/SPAN> '%a %b %d %H:%M:%S %z %Y'<\/SPAN>)<\/SPAN>als<\/SPAN> len(<\/SPAN>lid.<\/SPAN>vindenalles(<\/SPAN>'linqmap:city'<\/Span>,<\/span> namespaces<\/) > 0:<\/) Stad <span clasS=\"operator toKen\">=<\/) lid<SPAn clasS=\"punctuatiON toKen\"><\/>vinden<\(</span>'linqmap:city', namespaces<\)</span><\.>tekstanders:<\/span> Stad =<\/span> ''als len<(</span>lid<.>vindenalles<(</span>'linqmap:startNode', namespaces<)</span><)</span> > 0:<\/span> StartNode =<\/span> lid<.>vinden<(</span>'linqmap:startNode', namespaces<)</span><.>tekstanders:<\/span> StartNode =<\/span> ''als len<(</span>lid<.>vindenalles<(</span>'linqmap:endNode',"DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array()voor coords in line: xy = coords.split('],[') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed, Delay, Title, Type, DateReported, City, Street, Length, Level, UUID, StartNode, EndNode, Description, arcpy.Polyline(array)])verwijder curprint('fase 3')behalve:passerenprint('mislukte fase 3')anders:print('Fout')<\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><\/SPAN><<emojI_31 >><<emojI_32 >><<emojI_33 >><<emojI_34 >><<emojI_35 >><<emojI_36 >><<emojI_37 >><<emojI_38 >><<emojI_39 >><<emojI_40 >><<emojI_41 >><<emojI_42 >><<emojI_43 >><<emojI_44 >><<emojI_45 >><<emojI_46 >><<emojI_47 >><<emojI_48 >><<emojI_49 >><<emojI_50 >><<emojI_51 >><<emojI_52 >><<emojI_53 >><<emojI_54 >><<emojI_55 >><<emojI_56 >><<emojI_57 >><<emojI_58 >><<emojI_59 >><<emojI_60 >><<emojI_61 >><<emojI_62 >><<emojI_63 >><<emojI_64 ></ span >< span >> << emoj I _ 65 >> < / span >< span >> << emoj I _ 66 >> < / span >< span >> << emoj I _ 67 >> < / span >< span >> << emoj I _ 68 >> < / span >< span >> << emoj I _ 69 >> < / span >< span >> << emoj I _ 70 >> < / span >< span >> << emoj I _ 71 >> < / span >< span >> << emoj I _ 72 >> < / span >< span >> << emoj I _ 73 >> < / span >< span >> << emoj I _ 74 >> < / span >< span >> << emoj I _ 75 >> < / span >< span >> << emoj I _ 76 >> < / span >< span >> << emoj I _ 77 >> < / span >< span >> << emoj I _ 78 >> < / span >< span >> << emoj I _ 79 >> < / span >< span >> << emoj I _ 80 >> < / span >< span >> << emoj I _ 81 >> < / span >< span >> << emoj I _ 82 >> < / span >< span >> << emoj I _ 83 >> < / span >< span >> << emoj I _ 84 >> < / span >< span >> << emoj I _ 85 >> < / span >< span >> << emoj I _ 86 >> < / span >< span >> << emoj I _ 87 >> < / span >< span >> << emoj I _ 88 >> < / span >< span >> << emoj I _ 89 >> < / span >< span>> << em oj I _90>> > << e moj I _91>> > << e moj I _92>> > << e moj I _93>> > << e moj I _94>> > << e moj I _95>> > << e moj I _96>> > << e moj I _97>> > << e moj I _98>> > << e moj I _99>> > << e moj I _100>> > << e moj I _101>> > << e moj I _102>> > << e moj I _103>> > << e moj I _104>> > << e moj I _105>> > << e moj I _106>> > << e moj I _107>> > << e moj I _108>> > << e moj I _109>> > << e moj I _110>> > << e moj I _111>> > << e moj I _112>> > << e moj I _113>> </ sp a n> < sp a n>>> <<< EMOJ_I__114>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__115>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__116>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__117>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__118>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__119>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__120>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__121>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__122>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__123>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__124>>> <<< / SPAN>>> <<< SPAN>>> <<< EMOJ_I__125>>> <<< / SPAN >>><<<SP AN >>><<< EMOJ_I__126 >>><<<SP AN >>><<< EMOJ_I__127 >>><<<SP AN >>><<< EMOJ_I__128 >>><<<SP AN >>><<< EMOJ_I__129 >>><<<SP AN >>><<< EMOJ_I__130 >>><<<SP AN >>><<< EMOJ_I__131 >>><<<SP AN >>><<< EMOJ_I__132 >>><<<SP AN >>><<< EMOJ_I__133 >>><<<SP AN >>><<< EMOJ_I__134 >>><<<SP AN >>><<< EMOJ_I__135 >>><<<SP AN >>><<< EMOJ_I__136 >>><<<SP AN >>><<< EMOJ_I__137 >>><<<SP AN >>><<< EMOJ_I__138 >>><<<SP AN >>><<< EMOJ_I__139 >>><<<SP AN >>><<< EMOJ_I__140 >>><<<SP AN >>><<< EMOJ_I__141 >>><<<SP AN >>><<< EMOJ_I__142 >>><<<SP AN >>><<< EMOJ_I__143 >>><<<SP AN >>><<< EMOJ_I__144 >>><<<SP AN >>><<< EMOJ_I__145 >>><<<SP AN >>><<< EMOJ_I__146 >>><<<SP AN >>><<< EMOJ_I__147 >>><<<SP AN >>><<< EMOJ_I__148 >>><<<< SPAN>>>><<EM OJ_I __149>>>><<S PAN>>>><<E MOJ_I __150>>>><<S PAN>>>><<E MOJ_I __151>>>><<S PAN>>>><<E MOJ_I __152>>>><<S PAN>>>><<E MOJ_I __153>>>><<S PAN>>>><<E MOJ_I __154>>>><<S PAN>>>><<E MOJ_I __155>>>>
Would anyone be willing to post their most up to date python script. Have not been able to run mine successfully for a couple months now. Thank you
@daniel_marincounty Nice work, thank you for sharing.
@JasonSimpson Waze changed their Feed URL format a few months back. Here you may find details on new format which you will need to upload your new format version on the Waze Partner Hub.
If you’re not sure, here are examples of how the URLs look like:
Old URLs:
New URL:
https://www.waze.com/partnerhub-api/partners/11697892328/waze-feeds/58c6486b-430e-4835-83a2-8a6e12593d96?format=1 14
Can someone help me? I've had a waze feed running consistently for a couple years now, but the feed suddenly stopped in july. After researching the issue, Ive noticed that a URL feed has changed. After making the proper changes, I am still receiving an error when running the script in PRO. Below is the script.
import xml.etree.ElementTree as ET import urllib.request import arcpy from datetime import datetime namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' } tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/1103703####/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4###?format=1')) root = tree.getroot() channel_head = [] fcp = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Points/FeatureServer/0" fcl = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Lines/FeatureServer/0" for member in root.findall('channel/item'): channel = [] Title = member.find('title').text.upper() try: point = member.find('georss:point', namespaces) line = member.find('georss:line', namespaces) except: pass print('Fail Stage 0') if point != None: try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None: Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize() else: Subtype = '' UUID = member.find('linqmap:uuid', namespaces).text Reliability = member.find('linqmap:reliability', namespaces).text if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',') point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',') row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0]) cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"]) cursor.insertRow(row) del cursor print('stage 1') except: pass print('fail Stage 1') elif line != None: if Title != 'IRREGULARITY': try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(member.findall('linqmap:endNode', namespaces)) > 0: EndNode = member.find('linqmap:endNode', namespaces).text else: EndNode = '' if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' UUID = member.find('linqmap:uuid', namespaces).text Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() Length = member.find('linqmap:length', namespaces).text Level = member.find('linqmap:level', namespaces).text line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[') Speed = member.find('linqmap:speed', namespaces).text Delay = member.find('linqmap:delay', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() for coords in line: xy = coords.split(', ') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)]) del cur print('stage 2') except: pass print('fail Stage 2') else: try: Date = member.find('updateDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(member.findall('linqmap:endNode', namespaces)) > 0: EndNode = member.find('linqmap:endNode', namespaces).text else: EndNode = '' if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' UUID = member.find('linqmap:id', namespaces).text Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() Length = member.find('linqmap:length', namespaces).text Level = member.find('linqmap:trend', namespaces).text line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[') Speed = member.find('linqmap:speed', namespaces).text Delay = member.find('linqmap:delaySeconds', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() for coords in line: xy = coords.split(', ') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)]) del cur print('stage 3') except: pass print('fail Stage 3') else: print('Error')
And this is the error I am receiving when ran in PRO
Script2=====================Parameters
=====================Messages
Start Time: Monday, August 28, 2023 9:46:17 AMTraceback (most recent call last):File "C:\Users\jsimpson\Desktop\WazeAug23.py", line 7, in <module>tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/11037037395/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4e04b?format=1'))File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 540, in __init__self.parse(file)File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 580, in parseself._root = parser._parse_whole(source)xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0
Failed script Script2...Failed to execute (Script2).Failed at Monday, August 28, 2023 9:46:18 AM (Elapsed Time: 0.91 seconds)
Any help is appreciated.
I was able to copy @TrafficOpsGIS script into my own notebook and create the layers using the manual directions. However, it doesn't work when I run it in a notebook for some reason. If I run it as a .py it works 🙂 feel free to copy
import xml.etree.ElementTree as ET import urllib.request import arcpy from datetime import datetime namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' } tree = ET.ElementTree(file=urllib.request.urlopen('##URL')) root = tree.getroot() channel_head = [] fcp = "##URL" fcl = "##URL" for member in root.findall('channel/item'): channel = [] Title = member.find('title').text.upper() try: point = member.find('georss:point', namespaces) line = member.find('georss:line', namespaces) except: pass print('Fail Stage 0') if point != None: try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None: Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize() else: Subtype = '' UUID = member.find('linqmap:uuid', namespaces).text Reliability = member.find('linqmap:reliability', namespaces).text if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',') point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',') row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0]) cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"]) cursor.insertRow(row) del cursor print('stage 1') except: pass print('fail Stage 1') elif line != None: if Title != 'IRREGULARITY': try: Date = member.find('pubDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(member.findall('linqmap:endNode', namespaces)) > 0: EndNode = member.find('linqmap:endNode', namespaces).text else: EndNode = '' if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' UUID = member.find('linqmap:uuid', namespaces).text Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() Length = member.find('linqmap:length', namespaces).text Level = member.find('linqmap:level', namespaces).text line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[') Speed = member.find('linqmap:speed', namespaces).text Delay = member.find('linqmap:delay', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() for coords in line: xy = coords.split(', ') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)]) del cur print('stage 2') except: pass print('fail Stage 2') else: try: Date = member.find('updateDate').text DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y') if len(member.findall('linqmap:city', namespaces)) > 0: City = member.find('linqmap:city', namespaces).text else: City = '' if len(member.findall('linqmap:startNode', namespaces)) > 0: StartNode = member.find('linqmap:startNode', namespaces).text else: StartNode = '' if len(member.findall('linqmap:endNode', namespaces)) > 0: EndNode = member.find('linqmap:endNode', namespaces).text else: EndNode = '' if len(member.findall('linqmap:reportDescription', namespaces)) > 0: Description = member.find('linqmap:reportDescription', namespaces).text else: Description = '' if len(member.findall('linqmap:street', namespaces)) > 0: Street = member.find('linqmap:street', namespaces).text else: Street = '' UUID = member.find('linqmap:id', namespaces).text Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize() Length = member.find('linqmap:length', namespaces).text Level = member.find('linqmap:trend', namespaces).text line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[') Speed = member.find('linqmap:speed', namespaces).text Delay = member.find('linqmap:delaySeconds', namespaces).text cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"]) array = arcpy.Array() for coords in line: xy = coords.split(', ') array.add(arcpy.Point(xy[1], xy[0])) cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)]) del cur print('stage 3') except: pass print('fail Stage 3') else: print('Error')
The links to create the feature layers no longer work. Are there new ones?
Finalpt3pt2pt1
@RosyChawla2 , @MichelleNino Here is the script I use. See images 1-4 for complete script. Hope you are able to replicate. I run it directly in ArcPro Notebook. Please let me know if you have any questions on.
Hi All,
This script is exactly what we need at our county to be able to display data we get from Waze Connected Citizen Program. After proper indentation I tried running it with ArcPy 3.0 as well as ArcPy 2.7 Unfortunately I was not able to get it going.
Will appreciate any help.
Thanks,
-Rosy
@TrafficOpsGIS Hello! I would like to know if you've tried to run this script with python3.0? Or have you modified this code according to your needs?
Thank you!
Michelle
Can confirm that with some formatting for Python3.0 that this workflow does indeed still work! Thank you all for all your shared information!
Does this workflow still work? Thank you
I used this awesome article to get a nice dashboard up and running but after it running a couple months (and data piling up) I started running into performance issues. I thought it was the amount of records, so I created a new feed to test. That feed with far fewer records is having the same issue.
Anyone else experiencing some draw speed issues? it seems like the data is updating quickly (in my lists) but the map updates painfully slow.
Thanks
Mark
Hello Geethaka Fernando We followed your instructions with your provided python link. Had to fix some formatting errors as when it was pasted it didn't indent correctly. I'm running into the issue that this script, when ran in python versions 2.7.12 & 3.8.2 both can not find the library / module urllib.requests. Is there a specific version of python I need to download to get this to work? Thanks
Geethaka,
I'm Executive Director of a non-profit organization that exists to accelerate innovation and business growth around coastal resilience solutions.We're offering up to $500,000 to create a next generation traffic app that offers drivers real-time flooding and re-routing information.
If you're interested in learning more, visit riseresilience.org for more details, or email paulrobinson@riseresilience.org
Regards,
Paul Robinson
Geethaka Fernando thanks for this amazing script!!! I have a question regarding the DateReported and Length data. In the Waze documentation, it says the data is collected in UTC and kmh. Is the processed data in this format? I am asking because if that is the case, I would like to convert the data to my local time zone, as well as kmh to mph. Thanks!
Hey! Can anyone help me?
I'm getting no error running the script, but it's kind of strange because I live in northen hemisphere but my XML data has the "world-georss" on the url, so I ran both possible scripts (the script on the post and the script with solution that Geethaka Fernando gave to Eric Shreve ) and both worked... I even tried with and witout the "/0" at the end of the layer url and BOTH worked hahaha... What I mean by saying that all of this worked is that I got no errors running the script (I run it from the ArcGIS Pro IDLE) but no data is displaying on the "Data" tab from My Content / Waze Layers into ArcGIS Online... I don't know what else to do! Please, any one can help me???
I would really appreciate it!
This is a fantastic tutorial, thank you for sharing this!
Does anyone know of a way to modify the script so that it doesn't copy over duplicate rows every time the script is ran? I have two feature layers, one that holds all of the data, and is accumulated over time as the script is ran every 5 minutes, and another with a filter showing only the events that have been reported in the last 10 minutes. I am noticing that events are getting copied over multiple times and the data set is growing very fast. Thanks!
I was able to execute the python script, but I am getting the below error. Can you elaborate on how the script is supposed to access and edit the feature layer?
fail Stage 1cannot open 'https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer'
Ok, I figured this one out. I needed to use the individual feature service with its index number at the end. Like this:
https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureServer/0
Hello Gene, when I originally wrote this script it was for python 2.7 and it looks like from that error Noorhaizarinie Zaini gets is from trying to run it in python3. The parse function in python3 is different in some regard. you could try changing
to tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.fromstring</SPAN><SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">##URL for XML waze connection##'))</SPAN>tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>ElementTree<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit;">##URL for XML waze connection##')) </SPAN>
tree <SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN> ET<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.fromstring</SPAN><SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>file<SPAN class="" style="color: #a67f59; background: rgba(255, 255, 255, 0.5); border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">=</SPAN>urllib<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>request<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">.</SPAN>urlopen<SPAN class="" style="color: #999999; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">(</SPAN>'<SPAN class="" style="color: slategray; border: 0px; font-weight: inherit; margin: 0px; padding: 0px;">##URL for XML waze connection##'))</SPAN>
I am at the UC so I dont have a test environment to see if this works but I'm fairly certian it has to do with that line of code reading the xml from the url.
Hello Gee, I love this write up, thanks for sharing? Are you at the UC this year? If so I would love to meet up and see if you can help me quickly set this up. I have made my feature layers and altered the script, but I get the same error that Noorhaizarinie Zaini gets when executing it from Pro.
Thanks!
I got the same error. Did you get it solved?
Thanks Gee.
But could you please help me on this. I've run this script using ArcGIS Pro, but there are errors like the one below.
How to fix it?
Geethaka Fernando
https://community.esri.com/groups/departments-of-transportation/blog/2018/10/31/python-script-to-ingest-waze-data-to-arcgis-online
Thanks for sharing the code! BTW, is it the only way to use Waze data by becoming a CCP member?
Thanks Jonathan Pollack,
Have a look at this post - https://community.esri.com/groups/departments-of-transportation/blog/2019/02/04/traffic-study-reports-utilising-waze-data-and-arcgis-insights
Sometime in the near future, I'll update the script to ONLY ingest data into AGOL, when something changes in the XML feed.
Gee
Great job on this! Has anyone done anything to deal with duplicate records getting created as the data gets consumed and loaded into AGOL? Maybe filter by date or some other field when ingesting the data? Any ideas?
All good Joel.
In terms of error handling, the only thing I changed was - ability to handle missing elements and self closing tags.
For Example
<SPAN class="keyword token">if</SPAN> len<SPAN class="punctuation token">(</SPAN>member<SPAN class="punctuation token">.</SPAN>findall<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">></SPAN> <SPAN class="number token">0</SPAN> <SPAN class="operator token">and</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None<SPAN class="punctuation token">:</SPAN> Subtype <SPAN class="operator token">=</SPAN> member<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'linqmap:subtype'</SPAN><SPAN class="punctuation token">,</SPAN> namespaces<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>text<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'_'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">' '</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>capitalize<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN> Subtype <SPAN class="operator token">=</SPAN> <SPAN class="string token">''</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
And, I haven't come across any errors since changing this
Seeing people use this makes me wish I put some actual error handling in here
You need to have ArcGIS Pro or ArcMap installed to use the module arcpy
And if you intend on using ArcMap, you'll have to amend the syntax slightly to cater for Python 2.x
Alternatively, you could look into using the ArcGIS API for Python
Thanks, does it have to be ArcGIS Pro?
Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.