Python Script to ingest Waze data into ArcGIS Online

21034
46
10-31-2018 03:51 AM
by Anonymous User
Not applicable
17 46 21K

As part of the Waze Connected Citizen Program (CCP), partners gain access to real-time traffic congestions and incidents reported through the Waze app (in XML and JSON formats). There are already techniques and documentation on how to consume this data into an ArcGIS platform using GeoEvent.

In this blog, I will describe a technique for consuming this data into ArcGIS Online through Python and Windows Task Scheduler. The initial Python script was developed by Joel Meier which I modified to fit the workflow proposed. Also a special mention goes out to my colleagues Edmund Hall and Louis Tian who helped me debug. And last but not least I want to thank Andrew Stauffer‌ for helping us parter and get the most out of Waze CCP.

Creating ArcGIS Online feature layers to store Traffic information

Sign into ArcGIS Online > My Content page, click Create > Feature Layer, choose From URL and enter the exact URL of the point feature layer template. Afterwards, repeat the same process for the line feature layer. This will create new empty feature layers of the schema required (field names, types and spatial reference).

Point Feature Layer Template

https://services1.arcgis.com/E5n4f1VY84i0xSjy/arcgis/rest/services/WazeTrafficAlertsEmpty/FeatureSer...

Line Feature Layer Template

https://services1.arcgis.com/E5n4f1VY84i0xSjy/arcgis/rest/services/WazeTrafficJamsEmpty/FeatureServe...

Alternatively, visit https://developers.arcgis.com/layers/new to create Point and Line Geometry feature layers (WKID = 4326) with the following fields.

Spatial Reference

Point - Feature Layer

  • OBJECTID (type: esriFieldTypeOID, alias: OBJECTID, SQL Type: sqlTypeOther, length: 0, nullable: false, editable: false)*
  • DateReported (type: esriFieldTypeDate, alias: DateReported, SQL Type: sqlTypeTimestamp2, length: 0, nullable: true, editable: true)
  • Title (type: esriFieldTypeString, alias: Title, SQL Type: sqlTypeOther, length: 25, nullable: true, editable: true)
  • ReportType (type: esriFieldTypeString, alias: ReportType, SQL Type: sqlTypeOther, length: 25, nullable: true, editable: true)
  • ReportSubtype (type: esriFieldTypeString, alias: ReportSubtype, SQL Type: sqlTypeOther, length: 50, nullable: true, editable: true)
  • Street (type: esriFieldTypeString, alias: Street, SQL Type: sqlTypeOther, length: 256, nullable: true, editable: true)
  • City (type: esriFieldTypeString, alias: City, SQL Type: sqlTypeOther, length: 50, nullable: true, editable: true)
  • Description (type: esriFieldTypeString, alias: Description, SQL Type: sqlTypeOther, length: 1024, nullable: true, editable: true)
  • Reliability (type: esriFieldTypeInteger, alias: Reliability, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • UUID (type: esriFieldTypeString, alias: UUID, SQL Type: sqlTypeOther, length: 100, nullable: true, editable: true)
  • CreationDate (type: esriFieldTypeDate, alias: CreationDate, SQL Type: sqlTypeOther, length: 8, nullable: true, editable: false)*
  • Creator (type: esriFieldTypeString, alias: Creator, SQL Type: sqlTypeOther, length: 128, nullable: true, editable: false)*
  • EditDate (type: esriFieldTypeDate, alias: EditDate, SQL Type: sqlTypeOther, length: 8, nullable: true, editable: false)*
  • Editor (type: esriFieldTypeString, alias: Editor, SQL Type: sqlTypeOther, length: 128, nullable: true, editable: false)*

*pre-populated fields

Line - Feature Layer

  • OBJECTID (type: esriFieldTypeOID, alias: OBJECTID, SQL Type: sqlTypeOther, length: 0, nullable: false, editable: false)*
  • DateReported (type: esriFieldTypeDate, alias: DateReported, SQL Type: sqlTypeTimestamp2, length: 0, nullable: true, editable: true)
  • Title (type: esriFieldTypeString, alias: Title, SQL Type: sqlTypeOther, length: 25, nullable: true, editable: true)
  • ReportLevel (type: esriFieldTypeInteger, alias: ReportLevel, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • ReportType (type: esriFieldTypeString, alias: ReportType, SQL Type: sqlTypeOther, length: 25, nullable: true, editable: true)
  • Street (type: esriFieldTypeString, alias: Street, SQL Type: sqlTypeOther, length: 256, nullable: true, editable: true)
  • City (type: esriFieldTypeString, alias: City, SQL Type: sqlTypeOther, length: 50, nullable: true, editable: true)
  • Speed (type: esriFieldTypeDouble, alias: Speed, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • Delay (type: esriFieldTypeInteger, alias: Delay, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • Length (type: esriFieldTypeInteger, alias: Length, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • StartNode (type: esriFieldTypeString, alias: StartNode, SQL Type: sqlTypeOther, length: 256, nullable: true, editable: true)
  • EndNode (type: esriFieldTypeString, alias: EndNode, SQL Type: sqlTypeOther, length: 256, nullable: true, editable: true)
  • Description (type: esriFieldTypeString, alias: Description, SQL Type: sqlTypeOther, length: 1024, nullable: true, editable: true)
  • UUID (type: esriFieldTypeDouble, alias: UUID, SQL Type: sqlTypeOther, nullable: true, editable: true)
  • Shape__Length (type: esriFieldTypeDouble, alias: Shape__Length, SQL Type: sqlTypeDouble, nullable: true, editable: false)*
  • CreationDate (type: esriFieldTypeDate, alias: CreationDate, SQL Type: sqlTypeOther, length: 8, nullable: true, editable: false)*
  • Creator (type: esriFieldTypeString, alias: Creator, SQL Type: sqlTypeOther, length: 128, nullable: true, editable: false)*
  • EditDate (type: esriFieldTypeDate, alias: EditDate, SQL Type: sqlTypeOther, length: 8, nullable: true, editable: false)*
  • Editor (type: esriFieldTypeString, alias: Editor, SQL Type: sqlTypeOther, length: 128, nullable: true, editable: false)*

*pre-populated fields

Please make sure to use the exact field names and types, (alternatively you'd have to adjust the script).

Once feature layers are configured, please enable editing and set them to PRIVATE.

Please refer to "Waze Feed Technical Specifications [Get Waze Data]" from the Partner's Portal for exact definitions on field descriptions.

Configuring the Python script

Replace the comment sections (lines 6, 9 and 10) with your own XML feed, and URLs for your point and line feature layers.

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime
namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('##URL for XML waze connection##'))
root = tree.getroot()
channel_head = []
fcp = "##URL to Point - Feature Layer##"
fcl = "##URL to Line - Feature Layer##"
for member in root.findall('channel/item'):
 channel = []
 Title = member.find('title').text.upper()
 try:
 point = member.find('georss:point', namespaces)
 line = member.find('georss:line', namespaces)
 except:
 pass
 print('Fail Stage 0')
 if point != None:
 try:
 Date = member.find('pubDate').text
 DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

 if len(member.findall('linqmap:city', namespaces)) > 0:
 City = member.find('linqmap:city', namespaces).text
 else:
 City = ''

 if len(member.findall('linqmap:street', namespaces)) > 0:
 Street = member.find('linqmap:street', namespaces).text
 else:
 Street = ''

 if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
 Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
 else:
 Subtype = ''

 UUID = member.find('linqmap:uuid', namespaces).text
 Reliability = member.find('linqmap:reliability', namespaces).text

 if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
 Description = member.find('linqmap:reportDescription', namespaces).text
 else:
 Description = ''

 Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
 point = member.find('georss:point', namespaces).text.replace(' -',',-').split(' ')
 row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
 cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
 cursor.insertRow(row)
 del cursor
 print('stage 1')
 except:
 pass
 print('fail Stage 1')
 elif line != None:
 if Title != 'IRREGULARITY':
 try:
 Date = member.find('pubDate').text
 DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

 if len(member.findall('linqmap:city', namespaces)) > 0:
 City = member.find('linqmap:city', namespaces).text
 else:
 City = ''

 if len(member.findall('linqmap:startNode', namespaces)) > 0:
 StartNode = member.find('linqmap:startNode', namespaces).text
 else:
 StartNode = ''

 if len(member.findall('linqmap:endNode', namespaces)) > 0:
 EndNode = member.find('linqmap:endNode', namespaces).text
 else:
 EndNode = ''

 if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
 Description = member.find('linqmap:reportDescription', namespaces).text
 else:
 Description = ''

 if len(member.findall('linqmap:street', namespaces)) > 0:
 Street = member.find('linqmap:street', namespaces).text
 else:
 Street = ''

 UUID = member.find('linqmap:uuid', namespaces).text
 Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
 Length = member.find('linqmap:length', namespaces).text
 Level = member.find('linqmap:level', namespaces).text
 line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split(', ')
 Speed = member.find('linqmap:speed', namespaces).text
 Delay = member.find('linqmap:delay', namespaces).text
 cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
 array = arcpy.Array()
 for coords in line:
 xy = coords.split('],[')
 array.add(arcpy.Point(xy[1], xy[0]))
 cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
 del cur
 print('stage 2')
 except:
 pass
 print('fail Stage 2')
 else:
 try:
 Date = member.find('updateDate').text
 DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

 if len(member.findall('linqmap:city', namespaces)) > 0:
 City = member.find('linqmap:city', namespaces).text
 else:
 City = ''

 if len(member.findall('linqmap:startNode', namespaces)) > 0:
 StartNode = member.find('linqmap:startNode', namespaces).text
 else:
 StartNode = ''

 if len(member.findall('linqmap:endNode', namespaces)) > 0:
 EndNode = member.find('linqmap:endNode', namespaces).text
 else:
 EndNode = ''

 if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
 Description = member.find('linqmap:reportDescription', namespaces).text
 else:
 Description = ''

 if len(member.findall('linqmap:street', namespaces)) > 0:
 Street = member.find('linqmap:street', namespaces).text
 else:
 Street = ''

 UUID = member.find('linqmap:id', namespaces).text
 Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
 Length = member.find('linqmap:length', namespaces).text
 Level = member.find('linqmap:trend', namespaces).text
 line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split(', ')
 Speed = member.find('linqmap:speed', namespaces).text
 Delay = member.find('linqmap:delaySeconds', namespaces).text
 cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
 array = arcpy.Array()
 for coords in line:
 xy = coords.split('],[')
 array.add(arcpy.Point(xy[1], xy[0]))
 cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
 del cur
 print('stage 3')
 except:
 pass
 print('fail Stage 3')
 else:
 print('Error')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Scheduling the script

Once the Python script is up and running use Windows Task Scheduler to run the script at specific time intervals which will append new features to the ArcGIS Online feature layers (this technique will keep a history of all traffic information from when you first run the script).

Screenshot of traffic jams feature layer

Displaying Live Traffic Information

To display live traffic information - create feature layer views and define its features to only show features created in the last x number of minutes - x being how frequently you've setup the Python script to run.

I've setup the Python script to run every 4 minutes - subsequently the feature layer views to display features created in the last 4 minutes.

Define features to display live traffic information

Once setup, you can share these views publicly and display it on public facing applications.

Below are links to our feature layer views, and a map displaying these features alongside road/footpath closures.

*click here to find out more about our road/footpath work application process

Canberra Live Traffic

Email specific people when an Accident is reported

I used the following method, alternatively you could use similar Python scripts or create your own Python script to trigger an email when new features are added to a feature layer.

Run the email script of your choice on a new feature layer view (geometry type = point) that only displays Accidents (ReportType = "Accident"), where DateReported was in the last x number of minutes.

Define features to only display accidents for a specific time interval

And finally, setup windows task scheduler to capture any new features (accidents) that occur in the last x number of minutes.

I hope you can leverage these workflows to fit your own business needs. And please feel free to share your thoughts, comments or questions in the comments section below.


Cheers! - Gee Fernando

46 Comments
EricRodenberg
Esri Contributor

Nice write up...

by Anonymous User
Not applicable

I like this approach. What a great way to take advantage of all this data with such a small infrastructure investment. You could also change some of the formatting of the text in the alerts before pushing it into the feature service. For example changing "WEATHERHAZARD" to just "Hazard". I did this with GeoEvent Server but this this would be pretty simple to do in the python script. Here is an example we put together with GeoEvent Server for Indiana: Indiana Waze Operations Dashboard.

Great thanks for sharing this. Great job!

by Anonymous User
Not applicable

Thanks Eric Rodenberg

by Anonymous User
Not applicable

Thanks Tom Brenneman‌. Yeah, good pickup.

And thanks so much for sharing the dashboard. I love it

EricShreve
Occasional Contributor II

Nice article. I appreciate the time & effort that you put into this to make it work, as well as the script shareable. I just configured your code and attempted to run it and I am not getting past the 'Fail Stage 0' print function. Any clue on what I am missing? I have attached my code as well to cover up any loose ends.

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('https://na-georss.waze.com/rtserver/web/TGeoRSS?tk=ccp_partner&ccp_partner_name'))
root = tree.getroot()
channel_head = []
fcp = "https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/Waze_Data_Arizona/FeatureServer"
fcl = "https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/Waze_Data_Arizona_Line/FeatureSer..."
for member in root.findall('channel/item'):

    channel = []
Title = member.find('title').text.upper()
try:

    point = member.find('georss:point', namespaces)
    line = member.find('georss:line', namespaces)
except:

    pass
print('Fail Stage 0')
if point != None:

    try:
        Date = member.find('pubDate').text
        DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

        if len(member.findall('linqmap:city', namespaces)) > 0:
            City = member.find('linqmap:city', namespaces).text
        else:City = ''

        if len(member.findall('linqmap:street', namespaces)) > 0:
            Street = member.find('linqmap:street', namespaces).text
        else:Street = ''

        if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is str:
            Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
        else:Subtype = ''

        UUID = member.find('linqmap:uuid', namespaces).text
        Reliability = member.find('linqmap:reliability', namespaces).text

        if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
            Description = member.find('linqmap:reportDescription', namespaces).text
        else:Description = ''

        Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
        point = member.find('georss:point', namespaces).text.replace(' -',',-').split(' ')
        row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
        cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
        cursor.insertRow(row)
        del cursor
        print('stage 1')
        
    except:
    
        pass
    print('fail Stage 1')
elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')
                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''
                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''
                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''
                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''
                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''
                    
                    UUID = member.find('linqmap:uuid', namespaces).text
                    Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                    Length = member.find('linqmap:length', namespaces).text
                    Level = member.find('linqmap:level', namespaces).text
                    line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split(', ')
                    Speed = member.find('linqmap:speed', namespaces).text
                    Delay = member.find('linqmap:delay', namespaces).text
                    cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                    array = arcpy.Array()
                    for coords in line:
                        xy = coords.split('],[')
                        array.add(arcpy.Point(xy[1], xy[0]))
                        cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                        del cur
                        print('stage 2')
        
            except:
    
                pass
                print('fail Stage 2')
            else:
                try:
                    Date = member.find('updateDate').text
                    DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')
                    if len(member.findall('linqmap:city', namespaces)) > 0:
                        City = member.find('linqmap:city', namespaces).text
                    else:
                        City = ''
                    if len(member.findall('linqmap:startNode', namespaces)) > 0:
                        StartNode = member.find('linqmap:startNode', namespaces).text
                    else:
                        StartNode = ''
                    if len(member.findall('linqmap:endNode', namespaces)) > 0:
                        EndNode = member.find('linqmap:endNode', namespaces).text
                    else:
                        EndNode = ''
                    if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                        Description = member.find('linqmap:reportDescription', namespaces).text
                    else:
                        Description = ''
                    if len(member.findall('linqmap:street', namespaces)) > 0:
                        Street = member.find('linqmap:street', namespaces).text
                    else:
                        Street = ''
                        
                        UUID = member.find('linqmap:id', namespaces).text
                        Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                        Length = member.find('linqmap:length', namespaces).text
                        Level = member.find('linqmap:trend', namespaces).text
                        line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split(', ')
                        Speed = member.find('linqmap:speed', namespaces).text
                        Delay = member.find('linqmap:delaySeconds', namespaces).text
                        cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                        array = arcpy.Array()
                        for coords in line:
                            xy = coords.split('],[')
                            array.add(arcpy.Point(xy[1], xy[0]))
                            cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                            del cur
                            print('stage 3')
                except:
                    
                    pass
                    print('fail Stage 3')
                else:
                    print('Error')
MarkusSchlager1
Esri Contributor

Cédric Despierre Corporon‌, have you seen this very interesting blog post?

by Anonymous User
Not applicable

No worries. I've learnt so much from the GeoNet community.

And I'd like to help where I can in return

The URL to your XML feed seems incomplete - https://na-georss.waze.com/rtserver/web/TGeoRSS?tk=ccp_partner&ccp_partner_name

The feed to your XML URL should look like - https://world-georss.waze.com/rtserver/web/TGeoRSS?tk=?&ccp_partner_name=Test&polygon=34.582214,32.5...

Also, I noticed that your script included links to feature services rather than feature layers – e.g. https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/Waze_Data_Arizona_Line/FeatureSer...

 

I think in your case, adding a 0 to the end should fix this issue

https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/Waze_Data_Arizona_Line/FeatureSer...

And as long as the schemas of your feature layers are setup correctly; the script should run fine.

Let me know if you come across anymore issues.

Gee

EricShreve
Occasional Contributor II

Gee,

Thanks for the quick response. I formatted the code the way that you suggested: https://world-georss.waze.com/rtserver/web/TGeoRSS?tk=?&ccp_partner_name=My_Agency_Name&polygon=-114...

And I am getting an HTTP Status 503 - invalid token when I attempt to run the code in Python. When I add ccp_partner function similar to how the original URL is formatted the code works but I get the "Fail Stage 0", as well as "Fail Stage 3".

I also reproduced the same workflow that you mentioned when creating the Point & Line Feature Services. I am not sure what I am missing.

by Anonymous User
Not applicable

Hi Eric,

Can you make sure the the XML URL is the same as what's found on your Waze CCP Partner Portal > Get Waze Data, Waze XML Feed and the STATUS of the Waze XML Feed is Live.

Waze CCP Partner Portal > Get Waze Data, Waze XML Feed

Afterwards, make sure the XML feed is working properly by pasting it into Google Chrome. You may have to get back to Waze if the link is not working.

I'm also assuming that you're using Python from ArcGIS Pro to run the script ("C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe").

If that is the case, make sure that you're logged into ArcGIS Pro with the same credentials as the ArcGIS Online account (to ensure you have the right privileges to write into the feature layers).

If not, change the feature layers to public (feature layer overview > Share, Everyone), and run the script. *should only be performed in test scenarios

EricShreve
Occasional Contributor II

Hello Gee,

This is what my option looks like in Get Waze Data Portal. Do you think the "na-georss" vs "world-georss" could be impacting the way that the python script processes the XML Data?

Thank you. This is a tremendous script if I am able to get it working. 

by Anonymous User
Not applicable

Hi Eric‌,

Yes, this could be the reason.

I'd be more than happy to take a closer look (and come up with a solution) if you could email me your Waze XML Feed?

Gee

by Anonymous User
Not applicable

Hi again,

The issue was due to you guys living in the Northern Hemisphere 

Subsequently, the Python script wasn't passing lons and lats properly.

The modified script is attached below (I also emailed you this script updated with your own XML feed and feature layers).

Essentially anyone living in Northern America should be using this script rather than what is posted on the blog.

*If you're interested in the details please compare - lines 49, 93, 99, 141 & 147.

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime
namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('##URL for XML waze connection##'))
root = tree.getroot()
channel_head = []
fcp = "##URL to Point - Feature Layer##"
fcl = "##URL to Line - Feature Layer##"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')
            
            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''

            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''

            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''
                
            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text
            
            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''
            
            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''
                
                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''
                
                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Attached below is a screenshot of your area to demonstrate the script is working 

Waze Traffic in Arizona

OriGudes3
New Contributor II

Thanks, where do you actually run the code?

EricShreve
Occasional Contributor II

Gee,

That was the trick! I appreciate the quick response to fix the code. This is a tremendous script for users trying to get Waze data that do not have access to GeoEvent Server. Thanks Gee for making this available to the community!

-Eric

by Anonymous User
Not applicable

Hi Ori,

Use Python in ArcGIS Pro to run the code.

But, first copy the script into a text editor and save it with the extension .py

And if you have Python in ArcGIS Pro setup - Right click on that file > Run with ArcGIS Pro

Run Python code in ArcGIS Pro

by Anonymous User
Not applicable

No worries, glad I could help.

I also want to thank Joel Meier‌ for developing the initial python script.

Gee

by Anonymous User
Not applicable

Mark Chilcott‌ - this might interest you and the team

OriGudes3
New Contributor II

Thanks, does it have to be ArcGIS Pro?

by Anonymous User
Not applicable

You need to have ArcGIS Pro or ArcMap installed to use the module arcpy

And if you intend on using ArcMap, you'll have to amend the syntax slightly to cater for Python 2.x

Alternatively, you could look into using the ArcGIS API for Python

Gee

by Anonymous User
Not applicable

Seeing people use this makes me wish I put some actual error handling in here

by Anonymous User
Not applicable

All good Joel.

In terms of error handling, the only thing I changed was - ability to handle missing elements and self closing tags.

For Example

if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
    Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
else:
    Subtype = ''‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

And, I haven't come across any errors since changing this 

Gee

deleted-user-8TVt6oc0ewsX
Occasional Contributor

Great job on this!  Has anyone done anything to deal with duplicate records getting created as the data gets consumed and loaded into AGOL?  Maybe filter by date or some other field when ingesting the data?  Any ideas?

by Anonymous User
Not applicable

Thanks Jonathan Pollack‌,

Have a look at this post - https://community.esri.com/groups/departments-of-transportation/blog/2019/02/04/traffic-study-report...

Sometime in the near future, I'll update the script to ONLY ingest data into AGOL, when something changes in the XML feed.

Gee

NuozhouFu
New Contributor

Thanks for sharing the code! BTW, is it the only way to use Waze data by becoming a CCP member?

NoorhaizarinieZaini1
New Contributor

Thanks Gee.

But could you please help me on this. I've run this script using ArcGIS Pro, but there are errors like the one below.

How to fix it? 

@ Geethaka Fernando

 https://community.esri.com/groups/departments-of-transportation/blog/2018/10/31/python-script-to-ing...

Error when run with ArcGIS Pro

GeneSipes1
New Contributor III

I got the same error. Did you get it solved?

GeneSipes1
New Contributor III

Hello Gee, I love this write up, thanks for sharing? Are you at the UC this year? If so I would love to meet up and see if you can help me quickly set this up. I have made my feature layers and altered the script, but I get the same error that Noorhaizarinie Zaini gets when executing it from Pro. 

Thanks!

by Anonymous User
Not applicable

Hello Gene,  when I originally wrote this script it was for python 2.7 and it looks like from that error Noorhaizarinie Zaini gets is from trying to run it in python3. The parse function in python3 is different in some regard. you could try changing 

tree = ET.ElementTree(file=urllib.request.urlopen('##URL for XML waze connection##'))

to

tree = ET.fromstring(file=urllib.request.urlopen('##URL for XML waze connection##'))

I am at the UC so I dont have a test environment to see if this works but I'm fairly certian it has to do with that line of code reading the xml from the url.

GeneSipes1
New Contributor III

I was able to execute the python script, but I am getting the below error. Can you elaborate on how the script is supposed to access and edit the feature layer?

fail Stage 1
cannot open 'https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureS...'

Ok, I figured this one out. I needed to use the individual feature service with its index number at the end. Like this:

https://services1.arcgis.com/Hug9pbs2TYetbCha/arcgis/rest/services/Waze_Point_Feature_Layer/FeatureS...

GeneSipes1
New Contributor III

Does anyone know of a way to modify the script so that it doesn't copy over duplicate rows every time the script is ran? I have two feature layers, one that holds all of the data, and is accumulated over time as the script is ran every 5 minutes, and another with a filter showing only the events that have been reported in the last 10 minutes. I am noticing that events are getting copied over multiple times and the data set is growing very fast. Thanks!

PatriceCase1
New Contributor

This is a fantastic tutorial, thank you for sharing this!

ZairaRosas
New Contributor II

Hey! Can anyone help me?

I'm getting no error running the script, but it's kind of strange because I live in northen hemisphere but my XML data has the "world-georss" on the url, so I ran both possible scripts (the script on the post and the script with solution that Geethaka Fernando gave to Eric Shreve ) and both worked... I even tried with and witout the "/0" at the end of the layer url and BOTH worked hahaha... What I mean by saying that all of this worked is that I got no errors running the script (I run it from the ArcGIS Pro IDLE) but no data is displaying on the "Data" tab from My Content / Waze Layers into ArcGIS Online... I don't know what else to do! Please, any one can help me???

I would really appreciate it!

ErnestoCarreras2
New Contributor III

Geethaka Fernando‌ thanks for this amazing script!!! I have a question regarding the DateReported and Length data. In the Waze documentation, it says the data is collected in UTC and kmh. Is the processed data in this format? I am asking because if that is the case, I would like to convert the data to my local time zone, as well as kmh to mph. Thanks!

PaulRobinson1
New Contributor

Geethaka,

  I'm Executive Director of a non-profit organization that exists to accelerate innovation and business growth around coastal resilience solutions.We're offering up to $500,000 to create a next generation traffic app that offers drivers real-time flooding and re-routing information.

If you're interested in learning more, visit riseresilience.org for more details, or email paulrobinson@riseresilience.org 

Regards,

Paul Robinson

EmergencyManagement
New Contributor II

Hello Geethaka Fernando We followed your instructions with your provided python link. Had to fix some formatting errors as when it was pasted it didn't indent correctly. I'm running into the issue that this script, when ran in python versions 2.7.12 & 3.8.2 both can not find the library / module urllib.requests. Is there a specific version of python I need to download to get this to work? Thanks  

MarkCandland
New Contributor

I used this awesome article to get a nice dashboard up and running but after it running a couple months (and data piling up) I started running into performance issues. I thought it was the amount of records, so I created a new feed to test. That feed with far fewer records is having the same issue.

Anyone else experiencing some draw speed issues? it seems like the data is updating quickly (in my lists) but the map updates painfully slow.

Thanks

Mark

TrafficOpsGIS
New Contributor II

Does this workflow still work? Thank you

TrafficOpsGIS
New Contributor II

Can confirm that with some formatting for Python3.0 that this workflow does indeed still work! Thank you all for all your shared information!

MichelleNino
New Contributor

@TrafficOpsGIS Hello! I would like to know if you've tried to run this script with python3.0? Or have you modified this code according to your needs?

Thank you! 

Michelle

RosyChawla2
New Contributor

Hi All,

This script is exactly what we need at our county to be able to display data we get from Waze Connected Citizen Program. After proper indentation I tried running it with ArcPy 3.0 as well as ArcPy 2.7 Unfortunately I was not able to get it going.

Will appreciate  any help.

Thanks,

-Rosy

TrafficOpsGIS
New Contributor II

FinalFinalpt3pt3pt2pt2pt1pt1

@RosyChawla2 , @MichelleNino Here is the script I use. See images 1-4 for complete script. Hope you are able to replicate. I run it directly in ArcPro Notebook. Please let me know if you have any questions on.

daniel_marincounty
New Contributor II

The links to create the feature layers no longer work. Are there new ones?

daniel_marincounty
New Contributor II

I was able to copy @TrafficOpsGIS script into my own notebook and create the layers using the manual directions. However, it doesn't work when I run it in a notebook for some reason. If I run it as a .py it works 🙂 feel free to copy 

 

 

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('##URL'))

root = tree.getroot()
channel_head = []
fcp = "##URL"
fcl = "##URL"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')
                
                    

 

 

 

JasonSimpson
New Contributor III

Can someone help me? I've had a waze feed running consistently for a couple years now, but the feed suddenly stopped in july. After researching the issue, Ive noticed that a URL feed has changed. After making the proper changes, I am still receiving an error when running the script in PRO. Below is the script. 

 

import xml.etree.ElementTree as ET
import urllib.request
import arcpy
from datetime import datetime

namespaces = {'georss' : 'http://www.georss.org/georss', 'linqmap':'http://www.linqmap.com' }
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/1103703####/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4###?format=1'))

root = tree.getroot()
channel_head = []
fcp = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Points/FeatureServer/0"
fcl = "https://services7.arcgis.com/RL8aZgZHmJOdyN##/arcgis/rest/services/Waze_Lines/FeatureServer/0"
for member in root.findall('channel/item'):
    channel = []
    Title = member.find('title').text.upper()
    try:
        point = member.find('georss:point', namespaces)
        line = member.find('georss:line', namespaces)
    except:
        pass
        print('Fail Stage 0')
    if point != None:
        try:
            Date = member.find('pubDate').text
            DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

            if len(member.findall('linqmap:city', namespaces)) > 0:
                City = member.find('linqmap:city', namespaces).text
            else:
                City = ''
                
            if len(member.findall('linqmap:street', namespaces)) > 0:
                Street = member.find('linqmap:street', namespaces).text
            else:
                Street = ''
                
            if len(member.findall('linqmap:subtype', namespaces)) > 0 and member.find('linqmap:subtype', namespaces).text is not None:
                Subtype = member.find('linqmap:subtype', namespaces).text.replace('_', ' ').capitalize()
            else:
                Subtype = ''

            UUID = member.find('linqmap:uuid', namespaces).text
            Reliability = member.find('linqmap:reliability', namespaces).text

            if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                Description = member.find('linqmap:reportDescription', namespaces).text
            else:
                Description = ''

            Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
            #point = member.find('georss:point', namespaces).text.replace(' -', ',-').split(',')
            point = member.find('georss:point', namespaces).text.replace(' -',',-').split(',')
            row = Subtype, Type, Title, DateReported, City, Street, UUID, Reliability, Description, float(point[1]),float(point[0])
            cursor = arcpy.da.InsertCursor(fcp, ["ReportSubtype","ReportType","Title","DateReported","City","Street","UUID","Reliability","Description","SHAPE@X","SHAPE@Y"])
            cursor.insertRow(row)
            del cursor
            print('stage 1')
        except:
            pass
            print('fail Stage 1')
    elif line != None:
        if Title != 'IRREGULARITY':
            try:
                Date = member.find('pubDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:uuid', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:level', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delay', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 2')
            except:
                pass
                print('fail Stage 2')
        else:
            try:
                Date = member.find('updateDate').text
                DateReported = datetime.strptime(Date, '%a %b %d %H:%M:%S %z %Y')

                if len(member.findall('linqmap:city', namespaces)) > 0:
                    City = member.find('linqmap:city', namespaces).text
                else:
                    City = ''

                if len(member.findall('linqmap:startNode', namespaces)) > 0:
                    StartNode = member.find('linqmap:startNode', namespaces).text
                else:
                    StartNode = ''

                if len(member.findall('linqmap:endNode', namespaces)) > 0:
                    EndNode = member.find('linqmap:endNode', namespaces).text
                else:
                    EndNode = ''

                if len(member.findall('linqmap:reportDescription', namespaces)) > 0:
                    Description = member.find('linqmap:reportDescription', namespaces).text
                else:
                    Description = ''

                if len(member.findall('linqmap:street', namespaces)) > 0:
                    Street = member.find('linqmap:street', namespaces).text
                else:
                    Street = ''

                UUID = member.find('linqmap:id', namespaces).text
                Type = member.find('linqmap:type', namespaces).text.replace('_', ' ').capitalize()
                Length = member.find('linqmap:length', namespaces).text
                Level = member.find('linqmap:trend', namespaces).text
                line = member.find('georss:line', namespaces).text.replace('\n', '').replace(' ', '],[').replace('],[-',', -').split('],[')
                Speed = member.find('linqmap:speed', namespaces).text
                Delay = member.find('linqmap:delaySeconds', namespaces).text
                cur = arcpy.da.InsertCursor(fcl, ["Speed","Delay","Title","ReportType","DateReported","City","Street","Length","ReportLevel","UUID","StartNode","EndNode","Description","SHAPE@"])
                array = arcpy.Array()
                for coords in line:
                    xy = coords.split(', ')
                    array.add(arcpy.Point(xy[1], xy[0]))  
                cur.insertRow([Speed,Delay,Title,Type,DateReported,City,Street,Length,Level,UUID,StartNode,EndNode,Description,arcpy.Polyline(array)])
                del cur
                print('stage 3')
            except:
                pass
                print('fail Stage 3')
    else:
        print('Error')

 

And this is the error I am receiving when ran in PRO

Script2
=====================
Parameters

=====================
Messages

Start Time: Monday, August 28, 2023 9:46:17 AM
Traceback (most recent call last):
File "C:\Users\jsimpson\Desktop\WazeAug23.py", line 7, in <module>
tree = ET.ElementTree(file=urllib.request.urlopen('https://www.waze.com/partnerhub-api/partners/11037037395/waze-feeds/5e9eb202-2341-46d9-ad87-ee7c0ca4...'))
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 540, in __init__
self.parse(file)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\xml\etree\ElementTree.py", line 580, in parse
self._root = parser._parse_whole(source)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 0

Failed script Script2...
Failed to execute (Script2).
Failed at Monday, August 28, 2023 9:46:18 AM (Elapsed Time: 0.91 seconds)

 

 

Any help is appreciated. 

 

TrafficOpsGIS
New Contributor II

@JasonSimpson Waze changed their Feed URL format a few months back. Here you may find details on new format which you will need to upload your new format version on the Waze Partner Hub.

  • Please log in to the Partner Hub and use the new links for your feeds under “Get data from Waze” > “Waze data feed” (see image below)
  • The old URLs will be deprecated by end of May 2023

 

TrafficOpsGIS_0-1702394200209.jpeg

 

Screenshot 2023-03-14 at 13.23.532470×1604 226 KB

 

If you’re not sure, here are examples of how the URLs look like:

Old URLs:

  1. https://www.waze.com/il-rtserver/web/TGeoRSS?tk=ccp_partner&ccp_partner_name=testPartner&bottom=32.0...
  2. https://www.waze.com/partnerhub-api/waze-feed-access-token/845d2144-bf82-451c-98cd-2176ad57c2ba?form...

New URL:

https://www.waze.com/partnerhub-api/partners/11697892328/waze-feeds/58c6486b-430e-4835-83a2-8a6e1259...

TrafficOpsGIS
New Contributor II

@daniel_marincounty Nice work, thank you for sharing.