Send Email When a Feature is Added to an ArcGIS Online Hosted Feature Service

14997
140
06-02-2017 05:46 AM

Send Email When a Feature is Added to an ArcGIS Online Hosted Feature Service

With ArcGIS GeoEvent it is very easy to setup a notification when a new feature is added to an ArcGIS Online Hosted Feature Service.  Instructions can be found here.  If you do not have GeoEvent you can still accomplish this using Python.  Below is an example on how to do this. 

When editor tracking is enabled, new features are recorded in a field called 'created_date'.  The script queries this field within the service and compares the time to the current time (minus one hour).  If the time is greater than the current time, it will send an e-mail with a specified field's values for the new features.

Windows Task Scheduler can be setup to execute this script at a given interval (i.e. every hour).  This will report any new features within that last hour.

import requests, json, datetime, time, smtplib, math
from datetime import timedelta
from email.mime.text import MIMEText

# Disable warnings
requests.packages.urllib3.disable_warnings()

# Variables
username = 'agol' # AGOL Username
password = 'gis12345' # AGOL Password

URL = 'http://services.arcgis.com/dlFJXQQtlabc4qUk/arcgis/rest/services/PropertyDamage/FeatureServer/0/query' # Feature Service URL
uniqueID = 'OBJECTID' # i.e. OBJECTID
dateField = 'created_date' # Date field to query
hoursValue = 1 # Number of hours to check when a feature was added

fromEmail = 'user1@esri.com' # Email sender
toEmail = ['manager1@esri.com', 'manager2@esri.com'] # Email receiver(s)
smtpServer = 'smtp.gis.com' # SMPT Server Name
portNumber = 25 # SMTP Server port

# Create empty list for uniqueIDs
oidList = []

# Generate AGOL token
try:
print('Generating Token')
tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
r = requests.post(tokenURL, data=params, verify=False)
response = json.loads(r.content)
token = response['token']
except:
token = ''

# Return largest ObjectID
whereClause = '1=1'
params = {'where': whereClause, 'returnIdsOnly': 'true', 'token': token, 'f': 'json'}
r = requests.post(URL, data = params, verify = False)
response = json.loads(r.content)
try:
response['objectIds'].sort()
except Exception as e:
print("Error: {0}".format(e))

OIDs = response['objectIds']
count = len(response['objectIds'])
iteration = int(response['objectIds'][-1])
minOID = int(response['objectIds'][0]) - 1
OID = response['objectIdFieldName']

# Query service and check if created_date time is within the last hour
if count < 1000:
params = {'f': 'pjson', 'where': "1=1", 'outFields' : '{0}, {1}'.format(uniqueID, dateField), 'returnGeometry' : 'false', 'token' : token}
r = requests.post(URL, data=params, verify=False)
response = json.loads(r.content)
for feat in response['features']:
createDate = feat['attributes'][dateField]
createDate = int(str(createDate)[0:-3])
t = datetime.datetime.now() - timedelta(hours=hoursValue)
t = time.mktime(t.timetuple())
if createDate > t:
oidList.append(feat['attributes'][uniqueID])
else:
y = minOID
x = minOID + 1000

ids = response['objectIds']
newIteration = (math.ceil(iteration/1000.0) * 1000)
while y < newIteration:
if x > int(newIteration):
x = newIteration
where = OID + '>' + str(y) + ' AND ' + OID + '<=' + str(x)
print('Querying features with ObjectIDs from ' + str(y) + ' to ' + str(x))
params = {'f': 'pjson', 'where': where, 'outFields' : '{0}, {1}'.format(uniqueID, dateField), 'returnGeometry' : 'false', 'token' : token}
r = requests.post(URL, data=params, verify=False)
response = json.loads(r.content)
for feat in response['features']:
createDate = feat['attributes'][dateField]
createDate = int(str(createDate)[0:-3])
t = datetime.datetime.now() - timedelta(hours=hoursValue)
t = time.mktime(t.timetuple())
if createDate > t:
oidList.append(feat['attributes'][uniqueID])
x += 1000
y += 1000

print(oidList)

# Email Info
SUBJECT = 'New Features Added'
TEXT = "Features with {0}s {1} were added.".format(uniqueID, oidList)

# If new features exist, send email
if len(oidList) > 0:
smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)

msg = MIMEText(TEXT)
msg['Subject'] = SUBJECT
msg['From'] = fromEmail
msg['To'] = ", ".join(toEmail)

smtpObj.sendmail(fromEmail, toEmail, msg.as_string())
print("Successfully sent email")
smtpObj.quit()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
Comments

This is awesome to see! I have been working on a script of my own to do this based on fields other than creation date (it just fits the workflow better in our case) but I ended up getting access to the AGOL feature layer using the ArcRest module. If you are familiar with it I am curious if you would be able to explain to me the difference between using ArcRest and the method you've used in your own script.

I am very new to Python and coding In general so I am just trying to understand the mechanics of how you point the script to the feature layer here without using ArcRest.

Thank you so much for sharing this! I am sure it will be a life-saver for those of us without access to GeoEvent   

Hi Tiffany Taylor‌,

The ArcREST module simply uses the ArcGIS REST API, but in a more simplistic manner by leverage classes and function.  The above script is using the ArcGIS REST API, as well, to generate a token for ArcGIS Online, and to query the service. 

For example, ArcREST generates a token and queries a service in the following sample by importing AGOLTokenSecurityHandler and FeatureLayer classes.  If you were to dig into these classes, you will see similar code to what I have.

Thank you for the explanation! It definitely helped my understanding of how this works. 

Great script, but unfortunately i am receiving an error that i can not trouble shoot. I have updated all of the parameters to fit my agol sign in, i have included my feature service url, the only difference with my feature service url is that the ending does not contain a /0/Query at the end of the string. When i run my script i receive a KeyError: 'features' error. Any help is appreciated. I have included screen captures of the error. 

Snead‌ you will need to add the index number (i.e 0) for the layer you are polling followed by /query.  You can make sure the URL is correct by navigating to it.  You should see a page similar to the following:

Try the value ‘CreationDate’ for the variable dateField.

Thanks Jake and Bruce your comments, and they have helped move the error further down the code!

I have included a screen capture of my layer:

Here is the code with the error:

Appreciate your help, thanks. 

The error suggest there is no field named OBJECTID (this is set at line 9 "uniqueID").  The field names are listed on your layer screen when you scroll down.

Hi Jake,

Wonderful script, please can this be used for secure AGS feature services and not just hosted services?

Regards

Babatope Adebajo‌, yes you just need to update how the token is generated:

tokenURL = 'http://<server>/arcgis/admin/generateToken/'
params = {'f': 'pjson', 'username': username, 'password': password, 'client': 'requestip'}
req = urllib2.Request(tokenURL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
token = data['token']

Thanks Jake,

But i am having a Parse error on the 2nd line of this code, can it be with the 'client'?

Regards

Has anyone gotten this to work using outlook office 365?

Hi Jake,

This script is awesome. I was able to get it running in no time.

I was wondering if I could pull all the json down and configure it as html?

Any info would be great.

Thanks for all you do,

James Porter‌ you should be able to.  The query has an html value for the f parameter:

ArcGIS REST API 

@Jake Skinner

Thanks for providing this script and following up with issues, it's all been very helpful. 

One question I have is how to remove the 'u' from the beginning of the strings that are returned in the SMTP email. I do not want the OBJECTIDs to show up in the email so I am not targeting that list in the script. Instead, I'd like the email to return the addresses of the newly added features, which I managed to do. However the string that is returned has a 'u' in front of all strings that are listed in the email. Any idea on how to get rid of this?

Thanks,

Fred

fkapinos77‌ try adding str() in front of the variable you are returning.  Ex:  str(address)

Thanks Jake. I believe that is what I have as it is. Here is the whole thing, with sensitive parts redacted. I put it together piecemeal based on feedback in this thread as well as the previous thread. 

import urllib2, json, urllib, datetime, time, smtplib  
from datetime import timedelta 
 
Department = []
oidList = []
Street = []
Address = []


URL = 'REDACTED' 
params = {'f': 'pjson', 'where': "1=1", 'outfields' : 'OBJECTID, created_date, Department, Street, Address', 'returnGeometry' : 'false'} 
req = urllib2.Request(URL, urllib.urlencode(params)) 
response = urllib2.urlopen(req) 
data = json.load(response)
for feat in data['features']: 
    createDate = feat['attributes']['created_date'] 
    createDate = int(str(createDate)[0:-3]) 
    t = datetime.datetime.now() - timedelta(hours=24) 
    t = time.mktime(t.timetuple()) 
    if createDate > t: 
        Department.append(feat['attributes']['Department'])    
        oidList.append(feat['attributes']['OBJECTID'])
        Street.append(feat['attributes']['Street'])
        Address.append(feat['attributes']['Address'])

mergelist = zip(Address, Street, Department)

FROM = 'REDACTED' 
TO = ['REDACTED'] 

SUBJECT = 'New GIS Feature Added' 
TEXT = "Features with Addresses at" + " " + str(mergelist) + " " + "were added to the DPW GIS layer"

message =""" 'Subject: New GIS Feature Added'.format(SUBJECT, TEXT)
From: %s
To: %s
Subject: %s

%s
"""
% (FROM, ", ".join(TO), SUBJECT, TEXT)

gmailPwd = 'REDACTED'

if len(oidList) > 0:
    smtpObj = smtplib.SMTP(host='smtp.gmail.com', port= 587)
    smtpObj.starttls()
    smtpObj.login(FROM, gmailPwd)
    smtpObj.sendmail(FROM, TO, message) 
    print "Successfully sent email" 
    smtpObj.quit()
   
else:
    print "No new data found"
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I get this in the body of my email:

Thanks,

Fred

Try adding the str to the following:

if createDate > t:  
        Department.append(str(feat['attributes']['Department']))
        oidList.append(str(feat['attributes']['OBJECTID']))
        Street.append(str(feat['attributes']['Street']))
        Address.append(str(feat['attributes']['Address']))

Awesome Jake, that worked. Thanks. That was the last hurdle I had to implementing this. I'll post a more lengthy post on other issues I had that other people may find useful that aren't in this thread when I have a minute. 

I keep getting this error and am not sure if there is anything I can do to fix it.

Traceback (most recent call last):
  File "C:\Users\damona\Desktop\gmail test2.py", line 59, in <module>
    smtpObj = smtplib.SMTP(host='smtp.gmail.com', port= 465)
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 316, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 291, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "C:\Python27\ArcGIS10.5\lib\socket.py", line 575, in create_connection
    raise err
error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

Damon E.

I get a slightly different message when not behind our firewall

Traceback (most recent call last):
  File "C:\Users\damona\Desktop\gmail test2.py", line 59, in <module>
    smtpObj = smtplib.SMTP(host='smtp.gmail.com', port= 465)
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 317, in connect
    (code, msg) = self.getreply()
  File "C:\Python27\ArcGIS10.5\lib\smtplib.py", line 368, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
SMTPServerDisconnected: Connection unexpectedly closed

Actually, I do have another question. I am returning the text, but it is formatted with brackets and commas etc. ( This is in the email) 

I managed to format it using the following code, but it only returns 1 record. 

def newList():
    mergelist = zip(Address, Street, Department)
    for c1,c2,c3 in mergelist:
       return '{} {} {}\n'.format(c1, c2, c3)

FROM = 'REDACTED' 
TO = ['REDACTED'] 

SUBJECT = 'New GIS Feature Added' 
TEXT = "Features with Addresses at" + str(newList()) + " " + "\n\nwere added to the DPW GIS layer"
‍‍‍‍‍‍‍‍‍‍

Ideally, I'd like it do tabulate all the records that are appended to the lists in a readable format in the email:

62 Burford Avenue Water

26 Central Street Sewer

115 Elm Street Operations

I understand the formatting part, I just don't know why it isn't returning all the records in the list. Thanks again.

Thank you so much for sharing this, Jake! But for the life of me, I can't get it to work. I changed the variables appropriately (I think). I don't know how to troubleshoot since I don't have more information... The window prints this...

 

... and a bunch of text afterwards, and then closes a split second later. It closes so quickly that I haven't been able to print screen that text before the window disappears.

 

Any advice on how to move forward?

Oh, goodness. I think I'm getting the same error as Damon! I was able to get a bit more of the text before the box disappeared...

What did you end up doing, damoneisenach? Jake Skinner, any thoughts?

mangeles‌ can you share the service you are working with to a group in ArcGIS Online?  You can then invite my account (jskinner_CountySandbox) to this Group.  I can configure the above script to see if I can get it working and then share with you the results.

Thanks for your quick response, Jake! I really appreciate you taking your time to help resolve this.

Melinda Angeles‌ I was able to get this to work.  Here is the code I used:

import urllib2, json, urllib, datetime, time, smtplib
from datetime import timedelta

# Variables
username = 'jskinner_CountySandbox'        # AGOL Username
password = '******'    # AGOL Password

URL = 'https://services1.arcgis.com/vdNDkVykv9vEWFX4/arcgis/rest/services/address_errors/FeatureServer/0/qu...'       # Feature Service URL
uniqueID = 'OBJECTID'           # i.e. OBJECTID
dateField = 'CreationDate'      # Date field to query
hoursValue = 1                  # Number of hours to check when a feature was added

fromEmail = 'test@esri.com' # Email sender
toEmail = 'test@esri.com'   # Email receiver
smtpServer = 'smtp.gis.com'    # SMPT Server Name
portNumber = 25                 # SMTP Server port

# Create empty list for uniqueIDs
oidList = []

# Generate AGOL token
try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''

# Query service and check if created_date time is within the last hour
params = {'f': 'pjson', 'where': "1=1", 'outfields' : '{0}, {1}'.format(uniqueID, dateField), 'returnGeometry' : 'false', 'token' : token}
req = urllib2.Request(URL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
for feat in data['features']:
    createDate = feat['attributes'][dateField]
    createDate = int(str(createDate)[0:-3])
    t = datetime.datetime.now() - timedelta(hours=hoursValue)
    t = time.mktime(t.timetuple())
    if createDate > t:
        oidList.append(feat['attributes'][uniqueID])

print(oidList)

# Email Info
FROM = fromEmail
TO = [toEmail]
SUBJECT = 'New Features Added'
TEXT = "Features with {0}s {1} were added.".format(uniqueID, oidList)

message = """\
From: %s
To: %s
Subject: %s

%s
"""
% (FROM, ", ".join(TO), SUBJECT, TEXT)

# If new features exist, send email
if len(oidList) > 0:
    smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)
    smtpObj.sendmail(FROM, TO, message)
    print "Successfully sent email"
    smtpObj.quit()

I would check with IT to make sure you have the SMTP server correct.

Thanks, Jake! You are the definition of awesome!!

I'll post again if more questions come up.

So I guess I still have more questions 😛 Should I email you directly or keep this conversation here? Basically I'm getting the same errors but instead of [1,2,3] I'm getting [6,7]. And I'm assuming the same "Traceback"... errors. I've been using a gmail address so I believe I have the correct SMTP server. (both "smtp.gmail.com" and "mail.gmail.com" have been valid.)

I believe the issue is connecting to gmail's SMTP server.  I haven't had much luck using gmail.  You will need to authenticate with gmail with your e-mail/password.  Take a look at the following page:

How to Send Emails with Gmail using Python 

Are you able to use your company's SMTP server instead?

Did you ever get this to return multiple records? I am running into the same issue.

I wasn't able to get this going with multiple records.  I ended up running the script every minute and there are only 7 or 8 a day so the likelihood of multiple features being entered within that time period is very minimal (not to mention with this particular map, it's not apocalyptic if it does happen). I think in the future I will need to take another look at it. If you work it out in the meantime, please post it here. 

I did end up being able to send emails to 4 different people based on a Department field, and have multiple other fields appear formatted in the email. This script has been incredibly useful. 

Hi Jake,  Is it possible to use this script with Portal and IWA Authentication? I was having trouble figuring out how to do this with some of the documentation.

Will Farmer‌ yes, you will just need to update the token generation.  Below is example on how to generate a token using Portal with IWA:

import urllib, urllib2, json, ssl
import win32com.client, os

username = "user1@ESRI"
password = "gis12345"

tokenURL = 'https://portal.domain.com:7443/arcgis/sharing/rest/generateToken/'
params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'https://portal.domain.com'}
req = urllib2.Request(tokenURL, urllib.urlencode(params))
try:
    response = urllib2.urlopen(req)
except:
    gcontext = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
    response = urllib2.urlopen(req, context=gcontext)
data = json.load(response)
token = data['token']

I was browsing for a solution like this and know this convo is a little old but could use some help with this script; I've been troubleshooting for a day now but keep getting syntax errors of which I'm at a loss but suspect its related to the smtp server as well Does the script have a known problem for parsing against Outlook? My related script (copied directly from Jake's: thanks Jake!)  If someone could provide some insight, I'd appreciate it.

import urllib2, json, urllib, datetime, time, smtplib
from datetime import timedelta

# Variables
username = 'awhitley67'    # AGOL Username
password = 'xxxxxxxxxxxxx'    # AGOL Password

URL = 'https://services5.arcgis.com/ixWnd9c3gwqSeniI/arcgis/rest/services/RoadProblemReports/FeatureServer/...'  # Feature Service URL
uniqueID = 'OBJECTID'           # i.e. OBJECTID
dateField = 'CreationDate'      # Date field to query
hoursValue = 1                  # Number of hours to check when a feature was added

fromEmail = 'awhitley@peachtree-city.org' # Email sender
toEmail = 'awhitley@peachtree-city.org'   # Email receiver
smtpServer = 'smtp-mail.outlook.com'    # SMPT Server Name
portNumber = 587                 # SMTP Server port

# Create empty list for uniqueIDs
oidList = []

# Generate AGOL token
try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''

# Query service and check if created_date time is within the last hour
params = {'f': 'pjson', 'where': "1=1", 'outfields' : '{0}, {1}'.format(uniqueID, dateField), 'returnGeometry' : 'false', 'token' : token}
req = urllib2.Request(URL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
for feat in data['features']:
    createDate = feat['attributes'][dateField]
    createDate = int(str(createDate)[0:-3])
    t = datetime.datetime.now() - timedelta(hours=hoursValue)
    t = time.mktime(t.timetuple())
    if createDate > t:
        oidList.append(feat['attributes'][uniqueID])

print(oidList)

# Email Info
FROM = fromEmail
TO = [toEmail]
SUBJECT = 'New Features Added'
TEXT = "Features with {0}s {1} were added.".format(uniqueID, oidList)

message = """\
From: %s
To: %s
Subject: %s

%s
""" % (FROM, ", ".join(TO), SUBJECT, TEXT)

# If new features exist, send email
if len(oidList) > 0:
    smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)
    smtpObj.sendmail(FROM, TO, message)
    print "Successfully sent email"
    smtpObj.quit()

Anthony PTCGIS‌ what are the errors you are receiving?

Jake,

Thanks for replying and sorry for late reply on this. I couldn't get it to work with Outlook so I started from scratch and am now using a yahoo account. Oddly, starting fresh I get the following error;

Traceback (most recent call last):
  File "C:/Users/anthony/Documents/check_update.py", line 1, in <module>
    import urllib2, json, urllib, datetime, time, smtplib
ImportError: No module named 'urllib2'

Ah, that's a python 2 vs 3 error.

Ok, so now using Py 2 the error reads:

Traceback (most recent call last):
  File "C:\Users\twhitley\Documents\check_update.py", line 64, in <module>
    smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 311, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 286, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "C:\Python27\ArcGIS10.3\lib\socket.py", line 571, in create_connection
    raise err
error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

I've used smtp.yahoo.com and tested both ports 587 and 25

I assumed a proxy setting error so turned off proxy settings within Internet options. Still getting error:

Traceback (most recent call last):
  File "C:\Users\twhitley\Documents\check_update.py", line 66, in <module>
    smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 311, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "C:\Python27\ArcGIS10.3\lib\smtplib.py", line 286, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "C:\Python27\ArcGIS10.3\lib\socket.py", line 571, in create_connection
    raise err
error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

If anyone can shoot some ideas, it would be appreciated.

Great post.  Unfortunately I'm stuck with this error message 

Traceback (most recent call last):
File "X:/...", line 43, in <module>
for feat in data['features']:
KeyError: 'features'

Process finished with exit code 1

I believe I have the service defined correctly : URL ='https://services.arcgis.com/yygmGNIVQrHqSELP/arcgis/rest/services/HomelessCamp2019/FeatureServer/0/q...'

And the uniqueId in my case is OBJECTID_12

I have also tried both the time stamp field name as I see it : created_date and CreationDate.

Thank you in advance for your help!  

Code:

import urllib2, json, datetime, time, smtplib, urllib, sys
from datetime import timedelta
from smtplib import SMTP_SSL

#Varialbes
username =  'xxxxxx'  #AGOL UserName
password = 'xxxxxx'   #AGOL password

# Feature Service URL
URL ='https://services.arcgis.com/yygmGNIVQrHqSELP/arcgis/rest/services/HomelessCamp2019/FeatureServer/0/q...'

uniqueID = 'OBJECTID_12'           # i.e. OBJECTID
dateField = 'created_date'      # Date field to query
hoursValue = 24                  # Number of hours to check when a feature was added

fromEmail = 'xxxx@slocity.org' # Email sender
toEmail = 'xxxx@slocity.org'   # Email receiver
smtpServer = 'smtp.office365.com '    # SMPT Server Name
portNumber = 587                 # SMTP Server port
office365_user = 'xxxx@slocity.org'
office365_pwd= "xxxxx"

# Create empty list for uniqueIDs
oidList = []

# Generate AGOL token
try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''

# Query service and check if created_date time is within the last hour
params = {'f': 'pjson', 'where': "1=1", 'outfields' : '{0}, {1}'.format(uniqueID, dateField), 'returnGeometry' : 'false', 'token' : token}
req = urllib2.Request(URL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)
for feat in data['features']:
    createDate = feat['attributes'][dateField]
    createDate = int(str(createDate)[0:-3])
    t = datetime.datetime.now() - timedelta(hours=hoursValue)
    t = time.mktime(t.timetuple())
    if createDate > t:
        oidList.append(feat['attributes'][uniqueID])

print(oidList)

# Email Info
FROM = fromEmail
TO = [toEmail]
SUBJECT = 'New Features Added to Homeless Camps Map'
TEXT = "Features with {0}s {1} were added.".format(uniqueID, oidList)

message = """\
From: %s
To: %s
Subject: %s

%s
"""
% (FROM, ", ".join(TO), SUBJECT, TEXT)

# If new features exist, send email
if len(oidList) > 0:

    try:
        smtpObj = smtplib.SMTP(host=smtpServer, port=portNumber)
        smtpObj.ehlo()
        smtpObj.login(office365_user,office365_pwd)
        smtpObj.sendmail(FROM, TO, message)
        print "Successfully sent email"
        smtpObj.quit()

    except:
        sys.exit("email failed:")


					
				
			
			
			
				
			
			
			
			
			
			
		

mary andrews‌ can you share the service to an AGOL group and invite my user account (jskinner_CountySandbox)?

Thanks Jake, I sent you an invitation to our "Homeless Camp" Group. 

mary andrews‌ I could not reproduce this.  Could you add a print statement after the token generation portion to see if the token is being generated.  Ex:

# Generate AGOL token
try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''

print("Token: " + str(token))

Oh my,  you were right it was not generating a token because i miss typed my user name.  Silly me.

I finally got it to work for Office 360 by adding my username and password with these additional lines of code in the middle.  Jake, thank you for all the help and sharing this code!  

if len(oidList) > 0:
    smtpObj = smtplib.SMTP(host=smtpServer,port=portNumber)
    smtpObj.ehlo()
    smtpObj.starttls()
    smtpObj.ehlo
    smtpObj.login(office365_user, office365_pwd)
    smtpObj.sendmail(FROM, TO, message)
    print "Successfully sent email"
    smtpObj.close()

This is a great python script and I automated it for several of my feature services!  My co-worked asked me to write a script to email when the status of a field becomes a certain value.  I thought not problem it should be easy to modify this script to do just that.  Ha!  Apparently not.  I just cant get this code to work.  I finally have it running and returning "finished with exit code 0"  but it says  "No data with Status defined found"

when I know there are records meeting the criteria.   Any help would be much appreciated as I very new to python.  

import urllib2, json, smtplib, urllib, sys
from smtplib import SMTP_SSL



#Variables for your ArcGIS online account
username =  'xxx'  #AGOL UserName
password = 'xxx'   #AGOL password

# Feature Service URL
URL ='https://services.arcgis.com/yygmGNIVQrHqSELP/ArcGIS/rest/services/Hydrants_Survey_2019/FeatureServer...'


uniqueID = 'OBJECTID_1'           # i.e. OBJECTID
myBasin = 'BasinArea'
#CurrentStatus = "Status"


fromEmail = 'xx.org'  # Email sender
toEmailCC = 'xx.org'   # Email receiver
toEmail = 'xxslocity.org'   # Email receiver
smtpServer = 'smtp.office365.com'    # SMPT Server Name
portNumber = 587                 # SMTP Server port
#email password and user
office365_user = 'xx.org'
office365_pwd= 'xx!'


# Create empty list for uniqueIDs, etc.
oidList = []
BasinArea = []


# Generate AGOL token
try:
    print('Generating Token')
    tokenURL = 'https://www.arcgis.com/sharing/rest/generateToken'
    params = {'f': 'pjson', 'username': username, 'password': password, 'referer': 'http://www.arcgis.com'}
    req = urllib2.Request(tokenURL, urllib.urlencode(params))
    response = urllib2.urlopen(req)
    data = json.load(response)
    token = data['token']
except:
    token = ''

print("Token: " + str(token))

#params = {'f': 'pjson', 'where': "1=1",'outfields' : '{0}, {1}'.format(uniqueID, BasinArea), 'returnGeometry' : 'false', 'token' : token}
params = {'f': 'pjson', 'where': "1=1",'outfields' : 'OBJECTID_1, BasinArea', 'returnGeometry' : 'false', 'token' : token}
req = urllib2.Request(URL, urllib.urlencode(params))
response = urllib2.urlopen(req)
data = json.load(response)


for feat in data['features']:
    BasinName = feat['attributes'][myBasin]
    BasinName = 'Reservoir Canyon1'
    if BasinArea == BasinName:
        oidList.append(feat['attributes'][uniqueID])
        BasinArea.append(str(feat['attributes']['BasinArea']))


# Email Info

FROM = fromEmail
TO = [toEmail, toEmailCC]
SUBJECT = 'Hydrants Status Update'
TEXT ="Hydrant Status update(s) to the online map with the following: " + " " + str(BasinArea) + " " + "were added to the DPW GIS layer"
message = """ 'Subject: New GIS Feature Added'.format(SUBJECT, TEXT)
From: %s
To: %s
Subject: %s

%s
"""
% (FROM, ", ".join(TO), SUBJECT, TEXT)


# If  features with Status defined exist, send email
if len(oidList) > 0:
    smtpObj = smtplib.SMTP(host=smtpServer,port=portNumber)
    smtpObj.ehlo()
    smtpObj.starttls()
    smtpObj.ehlo
    smtpObj.login(office365_user, office365_pwd)
    smtpObj.sendmail(FROM, TO, message)
    print "Successfully sent email"
    smtpObj.close()
else:
    print "No data with Status defined found"

Working backwards:

- since you see the message, "No data with Status defined found", len(oidList) must be 0.

- oidList grows if BasinArea = BasinName, so this must never happen

- BasinName always = 'Reservoir Canyon1', while BasinArea always = [], so they're never equal

mary andrews I think you may want the following:

for feat in data['features']:
    BasinName = feat['attributes'][myBasin]
    if BasinName == 'Reservoir Canyon1'
        oidList.append(feat['attributes'][uniqueID])
        BasinArea.append(str(feat['attributes']['BasinArea']))

I'm attempting to implement your script into one of my work flows and received the error below. Any idea on how to resolve this?

For some reason, there's no key called 'features' in the data variable. Print data to see, then work your way up (printing response, req, etc.) to figure out what's going awry.

Version history
Revision #:
1 of 1
Last update:
‎06-02-2017 05:46 AM
Updated by:
 
Contributors