AnsweredAssumed Answered

BeautifulSoup Detect Change Trigger

Question asked by jpilbeam Champion on Apr 4, 2020
Latest reply on Apr 7, 2020 by bixb0012

I have a script that uses bs4 to scrape a webpage and grab a string named, "Last Updated 4/3/2020, 8:28 p.m.". I then assign this string to a variable and send it in an email. The script is scheduled to run once a day. However, the date and time on the website change every other day. So, instead of emailing every time I run the script I'd like to set up a trigger so that it sends only when the date is different. How do I configure the script to detect that change?


'''Checks municipal websites for changes in meal assistance during C19 pandemic'''

# Import requests (to download the page)
import requests
# Import BeautifulSoup (to parse what we download)
from bs4 import BeautifulSoup
# Import win32com (Python module that controls Outlook)
import win32com.client
from win32com.client import Dispatch, constants
import re
import urllib
import urllib.request

#list of urls
urls = ['']

#set the headers like we are a browser
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

#download the homepage
response = requests.get(urls[0], headers=headers)
#parse the downloaded homepage and grab all text
soup = BeautifulSoup(response.text, "lxml")
#Find string
last_update_fra = soup.findAll(string=re.compile("Last Updated"))

#Put something here (if else..?) to trigger an email.

#I left off email block...


Here's where the string is in the HTML:

<h1>COVID-19 News Updates</h1>

<div><br></div><div>Last Updated 4/3/2020, 12:08 p.m.</div>