Context: Our organization has developed a python script in Jupyter Notebooks that scrapes an electricity providers website and compares the power outages with some of our at risk residents' addresses. If there is a power outage then it uses an email function to alert staff of those power outages. We want to schedule this script as a stand-alone script on a local server with task scheduler to run every hour.
^^^ Since there are currently no power outages, I updated the script to email us all those residences that have power on. (Should be all of our at risk population). This works beautifully in Jupyter Notebooks.
Issue: When I export the notebook as a stand-alone python script and run it in PyCharm, it doesn't provide the same results. During the test for those with power on it shows zero and we do not get sent an email.
Example results in Jupyter Notebooks creating a list: (the list is called "off_records" but it's really grabbing "Power_Status" == "ON" for testing purposes)
This is the result of the same exact script using PyCharm:
Help! Any reason why these results would be different?
Thanks for your time,
Shari
It's apprent that from PyCharm, the variable "match" is not being evaluated as true. Unfortunately, we cannot really debug the cariable since its referencing something that is outside of what you provided.
Add more print statements to help you debug why your "if match" conditions is not being met.
Thanks for pointing that out. I'm not sure how to share this script with you to see it all as it has sensitive data within it.
Understandable. Unfortunately, without much more context, it's going to be difficult to offer any more advice beyond adding in more print statements for debugging. And double check the request you are making to the service you're scraping for data to ensure that it hasn't been altered from the notebook.
The layer with the sensitive info is locked not shared publicly but here's the script anyway. If something stands out to you please let me know.
"""Script checks Oncor API and compares with an existing layer of COR STEAR residents.
Scrapes Oncor website that includes power status. If power status is off then an email is sent to oem.
Written in AGOL Notebooks by Ryan Steel. Copied from there and edited by Shari on 08/14/2025.
Python 3"""
from arcgis.gis import GIS
import requests
from tqdm import trange
import smtplib
from datetime import datetime
def emailAlert(toList, subject, body):
email_user = 'noreplay@cor.gov'
message = 'Subject: {}\n\n{}'.format(subject, body)
server = smtplib.SMTP('outbound.cor.gov', 25)
server.sendmail(email_user, toList, message)
server.close()
try:
start_time = time.time()
# 1. Connect to your GIS
gis = GIS("home")
# 2. Connect to feature layer
print("Connecting to feature layer...")
layer_item = gis.content.get("c8018b05abc74086a18c95ff1bd77657")
layer = layer_item.layers[0]
# 3. Query all features (return all fields)
print("Querying existing features...")
features = layer.query(
where="1=1",
out_fields="*",
return_geometry=False
).features
print(features)
# 4 and 5 Create mapping of ESI to FID and Get list of ESIs from the layer
esi_to_fid = {
str(f.attributes["ESI"]).strip(): f.attributes["FID"]
for f in features
if f.attributes["ESI"] is not None
}
esi_list = list(esi_to_fid.keys())
print(esi_list)
# 6. Fetch power status and prepare updates
print("Fetching power status and preparing updates...")
updates = []
errors = []
current_date = datetime.now().strftime('%Y-%m-%d') # Get the current date in the desired format
for esi in trange(222, desc="Checking ESI power status"):
try:
url = f"https://www.oncor.com/outages/api/customerOutage/outage/checkStatus?esiid={esi}&source=ORS"
response = requests.get(url)
response.raise_for_status()
data = response.json()
# ✅ Correct JSON parsing
api_status = (
data.get("getOutageStatusResponse", {})
.get("powerStatus", {})
.get("status", "")
.strip()
.upper()
)
final_status = "OFF" if api_status == "OFF" else "ON"
fid = esi_to_fid.get(esi)
if fid is not None:
updates.append({
"attributes": {
"FID": fid,
"Power_Status": final_status,
"Last_Updated": current_date # Update the Last_Updated field
}
})
else:
errors.append((esi, "FID not found"))
except Exception as e:
errors.append((esi, str(e)))
# 7. Apply updates to feature layer
print(f"Updating {len(updates)} features in the layer...")
if updates:
result = layer.edit_features(updates=updates)
print("Update complete.")
else:
print("No updates to apply.")
# 8. Compile OFF records for email
print("Compiling OFF records for notification...")
off_records = []
for update in updates:
if update["attributes"]["Power_Status"] == "OFF":
fid = update["attributes"]["FID"]
match = next((f for f in features if f.attributes["FID"] == fid), None)
if match:
attr = match.attributes
full_name = f"{attr.get('First_Name', '')} {attr.get('Last_Name', '')}".strip()
address = f"{attr.get('Street_Address_1', '')} {attr.get('Apt_Suite_Unit_1', '')}, {attr.get('City_1', '')}, {attr.get('State_Province_1', '')} {attr.get('Postal_Code_1', '')}".strip()
phone = attr.get("Phone_1", "N/A")
off_records.append(f"Name: {full_name}\nAddress: {address}\nPhone: {phone}\n")
print(off_records)
# 9. Send email if any power OFF cases
# Convert off_records to a single string for the email body
email_body = "\n\n".join(off_records)
# Specify the recipients of the email
to_list = ['shari.forbes@cor.gov', 'ryan.steele@cor.gov'] # Replace with actual recipient emails
subject = f"Power OFF Notifications - {datetime.now().strftime('%Y-%m-%d')}"
# Send the email if there are any off_records
if off_records:
print("Sending email notification...")
emailAlert(to_list, subject, email_body)
print("Email sent.")
else:
print("No power OFF records to notify.")
end_time = time.time()
elapsed_time = ((end_time - start_time) / 60)
elapsed_timeR = round(elapsed_time, 2)
print("SUCCESS!!")
# Send success email alert
emailAlert(['shari.forbes'],
subject = 'STEAR Power Status python code success',
body = 'STEAR Power STatus python code ran successfully in ' + str(elapsed_timeR) + ' minutes.\n yay!')
except Exception as e:
print("Error: " + e.args[0])
So, i think the issue is with line 104:
match = next((f for f in features if f.attributes["FID"] == FID, None))
My guess would be that you are not receiving any matches in this variable assignment, and therefor match is always evaluating to None and not executing your `if match:` condition. You could add a print statement after line 104 to to get what data is being assigned to the match variable and dive into why it isn't finding the match in your features data. You may also want to verify your data is matching correctly for FID - as in that they are both the same data type (string to string, int to int, etc.)
THank you. I'll look into it.