Creating a Feature Layer from an API

LindaSlattery · ‎09-10-2021

I have an API and I am trying to use the data in ArcGIS Online. Does anyone know of a python script that I can use to convert the response into a feature layer? Or some other way to do it?

LindaSlattery · ‎09-13-2021

Outside of my username, password and URL, the only other thing that gets changed is the item ID. I created a blank table in Pro, added one row, and shared up to my AGOL. I made sure it was editable, and even made it public, just in case that was an issue. Could there be something up with the table? Here is the URL, if you want to look at it.

https://services6.arcgis.com/zxOMWqh0yAD6mMsJ/arcgis/rest/services/power_outages_county/FeatureServe...

JakeSkinner · ‎09-14-2021

@LindaSlattery I found the issue. The request needs to be a get rather than a post.

Here is the updated code:

import requests, json
from arcgis import GIS

# Variables
username = "jskinner_CountySandbox"                 # AGOL Username
password = "********"                               # AGOL Password
itemID = 'f214df9b8f5440149a20ccd5d452a95e'         # AGOL Table Item ID
dataURL = 'https://eagle-i.doe.gov/api/outagesummary/countymax24hoursummary?state=OH&county=Adams&eiApiKey=0000...'

# Disable warnings
requests.packages.urllib3.disable_warnings()

# Connect to AGOL
gis = GIS('https://www.arcgis.com', username, password)

# Get Table
fLayer = gis.content.get(itemID)
editTable = fLayer.tables[0]

# Get Data
r = requests.get(dataURL, data = params, verify=False)
response = json.loads(r.content)
data = response['data']
print(data)

# Create Dictionary of attributes
addFeatures =  {
        "attributes" : {
            "currentOutage" : data[0]['currentOutage'],
            "currentOutageRunStartTime" : data[0]['currentOutageRunStartTime'],
            "maxOutage1" : data[0]['maxOutage1'],
            "maxOutage1RunStartTime": data[0]['maxOutage1RunStartTime'],
            "maxOutage24": data[0]['maxOutage24'],
            "maxOutage24RunStartTime": data[0]['maxOutage24RunStartTime'],
            "totalCustomers": data[0]['totalCustomers'],
            "currentOutageHasOverrideData":data[0]['currentOutageHasOverrideData'],
            "maxOutage24HasOverrideData": data[0]['maxOutage24HasOverrideData'],
            "maxOutage1HasOverrideData": data[0]['maxOutage1HasOverrideData'],
            "countyName": data[0]['countyName'],
            "stateId": data[0]['stateId'],
            "stateName": data[0]['stateName'],
            "countyFIPSCode": data[0]['countyFIPSCode']
        }
    }

# Update Table
editTable.edit_features(adds=[addFeatures])

print("Finished")

LindaSlattery · ‎09-14-2021

Very, very close now. I got an error but realized it was missing the params = {'f': 'pjson'} code under Get Data. It runs now but does not update the table. All of the data prints in the Notebook.

JakeSkinner · ‎09-14-2021

@LindaSlattery you should not need the params = {'f':'pjson'}. The issue appears to be the type of field for totalCustomers. This is set to Small Integer:

This data type is limited to only a small range of values. I would recommend converting all fields from Small Integer to Long Integer, then republish the feature service.

LindaSlattery · ‎09-14-2021

I really appreciate you giving me all of this help. If I remove the params = {'f':'pjson'} line, I get the following error:

22 # Get Data
---> 23 r = requests.get(dataURL, data = params, verify=False)
24 response = json.loads(r.content)
25 data = response['data']

NameError: name 'params' is not defined

I added the line back in and changed the field type and it runs now, but only updates the first row of data. All of the rows appear in the notebook from the print command, but only the first record appends in the table. The response gives a row for each county.

JakeSkinner · ‎09-17-2021

@LindaSlattery here is the updated code that will first perform a truncate of the hosted table, and then update the table with all records.

import requests, json, arcpy
from arcgis import GIS

# Variables
username = "jskinner_CountySandbox"                 # AGOL Username
password = "********"                               # AGOL Password
itemID = 'f214df9b8f5440149a20ccd5d452a95e'         # AGOL Table Item ID
dataURL = 'https://eagle-i.doe.gov/api/outagesummary/countymax24hoursummary?state=OH&eiApiKey='

# Disable warnings
requests.packages.urllib3.disable_warnings()

# Connect to AGOL
print("Connecting to AGOL")
gis = GIS('https://www.arcgis.com', username, password)

# Get Table
print("Get Hosted Table")
fLayer = gis.content.get(itemID)
editTable = fLayer.tables[0]

# Truncate Table
print("Truncate table")
editTable.manager.truncate()

# Get Data
print("Retrieving Data")
r = requests.get(dataURL, verify=False)
response = json.loads(r.content)
data = response['data']

# Create Dictionary of attributes and update table
print("Updating hosted table")
for attr in data:
    addFeatures =  {
            "attributes" : {
                "currentOutage" : attr['currentOutage'],
                "currentOutageRunStartTime" : attr['currentOutageRunStartTime'],
                "maxOutage1" : attr['maxOutage1'],
                "maxOutage1RunStartTime": attr['maxOutage1RunStartTime'],
                "maxOutage24": attr['maxOutage24'],
                "maxOutage24RunStartTime": attr['maxOutage24RunStartTime'],
                "totalCustomers": attr['totalCustomers'],
                "currentOutageHasOverrideattr":attr['currentOutageHasOverrideData'],
                "maxOutage24HasOverrideData": attr['maxOutage24HasOverrideData'],
                "maxOutage1HasOverrideData": attr['maxOutage1HasOverrideData'],
                "countyName": attr['countyName'],
                "stateId": attr['stateId'],
                "stateName": attr['stateName'],
                "countyFIPSCode": attr['countyFIPSCode']
            }
        }

    # Update Table
    editTable.edit_features(adds=[addFeatures])

print("Finished")

LindaSlattery · ‎09-20-2021

Thank you so much, Jake! You are the best :). However, I did get an error with the arcpy in the first row, but it worked perfectly when I took out arcpy. I set up my scheduler and will be testing it out throughout the day before we deploy it for the watch office. Your help is much appreciated!

ArmstKP · ‎01-11-2023

@LindaSlattery

Thank you Linda for your example. I have a question for you. I used your example and receive this error. Would you happen to know why?

Kareem · ‎05-05-2022

Hi,
I have approached the same problem while building a web app that uses python requests to query a 3rd party API to get the building parcel polygon when the app user types his address, but I got 3 issues:

1- when I use something like the following line, I get the error " data doesn't have an address" and it doesn't cause the 'shape column contains a polygon.

df.spatial.to_featureclass('block_groups_19_kendall')

2- if I am successful to transform the data frame into a feature class which functions to use to plot it on the map?

3- which function to use to link the user address typed in the search bar widget as an attribute in my requests query?

or what function or module in python API equals the following line in python:

user_address = str(SearchBarWidget.Input_value)

I know this is a lot to answer I am kinda new to ArcGIS, and you can just direct me where to search or mention the function name and I will do my best,

Thank you in advance