Creating a Feature Layer from an API

[LindaSlattery]

I have an API and I am trying to use the data in ArcGIS Online. Does anyone know of a python script that I can use to convert the response into a feature layer? Or some other way to do it?

[jcarlson]

So, like I'd mentioned, this is going to vary quite a bit based on the response you're getting from your API request. Mine in particular required this because the Census API returns the column names in the first row.

I see you're working in a notebook, which is good. I would first just check out what resonse.json(), or maybe response.text looks like.

[KateNewell1]

Josh, I am trying to do something similar with the Census APIs, and this script was hugely helpful! Thank you! For some reason, the script throws an error when I get to the apply data types sections. I copied your script and ran it as is, and still get the error. Do you know why this might be? Below is the error I am getting:

File "<string>", line 47
df = df[list[df_dtypes.keys())].astype[df_dtypes]
^
SyntaxError: invalid syntax

[JakeSkinner]

@LindaSlattery here is an example how to update a hosted table in AGOL using Python and the ArcGIS for Python API with the data example you provided.  The hosted table's field names match the same as the Data object in you sample.  You should just have to update the parameters:

import requests, json
from arcgis import GIS

# Variables
username = "jskinner_CountySandbox"                 # AGOL Username
password = "********"                               # AGOL Password
itemID = 'f214df9b8f5440149a20ccd5d452a95e'         # AGOL Table Item ID
dataURL = 'https://eagle-i.doe.gov/api/outagesummary/countymax24hoursummary?state=OH&county=Adams&eiApiKey=0000...'

# Disable warnings
requests.packages.urllib3.disable_warnings()

# Connect to AGOL
gis = GIS('https://www.arcgis.com', username, password)

# Get Table
fLayer = gis.content.get(itemID)
editTable = fLayer.tables[0]

# Get Data
params = {'f': 'pjson'}
r = requests.post(dataURL, data = params, verify=False)
response = json.loads(r.content)
data = response['data']
print(data)

# Create Dictionary of attributes
addFeatures =  {
        "attributes" : {
            "currentOutage" : data[0]['currentOutage'],
            "currentOutageRunStartTime" : data[0]['currentOutageRunStartTime'],
            "maxOutage1" : data[0]['maxOutage1'],
            "maxOutage1RunStartTime": data[0]['maxOutage1RunStartTime'],
            "maxOutage24": data[0]['maxOutage24'],
            "maxOutage24RunStartTime": data[0]['maxOutage24RunStartTime'],
            "totalCustomers": data[0]['totalCustomers'],
            "currentOutageHasOverrideData":data[0]['currentOutageHasOverrideData'],
            "maxOutage24HasOverrideData": data[0]['maxOutage24HasOverrideData'],
            "maxOutage1HasOverrideData": data[0]['maxOutage1HasOverrideData'],
            "countyName": data[0]['countyName'],
            "stateId": data[0]['stateId'],
            "stateName": data[0]['stateName'],
            "countyFIPSCode": data[0]['countyFIPSCode']
        }
    }

# Update Table
editTable.edit_features(adds=[addFeatures])

print("Finished")

[LindaSlattery]

Thanks, Jake! and i hate to sound like an idiot, but when you say I will have to update the parameters, is that in the "params = " row? How would that be different from the addFeatures section?

Thanks so much for your help, this is very confusing to me 🙂

[JakeSkinner]

@LindaSlattery you will just need to update the section at the top of the script under  # Variables:

username = ArcGIS Online Username

password = ArcGIS Online Password

itemID = item ID of the ArcGIS Online hosted table.  This can be found in the URL when you view the item details

dataURL = the URL to retrieve data from the API you're querying

After that, you should be able to execute the script.

[LindaSlattery]

OK, I did all of that and got the following errors. That's why I thought I needed to add something else in teh params row.

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-1-8a00425e9f5e> in <module>
     21 params = {'f': 'pjson'}
     22 r = requests.post(dataURL, data = params, verify=False)
---> 23 response = json.loads(r.content)
     24 data = response['data']
     25 print(data)

/opt/conda/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

/opt/conda/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

/opt/conda/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[LindaSlattery]

OK, I did all of that and got the following errors. 

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-1-8a00425e9f5e> in <module>
     21 params = {'f': 'pjson'}
     22 r = requests.post(dataURL, data = params, verify=False)
---> 23 response = json.loads(r.content)
     24 data = response['data']
     25 print(data)

/opt/conda/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

/opt/conda/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

/opt/conda/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[LindaSlattery]

OK, so I did that and got the following error. Sorry for the screen shot, but whenever I copied and pasted I got an error in here. Just my lucky day with errors 🙂

[JakeSkinner]

@LindaSlattery did you update the dataURL variable with the correct URL including the token?

[LindaSlattery]

Yes, with the API key. I can put the URL into a browser window and the data gets returned, so the URL is correct.