I have an API and I am trying to use the data in ArcGIS Online. Does anyone know of a python script that I can use to convert the response into a feature layer? Or some other way to do it?
Solved! Go to Solution.
So, like I'd mentioned, this is going to vary quite a bit based on the response you're getting from your API request. Mine in particular required this because the Census API returns the column names in the first row.
I see you're working in a notebook, which is good. I would first just check out what resonse.json(), or maybe response.text looks like.
Josh, I am trying to do something similar with the Census APIs, and this script was hugely helpful! Thank you! For some reason, the script throws an error when I get to the apply data types sections. I copied your script and ran it as is, and still get the error. Do you know why this might be? Below is the error I am getting:
File "<string>", line 47
df = df[list[df_dtypes.keys())].astype[df_dtypes]
^
SyntaxError: invalid syntax
@LindaSlattery here is an example how to update a hosted table in AGOL using Python and the ArcGIS for Python API with the data example you provided. The hosted table's field names match the same as the Data object in you sample. You should just have to update the parameters:
import requests, json
from arcgis import GIS
# Variables
username = "jskinner_CountySandbox" # AGOL Username
password = "********" # AGOL Password
itemID = 'f214df9b8f5440149a20ccd5d452a95e' # AGOL Table Item ID
dataURL = 'https://eagle-i.doe.gov/api/outagesummary/countymax24hoursummary?state=OH&county=Adams&eiApiKey=0000...'
# Disable warnings
requests.packages.urllib3.disable_warnings()
# Connect to AGOL
gis = GIS('https://www.arcgis.com', username, password)
# Get Table
fLayer = gis.content.get(itemID)
editTable = fLayer.tables[0]
# Get Data
params = {'f': 'pjson'}
r = requests.post(dataURL, data = params, verify=False)
response = json.loads(r.content)
data = response['data']
print(data)
# Create Dictionary of attributes
addFeatures = {
"attributes" : {
"currentOutage" : data[0]['currentOutage'],
"currentOutageRunStartTime" : data[0]['currentOutageRunStartTime'],
"maxOutage1" : data[0]['maxOutage1'],
"maxOutage1RunStartTime": data[0]['maxOutage1RunStartTime'],
"maxOutage24": data[0]['maxOutage24'],
"maxOutage24RunStartTime": data[0]['maxOutage24RunStartTime'],
"totalCustomers": data[0]['totalCustomers'],
"currentOutageHasOverrideData":data[0]['currentOutageHasOverrideData'],
"maxOutage24HasOverrideData": data[0]['maxOutage24HasOverrideData'],
"maxOutage1HasOverrideData": data[0]['maxOutage1HasOverrideData'],
"countyName": data[0]['countyName'],
"stateId": data[0]['stateId'],
"stateName": data[0]['stateName'],
"countyFIPSCode": data[0]['countyFIPSCode']
}
}
# Update Table
editTable.edit_features(adds=[addFeatures])
print("Finished")
Thanks, Jake! and i hate to sound like an idiot, but when you say I will have to update the parameters, is that in the "params = " row? How would that be different from the addFeatures section?
Thanks so much for your help, this is very confusing to me 🙂
@LindaSlattery you will just need to update the section at the top of the script under # Variables:
username = ArcGIS Online Username
password = ArcGIS Online Password
itemID = item ID of the ArcGIS Online hosted table. This can be found in the URL when you view the item details
dataURL = the URL to retrieve data from the API you're querying
After that, you should be able to execute the script.
OK, I did all of that and got the following errors. That's why I thought I needed to add something else in teh params row.
--------------------------------------------------------------------------- JSONDecodeError Traceback (most recent call last) <ipython-input-1-8a00425e9f5e> in <module> 21 params = {'f': 'pjson'} 22 r = requests.post(dataURL, data = params, verify=False) ---> 23 response = json.loads(r.content) 24 data = response['data'] 25 print(data) /opt/conda/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 346 parse_int is None and parse_float is None and 347 parse_constant is None and object_pairs_hook is None and not kw): --> 348 return _default_decoder.decode(s) 349 if cls is None: 350 cls = JSONDecoder /opt/conda/lib/python3.7/json/decoder.py in decode(self, s, _w) 335 336 """ --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end() 339 if end != len(s): /opt/conda/lib/python3.7/json/decoder.py in raw_decode(self, s, idx) 353 obj, end = self.scan_once(s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError("Expecting value", s, err.value) from None 356 return obj, end JSONDecodeError: Expecting value: line 1 column 1 (char 0)
OK, I did all of that and got the following errors.
--------------------------------------------------------------------------- JSONDecodeError Traceback (most recent call last) <ipython-input-1-8a00425e9f5e> in <module> 21 params = {'f': 'pjson'} 22 r = requests.post(dataURL, data = params, verify=False) ---> 23 response = json.loads(r.content) 24 data = response['data'] 25 print(data) /opt/conda/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 346 parse_int is None and parse_float is None and 347 parse_constant is None and object_pairs_hook is None and not kw): --> 348 return _default_decoder.decode(s) 349 if cls is None: 350 cls = JSONDecoder /opt/conda/lib/python3.7/json/decoder.py in decode(self, s, _w) 335 336 """ --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end() 339 if end != len(s): /opt/conda/lib/python3.7/json/decoder.py in raw_decode(self, s, idx) 353 obj, end = self.scan_once(s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError("Expecting value", s, err.value) from None 356 return obj, end JSONDecodeError: Expecting value: line 1 column 1 (char 0)
OK, so I did that and got the following error. Sorry for the screen shot, but whenever I copied and pasted I got an error in here. Just my lucky day with errors 🙂
@LindaSlattery did you update the dataURL variable with the correct URL including the token?
Yes, with the API key. I can put the URL into a browser window and the data gets returned, so the URL is correct.