Pandas data frame to feature layer - empty results

2053
2
Jump to solution
08-25-2020 04:36 PM
YvetteBevis
Esri Contributor

I am attempting to scrape data from a website and publish it as a feature layer to ArcGIS Online. I have been using the sample notebook (HTML table to Pandas Data Frame to Portal Item | ArcGIS for Developers ) as guidance for the pandas data frame to portal item bit. 

my problem is that after importing the data frame the resulting feature collection is empty:

Can anyone help me understand why it's empty? Is there maybe a characteristic of the data frame that it doesn't like? Is there a small detail I've overlooked?

Thanks,

Yvette

0 Kudos
1 Solution

Accepted Solutions
by Anonymous User
Not applicable

Hi Yvette Bevis‌,

Pretty sure there just isn't enough info in the table for the arcgis python API to know how to geocode the location information, and as a result it's not generating any features. Try adding state and country columns to your pandas dataframe before the gis.content.import_data()

from arcgis.gis import GIS
gis = GIS("home")

import requests
import urllib.request
import pandas as pd
import json
from bs4 import BeautifulSoup

url="https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/contact-tracing"
r=requests.get(url)
html_content= r.text
soup=BeautifulSoup(html_content, "html.parser")
table=soup.find('table', id = "table60438")
row=table.findAll('tr')
cr_df = pd.read_html(str(table))[0]
cr_df['Country']="Australia"
cr_df['State']="QLD"
tracingTable=gis.content.import_data(cr_df)
tracingTable.query()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

View solution in original post

2 Replies
by Anonymous User
Not applicable

Hi Yvette Bevis‌,

Pretty sure there just isn't enough info in the table for the arcgis python API to know how to geocode the location information, and as a result it's not generating any features. Try adding state and country columns to your pandas dataframe before the gis.content.import_data()

from arcgis.gis import GIS
gis = GIS("home")

import requests
import urllib.request
import pandas as pd
import json
from bs4 import BeautifulSoup

url="https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/contact-tracing"
r=requests.get(url)
html_content= r.text
soup=BeautifulSoup(html_content, "html.parser")
table=soup.find('table', id = "table60438")
row=table.findAll('tr')
cr_df = pd.read_html(str(table))[0]
cr_df['Country']="Australia"
cr_df['State']="QLD"
tracingTable=gis.content.import_data(cr_df)
tracingTable.query()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
YvetteBevis
Esri Contributor

Thanks! That was the issue. I initially tried to add in the location information as a parameter during the import unsuccessfully but your way of doing it worked perfectly.