I am attempting to scrape data from a website and publish it as a feature layer to ArcGIS Online. I have been using the sample notebook (HTML table to Pandas Data Frame to Portal Item | ArcGIS for Developers ) as guidance for the pandas data frame to portal item bit.
my problem is that after importing the data frame the resulting feature collection is empty:
Can anyone help me understand why it's empty? Is there maybe a characteristic of the data frame that it doesn't like? Is there a small detail I've overlooked?
Thanks,
Yvette
Solved! Go to Solution.
Hi Yvette Bevis,
Pretty sure there just isn't enough info in the table for the arcgis python API to know how to geocode the location information, and as a result it's not generating any features. Try adding state and country columns to your pandas dataframe before the gis.content.import_data()
from arcgis.gis import GIS
gis = GIS("home")
import requests
import urllib.request
import pandas as pd
import json
from bs4 import BeautifulSoup
url="https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/contact-tracing"
r=requests.get(url)
html_content= r.text
soup=BeautifulSoup(html_content, "html.parser")
table=soup.find('table', id = "table60438")
row=table.findAll('tr')
cr_df = pd.read_html(str(table))[0]
cr_df['Country']="Australia"
cr_df['State']="QLD"
tracingTable=gis.content.import_data(cr_df)
tracingTable.query()
Hi Yvette Bevis,
Pretty sure there just isn't enough info in the table for the arcgis python API to know how to geocode the location information, and as a result it's not generating any features. Try adding state and country columns to your pandas dataframe before the gis.content.import_data()
from arcgis.gis import GIS
gis = GIS("home")
import requests
import urllib.request
import pandas as pd
import json
from bs4 import BeautifulSoup
url="https://www.qld.gov.au/health/conditions/health-alerts/coronavirus-covid-19/current-status/contact-tracing"
r=requests.get(url)
html_content= r.text
soup=BeautifulSoup(html_content, "html.parser")
table=soup.find('table', id = "table60438")
row=table.findAll('tr')
cr_df = pd.read_html(str(table))[0]
cr_df['Country']="Australia"
cr_df['State']="QLD"
tracingTable=gis.content.import_data(cr_df)
tracingTable.query()
Thanks! That was the issue. I initially tried to add in the location information as a parameter during the import unsuccessfully but your way of doing it worked perfectly.