AnsweredAssumed Answered

MissingSchema: Invalid URL ' ': No schema supplied

Question asked by jpilbeam Champion on Apr 13, 2020
Latest reply on Apr 13, 2020 by bixb0012

Using Python 3.6.9, I'm making requests with Requests by calling URLs from a text file. I put five of them here, for example. The URLs are all valid.

https://secure.shippingapis.com/ShippingAPI.dll?API=Verify&XML=<AddressValidateRequest USERID="564WILLC0589"><Address><Address1>2001 Gardner Cir W</Address1><Address2></Address2><City>Aurora</City><State>IL</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>

https://secure.shippingapis.com/ShippingAPI.dll?API=Verify&XML=<AddressValidateRequest USERID="564WILLC0589"><Address><Address1>2427 Oakfield Dr</Address1><Address2></Address2><City>Aurora</City><State>IL</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>

https://secure.shippingapis.com/ShippingAPI.dll?API=Verify&XML=<AddressValidateRequest USERID="564WILLC0589"><Address><Address1>2451 Avalon Ct</Address1><Address2></Address2><City>Aurora</City><State>IL</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>

https://secure.shippingapis.com/ShippingAPI.dll?API=Verify&XML=<AddressValidateRequest USERID="564WILLC0589"><Address><Address1>2516 Hillsboro Blvd</Address1><Address2></Address2><City>Aurora</City><State>IL</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>

https://secure.shippingapis.com/ShippingAPI.dll?API=Verify&XML=<AddressValidateRequest USERID="564WILLC0589"><Address><Address1>2623 Shenandoah Ct</Address1><Address2></Address2><City>Aurora</City><State>IL</State><Zip5></Zip5><Zip4></Zip4></Address></AddressValidateRequest>

 

What I intend it to do is iterate the list of URLs and request each one in the for loop. Then it prints out the BeautifulSoup object. It looks like an XML when it's printed which is what I want. While debugging, I see it loops once and prints the first URL as desired, but then throws an error at the top of the for loop. I don't see what's wrong with the URL?

''' (1) putting the URLs one by one in the browser, (2) get the resulting XMLs,
and (3) listing these XMLs as text in a .txt file '''



import requests
from bs4 import BeautifulSoup


txtfile = r'C:\Users\jpilbeam\USPSAPIWCHDUpdateAll.txt'

#convert text file into a list
with open (txtfile) as f:
    x = (list(map(str.strip ,f.readlines())))

    for i in x:
        #Request the URL
        response = requests.get(i)
        #see if the URL has been correctly encoded print(r.url)
        r_url = response.text

        #parse the downloaded homepage to get a beautifulsoup object
        new_xml = BeautifulSoup(r_url, features = "xml").prettify()
        print(new_xml)

Error:

>>> 
[Dbg]>>>
<?xml version="1.0" encoding="utf-8"?>
<AddressValidateResponse>
<Address>
  <Address2>
   2001 GARDNER CIR W
  </Address2>
  <City>
   AURORA
  </City>
  <State>
   IL
  </State>
  <Zip5>
   60503
  </Zip5>
  <Zip4>
   6213
  </Zip4>
</Address>
</AddressValidateResponse>
Traceback (most recent call last):
  File "\\gisfile\GISstaff\Jared\Python Scripts\ArcGISPro\CallToFromUSPS_II.py", line 27, in <module>
    response = requests.get(i)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\sessions.py", line 462, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\models.py", line 313, in prepare
    self.prepare_url(url, params)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\models.py", line 387, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '': No schema supplied. Perhaps you meant http://?
>>>

Outcomes