Certain URLs may be blocked from use in Data Pipelines. If that is the case, you will get an invalid URL error.
A workaround is to download the file from the URL and upload it as a file input. To learn more, see the doc for the File input type.
If you're interested in troubleshooting the error, you can leverage the requests Python module that ships with ArcGIS Pro and check for redirects. See the code sample below.
If you do not have ArcGIS Pro installed you can install the requests module into your Python environment. Package management is beyond the scope of this blog post.
Your code block:
# Import required modules
import requests
# Update the url value to be the url you want to use in Data Pipelines
url = r'https://domain.com/page/files/file.csv'
response = requests.get(url,headers={'User-Agent':'Google Chrome'})
# Inspect the response for any redirects
response.history
response.url
With the February 2024 update of ArcGIS Online, Data Pipelines supports URLs that redirect. However, some URLs may redirect to an unsupported location. See the Limitations section of the Public URL input documentation for more details on unsupported URLs.
If the URL does not appear to meet any of the limitations outlined in the doc, Esri Technical Support (http://esriurl.com/support) to report a bug.
