topic Re: Export files from file server using URL path in Python Questions

Export files from file server using URL path

shildebrand — Thu, 21 Sep 2023 17:43:24 GMT

I recently wrote this script to download files from a file server using a folder path. For example the "LINK" field has values such as this: "\\wapgis61\Scans\101-005\101-005.pdf". We have recently changed the source data to to reference a virtual directory which uses URL's. For example, the path referenced earlier is accessed with a URL like this: http://wapgis61/Scans/101-005/101-005.pdf . Does anyone know how to modify this script to extract files that a referenced with a URL instead of a UNC path?

import arcpy import os import shutil fc = arcpy.GetParameterAsText(0) dest_path = arcpy.GetParameterAsText(1) print(fc) cursor = arcpy.SearchCursor(fc) for row in cursor: field = "LINK" link = (row.getValue(field)) head, tail = os.path.split(str(link)) print ("Downloading: " + tail) shutil.copy(link, dest_path) print("Download Complete")

Re: Export files from file server using URL path

JakeSkinner — Thu, 21 Sep 2023 18:13:10 GMT

Hi @shildebrand ,

You can use the requests module to do this. Below is an example:

import requests # Variables url = 'https://some.server.com/test.pdf' exportLocation = r'c:\temp\test.pdf' response = requests.get(url) with open(exportLocation, 'wb') as f: f.write(response.content)

Re: Export files from file server using URL path

shildebrand — Thu, 21 Sep 2023 20:42:50 GMT

Thanks Jake! Do you know how to reference the a feature class field that has the URL in it? The script below returns an error showing that the "LINK" field is an invalid URL. I'm guessing I probably have an incorrect syntax?

import arcpy import os import requests fc = arcpy.GetParameterAsText(0) dest_path = arcpy.GetParameterAsText(1) print(fc) cursor = arcpy.SearchCursor(fc) for row in cursor: #url format = http://wapgis61/Scans/101-005/101-005.pdf url = "LINK" print ("Downloading: " + url) response = requests.get(url) with open(dest_path, 'wb') as f: f.write(response.content) print("Download Complete")

Re: Export files from file server using URL path

JakeSkinner — Fri, 22 Sep 2023 19:47:17 GMT

Try the following:

import arcpy import os import requests fc = arcpy.GetParameterAsText(0) dest_path = arcpy.GetParameterAsText(1) print(fc) with arcpy.da.SearchCursor(fc, ['LINK']) as cursor: for row in cursor: print("Downloading: " + row[0]) response = requests.get(row[0]) outputFile = row[0].split("/")[0] with open(dest_path + '\\' + outputFile, 'wb') as f: f.write(response.content) del cursor

Re: Export files from file server using URL path

shildebrand — Mon, 25 Sep 2023 15:53:26 GMT

Thanks Jake, with a slight modification to your suggestion, it worked! Just changed the index number to 5 in row 13. Thanks for your help!

import arcpy import os import requests fc = arcpy.GetParameterAsText(0) dest_path = arcpy.GetParameterAsText(1) print(fc) cursor = arcpy.SearchCursor(fc) with arcpy.da.SearchCursor(fc, ['LINK']) as cursor: for row in cursor: print("Downloading: " + row[0]) response = requests.get(row[0]) outputFile = row[0].split("/")[5] print(outputFile) with open(dest_path + '\\' + outputFile, 'wb') as f: print(f) f.write(response.content) del cursor