nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Downloading attachments from a URL in text field

mapmakerleah

Hi!

TLDR: I need to download images from text fields with URLs in a feature class, all sorted by their related address and I am not sure how I could run this process

I am not sure if this is the right community to post this in, but I am starting here since I hope to accomplish this using Pro if possible.

I have a feature class with multiple text fields that hold URL links to an image. I am hoping to download these and store them for record-keeping purposes. These are NOT in a related table, just fields in feature classes. I would need to sort these images together to the right information (in this case, an address).

The steps I would need to do are:

- Get images from the text field to download.

- sort information into folders on my hard drive (hopefully like Street > house number > images with field name as image name)

Has anyone done something like these before where attachments were not actually attachments just links through URLS? I know this Esri article exists, but it looks like that is for images added through attachments and not in a URL link.

Thanks in advance for any help!

Finde mehr Beiträge mit dem Tag

Content Management

Feature Services

Accepted answers

All comments

ChristopherCounsell

Python and a http request library. Go through the table, get attribute data, request the file, rename it based on table attributes and sort it into the folders.

You could give some prompts and a sample dataset to Chat GPT and it'll help generate the script for you.

MErikReedAugusta

Well this is excitingly-timed. I was just discussing this operation with a teammate at work this week. Disclaimer: I haven't had a chance to go through said teammate's code, yet, to peek under the hood. I also haven't used urllib, yet, so the below code is an untested first-pass from me going through the documentation tonight.

Caveat emptor!

Also, it's possible I misunderstood something in your post about how these photos will be filed away. The below code assumes there's an "address" field to create folders from and that all the photos in the URLs have proper filenames & file extensions.

If it doesn't work as intended, let me know & I'll try to take a second look at this next week when I have datasets to R&D with and/or when I've had a chance to compare notes with that teammate.

import pathlib
import urllib

# Warning: The lowercase "R" that you see before these strings is important!
#          Without it, any backslashes in your path will break things.
fc = r'Path\to\your\FeatureClass\here'
outDir = r'Path\where\you\want\the\photos\saved'

# This list should contain at minimum: The URL field and whichever field(s)
# you want to use to create subdirectories (if any).
# This also assumes the filename in the URL is fine; if not, add any fields
# you need for that new filename.  I didn't write that bit of code below,
# though.
fieldLst = ['URLField',
            'Address',
           ]

# Seach Cursor will iterate through your attribute table,
# reading the fields we specified in fieldLst, above
with arcpy.da.SearchCursor(fc, fieldLst) as cursor:
  for url, address in cursor:

    # First, we figure out the name of the subfolder from the address,
    # then we create that subfolder if it doesn't already exist.
    addressDir = pathlib.Path(outDir) / address
    pathlib.Path(addressDir).mkdir(parents=True, exist_ok=True)

    # Next, let's get the filename of the photo from that URL
    # Also, lots of urls need percent-encoded characters that will cause
    # us problems, like a space becoming %20.  unquote fixes that for us.
    parsedURL = urllib.parse.urlparse(url)
    photoName = urllib.parse.unquote(pathlib.Path(parsedURL.path).name)

    # Then, we need the path to where that photo is going to be saved
    photoPath = addressDir / photoName

    # Finally, we write that photo to file
    with open(photoPath, 'rb') as img:
      # This downloads the photo from the URL so we can write it
      with urllib.request.urlopen(url) as imgURL:
        # And this actually CREATES the photo file
        img.write(imgURL.read())

    # Because I'm generally paranoid & fastidious, let's clean up all those
    # variables we created along the way.  Cursors have pulled weird tricks
    # on me as they loop through, if I'm being careless.
    del blob, photoPath, photoName, parsedURL, addressDir

ChristopherCounsell

Nice, this is what I was thinking but glad you have one ready to go!

mapmakerleah

Hey! Super excited that this would work out. Attaching two little pieces of information that I think could help determine if this could work.

1) here is what information from our dataset looks like (this is from AGOL, but we have this in pro, too)

This would work as anticipated because there is a address (street) and house number, correct? There is also a full address field.

2) an X'd out version of a URL we are working with. This should also work because there is a .jpg at the end, correct?
https://mydoforms.xyz.com/imageViewer?blobKey=ag9zfm15ZG9mb3Jtcy1ocmRyFwsSCmJsb2Jfc3RvcmUYgIDY77KJggoM&blobName=xyz$$10042018122407$$Published$$1756$$P1$$1.jpg

Fingers crossed that this is a solution to the problem! I assume this could be run in ArcGIS Pro Python window as long as all the information is filled out? If you get to test it, I would love to hear about how it ran!

Thank you so much.

MErikReedAugusta

If you add "import arcpy" to the top of that script, it could even be run outside ArcGIS. (I forgot that line when I wrote that code over the weekend)

Schema & Fields

Given your schema, I would recommend using the "Address" field that's at the furthest-left. It's possible you might have some characters in there that won't work as a folder name, though. You'll want to look into "validating folder names" in Python, and insert that step between Lines 25 & 26.

That's another step I didn't bother with for the proof-of-concept code above but would generally be a best-practice.

Hyperlink & Photo

I think that will work. I can't remember if those dollar signs will cause any trouble and/or if the unquote method (Line 32) will take care of them. I'd say run it and see what errors you get.

Unfortunately, I probably won't have a chance to do any testing on this for a few days; gonna be a much busier week than I planned, this week.

mapmakerleah

Hi!

Just a few follow up questions. I am pretty new to python so I want to make sure I am understanding certain parts of this script right.

1) In the

fieldLst =

['URLField',
'Address',
]

Are you needing to reference the field names or aliases here?

2) Semi-follow up to question 1, as you can see, I have MANY fields with image URLS. Can I do all pictures for a specific address at one time so they will all be sorted into the same folder and then named with the field alias?

i.e. here is a better picture of our dataset, tons of URLS with images. (there is more than what is pictured with URLS

I also attached a sample of how I was hoping to develop the folder structure.

Essentially, would i have to run the script multiple times for each field with a URL? Or can this script (somehow) be utilized so that each record tied to an address downloads all related URL fields/pictures at once?

Sorry for so many questions! I will be happy to test things out and report on how they worked out, I just need to understand more of how I can get there 🙂

Nimm an der Diskussion teil

Angemeldete Mitglieder können Beiträge verfassen, Updates folgen und mehr. Neu hier? Registriere ein kostenloses Konto.

Schnellzugriffe

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.