topic Re: Extract Feature Attachments in Python Questions

Extract Feature Attachments

Ranga_Tolapi — Thu, 24 Oct 2013 08:13:06 GMT

"Feature attachments" is the new feature from ArcGIS 10 on wards. How to extract feature attachments and save to disk using Python?

Re: Extract Feature Attachments

XanderBakker — Sat, 11 Dec 2021 10:27:01 GMT

"Feature attachments" is the new feature from ArcGIS 10 on wards. How to extract feature attachments and save to disk using Python?

Hi,

In this blogpost (please read):
http://anothergisblog.blogspot.nl/2012/06/working-with-blob-data-at-101-arcpyda.html

... a nice example is included of how to read attachments:

from arcpy import da
import os
with da.SearchCursor(r"c:\temp\demo.gdb\table",['blobFieldname','fileName']) as cursor:
   for row in cursor:
      binaryRep = row[0]
      fileName = row[1]
      # save to disk
      open(r"c:\saveFolder" + os.sep + fileName, 'wb').write(binaryRep.tobytes())
      del row
      del binaryRep
      del fileName

You can find more on Cursor and blob fields (scroll down to end of page) through this link:
http://resources.arcgis.com/en/help/main/10.2/index.html#//002z0000001q000000

Kind regards,

Xander

Re: Extract Feature Attachments

Ranga_Tolapi — Tue, 29 Oct 2013 13:07:07 GMT

Hi Xander,

Thanks for your response.

Actually I was looking about "Feature Attachments", which involves a relationship table to store the attachments for features.

Please go through "Query attachments" section in below link, which talks about performing the same task in .Net.

http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/index.html#//0001000001qr000000

Re: Extract Feature Attachments

XanderBakker — Sat, 11 Dec 2021 10:27:04 GMT

Hi Xander,

Thanks for your response.

Actually I was looking about "Feature Attachments", which involves a relationship table to store the attachments for features.

Please go through "Query attachments" section in below link, which talks about performing the same task in .Net.

http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/index.html#//0001000001qr000000

Python is more limited than ArcObjects (although you could implement COM types in Python and use ArcObjects). If you only want to save the attachments and not use any related data from the features then the code remains mostly the same:

from arcpy import da
import os
# fc = r"C:\Project\_LearnPython\Attachments\test.gdb\yourFCname"
tbl = r"C:\Project\_LearnPython\Attachments\test.gdb\yourFCname__ATTACH"
fldBLOB = 'DATA'
fldAttName = 'ATT_NAME'
outFolder = r"C:\Project\_LearnPython\Attachments"

with da.SearchCursor(tbl,[fldBLOB,fldAttName]) as cursor:
   for row in cursor:
      binaryRep = row[0]
      fileName = row[1]
      # save to disk
      open(outFolder + os.sep + fileName, 'wb').write(binaryRep.tobytes())
      del row
      del binaryRep
      del fileName

In this case you access the attachment tabel ("yourFCname__ATTACH"). The name of the attachment is stored in the field "ATT_NAME" and the binary data in the field "DATA".

If you want to access the feature itself, you will have to use the "REL_OBJECTID" field from the attachment table and query the featureclass on OBJECTID = REL_OBJECTID...

Kind regards,

Xander

Re: Extract Feature Attachments

Ranga_Tolapi — Tue, 29 Oct 2013 14:39:12 GMT

Thank you Xander, appreciated.
I will try this later, as I am busy with my other critical tasks.

Re: Extract Feature Attachments

NicholasKnabe — Fri, 15 Nov 2013 12:36:13 GMT

How would I go about added the relationship data into my python script.

"If you want to access the feature itself, you will have to use the "REL_OBJECTID" field from the attachment table and query the featureclass on OBJECTID = REL_OBJECTID..."

I know you mentioned this way, but since it is a relationship does it need to be pathed to a geodatabase? Or how does it pull the relationship information with the pictures?

Thanks
-Nick

Re: Extract Feature Attachments

MichaelVolz — Fri, 15 Nov 2013 13:13:00 GMT

Xander:

Would you be able to extend your code in python to be able to access images already stored as a BLOB field in SQL Server 2012 (non-GIS) as attachments without having to copy the images into ESRI feature classes thus eliminating data duplication of attachment files?

Re: Extract Feature Attachments

XanderBakker — Sat, 11 Dec 2021 10:27:06 GMT

How would I go about added the relationship data into my python script.

"If you want to access the feature itself, you will have to use the "REL_OBJECTID" field from the attachment table and query the featureclass on OBJECTID = REL_OBJECTID..."

I know you mentioned this way, but since it is a relationship does it need to be pathed to a geodatabase? Or how does it pull the relationship information with the pictures?

Hi Nicholas,

If you would like to access the related information the belongs to the attachment, you will have to use the REL_OBJECTID in the attachment table to query the feature class. In the code below I created a function "QueryRelatedData" that retrieves information from the feature class. If the related OID = 2, the expression will be "OBJECTID = 2", which is used as a query definition to only extract 1 feature. Next the attribute in the field "SomeFieldInFeatureClass" is accessed and returned.

I didn't test the code below, so be careful.

def main():
    global arcpy
    import arcpy, os

    # fixed settings
    fldBLOB = 'DATA'
    fldAttName = 'ATT_NAME'
    fldRelOID = 'REL_OBJECTID'

    # your settings (edit these)
    fc = r"C:\Project\_LearnPython\Attachments\test.gdb\Meldingen"
    tbl = r"C:\Project\_LearnPython\Attachments\test.gdb\Meldingen__ATTACH"
    outFolder = r"C:\Project\_LearnPython\Attachments"
    fldRelatedInfo = 'SomeFieldInFeatureClass' # should be valid column in FC

    with arcpy.da.SearchCursor(tbl,[fldBLOB,fldAttName,fldRelOID]) as cursor:
        for row in cursor:
            binaryRep = row[0]
            fileName = row[1]
            relOID = row[2]

            # access related information
            myRelatedInfo = QueryRelatedData(fc, fldRelatedInfo, relOID)
            # do something with the related info

            # save attachment to disk
            open(outFolder + os.sep + fileName, 'wb').write(binaryRep.tobytes())
            del row
            del binaryRep
            del fileName


def QueryRelatedData(fc, fldRelatedInfo, relOID):
    fldOID = 'OBJECTID'
    expression = arcpy.AddFieldDelimiters(fc, fldOID) + " = {0}".format(relOID)
    with arcpy.da.SearchCursor(fc, (fldOID, fldRelatedInfo), where_clause=expression) as cursor:
        for row in cursor:
            return row[1]
            break
        del row

if __name__ == '__main__':
    main()

Kind regards,

Xander

Re: Extract Feature Attachments

XanderBakker — Mon, 18 Nov 2013 06:06:28 GMT

Would you be able to extend your code in python to be able to access images already stored as a BLOB field in SQL Server 2012 (non-GIS) as attachments without having to copy the images into ESRI feature classes thus eliminating data duplication of attachment files?

Hi Michael,

Could you explain this a little more? Do you have images stored in SQL Server 2012, and you want to use them as attachment, without the need to convert them to an Esri attachment table? If that's the case, I don't think ArcGIS will recognize them as attachments.

If you would like to extract the BLOB attachments stored in SQL Server 2012 by code, I think you should be able to do so. If the SQL Server 2012 table can be accessed by ArcGIS, you can loop through it and extract the attachments (like in the code, save it to disk).

After saving the files to disk, I guess you can use "Generate Attachment Match Table (Data Management)" and Add the attachments (which would duplicate or "triplicate" the data). More on this in the Help topic "Working with the Attachments geoprocessing tools".

Not sure, if this is the answer you are looking for.

Kind regards,

Xander

Re: Extract Feature Attachments

MichaelVolz — Mon, 18 Nov 2013 11:46:21 GMT

I'm not a Database Administrator (DBA), but the DBAs at my organization are always trying to minimize unnecessary data duplication which seems what is occurring with the attachments if you already have the data stored in another format other than feature classes that recognize attachments.

Don't you think it would be more efficient if attachments could just be created in the native format that the original attachment files were stored in?

Re: Extract Feature Attachments

XanderBakker — Mon, 18 Nov 2013 12:22:22 GMT

Hi Michael,

The DBAs (and you) are right; data duplication creates a lot of extra work (apart from the additional data storage). Maybe in the future we'll see some extra functionality in ArcGIS allowing us to create attachments in the form of "hyperlinks" to existing data.

I see two ideas that kinda want the same as you're proposing:
Geodatabase Attachments for large organizations
Geodatabase Attachments Storage Location Option

I think we should promote those ideas.

Kind regards,

Xander

Re: Extract Feature Attachments

MichaelVolz — Mon, 18 Nov 2013 12:59:34 GMT

I had already promoted both of these ideas weeks ago. Now I am hoping that other GIS users will join the bandwagon so that ESRI see the need for this additional functionality.

Re: Extract Feature Attachments

TrilliumLevine1 — Thu, 28 Aug 2014 09:24:16 GMT

Hello,

I'm using 10.0, which as far as I know doesn't/can't use the arcpy.da module. How then would I go about extracting image attachments to disk without that module? Any feedback is greatly appreciated!

Re: Extract Feature Attachments

XanderBakker — Fri, 29 Aug 2014 03:41:55 GMT

The functionality to extract the feature attachments is not limited to the da.cursor. The syntax will be different. The help provides some examples of how to use the cursor in 10.0:ArcGIS Desktop

If you have problems creating the proper syntax let me know and I will add the code sample for 10.0.

Kind regards, Xander

Re: Extract Feature Attachments

TrilliumLevine1 — Tue, 02 Sep 2014 09:03:40 GMT

Hi Xander,

A code sample for 10.0 would be great, thank you. I'm also wondering if there's a way using 10.0 to access attachments from a service and extract them to a local disk. Everything I've found on this type of operation is for 10.1 an above.

Thanks again!

Re: Extract Feature Attachments

XanderBakker — Wed, 03 Sep 2014 16:50:48 GMT

Observe code sample below. Once again I haven't tested the code, so be careful.

Changes in syntax can be found on lines 16, 18-20 + 35 and 37

def main():
    global arcpy
    import arcpy, os

    # fixed settings
    fldBLOB = 'DATA'
    fldAttName = 'ATT_NAME'
    fldRelOID = 'REL_OBJECTID'

    # your settings (edit these)
    fc = r"C:\Project\_LearnPython\Attachments\test.gdb\Meldingen"
    tbl = r"C:\Project\_LearnPython\Attachments\test.gdb\Meldingen__ATTACH"
    outFolder = r"C:\Project\_LearnPython\Attachments"
    fldRelatedInfo = 'SomeFieldInFeatureClass' # should be valid column in FC

    with arcpy.SearchCursor(tbl) as cursor:
        for row in cursor:
            binaryRep = row.getValue(fldBLOB)
            fileName = row.getValue(fldAttName)
            relOID = row.getValue(fldRelOID)

            # access related information
            myRelatedInfo = QueryRelatedData(fc, fldRelatedInfo, relOID)
            # do something with the related info

            # save attachment to disk
            open(outFolder + os.sep + fileName, 'wb').write(binaryRep.tobytes())
            del row
            del binaryRep
            del fileName

def QueryRelatedData(fc, fldRelatedInfo, relOID):
    fldOID = 'OBJECTID'
    expression = arcpy.AddFieldDelimiters(fc, fldOID) + " = {0}".format(relOID)
    with arcpy.SearchCursor(fc, where_clause=expression) as cursor:
        for row in cursor:
            return row.getValue(fldRelatedInfo)
            break
        del row

if __name__ == '__main__':
    main()

Re: Extract Feature Attachments

TrilliumLevine1 — Thu, 04 Sep 2014 08:27:02 GMT

Hi Xander,

Thanks a lot for taking the time to post this, I appreciate it!

Best regards

Re: Extract Feature Attachments

TrilliumLevine1 — Tue, 07 Oct 2014 15:59:04 GMT

For anyone who is still looking for answers on extracting attachments from a secured ArcGIS online-hosted feature service: I've been working with Brian Brown (who has been helpful byond the call of duty) on developing a script that I can use to download attachments and replicate the entire hosted feature service locally.

Brian's base code can be found on his github site: https://github.com/bgeomapping/arcgis-toolboxes

I made some minor modifications to it, which are included in the attached code:

Functionality to create a directory for attachments and another for the spatial data.
Functionality for unzipping and renaming the replicated geodatabase.
I need this to run daily and delete the download from 'yesterday', so I added functionality for that.
I also wanted to have the global id in the file name, and for it to add a file extension to the attachment even if one wasn't included as part of the the file name -- functionality for that is added.
As far as I can tell the base code works only for images, but since attachments can be virtually any format, I modified the code to handle any format.
Generate a report showing number of attachments downloaded per layer in feature service.

Other things to note: if the specified url ends in FeatureServer, the entire service will be replicated and all attachments for all layers with attachments in the service will be downloaded. If the specified url ends in FeatureServer/<layer index>, only that layer will be replicated and only attachments for that layer will be downloaded. Also, the code creates the following subdirectories for attachments: Layer > Feature OID > attachments, as well as an 'All attachments' folder -- I don't need the 'All attachments' portion, so it's commented out. This should work in both Python 2.6 and 2.7 (I've tested in 2.6, Brian in 2.7) and is written so that ArcGIS need not be installed on the machine running the script (in other words, no arcpy import).

I hope this helps some of you!!!

- Trill

Re: Extract Feature Attachments

TrilliumLevine1 — Sat, 11 Dec 2021 10:27:09 GMT

Hi Xander,

I'm working with this code sample in 10.2 to access related information from downloaded attachments. The first problem I'm running into is that my myFeatureClass_ATTACH has no REL_OBJECTID field, only a REL_GLOBALID field. When I adjust the code to use the expression to match the global id's instead of the object id's, I'm getting the following error:

I'm not sure why it's still trying to select OBJECTID, which is maybe the problem (?) -- anyway, see my modified code below. Please let me know if you have any ideas. Any guidance is much appreciated! Thanks -

Trill

def main():
    global arcpy
    import arcpy, os

    # fixed settings
    fldBLOB = 'DATA'
    fldAttName = 'ATT_NAME'
    fldRelGID = 'REL_GLOBALID'

    # your settings (edit these)
     ## related feature class
    fc = r"C:\Forstmobil_2014.12.01\Geodaten\Forstmobil_2014.12.01.gdb\Baumkontrolle_AGS_online"
    ## attachment table
    tbl = r"C:\Forstmobil_2014.12.01\Geodaten\Forstmobil_2014.12.01.gdb\Baumkontrolle_AGS_online__ATTACH
    fldRelatedInfo = 'CreationDate' # should be valid column in FC

    with arcpy.da.SearchCursor(tbl,[fldBLOB,fldAttName,fldRelGID]) as cursor:
        for row in cursor:
            binaryRep = row[0]
            fileName = row[1]
            relGID = row[2]
            # access related information
            myRelatedInfo = QueryRelatedData(fc, fldRelatedInfo, relGID)
            print myRelatedInfo

def QueryRelatedData(fc, fldRelatedInfo, relGID):
    fldGID = 'GlobalID'
    expression = arcpy.AddFieldDelimiters(fc, fldGID) + " = {0}".format(relGID)
    with arcpy.da.SearchCursor(fc, (fldGID, fldRelatedInfo), where_clause=expression) as cursor:
        for row in cursor:
            return row[1]
            break
        del row

if __name__ == '__main__':
    main()

Re: Extract Feature Attachments

XanderBakker — Mon, 01 Dec 2014 16:13:13 GMT

Strange, in your code there is no reference to the OBJECTID field. Maybe due to the attachments it is trying to include it, but if the relationship is based on GlobalID, then there will be no OBJECTID (?).

The only thing I saw missing is the quote at the end of line 14 (but I think that is a copy error).