Working with EXIF image metadata in Survey123 for ArcGIS

7293
10
12-12-2017 02:58 PM
Esri Frequent Contributor
11 10 7,293

Digital cameras typically embed metadata at the point of capture into image files. EXIF (Exchangeable Image File Format) is the most widely adopted standard for encoding this metadata within the photos you take.  Some common data fields include the camera make and model, its serial number, the date and time of image capture, the shutter speed, the aperture, the ISO speed setting...  If a GPS is available to the camera, EXIF will also include the location from which the photo was taken and also the direction.

All the above information can be critical so we added quite extensive support for EXIF data within Survey123 for ArcGIS.

Preserving and capturing EXIF data with your photos.

The Survey123 field app is designed to preserve and capture EXIF data whenever possible.  When a new image question is added to your survey, you can use the Survey123 app to either browse for an existing photo, or use your device's camera to capture a new one.

If you browse for an existing photo, the Survey123 app will preserve any existing EXIF data in case at some point you want to download the photo from ArcGIS and extract the EXIF metadata.  This is important particularly when you use high-end cameras that write EXIF tags of your interest, or when you simply are handed a photo from someone else.

If you choose to capture a new photo with the Survey123 field app, not only EXIF metadata will be populated according to the  capabilities of your device, but also a handy GPS button will be available so you can refresh the GPS coordinates from your device right when you take the photo. The reason why this is important is because the GPS in your device may not have fixed the best possible location when you shoot the photo. By using the GPS button in the Survey123's app camera widget you can help getting the best possible location into your EXIF's metadata.

Looking into basic EXIF metadata tags

The Survey123 website has built-in reporting capabilities that let you look at basic EXIF metadata in photos submitted. Simply go into the data tab, and expand the Individual Survey Response report for a particular record. Simply click on photos within the report to access basic EXIF metadata such as the coordinates and direction where the photo was taken, the device used to capture the photo and the date and time.

Extracting EXIF metadata using the pulldata() XLSForm function

My favorite feature in Survey123 in regards with EXIF is the ability to write expressions in your XLSForm that will automatically extract EXIF tags from your photos so you can build data validation rules in your smart form and even store this EXIF tags as attributes in your features.  This means that later on you will be able to easily query your EXIF metadata within ArcGIS without any post-processing.  A classic example is extracting the GPSImageDirection tag into an attribute, so you can later use it to rotate your symbols on a map, or the GpsLongitude and GpsLatitude tags to later map the location of the photos. Other handy tags include ImageWidth, ImageLength and ExtendedDigitalZoomRatio which give you information about the size of the image as well as the optical zoom level used (if any) when the photo was captured, and of course the ImageDateTime tag as well

You can extract the EXIF tags using the pulldata() function, which we extended specifically for this purpose. The syntax is as follows:


pulldata("@exif", ${image}, "GpsImageDirection")

The first parameter indicates that the pulldata() function will be used to extract EXIF metadata. The second parameter indicates the question in your XLSForm with the image. The third parameter defines the name of the EXIF tag to be extracted.

There are over 300 tags in the EXIF specification, although there is no guarantee that all of them will be populated. It varies a lot depending on the device.  The table below shows the most common ones.

EXIF GeotagsEXIF Basic Camera  and Image TagsExtended Tags
GpsVersionId
GpsLatitudeRef
GpsLatitude
GpsLongitudeRef
GpsLongitude
GpsAltitudeRef
GpsAltitude
GpsTimeStamp
GpsSatellites
GpsStatus
GpsMeasureMode
GpsDop
GpsSpeedRef
GpsSpeed
GpsTrackRef
GpsTrack
GpsImageDirectionRef
GpsImageDirection
GpsMapDatum
GpsDestLatitudeRef
GpsDestLatitude
GpsDestLongitudeRef
GpsDestLongitude
GpsDestBearingRef
GpsDestBearing
GpsDestDistanceRef
GpsDestDistance
GpsProcessingMethod
GpsAreaInformation
GpsDateStamp
GpsDifferential
ImageWidth
ImageLength
ImageBitsPerSample
ImageCompression
ImagePhotometricInterpretation
ImageOrientation
ImageSamplesPerPixel
ImagePlanarConfiguration
ImageYCbCrSubSampling
ImageXResolution
ImageYResolution
ImageResolutionUnit
ImageStripOffsets
ImageRowsPerStrip
ImageStripByteCounts
ImageTransferFunction
ImageWhitePoint
ImagePrimaryChromaciticies
ImageYCbCrCoefficients
ImageYCbCrPosition
ImageReferenceBlackWhite
ImageDateTime
ImageDescription
ImageMake
ImageModel
ImageSoftware
ImageArtist
ImageCopyright
ImageXPTitle
ImageXPComment
ImageXPAuthor
ImageXPKeywords
ImageXPSubject
ImageRating
ImageRatingPercent
ImageLensSpecification
ImageLensMake
ImageLensModel
ImageLensSerialNumber
ExtendedExifVersion
ExtendedFlashPixVersion
ExtendedColorSpace
ExtendedComponentsConfiguration
ExtendedCompressedBitsPerPixel
ExtendedPixelXDimension
ExtendedPixelYDimension
ExtendedMakerNote
ExtendedUserComment
ExtendedRelatedSoundFile
ExtendedDateTimeOriginal
ExtendedDateTimeDigitized
ExtendedSubSecTime
ExtendedSubSecTimeOriginal
ExtendedSubSecTimeDigitized
ExtendedImageUniqueId
ExtendedExposureTime
ExtendedFNumber
ExtendedExposureProgram
ExtendedSpectralSensitivity
ExtendedISOSpeedRatings
ExtendedOecf
ExtendedShutterSpeedValue
ExtendedApertureValue
ExtendedBrightnessValue
ExtendedExposureBiasValue
ExtendedMaxApertureValue
ExtendedSubjectDistance
ExtendedMeteringMode
ExtendedLightSource
ExtendedFlash
ExtendedFocalLength
ExtendedSubjectArea
ExtendedFlashEnergy
ExtendedSpatialFrequencyResponse
ExtendedFocalPlaneXResolution
ExtendedFocalPlaneYResolution
ExtendedFocalPlaneResolutionUnit
ExtendedSubjectLocation
ExtendedExposureIndex
ExtendedSensingMethod
ExtendedFileSource
ExtendedSceneType
ExtendedCfaPattern
ExtendedCustomRendered
ExtendedExposureMode
ExtendedWhiteBalance
ExtendedDigitalZoomRatio
ExtendedFocalLengthIn35mmFilm
ExtendedSceneCaptureType
ExtendedGainControl
ExtendedContrast
ExtendedSaturation
ExtendedSharpness
ExtendedDeviceSettingDescription
ExtendedSubjectDistanceRange

To get you started quickly with the use of the pulldata() function, I recommend you have a look at the EXIF XLSForm sample included with Survey123 Connect for ArcGIS.

  1. Open Survey123 Connect for ArcGIS
  2. Click on New Survey
  3. Find the EXIF survey in the Samples category
  4. Set a title to your new survey and create it
  5. The list of tags is located in the choices worksheet

Limitation: pulldata("@exif") will not work if your image question uses the multiline appearance.

In most cases I see the pulldata() function being used to simply persist EXIF tags from photos as attributes in your feature service, but you can also use them to build input data validation logic. For example, you can compare the location of a geopoint question in your survey with the location of a photo and prevent the user from submitting if the distance is bigger than a particular threshold. You could also display a warning if the user attempts to use a photo that was taken a long time ago etc.  This is were you get creative to get the best out of Survey123...

10 Comments
Regular Contributor

http://www.exiv2.org/tags.html - tag descriptions. Scroll to the bottom of the page for the GPS tags

New Contributor III

There is a small discrepancy between the image direction tag name used by Survey123 and the Exiv2.3 specification: GPSImgDirection -vs- GPSImageDirection. Note that one tag has 'image' abbreviated.

The Exiv2.3 specification uses the GPSImgDirection spelling while Survey123 only accepts the full GPSImageDirection spelling.

I tested both spellings on a form. I used iPhone X to test. When I add the abbreviated form pulldata("@exif", ${image}, "GPSImgDirection") to my survey, no data is pulled when using either the built in camera or a previously taken photo. However, using the full spelled out form, data was returned: pulldata("@exif", ${image}, "GpsImageDirection")

The same holds true for the GpsImageDirectionRef data.

Here's a test form on Dropbox that dumps every possible tag in the Esri list text fields so I can determine the best data type to use in future implementations. You can see the two different spellings versions in there.

MVP Regular Contributor

Can you post that test form again? Thanks!

New Contributor III

Sure. Sorry, I recently moved away from Dropbox. Here it is again.

Photo EXIF Data Pulling Test Form - Google Drive  

New Contributor III

Is it possible to populate a text field using the photo name survey123 assigns to a photo that has been taken?

For example, say I take a photo and survey123 assigns the following name image-20180627-210709.jpg, can I populate a text field with that filename? I keep getting requests at work for the reports to contain not only the image but also the image name.

Esri Esteemed Contributor

Hi Peter,

The name of the file can be used in the report template functionality using

${image1|getValue:"name"}

New Contributor III

Is access to the EXIF tags limited to only Survey123 or is possible to grab them with ArcGis API for Python or ArcPy?

It seems like survey123 doesn't get the location automatically about 20% of the time and I wanted to fix the position or fill in a null position with the picture location if available.

Esri Esteemed Contributor

Hi Michael,

It is possible, though you would need to bring in a module like https://www.math.u-psud.fr/~bousch/exifdump.py or ExifRead · PyPI  to process EXIF information.

New Contributor III

Awesome Thank you for the links. James this is perfect

New Contributor III

I am filling out a form that is linked to an existing feature service with attachments. I have add several columns to the attachments table to store exif information. I am wondering if I can use the method described above to push the exif information to the attachments table?

First of all is it possible and how would I do it? No worries if this functionality isn't possible. I have exifread and could populate the table after the fact.