Image Server inconsistent when reading TIFF tags

[VuDang]

Hello,

When I add a folder of TIFF files to an image service, only the first TIFF file will have its TIFF tags read into the RPDef file. The subsequent TIFF files although they have TIFF tags as well, will have the TIFF tags missing from the RPDef file.

This is TIFFTAGS section with all tags properly read.

<TIFFTAGS>
<GeoPixelScale count="3" type="float64">1.000000000000, 1.000000000000, 0.000000000000</GeoPixelScale>
<GeoTiePoints count="6" type="float64">0.000000000000, 0.000000000000, 0.000000000000, 591319.625000000000, 4789340.962000000300, 0.000000000000</GeoTiePoints>
<GeoKeyDirectory count="28" type="uint16">1, 1, 0, 6, 1024, 0, 1, 1, 1025, 0, 1, 1, 1026, 34737, 257, 0, 3072, 0, 1, 26917, 3073, 34737, 245, 257, 3076, 0, 1, 9001</GeoKeyDirectory>
<GeoASCIIParams count="1" type="string">...</GeoASCIIParams>
<Tag_42112 count="154" type="string">
<GDALMetadata>
<Item name="IMG_GUID">420c3dd1-61ee-4d8b-a659-c202816c47ae</Item>
<Item name="LAYER_TYPE" sample="0">athematic</Item>
</GDALMetadata>
</Tag_42112>
</TIFFTAGS>

This is the TIFFTAGS section missing some tags.

<TIFFTAGS>
<GeoPixelScale count="3" type="float64">1.000000000000, 1.000000000000, 0.000000000000</GeoPixelScale>
<GeoTiePoints count="6" type="float64">0.000000000000, 0.000000000000, 0.000000000000, 591319.625000000000, 4789340.962000000300, 0.000000000000</GeoTiePoints>
<GeoKeyDirectory count="28" type="uint16">1, 1, 0, 6, 1024, 0, 1, 1, 1025, 0, 1, 1, 1026, 34737, 257, 0, 3072, 0, 1, 26917, 3073, 34737, 245, 257, 3076, 0, 1, 9001</GeoKeyDirectory>
<GeoASCIIParams count="1" type="string">...</GeoASCIIParams>
<GDALMetadata count="1" type="string">...</GDALMetadata>
</TIFFTAGS>

[axel99]

I have similar issues. My solution was to write a geoprocessing script tool in python using GDAL and lxml. My approach was as follows to use tiff files with embedded GDALMetadata Tags
1. read the service table to get the location of the relevant rpdef
2. read the rpdef to get the location of the raster source file
3. extract the "IMG_GUID" attribute from the GDALMetadata xml embedded in the tiff file
4. add the GDALMetadata element to the .rpdef for each raster found and store the IMG_GUID value(ie. /ImageServer/RasterProcessDefinition/Rasters/Raster/AdditionalFormatInfo/metadata/GDALMetadata)
5. Delete all PAM elements (persistent auxiliary metadata). I have had some issues with the image service editor if this element existed in the rpdefs
6. Delete all references to Tag_42112

I the tiff files have associated .tif.aux.xml files the behavior is a little different when you add a raster to a service, below is an example of PAM stored in a .tif.aux.xml file. I have based my design on this behavior so I can support all raster formats even if they do not support internal metadata storage.
<PAMDataset>
  <Metadata>
    <MDI key="EVENTID">40d7aac1-d410-49b9-9b25-faeab1d85bfb</MDI>
  </Metadata>
  <PAMRasterBand band="1">
    <Metadata>
      <MDI key="LAYER_TYPE">athematic</MDI>
    </Metadata>
  </PAMRasterBand>
</PAMDataset>

It appears if the rasters have associated .tif.aux.xml the behavior is more consistent. so I guess you could create a .tif.aux.xml for all the raster sources. Make sure you have deleted the .aux files that some times get created automatically by ESRI applications or you may get alot of additional metadata added to the rpdef that is not wanted.

Below is an example of clean well structured metadata element that has worked for me
<metadata>
            <srs format="OpenGIS">PROJCS["NAD83 / UTM zone 10N",GEOGCS["NAD83",DATUM["North_American_Datum_1983",SPHEROID["GRS 1980",6378137,298.2572221010002,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6269"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4269"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-123],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","26910"]]</srs>
            <compressionmethod>none</compressionmethod>
            <recommendedbackend>tiff</recommendedbackend>
            <geokeys>
              <GTModelTypeGeoKey>geokey:S:1</GTModelTypeGeoKey>
              <GTRasterTypeGeoKey>geokey:S:1</GTRasterTypeGeoKey>
              <GTCitationGeoKey>geokey:A:"NAD83 / UTM zone 10N"</GTCitationGeoKey>
              <GeogCitationGeoKey>geokey:A:"NAD83"</GeogCitationGeoKey>
              <GeogAngularUnitsGeoKey>geokey:S:9102</GeogAngularUnitsGeoKey>
              <ProjectedCSTypeGeoKey>geokey:S:26910</ProjectedCSTypeGeoKey>
              <ProjLinearUnitsGeoKey>geokey:S:9001</ProjLinearUnitsGeoKey>
            </geokeys>
            <geopixelscale>2.000000000000, 2.000000000000, 0.000000000000</geopixelscale>
            <geotiepoints>0.000000000000, 0.000000000000, 0.000000000000, 566557.000000000000, 6633521.000000000000, 0.000000000000</geotiepoints>
            <TIFFTAGS>
              <GeoPixelScale count="3" type="float64">2.000000000000, 2.000000000000, 0.000000000000</GeoPixelScale>
              <GeoTiePoints count="6" type="float64">0.000000000000, 0.000000000000, 0.000000000000, 566557.000000000000, 6633521.000000000000, 0.000000000000</GeoTiePoints>
              <GeoKeyDirectory count="32" type="uint16">1, 1, 0, 7, 1024, 0, 1, 1, 1025, 0, 1, 1, 1026, 34737, 21, 0, 2049, 34737, 6, 21, 2054, 0, 1, 9102, 3072, 0, 1, 26910, 3076, 0, 1, 9001</GeoKeyDirectory>
              <GeoASCIIParams count="1" type="string">NAD83 / UTM zone 10N|NAD83|</GeoASCIIParams>
              <GDALMetadata count="1" type="string">...</GDALMetadata>
              <GDALNoDataValue count="1" type="string">-9999</GDALNoDataValue>
            </TIFFTAGS>
          <GDALMetadata><Item name="EVENTID">c5ee467e-8893-495d-82ac-28e289c94dc3</Item></GDALMetadata>
</metadata>

GDALMetadata(tag 42112) and GDALNoDataValue(tag 42113) are registered tiff tags, similar to the GeoTiff tags hence they show up under the <TIFFTAGS>, not sure why GDALMetadata tiff tag is being parsed as an unrecognized tiff tag in one of your examples. It�??s a bug I guess!

I consider "<GDALMetadata count="1" type="string">...</GDALMetadata>
" as valid. GDALMetadata is an attribute of the <TIFFTAGS> indicating the number of xml strings, hence the use of the ellipsis. Just my interpretation! 

I also notice you may not have defined a GDALNoDataValue, I know support for nodata in image server today is very weak, but in the next release I think ESRI has added allot of new support for the concept of nodata. Just an observation!