Converting 32-bit TIFF to 16-bit erroneous values

6934
10
02-06-2018 10:09 AM
JohnWall
Occasional Contributor

I have several thousand TIFF files that I need to convert from 32-bit to 16-bit, but the Copy Raster geoprocessing tool appears to be providing erroneous results.

As an example, I have a TIF in 32-bit format with values ranging 0.0241 to 0.0674. When I use Copy Rater and set the "NoData Value" to blank, "Pixel Type" to 16-bit unsigned, set the "Scale Pixel Value" to true, I get a TIFF with values ranging 0 to 32768, but the image appears blank no matter what histogram stretch I use. I have tried Calculating Statistics within ArcCatalog with no results. When I set "Scale Pixel Value" to false, I get values ranging 0 to 1 with an image that appears blank.

I am using ArcGIS 10.3.1.

Is there something I am missing?

0 Kudos
10 Replies
JieZhang
Esri Contributor

Hi John,

Is it possible for you to upload a small sample image here? We can take a look. You didn't miss anything based on your description.

Thanks

Jie

JohnWall
Occasional Contributor

Hi Jie,

Here's the GeoTIFF.

Thanks!

0 Kudos
DanPatterson_Retired
MVP Emeritus

could be software version

In PRO, if I got it right

JohnWall
Occasional Contributor

Hi Dan,

Thank you for your note! For other reasons I would like to stay in 10.3.1, personally. But, I'll give ArcGIS Pro a shot as well.

Thank you,

John

0 Kudos
JieZhang
Esri Contributor

Hi John,

This is a bug of this tool in this version of ArcMap.

If you have to stay at 10.3.1 and you have to use this tool, I suggest you submit a support case through this link Esri Support Contact Support. There is workaround to use the stretch raster function through the image analysis window and change the output pixel type. However you cannot automate this through scripting.

It would be the best if you can upgrade either to Pro or newer version of ArcMap.

Jie

CodyBenkelman
Esri Regular Contributor

John

It always helps if you can give us a complete story.  e.g. You did what many do NOT do (told us the version are you using), but what are your overall objectives?  What is the data?  Why are you converting thousands of files?

At risk of changing the subject, I am concerned any time I see questions about "processing large # of images" and "creating a lot of outputs".  That's especially true in a case like this, where you're starting with floating point data (where I presume the pixel value is some absolute measurement - I would have guessed orthometric height although your range above "0.0241 to 0.0674" sounds odd) and converting for some reason, to generate files with 16 bit values that now do NOT represent the true data values but must have some legend or lookup table if users ever need to determine the (approximate) original values...


Are the end users for these 16 bit files using some other software?  Have you considered putting all into one single mosaic dataset and applying a raster function to rescale from F32 to U16?  Your project might be completely done in an hour...

Cody B. 

JohnWall
Occasional Contributor

Hello Cody,

The image was created in Pix4D from UAV images recorded with a MicaSense RedEdge sensor. Although the camera records in 16-bit, Pix4D created 32-bit GeoTIFF outputs. I should be able to switch the output to 16-bit in Pix4D, but I have not been able to find that setting in the version of the software I have and I think that issue is maybe best left to a different forum. If that is something I should include, please let me know. Due to the bit-depth issue, I thought I should be able to convert the values in ArcGIS and be good to go.

I am converting thousands of files from roughly 70 surveys that were carried out by someone before me on a university/research project. These surveys cover a series of turfgrass plots and surveys were repeated on multiple days with the aim of observing a blight taking over the grass fields. Some surveys have RGB composite rasters/orthophotos/images while others have individual reflectance rasters which I was planning to convert to 16-bit and create a composite raster (R,G,B, NIR and RedEdge).

I thought the 0.0241 to 0.0674 sounded odd! I will go over the workflow I was provided with and see if I get different values from reprocessing.

The main reason I have been tasked with converting from 32- to 16-bit is the next person is suppose to carry out supervised and unsupervised image classification in ERDAS imagine. They picked that software since it is the package they are familiar with for that kind of work. They are hoping that the spectral signal in the images I process will be enough to classify the turfgrass blight automatically.

Please let me know if I did not answer your questions fully.
Thank you,

John

0 Kudos
CodyBenkelman
Esri Regular Contributor

John

this makes sense.

I'd encourage you to try doing all of this in ArcGIS Pro.  (If you don't have Pro now, you can get a trial copy)

It may be possible that a change in your workflow could output 16 bit integers from Pix4D, but if you have 70 surveys you don't want to have to reprocess all of those through Pix4D.

You should not need to convert to 16bit.  If you creating a mosaic dataset of all the imagery, you can treat it as one logical collection.  The mosaic dataset can also combine any desired bands into a multispectral view (e.g. 4,1,2 for color IR, 4 & 1 for NDVI, 5 & 1 for red-edge-NDVI, etc.), and this is done instantly via a function, not a process you need to run and create more output files.

For the classification, you or colleagues can use advanced classification tools and wizard in the Image Analyst Extension or Spatial Analyst. (also available as a free trial for ArcGIS Pro). This can use any bit depth, or you can apply an arithmetic function prior to classification to convert (on the fly) to 16 bit.

If you really need to proceed with converting to 16bit, then you can use OptimizeRasters.

Download from http://esriurl.com/OptimizeRasters and use following command

c:\Image_Mgmt_Workflows\OptimizeRasters\OptimizeRasters.py -config=Imagery_to_TIF_float2int.xml -input=c:\tmp\Q\inputdirectory -output=c:\tmp\Q\outputdirectory

(See http://s3.amazonaws.com/ImageManagementWorkflowsTeam/OptimizeRasters/Imagery_to_TIF_float2int.xml )

JohnWall
Occasional Contributor

Hi Cody,

Thank you for your suggests. I'll run these past the rest of the group.

At the risk of sounding like I've skipped over something, I understand how a mosaic can be created in ArcGIS, but I do not know how ortho images would be created from UAV data (i.e. non-nadir). Is there a workflow for that? Is that something that has been added between 10.3.1 and Pro?

Thank you,

John

0 Kudos