best large imagery file format to use?

Question

Hi folks -

I've been tasked - as a contractor to a contractor -with the task of making some Rather Large Imagery mosaics.  I have not been given any specs other than 'make it suitable for use by college students learning GIS (I am assuming using ESRI products).  The files are on the order of 3.7 GB in img format.  My question is, what is the 'best' format to use to store these files in terms of flexibility - given I have not been told what to use.  Would it be a GEoTiff or is there something equally user friendly, free to write and read, that offers better compression?

thanks,
Cyndy

EshaanSharma · Answer

MrSID vs JPEG2000: Why GIS Professionals Prefer MrSID. Real Compression Test Results That Set the Record Straight

Original GeoTiff: 1.06 GB 
100:1 Compressed JPEG2000: 5.6 MB 
100:1 Compressed MrSID: 1.6 MB

Duarte_Carreira · Answer

I also think wavelet compression formats are best for very large images. But JPEG2000 has an horrific performance, and others are expensive. You can get a free ECW encoder for images up to 500MB (lookup ERDAS' site).

You should consider creating Tiled GTIFF files with JPEG compression. From my experience, this format is very performant and compatible with a wide array of GIS software. You additionally need to create the pyramids. For more details look here:
http://www.gdal.org/frmt_gtiff.html
http://www.mail-archive.com/mapserver-users@lists.osgeo.org/msg01155.html

Best of luck.
Duarte

JohnSobetzer · Answer

The most likely candidate that I can think of is jpeg 2000 (JP2 commonly).  MrSid, ECW and JP2 all use wavelet compression which works well with photography and greatly reduces disk storage requirements.  Decompressing does use CPU cycles, but for many purposes only the area being displayed needs to be uncompressed, and they have built in pyramids.  ArcMap handles them all.  MrSid and ECW, however, are proprietary and require expensive encoders.  Jpeg2000 is open source and ESRI's own software can encode it.  I warn you that at least through 9.3.1, the ESRI encoding produced a JP2 file that was rather slow to display on a computer, and awful on ArcPad.  The best JP2s I've seen comes from the expensive LizardTech encoder that also does MrSid.  JP2's also supports 4 bands.  I haven't found a good free JP2 encoder.

If you weren't using photos, but rasters such as the USGS topos (DRGs), then I'd suggest using tif rasters with LZW compression.  The compression is very high and the tifs very fast.