JPEG2000 Compression for SDE Rasters very slow

[RandyKreuziger]

We are moving to new SQL/SDE servers so I was expecting the rasters saved in an SDE raster catalog to be a lot faster but that is not the case.  In fact they are slower but things aren't identical from system to system.

Current system:  SQL 2000 / SDE 9.1, 1 CPU   5 year old server, Raster Catalog on a 5 year old SAN
New system:      SQL 2008 / SDE 9.3.1,  2 CPU 4 Core,  Raster Catalog on new SAS drives

The raster catalog in 9.1 was created using 1m resolution NAIP data tiled by quarter township and using LZ77 compression with an average tile size of 4950 rows by 7050 columns.  Newer imagery is now available so I loaded it as is but decided to using JPEG2000 compression (75) with all tiles being
8192 by 8192. 

Using the new raster catalog in ArcMap doesn't seem that much slower than the old raster catalog but in ArcIMS applications it takes double the time to pan through the imagery.  So in ArcIMS where it takes 7 seconds to pan with the old service it now takes 14 seconds which is not acceptable. 

The only reason this data is going into ArcSDE is to maintain legacy ArcIMS applications.  I used the JPEG2000 compression to save some disk space on the new system but I suspect that is where the slow down is occuring.  Any thoughts?  Thanks!

[VinceAngelo]

I doubt this is JPEG2000's fault.  Instead look to your tile size -- 8192x8192 is *way* too large.
The *compressed* size of a single tile shouldn't exceed 8-16K; starting from 64Mb (assuming
8-bit data) is not the way to get  there.

You can see how large your tiles really are by doing a SQL query like --
     SELECT avg(datalength(block_data)/1024) AVGkb FROM SDE_blk_23

Generally speaking, anytime you need to increase the default 2Mb RASTERBUFSIZE giomgr.defs
parameter you will usually hurt raster performance. 

I use 96x96 tiles with LZ77 compression, and let the 128x128 default stand with lossy (no more
than 256x256 with heavy compression).

- V