Issues clipping large ecw imagery

[KevinBowden]

Is anyone having success clipping ecw imagery files?  ARC keeps 'crashing' on me, though at least now it sends an error report.  I need to clip files that will end up being around 20GB.  It will clip less than 1GB easily, but nothing over that size.  ARC also crashes when I try to import these raster datasets into a file geodatabse, and crashes again when I try to convert into a TIFF format.

cheers Allison

[RobertBerger]

Hi Allison,

What format are you trying to output to? And what version are you using? If your output is tiff and you use 9.3 or previous, then be aware that tiff has a size limit (I think 4GB). What is the "uncompressed" file size in the ecw dataset properties?
Thanks.

Robert

[KevinBowden]

Thanks Robert.

I've tried to export as JPG, TIFF, ECW, IMG files.
Am using v10 GIS, Windows 7, 64bit OP system, my virtual memory is set to 50GB.
6GB RAM, Quad Core CPU.
Graphics card mid-range - SyncMaster on NVIDIA GeForce GT240 with 1024MB of memory.  I'm wondering if the graphics card is part of the problem?
I tried exporting using v9 on laptop & 2x good desktop computers.  Nothing....
The compressed original file size of the ecw is 5GB, but when I export, it says that the new file size will be 20GB. 
I also tried to import the raster dataset into a geodatabase - GIS crashes.
Tried to convert the ECW into another format - still nothing.
I've ruled out the imagery as being the problem because I tried the same process using older imagery that's worked fine before (albeit a smaller dataset to begin with).

Is it the graphics card?  Or something else?

Allison

[MichaelStead]

Try png. I think it will allow the big filesize, you are going to end up with a massive file though. I think if you download the ecw sdk libraries and install GDAL independently you can do this in GDAL (from command line) and keep the ECW format. There is a note in the read me about licensing and files over 500MB so I think this may require $$ too.

I am not sure what the limitations are in Arc now, but previously you needed some additional licencing from ERDAS for files bigger than some limit (that you always have exceeded if you are looking at compression formats). I think the same is true of sid files.

[JohnSobetzer]

ArcGIS 10 has had issues with ecw file formats so maybe this is another.  Having said that, a 5GB ecw, sid or jp2 at typical 1:20 compression, is a huge file, and probably has to be decompressed to the full 20 GB size before it can be processed.  That may just be too much for the computer and/or software.  If you can get it to display in a project, you might: 1) use a clip option in the data frame properties to limit its display, 2) create a masking polygon that you place over it in the TOC to limit its display, 3) export directly to a preferred file format with georeferencing from the view, perhaps in smaller pieces that can be mosaiced, to provide enough resolution, or 4) export from a layout to a preferred raster format and then georeference that.

You might also try other programs like the LizardTech Geoviewer (free) and see if it can save to a different format.

[GordonSumerling]

Allison,

I think the reason you are getting the problem is the ECW ddl's being used by ArcGIS does limit the size of ECW that can be clipped. Also because ECW is proprietary format ArcGIS is not allowed to write out in that format. I think you will encounter the problem with any application apart from an ERDAS application as they all use the same dll set for ECW's

My suggestion would be to read the ECW into the new geodatabase Mosaic format. Then adjust the boundary of the mosiac. This will not touch the image itself but will limit the display area of the mosaic. It is in effect like a clip so it will look correct in ArcGIS. You can then perform processing and analysis on the image and it will only be inside the boundary of the mosaic.

Reagards
Gordon

[KevinBowden]

Thankyou everyone for your replies and helpful hints....all good to know for the future.  It turned out that there was an issue with how the ecw was originally processed before we even received it. 

The imagery issue is also registered as a bug with ESRI - NIM040374 & NIM064260.

ERDAS Imagine is not supporting ecw format anymore which is also contributing to the problem. 

v10 can read imagery over 500GB and clip it, but in this case, there was an error with the imagery itself.

Allison