“Shift” tool has performance issue,

3850
27
Jump to solution
07-20-2019 12:37 AM
JamalNUMAN
Legendary Contributor

“Shift” tool has performance issue,

 

I observed that the “Shift” tool has performance issue. Is it normal that a raster of 113 MB takes around 4 minutes in order to be shifted (-50000, -500000)? My machine is with 4 Cores X 64GB RAM

 

Is there any recommendation to increase the performance of this tool?

----------------------------------------
Jamal Numan
Geomolg Geoportal for Spatial Information
Ramallah, West Bank, Palestine
1 Solution

Accepted Solutions
DanPatterson_Retired
MVP Emeritus

yes... time for some scripting to

  • read the *.pnw, rename them *.pnw_old,
  • get the appropriate lines containing the X, Y values,
  • do the math and
  • save the file out to a new copy so as to retain the old.

You wouldn't want to do this manually... even though it would be quicker than shifting.

 

View solution in original post

27 Replies
DanPatterson_Retired
MVP Emeritus

Shift—Data Management toolbox | ArcGIS Desktop  probably the same in arcmap

but anyway....

you didn't specify a file extension, so it isn't the shift that is taking a long time, but it is the conversion to an esri grid in the ZZ folder.

If you want to shift specify and output type by giving the raster an extension like *.tif... (ecw isn't on ArcGIS Pro's list of output types)

JamalNUMAN
Legendary Contributor

Many thanks for the input Dan.

 

I tried it with jpg extension. it took around 2 min and 15 sec.

 

I observed that as the ecw file (113 MB) is shifted with jpg extension, then ovr file is created with a size of 150 MB. This means that the 113 ecw file is ending up with 135 MB for jpg in addition to 150 MB for the ovr. This means that the size is tripled just to shift the raster!

----------------------------------------
Jamal Numan
Geomolg Geoportal for Spatial Information
Ramallah, West Bank, Palestine
DanPatterson_Retired
MVP Emeritus

try another format... esri usually recommends *.tif

JamalNUMAN
Legendary Contributor

Now the size of the tif and ovr are much bigger: 1.5 GB for the tif and 350 MB for the ovr

----------------------------------------
Jamal Numan
Geomolg Geoportal for Spatial Information
Ramallah, West Bank, Palestine
DanPatterson_Retired
MVP Emeritus

did you try convert first, then shift? when you convert, you can specify the compression etc

JamalNUMAN
Legendary Contributor

I have converted ecw to tif and then the tif is shifted:

 

-The size of tif file is much bigger than ecw

-When shifting tif, the size stays the same (as expected)

----------------------------------------
Jamal Numan
Geomolg Geoportal for Spatial Information
Ramallah, West Bank, Palestine
DanPatterson_Retired
MVP Emeritus

In the raster to other format tool... did you go to the environments tab and check the compression being used?

Compression (Environment setting)—Geoprocessing | ArcGIS Desktop 

You can experiment with compression types and bit settings.  You haven't indicated what bit type the input data are but I can assume 8-bit unsigned from this

Raster file formats—ArcGIS Pro | ArcGIS Desktop 

But since ecw is proprietary, then you will have to go to something else.  tif will be larger, *.png is smaller and lossless (I used it for all imagery/graphics regardless whether for gis or not).

So if raster size is the issue, you are fighting a lost cause... SID and ecw are the squished choices, tif, png are widely used and jpeg has had its day

JamalNUMAN
Legendary Contributor

I’ll be trying to go for the png option as it is better than other formats in terms of size. However, the performance stays low as one raster needs around 4 minutes to be converted and shifted.

 

This means that the 1,600 rasters need around 4.5 days to be converted and shifted

----------------------------------------
Jamal Numan
Geomolg Geoportal for Spatial Information
Ramallah, West Bank, Palestine
DanPatterson_Retired
MVP Emeritus

Was it a projection issue Jamal?

If it was, I am wondering if a batch reprojection wouldn't be better.

Also, the png format produces a world file. it might be faster to just go and edit it in a text editor (or via a script) to just adjust (aka, shift) the coordinates there.  If the coordinates are embedded in the file then you can't do this, but if there is a world file, it can be done.

why don't you zip and post all the bits of the *.png file except the png… it the *.aux, *.pnw, *.ovr etc etc