Extract By Mask tons of satellite imagery (rasters) with different cloud mask using codes?

[ZoeChen]

Satellite images are stored in many folders named with different dates. Right now, I am using arcgis desktop 10.4.1 Extract By Mask (batch) to remove clouds. Each time of batch processing, I can only finish one folder's images ( which are 6 images in total) because different folders have different masks. The speed is way too slow because I have more than 500 images waiting for me to extract. Is it possible to do it with a loop?? Any ideas?? Thank you!

[DanPatterson_Retired]

background processing set?

Have you tried it in ArcGIS Pro?

Do you have a machine with lots of memory?

Does multiple instances of ArcMap or Pro work if you have lots of memory?

PS... you posted this as a Discussion, rather than a Question... was a Discussion your intent or are you looking for a solution?

[ZoeChen]

Thank you for replying! I am pretty new here so I did not even know I posted my question as a discussion. Definitely looking a solution here. I haven't tried it in arcgis Pro yet because I'm more familiar with desktop arcgis. And, yes I do have machines with lots of memory. 

[GünterDörffel]

Zoe,

to me this reads as if the images in one folder would belong together and ONE mask is to be used per folder?
PLEASE learn about MosaicDatsets - as there you can dynamically apply masks - and even have useful Python Raster functions available (see here) for masking. Together with MDCS (see here) you could easily automate the whole process. 

Regards
Guenter

[ZoeChen]

Thank you for replying! Actually i will make the work to be very simple here by adding all the rasters into one single folder and use extract by mask function in a loop. The hard part here is to figure out how to loop through the code with different masks. If I can figure out the code, I will post it! 

[ZoeChen]

By the way, the reason I do not use mosaic dataset function is because my satellite images are all located at one single location but with different dates. My professor said it would work if those images are located in different areas. Anyway thanks! Also I am trying to improve my python skill too, not very good at it so trying to make the work as simple as possible. 

[CodyBenkelman]

As Guenter said, please use the Mosaic Dataset, and look for the proper Raster Type of your satellite imagery.

What is a mosaic dataset?—Help | ArcGIS Desktop 

 If you create a Mosaic Dataset (just an empty container for many images) then use "Add Rasters" and choose the correct raster type, you can load all of your images at one time.  I presume you have a Cloud Mask band for every multispectral scene, and that can be used with the Mask function, applied to all multispectral scenes in the mosaic dataset.

Mask function—Help | ArcGIS Desktop 

[ZoeChen]

Thank you for replying! Actually i will make the work to be very simple here by adding all the rasters into one single folder and use extract by mask function in a loop. The hard part here is to figure out how to loop through the code with different masks. If I can figure out the code, I will post it! 

[ZoeChen]

By the way, the reason I do not use mosaic dataset function is because my satellite images are all located at one single location but with different dates. My professor said it would work if those images are located in different areas. Anyway thanks! Also I am trying to improve my python skill too, not very good at it so trying to make the work as simple as possible. 

[GünterDörffel]

DATE Support is a mayor part of MosaicDatasets – as a very simple example: Are you aware of the LANDSAT GLOBAL SERVICES for ArcGIS? Find information here: http://www.esri.com/software/landsat-imagery

Those ARE time-aware multi-spectral MosaicDataset-based Image Services you can add as layers to ArcGIS!

Von: Zoe Chen

Gesendet: Thursday, August 31, 2017 4:19 AM

An: Dörffel Günter <g.doerffel@mysynergis.com>

Betreff: Re: - Re: Extract By Mask tons of satellite imagery (rasters) with different cloud mask using codes?

GeoNet <https://community.esri.com/?et=watches.email.thread>

Re: Extract By Mask tons of satellite imagery (rasters) with different cloud mask using codes?

reply from Zoe Chen<https://community.esri.com/people/zoexili?et=watches.email.thread> in Imagery and Remote Sensing - View the full discussion<https://community.esri.com/message/711969-re-extract-by-mask-tons-of-satellite-imagery-rasters-with-different-cloud-mask-using-codes?commentID=711969&et=watches.email.thread#comment-711969>