RasterLayer deletion, possible file handle leak, and workflow

[JohnHouston]

I'm looking into an issue I'm having with Raster and RasterLayer. I periodically load a new RasterLayer every couple seconds and remove an old RasterLayer. See code below.

Esri::ArcGISRuntime::Raster* raster = new Esri::ArcGISRuntime::Raster(QString::fromStdString(filename), raster_group_layer_);
Esri::ArcGISRuntime::RasterLayer* raster_layer = new Esri::ArcGISRuntime::RasterLayer(raster, raster_group_layer_);
QList<QColor> colors = ...
Esri::ArcGISRuntime::ColormapRenderer* colormap_renderer = new Esri::ArcGISRuntime::ColormapRenderer(colors, this);
raster_layer->setRenderer(colormap_renderer);
raster_layer->setLayerId(QString::fromStdString(guid.toString()));

Esri::ArcGISRuntime::Layer* old_layer = findLayer(raster_group_layer_->layers(), guid.toString());

raster_group_layer_->layers()->removeOne(old_layer);
raster_group_layer_->layers()->append(raster_layer);

The behavior I observe is leaked open file handles. When I call "lsof -p <pid>" on my process I see the same tif open repeatedly; this grows as time goes on. I am certain these open file handles are from the code above (I commented it out and the file list did not grow). The file handle list grows until eventually the process dies due to too many open files.

• What is the recommended way to delete a Raster and RasterLayer? I've tried:

delete raster_layer;

and

group_layer_->layers()->removeOne(raster_layer);

I suspect the latter is correct since it does appear to remove the image from display.

• Do you observe similar behavior with RasterLayer file handle leaks? Reproduction of this should be straightforward, let me know if it isn't and I can invest in a fully reproducible example.

Also some other questions related to my workflow. Basically I have some sensor raster data in memory I want to throw onto the map. It changes frequently.

• Is there a way to just hand in a QImage and transform to the RasterLayer directly? As a workaround I write a tiff to memdisk, then create a new RasterLayer on each update. As you may guess this is expensive and hacky.
• Is there a way to simply say, "reload the file" or "load this other new file" so I don't need to make a new RasterLayer each time?

Anyways thank you for looking at this, any suggestions are welcome!

Edit: confirmed happening on v100.9 and v100.11.

[Tanner_Yould · Accepted solution]

Hi @JohnHouston, thanks for reaching out! I have a few suggestions that you may want to check out.

We have a sample that demonstrates how to animate images in a scene (take a look at the "Animate images with image overlay" sample code here). The ImageOverlay class only works in 3D scenes right now and not in 2D maps. Even if you're using a map, it may be worth taking a look to see how we create Image objects and then pass them to the ImageOverlay and then replace them without introducing a memory leak. 

You can also try an RAII technique by doing something like setting a specific parent object for all the objects you want to delete periodically.

Here's a bit of pseudocode that demonstrates the idea.

// assumes std::unique_ptr<QObject> m_parent is a member of the class...
Esri::ArcGISRuntime::Layer* old_layer = findLayer(raster_group_layer_->layers(), guid.toString());
raster_group_layer_->layers()->removeOne(old_layer);
// assumes C++14 support
// this causes all previous objects with this parent to destruct
m_parent = std::make_unique<QObject>();
Esri::ArcGISRuntime::Raster* raster = new Esri::ArcGISRuntime::Raster(QString::fromStdString(filename), m_parent.get());
Esri::ArcGISRuntime::RasterLayer* raster_layer = new Esri::ArcGISRuntime::RasterLayer(raster, m_parent.get());
QList<QColor> colors = ...
Esri::ArcGISRuntime::ColormapRenderer* colormap_renderer = new Esri::ArcGISRuntime::ColormapRenderer(colors, m_parent.get());
raster_layer->setRenderer(colormap_renderer);
raster_layer->setLayerId(QString::fromStdString(guid.toString()));
raster_group_layer_->layers()->append(raster_layer);

I hope this helps and don't hesitate to follow up if it doesn't, or you have further questions. Thank you!

[JohnHouston]

Thank you @Tanner_Yould for the detailed and thoughtful reply. I like your idea of RAII and deleting the parent explicitly via the unique_ptr in order to ensure the Raster destructor is called (it also prevents a leak with the colormap renderer I missed before). I gave your suggestion a shot but however I still see the file handle leak:

john@john-ubuntu:~/-----$ pidof ---- | xargs lsof -p

...

---- 155580 john 185r REG 0,27 51804 176 /dev/shm/a6da76fc-74ab-4fa9-9854-b207f04ff162.tif
---- 155580 john 186r REG 0,27 51804 176 /dev/shm/a6da76fc-74ab-4fa9-9854-b207f04ff162.tif
---- 155580 john 187r REG 0,27 51804 176 /dev/shm/a6da76fc-74ab-4fa9-9854-b207f04ff162.tif
---- 155580 john 188r REG 0,27 51804 176 /dev/shm/a6da76fc-74ab-4fa9-9854-b207f04ff162.tif

(process name in the listing changed to ----) In any case I'll check out the animated raster example you referenced and give that a shot next. Thank you again, I'll let you know here how it goes.

[JohnHouston]

OK I switched over to scene and used ImageOverlay, which worked great. Thank you again.

[Tanner_Yould]

That's great, I'm glad it worked out!