Memory not realeased

[JeanLonneux]

Hi, so i create some graphics and append them in overlay. When i hit 10000 graphics i delete them all and clear overlay yet the memory is not realeased.

Sample:

Simple::Simple(QObject* parent /* = nullptr */):
    QObject(parent),
    m_map(new Map(Basemap::imageryWithLabelsVector(this), this))
{
    overlay = new GraphicsOverlay(this);

    const QString styleLocation = "whatever";
    m_dictionarySymbolStyle = DictionarySymbolStyle::createFromFile(styleLocation, this);

    overlay->setRenderer(new DictionaryRenderer(m_dictionarySymbolStyle, this));
}

void Simple::addGraphic()

{
    Geometry geom;
    SpatialReference sr(4326);
    geom = Point(0, 0, sr);

    QVariantMap symbol;

    Graphic *graphic = new Graphic(geom, symbol);

    overlay->graphics()->append(graphic);

    if (overlay->graphics()->size() >= 10000)

    {
        qDeleteAll(*overlay->graphics());
        overlay->graphics()->clear();
    }
}

[JamesBallard1]

Hi @JeanLonneux .

   What OS are you running on? We've looked into similar reports in the past and found that the memory analyzing tools and OSes vary on how they report freed heap allocations. For example the same app would not drop the memory usage on Linux via top, but it would on Windows when checking the task manager.

Also, if you can provide a bit more code I can try it on our end as well. I want to be sure I'm replicating the same workflow when I test it out. For example, what is the symbol you're adding? Are you getting that from a call to searchSymbols or fetchSymbol?

It's hard to tell from the code provided, but can you double check that your symbol is also getting deleted when you delete the graphics?

[JeanLonneux]

Hi James thanks for the reply.

I'm currently using linux, tried some tools like top, htop, pmap and smem they all gave me the same result the memory goes only up not down.

I'll attach the sample project, hope that this give you some more information. And please feel free to make observations and corrections to this sample.

[JamesBallard1]

Hi @JeanLonneux ,

I've debugged the problem for a while on our end and I can see that there is definitely a new issue with 100.11. I have a potential workaround you can try for now. If you're able, can you test to see if you can reproduce the problem on your end with 100.10? I suspect you won't be able to, but I want to confirm I am narrowing in on the same thing you're seeing.

I modified your test app locally to create thousands more graphics, and much faster in order to exacerbate the problem. In my local test this is enough to trigger a leak:

void Process::dataIn(QString msg)
{
    for (int i = 0; i < 1000; i++)
    { // just create/delete a bunch of graphics
        std::make_unique<Graphic>();
    }
}

As a workaround with 100.11, can you try creating the Graphic objects in either the main thread, or any thread that will have a longer-lived event loop? We have some internal data structures that only get cleaned up when some internal objects emit the destroyed signal. As a quick test, if I shift that code to this part of the test app, the problem goes away entirely:

void Controller::process()
{
  for (int i = 0; i < 1000; i++)
  {
    std::make_unique<Graphic>();
  }

    // Create Thread to draw symbols
    QThread *thread = new QThread(this);
    // Create worker to draw
    Process *worker = new Process(my_Map);
    worker->moveToThread(thread);

    connect(this, SIGNAL(callProcess(QString)), worker, SLOT(dataIn(QString)));
    connect(thread, &QThread::finished, worker, &Process::deleteLater);

    thread->start();

//    callProcess("test");
}

The difference is that the "process" function happens on the main thread.

[JeanLonneux]

I've tested in version 100.10 with that

    for (int i = 0; i < 1000; i++)
    { // just create/delete a bunch of graphics
        std::make_unique<Graphic>();
    }

and it doesnt crash like in 100.11 but still can see an little increase in memory (perhaps some issue in the way im using qt) by the way qt version 5.15.2.

[JamesBallard1 · Accepted solution]

@JeanLonneux thanks for testing that. You didn't mention that 100.11 was crashing. Are you hitting a crash with 100.11?

As previously mentioned the Linux memory tools weren't reliable for me with previous tests to give an accurate representation of heap memory.  Here's a snippet of test code that contains no Runtime code at all to test this out.

If you create a Qt Widgets example app, you can drop this code into the constructor and run it. All it does is allocate and free a large amount of memory. 

{
  ui->setupUi(this);

  {
    qDebug() << "Allocating memory";
    std::vector<std::unique_ptr<double>> temp;
    constexpr std::uint64_t loop = 100'000'000;
    temp.reserve(loop);
    for (std::uint64_t i = 0; i < loop; i++)
    {
      temp.emplace_back(std::make_unique<double>());
    }

    qDebug() << "Deallocating memory";
  }

  qDebug() << "Deallocating memory - done";
}

For example when I ran that test code on Windows I can see memory shoot up and drop back down, which is what I would intuitively expect. However when I ran that code on Linux, top and other tools would not show any reduction in the app's memory usage. I've included a sample test project you can use to validate this on your end. I am interested to find that your memory tools are reported this sample increases, and then reduces used memory as expected.

[JeanLonneux]

James the crash started after i inserted that loop to create/delete a bunch o graphics, the memory usage go skyrocket and the application crashes. You are right about Linux tools, just tested it. Probably some Linux heap allocation behavior (i need to read more about that).

Anyway ill keep testing, probably will keep version 100.10 for now.

[JamesBallard1]

Thanks @JeanLonneux for the update.

If you are still seeing issues with 100.11 do let us know so we can look into it. For example, if you run your unaltered app with 100.10 vs 100.11 and you are seeing 100.11 use more memory and continuously rise, please let us know.

Thanks.