Sunday, April 26, 2009

plasma and memory usage

A long time bane of my Plasma life has been dealing with it ballooning in memory usage depending on the configuration.

Some of the leaks turned out to be related to certain graphics drivers, but that wasn't the majority.

Some were bona fide leaks in the Plasma code base, such as one I found and fixed the other day that, when the job tracking was turned on in the system tray, would result in a small allocation for each and every job (e.g. file transfer) started. This one wasn't easy to find because the allocations weren't huge and they were all parented QObjects meaning valgrind wasn't being very helpful. We've found and fixed a number of leaks of various sizes and things are feeling pretty good ... but there was still problems.

There was a bug report about memory usage jumping when the digital clock was set to update every second. This is a great example of a report that describes both the problem as well as a very well defined way to reproduce it. Now, I don't personally use a clock that updates every second, and Plasma tries its best not to update too much .. but when I ran `plasmoidviewer digital-clock` with it set to show seconds I watched the memory usage skyrocket with great sadness. :)

After hunting around in the clock code, in libplasma and in libplasmaclock for a number of hours I finally thought, "Maybe this isn't my bug after all." So I whipped up a small Qt only test app and was able to trigger the same problem in just a handful of lines of code. It turns out it's a problem with the caching in QGraphicsScene: it inserts pixmaps but never removes them from QPixmapCache. That shouldn't be a problem, however, since the cache should delete pixmaps when it gets too big and delete pixmaps when inserting using the same key. Unfortunately, for some reason, that isn't happening. By inserting a QPixmapCache::remove before both of the calls to QPixmapCache::insert in QGraphicsScene, the pixmap leaking stopped immediately.

I'm not sure that's the right fix, however, but at least I have an idea now of where we need to be looking. That it isn't in Plasma is nice (yay! not my fault!) but also a bit frustrating since it's slightly easier for me to fix things in Plasma. ;)

In any case, with all these patches applied on my system, a full on Plasma session is currently taking up 17,832k in resident memory and using 302 pixmaps that total 5904k in size on the X server. That's a total of 23736k of app and pixmap memory for my desktop, panel, widgets and wallpapers running using a full debug build.

Earlier tonight my Plasma had well in excess of 1000 pixmaps after running for not a very long time at all. Glancing at xrestop Plasma now has 292 pixmaps, or 10 less than when I wrote that last paragraph, so it certainly seems to not be leaking now. ;)

Yes, definitely headed in the right direction ...

282 pixmaps.... whee.

Update: Alexis just replied to the bug saying he'll be working on fixing this issue first thing on Monday. I just love working with the Trolls from Qt .. :)

9 comments:

Javier said...

Wow, that's an amazing improvement!
Great to see the pieces starting to fall together :]

maninalift said...

The number of pixmaps keeps dropping? Perhaps you have a leak of anti-pixmaps, the future of the universe could be in jeopardy.

vespas said...

this is excellent! the improvement in kde4 is astounding... maybe there could sometime be a 'performance sprint' as a direct goal for kde to focus on optimisations?

hosein-mec said...

Fantastic

Ian Monroe said...

Great to know you're working on this and fixing it. I've been having to restart plasma every few days.

notanavragelosr said...

Great to hear! I loves me some slim-and-trim KDE, especially on my dinosaur of a rig.

Did the Twitter plasmoid get its memory leak plugged? I used it for a bit and saw it devouring my RAM if left unchecked (left it up for 6 days straight and Plasma's memory usage was over 1GB! closed the applet and all that RAM came back). I started using Choqok when I noticed that... and completely forgot about filing a bug report until now. Whoops.

Aaron J. Seigo said...

@notanavragelosr: yes, i fixed those a while back.

Enderandrew said...

I am responding here to an older post which may not be noticed, but hopefully it will. Friends and family often turn to me for IT/computer assistance. I often volunteer to do free Linux installs for anyone I know. I have been installing openSUSE 11.1 and KDE 3 for people new to Linux for years, and lately I've decided to do KDE 4 installs for people. Sometimes I am doing these installs on netbooks or old computers with little memory.

Overall, a KDE 4 session is taking considerably more memory than a KDE 3 session, and I think part of that is that I can't seem to completely disable aspects like Nepomuk and Akondi, even for users who basically only use Firefox. If I go into System Settings and "disable" Nepomuk, I still see a process running in the background eating up a lot of memory.

Small things like this keep driving me back to KDE 3 installs for people who largely just operate out of a web browser.

Peter said...

I'm aware that this is a pretty crummy bug report, but I often have to kill and restart plasma as the memory usage according to htop skyrockets. As an example, just now it was using 40% memory on a 4GB system.

After killing it and restarting it, htop now reports 1.5% for plasma.