Skip to content

Changes the memory allocator to use jemalloc to address memory leak issues #3566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

bigbruno
Copy link

@bigbruno bigbruno commented Jun 6, 2025

In the past few days, I tried several approaches to resolve memory leak issues in Nemo. I have a folder with 29,000 subfolders that I was using for testing. Scrolling quickly through this folder causes memory usage to exceed 1 GB.

Even after leaving the folder, the memory consumption remains high. However, when using jemalloc, switching to another folder brings memory usage back to normal, around 50 MB.

I used this folder with a huge number of subfolders to make testing easier, but folders with a thousand files are already sufficient to identify the issue.

@mtwebster
Copy link
Member

Hi, this is very interesting, but I get the impression jemalloc is no longer actively maintained, perhaps there are other alternatives that would bring a similar benefit?

Also, does this have any performance cost?

For a quick check you can do:

NEMO_BENCHMARK_LOADING=1 nemo /path/to/lots/of/files

I usually also clear the disk cache first before I run this:

sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

Thanks!

@leigh123linux
Copy link
Contributor

Hi, this is very interesting, but I get the impression jemalloc is no longer actively maintained, perhaps there are other alternatives that would bring a similar benefit?

Maybe mimalloc, jemalloc is archived.

@bigbruno
Copy link
Author

bigbruno commented Jun 6, 2025

Hi, this is very interesting, but I get the impression jemalloc is no longer actively maintained, perhaps there are other alternatives that would bring a similar benefit?

Maybe mimalloc, jemalloc is archived.

I don't know why they keep the repository as archived, but it is constantly receiving updates, with the last updates made 4 days ago and in the Issues there is information that a new stable release should be released this semester.

@bigbruno
Copy link
Author

bigbruno commented Jun 6, 2025

Hi, this is very interesting, but I get the impression jemalloc is no longer actively maintained, perhaps there are other alternatives that would bring a similar benefit?

Also, does this have any performance cost?

For a quick check you can do:

NEMO_BENCHMARK_LOADING=1 nemo /path/to/lots/of/files

I usually also clear the disk cache first before I run this:

sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

Thanks!

Performance Benchmark Results

Test 1: Without changes or clean cache

Run Nemo Startup Time Folder Load Time Idle Folder Load Time
1 5.694530s 5.495440s 5.665437s
2 6.274312s 6.070279s 6.353217s
3 5.990715s 5.764852s 6.031537s

Test 2: With JEMALLOC without clean cache

Run Nemo Startup Time Folder Load Time Idle Folder Load Time
1 5.791489s 5.584155s 5.772395s
2 5.565344s 5.366286s 5.517763s
3 6.266869s 6.066841s 6.367481s

Test 3: Without changes and CLEAN cache

Run Nemo Startup Time Folder Load Time Idle Folder Load Time
1 6.564831s 6.269911s 6.702799s
2 6.714676s 6.405399s 6.570017s
3 6.444123s 5.983357s 6.298770s

Test 4: With JEMALLOC and CLEAN cache

Run Nemo Startup Time Folder Load Time Idle Folder Load Time
1 6.463416s 6.191013s 6.347061s
2 6.327779s 6.042773s 6.428277s
3 5.940651s 5.655058s 5.812127s

It seems to me that there is no significant difference in performance, but the small difference seems positive for Jemalloc.

@bigbruno bigbruno force-pushed the fix-memory-leak branch 2 times, most recently from 109476c to 7eaf1a8 Compare June 8, 2025 08:18
bigbruno added 2 commits June 12, 2025 10:27
…stem and at compilation time you can force it to be activated or not.
@bigbruno
Copy link
Author

Includes some changes to avoid compatibility issues.

When compiling nemo, if meson detects that jemalloc is installed, it will automatically use it in nemo, in addition to having the option to enable or disable it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants