Vidar is a fairly geeky guy who enjoys writing and using nifty software. He hopes this blog will present new niftyness to you.
I represent a company for which I do some technical blogging for. I’ve stumbled upon your LINUX ATE MY RAM! page from the social networking sphere, and thought I’d ask permission to blog about this and link to your site.
I’ve encountered newbies hitting the same roadblock, wanted to explain the common misunderstanding and link to your site at the bottom of my blog post.
Is that okay with you?
Absolutely! Anything that increases awareness of the wonders of disk cache has my support.
Hi. Thx for your program : http://www.vidarholen.net/contents/junk/mandel.html
I have tried to run it. After make I have :
gcc -O9 -o mandel mandel.c -lpthread -lm -g -std=c99 -Wall -Werror
cc1: warnings being treated as errors
mandel.c: In function ‘calc_pixel’:
mandel.c:292: error: ‘drgb’ may be used uninitialized in this function
make: *** [mandel] Błąd 1
Can you help with it ?
My gcc 4.3.4-6 from debian doesn’t complain, but yours is right. Try simply editing the makefile and removing the word “-Werror” on line 4. Does that help?
Yes. It works fine. I have AMD turion 64 X2 and Ubuntu 64-bit and 2 GB RAM. What number of threads should I use ?
2 should be fine.
You are right. ( Why you use -j 3 ? ).
I have changed one function
and it works 2 time faster.
Congrats for your blog! I’ve been reading your “linux ate my ram” mini-blog 🙂 Everythin ok, but I have one question. I have a server with 4GB ram, and memory monitored. I’ve observed that it always have enough memory (+/- buffers/cache line) with some graphs, but there are certain moments that the OS has needed a bit of swapping (only a few megs) but the free memory is always high. I’m pretty sure cause I generate memory graphs, and the amount of free memory is always high. What do you think about it?
The kernel can be configured to swap out unused applications to make room for more disk cache to speed up applications in use. The degree to which this is done is called ‘swappiness’, configurable in /proc/sys/vm/swappiness.
If swappiness is 0, the kernel will never swap out unused apps unless it’s completely out of RAM. If it’s 100, the kernel will frequently swap out unused apps to make room for disk cache to speed up in-use apps.
The default in Ubuntu is 60, so there you will sometimes see some swap use even with plenty of free memory.
Also, once something is swapped, it will just sit there until the memory is modified, freed or something else needs the swap space. This means that trace swap usage can be caused by a short memory spike long ago and/or of short duration (maybe shorter than your logging tool).
This is a sign of Linux’ effectiveness in not wasting work that has gone into writing pages to swap.
Fantastic Vidar. Great explanation 🙂
I really like your linux memory caching page and was wondering if I could upload it to and publish it to an internal network? I would of course credit your site.
Sure, anything to spread the page cache love!
I want to ask you if I could translate your linuxatemyram.com site to spanish and post it on a public forum?.
Regards and thank you for your nice explanation.
Sure! Just make sure to link to the original.
The bottom section of http://www.vidarholen.net/contents/junk/nicotine.html contains an error. A cigarette does not contain about 7mg of nicotine, but about 0.7mg; mine contains 0.6mg per cigarette. An aubergine has an average weight of 300g, so one aubergine contains about 30µg nicotine, which is 0.03mg nicotine; 0.6mg / 0.03mg is 20, so 20 aubergines holds the same amount of nicotine, as one of my light cigarettes does.
Congratulations for your https://github.com/koalaman/shellcheck! Is there any way to choose (enable/disable) the checks the shellcheck script will use? Is it possible (easily) to add additional checks if needed?
Thanks in advance,
For the latest, local git version, you can use shellcheck --exclude=SC1234,SC2345 file to exclude certain types of checks.
shellcheck --exclude=SC1234,SC2345 file
Any new checks requires modifying the source code. It’s conceptually very simple, and most of the checks are under 15 lines of code, but it assumes familiarity with Haskell and I’m sure the a lot of the utility functions seem quirky and obscure. If you’d like to try, you can just browse the list of commits and see the diffs for adding new checks.
Thanks for your prompt response Vitar.
I was able to clone, build and execute shellcheck using the exclude option.
Have you already thought in create a SonarQube (http://docs.codehaus.org/display/SONAR/Plugin+Library) plugin for that?
I’m not familiar with SonarQube and I have no plans to integrate directly with it. Maybe you can use ShellCheck’s CheckStyle XML output and SonarQube’s CheckStyle plugin to bridge the gap?
I think you’re slightly wrong on linuxatemyram. There is an indirect claim on the main page, and a pretty direct claim on the play.html page that caches/buffers won’t ever cause an application to swap. This is incorrect though.
Linux will swap out *unused* pages from applications to keep some disk caches alive, if that is possible. The threshold is controlled by the sysctl vm.swappiness. I assume that you cannot see this effect with munch.c, because it is actually active on most of its pages, thus it won’t get swapped.
This gets pretty important on desktop systems. The default of vm.swappiness=60 makes linux to start swapping application data at 40% RAM usage, if it sees it fitting. On desktops, this often means to swap out the file manager or any other currently unused application (web browser, email client…) which has to be swapped in first.
Vidar – Thanks for the linuxatemyram site….I work with many people new to linux and its very effective in giving them a basic understanding of how memory looks in linux.
I was reading through your explanation of the Linux Page Cache (http://www.linuxatemyram.com/), great article!
Now I’m trying to understand the behaviour of an application as it relates to the Linux Page cache, the situation is:
– Oracle 11g claims to be able to reserve/pre-allocate RAM on startup of a DB instance, so if you set the sga_max_memory parameter to some value in GiB, “X” it will, in theory, reserve this memory.
– I assume this means that when Oracle db starts up, X GiB will be removed from the Linux Page Cache, and this should be reflected in the output of “free -m”.
– When the oracle DB is shut down, I presume the memory would be released, and the Linux Page Cache should grow by “X GiB”.
However, when I do this test on a Linux system (Standard CentOS 6.5), I don’t see the pre-allocated amount of memory being moved out of the Linux Page Cache …
So based on this 3 possibilities come to mind:
a. Linux free command does not reflect memory that’s actually been reserved by the application (in which case “free” is misleading)
b. Linux will not release memory from Page Cache unless the application is “actively” using it … and ignores the application’s attempt pre-allocate memory. Though, in this case, I’m not sure how Linux knows the difference between an application pre-allocating memory, and “actively using it” … I would think they look the same from the Kernel’s PoV.
b. Oracle sga_max_memory does not actually pre-allocate and reserve memory on Linux upon instance startup. It’s still available until actually demanded by the instance. If this is true it would seem AMM/ASMM effectively does not work on Linux for oracle 11g.
c. Some misconfiguration of Oracle AMM/ASMM on Oracle 11g resulting in automatic memory management not really being enabled.
I’d appreciate any thoughts you have on this!