I am working on a large program which is always up and must run in as many Linux flavors as possible, i.e. be portable. I have created a number of general utility functions for common tasks it needs to do, and one of these utility functions goes out to the shell to give a Linux command and return the output from the command. Depending on the exact Linux command being performed, occasionally it can drain free memory and not free it immediately.
In one case, I need to call this function to give a shell command and get data back a hundred or so times in a loop, which executes in just a moment. I find that during that loop my free memory decreases dramatically. I presume that it's freed at some point, but during execution of the loop, it can get quite low, and I think it may have caused a program abend in one environment. If you have read this far and are wondering, the command being given is to determine if a service is up, and it does this for every service which has script in /etc/init.d. The command, for bluetooth, for example, is:
service bluetooth status
My question is specifically this, is there anything I can do to my code in this function so that it either uses less memory of frees it better. Here is the code:
int GeneralUtilities::ExecSystemCmd(string cmd, vector<string>& commandOutput)
{
FILE *fp = NULL;
char commandOutputLine[5000];
int numLines = 0;
try
{
cmd += " 2>&1"; // Get stderr. This may not be sufficiently portable.
/* Open the command for reading. */
fp = popen(cmd.c_str(), "r");
if (fp == NULL)
return(0);
/* Read the output a line at a time - output it. */
for (numLines = 0; (fgets(commandOutputLine, sizeof(commandOutputLine)-1, fp) != NULL); numLines++)
commandOutput.push_back(commandOutputLine);
/* close */
pclose(fp);
}
catch(...)
{
if (fp != NULL)
pclose(fp);
}
return numLines;
}
I have checked and the character array commandOutputLine[] has no bearing on this problem. How might I re-write this function to make it less of a memory hog or to make it free its memory at once? Thanks in advance.
Brandon
How, exactly, are you measuring "free" memory? It's a common beginner mistake to see "free" low and panic, but it doesn't usually mean you're out of memory -- "cached" is as good as free, so as long as "cached" is high, you've got plenty.
---------- Post updated at 04:55 PM ---------- Previous update was at 04:51 PM ----------
Another thing -- you're saving every single line to a structure in memory. When the output is enormous you'd be better off saving it to a temp file. If it's tiny, saving it to a temp file won't hurt performance because it'll all fit in cache.
Aside from that I don't see anything in this function which would be a memory hog. Depending on what command it's running though, that may end up converting lots of 'free' memory into 'cache' memory (again, harmless.)
I was judging free memory from:
grep MemFree /proc/meminfo
You are having the standard linux newbie memory freakout. We sometimes see a few of these a month. Take a deep breath and relax -- you're not running out of memory.
Memory that's otherwise sitting idle gets used for caches when there's anything to cache, because it's wasted just sitting around doing nothing. The kernel gives it up just as readily as any other unused memory -- you don't need to "free" anything or put your computer through weird convolutions to "release the cache". Everything is fine.
Just consider 'cache' and 'free' to get truer numbers on how much memory is available.
I took a look with "free -m" and the situation looked entirely different. Also, I found the true cause of that one system crash and it had nothing to do with memory. My program was going through the scipts in /etc/init.d and doing a "status" on them to see what processes were up. It turns out that in openSUSE 11.4, that directory contains a script called boot which isn't a startup script at all, but the first script run at system startup, which does a lot of very basic things like mounting file systems. So, when my program gave the command:
service boot status
it was simply ignoring the command line argument "status" and executing, causing my program to stop and often causing a crash of my whole virtual environment. It had nothing to do with running out of resources.
Thanks again, Corona.
Naturally it would, after the fact... But it reports the same information. You'll have to add free and cache.