I am writing some code that uses MPI and I was keeping noticing some memory leaks when running it with valgrind. While trying to identify where the problem was, I ended up with this simple (and totally useless) main:
#include "/usr/include/mpi/mpi.h"
int main(int argc,char** argv)
{
MPI_Init(&argc, &argv);
MPI_Finalize();
return 0;
}
As you can see, this code doesn't do anything and shouldn't create any problem. However, when I run the code with valgrind (both in the serial and parallel case), I get the following summary:
==28271== HEAP SUMMARY:
==28271== in use at exit: 190,826 bytes in 2,745 blocks
==28271== total heap usage: 11,214 allocs, 8,469 frees, 16,487,977 bytes allocated
==28271==
==28271== LEAK SUMMARY:
==28271== definitely lost: 5,950 bytes in 55 blocks
==28271== indirectly lost: 3,562 bytes in 32 blocks
==28271== possibly lost: 0 bytes in 0 blocks
==28271== still reachable: 181,314 bytes in 2,658 blocks
==28271== suppressed: 0 bytes in 0 blocks
I don't understand why there are these leaks. Maybe it's just me not able to read the valgrind output or to use MPI initialization/finalization correctly...
I am using OMPI 1.4.1-3 under ubuntu on a 64 bit architecture, if this can help.
Thanks a lot for your time!
The OpenMPI FAQ addresses issues with valgrind. This refers initalization issues and memory leaks during finalization - which should have no practical negative impact.
There are many situations, where Open MPI purposefully does not
initialize and subsequently communicates memory, e.g., by calling
writev. Furthermore, several cases are known, where memory is not
properly freed upon MPI_Finalize.
This certainly does not help distinguishing real errors from false
positives. Valgrind provides functionality to suppress errors and
warnings from certain function contexts.
In an attempt to ease debugging using Valgrind, starting with v1.5,
Open MPI provides a so-called Valgrind-suppression file, that can be
passed on the command line:
mpirun -np 2 valgrind
--suppressions=$PREFIX/share/openmpi/openmpi-valgrind.supp
You're not doing anything wrong. Memcheck false positives with valgrind are common, the best you can do is suppress them.
This page of the manual speaks more about these false positives. A quote near the end:
The wrappers should reduce Memcheck's false-error rate on MPI
applications. Because the wrapping is done at the MPI interface, there
will still potentially be a large number of errors reported in the MPI
implementation below the interface. The best you can do is try to
suppress them.
Related
When i am running valgrind by adding a leakage to my code,I am getting leakage as still reachable for first allocation of block and then showing as definitely lost for 9 blocks.Possibly lost is showing due to other portion of the code. Why is this so ?
main()
{
........
char *ptr;
For(i=0;i<10;i++)
{
ptr=malloc(sizeof * ptr);
}
.....
}
Report:
HEAP SUMMARY:
==13832== in use at exit: 202,328 bytes in 62 blocks
==13832== total heap usage: 332 allocs, 270 frees, 283,928 bytes allocated
==13832==
==13832== LEAK SUMMARY:
==13832== definitely lost: 90 bytes in 9 blocks
==13832== indirectly lost: 0 bytes in 0 blocks
==13832== possibly lost: 202,180 bytes in 49 blocks
==13832== still reachable: 58 bytes in 4 blocks
==13832== suppressed: 0 bytes in 0 blocks
==13832== Rerun with --leak-check=full to see details of leaked memory
Adding to Florian's answer, here are a few examples of where you would have interior pointers
A memory manager. For instance, you know that you are going to be allocating very many small blocks of memory of the same size, so you write a memory manager that allocates large blocks and the subdivides them into your small size. These sub-block allocations are interior pointers.
A memory debugger. In this case, you allocate more memory than requested, and the extra memory is used to pad the allocated memory. The pointer returned to the client is an interior pointer.
Data structures such as Pascal strings. Here you allocate memory for the string and its size. The size comes before the string so the pointer to the start of the string is an interior pointer.
The memcheck manual says this:
"Possibly lost". […] This means that a chain of one or more pointers to the block has been found, but at least one of the pointers is an interior-pointer. This could just be a random value in memory that happens to point into a block, and so you shouldn't consider this ok unless you know you have interior-pointers.
So this should normally happen only if you have nested data structures on the heap where pointers point into an allocation, at an offset, and not directly at the beginning.
When running valgrind with --leak-check=full the generated reports include information about memory "possibly lost".
There is some information on this in the valgrind manual, as well as some example reports.
http://valgrind.org/docs/manual/mc-manual.html
LEAK SUMMARY:
definitely lost: 4 bytes in 1 blocks
indirectly lost: 0 bytes in 0 blocks
possibly lost: 0 bytes in 0 blocks
still reachable: 95 bytes in 6 blocks
of which reachable via heuristic:
stdstring : 56 bytes in 2 blocks
length64 : 16 bytes in 1 blocks
newarray : 7 bytes in 1 blocks
multipleinheritance: 8 bytes in 1 blocks
suppressed: 0 bytes in 0 blocks
In my own system, i get plenty of "possibly" lost memory when executing my multi-threaded test binary with valgrind.
What exactly does it mean that valgrind reports memory as "possibly lost"? Was it lost or wasn't it in this particular execution. Memory leakage should be more black and white than "possibly lost" in my opinion.
Roughly the categories are
Still in use = there is a live pointer to the memory at exit
Definitely lost = there are no live pointers to the memory
Indirectly lost = there are pointers to the memory but the pointers themselves are in "definitely lost" memory.
Possibly lost = there is a pointer but not the start of the memory.
The main reasons that Valgrind will detect possibly lost are either
Some junk pointer that accidentally points into the block. You should consider this as a definite loss.
A memory manager that allocates sub-blocks or guard bands.
So as a rule, if you are not using a memory manager, consider your possible losses as definite ones.
I tried to write my first GTK+-program. Compilation went fine but valgrind says that there are memory leaks. I'm unable to find those so could anyone say what am I doing wrong? Or is it possible at all to write graphical Linux programs without memory leaks?
#include <gtk/gtk.h>
int main(int argc, char* argv[])
{
gtk_init(&argc, &argv);
GtkWidget* window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
gtk_window_set_title(GTK_WINDOW(window), "Hello World");
gtk_container_set_border_width(GTK_CONTAINER(window), 60);
GtkWidget* label = gtk_label_new("Hello, world!");
gtk_container_add(GTK_CONTAINER(window), label);
g_signal_connect(window, "destroy", G_CALLBACK(gtk_main_quit), NULL);
gtk_widget_show_all(window);
gtk_main();
return 0;
}
gcc -Wall gtkhello.c -o gtkhello $(pkg-config --cflags --libs gtk+-2.0)
valgrind -v ./gtkhello
...
==9395== HEAP SUMMARY:
==9395== in use at exit: 538,930 bytes in 6,547 blocks
==9395== total heap usage: 21,434 allocs, 14,887 frees, 2,964,543 bytes allocated
==9395==
==9395== Searching for pointers to 6,547 not-freed blocks
==9395== Checked 949,656 bytes
==9395==
==9395== LEAK SUMMARY:
==9395== definitely lost: 4,480 bytes in 30 blocks
==9395== indirectly lost: 5,160 bytes in 256 blocks
==9395== possibly lost: 180,879 bytes in 1,716 blocks
==9395== still reachable: 348,411 bytes in 4,545 blocks
==9395== suppressed: 0 bytes in 0 blocks
==9395== Rerun with --leak-check=full to see details of leaked memory
==9395==
==9395== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==9395== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
You're not doing anything wrong. GTK widgets use reference counting, but in your programme all the references are taken care of, so you're not (manually) leaking anything.
So why does Valgrind claim you are?
Firstly, GLib has its own "slab" memory allocator, called GSlice, which is generally faster than system malloc for small allocations. Unfortunately, it confuses Valgrind, but if you set the environment variable G_SLICE=always-malloc, GSlice is effectively turned off.
Secondly, you can set the G_DEBUG=gc-friendly, which is supposed to help Valgrind produce more accurate results (although it my experience it generally doesn't make any difference).
Both of these environment variables are listed in the GLib documentation.
Unfortunately, even if you do both of these things, Valgrind will still report that your app leaks memory. The reason for this is that GTK (and its underlying libraries) allocate some "static" memory at start-up that doesn't get freed until the programme quits. This isn't really a problem, because typically the programme ends as soon as gtk_main() returns, and then the OS frees any remaining resources, so you're not really leaking anything. Valgrind thinks you are though, and for this reason it would be nice to have a gtk_deinit() function, but sadly there isn't one.
The best you can do instead is to teach Valgrind to ignore these things, via a suppressions file. The Valgrind page on the Gnome Wiki has details on all this and more.
I'm currently working on some OJ system, and the system will give out the TIME and MEMORY usage after my program is run. I know that there is ./time that could check out time usage, how about memory? Or is there any command that could check both?
Use valgrind.
valgrind your_exec
When your process exit, valgrind will output a summary of bytes allocated/freed during the execution time and how much space was still allocated just before exit
Output example:
==840==
==840== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 17 from 1)
==840== malloc/free: in use at exit: 88,940 bytes in 163 blocks.
==840== malloc/free: 376 allocs, 213 frees, 208,624 bytes allocated.
my application is causing memory leak of 10mb when the first timeout occurs. Heare i am using linux timer functions (timer_create etc.,).
For the subsequent timeouts no issue is there. I doubt some problem with linux timers.
I debugged it with valgrind and purify. Even these tools are of no help to me. In both the tools, memory leaked is shown as few kb's. But my application is causing memory leak of 10mb for the first timeout.
If anybody faced this problem earlier, please help me.
To find out which bits of you code is causing the leak (if any), compile your code to include debug symbols (i.e. include -g flag if you're using gcc), then run your program via valgrind.
valgrind --leak-check=full ./your_program
The run will take a little longer than usual, but when your program ends, the output from valgrind should tell you how much memory you've leaked and where the cuplrits are.
Sample output:
==10934== HEAP SUMMARY:
==10934== in use at exit: 10 bytes in 10 blocks
==10934== total heap usage: 10 allocs, 0 frees, 10 bytes allocated
==10934==
==10934== 10 bytes in 10 blocks are definitely lost in loss record 1 of 1
==10934== at 0x4024F20: malloc (vg_replace_malloc.c:236)
==10934== by 0x8048402: main (a.c:8)
==10934==
==10934== LEAK SUMMARY:
==10934== definitely lost: 10 bytes in 10 blocks
==10934== indirectly lost: 0 bytes in 0 blocks
==10934== possibly lost: 0 bytes in 0 blocks
==10934== still reachable: 0 bytes in 0 blocks
==10934== suppressed: 0 bytes in 0 blocks
update
Since you're already using valgrind, perhaps you could try using the Massif tool that comes with it. It should be able to paint a more accurate picture of memory usage (compare to simply watching top).
Check out this tutorial to see how it can be used. You may need some additional options to get a sensible graph depending on the runtime and mem usage of your program. Some useful options are described a few pages later in the tutorial.
Good luck.