my application is causing memory leak of 10mb when the first timeout occurs. Heare i am using linux timer functions (timer_create etc.,).
For the subsequent timeouts no issue is there. I doubt some problem with linux timers.
I debugged it with valgrind and purify. Even these tools are of no help to me. In both the tools, memory leaked is shown as few kb's. But my application is causing memory leak of 10mb for the first timeout.
If anybody faced this problem earlier, please help me.
To find out which bits of you code is causing the leak (if any), compile your code to include debug symbols (i.e. include -g flag if you're using gcc), then run your program via valgrind.
valgrind --leak-check=full ./your_program
The run will take a little longer than usual, but when your program ends, the output from valgrind should tell you how much memory you've leaked and where the cuplrits are.
Sample output:
==10934== HEAP SUMMARY:
==10934== in use at exit: 10 bytes in 10 blocks
==10934== total heap usage: 10 allocs, 0 frees, 10 bytes allocated
==10934==
==10934== 10 bytes in 10 blocks are definitely lost in loss record 1 of 1
==10934== at 0x4024F20: malloc (vg_replace_malloc.c:236)
==10934== by 0x8048402: main (a.c:8)
==10934==
==10934== LEAK SUMMARY:
==10934== definitely lost: 10 bytes in 10 blocks
==10934== indirectly lost: 0 bytes in 0 blocks
==10934== possibly lost: 0 bytes in 0 blocks
==10934== still reachable: 0 bytes in 0 blocks
==10934== suppressed: 0 bytes in 0 blocks
update
Since you're already using valgrind, perhaps you could try using the Massif tool that comes with it. It should be able to paint a more accurate picture of memory usage (compare to simply watching top).
Check out this tutorial to see how it can be used. You may need some additional options to get a sensible graph depending on the runtime and mem usage of your program. Some useful options are described a few pages later in the tutorial.
Good luck.
Related
I am using gfortran 8.4 in Ubuntu with a deferred-length character variable as in the following example:
PROGRAM test
IMPLICIT NONE
CHARACTER(LEN=:),ALLOCATABLE :: str
str = '10'
END PROGRAM test
If I compile it using:
gfortran-8 test.f90 -o test -O0
When running the program using Valgrind I get a memory leak:
==29119== HEAP SUMMARY:
==29119== in use at exit: 2 bytes in 1 blocks
==29119== total heap usage: 22 allocs, 21 frees, 13,522 bytes allocated
==29119==
==29119== LEAK SUMMARY:
==29119== definitely lost: 2 bytes in 1 blocks
==29119== indirectly lost: 0 bytes in 0 blocks
==29119== possibly lost: 0 bytes in 0 blocks
==29119== still reachable: 0 bytes in 0 blocks
==29119== suppressed: 0 bytes in 0 blocks
However, compiling the program with:
gfortran-8 test.f90 -o test -O1
I get in Valgrind:
==29130== HEAP SUMMARY:
==29130== in use at exit: 0 bytes in 0 blocks
==29130== total heap usage: 21 allocs, 21 frees, 13,520 bytes allocated
==29130==
==29130== All heap blocks were freed -- no leaks are possible
I do not understand why I am getting this memory leak when no optimization is applied at compile time. Thanks in advance.
All variables declared in the main program or as module variables are implicitly save. Saved variables are not automatically deallocated. The Fortran standard does not mandate deallocation of arrays at the end of the program. They will be reclaimed by your OS anyway.
You can deallocate your arrays manually or if you wish to get automatic reallocation, you can move that logic - and the allocatable variables - into a subroutine that is entered from the main program. That way the local allocatable variables of that subroutine will be deallocated when the subroutine finishes.
Alternatively, you can also create a block using block and end block and declare the allocatable variables inside the block with all that it brings. They will be deallocated when the execution of the block is finished.
Technically what happens is that code generated by the compiler for your program does not maintain the pointers inside the allocatable descriptors until the moment valgrind would like to see them for them to be "still reachable". That is a technicality that you do not have to worry about.
It might not be perfectly nice to let the OS do the memory cleanup for a variable that has a lifetime until the end of a program, but it's still valid.
To avoid these false positive leaks in valgrind it is sufficient to enclose your code in a scope contained in the main program using the block construct.
I am using gfortran 8.4 in Ubuntu with a deferred-length character variable as in the following example:
PROGRAM test
IMPLICIT NONE
CHARACTER(LEN=:),ALLOCATABLE :: str
str = '10'
END PROGRAM test
If I compile it using:
gfortran-8 test.f90 -o test -O0
When running the program using Valgrind I get a memory leak:
==29119== HEAP SUMMARY:
==29119== in use at exit: 2 bytes in 1 blocks
==29119== total heap usage: 22 allocs, 21 frees, 13,522 bytes allocated
==29119==
==29119== LEAK SUMMARY:
==29119== definitely lost: 2 bytes in 1 blocks
==29119== indirectly lost: 0 bytes in 0 blocks
==29119== possibly lost: 0 bytes in 0 blocks
==29119== still reachable: 0 bytes in 0 blocks
==29119== suppressed: 0 bytes in 0 blocks
However, compiling the program with:
gfortran-8 test.f90 -o test -O1
I get in Valgrind:
==29130== HEAP SUMMARY:
==29130== in use at exit: 0 bytes in 0 blocks
==29130== total heap usage: 21 allocs, 21 frees, 13,520 bytes allocated
==29130==
==29130== All heap blocks were freed -- no leaks are possible
I do not understand why I am getting this memory leak when no optimization is applied at compile time. Thanks in advance.
All variables declared in the main program or as module variables are implicitly save. Saved variables are not automatically deallocated. The Fortran standard does not mandate deallocation of arrays at the end of the program. They will be reclaimed by your OS anyway.
You can deallocate your arrays manually or if you wish to get automatic reallocation, you can move that logic - and the allocatable variables - into a subroutine that is entered from the main program. That way the local allocatable variables of that subroutine will be deallocated when the subroutine finishes.
Alternatively, you can also create a block using block and end block and declare the allocatable variables inside the block with all that it brings. They will be deallocated when the execution of the block is finished.
Technically what happens is that code generated by the compiler for your program does not maintain the pointers inside the allocatable descriptors until the moment valgrind would like to see them for them to be "still reachable". That is a technicality that you do not have to worry about.
It might not be perfectly nice to let the OS do the memory cleanup for a variable that has a lifetime until the end of a program, but it's still valid.
To avoid these false positive leaks in valgrind it is sufficient to enclose your code in a scope contained in the main program using the block construct.
When running valgrind with --leak-check=full the generated reports include information about memory "possibly lost".
There is some information on this in the valgrind manual, as well as some example reports.
http://valgrind.org/docs/manual/mc-manual.html
LEAK SUMMARY:
definitely lost: 4 bytes in 1 blocks
indirectly lost: 0 bytes in 0 blocks
possibly lost: 0 bytes in 0 blocks
still reachable: 95 bytes in 6 blocks
of which reachable via heuristic:
stdstring : 56 bytes in 2 blocks
length64 : 16 bytes in 1 blocks
newarray : 7 bytes in 1 blocks
multipleinheritance: 8 bytes in 1 blocks
suppressed: 0 bytes in 0 blocks
In my own system, i get plenty of "possibly" lost memory when executing my multi-threaded test binary with valgrind.
What exactly does it mean that valgrind reports memory as "possibly lost"? Was it lost or wasn't it in this particular execution. Memory leakage should be more black and white than "possibly lost" in my opinion.
Roughly the categories are
Still in use = there is a live pointer to the memory at exit
Definitely lost = there are no live pointers to the memory
Indirectly lost = there are pointers to the memory but the pointers themselves are in "definitely lost" memory.
Possibly lost = there is a pointer but not the start of the memory.
The main reasons that Valgrind will detect possibly lost are either
Some junk pointer that accidentally points into the block. You should consider this as a definite loss.
A memory manager that allocates sub-blocks or guard bands.
So as a rule, if you are not using a memory manager, consider your possible losses as definite ones.
I'm trying to understand the output from Valgrind having executed it as follows:
valgrind --leak-check=yes "someprogram"
The output is here:
==30347==
==30347== HEAP SUMMARY:
==30347== in use at exit: 126,188 bytes in 2,777 blocks
==30347== total heap usage: 4,562 allocs, 1,785 frees, 974,922 bytes
allocated
==30347==
==30347== LEAK SUMMARY:
==30347== definitely lost: 0 bytes in 0 blocks
==30347== indirectly lost: 0 bytes in 0 blocks
==30347== possibly lost: 0 bytes in 0 blocks
==30347== still reachable: 126,188 bytes in 2,777 blocks
==30347== suppressed: 0 bytes in 0 blocks
==30347== Reachable blocks (those to which a pointer was found) are
not shown.
==30347== To see them, rerun with: --leak-check=full --show-reachable=yes
==30347==
==30347== For counts of detected and suppressed errors, rerun with: -v
==30347== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
According to the output, there are no lost bytes, but there seems to be still reachable blocks. So do I have a memory leak?
No.
You are most concerned with unreachable blocks. What you are seeing here is that there are active variables that are still "pointing" at reachable blocks of memory. They are still in scope.
An unreachable block would be, for instance, memory that you have allocated dynamically, used for a period of time and then all of the references to it have gone out of scope even though the program is still executing. Since you no longer have any handles pointing to them they are now unrecoverable, creating a memory leak.
Here is a quote from the Valgrind docs:
"still reachable" means your program is probably ok -- it didn't free some memory it could have. This is quite common and often reasonable. Don't use --show-reachable=yes if you don't want to see these reports.
I am writing some code that uses MPI and I was keeping noticing some memory leaks when running it with valgrind. While trying to identify where the problem was, I ended up with this simple (and totally useless) main:
#include "/usr/include/mpi/mpi.h"
int main(int argc,char** argv)
{
MPI_Init(&argc, &argv);
MPI_Finalize();
return 0;
}
As you can see, this code doesn't do anything and shouldn't create any problem. However, when I run the code with valgrind (both in the serial and parallel case), I get the following summary:
==28271== HEAP SUMMARY:
==28271== in use at exit: 190,826 bytes in 2,745 blocks
==28271== total heap usage: 11,214 allocs, 8,469 frees, 16,487,977 bytes allocated
==28271==
==28271== LEAK SUMMARY:
==28271== definitely lost: 5,950 bytes in 55 blocks
==28271== indirectly lost: 3,562 bytes in 32 blocks
==28271== possibly lost: 0 bytes in 0 blocks
==28271== still reachable: 181,314 bytes in 2,658 blocks
==28271== suppressed: 0 bytes in 0 blocks
I don't understand why there are these leaks. Maybe it's just me not able to read the valgrind output or to use MPI initialization/finalization correctly...
I am using OMPI 1.4.1-3 under ubuntu on a 64 bit architecture, if this can help.
Thanks a lot for your time!
The OpenMPI FAQ addresses issues with valgrind. This refers initalization issues and memory leaks during finalization - which should have no practical negative impact.
There are many situations, where Open MPI purposefully does not
initialize and subsequently communicates memory, e.g., by calling
writev. Furthermore, several cases are known, where memory is not
properly freed upon MPI_Finalize.
This certainly does not help distinguishing real errors from false
positives. Valgrind provides functionality to suppress errors and
warnings from certain function contexts.
In an attempt to ease debugging using Valgrind, starting with v1.5,
Open MPI provides a so-called Valgrind-suppression file, that can be
passed on the command line:
mpirun -np 2 valgrind
--suppressions=$PREFIX/share/openmpi/openmpi-valgrind.supp
You're not doing anything wrong. Memcheck false positives with valgrind are common, the best you can do is suppress them.
This page of the manual speaks more about these false positives. A quote near the end:
The wrappers should reduce Memcheck's false-error rate on MPI
applications. Because the wrapping is done at the MPI interface, there
will still potentially be a large number of errors reported in the MPI
implementation below the interface. The best you can do is try to
suppress them.