When running valgrind with --leak-check=full the generated reports include information about memory "possibly lost".
There is some information on this in the valgrind manual, as well as some example reports.
http://valgrind.org/docs/manual/mc-manual.html
LEAK SUMMARY:
definitely lost: 4 bytes in 1 blocks
indirectly lost: 0 bytes in 0 blocks
possibly lost: 0 bytes in 0 blocks
still reachable: 95 bytes in 6 blocks
of which reachable via heuristic:
stdstring : 56 bytes in 2 blocks
length64 : 16 bytes in 1 blocks
newarray : 7 bytes in 1 blocks
multipleinheritance: 8 bytes in 1 blocks
suppressed: 0 bytes in 0 blocks
In my own system, i get plenty of "possibly" lost memory when executing my multi-threaded test binary with valgrind.
What exactly does it mean that valgrind reports memory as "possibly lost"? Was it lost or wasn't it in this particular execution. Memory leakage should be more black and white than "possibly lost" in my opinion.
Roughly the categories are
Still in use = there is a live pointer to the memory at exit
Definitely lost = there are no live pointers to the memory
Indirectly lost = there are pointers to the memory but the pointers themselves are in "definitely lost" memory.
Possibly lost = there is a pointer but not the start of the memory.
The main reasons that Valgrind will detect possibly lost are either
Some junk pointer that accidentally points into the block. You should consider this as a definite loss.
A memory manager that allocates sub-blocks or guard bands.
So as a rule, if you are not using a memory manager, consider your possible losses as definite ones.
Related
I am using gfortran 8.4 in Ubuntu with a deferred-length character variable as in the following example:
PROGRAM test
IMPLICIT NONE
CHARACTER(LEN=:),ALLOCATABLE :: str
str = '10'
END PROGRAM test
If I compile it using:
gfortran-8 test.f90 -o test -O0
When running the program using Valgrind I get a memory leak:
==29119== HEAP SUMMARY:
==29119== in use at exit: 2 bytes in 1 blocks
==29119== total heap usage: 22 allocs, 21 frees, 13,522 bytes allocated
==29119==
==29119== LEAK SUMMARY:
==29119== definitely lost: 2 bytes in 1 blocks
==29119== indirectly lost: 0 bytes in 0 blocks
==29119== possibly lost: 0 bytes in 0 blocks
==29119== still reachable: 0 bytes in 0 blocks
==29119== suppressed: 0 bytes in 0 blocks
However, compiling the program with:
gfortran-8 test.f90 -o test -O1
I get in Valgrind:
==29130== HEAP SUMMARY:
==29130== in use at exit: 0 bytes in 0 blocks
==29130== total heap usage: 21 allocs, 21 frees, 13,520 bytes allocated
==29130==
==29130== All heap blocks were freed -- no leaks are possible
I do not understand why I am getting this memory leak when no optimization is applied at compile time. Thanks in advance.
All variables declared in the main program or as module variables are implicitly save. Saved variables are not automatically deallocated. The Fortran standard does not mandate deallocation of arrays at the end of the program. They will be reclaimed by your OS anyway.
You can deallocate your arrays manually or if you wish to get automatic reallocation, you can move that logic - and the allocatable variables - into a subroutine that is entered from the main program. That way the local allocatable variables of that subroutine will be deallocated when the subroutine finishes.
Alternatively, you can also create a block using block and end block and declare the allocatable variables inside the block with all that it brings. They will be deallocated when the execution of the block is finished.
Technically what happens is that code generated by the compiler for your program does not maintain the pointers inside the allocatable descriptors until the moment valgrind would like to see them for them to be "still reachable". That is a technicality that you do not have to worry about.
It might not be perfectly nice to let the OS do the memory cleanup for a variable that has a lifetime until the end of a program, but it's still valid.
To avoid these false positive leaks in valgrind it is sufficient to enclose your code in a scope contained in the main program using the block construct.
I am using gfortran 8.4 in Ubuntu with a deferred-length character variable as in the following example:
PROGRAM test
IMPLICIT NONE
CHARACTER(LEN=:),ALLOCATABLE :: str
str = '10'
END PROGRAM test
If I compile it using:
gfortran-8 test.f90 -o test -O0
When running the program using Valgrind I get a memory leak:
==29119== HEAP SUMMARY:
==29119== in use at exit: 2 bytes in 1 blocks
==29119== total heap usage: 22 allocs, 21 frees, 13,522 bytes allocated
==29119==
==29119== LEAK SUMMARY:
==29119== definitely lost: 2 bytes in 1 blocks
==29119== indirectly lost: 0 bytes in 0 blocks
==29119== possibly lost: 0 bytes in 0 blocks
==29119== still reachable: 0 bytes in 0 blocks
==29119== suppressed: 0 bytes in 0 blocks
However, compiling the program with:
gfortran-8 test.f90 -o test -O1
I get in Valgrind:
==29130== HEAP SUMMARY:
==29130== in use at exit: 0 bytes in 0 blocks
==29130== total heap usage: 21 allocs, 21 frees, 13,520 bytes allocated
==29130==
==29130== All heap blocks were freed -- no leaks are possible
I do not understand why I am getting this memory leak when no optimization is applied at compile time. Thanks in advance.
All variables declared in the main program or as module variables are implicitly save. Saved variables are not automatically deallocated. The Fortran standard does not mandate deallocation of arrays at the end of the program. They will be reclaimed by your OS anyway.
You can deallocate your arrays manually or if you wish to get automatic reallocation, you can move that logic - and the allocatable variables - into a subroutine that is entered from the main program. That way the local allocatable variables of that subroutine will be deallocated when the subroutine finishes.
Alternatively, you can also create a block using block and end block and declare the allocatable variables inside the block with all that it brings. They will be deallocated when the execution of the block is finished.
Technically what happens is that code generated by the compiler for your program does not maintain the pointers inside the allocatable descriptors until the moment valgrind would like to see them for them to be "still reachable". That is a technicality that you do not have to worry about.
It might not be perfectly nice to let the OS do the memory cleanup for a variable that has a lifetime until the end of a program, but it's still valid.
To avoid these false positive leaks in valgrind it is sufficient to enclose your code in a scope contained in the main program using the block construct.
When i am running valgrind by adding a leakage to my code,I am getting leakage as still reachable for first allocation of block and then showing as definitely lost for 9 blocks.Possibly lost is showing due to other portion of the code. Why is this so ?
main()
{
........
char *ptr;
For(i=0;i<10;i++)
{
ptr=malloc(sizeof * ptr);
}
.....
}
Report:
HEAP SUMMARY:
==13832== in use at exit: 202,328 bytes in 62 blocks
==13832== total heap usage: 332 allocs, 270 frees, 283,928 bytes allocated
==13832==
==13832== LEAK SUMMARY:
==13832== definitely lost: 90 bytes in 9 blocks
==13832== indirectly lost: 0 bytes in 0 blocks
==13832== possibly lost: 202,180 bytes in 49 blocks
==13832== still reachable: 58 bytes in 4 blocks
==13832== suppressed: 0 bytes in 0 blocks
==13832== Rerun with --leak-check=full to see details of leaked memory
Adding to Florian's answer, here are a few examples of where you would have interior pointers
A memory manager. For instance, you know that you are going to be allocating very many small blocks of memory of the same size, so you write a memory manager that allocates large blocks and the subdivides them into your small size. These sub-block allocations are interior pointers.
A memory debugger. In this case, you allocate more memory than requested, and the extra memory is used to pad the allocated memory. The pointer returned to the client is an interior pointer.
Data structures such as Pascal strings. Here you allocate memory for the string and its size. The size comes before the string so the pointer to the start of the string is an interior pointer.
The memcheck manual says this:
"Possibly lost". […] This means that a chain of one or more pointers to the block has been found, but at least one of the pointers is an interior-pointer. This could just be a random value in memory that happens to point into a block, and so you shouldn't consider this ok unless you know you have interior-pointers.
So this should normally happen only if you have nested data structures on the heap where pointers point into an allocation, at an offset, and not directly at the beginning.
I'm trying to understand the output from Valgrind having executed it as follows:
valgrind --leak-check=yes "someprogram"
The output is here:
==30347==
==30347== HEAP SUMMARY:
==30347== in use at exit: 126,188 bytes in 2,777 blocks
==30347== total heap usage: 4,562 allocs, 1,785 frees, 974,922 bytes
allocated
==30347==
==30347== LEAK SUMMARY:
==30347== definitely lost: 0 bytes in 0 blocks
==30347== indirectly lost: 0 bytes in 0 blocks
==30347== possibly lost: 0 bytes in 0 blocks
==30347== still reachable: 126,188 bytes in 2,777 blocks
==30347== suppressed: 0 bytes in 0 blocks
==30347== Reachable blocks (those to which a pointer was found) are
not shown.
==30347== To see them, rerun with: --leak-check=full --show-reachable=yes
==30347==
==30347== For counts of detected and suppressed errors, rerun with: -v
==30347== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
According to the output, there are no lost bytes, but there seems to be still reachable blocks. So do I have a memory leak?
No.
You are most concerned with unreachable blocks. What you are seeing here is that there are active variables that are still "pointing" at reachable blocks of memory. They are still in scope.
An unreachable block would be, for instance, memory that you have allocated dynamically, used for a period of time and then all of the references to it have gone out of scope even though the program is still executing. Since you no longer have any handles pointing to them they are now unrecoverable, creating a memory leak.
Here is a quote from the Valgrind docs:
"still reachable" means your program is probably ok -- it didn't free some memory it could have. This is quite common and often reasonable. Don't use --show-reachable=yes if you don't want to see these reports.
my application is causing memory leak of 10mb when the first timeout occurs. Heare i am using linux timer functions (timer_create etc.,).
For the subsequent timeouts no issue is there. I doubt some problem with linux timers.
I debugged it with valgrind and purify. Even these tools are of no help to me. In both the tools, memory leaked is shown as few kb's. But my application is causing memory leak of 10mb for the first timeout.
If anybody faced this problem earlier, please help me.
To find out which bits of you code is causing the leak (if any), compile your code to include debug symbols (i.e. include -g flag if you're using gcc), then run your program via valgrind.
valgrind --leak-check=full ./your_program
The run will take a little longer than usual, but when your program ends, the output from valgrind should tell you how much memory you've leaked and where the cuplrits are.
Sample output:
==10934== HEAP SUMMARY:
==10934== in use at exit: 10 bytes in 10 blocks
==10934== total heap usage: 10 allocs, 0 frees, 10 bytes allocated
==10934==
==10934== 10 bytes in 10 blocks are definitely lost in loss record 1 of 1
==10934== at 0x4024F20: malloc (vg_replace_malloc.c:236)
==10934== by 0x8048402: main (a.c:8)
==10934==
==10934== LEAK SUMMARY:
==10934== definitely lost: 10 bytes in 10 blocks
==10934== indirectly lost: 0 bytes in 0 blocks
==10934== possibly lost: 0 bytes in 0 blocks
==10934== still reachable: 0 bytes in 0 blocks
==10934== suppressed: 0 bytes in 0 blocks
update
Since you're already using valgrind, perhaps you could try using the Massif tool that comes with it. It should be able to paint a more accurate picture of memory usage (compare to simply watching top).
Check out this tutorial to see how it can be used. You may need some additional options to get a sensible graph depending on the runtime and mem usage of your program. Some useful options are described a few pages later in the tutorial.
Good luck.