I am doing a small Project. I am checking about memory leaks using the tool Valgrind. When I use this tool, I got the bellow information.
> 584 bytes in 74 blocks are definitely lost in loss record 103 of 104
> ==4628== at 0x402BE68: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==4628== by 0x41CF8D0: strdup (strdup.c:43)
> ==4628== by 0x8060B95: main (in mycall)
>
> LEAK SUMMARY:
> ==4628== definitely lost: 584 bytes in 74 blocks
> ==4628== indirectly lost: 0 bytes in 0 blocks
> ==4628== possibly lost: 0 bytes in 0 blocks
> ==4628== still reachable: 21,414 bytes in 383 blocks
> ==4628== suppressed: 0 bytes in 0 blocks
> ==4628==
> ==4628== For counts of detected and suppressed errors, rerun with: -v
> ==4628== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
These are the codings I have used the function strdup. I have used in LEX code.
{string} {
yylval.string = strdup(yytext + 1);
yylval.string[yyleng - 2] = 0;
return PPSTRING;
}
{numvar} { yylval.string = strdup(yytext);return(PPNUMVAR); }
{sysnumvar} { yylval.string = (char *) strdup(yytext);return(PPSYSNUMVAR); }
I don't know in which point the memory has been leaked.
strdup function allocate necessary memory to store the sourcing string implicitly, you need to free the returned string (i.e., yylval.string in your code) manually.
Related
I've a very large program (okay, only 13,000 lines of code according to cloc) which leaks. I know because over time, it uses more and more resident memory.
I have the sanitizer option turned on, but on a clean exit, all my C++ software will properly clean everything as expected. So I don't see anything growing in the sanitizer output.
What would be useful in this case, is a way to call a function which displays the (large) list of allocated buffers while running the code. I can then look at a diff of two such outputs and see what was allocated anew. The leaked buffers will be in there...
At this point, though, I just don't see any header with sanitizer functions I could call to see such a list. Does it exist?
Lsan interface is available in sanitizer/lsan_interface.h but AFAIK it has no API to print allocation info. The best you can get is compile your code with Asan (which includes Lsan as well) and use __asan_print_accumulated_stats to get basic allocation statistics:
$ cat tmp.c
#include <sanitizer/asan_interface.h>
#include <stdlib.h>
int main() {
malloc(100);
__asan_print_accumulated_stats();
return 0;
}
$ gcc -fsanitize=address -g tmp.c && ./a.out
Stats: 0M malloced (0M for red zones) by 2 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 0 calls
Stats: 0M really freed by 0 calls
Stats: 0M (0M-0M) mmaped; 5 maps, 0 unmaps
mallocs by size class: 7:1; 11:1;
Stats: malloc large: 0
Stats: StackDepot: 2 ids; 0M allocated
Stats: SizeClassAllocator64: 0M mapped in 256 allocations; remains 256
07 (112): mapped: 64K allocs: 128 frees: 0 inuse: 128 num_freed_chunks 457 avail: 585 rss: 4K releases: 0
11 (176): mapped: 64K allocs: 128 frees: 0 inuse: 128 num_freed_chunks 244 avail: 372 rss: 4K releases: 0
Stats: LargeMmapAllocator: allocated 0 times, remains 0 (0 K) max 0 M; by size logs:
=================================================================
==15060==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 100 byte(s) in 1 object(s) allocated from:
#0 0x7fdf2194fb40 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xdeb40)
#1 0x559ca08a7857 in main /home/yugr/tmp.c:5
#2 0x7fdf214a1bf6 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21bf6)
SUMMARY: AddressSanitizer: 100 byte(s) leaked in 1 allocation(s).
Unfortunately there is no way to print exact allocations.
I am learning pthreads programming.
I understood that there are two states of thread:
1. Joinable
2. Detachable
In case of Joinable, we need to call pthread_join to free the resources(stack), whereas in case of detached there is no need to call pthread_join and the resources will be freed on thread exit.
I wrote a sample program to observe the behavior
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
void *threadFn(void *arg)
{
pthread_detach(pthread_self());
sleep(1);
printf("Thread Fn\n");
pthread_exit(NULL);
}
int main(int argc, char *argv[])
{
pthread_t tid;
int ret = pthread_create(&tid, NULL, threadFn, NULL);
if (ret != 0) {
perror("Thread Creation Error\n");
exit(1);
}
printf("After thread created in Main\n");
pthread_exit(NULL);
}
When i try to check any mem leaks with valgrind it gave me leaks of 272 bytes. Can you show me why is the leak happening here.
$valgrind --leak-check=full ./app
==38649==
==38649== HEAP SUMMARY:
==38649== in use at exit: 272 bytes in 1 blocks
==38649== total heap usage: 7 allocs, 6 frees, 2,990 bytes allocated
==38649==
==38649== 272 bytes in 1 blocks are possibly lost in loss record 1 of 1
==38649== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==38649== by 0x40134A6: allocate_dtv (dl-tls.c:286)
==38649== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==38649== by 0x4E44227: allocate_stack (allocatestack.c:627)
==38649== by 0x4E44227: pthread_create##GLIBC_2.2.5 (pthread_create.c:644)
==38649== by 0x108902: main (2.c:18)
==38649==
==38649== LEAK SUMMARY:
==38649== definitely lost: 0 bytes in 0 blocks
==38649== indirectly lost: 0 bytes in 0 blocks
==38649== possibly lost: 272 bytes in 1 blocks
==38649== still reachable: 0 bytes in 0 blocks
==38649== suppressed: 0 bytes in 0 blocks
==38649==
==38649== For counts of detected and suppressed errors, rerun with: -v
==38649== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Your expectation is correct that there shouldn't be any leaks in main thread once you call pthread_exit.
However, what you observe is a quirk of the implementation you're using (which is likely to be glibc) - pthreads library (glibc implementation) re-uses the initially allocated stack for threads - like a cache so that previously allocated stacks can be re-used whenever possible.
Valgrind simply reports what it "sees" (something was allocated but not de-allocated). But it's not a real leak, so you don't need to worry about this.
If you "reverse" the logic (main thread exits as the last thread) then you wouldn't see leaks because the initially allocated stack space is properly free'd by the main thread. But this leak isn't a real leak in any case and you can safely ignore this.
You can also setup a suppression file so that Valgrind doesn't complain about this (which is to inform Valgrind that "I know this isn't not real leak, so don't report this"), such as:
{
Pthread_Stack_Leaks_Ignore
Memcheck:Leak
fun:calloc
fun:allocate_dtv
fun:_dl_allocate_tls
fun:allocate_stack
fun:pthread_create*
}
I am trying to parallelize an algorithm using DAG-Scheduling via OpenMP tasks and there many programs are killed by the Linux kernel due to Out-Of-Memory after a calls to the parallelized code although the allocated memory is only 1% of the servers main memory. But this happens only if I use the Intel Compilers from 2015, 2016 or even the new 2017 edition.
Here is a small example building the same task dependency graph as the algorithm crashing:
PROGRAM OMP_TASK_PROBLEM
IMPLICIT NONE
INTEGER M, N
PARAMETER(M = 256, N=256)
DOUBLE PRECISION X(M,N)
X(1:M,1:N) = 0.0D0
CALL COMPUTE_X(M, N, X, M)
! WRITE(*,*) X (1:M, 1:N)
END PROGRAM
SUBROUTINE COMPUTE_X(M,N, X, LDX)
IMPLICIT NONE
INTEGER M, N, LDX
DOUBLE PRECISION X(LDX, N)
INTEGER K, L, KOLD, LOLD
!$omp parallel default(shared)
!$omp master
L = 1
DO WHILE ( L .LE. N )
K = M
DO WHILE (K .GT. 0)
IF ( K .EQ. M .AND. L .EQ. 1) THEN
!$omp task depend(out:X(K,L)) firstprivate(K,L) default(shared)
X(K,L) = 0
!$omp end task
ELSE IF ( K .EQ. M .AND. L .GT. 1) THEN
!$omp task depend(out:X(K,L)) depend(in:X(K,LOLD)) firstprivate(K,L,LOLD) default(shared)
X(K,L) = 1 + X(K,LOLD)
!$omp end task
ELSE IF ( K .LT. M .AND. L .EQ. 1) THEN
!$omp task depend(out:X(K,L)) depend(in:X(KOLD,L)) firstprivate(K,L,KOLD) default(shared)
X(K,L) = 2 + X(KOLD,L)
!$omp end task
ELSE
!$omp task depend(out:X(K,L)) depend(in:X(KOLD,L),X(K,LOLD)) firstprivate(K,L,KOLD, LOLD) default(shared)
X(K,L) = X(KOLD, L) + X(K,LOLD)
!$omp end task
END IF
KOLD = K
K = K - 1
END DO
LOLD = L
L = L + 1
END DO
!$omp end master
!$omp taskwait
!$omp end parallel
END SUBROUTINE
After compiling it using ifort -qopenmp -g omp_test.f90 and running it via valgrind it reports:
==21071== LEAK SUMMARY:
==21071== definitely lost: 0 bytes in 0 blocks
==21071== indirectly lost: 0 bytes in 0 blocks
==21071== possibly lost: 118,489,088 bytes in 113 blocks
==21071== still reachable: 1,286 bytes in 35 blocks
==21071== suppressed: 0 bytes in 0 blocks
where the --leak-check=full option shows the following details:
==21071== 1,048,576 bytes in 1 blocks are possibly lost in loss record 22 of 26
==21071== at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==21071== by 0x516A807: bget(kmp_info*, long) (kmp_alloc.c:741)
==21071== by 0x516A52D: ___kmp_fast_allocate (kmp_alloc.c:2012)
==21071== by 0x51CD917: __kmp_task_alloc (kmp_tasking.c:991)
==21071== by 0x51CD886: __kmpc_omp_task_alloc (kmp_tasking.c:1131)
==21071== by 0x4034FA: compute_x_ (omp_task_problem.f90:31)
==21071== by 0x51DBCB2: __kmp_invoke_microtask (in /scratch/software/intel-2016/compilers_and_libraries_2016.3.210/linux/compiler/lib/intel64_lin/libiomp5.so)
==21071== by 0x51AA436: __kmp_invoke_task_func (kmp_runtime.c:7058)
==21071== by 0x51AB60A: __kmp_fork_call (kmp_runtime.c:2397)
==21071== by 0x5184517: __kmpc_fork_call (kmp_csupport.c:339)
==21071== by 0x403353: compute_x_ (omp_task_problem.f90:24)
==21071== by 0x402F4C: MAIN__ (omp_task_problem.f90:10)
==21071==
==21071== 24,117,248 bytes in 23 blocks are possibly lost in loss record 23 of 26
==21071== at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==21071== by 0x516A807: bget(kmp_info*, long) (kmp_alloc.c:741)
==21071== by 0x516A52D: ___kmp_fast_allocate (kmp_alloc.c:2012)
==21071== by 0x51CB5E0: __kmpc_omp_task_with_deps (kmp_taskdeps.cpp:452)
==21071== by 0x403D54: compute_x_ (omp_task_problem.f90:43)
==21071== by 0x51DBCB2: __kmp_invoke_microtask (in /scratch/software/intel-2016/compilers_and_libraries_2016.3.210/linux/compiler/lib/intel64_lin/libiomp5.so)
==21071== by 0x51AA436: __kmp_invoke_task_func (kmp_runtime.c:7058)
==21071== by 0x51AB60A: __kmp_fork_call (kmp_runtime.c:2397)
==21071== by 0x5184517: __kmpc_fork_call (kmp_csupport.c:339)
==21071== by 0x403353: compute_x_ (omp_task_problem.f90:24)
==21071== by 0x402F4C: MAIN__ (omp_task_problem.f90:10)
==21071== by 0x402E1D: main (in ./a.out)
...
The line numbers of the compute_x_ function in the backtrace correspond to the !$omp task statements. These memory leaks accumulated rapidly to an amount of memory such that the program crashes. But changing the call of COMPUTE_X to
DO J = 1, 10000
CALL COMPUTE_X(M,N, X, M)
END DO
one can even use top to see that the memory requirements of the application explode.
Using gcc-6.2 for this valgrind ends up with:
==21246== LEAK SUMMARY:
==21246== definitely lost: 0 bytes in 0 blocks
==21246== indirectly lost: 0 bytes in 0 blocks
==21246== possibly lost: 8,640 bytes in 15 blocks
==21246== still reachable: 4,624 bytes in 4 blocks
==21246== suppressed: 0 bytes in 0 blocks
==21246==
where the leaks are only from the first initialization of the OpenMP runtime system.
So my question is: Why does the Intel Compiler/Intel OpenMP runtime system produce theses leaks or alternatively is there an error in the way I have
designed the task parallelism.
I've get a memory leak reported by Valgrind with -pg enabled when compiling the following simple code.
#include <iostream>
#include <boost/filesystem.hpp>
#define BOOST_FILESYSTEM_VERSION 3
using boost::filesystem::path;
using namespace std;
int main() {
path ptDir;
ptDir = "/home/foo/bar";
if (true == is_directory((const path &)ptDir)){
cout << "ptDir: " << ptDir << endl;
}
}
The full compile option was as follows.
g++ -pg -g test.cpp -lboost_system -lboost_filesystem
The command of running valgrind is:
valgrind --gen-suppressions=all --track-origins=yes --error-limit=no --leak-check=full --show-reachable=yes -v --trace-children=yes --track-fds=yes --log-file=vg.log ./a.out
Then valgrind gave me a memory leak error.
==9598== HEAP SUMMARY:
==9598== in use at exit: 4,228 bytes in 1 blocks
==9598== total heap usage: 136 allocs, 135 frees, 17,984 bytes allocated
==9598==
==9598== Searching for pointers to 1 not-freed blocks
==9598== Checked 130,088 bytes
==9598==
==9598== 4,228 bytes in 1 blocks are still reachable in loss record 1 of 1
==9598== at 0x402A5E6: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==9598== by 0x4260F44: monstartup (gmon.c:134)
==9598== by 0xBED6B636: ???
==9598== by 0x4E45504E: ???
==9598==
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:monstartup
obj:*
obj:*
}
==9598== LEAK SUMMARY:
==9598== definitely lost: 0 bytes in 0 blocks
==9598== indirectly lost: 0 bytes in 0 blocks
==9598== possibly lost: 0 bytes in 0 blocks
==9598== still reachable: 4,228 bytes in 1 blocks
==9598== suppressed: 0 bytes in 0 blocks
==9598==
==9598== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==9598== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Is this correct? I'm using Ubuntu 12.04/ Kernel 3.2.0-32-generic
Valgrind FAQ:
"still reachable" means your program is probably ok -- it didn't free some
memory it could have. This is quite common and often reasonable. Don't use
--show-reachable=yes if you don't want to see these reports.
The fact that the allocatio comes from:
by 0x4260F44: monstartup (gmon.c:134)
indicates that it's a side effect of -pg - which is nothing you can do anything about. Don't mix -pg and valgrind is my suggestion.
I've been chasing a memory leak (reported by 'valgrind --leak-check=yes') and it appears to be coming from ALSA. This code has been in the free world for some time so I'm guessing that it's something I'm doing wrong.
#include <stdio.h>
#include <stdlib.h>
#include <alsa/asoundlib.h>
int main (int argc, char *argv[])
{
snd_ctl_t *handle;
int err = snd_ctl_open( &handle, "hw:1", 0 );
printf( "snd_ctl_open: %d\n", err );
err = snd_ctl_close(handle);
printf( "snd_ctl_close: %d\n", err );
}
The output looks like this:
[root#aeolus alsa]# valgrind --leak-check=yes ./test2
==16296== Memcheck, a memory error detector
==16296== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==16296== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==16296== Command: ./test2
==16296==
snd_ctl_open: 0
snd_ctl_close: 0
==16296==
==16296== HEAP SUMMARY:
==16296== in use at exit: 22,912 bytes in 1,222 blocks
==16296== total heap usage: 1,507 allocs, 285 frees, 26,236 bytes allocated
==16296==
==16296== 4 bytes in 2 blocks are possibly lost in loss record 1 of 62
==16296== at 0x4007100: malloc (vg_replace_malloc.c:270)
==16296== by 0x340F7F: strdup (in /lib/libc-2.5.so)
==16296== by 0x624C6B5: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624CA5B: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624CD81: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624F311: snd_config_update_r (in /lib/libasound.so.2.0.0)
==16296== by 0x624FAD7: snd_config_update (in /lib/libasound.so.2.0.0)
==16296== by 0x625DA22: snd_ctl_open (in /lib/libasound.so.2.0.0)
==16296== by 0x804852F: main (test2.cpp:9)
and continues for some pages to
==16296== 2,052 bytes in 57 blocks are possibly lost in loss record 62 of 62
==16296== at 0x4005EB4: calloc (vg_replace_malloc.c:593)
==16296== by 0x624A268: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624A38F: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624CA33: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624CCC9: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624CD81: ??? (in /lib/libasound.so.2.0.0)
==16296== by 0x624F311: snd_config_update_r (in /lib/libasound.so.2.0.0)
==16296== by 0x624FAD7: snd_config_update (in /lib/libasound.so.2.0.0)
==16296== by 0x625DA22: snd_ctl_open (in /lib/libasound.so.2.0.0)
==16296== by 0x804852F: main (test2.cpp:9)
==16296==
==16296== LEAK SUMMARY:
==16296== definitely lost: 0 bytes in 0 blocks
==16296== indirectly lost: 0 bytes in 0 blocks
==16296== possibly lost: 22,748 bytes in 1,216 blocks
==16296== still reachable: 164 bytes in 6 blocks
==16296== suppressed: 0 bytes in 0 blocks
==16296== Reachable blocks (those to which a pointer was found) are not shown.
==16296== To see them, rerun with: --leak-check=full --show-reachable=yes
==16296==
==16296== For counts of detected and suppressed errors, rerun with: -v
==16296== ERROR SUMMARY: 56 errors from 56 contexts (suppressed: 19 from 8)
This came about as I'm using ALSA in a project and started seeing this huge leak...or at least the report of said leak.
So the question is: is it me, ALSA or valgrind that's having issues here?
http://git.alsa-project.org/?p=alsa-lib.git;a=blob;f=MEMORY-LEAK;hb=HEAD says:
Memory leaks - really?
----------------------
Note that some developers are thinking that the ALSA library has some memory
leaks. Sure, it can be truth, but before contacting us, please, be sure that
these leaks are not forced.
The biggest reported leak is that the global configuration is cached for
next usage. If you do not want this feature, simply, call
snd_config_update_free_global() after all snd_*_open*() calls. This function
will free the cache.
The biggest reported leak is that the global configuration is cached for next usage.
If you do not want this feature, simply call snd_config_update_free_global() after all snd_*_open*() calls.
This function will free the cache." <---- Valgrind still detects leaks.
This can be fixed if you call snd_config_update_free_global() after snd_pcm_close(handle);
Perhaps this will work (source):
diff --git a/src/pcm/pcm.c b/src/pcm/pcm.c
--- a/src/pcm/pcm.c
+++ b/src/pcm/pcm.c
## -2171,7 +2171,12 ## static int snd_pcm_open_conf(snd_pcm_t **pcmp, const char *name,
if (open_func) {
err = open_func(pcmp, name, pcm_root, pcm_conf, stream, mode);
if (err >= 0) {
- (*pcmp)->open_func = open_func;
+ if ((*pcmp)->open_func) {
+ /* only init plugin (like empty, asym) */
+ snd_dlobj_cache_put(open_func);
+ } else {
+ (*pcmp)->open_func = open_func;
+ }
err = 0;
} else {
snd_dlobj_cache_put(open_func);
I tried it myself, but to no avail. My core temp heats up ~10 °F, most likely due to similar memory leak. Here's some of what valgrind gave me, even after using the patch above:
==869== 16,272 bytes in 226 blocks are possibly lost in loss record 103 of 103
==869== at 0x4C28E48: calloc (vg_replace_malloc.c:566)
==869== by 0x5066E61: _snd_config_make (in /usr/lib64/libasound.so.2)
==869== by 0x5066F58: _snd_config_make_add (in /usr/lib64/libasound.so.2)
==869== by 0x50673B9: parse_value (in /usr/lib64/libasound.so.2)
==869== by 0x50675DE: parse_array_def (in /usr/lib64/libasound.so.2)
==869== by 0x5067680: parse_array_defs (in /usr/lib64/libasound.so.2)
==869== by 0x5067A8E: parse_def (in /usr/lib64/libasound.so.2)
==869== by 0x5067BC7: parse_defs (in /usr/lib64/libasound.so.2)
==869== by 0x5067A6F: parse_def (in /usr/lib64/libasound.so.2)
==869== by 0x5067BC7: parse_defs (in /usr/lib64/libasound.so.2)
==869== by 0x5067A6F: parse_def (in /usr/lib64/libasound.so.2)
==869== by 0x5067BC7: parse_defs (in /usr/lib64/libasound.so.2)
The number of bytes lost just keeps going up and up.