All.
Let's suppose that process'A allocate a lot of pages by such as below code.
And process'A periodically executes this code so it happens memory leak.
// allocates 1Mb
for(i=0;i<10;i++)
{
page_p=alloc_pages(gfp_mask, 8);
}
BTW, what become of the allocated pages after killing process without free page?
Allocated pages are permanently leak?
In Linux you have virtual memory, which is a per process memory map. The processes memory is allocated from this map, and the OS maps this memory into physical memory, either RAM or swap.
When a process exits, the OS removes the processes memory map, and another process can reuse it. So leaked memory is only leaked when the process is running.
Related
I'm new to Linux and computer architecture, just some questions on how process and thread related to virtual memory and physical memory RAM.Below is my questions.
Q1-When there is two processes(process A and process B) running concurrently, if process A is running now, the process B's states like register values, heap objects etc have to be pushed to store on disk (Virtual Memory), and when the next context switch happens, process B will be "recovery" from disk to RAM, process A's state will be pushed to disk, is my understanding correct?
Q2- If my understanding in Q1 is correct, why not just save all processes on RAM too? normally we have large RAM like 16gb,32gb etc, how about just store every process's state on RAM, and when there is too many processes and RAM is going to run out, then further processes' states will be stored to disk?
Q3-How about threads? if there is multiple threads (e.g thread A and thread B), when thread A is running, does thread B's state will be pushed to stored on disk too?
is my understanding correct?
No, it's wrong. Waiting or blocked processes don't get swapped to disc. They wait in memory. Virtual memory is not on disc.
Also on a system with two processors, two processes are running concurrently, so both processes A and B can be running at the same time.
why not just save all processes on RAM too?
This is exactly what happens. All processes memory kindly waits in RAM until scheduler switches to this process.
Side note: If there is no RAM available and the system has swap available and this process is idle for some defined time, than it may get swapped on disc, ie. the processes memory may get moved to disc. But this doesn't happen immediately, it happens after a long time and in certain situation
will be pushed to stored on disk too?
No.
Virtual memory is not about physical location of the memory. It's the other way round - virtual memory is a of abstraction that allows system to modify the physical (maybe if any) location of the memory. A simplest explanation I give: there is a special cpu register that is added to each address upon dereferencing. A user space program does *(int*)4 but he doesn't get the value behind 4th byte in RAM, the special cpu register value is added to the pointer value upon dereferencing. The register value is configured by the system, can be different in different programs. So you can have exact same pointer values in two programs, but they both point to different locations. Of cause, this is over-over-simplification.
I always read that, at any given time, the processor can only run one process at a time. So one and only one process is in state running.
However, we can have a number of runnable processes. These are all of these processes who are waiting for the scheduler to schedule their execution.
At any given time, do all these runnable processes exist in user address space? Or has the currently running process in user address space, and it is only once they are scheduled that they are brought back to RAM from disk. In which case, does it mean that the kernel keeps the process task descriptor in its list of all runnable processes even if they are in disk? I guess you can tell I am confused.
If CPU supports virtual memory addressing, each process has a unique view of the memory. Two different processes that tries to read from the same memory address, will map to different location in physical memory, unless the memory maps tells otherwize (shared memory, like DLL files are mapped read only like this for instance)
If CPU does not support virtual memory, but only memory protection, the memory from the other processes will be protected away, so that the running process can only access its own memory.
I detected my service process leaking memory on a Linux server, it takes 1.2G of physical memory and consumes more and more.
While I am looking at the code for the memory leak, I notice the process is restarted (This process if managed by supervisord, so it is restarted if killed). There is no error log or panic in the log of the process. So my guess is that it is killed by the kernel.
When does the kernel kill a process that is leaking memory? When it consumes too much memory? or it allocates memory too fast?
Memory leaks can cause your system memory to get low. If the memory gets very low, the OOM(Out Of Memory) killer will be invoked to try to recover from low memory state. The OOM Killer will terminate one or more processes that consume more memory and are of least importance(low priority). Normally, the OOM killer will be invoked incase there is no user address space available or if there is no page available.
OOM killer uses select_bad_process(),badness() to determine and kill the process. These functions determine the process by assigning points/score for all the processes based on various factors such as VM size of process, VM size of its children, uptime, priority, whether it does any hardware access, whether it is swapper or init or kernel thread. The process with highest points/ score(badness) gets terminated/killed.
Also, checkout whether the overcommit behaviour of kernel (/proc/sys/vm/overcommit_memory, /proc/sys/vm/overcommit_ratio) and the limit on the address space for the processes are appropriate.
Valgrind is a very handy tool in such scenarios in identifying memory leaks.
When a process P1 is in a blocked or suspended state, will the memory management system swap it out of main memory for room for an active process?
And if the process is determined to come back where is the Program's procedure call stack, Contents of program counter (PC) and Contents of program status word (PSW) stored? Does the OS keep it all in secondary memory or is part of the suspended/blocked process of P1 kept in main memory?
So I'm guessing when a process is swapped out of memory and put in a
suspended state, all of its resident pages are moved out. When the
process is resumed, all of the pages that were previously in main
memory are returned to main memory
Think in terms of pages, not processes.
Even an active process may have many pages evicted out of physical memory and into swap if the system is under memory pressure.
So, sure, a suspended process may have effectively all of its pages swapped out entirely.
But it is unlikely to have all pages swapped in simply because the process woke up. Doing so would be a waste of CPU, I/O and memory. Instead, pages will be brought back as needed (general case -- some pagers may bring back sets of pages heuristically).
If a process is active, then it won't be swapped out, so the dynamic state of the lowest call stack (all the register noise, red zone on stack, etc... ) isn't in play when the swap happens.
I.e. for a process to be swapped out the threads need to be blocked on something, typically a call into the kernel or into a system library that is blocking. Registers will be out of play, etc... Thus, the execution state that needs to be swapped out is pretty straightforward as the call return state will be preserved in the thread state itself (as the thread is blocked).
In fact, things like the PC and the PSW are preserved more as a part of the context switching subsystem than paging. I.e. on a typical system, you'll likely have several hundred, maybe thousands, of threads running at once across the N physical cores of the CPU. The concurrency support of the architecture is where you'll find how that state is maintained.
I'm writing a device driver that, among other things, allocates a block of memory with kmalloc. This memory is freed when the user program closes the file. In one of my experiments, the user program crashed without closing the file.
Would anything have freed this memory?
In another experiment, I moved the kfree() from the close() function to the module_exit() function. When I ran the user program twice consecutively, I called kmalloc again with the same pointer as before, without freeing it first. Thus, I lost a pointer to that memory, and cannot free it.
Is this memory lost to the system until I reboot, or will it be freed when I unload the driver?
Kernel memory is never freed automatically. This includes kmalloc.
All memory related to an open file descriptor should be released when the file is closed.
When a process exits, for any reason whatsoever (including kill -9), all open file descriptors are closed, and the driver's close function is called. So if you free there, nothing the process can do will make the memory stay after the process dies.
Please don't relate your user-space experience with Kernel programming.
What do I mean by this?
Normal processes get a clean-up for them once they exit, that's not the case with kernel modules because they're not really processes.
Technically, when you load a module and then call kmalloc, what you did was that you asked the kernel to allocate some memory for you in the kernel space, it's technically a new memory for the whole kernel so even if you unload your module, that allocated kernel memory is there unless explicitly freed.
In simple terms answering your question:
Every kmalloc needs a kfree, else the memory will remain there as long as the system is up.