Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
What happens when physical memory is fully occupied by process and a new process(similar priority) is introduced. How does the Memory Management unit handle the pages(resource) requested by the new and old processes(same priority tasks).
So I mean to ask how swapping of memory done for similar priority process and physical memory is full on the other side. Please explain with an example?
You should not care about what happenning in that case, and on current Linux deskops & laptops it is an improbable case (because usually the kernel steals page from the filesystem cache).
When a new program is started with the execve(2) syscalls, new memory mappings are set up (as if nearly done by mmap(2)), possibly with copy-on-write mechanism. Once the program is accessing them, the kernel will page-fault and ultimately load the page in physical RAM. It may have to choose which pages should be stealed. If they are dirty, it has to write them to some swap zone (or to some mmap-ed file if the mapping is MAP_SHARED). Otherwise, it just reuses them (and reassign the physical pages).
If all memory resources are used, memory overcommit may happen
The MMU is used by the linux kernel for virtual memory management. Applications see on some virtual address space (look into /proc/ e.g. with cat /proc/self/maps to understand it).
The MMU is doing the virtual to physical address translation and is giving page faults. The kernel is responsible for configuring the MMU (i.e. setting up the virtual address space translation mechanism) and for handling page faults (which are usually invisible to the application -e.g. because the kernel would fetch a page from the disk, the filesystem or the swap area-, except as the SIGSEGV signal which occurs when a "non-existent" page is accessed).
please take time to read all the links given here.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
The kernel boots with default_hugepagesz=1G option, which defines size of the default page size. So when an application want large memory, the kernel will allocate it with 1G pages.
If the kernel boots with hugepages=N, i.e. allocate N huge pages at boot. So in this case, the kernel will automatically take a page from this pool, thus saving time on allocating memory?
When this pool runs out of available pages, how will the kernel allocate huge memory?
hugepages kernel options reserves contiguous ranges of page frames (RAM), so that the user can allocate that many huge pages without fail.
When there are no reserved contiguous huge pages the kernel tries to find more, which may fail when the physical memory becomes fragmented.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 7 years ago.
Improve this question
I found at several places that Linux uses pages and a paging mechanism but I didn't find anywhere where this file is or how to configure it.
All the information I found is about the Linux swap file / partition. There is a difference between paging and swapping:
Paging moves pages (a small frame which contains a piece of data - usually 4 KB but can vary between different OS's) from main memory to a backbend storage, happens always as a normal function of the operating system.
Swapping moves an entire process to storage and happens when the system is memory stressed or on windows 8 when a new application is hibernating.
Does Linux uses it's swap file / partition for both cases?
If so, how could I see how many page are currently paged out? This information is not there in vmstat, free or swapon commands (or that I fail to see it).
Or is there another file used for paging?
If so, how can I configure it (and watch it's usage)?
Or perhaps Linux does not use paging at all and I was mislead?
I would appreciate if the answers will be specific to red hat enterprise Linux both versions 6 and 7 but also a general answer about all Linux's will be good.
Thanks in advance.
On Linux, the swap partition(s) are used for paging.
Linux does not respond to memory pressure by swapping out whole processes. The virtual memory system does demand paging, page by page. Under extreme memory pressure, one or more processes will be killed by the OOM killer. (There are some useful links to documentation in the first NOTE in man malloc)
There is a line in the top header which shows swap partition usage, but if that is all the information you want, use
swapon -s
man swapon for more information.
The swap partition usage is not the same as the number of unmapped pages. A page might be memory-mapped to a file using the mmap call; since that page has backing store in the file, there is no need to also write it to a swap partition, and the system won't use swap space for that. But swap partition usage is a pretty good indicator.
Also note that Linux (unlike Windows) does not allocate swap space for pages when they are allocated. Instead, it adds the new page to the virtual memory map without any backing store. and allocates the swap space when the page needs to be swapped out. The consequence (as described in the malloc manpage referenced earlier) is that a malloc call may succeed in allocating virtual memory, but a subsequent attempt to use that virtual memory may fail.
Although Linux retains the term 'swap partition' as a historical relic, it actually performs paging. So your expectation is borne out; you were just thrown by the archaic terminology.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
How the OS is able to do this
With virtual memory, programs running on the system can allocate far
more memory than is physically available;
It is in practice "a little more memory", not "far more memory", otherwise you are experimenting thrashing.
Every desktop, latop or server processor has an MMU. It is used by the virtual memory system to give a virtual address space thru paging & the page cache. When the kernel gets a page fault, it could fetch a page from disk -e.g. in a segment of an ELF executable or shared object or some other mapped file, or some pages from the swap area- or send a SIGSEGV signal, see signal(7).
On Linux, several system calls can change the address space: mmap(2) and munmap (and also the obsolete sbrk, etc...) and execve(2). You might advise the kernel using madvise(2)
You could use cat /proc/$somepid/maps (e.g. cat /proc/$$/maps in your shell) to understand the address space map of some process. See proc(5).
Follow all the links above and read also Advanced Linux Programming and Operating Systems: Three Easy Pieces
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
AFAIK there's a partition of a process memory that stores kernel related data and it's marked as read-only.
I can't find a factual explanation for why this happens, what is the purpose of this area and why should you include it in every process memory space ?
Just like the user-mode memory space, the kernel needs its own code section (RX), data section (R/RW), and stack frames for the threads (RW).
I would not say that it needs to be included in process memory space, but rather say that it's where the kernel always resides. Unlike the process memory space that gets replaced whenever there's a context switch between processes, the kernel space (>=0xC0000000 in 32bit and >=0xFFFFFFFF80000000 in 64bit), in its entirety, never gets replaced.
This is a necessary requirement since there's only one kernel on a system and it must remain at the same place in the memory (virtual) at all times for handling system calls, interrupts, and running various kernel tasks.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
Does virtual memory exists somewhere in our computer system in reality(i.e on hard disk )?
if not how a mapping from virtual memory to real data in hard disk is made if data is not in main memory i.e.( page fault occurs ).Is there any table that maintains the mapping from virtual memory to hard disk data..
Memory is so called virtual because a process sees its address-space as a contiguous chunk of available memory, using all the breadth of the underlying address bus width, let say 4GB for a 32bits system. So every single process has a 4GB address-space, yet this memory is not fully backed by physical memory on a 1-to1 basis. And even though you have 4GB of physical memory to back the 4GB address-space of the process, where would go the kernel, the others process? This memory has to be virtual.
Yes, tables maintain the process address-space. To make it simple, some of the pages are currently mapped on the volatile physical memory, but some others are not. They are backed by a memory file on the HDD. When a page-fault occurs, the page-fault will check if that page is mapped on the physical-memory (usually it’s a bit inside the page’s attributes), and if not, it will fetch it from the memory mapped file on the HDD, and replace with it an old page mapped to the physical memory.
Hope this help.
Yes virtual memory really does exist and yes there is a table that maintains the mapping. Look for page table in wikipedia for instance. In fact most of the virtual memory article will answer your question in full.
Most of your questions are answered by http://en.wikipedia.org/wiki/Virtual_memory.
A backing store must exist for virtual memory. This is usually a hard disk. Basically its some other device that ua usually slower than RAM but is much bigger in capacity.
When a page fault occurs, the page is obtained from the backing store
The page table contains information on where in the backing store the page is to found
Short answer, no :) Virtual Memory is virtual!
Especially if you consider virtual memory as "the memory that can be addressed by a process". On 64 bit systems, the whole disk hardly could back the entire virtual memory. So "in reality", as you asked, I would say no.
Long(-ish) answer: virtual memory exists as a series of data structures in the kernel. They mostly keep trace of which page/segment is currently reserved, allocated, mapped to a file or mapped to physical memory.
Also, the answer is different if what you look at is "allocated virtual memory". This always exists in one form or another (usually, pages backed by hard-disk swap space).
Yes, most used bytes of virtual memory exist somewhere. I say "most" because pages that map registers of some special hardware can have holes. But all memory allocated by you app exists either in RAM or on hard disk.
The wikipedia article explains all the details: http://en.wikipedia.org/wiki/Virtual_memory