Linux creates virtual memory pages for every program to use, and the OS handles mapping the virtual addresses to genuine hardware addresses, correct?
But how does Windows do this? Do Windows programs actually have memory that translates to real hardware addresses? I'm also aware that windows can use hard disk memory when RAM is over used, and this process is again called virtual memory, but I believe this is an entirely different concept?
Windows and Linux (at least on Intel 32/64 bit systems) both implement virtual memory using the same mechanism: hardware supported page tables. The OS and the hardware cooperate together to do the address mapping.
The entire concept of separating the logical addresses a program uses from the physical addresses is what is called virtual memory. The use of the hard disk as a backing store is an implementation of virtual memory that uses a swap file to increase the amount of virtual memory to an amount greater than the physical memory installed in the system.
Virtual memory is a pretty deep and wide subject. Maybe start with this Wiki article an Memory Management and then hit the googles for a deeper understanding.
Related
I am currently studying about virtual memory in operating system and I have few questions.
Is swap partition or swap file same as virtual memory in terms of Linux?
If yes, then in case I've no swapping enabled in my Linux system, does that mean my system has no virtual memory?
I have also read that virtual memory makes system more secure because with virtual memory, CPU generates virtual addresses which are then translated to actual physical addresses by MMU, therefore securing the system because no process can actually interact with the actual physical memory. So if I just enable swapping on my Linux system, will my CPU start generating virtual addresses and currently it's directly generating physical addresses as I have no swap partition?
How does CPU know if virtual memory is present or not?
Having no swap file/partition doesn't imply that you don't have virtual memory. Modern operating-systems always use paging/virtual memory no matter what.
Is swap partition or swap file same as virtual memory in terms of Linux?
No swap file and virtual memory is not the same in terms of any OS. Virtual memory just says that all memory accesses are going to be translated by the MMU using the page tables. Modern OSes always use paging.
If yes, then in case I've no swapping enabled in my Linux system, does that mean my system has no virtual memory?
Your system certainly has virtual memory. To use long mode (64bits mode), the OS must enable paging. I doubt that you have a system old enough to not use paging. Page swapping to the hard-disk is not virtual memory. It is more like a feature of virtual memory that can be used to extend physical memory because a page which isn't required immediatly can be swapped to the hard-disk momentarily.
I have also read that virtual memory makes system more secure because with virtual memory, CPU generates virtual addresses which are then translated to actual physical addresses by MMU, therefore securing the system because no process can actually interact with the actual physical memory. So if I just enable swapping on my Linux system, will my CPU start generating virtual addresses and currently it's directly generating physical addresses as I have no swap partition?
Your computer certainly has paging/virtual memory enabled. Having no swap partition doesn't mean that you don't have virtual memory. Paging can also be used to avoid fragmentation of RAM and for security. You are right that paging is securing your system because the page tables prevent a process from accessing the memory of another process. It also has ring privilege on a page to page basis which allows to differentiate between kernel mode and user mode code.
How does CPU know if virtual memory is present or not?
The OS just enables paging by setting a bit in a control register. Then the CPU starts blindly translating every memory accesses using the MMU.
No. Swap file is not the same as virtual memory.
Once the firmware/kernel sets up the necessary registers and/or in-memory data structures and switches the processor mode, virtual memory mappings are used for accessing the physical memory.
Yes, the inability of processes to refer to memory locations without a mapping allows the kernel to employ isolation and access control mechanisms.
Through active mappings, different virtual addresses can map to the same physical memory region at different times. The kernel can maintain the illusion that a larger amount of memory is available that the capacity of the actual physical memory, where only a subset of the virtual memory resides in the physical memory at any given time. The rest is stored in the swap file.
Accesses to virtual addresses where the corresponding data is currently in the swap file are trapped by the kernel (via a page fault) and might lead to the kernel swapping the data in, and swapping some other data from physical memory out.
If you disable the swap file, the kernel has no place store the swapped out data. This reduces the amount of virtual memory available.
I am reading about ARM Trustzone at this link. I understand that using TrustZone, one can partition the memory into secure and non-secure regions. Vendors may use this to run a secure OS.
What I am curious about is that what is the granularity support for this partition ? Is it just that there can be a block of memory marked "secure" and there can be only one such block of memory per OS ? Does TrustZone have the capacity to partition memory for individual processes ?
Lets say I have a .so file (hypothetical example) for a Linux application. Could it be possible that the same code in process A could be marked secure in virtual address 0x1000 to 0x2000, while in process B could be marked secure in virtual address 0x5000 to 0x6000 ?
TrustZone partitioning happens at the physical memory level, so the process-level parts of your question don't really apply. Note that Linux as the non-secure OS can't even see secure memory, so having virtual mappings for inaccessible addresses would be of little use; however the secure OS does have the ability to map both secure and non-secure physical addresses by virtue of the NS bit in its page table entries.
As for how that physical partitioning goes, it depends on the implementation. The TZC-380 your link refers to supports 2-16 regions with a minimum 32KB granularity; its successor the TZC-400 has 9 regions, and goes all the way down to 4KB granularity. Other implementations may be different still, although granularity below 4KB is unlikely since that would be pretty much unusable for the CPU with its MMU on. Also, there are usually some things in a system which are going to be hardwired to the secure memory map only (the TZC's programming interface, for one), and that often includes some dedicated secure SRAM.
I'm working with a trustzone (ARM) and unlike memory access in linux, the trusted applet does not have access to the virtual-physical memory mapping. It deals strictly with physical memory.
How would I go about translating from virtual to physical addresses?
So if i understand you don't know which memory area your application can use as it is detached from the virtual address space set in the CPU?
In my opinion you have only two choices:
Somehow tell the OS that a section of memory (physical) is reserved for the application. Let's say from 0xfff0 to 0xffff
The Page Table with the mapping is usually at the end of the kernel in memory. Try to get that address and you can search it for a free adress space (memory block).
How does xen handle E820 memory map for domU kernels? In my specific problem I am trying to map Non-Volatile RAM to domU kernels. The dom0 memory map returns the following relevant parts.
100000000-17fffffff : System RAM (4GB to 6GB)
180000000-37fffffff : reserved (6GB to 14GB)
The second line corrosponds to the NVRAM which is a region from 6GB to 14GB in the dom0 kernel. How can I map this NVRAM region to the domU kernel which does not map this region at all.
Ultimately I want to the nvram region to be available in other domU VMs so any solutions or advice would be highly helpful.
P.S. :: If I attempt to write to this region from the domU kernel will Xen intercept this write operation. Actually this is just a memory region write which should not be a problem, but it might appear as a hardware access.
Guest domains in Xen have two different models for x86:
1. Hardware Virtual Machine (HVM) : It takes benefit of Intel VT or AMD SVM extensions to enable true virtualization on x86 platform
2. Paravirtualized (PV) : This mode adds modifications in the source code of the operating system to get rid of the x86 virtualization problems and also add performance boost to the system.
These two different models handle the E820 memory map differently. E820 memory map basically gives an OS the physical address space to operate on along with the location of I/O devices. In PV mode I/O devices are available through Xenstore. The domain builder only provides a console device during boot to the pv guest. All other I/O devices have to be mapped by the guest. The guest in this mode starts execution in protected mode instead of real mode for x86. The domain builder maps the start_info pages into the guest domain's physical address space. This start_info pages contain most of the information to initialize a kernel such as number of available pages, number of CPUs, console information, Xenstore etc. E820 memory map in this context would just consist of the number of available memory pages because BIOS is not emulated and I/O device information is provided separately through Xenstore.
On the otherhand, in HVM guest BIOS and other devices have to be emulated by Xen. This mode should support any unmodified OS, thus we cannot use the previous method. BIOS emulation is done via code borrowed from Bochs, while devices are emulated using QEMU code. Here an OS is provided with an E820 memory map, build by the domain builder. The HVM domain builder would typically pass the memory layout information to the Bochs emulator which then performs the required task.
To get hold of the NVRAM pages you will have to build a separate MMU for NVRAM. This MMU should handle all the NVM pages and allocate/free it on demand just like the RAM pages. It is a lot of work.
The machine on which I develop has more memory than the one on which the code will eventually run. I dont have access tothe machine on which it will actually run. This is a 64 bit application and I intend to use the address space but cap physical allocation. I dont want to lock down virtual memory, only physical memory. Is there a way to set limits on a linux machine such that it mimics a system with low RAM. I think ulimit does not differentiate between reserved address space vs actual allocation. If there is a way to do it without rebooting with different kernel parameters or, pulling out extra RAM that would be great. May be some /proc tricks.
See https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process which suggests using "timeout" from here: https://github.com/pshved/timeout .
If You can change boot command line of the kernel and want to restrict available memory use
mem=
boot parameter.
For more information check:
https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html