Consider a bare-metal Risc-V supervisor-mode program that is using MMU with page-based virtual memory translation (say, Sv32). And there is a requirement to perform a programmatic translation from virtual address to physical (think of cases that it might be needed to pass to some machine ECALL, or external DMA engine). That is a function similar to following is needed:
uintptr_t virt_to_phys(uintptr_t virt)
{
// .... Calculate physical address phys
return phys;
}
Is there a way to utilize the "existing" hardware page table walk to perform the calculation? Or the only way is to perform a "manual" page table walk, or have some separate lookup table in order to implement this (or some other way I couldn't think of)?
Related
On the surface, this appears to be a silly question. Some patience please.. :-)
Am structuring this qs into 2 parts:
Part 1:
I fully understand that platform RAM is mapped into the kernel segment; esp on 64-bit systems this will work well. So each kernel virtual address is indeed just an offset from physical memory (DRAM).
Also, it's my understanding that as Linux is a modern virtual memory OS, (pretty much) all addresses are treated as virtual addresses and must "go" via hardware - the TLB/MMU - at runtime and then get translated by the TLB/MMU via kernel paging tables. Again, easy to understand for user-mode processes.
HOWEVER, what about kernel virtual addresses? For efficiency, would it not be simpler to direct-map these (and an identity mapping is indeed setup from PAGE_OFFSET onwards). But still, at runtime, the kernel virtual address must go via the TLB/MMU and get translated right??? Is this actually the case? Or is kernel virtual addr translation just an offset calculation?? (But how can that be, as we must go via hardware TLB/MMU?). As a simple example, lets consider:
char *kptr = kmalloc(1024, GFP_KERNEL);
Now kptr is a kernel virtual address.
I understand that virt_to_phys() can perform the offset calculation and return the physical DRAM address.
But, here's the Actual Question: it can't be done in this manner via software - that would be pathetically slow! So, back to my earlier point: it would have to be translated via hardware (TLB/MMU).
Is this actually the case??
Part 2:
Okay, lets say this is the case, and we do use paging in the kernel to do this, we must of course setup kernel paging tables; I understand it's rooted at swapper_pg_dir.
(I also understand that vmalloc() unlike kmalloc() is a special case- it's a pure virtual region that gets backed by physical frames only on page fault).
If (in Part 1) we do conclude that kernel virtual address translation is done via kernel paging tables, then how exactly does the kernel paging table (swapper_pg_dir) get "attached" or "mapped" to a user-mode process?? This should happen in the context-switch code? How? Where?
Eg.
On an x86_64, 2 processes A and B are alive, 1 cpu.
A is running, so it's higher-canonical addr
0xFFFF8000 00000000 through 0xFFFFFFFF FFFFFFFF "map" to the kernel segment, and it's lower-canonical addr
0x0 through 0x00007FFF FFFFFFFF map to it's private userspace.
Now, if we context-switch A->B, process B's lower-canonical region is unique But
it must "map" to the same kernel of course!
How exactly does this happen? How do we "auto" refer to the kernel paging table when
in kernel mode? Or is that a wrong statement?
Thanks for your patience, would really appreciate a well thought out answer!
First a bit of background.
This is an area where there is a lot of potential variation between
architectures, however the original poster has indicated he is mainly
interested in x86 and ARM, which share several characteristics:
no hardware segments or similar partitioning of the virtual address space (when used by Linux)
hardware page table walk
multiple page sizes
physically tagged caches (at least on modern ARMs)
So if we restrict ourselves to those systems it keeps things simpler.
Once the MMU is enabled, it is never normally turned off. So all CPU
addresses are virtual, and will be translated to physical addresses
using the MMU. The MMU will first look up the virtual address in the
TLB, and only if it doesn't find it in the TLB will it refer to the
page table - the TLB is a cache of the page table - and so we can
ignore the TLB for this discussion.
The page table
describes the entire virtual 32 or 64 bit address space, and includes
information like:
whether the virtual address is valid
which mode(s) the processor must be in for it to be valid
special attributes for things like memory mapped hardware registers
and the physical address to use
Linux divides the virtual address space into two: the lower portion is
used for user processes, and there is a different virtual to physical
mapping for each process. The upper portion is used for the kernel,
and the mapping is the same even when switching between different user
processes. This keep things simple, as an address is unambiguously in
user or kernel space, the page table doesn't need to be changed when
entering or leaving the kernel, and the kernel can simply dereference
pointers into user space for the
current user process. Typically on 32bit processors the split is 3G
user/1G kernel, although this can vary. Pages for the kernel portion
of the address space will be marked as accessible only when the processor
is in kernel mode to prevent them being accessible to user processes.
The portion of the kernel address space which is identity mapped to RAM
(kernel logical addresses) will be mapped using big pages when possible,
which may allow the page table to be smaller but more importantly
reduces the number of TLB misses.
When the kernel starts it creates a single page table for itself
(swapper_pg_dir) which just describes the kernel portion of the
virtual address space and with no mappings for the user portion of the
address space. Then every time a user process is created a new page
table will be generated for that process, the portion which describes
kernel memory will be the same in each of these page tables. This could be
done by copying all of the relevant portion of swapper_pg_dir, but
because page tables are normally a tree structures, the kernel is
frequently able to graft the portion of the tree which describes the
kernel address space from swapper_pg_dir into the page tables for each
user process by just copying a few entries in the upper layer of the
page table structure. As well as being more efficient in memory (and possibly
cache) usage, it makes it easier to keep the mappings consistent. This
is one of the reasons why the split between kernel and user virtual
address spaces can only occur at certain addresses.
To see how this is done for a particular architecture look at the
implementation of pgd_alloc(). For example ARM
(arch/arm/mm/pgd.c) uses:
pgd_t *pgd_alloc(struct mm_struct *mm)
{
...
init_pgd = pgd_offset_k(0);
memcpy(new_pgd + USER_PTRS_PER_PGD, init_pgd + USER_PTRS_PER_PGD,
(PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t));
...
}
or
x86 (arch/x86/mm/pgtable.c) pgd_alloc() calls pgd_ctor():
static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd)
{
/* If the pgd points to a shared pagetable level (either the
ptes in non-PAE, or shared PMD in PAE), then just copy the
references from swapper_pg_dir. */
...
clone_pgd_range(pgd + KERNEL_PGD_BOUNDARY,
swapper_pg_dir + KERNEL_PGD_BOUNDARY,
KERNEL_PGD_PTRS);
...
}
So, back to the original questions:
Part 1: Are kernel virtual addresses really translated by the TLB/MMU?
Yes.
Part 2: How is swapper_pg_dir "attached" to a user mode process.
All page tables (whether swapper_pg_dir or those for user processes)
have the same mappings for the portion used for kernel virtual
addresses. So as the kernel context switches between user processes,
changing the current page table, the mappings for the kernel portion
of the address space remain the same.
The kernel address space is mapped to a section of each process for example on 3:1 mapping after address 0xC0000000. If the user code try to access this address space it will generate a page fault and it is guarded by the kernel.
The kernel address space is divided into 2 parts, the logical address space and the virtual address space. It is defined by the constant VMALLOC_START. The CPU is using the MMU all the time, in user space and in kernel space (can't switch on/off).
The kernel virtual address space is mapped the same way as user space mapping. The logical address space is continuous and it is simple to translate it to physical so it can be done on demand using the MMU fault exception. That is the kernel is trying to access an address, the MMU generate fault , the fault handler map the page using macros __pa , __va and change the CPU pc register back to the previous instruction before the fault happened, now everything is ok. This process is actually platform dependent and in some hardware architectures it mapped the same way as user (because the kernel doesn't use a lot of memory).
In nutshell, as I understand memory management, processor produces virtual addresses. These addresses are translated to corresponding physical addresses using per-process address table by MMU (with TLBs and page-faults in-between, as and when needed).
My question is does processor always produces Virtual addresses? In terms of Address-spaces(user/kernel), Processor modes (user/kernel) and contexts (process/system) when all times does processor produce physical addresses?
Memory typically knows nothing about virtual addresses or segments, which are CPU concepts, it is just memory, a collection of addressable and readable/writable bits. The processor talks to memory using physical addresses. Many simple processors (especially old ones or for special embedded uses) have no MMUs, virtual addresses or privileged modes. Those that have MMUs and virtual addresses normally start either with those disabled or they at first use fixed mapping because otherwise nothing would be able to work if there's no mapping at all.
So, physical addresses are always in use, while for virtual addresses it depends on the CPU and the software in use.
Processor is unaware about whether it is physical address or virtual address , it is the job of respective MMU to do the translation.
Processor has to place the address on it address bus , so now the path depends whether MMU is enabled or disabled. if MMU is enabled it will follow the path of MMU translation and respective physical address will get placed on address bus and if MMU is disabled the same address generated by respective instruction will be placed on address bus.
so it is responsibility of programmer if MMU is disabled all address access should be physical address otherwise it will be a exception or abort in system
Generally the CPU ONLY knows "virtual addresses". Ie, when you do any assembly programming, any "load regs, *(memory ptr)" or such-like operation, the addresses are virtual addresses.
The following diagram illustrates well the concept:
http://slideplayer.com/slide/4394245/ (page 7)
Any addresses coming out of CPU, is always virtual. But if the processor has a MMU, then the MMU will intercept the addresses and convert it to physical addresses (through Page Table mechanism) before putting it on the memory bus. So if you sniff the memory bus, you will see physical addresses.
References:
https://www.quora.com/What-is-the-return-address-of-kmalloc-Physical-or-Virtual
Yes when a system works on VM then instructions/programs produce only VA even at the time of booting, addresses generated are not the same as physical despite page tables are still not in existence.
But processor need physical addresses to access memory. So there is mechanism to produce physical addresses from virtual addresses which is called address translation done through page tables. Yes whatever you see in kernel mode/user mode, register values all are virtual addresses. but when processor actually perform computation, processor always uses physical addresses
My question is does processor always produces Virtual addresses? In terms of Address-spaces(user/kernel), Processor modes (user/kernel) and contexts (process/system) when all times does processor produce physical addresses?
An x86 CPU operates with 3 different "kinds" of addresses:
physical: the actual addresses that are used to select the byte in memory. Used for things like segment base addresses, interrupt descriptor table, global descriptor table.
logical: these are addresses that you use most of the time when paging is disabled. They're converted to physical addresses by adding the respective segments (physical) base address.
virtual: the addresses used by most instructions when paging is enabled. These are translated into logical addresses using page directories and tables.
Which kind of address is used isn't dependent on the current privilege level (CPL; "system mode", "user mode" or between). It depends on the state of the processor (paging enabled or not) and the actual instruction (lidt for example). Though I guess it's safe to assume that there won't be physical addressing in "user mode" (CPL > 0) since instructions using physical addresses are usually privileged instructions.
Im currently trying to understand systems programming for Linux and have a hard time understanding how virtual to physical memory mappings work.
What I understand so far is that two processes P1 and P2 can make references to the same virtual adress for example 0xf11001. Now this memory adress is split up into two parts. 0xf11 is the page number and 0x001 is the offset within that page (assuming 4096 page size is used). To find the physical adress the MMU has hardware registeres that maps the pagenumber to a physical adress lets say 0xfff. The last stage is to combine 0xfff with 0x001 to find the physical 0xfff001 adress.
However this understanding makes no sens, the same virtual adresses would still point to the same physical location??? What step is I missing inorder for my understanding to be correct???
You're missing one (crucial) step here. In general, MMU doesn't have hardware registers with mappings, but instead one register (page table base pointer) which points to the physical memory address of the page table (with mappings) for the currently running process (which are unique to every process). On context switch, kernel with change this register's value, so for each running process different mapping will be performed.
Here's nice presentation on this topic: http://www.eecs.harvard.edu/~mdw/course/cs161/notes/vm.pdf
I am little confused about how linux takes advantage of ARMv7 MMU hardware for its 3 level page table walk. MMU has only 2 registers ttbr0 and ttbr1 (one for kernel and other for user-space). How does mmu know know multi-level page table walk of linux?
Thanks,
Hvr
If the upper N bits of the virtual address are all zero then the translation starts at TTBR0 else TTBR1. N comes from the TTBCR. The TTBRn registers contain the physical address of the base of the first-level table. The appropriate entry of the first-level table is loaded and various bits of the entry determine if the translation uses a second-level table and if so what its physical address is.
The MMU can be configured to use Short Descriptors (32-bit physical addresses) or Long Descriptors (40-bit physical addresses). When using short descriptors, at most only two-levels of translation table can be used. When using long descriptors, there can be three levels.
This ignores stage 2 translations (Hypervisors). All is documented in the ARMARM for v7-A&R section B3.3:
http://infocenter.arm.com/help/topic/com.arm.doc.ddi0406c/index.html
I am writing a Linux kernel module that needs to map a specific physical address to a specific virtual one, and I just can't find a way to do it.
OK, This is my current solution. To map phys_addr to virt_addr I use this code:
page = pfn_to_page(virt_addr >> PAGE_SHIFT);
pte = get_locked_pte(&init_mm, phys_addr, &ptl);
set_pte_at(&init_mm, phys_addr, pte, mk_pte(page, VM_READ | VM_WRITE | VM_EXEC));
spin_unlock(ptl);
flush_tlb_all();
Some explenations: I use the pfn_to_page func to get the page struct corresponding to my virt_addr. I get the page table entry (pte) with the get_locked_pte func which needs the physical address corresponding to the wanted pte and an uninitalized spinlock (ptl). Then, I actually map the page using set_pte_at func and the mk_pte macro, unlock the spinlock and flush the tlb cache.
This solution seems to work pretty well, though it does not survive a context switch.
Could you tell us the kernel version and CPU architecture/type you are using? Generally speaking, if the specific virtual address you want to mapping to does not overlap with kernel virtual address(such as 0xC0000000), and also if the physical address which your device is using does not overlap with system memory physical address range, you can using the low-level functions(if there is not, you can using assemble language to set up MMU TLB entries directly during kernel booting up) to set up MMU TLB entries to map the specific address to a specific virtual one during kernel booting up. I can provide one example based on 2.6.10 kernel version and Freescale PowerPC CPU, there is a function io_block_mapping to do the thing you want.