How to access physical addresses from user space in Linux? - linux

Question

On a ARM based system running Linux, I have a device that's memory mapped to a physical address. From a user space program where all addresses are virtual, how can I read content from this address?

Answer 1

busybox devmem
busybox devmem is a tiny CLI utility that mmaps /dev/mem.
You can get it in Ubuntu with: sudo apt-get install busybox
Usage: read 4 bytes from the physical address 0x12345678:
sudo busybox devmem 0x12345678
Write 0x9abcdef0 to that address:
sudo busybox devmem 0x12345678 w 0x9abcdef0
Source: https://github.com/mirror/busybox/blob/1_27_2/miscutils/devmem.c#L85
mmap MAP_SHARED
When mmapping /dev/mem, you likely want to use:
open("/dev/mem", O_RDWR | O_SYNC);
mmap(..., PROT_READ | PROT_WRITE, MAP_SHARED, ...)
MAP_SHARED makes writes go to physical memory immediately, which makes it easier to observe, and makes more sense for hardware register writes.
CONFIG_STRICT_DEVMEM and nopat
To use /dev/mem to view and modify regular RAM on kernel v4.9, you must fist:
disable CONFIG_STRICT_DEVMEM (set by default on Ubuntu 17.04)
pass the nopat kernel command line option for x86
IO ports still work without those.
See also: mmap of /dev/mem fails with invalid argument for virt_to_phys address, but address is page aligned
Cache flushing
If you try to write to RAM instead of a register, the memory may be cached by the CPU: How to flush the CPU cache for a region of address space in Linux? and I don't see a very portable / easy way to flush it or mark the region as uncacheable:
How to write kernel space memory (physical address) to a file using O_DIRECT?
How to flush the CPU cache for a region of address space in Linux?
Is it possible to allocate, in user space, a non cacheable block of memory on Linux?
So maybe /dev/mem can't be used reliably to pass memory buffers to devices?
This can't be observed in QEMU unfortunately, since QEMU does not simulate caches.
How to test it out
Now for the fun part. Here are a few cool setups:
Userland memory
allocate volatile variable on an userland process
get the physical address with /proc/<pid>/maps + /proc/<pid>/pagemap
modify the value at the physical address with devmem, and watch the userland process react
Kernelland memory
allocate kernel memory with kmalloc
get the physical address with virt_to_phys and pass it back to userland
modify the physical address with devmem
query the value from the kernel module
IO mem and QEMU virtual platform device
create a platform device with known physical register addresses
use devmem to write to the register
watch printfs come out of the virtual device in response
Bonus: determine the physical address for a virtual address
Is there any API for determining the physical address from virtual address in Linux?

Answer 2

You can map a device file to a user process memory using mmap(2) system call. Usually, device files are mappings of physical memory to the file system.
Otherwise, you have to write a kernel module which creates such a file or provides a way to map the needed memory to a user process.
Another way is remapping parts of /dev/mem to a user memory.
Edit:
Example of mmaping /dev/mem (this program must have access to /dev/mem, e.g. have root rights):
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
if (argc < 3) {
printf("Usage: %s <phys_addr> <offset>\n", argv[0]);
return 0;
}
off_t offset = strtoul(argv[1], NULL, 0);
size_t len = strtoul(argv[2], NULL, 0);
// Truncate offset to a multiple of the page size, or mmap will fail.
size_t pagesize = sysconf(_SC_PAGE_SIZE);
off_t page_base = (offset / pagesize) * pagesize;
off_t page_offset = offset - page_base;
int fd = open("/dev/mem", O_SYNC);
unsigned char *mem = mmap(NULL, page_offset + len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, page_base);
if (mem == MAP_FAILED) {
perror("Can't map memory");
return -1;
}
size_t i;
for (i = 0; i < len; ++i)
printf("%02x ", (int)mem[page_offset + i]);
return 0;
}