Is there a way to share memory between OAL and kernel driver? Between OAL and kernel there is NKGlobal structure which can be used to share memory. I would like a similar approach to share the memory. I am using Windows Embedded Compact 2013 on Xilinx board.
Thus far, in the OAL layer I did VirtToPhys to an allocated memory to get the physical address. In the kernel driver I tried VirtualAllocCopyEx but I dont know what to pass in the pAddr.
LPVOID VirtualAllocCopyEx (
HANDLE hSrcProc,
HANDLE hDstProc,
LPVOID pAddr,
DWORD cbSize,
DWORD dwProtect
);
If the driver is running in kernel mode it's sharing the same address space of the OAL, so you can pass pointers directly. You can allocate your buffer in the driver or in the OAL and share a value using an IOCTL implemented in OEMIoControl in your OAL (if your BSP uses the common production quality layer you may check how other IOCTLs are implemented, usually you need to add a record in a table of structs and implement a function to manage your specific IOCTL).
Both components can access the memory using that pointer.
Related
I have a custom device driver that implements an mmap operation to map a shared RAM buffer (outside of the OS) to userspace. The buffer is reserved by passing mem=32M as a boot argument for the OS, leaving the rest of the 512MB available as a buffer. I would like to perform zero-copy operations from the mapped memory, which is not possible if the vm_flags include VM_PFNMAP and VM_IO.
My driver currently performs the mapping by calling vm_iomap_memory(vma, start, size), which in turn calls io_remap_pfn_range and remap_pfn_range, which sets up the vma with the VM_PFNMAP and VM_IO set. This works to map the memory to userspace, but zero-copy socket operations fail at get_user_pages due either to the VM_PFNMAP flags being set or the struct page being missing. The comments for remap_pfn_range show this is intended behavior, as pfn-mapped memory should not be treated as 'normal'. However, for my case it is just a block of reserved RAM, so I don't see why it should not be treated as normal. I have set up cache invalidation/flushing to manually manage the memory.
I have tried unsetting the VM_PFNMAP and VM_IO flags on the vm_area_struct both during and after the mapping, but get_user_pages still fails. I have also looked at the dma libraries but it looks like they rely on a call to remap_pfn_range behind the scenes.
My question is how do I map physical memory as a normal, non-pfn, struct page-backed userspace address? Or is there some other way I should be looking at it? Thanks!
I've found the solution to mapping a memory buffer outside the Kernel that requires a correction to several wrong starting points that I mentioned above. It's not possible to post full source code here, but the steps to get it working are:
Device tree: Define reserved memory region for buffer with no associated driver. Do not use mem or memmap bootargs. Kernel will confine itself to using memory outside of this reserved space for itself, but now will be able to make struct pages for reserved memory.
In a device driver (a LKM in my case), mapping physical address to kernel virtual address requires using using memremap instead of ioremap, as it is real memory we are mapping.
In device driver mmap routine, do not use any variant of remap_pfn_range to setup the vma for usespace, instead assign a custom fault nopage routine to the vma->vm_ops.fault to look up the page when the userspace virtual address is used. This approach is described in lddv3 ch15.
The nopage function in the driver should use the vm_fault structure argument that is passed to it to calculate the offset into the vma for the address that needs a page. Then use that offset to calculate an kernel virtual address (against the memremap'd address), and get the page with a call to page = virt_to_page(pageptr);, followed by a call to get_page(page);, and assign it to the vm_fault structure with vmf->page = page; The latter part of this is illustrated in lddv3 chapter 15 as well.
The memory mapped in this fashion using mmap against the custom device driver can be used just like normal malloc'd memory as far as I can tell. There are probably ways to achieve a similar result with the DMA libraries, but I had constraints preventing that route, or associating the device tree node with the driver.
I've been struggling with this one, would really appreciate some help. I want to use the internal SRAM (stepping stone - not used after boot) of my At91sam9g45 to speed up some intensive computations and am having trouble meeting all the following conditions:
Memory is accessible from user space. This was easy using the user space mmap() and then kernel remap_pfn_range(). Using the pointer returned, my user space programs can read/write to the SRAM.
Using the kernel DMA API call dma_async_memcpy_buf_to_buf() to do a memcpy using DMA. Within my basic driver, I want to call this operation to copy data from DDR( allocated with kmalloc()) into the SRAM buffer.
So my problem is that I have the user space and physical addresses, but no kernel-space DMA API friendly mapping.
I've tried using ioremap and using the fixed virutal address provided to iotable_init(). None of these seems to result in a kernel virtual address that can be used with something like virt_to_bus (which works for the kmalloc addresses and i think is used within the DMA API).
There's way around and thats just triggering the DMA manually using the physical addresses, but I'd like to try and figure this out. I've been reading through LDD3 and googling, but i can't see any examples of using non-kmalloc memory for the DMA API (except for PCI buses).
I am working on a small embedded system. When my linux boots up into user space, I know where are my devices in the physical memory. I want to map them into user space virtual addresses. Currently, I am doing it through a kernel module. I use vmalloc/kmalloc (depending on the size) and then I use ioremap_page_range on that returned virtual addresses to map my physical addresses. I dont think that is the correct way to go about. First of all I am allocating memory and then I am asking kernel to remap that virtual address space to some different physical address space. (Initially mapped physical->virtual in vmcall/kmalloc is kinda useless as I dont care about those physical pages. This is definitely not good.)
Instead of this is there a better way to map the known physical memory into user space process. (I know other than my user space process, no one gonna touch that memory.)
Thanks
What you are trying to do is accessing what is called IO memory. I can only encourage you to read the Linux Device Drivers (LDD) book and more specifically the chapter 9.
To "allocate" such an area, you need to call
struct resource *request_mem_region(unsigned long start, unsigned long len, char *name)
. Before your driver can access it, you have to assign it a virtual address, this is done with a call to
void *ioremap(unsigned long phys_addr, unsigned long size)
To ensure that your driver will then work on different architectures/platforms, be sure to use some accessor function to such areas ( ioread8/16/32 or iowrite8/16/32 and all of their variants).
In Kernel module, remap_pfn_range() can be used to convert the physical address to virtual address. The following link will be helpful.
How remap_pfn_range remaps kernel memory to user space?
In Kernel module, remap_pfn_range() can be used to convert the physical address to virtual address. When you don't have a actual devices you can:
1) create a virtual device and,
2) use mmap to those virtual devices to access the very same kernel memory through remap_pfn_range virtual mapping of that process.
3) Usually in dedicated environments you may addition want to pin those physical pages lest they are taken away from your process.
4) You also share these physical addresses with different processes but will need to handle synchronization, independently through other IPC mechanisms as to each process they will look as different addresses.
I have a PCI device that needs to read and write from userspace. I'm trying to use zero copy; is there a way to allocate, pin, and get the physical address of a userspace address completely within userspace or do I need to have a kernel module that, say, calls virt_to_phys or get_user_pages? The device's memory is mapped into userspace memory via MMIO so I can pass it any data that's needed. Thanks.
It was a total hack, but I limited Linux to a range of memory and used MMIO to allocate memory for my device that the kernel was unaware of.
Basically you need the memory to be DMA-able, and as far as I know only a kernel module can do that. See http://lxr.free-electrons.com/source/Documentation/PCI/PCI-DMA-mapping.txt
LDD3 (p:453) demos dma_map_single using a buffer passed in as a parameter.
bus_addr = dma_map_single(&dev->pci_dev->dev, buffer, count, dev->dma_dir);
Q1: What/where does this buffer come from?
kmalloc?
Q2: Why does DMA-API-HOWTO.txt state I can use raw kmalloc to DMA into?
Form http://www.mjmwired.net/kernel/Documentation/DMA-API-HOWTO.txt
L:51 If you acquired your memory via the page allocator kmalloc() then you may DMA to/from that memory using the addresses returned from those routines.
L:74 you cannot take the return of a kmap() call and DMA to/from that.
So I can pass the address returned from kmalloc to my hardware device?
Or should I run virt_to_bus on it first?
Or should I pass this into dma_map_single?
Q3: When the DMA transfer is complete, can I read the data in the kernel driver via the kmalloc address?
addr = kmalloc(...);
...
printk("test result : 0x%08x\n", addr[0]);
Q4: Whats the best way to get this to user-space?
copy_to_user?
mmap the kmalloc memory?
others?
kmalloc is indeed one source to get the buffer. Another can be alloc_page with the GFP_DMA flag.
The meaning is that the memory that kmalloc returns is guaranteed to be contiguous in physical memory, not just virtual memory, so you can give the bus address of that pointer to your hardware. You do need to use dma_map_single() on the address returned which depending on exact platform might be no more then wrapper around virt_to_bus or might do more then do (set up IOMMU or GART tables)
Correct, just make sure to follow cache coherency guidelines as the DMA guide explains.
copy_to_user will work fine and is the easiest answer. Depending on your specific case it might be enough or you might need something with better performance. You cannot normaly map kmalloced addresses to user space, but you can DMA into user provided address (some caveats apply) or allocate user pages (alloc_page with GFP_USER)
Good luck!