DMA engine is not responding correctly on PowerPC linux. When my PCIe device sends a read / write request to host, timeout happens. I have 1GB of RAM at lower address range.
I have called the following functions
pci_device_enable()
pci_set_master()
DMA memory is allocated at 0x0F00_0000 and am able to access this memory from user space from host. How can I debug this problem. What could be the reason?
Please check if the PCIe device is using the correct address for the DMA memory. You only mention one address, but the address(es) of the DMA memory in user and kernel space will be different than the address of the DMA memory as seen from the PCIe bus.
You don't say how you are allocating the DMA memory and there are several different ways.
For instance, if you use the following call to get the DMA memory, the dma_addr_t is the address
to use in the PCIe device.
void *pci_alloc_consistent(struct pci_dev *dev, size_t size,
dma_addr_t *dma_handle);
Check if you defined CONFIG_FSL_DMA part of your kernel config to compile the DMA driver. Also make sure your device tree has an entry for the DMA device on chip.
Related
In my opinion, SPI and DMA are both controllers.
SPI is a communication tool and DMA can transfer data without CPU.
The system API such as spi_sync() or spi_async(), are controlled by the CPU.
So what is the meaning of SPI with DMA, does it mean DMA can control the SPI API without CPU? Or the SPI control uses CPU but the data transfer to DMA directly?
SPI is not a tool, it is a communication protocol. Typical micro controllers have that protocol implemented in hardware which can accessed by read/write to dedicated registers in the address space of the given controller.
DMA on micro controllers is typically designed to move content of registers to memory and visa versa. DMA can sometimes configured to write a special amount of read/writes or increasing or decreasing source and target address of memory and so on.
If you have a micro controller which have SPI with DMA support, it typically means that you can have some data in the memory which will be transferred to the SPI unit to send multiple data bytes without intervention of the cpu core itself. Or read an amount of data bytes from SPI to memory automatically without wasting cpu core.
How such DMA SPI transfers are configured is written in the data sheets of the controllers. There are a very wide span of types so no specific information can be given here without knowing the micro type.
The linux APIs for dealing with SPI are abstracting the access of DMA and SPI by using the micro controller specific implementations in the drivers.
It is quite unclear if you want to use the API to access your SPI or you want to implement a device driver to make the linux API working on your specific controller.
It is not possible to give you a general introduction to write a kernel driver here or clarify register by register from your data sheets. If you need further information you have to make your question much more specific!
I am having trouble getting bus-mastering DMA working on a device driver in Ubuntu with kernel 5.3.0-68-generic.
I have enabled bus mastering with pci_set_master (when using lspci -v, the PCIe device will have the bus_master flag) and I am allocating a DMA buffer with dma_alloc_coherent.
I take the dma_addr_t returned by the dma alloc and pass that to the device and then I use the kernel virtual address with a chrdev mmap driver to map the address into userspace (using remap_pfn_range) where a userspace driver can populate the DMA memory region.
It doesn't appear that the the PCIe device can see the memory updates in the DMA region, is there perhaps some dma, iommu, or pci settings I need to enable to allow the PCIe device to read back into system memory as the bus master?
Existing kernel drivers such as xilinx have specific way to be registered (as tty device), if they are mapped directly to cpu memory map as done here with device tree:
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842249/Uartlite+Driver
But in other cases, there is a PCIe device (like FPGA which has the xilinx uart IPs) which is connected to and the cpu.
How should we make the uart get registered when using PCIe device ?
The device tree I try to register into PCIe is uartlite driver:
https://github.com/Xilinx/linux-xlnx/blob/master/drivers/tty/serial/uartlite.c
I think that what I probably need to do is:
Write a custom pci driver.
Need to prepare platform_device struct and then call the uart probe routine from pci driver:
ulite_probe(struct platform_device *pdev)
I've seen related question with others using FPGA with multiple device connected, but seems that there is no docuemnt, or tutorial which describes how to do this.
Any comment, example or document is appreciated.
So something like a ARM CPU connected to an Artix FPGA over PCIe right?
Yes, you would need a custom PCIe driver. The PCIe configuration and data spaces would have to be mapped. Have a look at pci_resource_{start, len} and pci_remap_bar functions. You can then use pci_get_device to get a pointer to the struct device and retrieve the virtual address of the PCIe configuration space. The UART driver can then use the struct device pointer and it's register map should be at some offset to the virtual address of the PCIe configuration space as per your design. You can invoke the probe call of UARTlite IP driver in your own driver.
"Existing kernel drivers such as xilinx have specific way to be registered (as tty device), if they are mapped directly to cpu memory map as done here with device tree". Note that this is true if we are only talking of tty devices. A GPIO peripheral IP won't be expose as tty but in /sys/class/gpio.
I've finished developing a pcie driver for an FPGA under a linux distributiuon. Everything works fine. But I'm wondering where the base address register in the PCI Endpoint of the FPGA gets the base address. When I've generated the PCIe Endpoint I was able to set up the length of the BAR, but not more.
In the PCIe driver I do the standard functions like pci_enable_device, but I do not set up specifically a base address.
So does the operating system set up the base address during startup? or how does it work?
By the side I would like to know what initialisations the operating system gernerally do if an pcie pcie device is connected. Since I do see my pci device in lspci even if the driver is unloaded.
Kind regards
Thomas
The address allocation for the PCI devices are generally done at the BIOS level. Let us refer to the x86 platform. If we look closely at the system address map, it would be something like this (image taken from BIOS DISASSEMBLY NINJUTSU, by Darmawan Salihun)
On the address map, there is a dedicated space to map the PCI memory regions. The same could be replicated using the output of /proc/iomem.
This implementation is platform dependent, and as the BIOS "knows" about the platform, it would set aside the addresses dedicated to the PCI slots. When a device is plugged into the slot, the BIOS interacts with the firmware on the device and actually sets up the memory regions for the device, such that the OS could make use of it.
Now coming to the drivers part. In Linux, the drivers follow a specific standard known as the 'Linux Device Model', which constitutes a Core Layer(PCI core), Host Controller Drivers(PCI Controller/Masters) and Client Drivers(PCI devices). When the PCI device(client) is plugged into the slot, the corresponding host controller knows about the attachment and it further informs the PCI core about it, and hence appears in the output of lspci.
lspci shows the devices which are identified by the host controller, in which case, it may or may not be tied to a driver. The core further traverses the drivers in the system, finds a matching one, and attaches to this device.
So, the reason you are seeing the device in the output of lspci is because the host controller has identified the device, and has informed the PCI core. It doesn't matter even if any driver is attached to the device or not.
On most consumer grade computers, BAR allocation seem to be done in the BIOS.
I suppose that in a hotplug capable architecture this must be done or at least triggered by the OS.
I need to find a way to trigger DMA operations easily at my command to facilitate hardware debugging. Is it possible to initialize a DMA read on existing PCI device (e.g. sound card or netcard) in my Linux, by writing directly to its registers? Or do I have to write a custom driver and invoke it by insmod?
There is no standard way to start a DMA operation. Generally, you need to prepare a DMA buffer on the host and setup DMA registers on your device, load DMA address(s), size(s) etc.
However, there is no single standard for DMA registers for PCI devices.
You need to find the specification document of your PCI device. In that spec, look for DMA chapter (this is also called PCI "master access" as opposed to "target access").
You will find there:
- If scatter-gather or contiguous DMA are supported.
- How to setup DMA registers, one of them is usually called DMA CSR - "DMA command/status register".
- If the device supports complicated DMA layout (one or many ring buffers, chain of DMA descriptors etc.)
But the good thing is that many PCI devices support 0-size DMA.
Which does not do any memory access but just triggers a "DMA complete" interrupt. This can be a very convenient place to start for you.