I am trying to write a network device driver for Linux. The device that I have has an API available that allows me to access all of the features I need through a shared object that exists in userspace.
I want to write a network driver such that I can make the device show up as a CAN interface. However, in order to interact with the device I need to use a specific shared object that exists in userspace.
The reason that I need a network device driver is to expose a CAN Interface that can be interacted with via the SocketCAN utilities.
Is there a way that I can write a network device driver in userspace? Or what would the best way for me to architect a solution?
Tl;Dr
Need to write a device driver for a device which can only be interacted with from userspace via a supplied shared object which exposes the API. I need the device to show up as a network interface in order to utilize the SocketCAN utilities and other applications that communicate with CAN interfaces in Linux.
What are my options here? What can I do?
Thanks!
So you are saying that there is no driver for your network device in kernel at all, and it can be only accessed via some user-space library? In that case shared library you mentioned should be communicating with your network device by memory mapping your /dev/mem file, in order to be able to read/write to hardware registers. Or perhaps by using some UIO.
So your driver should be also developed in user-space then... Then the actual question you should ask is how to use kernel CAN API from user-space? And is it possible at all in the first place? For answers I guess you should look at Documentation/networking/can.txt. And if the answer is "no" (means you can't expose CAN interface from user-space), then you should develop also some kernel driver which would interact with your user-space part, exposing CAN interface.
In ideal world the whole driver architecture would look like this:
But you need to use some (proprietary, if I understand correctly) shared library API to interact with your device. So I propose you to use next driver architecture, which depicted on the image below:
blue color stands for parts that need to be developed
magenta is for already existing code
In a nutshell, your app and driver both make a shim between SocketCAN API and shared library API.
So you need to develop 2 components:
Driver (on kernel side). It's in charge of:
talking to SocketCAN utilities
talking to your user-space application
Application (in user-space); it's probably should be a daemon, as it's gonna be running constantly. It's in charge of:
talking to shared library
talking to your driver
The last question remains is which kernel API to use to interact between your kernel space driver and user-space application (marked as IPC on picture). It strictly depends on which kind of data you are going to send between two, and how much of data you will want to send, and which way of sending is most appropriate for your task. It may also depend on your shared library API: you probably don't want to spend much of CPU time to convert messages format (as you already have triple context switching with this driver architecture, which is not really nice for performance). So it's probably should be something packet-oriented, like Netlink.
Next reading can be useful to figure out which IPC to use:
Kernel Space - User Space Interfaces
Linux kernel interfaces
Related
I tried to generate code for a device changing the implementation type to SharedLibrary and renaming the Entry Point to Device_Name.so. I was able to generate and build, but in main.cpp it kept a main function not a make_component to be called by ComponentHost. the device constructors deals with arguments that ComponentHost doesn't, like the Device Manager IOR. I believe this functionality extension implies changing the source code of ComponenHost. Is it part of REDHAWK roadmap? any comments on how can I make it work?
So are you trying to use the shared process space within a node to communicate between devices and services? Because I don't believe that there is tooling specifically for this yet, but I think there is a way to do this. Just to be clear I haven't tried this, but based on the test used by the bulkio ports to determine local vs remote transport usage, I think this will work.
If you look at the persona pattern, you'll see that there is a Programmable Device which is responsible for loading Persona Devices. Most of the details for this aren't necessary for what you're trying to do, but the pattern should be helpful. To accomplish communication between Devices using shared memory, you could generate a Programmable device whose sole purpose is to forward parameters from the DeviceManager to the Personas. The Personas would then act as your normal Devices normally do, just launched in the same process space as one another.
The code generators for the Programmable and Persona Devices are not yet integrated into the IDE, so you'll have to create a new Device project in eclipse for each Device you want (so that you'll have the spd files). Be sure to add the appropriate AggregateDevice interface to your Devices. This let's the framework know that multiple devices can technically be considered one entity, but you can also individually communicate with each. Also make sure that the Programmable is an Executable Device, since it needs to launch the Persona Devices. Then, from the command line, you can run redhawk-codegen - - pgdevice </path/to/programmable/spd> to generate a Programmable Device, and redhawk-codegen - - persona </path/to/persona/spd> to generate your Persona Device(s).
Once all of this is done, you'll notice the main function for your Programmable launches the Device like you described in your question. However, the main function for the Personas has code to launch the Device as either a standalone Device or as simply an object in its own thread.
This should allow the bulkio ports of the Programmable and Personas to communicate with each other via shared memory. Obviously this will break down if you attempt to push data out of the process, at least until someone adds interprocess shared memory via something like shm. Not sure if that's on the road map, but it would certainly be neat.
Update: It appears that interprocess shared memory was added in RH 2.1.2, so you should be able to communicate between collocated Devices, Services, and Components using that mechanism. This renders the above unnecessary, but I'm going to leave it for earlier versions of RH.
Let me know if you have any questions!
As of RH 2.1.2, the default behavior for Devices/Services/Components whose user code uses redhawk::buffer for the data memory allocator and the stream API for interaction with the bulkio port is to use a shared memory transport between C++ entities that are running in different processes
I need your advice.
I want to achieve transactions between my pc and fpga by pcie slot.
I try to install a driver to do that but the driver don't see the fpga.
I run on terminal $ lspci and I take this picture:
enter image description here
Is there any way to connect xdma module with my fpga manually...??
What you see in the "Used By" column of lsmod does not mean whether some user-space program is using the module, but only if it is being used by other kernel modules. For example if you check the usb_core line from the lsmod output, you will see that it is being used by a bunch of USB/HID related modules.
The kernel module (or device deriver) is only a way to enable your system to comunicate with the particular piece of hardware. It only exposes its interface so the rest of the kernel (or userspace programs) can interact with it.
You will need for sure some user level software or application that makes use of the interface exposed by the kernel module.
I don't know for your particular case, but the interface that the driver exposes depends on the type of the hardware and the module implementation itself. It may expose some files to /sysfs, or add a new syscall (this is very unlikely, but it's a possibility), or make special usage of ioctl.
So, you should check for the module documentation, or look for some user-space program that knows how to interface with the driver.
I know that user applications can run only in user mode, which is for system security. On the contrary most drivers run in kernel mode, to access I/O devices. Nevertheless some driver run in user mode, but are allowed to access I/O devices. So I have the following question. What is the main difference between drivers and user applications? Can't user application be allowed to access I/O devices like some drivers do?
Thanks.
First of all, some preview from this link :-
Applications run in user mode, and core operating system components
run in kernel mode. Many drivers run in kernel mode, but some drivers
run in user mode.
When you start a user-mode application, Windows(/any OS) creates a process for
the application. The process provides the application with a private
virtual address space and a private handle table. Because an
application's virtual address space is private, one application cannot
alter data that belongs to another application.
In addition to being private, the virtual address space of a user-mode
application is limited. A processor running in user mode cannot access
virtual addresses that are reserved for the operating system. Limiting
the virtual address space of a user-mode application prevents the
application from altering, and possibly damaging, critical operating
system data.
All code that runs in kernel mode shares a single virtual address
space. This means that a kernel-mode driver is not isolated from other
drivers and the operating system itself. If a kernel-mode driver
accidentally writes to the wrong virtual address, data that belongs to
the operating system or another driver could be compromised.
Also, from this link
Software drivers
Some drivers are not associated with any hardware device at all. For
example, suppose you need to write a tool that has access to core
operating system data structures, which can be accessed only by code
running in kernel mode. You can do that by splitting the tool into two
components. The first component runs in user mode and presents the
user interface. The second component runs in kernel mode and has
access to the core operating system data. The component that runs in
user mode is called an application, and the component that runs in
kernel mode is called a software driver. A software driver is not
associated with a hardware device.
Also, software drivers() always run in kernel mode. The main reason
for writing a software driver is to gain access to protected data that
is available only in kernel mode. But device drivers do not always
need access to kernel-mode data and resources. So some device drivers
run in user mode.
What is the main difference between drivers and user applications?
The difference is same as that between a sub-marine and a ship. Drivers are hardware-dependent and operating-system-specific. They usually provide the interrupt handling required for any necessary asynchronous time-dependent hardware interface. Hence, almost all of them run in kernel mode. Whereas, as specified in the second paragraph, to prevent applications damaging critical OS data, the user applications are bound to run in user space.
Also, not all drivers communicate directly with a device. For a given I/O request (like reading data from a device), there are often several drivers, layered in a stack, that participate in the request. The one driver in the stack that communicates directly with the device is called the function driver; the drivers that perform auxiliary processing are called filter drivers.
Can't user application be allowed to access I/O devices like some
drivers do?
The application calls a function implemented by the operating system, and the operating system calls a function implemented by the driver. The driver knows how to communicate with the device hardware to get the data. After the driver gets the data from the device, it returns the data to the operating system, which returns it to the application.
Applications connect to IO devices through APIs/interfaces presented by device drivers(rather the OS). The OS handles most hardware/software interaction. Hardware vendors write "plugins/modules/drivers" which allows the OS to control their specific hardware. So using the interfaces provided by the OS, you can write your Application to access IO devices.
So, you can't have a user application directly access the hardware without the help of drivers because it's all drivers below the hierarchy to access the device as device drivers are written in low-level languages which can communicate with hardware, whereas user-applications are written in high-level languages.
Also, check this answer to get more idea about drivers address space in various OS'.
Your question lacks any concrete operating system tags, so I'll answer generically about the subject
Yes, drivers can be very similar to a user application in a microkernel-based operating system. In a microkernel, usually a driver is just a userspace app that queries the kernel for some generic capabilities and then functions normally in userspace
Take for example a hypothetical implementation of a pagein/pageout driver in such an OS.
When a page fault happens in user mode, the transition is made to kernel as usual with synchronous exceptions, but a microkernel would queue the pagein request to a userspace driver via local message queue.
After that (still in the exception handler) the faulting process is made blocking and dequeued.
Then the userspace driver app does the necessary page-in and notifies the microkernel on the work done again via local message queue. The kernel is then ready to reschedule the faulting process as it will have its memory mappings ready.
Such functioning incurs a performance overhead but it has its advantages, mainly that the drivers can be implemented and tested just like any other user app.
In a monolithic kernel such disposition is impossible and that case is better described by other answers.
My main question is there piece of code running in X-Server process memory (Excluded drivers - which we all know can be written in different manners) is directly accessing memory in GPU card?
Or it employs drivers and drm, or any other interface for communication with GPU and queuing draw/render/clear/... commands?
I know question seems lame, but I am interested in specifics?
EDIT:
More specifically: to my understanding kernel communicates with hardware with assistance from drivers, and exposes API to the rest (if I am wrong please correct me).
In this context can X-Server circumvent DMA-API (I am only guessing DMA IO is responsible for communication with periferials) located in kernel to communicate and exchange data with GPU card (in a direct way - without anyones assistance == without kernel, drivers, ...)?
And what would be bare minimum requirement for X-Server to communicate with GPU. I am aiming to understand how this communication is done on low level.
It is entirely possible that on Linux a given X server accesses part of the video card memory directly as a framebuffer. It's not the most efficient way of displaying things, but it works.
I am currently writing a CUDA application and am running into a few IO issues "feeding the beast."
I am wondering if there is any way that I can directly read data from a RAID controller or NIC and have that data sent directly to the GPU. What I'm trying to accomplish is shown directly on slide #3 of the following presentation: http://developer.download.nvidia.com/devzone/devcenter/cuda/docs/GPUDirect_Technology_Overview.pdf.
That being said, apparently this has been answered already here: Is it possible to access hard disk directly from gpu?, however the presentation that I've attached leads to believe all I need is to set an environment variable in Linux (but it doesn't offer any useful code snippets/examples).
Therefore, I'm wondering if it is possible to read data directly from a NIC/RAID controller into the GPU and what would be required to do so? Would I need to write my own driver for the hardware? Are there any examples where certain copies are avoided?
Thanks in advance for the help.
GPUDirect is a technology "umbrella term", which in general is a brand referring to technologies that enable direct data transfer to and/or from a GPU, somehow bypassing unnecessary trips through host memory.
GPUDirect v1 is a technology that works with specific infiniband adapters, and enables the sharing of a data buffer between the GPU driver and the IB driver. This technology has mostly been superceded by GPUDirect (v3) RDMA. This v1 technology does not enable general usage with any NIC. The environment variable reference:
however the presentation that I've attached leads to believe all I need is to set an environment variable in Linux
refers to enabling GPUDirect v1. It is not a general purpose NIC enabler.
GPUDirect v2 is also called GPUDirect Peer-to-Peer, and it is for transfer of data between two CUDA GPUs on the same PCIE fabric only. It does not enable interoperability with any other kind of device.
GPUDirect v3 is also called GPUDirect RDMA.
Therefore, I'm wondering if it is possible to read data directly from a NIC/RAID controller into the GPU and what would be required to do so?
Today, the canonical use case for GPUDirect RDMA is with a Mellanox Infiniband (IB) adapter. (It can also be made to work, perhaps with assistance from Mellanox, using a Mellanox Ethernet Adapter and RoCE). If this fits your definition of "NIC", then it's possible by loading a proper software stack, assuming you have appropriate hardware. The GPU and the IB device need to be on the same PCIE fabric, which means they need to be attached to the same PCIE root complex (effectively, connected to the same CPU socket). When used with a Mellanox IB adapter, typical usage would involve a GPUDirect RDMA-aware MPI.
If you have your own unspecified NIC or RAID controller, and you don't already have a GPUDirect RDMA linux device driver for it, then it's not possible to use GPUDirect. (If there is a GPUDirect RDMA driver for it, contact the manufacturer or driver provider for assistance.) If you have access to the driver source code, and are familiar with writing your own linux device drivers, you could try crafting your own GPUDirect driver. The steps involved are beyond the scope of my answer, but the starting point is documented here.
Would I need to write my own driver for the hardware?
Yes, if you don't already have a GPUDirect RDMA driver for it, one would need to be written.
Are there any examples where certain copies are avoided?
The GPUDirect RDMA MPI link gives examples and explains how GPUDirect RDMA can avoid unnecessary device<->host data copies during the transfer of data from GPU to IB adapter. In general, data can be transferred directly (over PCIE) from memory on the GPU device to memory on the IB device (or vice-versa) with no trip through host memory (GPUDirect v1 did not achieve this.)
UPDATE: NVIDIA has recently announced a new GPU Direct technology called GPU Direct Storage.