I have been asked to integrate a custom JPEG encoder kernel module to the linux tree. The description is too generic. Can anyone suggest where in kernel tree should this go? I mean under what category in the drivers? I am assuming this is going to be compiled as a module and not statically linked to the kernel. If I generalize the question where should any custom kernel module live in the kernel tree? Assume the kernel module is a video/audio decoder/encoder. In this case it is a JPEG encoder as I said.
Any help will be highly appreciated.
Thanks.
When I posted this question I did not have clarity as how drivers are categorized and placed in the kernel tree. So explored and this is what I found so far:
If I am integrating/writing a new driver e.g. Ring Oscillator (this device simply generates some frequencies given a input period value, the frequency number is fed to a random number generator). To my understanding this should go under linux/drivers/misc/ whereas someone argued this should go under linux/drivers/misc/. But apart from that there seems to no strict rule where this kind of drivers should go. So it is quite up to your discretion and judgment where you ultimately place it. I have given the details of the steps involved here.
I also had to integrate a jpeg encoder and I was confused where this driver should go. I initially thought I will place it under linux/drivers/media/ as suggested in the comments. But this turned out to be a matter of preference. Finally I integrated it as a new buildroot package. In case you are interested I have described it here.
This is my understanding so far. If anyone thinks if I have missed anything please kindly point out.
Related
I wanted to see how the conv1d module is implemented
https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#Conv1d. So I looked at functional.py but still couldn’t find the looping and cross-correlation computation.
Then I searched Github by keyword ‘conv1d’, checked conv.cpp https://github.com/pytorch/pytorch/blob/eb5d28ecefb9d78d4fff5fac099e70e5eb3fbe2e/torch/csrc/api/src/nn/modules/conv.cpp 1 but still couldn’t locate where the computation is happening.
My question is two-fold.
Where is the source code that "conv1d” is implemented?
In general, if I want to check how the modules are implemented, where is the best place to find? Any pointer to the documentation will be appreciated. Thank you.
It depends on the backend (GPU, CPU, distributed etc) but in the most interesting case of GPU it's pulled from cuDNN which is released in binary format and thus you can't inspect its source code. It's a similar story for CPU MKLDNN. I am not aware of any place where PyTorch would "handroll" it's own convolution kernels, but I may be wrong. EDIT: indeed, I was wrong as pointed out in an answer below.
It's difficult without knowing how PyTorch is structured. A lot of code is actually being autogenerated based on various markup files, as explained here. Figuring this out requires a lot of jumping around. For instance, the conv.cpp file you're linking uses torch::conv1d, which is defined here and uses at::convolution which in turn uses at::_convolution, which dispatches to multiple variants, for instance at::cudnn_convolution. at::cudnn_convolution is, I believe, created here via a markup file and just plugs in directly to cuDNN implementation (though I cannot pinpoint the exact point in code when that happens).
Below is an answer that I got from pytorch discussion board:
I believe the “handroll”-ed convolution is defined here: https://github.com/pytorch/pytorch/blob/master/aten/src/THNN/generic/SpatialConvolutionMM.c 3
The NN module implementations are here: https://github.com/pytorch/pytorch/tree/master/aten/src
The GPU version is in THCUNN and the CPU version in THNN
My task is to create a program to open and close an electronic valve that is plugged into GPIO ports on my BeagleBone, by using TTL signals.
Questions:
Can I do this?
How do I make an executable file to do this?
Can someone refer me to documentation on this?
Am I going about this in the wrong way?
Thank you.
P.S. If you couldn't already tell I am very new to this.
Yes
There are many ways. It's actually a pretty standard Linux computer and you can use any of a "million" different programming languages to achieve this. This also means you don't have to look for "Beaglebone" specific instructions (beyond the GPIO info below), but your problem is just "How do I write a program that can write text to a file on Linux?". Bonus: This sounds easy and it is easy!
Yes, take a look here for the hardware specific part:
https://github.com/adafruit/adafruit-beaglebone-io-python/issues/157
It describes fairly well both the new and the old sysfs interfaces you can use to manipulate GPIOs.
Depending on the language of your choice, there may already be bindings or a library to abstract this.
No (only based on the information you provided, are there other requirements?)
We all were new at this at some point, don't worry.
Sidenote: It's generally a good idea to make sure that you are running the latest firmware. In case of the BB-Family you can find them here: http://beagleboard.org/latest-images
I am currently a student at a university studying a computing related degree and my current project is focusing on finding vulnerabilities in the Linux kernel. My aim is to both statically audit as well as 'fuzz' the kernel (targeting version 3.0) in an attempt to find a vulnerability.
My first question is 'simple' is fuzzing the Linux kernel possible? I have heard of people fuzzing plenty of protocols etc. but never much about kernel modules. I also understand that on a Linux system everything can be seen as a file and as such surely input to the kernel modules should be possible via that interface shouldn't it?
My second question is: which fuzzer would you suggest? As previously stated lots of fuzzers exist that fuzz protocols however I don't see many of these being useful when attacking a kernel module. Obviously there are frameworks such as the Peach fuzzer which allows you to 'create' your own fuzzer from the ground up and are supposedly excellent however I have tried repeatedly to install Peach to no avail and I'm finding it difficult to believe it is suitable given the difficulty I've already experienced just installing it (if anyone knows of any decent installation tutorials please let me know :P).
I would appreciate any information you are able to provide me with this problem. Given the breadth of the topic I have chosen, any idea of a direction is always greatly appreciated. Equally, I would like to ask people to refrain from telling me to start elsewhere. I do understand the size of the task at hand however I will still attempt it regardless (I'm a blue-sky thinker :P A.K.A stubborn as an Ox)
Cheers
A.Smith
I think a good starting point would be to extend Dave Jones's Linux kernel fuzzer, Trinity: http://codemonkey.org.uk/2010/12/15/system-call-fuzzing-continued/ and http://codemonkey.org.uk/2010/11/09/system-call-abuse/
Dave seems to find more bugs whenever he extends that a bit more. The basic idea is to look at the system calls you are fuzzing, and rather than passing in totally random junk, make your fuzzer choose random junk that will at least pass the basic sanity checks in the actual system call code. In other words, you use the kernel source to let your fuzzer get further into the system calls than totally random input would usually go.
"Fuzzing" the kernel is quite a broad way to describe your goals.
From a kernel point of view you can
try to fuzz the system calls
the character- and block-devices in /dev
Not sure what you want to achieve.
Fuzzing the system calls would mean checking out every Linux system call (http://linux.die.net/man/2/syscalls) and try if you can disturb regular work by odd parameter values.
Fuzzing character- or block-drivers would mean trying to send data via the /dev-interfaces in a way which would end up in odd result.
Also you have to differentiate between attempts by an unprivileged user and by root.
My suggestion is narrowing down your attempts to a subset of your proposition. It's just too damn broad.
Good luck -
Alex.
One way to fuzzing is via system call fuzzing.
Essentially the idea is to take the system call, fuzz the input over the entire range of possible values - whether it remain within the specification defined for the system call does not matter.
I have to add a system call in linux kernel that will print the process tree showing only the PIDs to user code. I have to use copy_to_user here. But I am not understanding the use of this function. Could any of u give an example of how it works, including the user-side code and added system code?.....Any easy/simple example would be great for me...:)
Thanks.
I suggest you read through the Linux Device Driver book. It's freely available online at http://lwn.net/Kernel/LDD3/. Although it's geared towards device drivers, it covers most of the key aspects for communicating between kernel and user space and includes multiple examples.
By the way, this sounds like a homework question. If so, your question should have the 'homework' Tag associated with it.
I am in the process of tackling the Linux Kernel learning curve and trying to get my head round the information stored in nested struct specifically to resolve a ALSA driver issue.
Hence, I am spending a lot of my time in the source code tracing through structures that have pointers to other structures that in turn have pointers to yet other structures...by which time my head has become so full that I start to loose track of the big picture!
Can anybody point me at either a tool or a website (along the lines of the highly usful Linux Cross Reference http://lxr.linux.no/) that will allow me to, ideally graphically, expand down through the nested struct of the source code?
At the moment we are developing for an Embedded PowerPC in Eclipse CDT version 4.0 but wouldn't be opposed to switching tool chains.
Regards
KermitG
This may sound old fashion but I've found that tracing through data structures with a pencil and paper helps you reverse engineer the code better than tools that automagically do this. So, my recommendation is that you draw them yourself so that you don't have to keep it all in your head. Once you've done this your learning curve becomes a lot less steep.
Just a copy/paste of my comment, so that this question has at least 1 answer.
Or alternatively you could use something like Doxygen to generate the diagrams for you. It's worth noting a lot of the DocBook books get their structures directly from annotated code.
I am currently using Kdevelop4 (svn version) to walk through the Linux kernel. The navigation capabilities are great, but it takes a big while to parse it (just give it the directories you need, omitting all drivers you are not interested in for example) and is still a little bit crashy.
Once the stability improves and the parser can cache previously parsed data, I think this will become the most convenient way to walk through the kernel.