I am a non CS/IT student, but having knowledge of C, Java, DS and Algorithms. Now-a-days I am focusing on operating system and had gained some of its concepts. But I want some practical knowledge of it. Merely writing algo code in java/c has no fun in doing. I have gone through many articles where they mentioned we can customize source code of Linux-kernel.
I want to start customizing the kernel as I move ahead in the learning of OS concepts and apply the same. It will make two goals achievable 1. I will gain practical idea of the operating system 2. I will have a project.
Problem which I face-
1. From where to get the source code? Which source code should I download? Also the documentation if possible.
https://www.kernel.org/
I went in there but there are so many of them which one will be better?
2. How will I customize the code once I have it?
Please give me suggestions with detail about how I should start this journey (of changing source code to customize Linux).
Moreover I am using Windows 8.
I recommend first reading several books on OSes and on programming. You need a broad CS culture (if possible get a CS degree)
I am a non CS/IT student,
You'll better become one, or else spend years of work to learn all the stuff a CS graduate student has learnt.
First, you need to be very familiar with Linux programming on user side (application programs). So read at least Advanced Linux Programming and study the source code of several programs, including shells (and some kind of servers). Read also carefully syscalls(2). Explore the state of your kernel (e.g. thru proc(5)...). Look into https://kernelnewbies.org/
I also recommend learning several programming languages. You should in particular read SICP, an excellent introduction to programming. Read also some book like programming language pragmatics. Read something about continuation and continuation passing style. Read the Dragon book. Read some Introduction to Algorithms. Read something about computer architecture and instruction set architecture
Merely writing algo code in java/c has no fun in doing.
But the kernel is also written in C (mostly) and full of algorithmic code. What makes you think you'll get more fun in it?
I want to start customizing the kernel as I move ahead in the learning of OS concepts and apply the same.
But why? Why don't you also consider studying and contributing to some user-level code
I would recommend first reading a good book on OSes in general, notably Operating Systems: Three Easy Pieces. Look also on OSdev.
At last, the general advice about kernel programming is don't. A common mistake is to try adding code inside the kernel to solve some issue that can and should be solved in user-land.
How will I customize the code once I have it?
You probably should not customize the kernel, but if you did you'll use familiar tools (a good source code editor like emacs or vim, a compiler and linker on the command line, a build automation tool like make). Patching the kernel is similar to patching some other free software. But testing your kernel is harder (because you'll often reboot).
You'll also find several books explaining the Linux kernel.
If you still want to customize the kernel you should first try to code some kernel module.
Moreover I am using Windows 8.
This is a huge mistake. You first need to be an advanced Linux user. So wipe out Windows from your computer, and install some Linux distribution -I recommend Debian- (and use only Linux, no more Windows). Become familiar with command line.
I seriously recommend to avoid working on the kernel as your first project.
I strongly recommend looking at some existing user-land free software project first (there are thousands of them, notably on github, e.g. choose some package in your distribution, study its source code, work on it, propose the patch to the community). Be able to build from source code a lot of things.
A wise man once said you "must act your way into right thinking, as you cannot think your way into right acting". In your case, you'll need to act as an experienced programmer would act, which means before we write any code, we need to answer some questions.
What do we want to change?
Why do we want to change it?
What are the repercussions of this change (ie what other functions - out of all the 10's of millions of lines of source code - call this function)?
After we've made the change, how are we going to compile it? In other words, there is a defined process for this. What is it?
After we compile our new kernel/module, how are we going to test it?
A good start, in addition to the answer that was just posted, would be to run LFS (Linux from Scratch). Get a successful install of that and use it as a starting point.
Now, since we're experienced programmers, we know that tinkering with a 10M+ line codebase is a recipe for trouble; we need a bit more direction than that. Here's a list of bugs that need to be fixed: https://bugzilla.kernel.org/buglist.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=7d
I, for one, would be glad to see the one called "AUFS hangs on fanotify" go away, as I use AUFS with Docker on a daily basis.
If, down the line, you decide you'd rather hack on something besides the kernel, there are plenty of other options.
From your question it follows that you've already gained some concepts of an operating system. However, if you feel that it's still insufficient, it is OK to spend more time on learning. An operating system (mainly, a kernel) has certain tasks to perform like memory management (or memory protection), multiprogramming, hardware abstraction and so on. Neither of the topics may be neglected - they are all as important. So, if you have some time, you may refer to such useful books as "Modern Operating Systems" by Andrew Tanenbaum. Special books like that will shed much light on all important aspects of a modern OS. Suffice it to say, Linux kernel itself was started by Linus Torvalds because of a strong inspiration by MINIX - an educational project by A. Tanenbaum.
Such a cumbersome project like an OS kernel (BSD, Linux, etc.) contains lots of code. Many people are collaborating to write or enhance whatever parts of the kernel. So, there is a common and inevitable need to use a version control system. So, if you have an intention to submit your code to the kernel in future, you also have to have hands on with version control. Particularly, Linux relies on Git SCM (software configuration management - a synonym for version control).
So, once you have some knowledge of Git, you can install it on your computer and download Linux source code: git clone https://github.com/torvalds/linux.git
Determine your goals at Linux kernel modification. What do you want to achieve? Perhaps, you have a network card which you suspect to miss some features in Linux? Take a look at the other vendors' drivers and make an attempt to fix the driver of interest to include the features. Of course, this will require some knowledge of the HW, and, if the features are HW dependent, you will unlikely succeed to elaborate your code without special knowledge. But, in general, - if you are trying to make an enhancement, it assumes that you are an experienced Linux user yourself. Otherwise, how will you understand that some fixes/enhancements/etc. are required? So, I can't help but agree with the proposal to postpone Windows 8 for a while and start using some Linux distribution (eg. Debian).
If you succeed to determine your goals (eg. if you find a paper describing some desired changes in Linux kernel or if you decide to enhance some device drivers / write your own), you will be able to try it hands on. However, you still might need some helpful books, but, in this case, some Linux-specific ones. Also, writing C code for the kernel itself will require one important detail - you will need to comply with a so called coding standard, otherwise Linux kernel maintainers will not be able to accept your patches.
So, I made an attempt to outline some tips based on your current question. Of course, the job of kernel development has far more broad prerequisites, but these are which are just obvious.
I'd like to understand & edit(adding printk, etc.) the linux kernel for a craneboard, which I obtained from github. I'm a newbie. I have gone through certain questions related to this in stackoverflow, but they all concentrate on linux in PC (not in embedded!). I don't know where to start. Any kind of guidance is accepted. Thanks in advance.
Dear did you got chance to look at README located at
https://github.com/craneboard/craneboard-kernel
if you need to add any information or some logging i.e. adding printk in kernel itslef, you need to recompile it, it also shows how to do it. But i am interested to know why you need to put logging information in kernel? Are you going to fix some bug in kernel?
Printk is same like a C Printf, but with few differences http://www.makelinux.net/books/lkd2/ch18lev1sec3
Also have a look at http://processors.wiki.ti.com/index.php/CraneBoard
It might also be a good start to look at https://linuxlink.timesys.com/files/pdf/Timesys-EmbeddedLinuxTutorial.pdf
You'll need an ARM toolchain before you can compile the kernel. Here is a good tutorial.
I'm a newbie on Linux programming and I don't know exactly where I can get information of functions for example. There is a directory Documentations in my sources. But I can't find semaphore stuff there for example. Is there a documentation for all the function somewhere in the sources? Or not obligatorily?
The Linux Kernel Newbies site is a good point to start.
And, of course, the kernel itself is the best reference. To get started. Also have a look at kernel.org/doc. For semaphores, look at this page here.
I am currently a student at a university studying a computing related degree and my current project is focusing on finding vulnerabilities in the Linux kernel. My aim is to both statically audit as well as 'fuzz' the kernel (targeting version 3.0) in an attempt to find a vulnerability.
My first question is 'simple' is fuzzing the Linux kernel possible? I have heard of people fuzzing plenty of protocols etc. but never much about kernel modules. I also understand that on a Linux system everything can be seen as a file and as such surely input to the kernel modules should be possible via that interface shouldn't it?
My second question is: which fuzzer would you suggest? As previously stated lots of fuzzers exist that fuzz protocols however I don't see many of these being useful when attacking a kernel module. Obviously there are frameworks such as the Peach fuzzer which allows you to 'create' your own fuzzer from the ground up and are supposedly excellent however I have tried repeatedly to install Peach to no avail and I'm finding it difficult to believe it is suitable given the difficulty I've already experienced just installing it (if anyone knows of any decent installation tutorials please let me know :P).
I would appreciate any information you are able to provide me with this problem. Given the breadth of the topic I have chosen, any idea of a direction is always greatly appreciated. Equally, I would like to ask people to refrain from telling me to start elsewhere. I do understand the size of the task at hand however I will still attempt it regardless (I'm a blue-sky thinker :P A.K.A stubborn as an Ox)
Cheers
A.Smith
I think a good starting point would be to extend Dave Jones's Linux kernel fuzzer, Trinity: http://codemonkey.org.uk/2010/12/15/system-call-fuzzing-continued/ and http://codemonkey.org.uk/2010/11/09/system-call-abuse/
Dave seems to find more bugs whenever he extends that a bit more. The basic idea is to look at the system calls you are fuzzing, and rather than passing in totally random junk, make your fuzzer choose random junk that will at least pass the basic sanity checks in the actual system call code. In other words, you use the kernel source to let your fuzzer get further into the system calls than totally random input would usually go.
"Fuzzing" the kernel is quite a broad way to describe your goals.
From a kernel point of view you can
try to fuzz the system calls
the character- and block-devices in /dev
Not sure what you want to achieve.
Fuzzing the system calls would mean checking out every Linux system call (http://linux.die.net/man/2/syscalls) and try if you can disturb regular work by odd parameter values.
Fuzzing character- or block-drivers would mean trying to send data via the /dev-interfaces in a way which would end up in odd result.
Also you have to differentiate between attempts by an unprivileged user and by root.
My suggestion is narrowing down your attempts to a subset of your proposition. It's just too damn broad.
Good luck -
Alex.
One way to fuzzing is via system call fuzzing.
Essentially the idea is to take the system call, fuzz the input over the entire range of possible values - whether it remain within the specification defined for the system call does not matter.
Looking for resources that can help getting 'into' the Linux code. Could not get much help on Google. I have no issues on the distro covered by the book/resource, but will like if Fedora is the base. Also, it would be great if the resource is well maintained and updated.
this looks promising:
http://kernelnewbies.org/KernelHacking
Note that you will need to be familiar with Operating Systems concepts to even understand the concepts of how memory is allocated, how processes are scheduled, and whatnot. Also, the code of the linux kernel is monstrously complex.
You are undertaking a daunting task. But have fun with it. You might want to start with a small device driver's code.
http://www.makelinux.net/kernel_map may be a good place to start. A clickable map which takes you to the part of the source you clicked. Readable in a browser.
If you are talking about the Linux kernel, kernel newbies is absolutely awesome. Besides that, I don't think theres a single resource to recommend.
If you are trying to understand the linux kernel source code, then the exact distribution you are using, is not very relevant.
I would recommend the book from Robert Love: Linux Kernel Development, currently in its third edition. It will give you an understanding of the main parts of the kernel.