When I use top the iowait on the host is really high.
iostat tells me which disk is utilized more but I want to find out which process is the culprit?
I am trying to find this out on a red hat linux host. Any suggestions.
EDIT: My linux flavor does not either have atop or ntop and since building kernel is not an option for me don't ask me why :) (since this is not my personal box). are there any other alternatives
I usually use atop. There's a really good article at Debian Package A Day about it. It does require kernel patching (although Ubuntu already has the patch applied, I'm not sure about any other distributions.)
Use iotop.
Or you can get it standalone, it's a simple python script which requires a recent kernel (can't remember, but at least > 2.6.20)
Related
I am looking if there is any API through which we can get OS distribution name and version from a Linux kernel module?
For example, I am using SLES 12 service pack 4. This information is present in /etc/os-release file. I want to know if there is any way to get this information from kernel code.
linux:/ # cat /etc/os-release
NAME="SLES"
VERSION="12-SP4"
VERSION_ID="12.4"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP4"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp4"
linux:/ #
There's no kernel API for detecting the current OS distribution, simply because it's not really needed. The Linux kernel itself is distribution-agnostic, and it couldn't care less which distribution is being run on top of it (having the kernel depend on what's being run on top of it wouldn't make much sense).
If you really want, you can open, read and parse the file yourself from kernel space. See more in this other post for an example, and in particular this answer for modern kernels. In any case, remember that filesystem interaction from kernel space is generally discouraged, and could easily lead to bugs and compromise the security of the kernel if done wrong, so be careful.
If you are developing a kernel module, I would suggest you to parse the /etc/os-release file from userspace when compiling/installing the module and use a set of #defines, or even module parameters. In any case, you should ask yourself why you need this information in kernel code in the first place, as you really shouldn't.
The question is how to execute aout-format binary (I mean old format which for example used on FreeBSD before it has migrated to ELF) on Linux system. Is there a possibility to do so without extra coding (is there some existing solution)? Probably it should be in form of kernel module or patch for the Linux kernel. Another solution could be user-space launcher (may be even run-time linker). I have searched for something similar but was unable to found something. I have not yet checked difference in system calls interfaces, if you have some comments about that, you are welcome to provide them.
P.S. I know that writing user-space launcher for aout static binary is quite trivial but the question is about some existing solution.
Check for CONFIG_BINFMT_AOUT in your kernel config.
If your kernel has /proc/config.gz:
zgrep CONFIG_BINFMT_AOUT /proc/config.gz
On Ubuntu and the like:
grep CONFIG_BINFMT_AOUT /boot/config-$(uname -r)
Kernel option was CONFIG_BINFMT_AOUT, not sure if it's still around or necessary.
I recently read a post (admittedly its a few years old) and it was advice for fast number-crunching program:
"Use something like Gentoo Linux with 64 bit processors as you can compile it natively as you install. This will allow you to get the maximum punch out of the machine as you can strip the kernel right down to only what you need."
can anyone elaborate on what they mean by stripping down the kernel? Also, as this post was about 6 years old, which current version of Linux would be best for this (to aid my google searches)?
There is some truth in the statement, as well as something somewhat nonsensical.
You do not spend resources on processes you are not running. So as a first instance I would try minimise the number of processes running. For that we quite enjoy Ubuntu server iso images at work -- if you install from those, log in and run ps or pstree you see a thing of beauty: six or seven processes. Nothing more. That is good.
That the kernel is big (in terms of source size or installation) does not matter per se. Many of this size stems from drivers you may not be using anyway. And the same rule applies again: what you do not run does not compete for resources.
So think about a headless server, stripped down -- rather than your average desktop installation with more than a screenful of processes trying to make the life of a desktop user easier.
You can create a custom linux kernel for any distribution.
Start by going to kernel.org and downloading the latest source. Then choose your configuration interface (you have the choice of console text, 'config', ncurses style 'menuconfig', KDE style 'xconfig' and GNOME style 'gconfig' these days) and execute ./make whateverconfig. After choosing all the options, type make to create your kernel. Then make modules to compile all the selected modules for this kernel. Then, make install will copy the files to your /boot directory, and make modules_install, copies the modules. Next, go to /boot and use mkinitrd to create the ram disk needed to boot properly, if needed. Then you'll add the kernel to your GRUB menu.lst, by editing menu.lst and copying the latest entry and adding a similar one pointing to the new kernel version.
Of course, that's a basic overview and you should probably search for 'linux kernel compile' to find more detailed info. Selecting the necessary kernel modules and options takes a bit of experience - if you choose the wrong options, the kernel might not be bootable and you'll have to start over, which is a pain because selecting the options and compiling the kernel can take 15-30 minutes.
Ultimately, it isn't going to make a large difference to compile a stripped-down custom kernel unless your given task is very, very performance sensitive. It makes sense to remove things you're never going to use from the kernel, though, like say ISDN support.
I'd have to say this question is more suited to SuperUser.com, by the way, as it's not quite about programming.
What scheduling algorithms does Linux kernel use?
Where can I get more info about linux's kernel? (OS first course... student level)
The linux kernel has several different available scheduling algorithms both for the process scheduling and for I/O scheduling. Download it from www.kernel.org and call
make menuconfig
You will get a full list of all available options with a built-in help.
One guy that once came up with his O(1) scheduler is Con Kolivas. Definitively have to have a look at what he did. I was once a great break-through.
Note: As Abdullah Shahin noted, this answer is about IO queing scheduler, not for processes.
If you just want to check what scheduler your linux system is using and which are available you can run the following command:
cat /sys/block/sda/queue/scheduler
The one between the [] is the one it's using at the moment. The other ones are available.
To change it:
sudo bash -c 'echo deadline > /sys/block/sda/queue/scheduler'
Be carefull to set it back to default though, unless you know what you are doing and want.
Default (in newer Ubuntu distros at least) is CFQ (Completely Fair Scheduling):
http://en.wikipedia.org/wiki/CFQ
Interview with the creator (Jens Axboe):
http://kerneltrap.org/node/7637
As others have already mentioned, there are several scheduling algorithms available, according to the intended use.
Check this article if you want to learn more about scheduling in Linux.
i believe "completely fair scheduler" is in use with latest kernels. I think you can good amount of information if you just search for it in google.
link : http://en.wikipedia.org/wiki/Completely_Fair_Scheduler
A new addition to Linux Kernel is EDF (Earliest Deadline First) for guaranteed RealTime support
http://lkml.org/lkml/2009/9/22/186
http://www.evidence.eu.com/content/view/313/390/
I think the Linux kernel actually has a few different schedulers you can choose from at compile-time. To find out more about the Linux kernel, you can download the kernel source code (or browse it online) and look in the Documentation directory. For example, the scheduler subdirectory might be helpful. You can also just look at the code itself, obviously.
Modern GNU/Linux distributions use CFS (Completely Fair Scheduler). You may read more on that in the 4th chapter of this book:
Linux Kernel Development 3rd Edition by Robert Love
You will find many interesting and easy to understand explanations there. I enjoyed a lot.
Linux Kernel allows three different scheduling algorithms mainly
shortest job first
Round Robin Scheduling
Priority based preemptive scheduling algorithm.
The third scheduling method which it differs with lower version of Linux versions such as 2.4
Can anyone point me to a good tutorial on creating a bootable Linux CD from scratch?
I need help with a fairly specialized problem: my firm sells an expansion card that requires custom firmware. Currently we use an extremely old live CD image of RH7.2 that we update with current firmware. Manufacturing puts the cards in a machine, boots off the CD, the CD writes the firmware, they power off and pull the cards. Because of this cycle, it's essential that the CD boot and shut down as quickly as possible.
The problem is that with the next generation of cards, I have to update the CD to a 2.6 kernel. It's easy enough to acquire a pre-existing live CD - but those all are designed for showing off Linux on the desktop - which means they take forever to boot.
Can anyone fix me up with a current How-To?
Update:
So, just as a final update for anyone reading this later - the tool I ended up using was "livecd-creator".
My reason for choosing this tool was that it is available for RedHat-based distributions like CentOs, Fedora and RHEL - which are all distributions that my company supports already. In addition, while the project is very poorly documented it is extremely customizable. I was able to create a minimal LiveCD and edit the boot sequence so that it booted directly into the firmware updater instead of a bash shell.
The whole job would have only taken an hour or two if there had been a README explaining the configuration file!
There are a couple of interesting projects you could look into.
But first: does it have to be a CD-ROM? That's probably the slowest possible storage (well, apart from tape, maybe) you could use. What about a fast USB stick or a an IEE1394 hard-disk or maybe even an eSATA hard-disk?
Okay, there are several Live-CDs that are designed to be very small, in order to e.g. fit on a business card sized CD. Some were also designed to be booted from a USB stick, back when that meant 64-128 MiByte: Damn Small Linux is one of the best known ones, however it uses a 2.4 kernel. There is a sister project called Damn Small Linux - Not, which has a 2.6 kernel (although it seems it hasn't been updated in years).
Another project worth noting is grml, a Live-CD for system administration tasks. It does not boot into a graphic environment, and is therefore quite fast; however, it still contains about 2 GiByte of software compressed onto a CD-ROM. But it also has a smaller flavor, aptly named grml-small, which only contains about 200 MiByte of software compressed into 60 MiByte.
Then there is Morphix, which is a Live-CD builder toolkit based on Knoppix. ("Morphable Knoppix"!) Morphix is basically a tool to build your own special purpose Live-CD.
The last thing I want to mention is MachBoot. MachBoot is a super-fast Live-CD. It uses various techniques to massively speed up the boot process. I believe they even trace the order in which blocks are accessed during booting and then remaster the ISO so that those blocks are laid out contiguously on the medium. Their current record is less than 6 seconds to boot into a full graphical desktop environment. However, this also seems to be stale.
One key piece of advice I can give is that most LiveCDs use a compressed filesystem called squashfs to cram as much data on the CD as possible. Since you don't need compression, you could run the mksquashfs step (present in most tutorials) with -noDataCompression and -noFragmentCompression to save on decompression time. You may even be able to drop the squashfs approach entirely, but this would require some restructuring. This may actually be slower depending on your CD-ROM read speed vs. CPU speed, but it's worth looking into.
This Ubuntu tutorial was effective enough for me to build a LiveCD based on 8.04. It may be useful for getting the feel of how a LiveCD is composed, but I would probably not recommend using an Ubuntu LiveCD.
If at all possible, find a minimal LiveCD and build up with only minimal stripping out, rather than stripping down a huge LiveCD like Ubuntu. There are some situations in which the smaller distros are using smaller/faster alternatives rather than just leaving something out. If you want to get seriously hardcore, you could look at Linux From Scratch, and include only what you want, but that's probably more time than you want to spend.
Creating Your Own Custom Ubuntu 7.10 Or Linux Mint 4.0 Live-CD With Remastersys
Depends on your distro. Here's a good article you can check out from LWN.net
There is a book I used which covers a lot of distros, though it does not cover creating a flash-bootable image. The book is Live Linux(R) CDs: Building and Customizing Bootables. You can use it with supplemental information from your distro of choice.
So, just as a final update for anyone reading this later - the tool I ended up using was "livecd-creator".
My reason for choosing this tool was that it is available for RedHat-based distributions like CentOs, Fedora and RHEL - which are all distributions that my company supports already. In addition, while the project is very poorly documented it is extremely customizable. I was able to create a minimal LiveCD and edit the boot sequence so that it booted directly into the firmware updater instead of a bash shell.
The whole job would have only taken an hour or two if there had been a README explaining the configuration file!
Debian Live provides the best tools for building a Linux Live CD. Webconverger uses Debian Live for example.
It's very easy to use.
sudo apt-get install live-helper # from Debian unstable, which should work fine from Ubuntu
lh_config # edit config/* to your liking
sudo lh_build