I have a user-space program (Capstone). I would like to use it in FreeBSD kernel. I believe most of them have the same function name, semantics and arguments (in FreeBSD, the kernel printf is also named printf). First I built it as libcapstone.a library, and link my program with it. Since the include files are different between Linux user-space and FreeBSD kernel, the library cannot find symbols like sprintf, memset and vsnprintf. How could I make these symbols (from FreeBSD kernel) visible to libcapstone.a?
If directly including header files like <sys/systm.h> in Linux user-space source code, the errors would be like undefined type u_int or u_char, even if I add -D_BSD_SOURCE to CFLAGS.
Additionally, any other better ways to do this?
You also need ; take a look at kernel man pages, eg "man 9 printf". They list required includes at the top.
Note, however, that you're trying to do something really hard. Some basic functions (eg printf) might be there; others differ completely (eg malloc(9)), and most POSIX APIs are simply not there. You won't be able to use open(2), socket(2), or fork(2).
Related
I know that kernel modules are used to write device drivers. You can add new system calls to the Linux kernel and use it to communicate with other devices.
I also read that ioctl is a system call used in linux to implement system calls which are not available in the kernel by default.
My question is, why wouldn't you just write a new kernel module for your device instead of using ioctl? why would ioctl b useful where kernel modules exist?
You will need to write a kernel driver in either case, but you can choose between adding a new syscall and adding a ioctl.
Let's say you want to add a feature to get the tuner settings for a video capturing device.
If you implement it as a syscall:
You can't just load a module, you need to change the kernel itself
Hundreds of drivers could each add dozens of syscalls each, kludging up the table with thousands of global functions that must be kept forever.
For the driver to have any reach, you will need to convince kernel maintainers that this burden is worthwhile.
You will need to upstream the definition into glibc, and people must upgrade before they can write programs for it
If you implement it as an ioctl:
You can build your module for an existing kernel and let users load it, without having to get kernel maintainers involved
All functions are simple per-driver constants in the applicable header file, where they can easily be added or removed
Everyone can start programming with it just by including the header
Since an ioctl is much easier, more flexible, and exactly meant for all these driver specific function calls, this is generally the preferred method.
I also read that ioctl is a system call used in linux to implement system calls which are not available in the kernel by default.
This is incorrect.
System calls are (for Linux) listed in syscalls(2) (there are hundreds of them between user space and kernel land) and ioctl(2) is one of them. Read also wikipage on ioctl and on Unix philosophy and Linux Assembler HowTo
In practice, ioctl is mostly used on device files, and used for things which are not a read(2) or a write(2) of bytes.
For example, a sound is made by writing bytes to /dev/audio, but to change the volume you'll use some ioctl. See also fcntl(2) playing a similar role.
Input/output could also happen (somehow indirectly ...) thru mmap(2) and related virtual address space system calls.
For much more, read Advanced Linux Programming and Operating Systems: Three Easy Pieces. Look into Osdev for more hints about coding your own OS.
A kernel module could implement new devices, or new ioctl, etc... See kernelnewbies for more. I tend to believe it might sometimes add a few new syscalls (but this was false in older linux kernels like 3.x ones)
Linux is mostly open source. Please download then look inside source code. See also Linux From Scratch.
IIRC, Linux kernel 1.0 did not have any kernel modules. But that was around 1995.
What are the differences bettween linux system call mmap(2) and posix mmap(3) function?
How to distinguish which one is used when broswing the source code,since they have the same header file.For details, see below.
I am running on Ubuntu.I do not think it matters what operating system you are using.The mannual page really does not supply much useful information indeed.
As per the reply of Jörg W Mittag, I think the mmap must be posix function when i am broswing the source code.But i wonder that why i need not to explicitly link to posix library when using the mmamp(3) function .I mean no extra link flag is needed when complie the source code.
As per the reply of Faschingbauer,some question arise if we make the conclusion that no posix mmap is implenmented.You see, there are some posix function implemented（eg, shm_opn、sem_open, mq_open）.In the mean time，there are corresponding ones with the same functions(eg, shmget,semget, msgget).How to explain it?At least, I think some posix functions are implemented by linux.
#log for "man 2 mmap"
MMAP(2) Linux Programmer's Manual
NAME
mmap, munmap - map or unmap files or devices into memory
SYNOPSIS
#include <sys/mman.h>
#log for "man 3 mmap"
MMAP(3POSIX) POSIX Programmer's Manual
PROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for details of Linux behavior), or the interface may not be implemented on Linux.
NAME
mmap — map pages of memory
SYNOPSIS
#include <sys/mman.h>
POSIX vs. Linux
First, some facts:
POSIX is a standard, made by a standards body. POSIX does not
implement anything, but rather define feature set and behavior of
interfaces. Part of this definition is a number of man pages - the
"POSIX Programmer's Manual"
Linux implements the POSIX standard, just like other UNIX
operating systems do. (I do not know if Linux is "POSIX certified",
nor do I care.) In implementing the POSIX standard, Linux takes the
freedom to extend the standard with Linux specific features; hence
it brings its own set of manual pages, the "Linux Programmer's
Manual".
Looking at the Linux ("man 2 mmap") man page, you can see that it
defines, for example, the MAP_LOCKED bit in the flags argument
(btw. MAP_LOCKED makes a separate call to mlock() unnecessary). This
flag does not appear in the POSIX man page ("man 3 mmap"), because it
is not required by the POSIX standard for a conforming implementation.
That said, there is no way to use an alternative implementation of
mmap() in Linux. If some source code that you are reading uses mmap(),
and you are on Linux, then the Linux implementation of mmap() is used, simply
because there is no POSIX implementation of it.
Respectively, the POSIX version is contained in the Linux
implementation. Linux is "compatible" with POSIX, so to say - it does
not redefine any feature required by POSIX (this would mean to violate the standard), but only adds extensions
like the MAP_LOCKED above.
Manual Pages
Hm. My personal opinion is that the POSIX version of, say, the mmap
man page is only there to confuse users. If you inadvertently hit the
section "3" mmap() man page, and you don't know what the relationship
is between POSIX and Linux, then you are indeed seriously confused at
best, or on the wrong track at worst.
I suggest you omit the section number and just say "man mmap" - this searches all the manual
sections for occurences of "mmap" and stops at the first (see "man man" for the exact definition).
(This does not work as envisioned with "man write" when you are
searching for the definition of the write() system call, because there
is a command "write" with the same same in section "1" :-) )
System Calls
As stated by "man man", manual section "2" contains system call
documentation. Knowing that mmap() is implemented by the Linux kernel
(because it is the kernel who implements core OS functionality like
memory management) can only help to clear up the confusion as to
whether the documentation you are reading is what you want.
What are the differences bettween linux system mmap(2) and posix mmap(3) function?
Section 2 documents syscalls. Section 3 documents functions. Therefore, mmap(2) is not a function at all, it is a syscall.
How to distinguish which one is used when broswing the source code?
If it is a function call, it is mmap(3). If it is a syscall, it is mmap(2). Since it is impossible to portably call syscalls from C, there will always be some sort of macro or wrapper function for the syscall.
Also, unless you are reading the source code of the runtime library for a C compiler (e.g. GCC's) or the source code of a POSIX library (such as Glibc, dietlibc, µClibc, or musl), it is highly unlikely that you will find any syscalls in the code.
But i wonder that why i need not to explicitly link to posix library when using the mmamp(3) function .
You need not link another library because mmap is contained in GLIBC; you can see this e. g. with
nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep mmap
00000000000e4610 W mmap
00000000000e4610 W mmap64
I wanted to know whether an instruction is from the application itself or from the library code.
I observed some application code/data are located at about 0x000055xxxx while libraries and mmaped regions are by default located at 0x00007fcxxxx. Can I use for example, 0x00007f00...00 as a boundary to tell instruction is from the application itself or from the library?
How can I configure this boundary in Linux kernel?
Updated.
Can I prevent (or detect) a syscall instruction being issued from application code (only allow it to go through libc). Maybe we can do a binary scan, but due to the variable length of instruction, it's hard to prevent unintended syscall instruction.
Do it the other way. You need to learn a lot.
First, read a lot more about operating systems. So read the Operating Systems: Three Easy Pieces textbook.
Then, learn more about ASLR.
Read also Drepper's How to write shared libraries and Levine's Linkers and loaders book.
You want to use pmap(1) and proc(5).
You probably want to parse the /proc/self/maps pseudo-file from inside your program. Or use dladdr(3).
To get some insight, run cat /proc/$$/maps and cat /proc/self/maps in a Linux terminal
I wanted to know whether an instruction is from userspace or from library code.
You are confused: both library code and main executable code are userspace.
On Linux x86_64, you can distinguish kernel addresses from userpsace addresses, because the kernel addresses are in the FFFF8000'00000000 through FFFFFFFF'FFFFFFFF range on current (48-bit) implementations. See canonical form address description here.
I observed some application code/data are located at about 0x000055xxxx while libraries and mmaped regions are by default located at 0x00007fcxxxx. Can I use for example, 0x00007f00...00 as a boundary to tell instruction is from the application itself or from the library?
No, in general you can't. An application can be linked to load anywhere within canonical address space (though most applications aren't).
As Basile Starynkevitch already answered, you'll need to parse /proc/$pid/maps, or know what address the executable is linked to load at (for non-PIE binary).
I am making a /proc entry for my driver. So, in the read callback function the first argument is the location into which we write the data intended for the user. I searched on how to write the data in it and i could see that everybody is using sprintf for this purpose. I am surprised to see that it works in kernel space. However this should be wrong to use a user space function in kernel space. Also i cant figure out how to write in that location without using any user space function like strcpy, sprintf, etc. I am using kernel version 3.9.10. Please suggest me how i should do this without using sprintf or any other user space function.
Most of the 'normal' user-space functions would make no sense in kernel code, so they are not available in the kernel.
However, some functions like sprintf, strcpy, or memcpy are useful in kernel code, so the kernel implements them (more or less completely) and makes them available for drivers.
See include/linux/kernel.h and string.h.
sprintf is a kernel-space function in Linux. It is totally separate from its user-space namesake and may or may not work identically to it.
Just because a function in user-space exist, it does not mean an identically named function in kernel-space cannot.
Both methods are used to gather system calls also parameters and return values of them. When we prefer LD_PRELOAD and why? Maybe we can say that we can only gather syscalls via strace but we can gather library calls with LD_PRELOAD trick. However, there is another tracer for libraries whose name is ltrace.
strace is using the ptrace(2) syscall (with PTRACE_SYSCALL probably), so will catch every system call (thru kernel hooks installed by ptrace). It will work on any executable, even on statically linked ones, or those using something else than your distribution's GNU Glibc (like e.g. musl-libc, or some assembly written utility like old versions of busybox).
LD_PRELOAD tricks use the dynamic loader e.g. /lib64/ld-linux-x86-64.so.2 or /lib/ld.so (see ld.so(8) man page) etc... so won't work with statically linked executables (or those using something else than your dynamic loader and your GNU libc).
ltrace is probably also ptrace based.
And all these are being free software, you could study their source code (and improve it).