I am trying to understand linux syscalls mechanism. I am reading a book and it in the book it says that exit function look like that(with gdb):
mov $0x0,%ebx
mov $0x1,%eax
80 int $0x80
I understand that this is a syscall to exit, but in my Debian it looks like that:
jmp *0x8049698
push $0x8
jmp 0x80482c0
maybe can someone explain me why it's not the same? When I try to do disas on 0x80482c0
gdb prints me:
No function contains specified address.
Also, can someone give me a good reference to Linux Internals material(as Windows internals)?
Thanks!
The function you most likely called is exit() from C Standard Library (see man 3 exit). This function is a library function which, in turn, calls SYS_exit system call, but not being a system call itself. You will not see that good looking int 0x80 code in your C program disassembly. All existing functions (exit(), syscall(), etc.) are called from some library, so your program is only doing call to that library, and those functions are not belong to your program.
If you want to see exactly that int 0x80 code -- you can inline that asm code in your C application. But this is considered a bad practice, though, as your code become architecture-dependent (only applicable to x86 architecture, in your case).
can someone give me a good reference to Linux Internals material
The code itself is the best up-to-date reference. All books are more or less outdated. Also look into Documentation/ directory in kernel sources.
Related
I decided yesterday to learn assembly (NASM syntax) after years of C++ and Python and I'm already confused about the way to exit a program. It's mostly about ret because it's the suggested instruction on SASM IDE.
I'm speaking for main obviously. I don't care about x86 backward compatibility. Only the x64 Linux best way. I'm curious.
If you use printf or other libc functions, it's best to ret from main or call exit. (Which are equivalent; main's caller will call the libc exit function.)
If not, if you were only making other raw system calls like write with syscall, it's also appropriate and consistent to exit that way, but either way, or call exit are 100% fine in main.
If you want to work without libc at all, e.g. put your code under _start: instead of main: and link with ld or gcc -static -nostdlib, then you can't use ret. Use mov eax, 231 (__NR_exit_group) / syscall.
main is a real & normal function like any other (called with a valid return address), but _start (the process entry point) isn't. On entry to _start, the stack holds argc and argv, so trying to ret would set RIP=argc, and then code-fetch would segfault on that unmapped address. Nasm segmentation fault on RET in _start
System call vs. ret-from-main
Exiting via a system call is like calling _exit() in C - skip atexit() and libc cleanup, notably not flushing any buffered stdout output (line buffered on a terminal, full-buffered otherwise).
This leads to symptoms such as Using printf in assembly leads to empty output when piping, but works on the terminal (or if your output doesn't end with \n, even on a terminal.)
main is a function, called (indirectly) from CRT startup code. (Assuming you link your program normally, like you would a C program.) Your hand-written main works exactly like a compiler-generate C main function would. Its caller (__libc_start_main) really does do something like int result = main(argc, argv); exit(result);,
e.g. call rax (pointer passed by _start) / mov edi, eax / call exit.
So returning from main is exactly1 like calling exit.
Syscall implementation of exit() for a comparison of the relevant C functions, exit vs. _exit vs. exit_group and the underlying asm system calls.
C question: What is the difference between exit and return? is primarily about exit() vs. return, although there is mention of calling _exit() directly, i.e. just making a system call. It's applicable because C main compiles to an asm main just like you'd write by hand.
Footnote 1: You can invent a hypothetical intentionally weird case where it's different. e.g. you used stack space in main as your stdio buffer with sub rsp, 1024 / mov rsi, rsp / ... / call setvbuf. Then returning from main would involve putting RSP above that buffer, and __libc_start_main's call to exit could overwrite some of that buffer with return addresses and locals before execution reached the fflush cleanup. This mistake is more obvious in asm than C because you need leave or mov rsp, rbp or add rsp, 1024 or something to point RSP at your return address.
In C++, return from main runs destructors for its locals (before global/static exit stuff), exit doesn't. But that just means the compiler makes asm that does more stuff before actually running the ret, so it's all manual in asm, like in C.
The other difference is of course the asm / calling-convention details: exit status in EAX (return value) or EDI (first arg), and of course to ret you have to have RSP pointing at your return address, like it was on function entry. With call exit you don't, and you can even do a conditional tailcall of exit like jne exit. Since it's a noreturn function, you don't really need RSP pointing at a valid return address. (RSP should be aligned by 16 before a call, though, or RSP%16 = 8 before a tailcall, matching the alignment after call pushes a return address. It's unlikely that exit / fflush cleanup will do any alignment-required stores/loads to the stack, but it's a good habit to get this right.)
(This whole footnote is about ret vs. call exit, not syscall, so it's a bit of a tangent from the rest of the answer. You can also run syscall without caring where the stack-pointer points.)
SYS_exit vs. SYS_exit_group raw system calls
The raw SYS_exit system call is for exiting the current thread, like pthread_exit().
(eax=60 / syscall, or eax=1 / int 0x80).
SYS_exit_group is for exiting the whole program, like _exit.
(eax=231 / syscall, or eax=252 / int 0x80).
In a single-threaded program you can use either, but conceptually exit_group makes more sense to me if you're going to use raw system calls. glibc's _exit() wrapper function actually uses the exit_group system call (since glibc 2.3). See Syscall implementation of exit() for more details.
However, nearly all the hand-written asm you'll ever see uses SYS_exit1. It's not "wrong", and SYS_exit is perfectly acceptable for a program that didn't start more threads. Especially if you're trying to save code size with xor eax,eax / inc eax (3 bytes in 32-bit mode) or push 60 / pop rax (3 bytes in 64-bit mode), while push 231/pop rax would be even larger than mov eax,231 because it doesn't fit in a signed imm8.
Note 1: (Usually actually hard-coding the number, not using __NR_... constants from asm/unistd.h or their SYS_... names from sys/syscall.h)
And historically, it's all there was. Note that in unistd_32.h, __NR_exit has call number 1, but __NR_exit_group = 252 wasn't added until years later when the kernel gained support for tasks that share virtual address space with their parent, aka threads started by clone(2). This is when SYS_exit conceptually became "exit current thread". (But one could easily and convincingly argue that in a single-threaded program, SYS_exit does still mean exit the whole program, because it only differs from exit_group if there are multiple threads.)
To be honest, I've never used eax=252 / int 0x80 in anything, only ever eax=1. It's only in 64-bit code where I often use mov eax,231 instead of mov eax,60 because neither number is "simple" or memorable the way 1 is, so might as well be a cool guy and use the "modern" exit_group way in my single-threaded toy program / experiment / microbenchmark / SO answer. :P (If I didn't enjoy tilting at windmills, I wouldn't spend so much time on assembly, especially on SO.)
And BTW, I usually use NASM for one-off experiments so it's inconvenient to use pre-defined symbolic constants for call numbers; with GCC to preprocess a .S before running GAS you can make your code self-documenting with #include <sys/syscall.h> so you can use mov $SYS_exit_group, %eax (or $__NR_exit_group), or mov eax, __NR_exit_group with .intel_syntax noprefix.
Don't use the 32-bit int 0x80 ABI in 64-bit code:
What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code? explains what happens if you use the COMPAT_IA32_EMULATION int 0x80 ABI in 64-bit code.
It's totally fine for just exiting, as long as your kernel has that support compiled in, otherwise it will segfault just like any other random int number like int 0x7f. (e.g. on WSL1, or people that built custom kernels and disabled that support.)
But the only reason you'd do it that way in asm would be so you could build the same source file with nasm -felf32 or nasm -felf64. (You can't use syscall in 32-bit code, except on some AMD CPUs which have a 32-bit version of syscall. And the 32-bit ABI uses different call numbers anyway so this wouldn't let the same source be useful for both modes.)
Related:
Why am I allowed to exit main using ret? (CRT startup code calls main, you're not returning directly to the kernel.)
Nasm segmentation fault on RET in _start - you can't ret from _start
Using printf in assembly leads to empty output when piping, but works on the terminal stdout buffer (not) flushing with raw system call exit
Syscall implementation of exit() call exit vs. mov eax,60/syscall (_exit) vs. mov eax,231/syscall (exit_group).
Can't call C standard library function on 64-bit Linux from assembly (yasm) code - modern Linux distros config GCC in a way that call exit or call puts won't link with nasm -felf64 foo.asm && gcc foo.o.
Is main() really start of a C++ program? - Ciro's answer is a deep dive into how glibc + its CRT startup code actually call main (including x86-64 asm disassembly in GDB), and shows the glibc source code for __libc_start_main.
Linux x86 Program Start Up
or - How the heck do we get to main()? 32-bit asm, and more detail than you'll probably want until you're a lot more comfortable with asm, but if you've ever wondered why CRT runs so much code before getting to main, that covers what's happening at a level that's a couple steps up from using GDB with starti (stop at the process entry point, e.g. in the dynamic linker's _start) and stepi until you get to your own _start or main.
https://stackoverflow.com/tags/x86/info lots of good links about this and everything else.
I am learning assembly, but I'm having trouble understanding how a program is executed by a CPU when using gnu/linux on arm. I will elaborate.
Problem:
I want my program to return 5 as it's exit status.
Assembly for this is:
.text
.align 2
.global main
.type main, %function
main:
mov w0, 5 //move 5 to register w0
ret //return
I then assemble it with:
as prog.s -o prog.o
Everything ok up to here. I understand I then have to link my object file to the C library in order to add additional code that will make my program run. I then link with(paths are omitted for clarity):
ld crti.o crtn.o crt1.o libc.so prog.o ld-linux-aarch64.so.1 -o prog
After this, things work as expected:
./prog; echo $?
5
My problem is that I can't figure out what the C standard library is actually doing here. I more or less understand crti/n/1 are adding entry code to my program (eg the .init and .start sections), but no clue what's libc purpose.
I am interested at what would be a minimal assembly implementation of "returning 5 as exit status"
Most resources on the web focus on the instructions and program flow once you are in main. I am really interested at what are all the steps that go on once I execute with ./. I am now going through computer architecture textbooks, but I hope I can get a little help here.
The C language starts at main() but for C to work you in general need at least a minimal bootstrap. For example before main can be called from the C bootstrap
1) stack/stackpointer
2) .data initialized
3) .bss initalized
4) argc/argv prepared
And then there is C library which there are many/countless C libraries and each have their own designs and requirements to be satisfied before main is called. The C library makes system calls into the system so this starts to become system (operating system, Linux, Windows, etc) dependent, depending on the design of the C library that may be a thin shim or heavily integrated or somewhere in between.
Likewise for example assuming that the operating system is taking the "binary" (binary formats supported by the operating system and rules for that format are defined by the operating system and the toolchain must conform likewise the C library (even though you see the same brand name sometimes assume toolchain and C library are separate entities, one designed to work with the other)) from a non volatile media like a hard drive or ssd and copying the relevant parts into memory (some percentage of the popular, supported, binary file formats, are there for debug or file format and not actually code or data that is used for execution).
So this leaves a system level design option of does the binary file format indicate .data, .bss, .text, etc (note that .data, .bss, .text are not standards just convention most people know what that means even if a particular toolchain did not choose to use those names for sections or even the term sections).
If so the operating systems loader that takes the program and loads it into memory can choose to put .data in the right place and zero .bss for you so that the bootstrap does not have to. In a bare-metal situation the bootstrap would normally handle the read/write items because it is not loaded from media by some other software it is often simply in the address space of the processor on a rom of some flavor.
Likewise argv/argc could be handled by the operating systems tool that loads the binary as it had to parse out the location of the binary from the command line assuming the operating system has/uses a command line interface. But it could as easily simply pass the command line to the bootstrap and the bootstrap has to do it, these are system level design choices that have little to do with C but everything to do with what happens before main is called.
The memory space rules are defined by the operating system and between the operating system and the C library which often contains the bootstrap due to its intimate nature but I guess the C library and bootstrap could be separate. So linking plays a role as well, does this operating system support protection is it just read/write memory and you just need to spam it in there or are there separate regions for read/only (.text, .rodata, etc) and read/write (.data, .bss, etc). Someone needs to handle that, linker script and bootstrap often have a very intimate relationship and the linker script solution is specific to a toolchain not assumed to be portable, why would it, so while there are other solutions the common solution is that there is a C library with a bootstrap and linker solution that are heavily tied to the operating system and target processor.
And then you can talk about what happens after main(). I am happy to see you are using ARM not x86 to learn first, although aarch64 is a nightmare for a first one, not the instruction set just the execution levels and all the protections, you can go a long long way with this approach but there are some things and some instructions you cannot touch without going bare metal. (assuming you are using a pi there is a very good bare-metal forum with a lot of good resources).
The gnu tools are such that binutils and gcc are separate but intimately related projects. gcc knows where things are relative to itself so assuming you combined gcc with binutils and glibc (or you just use the toolchain you found), gcc knows relative to where it executed to find these other items and what items to pass when it calls the linker (gcc is to some extent just a shell that calls a preprocessor a compiler the assembler then linker if not instructed not to do these things). But the gnu binutils linker does not. While as distasteful as it feels to use, it is easier just to
gcc test.o -o test
rather than figure out for that machine that day what all you need on the ld command line and what paths and depending on design the order on the command line of the arguments.
Note you can probably get away with this as a minimum
.global main
.type main, %function
main:
mov w0, 5 //move 5 to register w0
ret //return
or see what gcc generates
unsigned int fun ( void )
{
return 5;
}
.arch armv8-a
.file "so.c"
.text
.align 2
.p2align 4,,11
.global fun
.type fun, %function
fun:
mov w0, 5
ret
.size fun, .-fun
.ident "GCC: (GNU) 10.2.0"
I am used to seeing more fluff in there:
.arch armv5t
.fpu softvfp
.eabi_attribute 20, 1
.eabi_attribute 21, 1
.eabi_attribute 23, 3
.eabi_attribute 24, 1
.eabi_attribute 25, 1
.eabi_attribute 26, 2
.eabi_attribute 30, 2
.eabi_attribute 34, 0
.eabi_attribute 18, 4
.file "so.c"
.text
.align 2
.global fun
.syntax unified
.arm
.type fun, %function
fun:
# args = 0, pretend = 0, frame = 0
# frame_needed = 0, uses_anonymous_args = 0
# link register save eliminated.
mov r0, #5
bx lr
.size fun, .-fun
.ident "GCC: (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609"
.section .note.GNU-stack,"",%progbits
Either way, you can look up each of the assembly language items and decide if you really need them or not, depends in part on if you feel the need to use a debugger or binutils tools to tear apart the binary (do you really need to know the size of fun for example in order to learn assembly language?)
If you wish to control all of the code and not link with a C library you are more than welcome to you need to know the memory space rules for the operating system and create a linker script (the default one may in part be tied to the C library and is no doubt overly complicated and not something you would want to use as a starting point). In this case being two instructions in main you simply need the one address space valid for the binary, however the operating system enters (ideally using the ENTRY(label), which could be main if you want but often is not _start is often found in linker scripts but is not a rule either, you choose. And as pointed out in comments you would need to make the system call to exit the program. System calls are specific to the operating system and possibly version and not specific to a target (ARM), so you would need to use the right one in the right way, very doable, your whole project linker script and assembly language could be maybe a couple dozen lines of code total. We are not here to google those for you so you would be on your own for that.
Part of your problem here is you are searching for compiler solutions when the compiler in general has absolutely nothing to do with any of this. A compiler takes one language turns it into another language. An assembler same deal but one is simple and the other usually machine code, bits. (some compilers output bits not text as well). It is equivalent to looking up the users manual for a table saw to figure out how to build a house. The table saw is just a tool, one of the tools you need, but just a generic tool. The compiler, specific gnu's gcc, is generic it does not even know what main() is. Gnu follows the Unix way so it has a separate binutils and C library, separate developments, and you do not have to combine them if you do not want to, you can use them separately. And then there is the operating system so half your question is buried in operating system details, the other half in a particular C library or other solution to connect main() to the operating system.
Being open source you can go look at the bootstrap for glibc and others and see what they do. Understanding this type of open source project the code is nearly unreadable, much easier to disassemble sometimes, YMMV.
You can search for the Linux system calls for arm aarch64 and find the one for exit, you can likely see that the open source C libraries or bootstrap solutions you find that are buried under what you are using today, will call exit but if not then there is some other call they need to make to return back to the operating system. It is unlikely it is a simple ret with a register that holds a return value, but technically that is how someone could choose to do it for their operating system.
I think you will find for Linux on arm that Linux is going to parse the command line and pass argc/argv in registers, so you can simply use them. And likely going to prep .data and .bss so long as you build the binary correctly (you link it correctly).
Here's a bare minimum example.
Run it with:
gcc -c thisfile.S && ld thisfile.o && ./a.out
Source code:
#include <sys/syscall.h>
.global _start
_start:
movq $SYS_write, %rax
movq $1, %rdi
movq $st, %rsi
movq $(ed - st), %rdx
syscall
movq $SYS_exit, %rax
movq $1, %rdi
syscall
st:
.ascii "\033[01;31mHello, OS World\033[0m\n"
ed:
I created a syscall same as /usr/share/examples/kld/syscall/module/syscall.c with a little change in message.
I used kldload and module loaded. now I want to call the syscall.
what is this syscall number so I can call it?
or what is the way to call this syscall?
I suggest you take a look at Designing BSD rootkits, that's how I learned kernel programming on FreeBSD, there's even a section that talks all about making your own syscalls.
Well, if you check /usr/share/examples/kld/syscall directory you will see it contains a test program..... but hey, let's assume the program is not there.
Let's take a look at part of the module itself:
/*
* The offset in sysent where the syscall is allocated.
*/
static int offset = NO_SYSCALL;
[..]
case MOD_LOAD :
printf("syscall loaded at %d\n", offset);
break;
The module prints syscall number on load, so the job now is to learn how to call it... a 'freebsd call syscall' search on google...
Reveals: http://www.freebsd.cz/doc/en/books/developers-handbook/x86-system-calls.html (although arguably not something to use on amd64) and.. https://www.freebsd.org/cgi/man.cgi?query=syscall&sektion=2 - a manual page for a function which allows you to call arbitrary syscalls.
I strongly suggest you do some digging on your own. If you don't, there is absolutely no way you will be able to write any kernel code.
I need some reference but a good one, possibly with some nice examples. I need it because I am starting to write code in assembly using the NASM assembler. I have this reference:
http://bluemaster.iu.hio.no/edu/dark/lin-asm/syscalls.html
which is quite nice and useful, but it's got a lot of limitations because it doesn't explain the fields in the other registers. For example, if I am using the write syscall, I know I should put 1 in the EAX register, and the ECX is probably a pointer to the string, but what about EBX and EDX? I would like that to be explained too, that EBX determines the input (0 for stdin, 1 for something else etc.) and EDX is the length of the string to be entered, etc. etc. I hope you understood me what I want, I couldn't find any such materials so that's why I am writing here.
Thanks in advance.
The standard programming language in Linux is C. Because of that, the best descriptions of the system calls will show them as C functions to be called. Given their description as a C function and a knowledge of how to map them to the actual system call in assembly, you will be able to use any system call you want easily.
First, you need a reference for all the system calls as they would appear to a C programmer. The best one I know of is the Linux man-pages project, in particular the system calls section.
Let's take the write system call as an example, since it is the one in your question. As you can see, the first parameter is a signed integer, which is usually a file descriptor returned by the open syscall. These file descriptors could also have been inherited from your parent process, as usually happens for the first three file descriptors (0=stdin, 1=stdout, 2=stderr). The second parameter is a pointer to a buffer, and the third parameter is the buffer's size (as an unsigned integer). Finally, the function returns a signed integer, which is the number of bytes written, or a negative number for an error.
Now, how to map this to the actual system call? There are many ways to do a system call on 32-bit x86 (which is probably what you are using, based on your register names); be careful that it is completely different on 64-bit x86 (be sure you are assembling in 32-bit mode and linking a 32-bit executable; see this question for an example of how things can go wrong otherwise). The oldest, simplest and slowest of them in the 32-bit x86 is the int $0x80 method.
For the int $0x80 method, you put the system call number in %eax, and the parameters in %ebx, %ecx, %edx, %esi, %edi, and %ebp, in that order. Then you call int $0x80, and the return value from the system call is on %eax. Note that this return value is different from what the reference says; the reference shows how the C library will return it, but the system call returns -errno on error (for instance -EINVAL). The C library will move this to errno and return -1 in that case. See syscalls(2) and intro(2) for more detail.
So, in the write example, you would put the write system call number in %eax, the first parameter (file descriptor number) in %ebx, the second parameter (pointer to the string) in %ecx, and the third parameter (length of the string) in %edx. The system call will return in %eax either the number of bytes written, or the error number negated (if the return value is between -1 and -4095, it is a negated error number).
Finally, how do you find the system call numbers? They can be found at /usr/include/linux/unistd.h. On my system, this just includes /usr/include/asm/unistd.h, which finally includes /usr/include/asm/unistd_32.h, so the numbers are there (for write, you can see __NR_write is 4). The same goes for the error numbers, which come from /usr/include/linux/errno.h (on my system, after chasing the inclusion chain I find the first ones at /usr/include/asm-generic/errno-base.h and the rest at /usr/include/asm-generic/errno.h). For the system calls which use other constants or structures, their documentation tells which headers you should look at to find the corresponding definitions.
Now, as I said, int $0x80 is the oldest and slowest method. Newer processors have special system call instructions which are faster. To use them, the kernel makes available a virtual dynamic shared object (the vDSO; it is like a shared library, but in memory only) with a function you can call to do a system call using the best method available for your hardware. It also makes available special functions to get the current time without even having to do a system call, and a few other things. Of course, it is a bit harder to use if you are not using a dynamic linker.
There is also another older method, the vsyscall, which is similar to the vDSO but uses a single page at a fixed address. This method is deprecated, will result in warnings on the system log if you are using recent kernels, can be disabled on boot on even more recent kernels, and might be removed in the future. Do not use it.
If you download that web page (like it suggests in the second paragraph) and download the kernel sources, you can click the links in the "Source" column, and go directly to the source file that implements the system calls. You can read their C signatures to see what each parameter is used for.
If you're just looking for a quick reference, each of those system calls has a C library interface with the same name minus the sys_. So, for example, you could check out man 2 lseek to get the information about the parameters forsys_lseek:
off_t lseek(int fd, off_t offset, int whence);
where, as you can see, the parameters match the ones from your HTML table:
%ebx %ecx %edx
unsigned int off_t unsigned int
When a syscall returns, I get the syscall return value in %eax, however on entry I am getting -38, which is 0xFFFFFFDA in hex. This is for both write/read. What is this number? Can it be used to safely differentiate an entry from an exit?
The -38 in eax on syscall entry is apparently ENOSYS (Function not implemented), and is put there by syscall_trace_entry in arch/x86/kernel/entry_32.S. I suppose it's safe to assume that it will always be there on syscall entry, however it can also be there on syscall exit, if the syscall returns ENOSYS.
Personally, I have always just kept track of whether I'm in syscall entry or exit when using ptrace, although I have seen some code relying on the ENOSYS too. (I'm assuming you're using ptrace) I guess that won't work if the process happens to be inside a syscall when you attach to it, but I have been lucky enough to not bump into that problem.
I took a quick look at strace sources, and I guess it keeps track of the state too, since there was a comment saying "We are attaching to an already running process. Try to figure out the state of the process in syscalls, to handle the first event well." and slightly after that it said "The process is asleep in the middle of a syscall. Fake the syscall entry event.".
In short, the value can't be safely used to differentiate an entry from an exit. That said, I'm not sure that tracking it manually is the best method, since I haven't really got any source which would definitely tell you to use that technique, sorry. :)
I still not get when you get the -38 in eax, but when doing a syscall eax contains a number that defines the syscall (in a 2.6 Kernel you can have a look at arch/x86/include/asm/unistd_64.h to see the numbers for each call).
So the sequence is the following:
your programm
set eax to syscall (dep on call, also some other regs)
init syscall (via int 0x80)
result of syscall in eax
your programm again
Maybe your question is not so formulated, but if you are not writing kernel code/driver the easiest way to tell, wether you are before syscall entry or after syscall exit is: TRUE when you are in your code ;-). The entry/exit itself happen (more or less) instant in one instruction, so either you are in the syscall (then you would know because it must be some kernel code or the blocking call) or you are not (almost everytime when you debug your code).