I am currently writing a compiler (http://curly-lang.org if you're curious), and have been encountering a strange bug when trying to run the generated ELF binaries on the latest Linux kernel. The same binaries run fine on older kernels (I've tried on several Ubuntu boxes, uname 4.4.0-1049-aws), but on my updated Arch box (uname 4.17.11-arch1), I can't even open them under GDB.
The error message given by GDB is During startup program terminated with signal SIGSEGV, Segmentation fault, which as I understand is indicative of a failure to load the program segments before the first instruction is ever run.
I compiled a minimal ELF executable with GCC/NASM to try and reproduce the problem, but the GCC-produced executable loads without a hitch, whereas my programs definitely do not.
Here are the printouts of readelf -a for both executables, for reference. The first is the program generated by my compiler :
$ readelf -a my-program
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x400040
Start of program headers: 2400 (bytes into file)
Start of section headers: 2568 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 3
Size of section headers: 64 (bytes)
Number of section headers: 6
Section header string table index: 1
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] STRTAB 0000000000000000 00000b88
000000000000009e 0000000000000000 0 0 0
[ 2] .init PROGBITS 0000000000400040 00000040
000000000000006b 0000000000000000 AX 0 0 0
[ 3] .text PROGBITS 00000000004000ab 000000ab
0000000000000824 0000000000000000 AX 0 0 0
[ 4] .data PROGBITS 00000000008008cf 000008cf
0000000000000091 0000000000000000 WA 0 0 0
[ 5] .symtab SYMTAB 0000000000000000 00000c26
00000000000000c0 0000000000000018 1 8 0
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000040 0x0000000000400040 0x0000000000000000
0x000000000000006b 0x000000000000006b R E 0x1000
LOAD 0x00000000000000ab 0x00000000004000ab 0x0000000000000000
0x0000000000000824 0x0000000000000824 R E 0x1000
LOAD 0x00000000000008cf 0x00000000008008cf 0x0000000000000000
0x0000000000000091 0x0000000000000091 RW 0x1000
Section to Segment mapping:
Segment Sections...
00 .init
01 .text
02 .data
There is no dynamic section in this file.
There are no relocations in this file.
The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported.
Symbol table '.symtab' contains 8 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 00000000004004e0 0 NOTYPE LOCAL HIDDEN 3 .text.argument
1: 00000000004005a0 0 NOTYPE LOCAL HIDDEN 3 .text.constant
2: 0000000000400270 0 NOTYPE LOCAL HIDDEN 3 .text.memextend-page
3: 0000000000400210 0 NOTYPE LOCAL HIDDEN 3 .text.memextend-pool-32
4: 00000000004002b0 0 NOTYPE LOCAL HIDDEN 3 .text.unit
5: 00000000004005f0 0 NOTYPE LOCAL HIDDEN 3 .text.write
6: 00000000008008d0 0 NOTYPE LOCAL HIDDEN 4 .data.brkaddr
7: 0000000000400040 0 NOTYPE LOCAL HIDDEN 2 .init.brkaddr-init
No version information found in this file.
And for the GCC-generated program :
$ readelf -a gcc-program
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x400110
Start of program headers: 64 (bytes into file)
Start of section headers: 336 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 3
Size of section headers: 64 (bytes)
Number of section headers: 5
Section header string table index: 4
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .note.gnu.build-i NOTE 00000000004000e8 000000e8
0000000000000024 0000000000000000 A 0 0 4
[ 2] .text PROGBITS 0000000000400110 00000110
0000000000000010 0000000000000000 AX 0 0 16
[ 3] .data PROGBITS 0000000000600120 00000120
0000000000000001 0000000000000000 WA 0 0 4
[ 4] .shstrtab STRTAB 0000000000000000 00000121
000000000000002a 0000000000000000 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x0000000000000120 0x0000000000000120 R E 0x200000
LOAD 0x0000000000000120 0x0000000000600120 0x0000000000600120
0x0000000000000001 0x0000000000000001 RW 0x200000
NOTE 0x00000000000000e8 0x00000000004000e8 0x00000000004000e8
0x0000000000000024 0x0000000000000024 R 0x4
Section to Segment mapping:
Segment Sections...
00 .note.gnu.build-id .text
01 .data
02 .note.gnu.build-id
There is no dynamic section in this file.
There are no relocations in this file.
The decoding of unwind sections for machine type Advanced Micro Devices X86-64 is not currently supported.
No version information found in this file.
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 1a3e678b08996ee6a9d289c3f76c7c52cd4a30aa
As you can see, I've tried to mirror GCC's segment placement (~0x400000 for the code, and ~0x800000 for the data), and the two ELF headers are strictly identical. The only meaningful difference I can think of is that my custom binaries have two LOAD segments (one for the initialization code, one for the rest) that share the same page, whereas GCC only produces a single code LOAD segment. That shouldn't pose a problem, though, since they both share the same permissions and don't overlap.
Other than that, I do not see what could possibly keep the first program from loading correctly. If anyone well-versed in the arcanes of the Linux ELF loader could enlighten me, that would be greatly appreciated.
Thank you for your attention,
Nevermind everyone, it was the page-sharing segments that were causing the problem all along.
Thinking the problem was probably in the kernel loader, I should have thought about running dmesg much earlier, where I would have noticed the following message, clear as day :
[54178.211348] 12766 (my-program): Uhuuh, elf segment at 0000000000400000 requested but the memory is mapped already
Apparently, some benevolent mastermind decided 3 months ago that it would be good to actually catch double-mapping errors instead of just letting them go silently as we always did in the ELF loader.
It's not that my binaries used to be correct, it's that the error they were causing wasn't caught before. I don't know if I should be proud or ashamed for my bugs to have eluded detection all this time.
Anyway, I leave this answer to warn anyone foolish enough to map multiple segments on a single page in an ELF binary : do not. There is no try.
PS: #rodrigo: Thanks for your answer, I didn't even notice the PhysAddr before you pointed them out. The manual says that they're used "on systems where physical addressing is relevant", which doesn't seem to be the case here, but I'll remember to keep a lookout for them next time.
Related
If I have to design a simulator of sorts, can the ending address of .text section (found out by starting address and size of .text) of an ELF be considered the ending place for the instructions?
In the sense, can the last address of .text section be treated as the last instruction address?
can the last address of .text section be treated as the last instruction address?
The last address of .text section is indeed (the end of) the last instruction in that section.
But nothing in ELF specification prevents the file from having executable instructions in other sections. Indeed, this is very common. For example:
readelf -WS /bin/ls | grep ' AX'
[11] .init PROGBITS 0000000000004000 004000 000017 00 AX 0 0 4
[12] .plt PROGBITS 0000000000004020 004020 0006b0 10 AX 0 0 16
[13] .plt.got PROGBITS 00000000000046d0 0046d0 000018 08 AX 0 0 8
[14] .text PROGBITS 00000000000046f0 0046f0 01254e 00 AX 0 0 16
[15] .fini PROGBITS 0000000000016c40 016c40 000009 00 AX 0 0 4
All of .init, .fini, .text etc. contain executable instructions.
Update:
So if I can run the following command on the elf input file, can I consider the last address in the last executable section as end of instructions?
If you are writing an emulator, surely you wouldn't want to run readelf and parse its output, but instead would parse the info directly by reading the ELF file and parsing its contents.
Also note that an executable can mmap(..., PROT_READ|PROT_EXEC, ...) or mprotect additional executable sections, so knowing the last address of instruction in the ELF file is somewhat pointless.
is it possible to separate the .text into the 4K aligned pages. For example, I wrote a simple program and used readelf -a a.out to dump the section headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
... ...
[11] .init PROGBITS 00000470 000470 000023 00 AX 0 0 4
[12] .plt PROGBITS 000004a0 0004a0 000060 04 AX 0 0 16
[13] .text PROGBITS 00000500 000500 000342 00 AX 0 0 16
[14] .fini PROGBITS 00000844 000844 000014 00 AX 0 0 4
How can I put .text into a 4K aligned page (e.g., Addr for .text be 0x1000)?
Thank you!
The tool responsible of the issue you post is ld(1) (the linker) It has a complete manual describing the scriptable language used to align pages and create the final program. Just have a read to that manual (I'm referring to the ld manual, not the manual page)
Once you have read it, you can create a miniscript file that, based on the standard linux script, forces *(.text) segments to be page aligned and use it to link your program.
I am currently trying to create simple ELF executable using ELFIO library based on their docs (page 8) [1]. This is output from readelf when run on the slightly modified version of their code (nothing important, just different string and address base. Alignments, flags etc. are untouched.)
ELF Header:
Magic: 7f 45 4c 46 01 01 01 03 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - GNU
ABI Version: 0
Type: EXEC (Executable file)
Machine: Intel 80386
Version: 0x1
Entry point address: 0x400000
Start of program headers: 52 (bytes into file)
Start of section headers: 4168 (bytes into file)
Flags: 0x0
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 2
Size of section headers: 40 (bytes)
Number of section headers: 4
Section header string table index: 1
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .shstrtab STRTAB 00000000 00102e 000017 00 0 0 0
[ 2] .text PROGBITS 00400000 001000 00001d 00 AX 0 0 16
[ 3] .data PROGBITS 00400020 001020 00000e 00 WA 0 0 4
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x001000 0x00400000 0x00400000 0x0001d 0x0001d R E 0x1000
LOAD 0x001020 0x00400020 0x00400020 0x0000e 0x0000e RW 0x10
Section to Segment mapping:
Segment Sections...
00 .text
01 .data
This executable file runs fine. I wanted to move data section a little bit further from text section, so I have chosen address 0x400040. here you can see that it is all I did.
ELF Header:
Magic: 7f 45 4c 46 01 01 01 03 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - GNU
ABI Version: 0
Type: EXEC (Executable file)
Machine: Intel 80386
Version: 0x1
Entry point address: 0x400000
Start of program headers: 52 (bytes into file)
Start of section headers: 4168 (bytes into file)
Flags: 0x0
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 2
Size of section headers: 40 (bytes)
Number of section headers: 4
Section header string table index: 1
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .shstrtab STRTAB 00000000 00102e 000017 00 0 0 0
[ 2] .text PROGBITS 00400000 001000 00001d 00 AX 0 0 16
[ 3] .data PROGBITS 00400040 001020 00000e 00 WA 0 0 4
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x001000 0x00400000 0x00400000 0x0001d 0x0001d R E 0x1000
LOAD 0x001020 0x00400040 0x00400040 0x0000e 0x0000e RW 0x10
Section to Segment mapping:
Segment Sections...
00 .text
01 .data
However, this file keeps seg. faulting on its start. I can't really understand what is going on. At first I though it has something to do with alignment of sections and segments but they seem OK to me. I have also suspected ASLR (because the address of Hello World is hardcoded and there are no relocations) and tried to turned it off in /proc/sys/kernel/randomize_va_space. I have also checked linux kernel source code, especially file fs/binfmt_elf.c, but to fully understand what is going on there I will need more time. So I would like to ask for your help. If you know what is wrong with the address 0x400040, why this file keeps seg. faulting, or you can point me to a certain lines of code in kernel, then I would really appreciate that.
Edit: I have also tried gdb but bt provides No stack. Setting read watchpoint on entry point address does not do anything.
Edit 2: After playing around with the file I found out that moving the file offset of second segment from 0x1020 to 0x1040 causes the file to work fine. So now my question is, does file offset and virtual address need to be in some relation?
[1] http://elfio.sourceforge.net/elfio.pdf
So now my question is, does file offset and virtual address need to be in some relation?
Yes: file offset must be congruent to VirtAddr and PhysAddr modulo page size, because the file is mmaped directly (by the kernel loader).
If you do not maintain that relationship, the kernel will create a new process, attempt to map your executable into it, discover that you've violated basic constraint, and will terminate your process (with SIGKILL, I believe). Your process never executes a single instruction in user-space.
No stack frame is created then it means there is some problem occurs before entry point.
Please tell me whether the file is linked with any library modules like libc because they will provide initial setup to call our entry point function.
Use gdb and make breakpoint at _start. If you don't linked with library then the task of setting up stack, arguments, environment will be your responsibility.
According to readelf:
----------------------------------------------------------------------
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[24] .data PROGBITS 0000000000601040 00001040
0000000000000051 0000000000000000 WA 0 0 32
----------------------------------------------------------------------
Section to Segment mapping:
Segment Sections...
00
01
02
03 .init_array .fini_array .jcr .dynamic .got .got.plt .data .bss
----------------------------------------------------------------------
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR
INTERP
LOAD
LOAD 0x0000000000000e10 0x0000000000600e10 0x0000000000600e10
0x0000000000000281 0x0000000000000288 RW 200000
As you can see above .data segment has W (Write) and A (Alloc) permissions and .data is loaded in a LOAD section with R (Read) W (Write).
However, the shellcode in the .data section is executable, according to GDB:
0x601060 <bytecode>: xor rax,rax
=> 0x601063 <bytecode+3>: xor rdi,rdi
And I don't know why. Is this correct? What am I missing?
However, the shellcode in the .data section is executable, according to GDB:
The GDB output does not tell you that the .data section is executable. GDB will happily disassemble any memory you ask it to disassemble.
Try this:
(gdb) set $p = (void (*)(void))&bytecode
(gdb) call $p()
This should result in a SIGSEGV on the first instruction of bytecode, because it in fact is not executable.
Help me please with gnu assembler for arm926ejs cpu.
I try to build a simple program(test.S):
.global _start
_start:
mov r0, #2
bx lr
and success build it:
arm-none-linux-gnueabi-as -mthumb -o test.o test.S
arm-none-linux-gnueabi-ld -o test test.o
but when I run the program in the arm target linux environment, I get an error:
./test
Segmentation fault
What am I doing wrong?
Can _start function be the thumb func?
or
It is always arm func?
Can _start be a thumb function (in a Linux user program)?
Yes it can. The steps are not as simple as you may believe.
Please use the .code 16 as described by others. Also look at ARM Script predicate; my answer shows how to detect a thumb binary. The entry symbol must have the traditional _start+1 value or Linux will decide to call your _start in ARM mode.
Also your code is trying to emulate,
int main(void) { return 2; }
The _start symbol must not do this (as per auselen). To do _start to main() in ARM mode you need,
#include <linux/unistd.h>
static inline void exit(int status)
{
asm volatile ("mov r0, %0\n\t"
"mov r7, %1\n\t"
"swi #7\n\t"
: : "r" (status),
"Ir" (__NR_exit)
: "r0", "r7");
}
/* Wrapper for main return code. */
void __attribute__ ((unused)) estart (int argc, char*argv[])
{
int rval = main(argc,argv);
exit(rval);
}
/* Setup arguments for estart [like main()]. */
void __attribute__ ((naked)) _start (void)
{
asm(" sub lr, lr, lr\n" /* Clear the link register. */
" ldr r0, [sp]\n" /* Get argc... */
" add r1, sp, #4\n" /* ... and argv ... */
" b estart\n" /* Let's go! */
);
}
It is good to clear the lr so that stack traces will terminate. You can avoid the argc and argv processing if you want. The start shows how to work with this. The estart is just a wrapper to convert the main() return code to an exit() call.
You need to convert the above assembler to Thumb equivalents. I would suggest using gcc inline assembler. You can convert to pure assembler source if you get inlines to work. However, doing this in 'C' source is probably more practical, unless you are trying to make a very minimal executable.
Helpful gcc arguements are,
-nostartfiles -static -nostdlib -isystem <path to linux user headers>
Add -mthumb and you should have a harness for either mode.
Your problem is you end with
bx lr
and you expect Linux to take over after that. That exact line must be the cause of Segmentation fault.
You can try to create a minimal executable then try to bisect it to see the guts and understand how an executable is expected to behave.
See below for a working example:
.global _start
.thumb_func
_start:
mov r0, #42
mov r7, #1
svc #0
compile with
arm-linux-gnueabihf-as start.s -o start.o && arm-linux-gnueabihf-ld
start.o -o start_test
and dump to see the guts
$ arm-linux-gnueabihf-readelf -a -W start_test
Now you should notice the odd address of _start
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x8055
Start of program headers: 52 (bytes into file)
Start of section headers: 160 (bytes into file)
Flags: 0x5000000, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 1
Size of section headers: 40 (bytes)
Number of section headers: 6
Section header string table index: 3
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .text PROGBITS 00008054 000054 000006 00 AX 0 0 4
[ 2] .ARM.attributes ARM_ATTRIBUTES 00000000 00005a 000014 00 0 0 1
[ 3] .shstrtab STRTAB 00000000 00006e 000031 00 0 0 1
[ 4] .symtab SYMTAB 00000000 000190 0000e0 10 5 6 4
[ 5] .strtab STRTAB 00000000 000270 000058 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x000000 0x00008000 0x00008000 0x0005a 0x0005a R E 0x8000
Section to Segment mapping:
Segment Sections...
00 .text
There is no dynamic section in this file.
There are no relocations in this file.
There are no unwind sections in this file.
Symbol table '.symtab' contains 14 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 00000000 0 NOTYPE LOCAL DEFAULT UND
1: 00008054 0 SECTION LOCAL DEFAULT 1
2: 00000000 0 SECTION LOCAL DEFAULT 2
3: 00000000 0 FILE LOCAL DEFAULT ABS start.o
4: 00008054 0 NOTYPE LOCAL DEFAULT 1 $t
5: 00000000 0 FILE LOCAL DEFAULT ABS
6: 0001005a 0 NOTYPE GLOBAL DEFAULT 1 _bss_end__
7: 0001005a 0 NOTYPE GLOBAL DEFAULT 1 __bss_start__
8: 0001005a 0 NOTYPE GLOBAL DEFAULT 1 __bss_end__
9: 00008055 0 FUNC GLOBAL DEFAULT 1 _start
10: 0001005a 0 NOTYPE GLOBAL DEFAULT 1 __bss_start
11: 0001005c 0 NOTYPE GLOBAL DEFAULT 1 __end__
12: 0001005a 0 NOTYPE GLOBAL DEFAULT 1 _edata
13: 0001005c 0 NOTYPE GLOBAL DEFAULT 1 _end
No version information found in this file.
Attribute Section: aeabi
File Attributes
Tag_CPU_arch: v4T
Tag_THUMB_ISA_use: Thumb-1
here answer.
Thanks for all.
http://stuff.mit.edu/afs/sipb/project/egcs/src/egcs/gcc/config/arm/README-interworking
Calls via function pointers should use the BX instruction if the call is made in ARM mode:
.code 32
mov lr, pc
bx rX
This code sequence will not work in Thumb mode however, since the mov instruction will not set the bottom bit of the lr register. Instead a branch-and-link to the _call_via_rX functions should be used instead:
.code 16
bl _call_via_rX
where rX is replaced by the name of the register containing the function address.