Different memory alignment for different buffer sizes - linux

Question

I'm trying to educate myself regarding stack overflows and played around a bit with these -fno-stack-protector flag and tried to understand how memory is managed in a process.
I compiled the following code (using Ubuntu 18.04.1 LTS (x86_64), gcc 7.3.0., ASLR disabled)
int main (int argc, char *argv[])
{
char buff[13];
return 0;
}
as follows: gcc -g -o main main.c -fno-stack-protector. I then evoked gdb main, b 4, run and as can be seen from the the following outputs
(gdb) print &buff
$2 = (char (*)[13]) 0x7fffffffd963
0x7fffffffd963: 0xff 0xff 0x7f 0x00 0x00 0x00 0x00 0x00
0x7fffffffd96b: 0x00 0x00 0x00 0x00 0x00 0x10 0x46 0x55
0x7fffffffd973: 0x55 0x55 0x55 0x00 0x00 0x97 0x5b 0xa0
0x7fffffffd97b: 0xf7 0xff 0x7f 0x00 0x00 0x01 0x00 0x00
(gdb) info frame 0
Stack frame at 0x7fffffffd980:
[...]
Saved registers:
rbp at 0x7fffffffd970, rip at 0x7fffffffd978
the 13 bytes allocated for the buffer follow directly after the saved base pointer rbp.
After increasing the buffer size from 13 to 21 I got the following results:
(gdb) print &buff
$3 = (char (*)[21]) 0x7fffffffd950
(gdb) x/48bx buff
0x7fffffffd950: 0x10 0x46 0x55 0x55 0x55 0x55 0x00 0x00
0x7fffffffd958: 0xf0 0x44 0x55 0x55 0x55 0x55 0x00 0x00
0x7fffffffd960: 0x50 0xda 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffd968: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffd970: 0x10 0x46 0x55 0x55 0x55 0x55 0x00 0x00
0x7fffffffd978: 0x97 0x5b 0xa0 0xf7 0xff 0x7f 0x00 0x00
(gdb) info frame 0
Stack frame at 0x7fffffffd980:
[...]
Saved registers:
rbp at 0x7fffffffd970, rip at 0x7fffffffd978
Now there are additional 11 bytes after the rbp before the buffer follows.
In the second case, why are there 11 additional bytes? Is this due to the alignment of the stack, e.g. does the buffer have to be 16 bytes aligned (a multiple of 16) starting from rbp?
Why is the memory layout different in the first case, there seems to be no alignment?

Answer 1

The x86-64 System V ABI requires 16-byte alignment for local or global arrays that are 16 bytes or larger, and for all C99 VLAs (which are always local).
An array uses the same alignment as its elements, except that a local or global
array variable of length at least 16 bytes or a C99 variable-length array variable
always has alignment of at least 16 bytes.4
4 The alignment requirement allows the use of SSE instructions when operating on the array.
The compiler cannot in general calculate the size of a variable-length array (VLA), but it is expected
that most VLAs will require at least 16 bytes, so it is logical to mandate that VLAs have at
least a 16-byte alignment.
Fixed-size arrays smaller than one SIMD vector (16 bytes) don't have this requirement, so they can pack efficiently in the stack layout.
Note that this doesn't apply to arrays inside structs, only to locals and globals.
(For dynamic storage, the alignment of a malloc return value must be aligned enough to hold any object up to that size, and since x86-64 SysV has maxalign_t of 16 bytes, malloc must also return 16-byte aligned pointers if the size is 16 or higher. For smaller allocations, it could return only 8B-aligned for an 8B allocation if it wanted to.)
The requirement for local arrays makes it safe to write code that passes their address to a function that requires 16-byte alignment, but this is mostly not something the ABI itself really needs to specify.
It's not something that different compilers have to agree on to link their code together, the way struct layout or the calling convention is (which registers are call-clobbered, or used for arg-passing...). The compiler basically owns the stack layout for the function it's compiling, and other functions can't assume or depend on anything about it. They'd only get pointers to your local vars if you pass pointers as function args, or store pointers into globals.
Specifying it for globals is useful, though: it makes it safe for compiler-generated auto-vectorized code to assume alignment for global arrays, even when it's an extern int[] in an object file compiled by another compiler.