segfault on write() with ~8MB buffer (OSX, Linux) - linux

Question

I was curious what kind of buffer sizes write() and read() could handle on Linux/OSX/FreeBSD, so I started playing around with dumb programs like the following:
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
int main( void ) {
size_t s = 8*1024*1024 - 16*1024;
while( 1 ) {
s += 1024;
int f = open( "test.txt", O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR | S_IXUSR );
char mem[s];
size_t written = write( f, &mem[0], s );
close( f );
printf( "(%ld) %lu\n", sizeof(size_t), written );
}
return 0;
}
This allowed me to test how close to a seeming "8MB barrier" I could get before segfaulting. Somewhere around the 8MB mark, my program dies, here's an example output:
(8) 8373248
(8) 8374272
(8) 8375296
(8) 8376320
(8) 8377344
(8) 8378368
(8) 8379392
(8) 8380416
(8) 8381440
(8) 8382464
Segmentation fault: 11
This is the same on OSX and Linux, however my FreeBSD VM is not only much faster at running this test, it also can go on for quite a ways! I've successfully tested it up to 511MB, which is just a ridiculous amount of data to write in one call.
What is it that makes the write() call segfault, and how can I figure out the maximum amount that I can possibly write() in a single call, without doing something ridiculous like I'm doing right now?
(Note, all three operating systems are 64-bit, OSX 10.7.3, Ubuntu 11.10, FreeBSD 9.0)

Answer 1

The fault isn't within write(), it's a stack overflow. Try this:
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
int main( void )
{
void *mem;
size_t s = 512*1024*1024 - 16*1024;
while( 1 )
{
s += 1024;
int f = open( "test.txt", O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR | S_IXUSR );
mem = malloc(s);
size_t written = write( f, mem, s );
free(mem);
close( f );
printf( "(%ld) %lu\n", sizeof(size_t), written );
}
return 0;
}