File upload fails when file is over 60K in size - linux

Question

I have been working to convert an in-house application away from using FTP, as the security team has told us to get off FTP. So I've been using HTTP uploads instead, and for the most part it works very well. Our environment is a mishmash of Linux, HP-UX, Solaris, and AIX. On our Linux servers, curl is universally available, so I have been using curl's POST capabilities for uploads, and it's worked flawlessly. Unfortunately, the Unix machines rarely have curl, or even wget, so I wrote a GET script with perl which works fine, and the POST script I wrote for perl(lifted and adapted from elsewhere on the web) works brilliantly for Unix, up until the data being uploaded is greater than about 60K(which curl handles fine in Linux, btw). Beyond that, the Apache error log starts spitting out:
CGI.pm: Server closed socket during multipart read (client aborted?).
No such error ever occurs when I use curl for the upload. Here's my POST script, using Socket, since LWP is not available on every server, and not at all on any of the Unix servers.
#!/usr/bin/perl -w
use strict;
use Socket;
my $v = 'dcsm';
my $upfile = $ARGV[0] or die 'Upload File not found or not specified.' . "\n";
my $hostname = $ARGV[1] or die 'Hostname not specified.' . "\n";
$| = 1;
my $host = "url.mycompany dot com";
my $url = "/csmtar.cgi";
my $start = times;
my ( $iaddr, $paddr, $proto );
$iaddr = inet_aton($host);
$paddr = sockaddr_in( 80, $iaddr );
$proto = getprotobyname('tcp');
unless ( socket( SOCK, PF_INET, SOCK_STREAM, $proto ) ) {
die "ERROR : init socket: $!";
}
unless ( connect( SOCK, $paddr ) ) {
die "no connect: $!\n";
}
my $length = 0;
open( UH, "< $upfile" ) or warn "$!\n";
$length += -s $upfile;
my $boundary = 'nn7h23ffh47v98';
my #head = (
"POST $url HTTP/1.1",
"Host: $host",
"User-Agent: z-uploader",
"Content-Length: $length",
"Content-Type: multipart/form-data; boundary=$boundary",
"",
"--$boundary",
"Content-Disposition: form-data; name=\"hostname\"",
"",
"$hostname",
"--$boundary",
"Content-Disposition: form-data; name=\"ren\"",
"",
"true",
"--$boundary",
"Content-Disposition: file; name=\"filename\"; filename=\"$upfile\"",
"--$boundary--",
"",
"",
);
my $header = join( "\r\n", #head );
$length += length($header);
$head[3] = "Content-Length: $length";
$header = join( "\r\n", #head );
$length = -s $upfile;
$length += length($header);
select SOCK;
$| = 1;
print SOCK $header;
while ( sysread( UH, my $buf, 8196 ) ) {
if ( length($buf) < 8196 ) {
$buf = $buf . "\r\n--$boundary";
syswrite SOCK, $buf, length($buf);
} else {
syswrite SOCK, $buf, 8196;
}
print STDOUT '.',;
}
close UH;
shutdown SOCK, 1;
my #data = (<SOCK>);
print STDOUT "result->#data\n";
close SOCK;
Anybody see something that jumps out at them?
UPDATE:
I made the following updates, and the errors appear to be unchanged.
To address the content-length issue, and attempt to eliminate the potential for the loop equalling the exact number of characters before appending the final boundary, I made the following code update.
my $boundary = 'nn7h23ffh47v98';
my $content = <<EOF;
--$boundary
Content-Disposition: form-data; name="hostname"
$hostname
--$boundary
Content-Disposition: file; name="filename"; filename="$upfile"
--$boundary--
EOF
$length += length($content);
my $header = <<EOF;
POST $url HTTP/1.1
Host: $host
User-Agent: z-uploader
Content-Length: $length
Content-Type: multipart/form-data; boundary=$boundary
EOF
$header .= $content;
select SOCK;
$| = 1;
print SOCK $header;
my $incr = ($length + 100) / 20;
$incr = sprintf("%.0f", $incr);
while (sysread(UH, my $buf, $incr )) {
syswrite SOCK, $buf, $incr;
}
syswrite SOCK, "\n--$boundary", $incr;

Answer 1

You are asking if there's "something that jumps out", from looking at the code.
Two things jump out at me:
1) The Content-Length parameter in a POST HTTP message specifies the exact byte count of the entity portion of the HTTP message. See section 4.4 of RFC 2616.
You are setting the Content-Length: header to the exact size of the of the file you're uploading. Unfortunately, in addition to the file itself, you are also sending the MIME headers.
The "entity" portion of the HTTP message, as defined by RFC 2616, essentially consists of everything after the blank line following the last header of the HTTP message header. Everything below that point must be included in the Content-Length: header. The Content-Length header is NOT the size of the file you're uploading, but the same of the HTTP message's entire entity portion, which follows the header.
2) Ignoring the broken Content-Length: header, if the size of the file happens to be an exact multiple of 8196 bytes, the MIME document you are constructing will most likely be corrupted. Your last sysread() call will get the last 8196 bytes in the file, which you will happy copy through, the next call to sysread() will return 0, and you will terminate the loop, without emitting the trailing boundary delimiter. The MIME document will be corrupt.