Read numbers from file until line < n - linux

Question

As shocked as I am, I can't find this anywhere, and my bash skills are still sub-par.
I have a text file of prime numbers:
2\n
3\n
5\n
7\n
11\n
etc...
I want to pull all primes under 2^32 (4294967296) plus one additional prime number, and save these primes to the own text file formatted the same way. Also, my file has just over 1.3 billion lines so far, so stopping after the limit would be ideal.
Update: Problem.
The bash script has been looping through these 11 numbers for quite some time without me noticing:
4232004449
4232004479
4232004493
4232004509
4232004527
4232004533
4232004559
4232004589
4232004593
4232004613
004437
What's even weirder is I grepped primes.txt (the original) and "^004437" was nowhere to be found. Is this some kind of limitation of bash?
Update: Solution
It appears to be some kind of limitation of something, I really don't know what. I'm re-chosing the perl script as my answer because not only did it work, but it created the ~2GB from nothing in ~80 seconds and included the additional prime. Go here for a solution to the bash error.

Answer 1

$ perl -lne 'print; last if $_ > 2**32' < myprimes.txt > myprimes2.txt
Gives you the input series of primes up to one prime past 2**32, then stops. Does not read source file into memory.

Answer 2

In shell, without loading the whole 1.3 billion numbers into memory, you can use:
n=4294967296
last=0
while read number
do
if [ $last -gt $n ]
then break
fi
echo $number
last=$number
done < primes.txt > primes2.txt
You could lose the last variable too:
n=4294967296
while read number
do
echo $number
if [ $number -gt $n ]
then break
fi
done < primes.txt > primes2.txt

Answer 3

This is very easy to do in Bash! Just cat the file primes.txt to read it, go through each number, check that the number is less than 2^32, and if it is, append it to primes2.txt.
The exact code is below.
#!/bin/bash
n=4294967296; # 2^32
for i in `cat primes.txt`
do
if [ $i -le $n ]
then
echo $i >> primes2.txt;
fi
done
Or you can use this simple Python solution, which does not require loading the entire file into memory.
new_primes = open('primes2.txt', 'a')
n = 2**32
[new_primes.write(p) for p in open('primes.txt', 'r') if int(p) < n]

Answer 4

I would recommend doing something like this in Perl:
EDIT: Hm, it was probably the array that used up all your RAM - this should be more friendly to your resources.
#!/usr/bin/env perl
use warnings;
use strict;
my $max_value = ( 2 ** 32);
my $input_file = 'primes.txt';
my $output_file = 'primes2.txt';
open( my $INPUT_FH, '<', $input_file )
or die "could not open file: $!";
open ( my $OUTPUT_FH, '>', $output_file )
or die "could not open file: $!";
foreach my $prime ( <$INPUT_FH> ) {
chomp($prime);
unless ( $prime >= $max_value ) { print $OUTPUT_FH "$prime","\n"; }
}