Crystal recursive function resulting in a signal 11 (invalid memory access)? - tail-recursion

I wanted to test recursive functions in Crystal, so I wrote something like...
def repeat(n)
return if n < 0
# Useless line just to do something
n + 1
repeat(n - 1)
end
repeat(100_000_000)
I didn't expect this to work with either crystal recurse.cr or crystal build recurse.cr, because neither one of those can optimize for tail call recursion - as expected, both resulted in stack overflows.
When I used the --release flag, it was totally fine - and incredibly fast.
If I did the following...
NUM = 100_000
def p(n)
puts n if n % (NUM / 4) == 0
end
def recursive_repeat(n)
return if n < 0
p(n)
recursive_repeat(n-1)
end
recursive_repeat(NUM)
... everything is fine - I don't even need to build it.
If I change to a much larger number and a non-recursive function, again it's fine...
NUM = 100_000_000
def p(n)
puts n if n % (NUM / 4) == 0
end
def repeat(num)
(0..num).each { |n| p(n) }
end
repeat(NUM)
If I instead use recursion, however...
NUM = 100_000
def p(n)
puts n if n % (NUM / 4) == 0
end
def recursive_repeat(n)
return if n < 0
p(n)
recursive_repeat(n-1)
end
recursive_repeat(NUM)
...I get the following output:
100000000
Invalid memory access (signal 11) at address 0x7fff500edff8
[4549929098] *CallStack::print_backtrace:Int32 +42
[4549928595] __crystal_sigfault_handler +35
[140735641769258] _sigtramp +26
[4549929024] *recursive_repeat<Int32>:Nil +416
[4549929028] *recursive_repeat<Int32>:Nil +420 (65519 times)
[4549926888] main +2808
How does using puts like that trigger a stack overflow - especially given that it would only write 5 lines total?
Shouldn't this still be TCO?
Edit
Okay, to add to the weirdness, this works...
NUM = 100_000_000
def p(n)
puts n if n % (NUM / 4) == 0
end
def repeat(num)
(0..num).each { |n| p(n) }
end
repeat(NUM)
def recursive_repeat(n)
return if n < 0
p(n)
recursive_repeat(n-1)
end
recursive_repeat(NUM)
... but removing the call to repeat(NUM), literally keeping every other line the same, will again result in the error.

Related

Loop optimization in QB64

Have a loop in QB64 concerning loop optimization:
DIM N AS DOUBLE, X(100000000) AS DOUBLE
T! = TIMER
FOR N = 1 to 100000000
IF X(N) THEN
PRINT X(N)
EXIT FOR
END IF
NEXT
PRINT TIMER - T!
is it any faster than:
DIM N AS DOUBLE, X(100000000) AS DOUBLE
T! = TIMER
FOR N = 1 to 100000000
IF X(N) <> 0 THEN
PRINT X(N)
EXIT FOR
END IF
NEXT
PRINT TIMER - T!
EDITED: 09-18-2018 to include variable types
I written this code to evaluate your test:
REM Delete REM to enable console runs
REM $CONSOLE:ONLY
REM _DEST _CONSOLE
DIM SHARED N AS DOUBLE, X(100000000) AS DOUBLE
S# = 0: ZC% = 0
T% = 10
IF COMMAND$ <> "" THEN
T% = VAL(COMMAND$)
END IF
IF T% > 999 THEN T% = 999
FOR I% = 1 TO T%
A# = TRYA
B# = TRYB
D# = A# - B#
PRINT USING "Case A ... : #.########"; A#
PRINT USING "Case B ... : #.########"; B#
PRINT USING "Diff ..... : #.########"; D#;
A$ = ""
IF ABS(D#) < 0.00000001 THEN
ZC% = ZC% + 1
A$ = "*"
END IF
S# = S# + A# - B#
PRINT A$
PRINT
REM INKEY$ doesn't work in console mode!
A$ = INKEY$
IF A$ = CHR$(27) THEN
I% = I% + 1: EXIT FOR
END IF
NEXT
PRINT USING "Avrg A - B : #.########"; S# / (I% - 1)
PRINT USING "0 diff:### on ### tryes"; ZC%, (I% - 1)
PRINT
PRINT "Hit a key to exit!"
REM INPUT$ doesn't work in console mode!
A$ = INPUT$(1)
SYSTEM
FUNCTION TRYA#
T# = TIMER
FOR N = 1 TO 100000000
IF X(N) THEN
PRINT X(N)
EXIT FOR
END IF
NEXT
A# = TIMER - T#
TRYA = A#
END FUNCTION
FUNCTION TRYB#
T# = TIMER
FOR N = 1 TO 100000000
IF X(N) <> 0 THEN
PRINT X(N)
EXIT FOR
END IF
NEXT
A# = TIMER - T#
TRYB = A#
END FUNCTION
The two different routines are inserted into two functions: TRYA and TRYB.
I launched this SW with a loop that runs 999 times the functions and the result is:
Avrg. A - B: 0.00204501
0 diff:359 on 999 tryes
then I launched with a 10 times loop and the result is:
Avrg. A - B: -.01640625
0 diff: 1 on 10 tryes
then I launched with a 15 times loop and the result is:
Avrg. A - B: 0.00026042
0 diff: 5 on 15 tryes
Cause we launch the SW in a multi-thread ambient I don't believe this is a very good test, but there's some results:
In two cases the results of no difference (0 diff) is a third of all loops.
In two cases it seems the function TRYA is slower.
In one case it seems the function TRYB is slower.
Looking at these results, I think, we may consider the two functions equivalent!
You obtain more than 10 loops running the code from command line (or modifying the command$ parameter into the QB64 menu) as:
# ./test n
Where n is the number of loops you desire.
The SW was compiled using gcc with -O3 optimizations option. (To do this you have to modify the file [/opt/]qb64/internal/c/makeline_lnx.txt)

issue with printing in haskell and function declaration

this is my code:
sumDig i = if (i == 0) then 0 else ((mod i 10) + somaDig ((i-(mod i 10)/10)))
main = do
print (sumDig 4)
it's not working and I dont know why... it doesn't compile and the compiler message is not helping
the function is just to sum all characters of a given number, eg: 123 = 1 + 2 + 3 = 6, and it does that using n mod 10 + recursion from n/10
You've got a few errors.
The error about ambiguous types can be fixed by adding a type annotation to sumDig
sumDig :: Int -> Int
Another error is that somaDig is undefined. Did you mean to type sumDig?
If so, then the last compile error is the use of / on integers. You probably want div instead:
sumDig :: Int -> Int
sumDig i = if (i == 0) then 0 else ((mod i 10) + sumDig (((i-(mod i 10)) `div` 10)))

groovy fibonacchi trampoline and memoize

I am trying to evaluate fibonacchi sequence for 10000000.
Using basic trampoline it looks like this
def rFibonacchi
rFibonacchi = {
BigInteger n, prev = 0, next = 1 ->
(n < 2) ? prev : rFibonacchi.trampoline(n - 1, prev, next + prev)
}.trampoline()
But using trampoline & memoize combo I am constantly getting OutOfMemoryError.
def tFibonacchi, mFibonacchi
mFibonacchi = { BigInteger n, prev, next ->
n < 2 ? prev : tFibonacchi.trampoline(n - 1, next, prev + next)
}.memoize()
tFibonacchi = { BigInteger n, prev = 0, next = 1 ->
mFibonacchi(n, next, prev)
}.trampoline()
tFibonacchi(10000000); // GC overhead limit exceed
Is it my algorithm's issue?
Your algorithm doesn't get any bonus from using memoization. To quote the groovy docs on memoization:
Memoization allows the result of the call of a closure to be cached. It is interesting if the computation done by a function (closure) is slow, but you know that this function is going to be called often with the same arguments. A typical example is the Fibonacci suite. A naive implementation may look like this:
def fib
fib = { long n -> n<2?n:fib(n-1)+fib(n-2) }
assert fib(15) == 610 // slow!
It is a naive implementation because 'fib' is often called recursively with the same arguments, leading to an exponential algorithm:
computing fib(15) requires the result of fib(14) and fib(13)
computing fib(14) requires the result of fib(13) and fib(12)
Since calls are recursive, you can already see that we will compute the same values again and again, although they could be cached. This naive implementation can be "fixed" by caching the result of calls using memoize:
fib = { long n -> n<2?n:fib(n-1)+fib(n-2) }.memoize()
assert fib(25) == 75025 // fast!
The cache works using the actual values of the arguments.
You are using an improved Fibonacci algorithm to the above one. Yours is more iterative and it never calls mFibonacchi twice with all of the same arguments. This causes groovy to cache the result of each call, but never actually use this cache, which leads to the memory overflow. The memoization is actually an issue.
Your algorithm is equivalent to:
BigInteger fibonacchi(BigInteger n) {
BigInteger prev = 0, next = 1
for (; n > 2; n--) {
BigInteger temp = prev
prev = next
next = prev + temp
}
return prev
}

Tail recursion with Groovy

I coded 3 factorial algorithms:
I expect to fail by stack overflow. No problem.
I try a tail recursive call, and convert the previous algorithm from recursive to iterative. It doesn't work, but I don't understand why.
I use trampoline() method and it works fine as I expect.
def factorial
factorial = { BigInteger n ->
if (n == 1) return 1
n * factorial(n - 1)
}
factorial(1000) // stack overflow
factorial = { Integer n, BigInteger acc = 1 ->
if (n == 1) return acc
factorial(n - 1, n * acc)
}
factorial(1000) // stack overflow, why?
factorial = { Integer n, BigInteger acc = 1 ->
if (n == 1) return acc
factorial.trampoline(n - 1, n * acc)
}.trampoline()
factorial(1000) // It works.
There is no tail recursion in Java, and hence there is none in Groovy either (without using something like trampoline() as you have shown)
The closest I have seen to this, is an AST transformation which cleverly wraps the return recursion into a while loop
Edit
You're right, Java (and hence Groovy) do support this sort of tail-call iteration, however, it doesn't seem to work with Closures...
This code however (using a method rather than a closure for the fact call):
public class Test {
BigInteger fact( Integer a, BigInteger acc = 1 ) {
if( a == 1 ) return acc
fact( a - 1, a * acc )
}
static main( args ) {
def t = new Test()
println "${t.fact( 1000 )}"
}
}
When saved as Test.groovy and executed with groovy Test.groovy runs, and prints the answer:
402387260077093773543702433923003985719374864210714632543799910429938512398629020592044208486969404800479988610197196058631666872994808558901323829669944590997424504087073759918823627727188732519779505950995276120874975462497043601418278094646496291056393887437886487337119181045825783647849977012476632889835955735432513185323958463075557409114262417474349347553428646576611667797396668820291207379143853719588249808126867838374559731746136085379534524221586593201928090878297308431392844403281231558611036976801357304216168747609675871348312025478589320767169132448426236131412508780208000261683151027341827977704784635868170164365024153691398281264810213092761244896359928705114964975419909342221566832572080821333186116811553615836546984046708975602900950537616475847728421889679646244945160765353408198901385442487984959953319101723355556602139450399736280750137837615307127761926849034352625200015888535147331611702103968175921510907788019393178114194545257223865541461062892187960223838971476088506276862967146674697562911234082439208160153780889893964518263243671616762179168909779911903754031274622289988005195444414282012187361745992642956581746628302955570299024324153181617210465832036786906117260158783520751516284225540265170483304226143974286933061690897968482590125458327168226458066526769958652682272807075781391858178889652208164348344825993266043367660176999612831860788386150279465955131156552036093988180612138558600301435694527224206344631797460594682573103790084024432438465657245014402821885252470935190620929023136493273497565513958720559654228749774011413346962715422845862377387538230483865688976461927383814900140767310446640259899490222221765904339901886018566526485061799702356193897017860040811889729918311021171229845901641921068884387121855646124960798722908519296819372388642614839657382291123125024186649353143970137428531926649875337218940694281434118520158014123344828015051399694290153483077644569099073152433278288269864602789864321139083506217095002597389863554277196742822248757586765752344220207573630569498825087968928162753848863396909959826280956121450994871701244516461260379029309120889086942028510640182154399457156805941872748998094254742173582401063677404595741785160829230135358081840096996372524230560855903700624271243416909004153690105933983835777939410970027753472000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
As a guess, I would say that the JVM does not know how to optimise closures (like it does with methods), so this tail call does not get optimised out in the bytecode before it is executed
Starting with version 2.3 Groovy supports tail recursion with the #TailRecursive annotation for methods:
http://java.dzone.com/articles/groovy-goodness-more-efficient

How to highlight the differences between subsequent lines in a file?

I do a lot of urgent analysis of large logfile analysis. Often this will require tailing a log and looking for changes.
I'm keen to have a solution that will highlight these changes to make it easier for the eye to track.
I have investigated tools and there doesn't appear to be anything out there that does what I am looking for. I've written some scripts in Perl that do it roughly, but I would like a more complete solution.
Can anyone recommend a tool for this?
Levenshtein distance
Wikipedia:
Levenshtein distance between two strings is minimum number of operations needed to transform one string into the other, where an operation is an insertion, deletion, or substitution of a single character.
public static int LevenshteinDistance(char[] s1, char[] s2) {
int s1p = s1.length, s2p = s2.length;
int[][] num = new int[s1p + 1][s2p + 1];
// fill arrays
for (int i = 0; i <= s1p; i++)
num[i][0] = i;
for (int i = 0; i <= s2p; i++)
num[0][i] = i;
for (int i = 1; i <= s1p; i++)
for (int j = 1; j <= s2p; j++)
num[i][j] = Math.min(Math.min(num[i - 1][j] + 1,
num[i][j - 1] + 1), num[i - 1][j - 1]
+ (s1[i - 1] == s2[j - 1] ? 0 : 1));
return num[s1p][s2p];
}
Sample App in Java
String Diff
Application uses LCS algorithm to concatenate 2 text inputs into 1. Result will contain minimal set of instructions to make one string for the other. Below the instruction concatenated text is displayed.
Download application:
String Diff.jar
Download source:
Diff.java
I wrote a Python script for this purpose that utilizes difflib.SequenceMatcher:
#!/usr/bin/python3
from difflib import SequenceMatcher
from itertools import tee
from sys import stdin
def pairwise(iterable):
"""s -> (s0,s1), (s1,s2), (s2, s3), ...
https://docs.python.org/3/library/itertools.html#itertools-recipes
"""
a, b = tee(iterable)
next(b, None)
return zip(a, b)
def color(c, s):
"""Wrap string s in color c.
Based on http://stackoverflow.com/a/287944/1916449
"""
try:
lookup = {'r':'\033[91m', 'g':'\033[92m', 'b':'\033[1m'}
return lookup[c] + str(s) + '\033[0m'
except KeyError:
return s
def diff(a, b):
"""Returns a list of paired and colored differences between a and b."""
for tag, i, j, k, l in SequenceMatcher(None, a, b).get_opcodes():
if tag == 'equal': yield 2 * [color('w', a[i:j])]
if tag in ('delete', 'replace'): yield color('r', a[i:j]), ''
if tag in ('insert', 'replace'): yield '', color('g', b[k:l])
if __name__ == '__main__':
for a, b in pairwise(stdin):
print(*map(''.join, zip(*diff(a, b))), sep='')
Example input.txt:
108 finished /tmp/ts-out.5KS8bq 0 435.63/429.00/6.29 ./eval.exe -z 30
107 finished /tmp/ts-out.z0tKmX 0 456.10/448.36/7.26 ./eval.exe -z 30
110 finished /tmp/ts-out.wrYCrk 0 0.00/0.00/0.00 tail -n 1
111 finished /tmp/ts-out.HALY18 0 460.65/456.02/4.47 ./eval.exe -z 30
112 finished /tmp/ts-out.6hdkH5 0 292.26/272.98/19.12 ./eval.exe -z 1000
113 finished /tmp/ts-out.eFBgoG 0 837.49/825.82/11.34 ./eval.exe -z 10
Output of cat input.txt | ./linediff.py:
http://neil.fraser.name/software/diff_match_patch/svn/trunk/demos/demo_diff.html
.. this look promising, will update this with more info when Ive played more..

Resources