Julia-Lang UndefVarError: loop_inner is not defined - multithreading

Question

im trying to use Multi-Threadding (or better Multi-Processor) in Julia-Lang. Just using Base.Threads just made my Application Slower so i wanted to try Distributed.
module Parallel
# ... includes ..
using Distributed
#Distributed.everywhere include("...jl")
#... Includes Needed in Proccesses
export loop_inner
#Distributed.everywhere function loop_inner(parentValue, value, i, depth)
...
end
function langfordSequence(parentValue, depth)
...
if depth < 4 && depth > 1
futures = [#spawnat :any loop_inner(parentValue, value, i, depth) for i = 0:possibilites]
return sum(fetch.(futures))
% 2) PROBLEMATIC LINE ^^
else
return sum([loop_inner(parentValue, value, i, depth) for i = 0:possibilites])
% 1) PROBLEMATIC LINE ^^
end
end
end
But i run into
$julia -L ...jl -L Parallel.jl main.jl -p 8
ERROR: LoadError: UndefVarError: loop_inner not defined (At Line [see Code Above at % 1) PROBLEMATIC LINE ^^ )
I hope someone can tell me what im doing wrong.
If im chaning if depth < 4 && depth > 1 to if depth < 4 im getting UndefVarError: Parallel not defined (At Line [see Code Above at % 2) PROBLEMATIC LINE ^^ )
Ty in advance

Answer 1

It does not work because each worker process needs to separately load your module.
Your module should look like this:
module MyParallel
include("somefile.jl")
export loop_inner
function loop_inner(parentValue, value, i, depth)
end
end
Now you use it in the following way (this assumes that the module is inside your private package):
using Distributed
using Pkg
Pkg.activate(".") # or wherever is the package
using MyParallel # enforces module compilation which should not occur in parallel
addprocs(4) # should always happen before any `#everywhere` macro or use `-p` command line option instead.
#everywhere using Pkg, Distributed
#everywhere Pkg.activate(".")
#everywhere using MyParallel
Now you are ready to work with the MyParallel module.
EDIT
To make it more clear. The important goal of a module is to provide a common namespace for a set of functions, types and global (module) variables. If you put any distributed code inside a module you obviously break this design because each Julia's worker is a totally separate system process, having it's own memory and namespace. Hence the good design, in my opinion, is to keep all the do-the-work code inside the module and distributed computation management outside the module. Perhaps in some scenarios one might want the distributed orchestration code to be in a second module - but normally it is just more convenient to keep such code outside of the module.