In Julia, how to create a macro consisting of several optional macros? - multithreading

Question

In Julia, I am trying out different parallelization libraries, to make my program more performant, and to check if memory consumption is the same as with no parallelization. The unfortunate effect of this is a lot of duplication.
Is there a way to organize my code so that I write the algorithm only once and then some macro with a parameter decides how the code is parallelized? My question is similar to this one. For example, my MWE
using ThreadsX, Folds, FLoops, Polyester
create_data = (n,s) -> [rand(1:n,r) for j=1:n for r∈[rand(1:s)]]
function F!(method ::Int, L ::Vector{Vector{Int}}) ::Nothing
n = length(L)
if method==0 for j=1:n sort!(L[j]) end end
if method==1 Threads.#threads for j=1:n sort!(L[j]) end end
if method==2 ThreadsX.foreach(1:n) do j sort!(L[j]) end end
if method==3 Folds.foreach(1:n) do j sort!(L[j]) end end
if method==4 FLoops.#floop for j=1:n sort!(L[j]) end end
if method==5 Polyester.#batch for j=1:n sort!(L[j]) end end
return nothing end
for mtd=0:5
L = create_data(10^6,10^3);
#time F!(mtd,L) end
returns
17.967120 seconds
4.537954 seconds (38 allocations: 3.219 KiB)
4.418978 seconds (353 allocations: 27.875 KiB)
5.583201 seconds (54 allocations: 3.875 KiB)
5.542852 seconds (53 allocations: 3.844 KiB)
4.263488 seconds (3 allocations: 80 bytes)
so there are different performances already for a very simple problem.
In my actual case, instead of sort!(L[j]) I have lots of intensive code with several Arrays, Vector{Vector}s, Dicts, ..., where different threads read from occasionally the same place, but write to different places, allocate space in memory, mutate the input, etc. Is there a way to create a new macro #Parallel so that my code would be just
function F!(method ::Int, L ::Vector{Vector{Int}}) ::Nothing
n = length(L)
#Parallel(method) for j=1:n sort!(L[j]) end
return nothing end
Note that I have never created a macro, I only used them thus far, so some explanation would be welcome.

Answer 1

A macro-based solution is possible, but seems unnecessary to me. I'd rather organize code like this:
function testfun!(x::Vector{Int})
# here goes the repetitive part
return sort!(x)
end
# entry point for dispatch, and set up data
function F(kernel!, M, N, strategy::Symbol)
L = create_data(M, N)
return F!(kernel!, L, Val{strategy}())
end
# and a function to run the loop for every strategy
function F!(kernel!, L, ::Val{:baseline})
n = length(L)
for j=1:n kernel!(L[j]) end
end
function F!(kernel!, L, ::Val{:Threads})
n = length(L)
Threads.#threads for j=1:n kernel!(L[j]) end
end
function F!(kernel!, L, ::Val{:ThreadsX})
n = length(L)
ThreadsX.foreach(1:n) do j kernel!(L[j]) end
end
# ...
# + rest of the loop functions for Floops, Folds, etc.
function dotests()
for strategy = (:baseline, :Threads, :ThreadsX, ...)
#benchmark F(testfun!, 10^6, 10^3, strategy)
end
end
This is showing a dispatch-based approach. You could equally well use dictionaries or conditions. The important point is separating the "runner" F! from the "kernel" function.