How to interpolate into a Julia "for" expression? - metaprogramming

Question

I was writing a macro #vcomp (vector comprehension) based on Python's list comprehensions with a conditional clause to filter elements in a succinct way.
macro vcomp(comprehension::Expr, when::Symbol, condition)
comp_head, comp_args = comprehension.head, comprehension.args
comp_head ∉ [:comprehension, :typed_comprehension] && error("#vcomp not a comprehension")
when ≠ :when && error("#vcomp expected `when`, got: `$when`")
T = comp_head == :typed_comprehension ? comp_args[1] : nothing
if VERSION < v"0.5-"
element = comp_head == :comprehension ? comp_args[1] : comp_args[2]
sequence = comp_head == :comprehension ? comp_args[2] : comp_args[3]
else
element = comp_head == :comprehension ? comp_args[1].args[1] : comp_args[2].args[1]
sequence = comp_head == :comprehension ? comp_args[1].args[2] : comp_args[2].args[2]
end
result = T ≠ nothing ? :($T[]) : :([])
block = Expr(:let, Expr(:block,
Expr(:(=), :res, result),
Expr(:for, sequence,
Expr(:if, condition,
Expr(:call, :push!, :res, element))),
:res))
return esc(block)
end
Used like this:
julia> #vcomp Int[i^3 for i in 1:10] when i % 2 == 0
5-element Array{Int64,1}:
8
64
216
512
1000
Which expand to this:
julia> macroexpand(:(#vcomp Int[i^3 for i in 1:15] when i % 2 == 0))
:(let
res = Int[]
for i = 1:15
if i % 2 == 0
push!(res,i ^ 3)
end
end
res
end)
I was expecting to be able to write block like this:
block = quote
let
res = $result
for $sequence
if $condition
push!(res, $element)
end
end
res
end
end
Which gives the following error:
ERROR: syntax: invalid iteration specification
Instead of the way I came up with:
block = Expr(:let, Expr(:block,
Expr(:(=), :res, result),
Expr(:for, sequence,
Expr(:if, condition,
Expr(:call, :push!, :res, element))),
:res))
However I was able to do it using Expr(:for, ...) directly as shown above and as far as I understand this is a parser error (is this a bug?). I have also been unable to find examples of this kind of interpolation, this is what I've tried:
julia> ex₁ = :(i in 1:10)
:($(Expr(:in, :i, :(1:10))))
julia> ex₂ = :(i = 1:10)
:(i = 1:10)
julia> quote
for $ex₁
ERROR: syntax: invalid iteration specification
julia> quote
for $ex₂
ERROR: syntax: invalid iteration specification
Construct whole expression and inspect it:
julia> ex₃ = quote
for i in 1:10
print(i)
end
end
quote # none, line 2:
for i = 1:10 # none, line 3:
print(i)
end
end
julia> ex₃.args
2-element Array{Any,1}:
:( # none, line 2:)
:(for i = 1:10 # none, line 3:
print(i)
end)
julia> ex₃.args[2].args
2-element Array{Any,1}:
:(i = 1:10)
quote # none, line 3:
print(i)
end
julia> ex₃.args[2].args[1]
:(i = 1:10)
julia> ex₃.args[2].args[1] == ex₂ # what's the difference then?
true
This works but is less readable:
julia> ex₄ = Expr(:for, ex₁, :(print(i)))
:(for $(Expr(:in, :i, :(1:10)))
print(i)
end)
julia> ex₅ = Expr(:for, ex₂, :(print(i)))
:(for i = 1:10
print(i)
end)
julia> eval(ex₃)
12345678910
julia> eval(ex₄)
12345678910
julia> eval(ex₅)
12345678910
Is there a way I can use the more terse syntax instead? I find the current implementation difficult to read and reason about compared to what I was expecting to write.

Answer 1

First of all, I belive that comprehensions with guards are coming to Julia (in v0.5?).
To answer your question: The parser wants to be able to verify that its input is syntactically correct without looking into the actual value that is interpolated. Try eg
x, y = :i, :(1:10)
quote
for $x = $y
end
end
Now the parser can recognize the relevant parts of the syntax. (And you should get the same AST if you use for $x in $y instead.)