This is a dumb question, so I apologise if so. This is for Julia, but I guess the question is not language specific.
There is advice in Julia that global variables should not be used in functions, but there is a case where I am not sure if a variable is global or local. I have a variable defined in a function, but is global for a nested function. For example, in the following,
a=2;
f(x)=a*x;
variable a is considered global. However, if we were to wrap this all in another function, would a still be considered global for f? For example,
function g(a)
f(x)=a*x;
end
We don't use a as an input for f, so it's global in that sense, but its still only defined in the scope of g, so is local in that sense. I am not sure. Thank you.
You can check directly that what #DNF commented indeed is the case (i.e. that the variable a is captured in a closure).
Here is the code:
julia> function g(a)
f(x)=a*x
end
g (generic function with 1 method)
julia> v = g(2)
(::var"#f#1"{Int64}) (generic function with 1 method)
julia> dump(v)
f (function of type var"#f#1"{Int64})
a: Int64 2
In this example your function g returns a function. I bind a v variable to the returned function to be able to inspect it.
If you dump the value bound to the v variable you can see that the a variable is stored in the closure.
A variable stored in a closure should not a problem for performance of your code. This is a typical pattern used e.g. when doing optimization of some function conditional on some parameter (captured in a closure).
As you can see in this code:
julia> #code_warntype v(10)
MethodInstance for (::var"#f#1"{Int64})(::Int64)
from (::var"#f#1")(x) in Main at REPL[1]:2
Arguments
#self#::var"#f#1"{Int64}
x::Int64
Body::Int64
1 ─ %1 = Core.getfield(#self#, :a)::Int64
│ %2 = (%1 * x)::Int64
└── return %2
everything is type stable so such code is fast.
There are some situations though in which boxing happens (they should be rare; they happen in cases when your function is so complex that the compiler is not able to prove that boxing is not needed; most of the time it happens if you assign value to the variable captured in a closure):
julia> function foo()
x::Int = 1
return bar() = (x = 1; x)
end
foo (generic function with 1 method)
julia> dump(foo())
bar (function of type var"#bar#6")
x: Core.Box
contents: Int64 1
julia> #code_warntype foo()()
MethodInstance for (::var"#bar#1")()
from (::var"#bar#1")() in Main at REPL[1]:3
Arguments
#self#::var"#bar#1"
Locals
x::Union{}
Body::Int64
1 ─ %1 = Core.getfield(#self#, :x)::Core.Box
│ %2 = Base.convert(Main.Int, 1)::Core.Const(1)
│ %3 = Core.typeassert(%2, Main.Int)::Core.Const(1)
│ Core.setfield!(%1, :contents, %3)
│ %5 = Core.getfield(#self#, :x)::Core.Box
│ %6 = Core.isdefined(%5, :contents)::Bool
└── goto #3 if not %6
2 ─ goto #4
3 ─ Core.NewvarNode(:(x))
└── x
4 ┄ %11 = Core.getfield(%5, :contents)::Any
│ %12 = Core.typeassert(%11, Main.Int)::Int64
└── return %12
Related
I have a computationally expensive vector I want to index into inside a function, but since the table is never used anywhere else, I don't want to pass the vector around, but access the precomputed values like a memoized function.
The idea is:
cachedFunction :: Int -> Int
cachedFunction ix = table ! ix
where table = <vector creation>
One aspect I've noticed is that all memoization examples I've seen deal with recursion, where even if a table is used to memoize, values in the table depend on other values in the table. This is not in my case, where computed values are found using a trial-and-error approach but each element is independent from another.
How do I achieve the cached table in the function?
You had it almost right. The problem is, your example basically scoped like this:
┌────────────────────────────────┐
cachedFunction ix = │ table ! ix │
│where table = <vector creation> │
└────────────────────────────────┘
i.e. table is not shared between different ix. This is regardless of the fact that it happens to not depend on ix (which is obvious in this example, but not in general). Therefore it would not be useful to keep it in memory, and Haskell doesn't do it.
But you can change that by pulling the ix argument into the result with its associated where-block:
cachedFunction = \ix -> table ! ix
where table = <vector creation>
i.e.
┌────────────────────────────────┐
cachedFunction = │ \ix -> table ! ix │
│where table = <vector creation> │
└────────────────────────────────┘
or shorter,
cachedFunction = (<vector creation> !)
In this form, cachedFunction is a constant applicative form, i.e. despite having a function type it is treated by the compiler as a constant value. It's not a value you could ever evaluate to normal form, but it will keep the same table around (which can't depend on ix; it doesn't have it in scope) when using it for evaluating the lambda function inside.
According to this answer, GHC will never recompute values declared at the top-level of a module. So by moving your table up to the top-level of your module, it will be evaluated lazily (once) the first time it's ever needed, and then it will never be requested again. We can see the behavior directly with Debug.Trace (example uses a simple integer rather than a vector, for simplicity)
import Debug.Trace
cachedFunction :: Int -> Int
cachedFunction ix = table + ix
table = traceShow "Computed" 0
main :: IO ()
main = do
print 0
print $ cachedFunction 1
print $ cachedFunction 2
Outputs:
0
"Computed"
1
2
We see that table is not computed until cachedFunction is called, and it's only computed once, even though we call cachedFunction twice.
Here's a psuedocode implementation of what I would be looking for within Julia:
struct Example
field1::Float64
field2::Float64
end # End struct
example = Example(1., 2.)
function modifystruct(mystruct, fieldname)
mystruct.fieldname +=10
return mystruct
end
modifystruct(example, field1)
# In this instance I would want to have example.field1 = 11.
How would I actually do this? I want to provide the fieldname as something like a string, and have my struct."whateverfieldname" get modified as such. I should add that I do NOT want to code something in like this:
function modifystruct(mystruct, fieldname)
if fieldname = "fieldname1"
mystruct.field1 +=10
end
if fieldname = "fieldname2"
mystruct.field2 +=10
end
return mystruct
end
Largely due to how versatile I want this code to be. I may be using different types of structs for my program, so the closest I can get to directly accessing by the name of the field, the better. Is there any method or implementation that can do this for me?
Sure, that's setproperty!(value, name, x) and getproperty(value, name):
function modifystruct(mystruct, fieldname)
new_field = getproperty(mystruct, fieldname) + 10
setproperty!(mystruct, fieldname, new_field)
return mystruct
end
As DecowVR rightly notes, this requires mystruct to be mutable.
If you want to do this repeatedly and with nested properties, you might be interested in lenses such as those provided by Setfield.jl.
Firstly, whould be noticed that in order to be able to modify an struct, it needs to be mutable:
julia> mutable struct Example
field1::Float64
field2::Float64
end
julia> example = Example(1., 2.)
Example(1.0, 2.0)
And now, a simple aproach would be to use Julia Symbols. A symbol is nothing else but an expression like :var. Can be used as shown:
julia> example.:field1
1.0
However, if we create a variable that stores the symbol, it won't work:
julia> v = :field1
:field1
julia> example.v
ERROR: type Example has no field v
Stacktrace:
[1] getproperty(x::Example, f::Symbol)
# Base ./Base.jl:42
[2] top-level scope
# REPL[18]:1
This is due to the order in which the Julia Interpreter works. If we want to evaluate firstly the variable, and then the expression, it is as easy as:
julia> #eval example.$v
1.0
So the complete function would be as follows:
julia> function modify_struct(mystruct::Example, fieldname::Symbol)
#eval $mystruct.$fieldname += 10
end
What is the correct way to store a function in a struct constructor? And retrieve it? And change its parameters?
I wrote some code back in version 0.4 to 0.5, which unsurprisingly no longer works. In words: I have a struct-cum-method named Model in which I define parameters and functional forms. And a similarly structured Solution in which I solve instances of the model. My purpose is to make multiple simulations of the model for different sets of parameters and functional forms. The code below is part of a module inside a package. I don't have a good handle on how to deal with the function type in Julia 1.6 (something like the code below used to work).
# Non-Parametric version of struct + outer method constructor
# Model parameters and functions
struct Model
f::Function
p::Float64
n::Int64
end
function Model(;
f::Function = x -> x + p,
p::Float64 = 2.0,
n::Int64 = 4
)
Model(f, p, n)
end
This is the output of Model():
julia> m = Model()
Model(var"#2#4"(), 2.0, 4)
julia> Model(p = 1.0)
Model(var"#3#5"(), 1.0, 4)
julia> m.f
#2 (generic function with 1 method)
julia> m.p
2.0
julia> m.n
4
julia> Model(f = x -> x - p)
Model(var"#15#16"(), 2.0, 4)
julia> m.f(1.0)
ERROR: UndefVarError: p not defined
Stacktrace:
[1] (::var"#2#4")(x::Float64)
# Main ./REPL[2]:2
[2] top-level scope
# REPL[4]:1
I'd be grateful for pointers on where the code goes wrong and how to fix it. If at all recommended, I'd like a parametric struct.
You need to use Base.#kwdef or an #with_kw equivalent from the Parameters package (I prefer the latter since it provides nicer formatting for the console)
Base.#kwdef struct Model
p::Float64 = 2.0
n::Int64 = 4
f::Function = x -> x + p
end
This now can be used such as:
julia> Model(p=7).f(11)
18
Regarding solution you should separate the actual solution structure from the constructor. Hence you will have a function solve(m::Model) (or solve! if m is mutated in the process) that should yields a Solution object.
For creating the Solution object use the same pattern as above.
I'm learning about scope in Python 3, and this example has me confused. Comparing behaviors of a list variable and a string variable when called inside a function:
foo1 = []
foo2 = ''
def f():
foo1.append(3)
global foo2
foo2 += 'c'
print('foo1-in-f:',foo1)
print('foo2-in-f:',foo2)
print('foo1-before:',foo1)
print('foo2-before:',foo2)
f()
print('foo1-after:',foo1)
print('foo2-after:',foo2)
The output, as expected, is:
foo1-before: []
foo2-before:
foo1-in-f: [3]
foo2-in-f: c
foo1-after: [3]
foo2-after: c
I'm confused why the string must be declared global, as in the line global foo2, but the list is not declared global, as in there in no line global foo1.
I ran the code omitting the line global foo2 and unsurprisingly got UnboundedLocalError: local variable 'foo2' referenced before assignment. But, why am I not getting this error for foo1?
Any insights appreciated. I want to make sure I'm understanding how this really works.
In a Python function, all variable references are assumed to be global unless named locally. All new objects are created in local scope and there are restrictions on transferring or modifying objects to another scope.
You can do:
a=1
def f(): return a+1 # unnamed integer object created and returned
>>> f()
2
You can modify the contents of a global mutable variable since this does not assign a locally named object to the global scope:
ls=['string']
def f():
ls.append('another') # unnamed string object created and added to ls
ls[0]+='_modified' # ls[0] read, new object created with +=,
# new object added to ls[0]
>>> f()
>>> ls
['string_modified', 'another']
But it would be an error to use ls+=[something] since the assignment += is treated as ls being local in scope then reassigned to the global scope:
ls=[]
def f():
ls+=['new entry'] # UnboundLocalError
The issue you are seeing is not necessarily modifying global variables. It is with reassigning the name that the function has used locally to the global scope.
There is a FAQ on this issue on the Python website.
There is an expanded FAQ on Eli Bendersky's blog.
You need to know how variable scopes work in Python. Python does not require you to declare variables, but assumes that a variable assigned in the body of a function is local. You can see this reflected by the compiler in the generated bytecode:
foo1 = []
foo2 = ''
def f():
foo1.append(3)
foo2 += 'c'
from dis import dis
dis(f)
4 0 LOAD_GLOBAL 0 (foo1)
2 LOAD_METHOD 1 (append)
4 LOAD_CONST 1 (3)
6 CALL_METHOD 1
8 POP_TOP
5 10 LOAD_FAST 0 (foo2)
12 LOAD_CONST 2 ('c')
14 INPLACE_ADD
16 STORE_FAST 0 (foo2)
18 LOAD_CONST 0 (None)
20 RETURN_VALUE
foo1 is loaded from the global context because foo1.append(3) is an item assignment operation - it modifies the actual reference. AFAIK Python strings are immutable which means that you need copy the value before doing the assignment, which is what you are doing inside the function because foo2 += 'c' will actually create a new string. Try running foo1 += [3] and you will get the same UnboundLocalError for foo1.
foo1.append(3) is an item assignment operation which is equivalent to foo1[len(foo1):] = [3]. This kind of operation is not possible for Python string for reasons stated above - try running foo2[:] = 'c' and you will get the error 'str' object does not support item assignment.
Now, the global keyword basically tells the interpreter to treat foo2 as a global variable in spite of the assignment within the function.
Suppose I have some immutable struct, e.g.
struct Person
name::Symbol
age::Int
end;
I want to write a function
function copyWithModification(original_person::Person, fieldToChange::String, valueForNewField)::Person
that returns a new Person struct just like the old one except that the value of the field specified in fieldToChange has been set to valueForNewField. How do I do this?
My current attempt uses Setfield and metaprogramming
using Setfield
function copyWithModification(original_person::Person, fieldToChange::String, valueForNewField)::Person
return eval(Meta.parse("#set original_person." * fieldToChange * " = " * string(valueForNewField)))
end
This does not work because the eval is performed in global scope and thus does not have access to the original_person object:
julia> struct Person
name::Symbol
age::Int
end;
julia> using Setfield
julia> function copyWithModification(original_person::Person, fieldToChange::String, valueForNewField)::Person
return eval(Meta.parse("#set original_person." * fieldToChange * " = " * string(valueForNewField)))
end
copyWithModification (generic function with 1 method)
julia> person_local_scope = Person(:test, 10)
Person(:test, 10)
julia> copyWithModification(person_local_scope, "age", 20)
ERROR: UndefVarError: original_person not defined
Stacktrace:
[1] top-level scope at /Users/lionstarr/.julia/packages/Setfield/XM37G/src/sugar.jl:182
[2] eval at ./boot.jl:330 [inlined]
[3] eval(::Expr) at ./client.jl:425
[4] copyWithModification(::Person, ::String, ::Int64) at ./REPL[3]:2
[5] top-level scope at REPL[5]:1
julia>
I should note I am not concerned with the performance of this code; it will only be called once or twice. The point is to save code replication and human error as the struct I actually want to use this code on is a lot larger.
If you are not concerned with performance, using plain introspection is fine and very simple in your case:
function copy_with_modification1(original::T, field_to_change, new_value) where {T}
val(field) = field==field_to_change ? new_value : getfield(original, field)
T(val.(fieldnames(T))...)
end
For example, it yields the following results:
julia> struct Person
name::Symbol
age::Int
end
julia> p = Person(:Joe, 42)
Person(:Joe, 42)
julia> using BenchmarkTools
julia> #btime copy_with_modification1($p, :age, 43)
666.924 ns (7 allocations: 272 bytes)
Person(:Joe, 43)
In order to regain efficiency, the same kind of technique can be implemented in such a way that listing fields happens at compile-time. Here is an example using a generated function:
# Can't have closures inside generated functions, so the helper function
# is declared outside
function val_(original, field, field_to_change, new_value)
field == field_to_change ? new_value : getfield(original, field)
end
#generated function copy_with_modification2(original, field_to_change, new_value)
# This is the "compile-time" part
T = original # here `original` refers to the type of the argument
fields = fieldnames(T) # fieldnames is called compile-time
# This is the "run-time" part
quote
# We broadcast only over `fields`, other arguments are treated as scalars
$T(val_.(Ref(original), $fields, Ref(field_to_change), Ref(new_value))...)
end
end
The performance is now much better:
julia> #btime copy_with_modification2($p, :age, 43)
2.533 ns (0 allocations: 0 bytes)
Person(:Joe, 43)
Such function is already defined in Setfield and there is no need to reinvent the wheel!
julia> using Setfield
julia> p = Person(:Smith, 10)
Person(:Smith, 10)
julia> setproperties(p, age=20)
Person(:Smith, 20)
More than one field can be set at a time, see ?setproperties for more details.
You don't need to use metaprogramming for this. I think this "normal" function manages to do what you need.
function Person(p :: Person,fieldtochange,newvalue)
newparams = [] # This array will store a new list of parameters
# This loop will iterate in all the fields (obtained via [fieldnames][1])
# of the struct Person and compare with the given field,
# if it coincides, adds the new value to the newparams array,
# if not, get the values of the original person using
# getproperty and add them to the array.
for currentfield in fieldnames(Person)
if currentfield == fieldtochange
push!(newparams,newvalue)
else
push!(newparams,getproperty(p,currentfield)) #[2]
end
end
return Person(newparams...) #Construct a new person with the new parameters
# using '...' for [splatting][3].
end
In this case I named the function "Person" to make it another constructor but you can change the name to the one you wanted.
1 https://docs.julialang.org/en/v1/base/base/#Base.fieldnames
[2] https://docs.julialang.org/en/v1/base/base/#Base.getproperty
[3] https://docs.julialang.org/en/v1/base/base/#...