I think this is a newbie question but why it fails on the last assertion?
I was thinking that the closure binds the value on it, so changing it from the closure will change the value outside the closure.
def value = 5
def foo(n){
return {
++n
}
}
def test = foo(value)
assert test() == 6
assert test() == 7
assert value == 7
Thanks for help.
That does seem to be strange behaviour but I think it is correct. All objects event integegers are passed by reference. The call foo(value) passes value into the function. The variable 'n' is a reference to the same object as 'value' references. Essentially you have two pointers to the same object. When you increment 'n', you are incrementing that variable only.
Since the Integer class is immutable ++n is actually doing something like:
n = n + 1
This is assigning the incremented value to the variable n. The variable 'value' declared at the top is still pointing to the original object 5.
Remember that Integer (the runtime type of value) is immutable. So although n and value initially refer to the same object, when you execute ++n it creates a new Integer. The value reference is not updated to refer to this new object, so when execution completes it is still referring to the initial object (5).
You are assigning a new value to the name n, which is different than the name value. You can get the behaviour you want by making value a mutable object. Then you can change it instead of creating and assigning a new object.
Here's a simple example that wraps the value in a list:
def value = [5]
def foo(n){
return {
n[0]++
n
}
}
def test = foo(value)
assert test() == [6]
assert test() == [7]
assert value == [7]
Related
this code works well when I try to change dictionary in closure
def a():
x = {'a':'b'}
def b():
x['a'] = 'd'
print(x)
return b
>>> b = a()
>>> b()
{'a':'d'}
output is meeting my expectation. but why the code below doesn't work?
def m():
x = 1
def n():
x += 1
print(x)
return n
>>> n = m()
>>> n()
UnboundLocalError: local variable 'x' referenced before assignment
Honestly, I've known that we can use nonlocal x statement to solve this problem
But can anybody explain the reason more deeply for me? what the difference between a dictionary and a number
Thanks!
Python has a great FAQ specifically on this.
In short, when you modify a dictionary, or any mutable object, you modify the same object, so you don't re-assign the variable. In case of an integer, since it's immutable, by doing a += you create a new object and put it into x. Since x is now defined inside the inner function, but you're trying to bring data from the outer function, you have an issue.
You can check if it's the same object using id().
How does one capture a value (as opposed to a reference) in a python closure?
What I have tried:
Here is a function which makes a list of parameter-less functions, each of which spits out a string:
def makeListOfFuncs( strings):
list = []
for str in strings:
def echo():
return str
list.append( echo)
return list
If closures in python worked like every other language, I would expect that a call like this:
for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
print( animalFunc())
... would yield output like this:
bird
fish
zebra
But in python instead, we get:
zebra
zebra
zebra
Apparently what is happening is that the closure for the echo function is capturing the reference to str, as opposed to the value in the call frame at the time of closure construction.
How can I define makeListOfFuncs, so that I get 'bird', 'fish', 'zebra'?
I found the answer here on a blog post by Bob Kerner.
def makeListOfFuncs( strings):
list = []
for str in strings:
def generateFunc( str):
def echo():
return str
return echo
list.append( generateFunc( str))
return list
for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
print( animalFunc())
Python closures are weird.
Python closure is not weird if you know how closure internally works in python.
def makeListOfFuncs( strings):
list = []
for str in strings:
def echo():
return str
list.append( echo)
return list
You are returning a list of closures. variable "str" is shared between scopes, it is in two different scopes. When python sees it, it creates an intermediary object. this object contains a reference to another object which is "str". and for each closure, it is the same cell. You can test it:
closures_list= makeListOfFuncs( ['bird', 'fish', 'zebra'])
# Those will return same cell address
closures_list[0].__closure__
closures_list[1].__closure__
closures_list[2].__closure__
When you call makeListOfFuncs, it will run the for-loop and at the end of the loop, the intermediary object will point to "zebra". So when you call each closure print( animalFunc()) , it will visit the intermediary obj which will reference to "zebra".
You have to think about what happens when python creates a function and when it evaluates it.
def echo():
return str
We need to somehow able to capture the value of "str" as the function being created. Because if you wait until function gets evaluated, then it is too late. Another solution would be defining a default value:
def makeListOfFuncs( strings):
list = []
for str in strings:
def echo(y=str):
return y
list.append( echo)
return list
for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
print( animalFunc())
This solution works because default values get evaluated at creation time. So when echo is created, the default value of "str" will be used. the default value will not point to the cell.
in the first iteration, the default value will be "bird". Python will not see this as a free variable. in this case we are not even creating closure, we are creating function. in next iteration "fish" and in the last iteration "zebra" will be printed
This question already has answers here:
Are Python variables pointers? Or else, what are they?
(9 answers)
Closed 3 years ago.
When I use the function that doesn't return anything, the input parameters remain globally unchanged.
For example:
def func(var):
var += 1
a = 0
for i in range(3):
func(a)
print(a)
logically results in
0
0
0
By it doesn't seem to work the same when the input is numpy array:
import numpy as np
def func(var):
var += 1
a = np.zeros(3)
for i in range(3):
func(a)
print(a)
Output:
[1. 1. 1.]
[2. 2. 2.]
[3. 3. 3.]
Thus, all modifications were applied to array globally, not inside the function. Why is it happening? And, more generally, are there any special rules on how to handle np arrays as functions input?
In Python, any value passed to a function is passed by object reference. So, in the first case, where you pass a number to your function, var is set to a reference to the object that reresents the number 1. In Python, even numbers are objects. To icrement var in this case actually means to set it with a reference to the object that represents the number 1+1, which is the object that represents 2. Note that object that represents the number 1 is not changed. Within the function, it is replaced.
When you pass a numpy array to your function it is likewise passed in by object reference. Hence, var now holds a reference to your array a. Incrementing an array by arr += 1 means to add 1 to each of its elements. Hence, the actual content of the object that var references has to change. However, var still references the same object are incrementation.
Take a look at the following code:
import numpy as np
def func(vals):
print('Before inc: ', id(vals))
vals += 1
print('After inc: ', id(vals))
When you pass in a number literal, vals is set to a reference of the object representing the respective number. This object has a unique id, which you can return using the id function. After incrementation, vals is a reference to the object representing a number which is one greater the first one. You can verify that by calling id again after incrementation. So, the output of the above function is something like:
Before inc: 4351114480
After inc: 4351114512
Note that there are two different objects. When now pass in an numpy array like, the resulting ids are the same:
a = np.zeros(3)
func(a)
Before inc: 4496241408
After inc: 4496241408
If you want to modify an array inside of a function and don't want that the apply changes take effect outside of the function, you have to copy your array:
def func(vals):
_vals = vals.copy()
# doing stuff with `_vals` won't change the array passed to `vals`
+= for int (and float, str, ...) creates a new value. Such types are known as immutable, because each individual object cannot be changed.
>>> i = 1
>>> id(i)
4550541072
>>> i += 1
>>> id(i) # different id
4550541104
This means incrementing such a value inside a function creates a new value inside the function. Any references outside of the function are unaffected.
+= for np.array (and list, Counter, ...) modifies the content. Such types are known as mutable, because each individual object can be changed.
>>> l = [0, 1, 2, 3]
>>> id(l)
4585425088
>>> l += [4]
>>> id(l)
4585425088
This means incrementing such a value inside a function changes the value of the object visible inside and outside of the function. Any references inside and outside of the function point to the exact same object, and show its changed value.
I Have two maps:
def map = ['a': 3, 'b': 4, 'c':5]
def map2= ['a': 3, 'b': 4, 'c':4]
I want to take the maximum value of the map like this:
def newMap = map.max {it.value}
and my output is correct ('c':5), my problem is with the second map because there is more than one max value. On this case I want to change the key so I can know that there was more than one max value. I want my output to be:
def newMap2 = map2.max {it.value}
assert newMap2 == ['+than1': 4]
Can I alter the key of the map in this specific case using groovy functions?
Can I achieve this inside the max closure?
I do not want to alter the key if there isn't more than one max value.
Keep in mind that map.max(Closure cl) returns Map.Entry<String, Integer> and not a map. So if you expect a map with a single key, you will have to create one from the result you get.
Map.max(Closure cl) searches for the maximum value and returns it. There is no variant that allows you to modify function behavior in case of two entries holding maximum value. According to docs:
Selects an entry in the map having the maximum
calculated value as determined by the supplied closure.
If more than one entry has the maximum value,
an arbitrary choice is made between the entries having the maximum value.
In practice: the first entry found with maximum value is returned.
Groovy however offers a different collection function that can be used to achieve what you expect - Collection.inject(initialValue, closure) function. This is an equivalent of popular fold function known very well in functional programming paradigm. It starts with some initial value and it iterates a collection and applies a function to every element (this function returns a new value that replaces value passed as initial value) to reduce a list of elements to a single element in a single iteration. Collection.inject() javadoc gives a very descriptive example of summing all numbers from a list:
assert 0+1+2+3+4 == [1,2,3,4].inject(0) { acc, val -> acc + val }
Now let's take a look how we can use this function to achieve expected result. Consider following example:
def map2= ['a': 3, 'b': 4, 'c':4]
Tuple2<String, Integer> result = map2.inject(new Tuple2<String, Integer>(null, null)) { Tuple2<String, Integer> tuple, entry ->
entry.value >= tuple.second ?
new Tuple2<>(entry.value == tuple.second ? '+1than1' : entry.key, entry.value) :
tuple
}
assert result.first == '+1than1'
assert result.second == 4
Here we start with new Tuple2<String, Integer>(null, null) as an initial value - a Tuple2 represents a pair of two values. In our case it represents a key and the maximum value. The closure checks if current entry value is higher or equal the one we store in the tuple and if this is true it checks if the value is the same as the one we have already found - if this is true it uses +than1 as a key instead of a key taken from map. Otherwise it simply uses entry key without any modification. When the value is lower then the one we currently store in the tuple, existing tuple gets returned. And finally, we get a tuple where tuple.first holds a key and tuple.second holds a maximum value.
Of course to make the inject part more readable it is worth extracting a data class that represents your expected result and behavior. You can implement a function like compareAndReplace there to define specific behavior when another maximum value is found. Something like this:
import groovy.transform.Immutable
#Immutable
class Result {
static Result EMPTY = new Result(null, null)
String key
Integer value
Result compareAndReplace(Map.Entry<String, Integer> entry, String key = '+1than1') {
entry.value >= value ?
new Result(entry.value == value ? key : entry.key, entry.value) :
this
}
}
def map2 = ['a': 3, 'b': 4, 'c': 4]
Result result = map2.inject(Result.EMPTY) { Result result, entry -> result.compareAndReplace(entry) }
assert result.key == '+1than1'
assert result.value == 4
You can do it in a one-liner but it's still a little obscure:
println map2.groupBy{ it.value }.max{ it.key }.value.with{ it.size() > 1 ? ['+than1': it.values()[0]] : it }
I'd probably at least extract out the last Closure:
def mergedKey = { it.size() > 1 ? ['+than1': it.values()[0]] : it }
def newMap2 = map2.groupBy{ it.value }.max{ it.key }.value.with(mergedKey)
assert newMap2 == ['+than1': 4]
n = 3
d = {'x':n}
d['x'] += 1
print(n)
When I run it, I get
3
How do I make n = 4?
You can't do this, at least, not in any simple way.
The issue is very similar when you're just dealing with two variables bound to the same object. If you rebind one of them with an assignment, you will not see the new value through the other variable:
a = 3
b = a
a += 1 # binds a to a new integer, 4, since integers are immutable
print(b) # prints 3, not 4
One exception is if you are not binding a new value to the variable, but instead modifying a mutable object in-place. For instance, if instead of 1 you has a one-element list [1], you could replace the single value without creating a new list:
a = [3]
b = a
a[0] += 1 # doesn't rebind a, just mutates the list it points to
print(b[0]) # prints 4, since b still points to the same list as a
So, for your dictionary example you could take a similar approach and have n and your dictionary value be a list or other container object that you modify in-place.
Alternatively, you could store the variable name "n" in your dictionary and then rather than replacing it in your other code, you could use for a lookup in the globals dict:
n = 3
d = {"x": "n"} # note, the dictionary value is the string "n", not the variable n's value
globals()[d["x"]] += 1
print(n) # this actually does print 4, as you wanted
This is very awkward, of course, and only works when n is a global variable (you can't use the nominally equivalent call to locals in a function, as modifying the dictionary returned by locals doesn't change the local variables). I would not recommend this approach, but I wanted to show it can be done, if only badly.
You could use a class to contain the data values to enable additions. Basically you are creating a mutable object which acts as an integer.
It is a work around, but lets you accomplish what you want.
Note, that you probably need to override a few more Python operators to get full coverage:
class MyInt(object):
val = 0
def __init__(self,val):
self.val = val
def __iadd__(self,val):
self.val = self.val + val
def __repr__(self):
return repr(self.val)
n = MyInt(3)
print(n)
d = {'x':n}
d['x'] += 1
print(n)