When a GString will change its toString representation - groovy

I am reading the Groovy closure documentation in https://groovy-lang.org/closures.html#this. Having a question regarding with GString behavior.
Closures in GStrings
The document mentioned the following:
Take the following code:
def x = 1
def gs = "x = ${x}"
assert gs == 'x = 1'
The code behaves as you would expect, but what happens if you add:
x = 2
assert gs == 'x = 2'
You will see that the assert fails! There are two reasons for this:
a GString only evaluates lazily the toString representation of values
the syntax ${x} in a GString does not represent a closure but an expression to $x, evaluated when the GString is created.
In our example, the GString is created with an expression referencing x. When the GString is created, the value of x is 1, so the GString is created with a value of 1. When the assert is triggered, the GString is evaluated and 1 is converted to a String using toString. When we change x to 2, we did change the value of x, but it is a different object, and the GString still references the old one.
A GString will only change its toString representation if the values it references are mutating. If the references change, nothing will happen.
My question is regarding the above-quoted explanation, in the example code, 1 is obviously a value, not a reference type, then if this statement is true, it should update to 2 in the GString right?
The next example listed below I feel also a bit confusing for me (the last part)
why if we mutate Sam to change his name to Lucy, this time the GString is correctly mutated??
I am expecting it won't mutate?? why the behavior is so different in the two examples?
class Person {
String name
String toString() { name }
}
def sam = new Person(name:'Sam')
def lucy = new Person(name:'Lucy')
def p = sam
def gs = "Name: ${p}"
assert gs == 'Name: Sam'
p = Lucy. //if we change p to Lucy
assert gs == 'Name: Sam' // the string still evaluates to Sam because it was the value of p when the GString was created
/* I would expect below to be 'Name: Sam' as well
* if previous example is true. According to the
* explanation mentioned previously.
*/
sam.name = 'Lucy' // so if we mutate Sam to change his name to Lucy
assert gs == 'Name: Lucy' // this time the GString is correctly mutated
Why the comment says 'this time the GString is correctly mutated? In previous comments it just metioned
the string still evaluates to Sam because it was the value of p when the GString was created, the value of p is 'Sam' when the String was created
thus I think it should not change here??
Thanks for kind help.

These two examples explain two different use cases. In the first example, the expression "x = ${x}" creates a GString object that internally stores strings = ['x = '] and values = [1]. You can check internals of this particular GString with println gs.dump():
<org.codehaus.groovy.runtime.GStringImpl#6aa798b strings=[x = , ] values=[1]>
Both objects, a String one in the strings array, and an Integer one in the values array are immutable. (Values are immutable, not arrays.) When the x variable is assigned to a new value, it creates a new object in the memory that is not associated with the 1 stored in the GString.values array. x = 2 is not a mutation. This is new object creation. This is not a Groovy specific thing, this is how Java works. You can try the following pure Java example to see how it works:
List<Integer> list = new ArrayList<>();
Integer number = 2;
list.add(number);
number = 4;
System.out.println(list); // prints: [2]
The use case with a Person class is different. Here you can see how mutation of an object works. When you change sam.name to Lucy, you mutate an internal stage of an object stored in the GString.values array. If you, instead, create a new object and assigned it to sam variable (e.g. sam = new Person(name:"Adam")), it would not affect internals of the existing GString object. The object that was stored internally in the GString did not mutate. The variable sam in this case just refers to a different object in the memory. When you do sam.name = "Lucy", you mutate the object in the memory, thus GString (which uses a reference to the same object) sees this change. It is similar to the following plain Java use case:
List<List<Integer>> list2 = new ArrayList<>();
List<Integer> nested = new ArrayList<>();
nested.add(1);
list2.add(nested);
System.out.println(list2); // prints: [[1]]
nested.add(3);
System.out.println(list2); // prints: [[1,3]]
nested = new ArrayList<>();
System.out.println(list2); // prints: [[1,3]]
You can see that list2 stores the reference to the object in the memory represented by nested variable at the time when nested was added to list2. When you mutated nested list by adding new numbers to it, those changes are reflected in list2, because you mutate an object in the memory that list2 has access to. But when you override nested with a new list, you create a new object, and list2 has no connection with this new object in the memory. You could add integers to this new nested list and list2 won't be affected - it stores a reference to a different object in the memory. (The object that previously could be referred to using nested variable, but this reference was overridden later in the code with a new object.)
GString in this case behaves similarly to the examples with lists I shown you above. If you mutate the state of the interpolated object (e.g. sam.name, or adding integers to nested list), this change is reflected in the GString.toString() that produces a string when the method is called. (The string that is created uses the current state of values stored in the values internal array.) On the other hand, if you override a variable with a new object (e.g. x = 2, sam = new Person(name:"Adam"), or nested = new ArrayList()), it won't change what GString.toString() method produces, because it still uses an object (or objects) that is stored in the memory, and that was previously associated with the variable name you assigned to a new object.

That's almost the whole story, as you can use a Closure for your GString evaluation, so in place of just using the variable:
def gs = "x = ${x}"
You can use a closure that returns the variable:
def gs = "x = ${-> x}"
This means that the value x is evaluated at the time the GString is changed to a String, so this then works (from the original question)
def x = 1
def gs = "x = ${-> x}"
assert gs == 'x = 1'
x = 2
assert gs == 'x = 2'

Related

Groovy from 2nd dimentional array to 1 dimentional as string with prefix

I have an 2 dimentional array:
def test = [[88,3,2],[22,33,4],[88,3,3]]
test.sort
what i need now is to create each item into string and prefix it with string "test-"
so the end result would ne one dimentional array:
def endResult = ["test-88.3.2"],["test-88.3.3"],["test-22.33.4"]
if i do:
test.each {println it.join(".")}
it prints the first part but as written i need to save it and add prefix
im new to groovy any help would be great
The each method does not produce any result - it only iterates the input collection and allows you to do something with each element (e.g. print it to the console like in the example you showed in your question.)
If you want to manipulate each element of the collection and store it as a new collection, you can use the collect method which also takes a closure as a parameter. This closure is applied to each element of the input collection, and the return value of this closure is used to return a new collection from the collect method.
Something like this should do the trick for you:
def test = [[88,3,2],[22,33,4],[88,3,3]]
def endResult = test.collect { 'test-' + it.join('.') }
println endResult // [test-88.3.2, test-22.33.4, test-88.3.3]
It's worth mentioning that the closure we passed to the collect method uses so-called implicit return - there is no return keyword, but the value it produces from 'test-' + it.join('.') is returned implicitly.

Setting instance members in function - what's going on here? [duplicate]

Are parameters passed by reference or by value? How do I pass by reference so that the code below outputs 'Changed' instead of 'Original'?
class PassByReference:
def __init__(self):
self.variable = 'Original'
self.change(self.variable)
print(self.variable)
def change(self, var):
var = 'Changed'
See also: Why can a function modify some arguments as perceived by the caller, but not others?
Arguments are passed by assignment. The rationale behind this is twofold:
the parameter passed in is actually a reference to an object (but the reference is passed by value)
some data types are mutable, but others aren't
So:
If you pass a mutable object into a method, the method gets a reference to that same object and you can mutate it to your heart's delight, but if you rebind the reference in the method, the outer scope will know nothing about it, and after you're done, the outer reference will still point at the original object.
If you pass an immutable object to a method, you still can't rebind the outer reference, and you can't even mutate the object.
To make it even more clear, let's have some examples.
List - a mutable type
Let's try to modify the list that was passed to a method:
def try_to_change_list_contents(the_list):
print('got', the_list)
the_list.append('four')
print('changed to', the_list)
outer_list = ['one', 'two', 'three']
print('before, outer_list =', outer_list)
try_to_change_list_contents(outer_list)
print('after, outer_list =', outer_list)
Output:
before, outer_list = ['one', 'two', 'three']
got ['one', 'two', 'three']
changed to ['one', 'two', 'three', 'four']
after, outer_list = ['one', 'two', 'three', 'four']
Since the parameter passed in is a reference to outer_list, not a copy of it, we can use the mutating list methods to change it and have the changes reflected in the outer scope.
Now let's see what happens when we try to change the reference that was passed in as a parameter:
def try_to_change_list_reference(the_list):
print('got', the_list)
the_list = ['and', 'we', 'can', 'not', 'lie']
print('set to', the_list)
outer_list = ['we', 'like', 'proper', 'English']
print('before, outer_list =', outer_list)
try_to_change_list_reference(outer_list)
print('after, outer_list =', outer_list)
Output:
before, outer_list = ['we', 'like', 'proper', 'English']
got ['we', 'like', 'proper', 'English']
set to ['and', 'we', 'can', 'not', 'lie']
after, outer_list = ['we', 'like', 'proper', 'English']
Since the the_list parameter was passed by value, assigning a new list to it had no effect that the code outside the method could see. The the_list was a copy of the outer_list reference, and we had the_list point to a new list, but there was no way to change where outer_list pointed.
String - an immutable type
It's immutable, so there's nothing we can do to change the contents of the string
Now, let's try to change the reference
def try_to_change_string_reference(the_string):
print('got', the_string)
the_string = 'In a kingdom by the sea'
print('set to', the_string)
outer_string = 'It was many and many a year ago'
print('before, outer_string =', outer_string)
try_to_change_string_reference(outer_string)
print('after, outer_string =', outer_string)
Output:
before, outer_string = It was many and many a year ago
got It was many and many a year ago
set to In a kingdom by the sea
after, outer_string = It was many and many a year ago
Again, since the the_string parameter was passed by value, assigning a new string to it had no effect that the code outside the method could see. The the_string was a copy of the outer_string reference, and we had the_string point to a new string, but there was no way to change where outer_string pointed.
I hope this clears things up a little.
EDIT: It's been noted that this doesn't answer the question that #David originally asked, "Is there something I can do to pass the variable by actual reference?". Let's work on that.
How do we get around this?
As #Andrea's answer shows, you could return the new value. This doesn't change the way things are passed in, but does let you get the information you want back out:
def return_a_whole_new_string(the_string):
new_string = something_to_do_with_the_old_string(the_string)
return new_string
# then you could call it like
my_string = return_a_whole_new_string(my_string)
If you really wanted to avoid using a return value, you could create a class to hold your value and pass it into the function or use an existing class, like a list:
def use_a_wrapper_to_simulate_pass_by_reference(stuff_to_change):
new_string = something_to_do_with_the_old_string(stuff_to_change[0])
stuff_to_change[0] = new_string
# then you could call it like
wrapper = [my_string]
use_a_wrapper_to_simulate_pass_by_reference(wrapper)
do_something_with(wrapper[0])
Although this seems a little cumbersome.
The problem comes from a misunderstanding of what variables are in Python. If you're used to most traditional languages, you have a mental model of what happens in the following sequence:
a = 1
a = 2
You believe that a is a memory location that stores the value 1, then is updated to store the value 2. That's not how things work in Python. Rather, a starts as a reference to an object with the value 1, then gets reassigned as a reference to an object with the value 2. Those two objects may continue to coexist even though a doesn't refer to the first one anymore; in fact they may be shared by any number of other references within the program.
When you call a function with a parameter, a new reference is created that refers to the object passed in. This is separate from the reference that was used in the function call, so there's no way to update that reference and make it refer to a new object. In your example:
def __init__(self):
self.variable = 'Original'
self.Change(self.variable)
def Change(self, var):
var = 'Changed'
self.variable is a reference to the string object 'Original'. When you call Change you create a second reference var to the object. Inside the function you reassign the reference var to a different string object 'Changed', but the reference self.variable is separate and does not change.
The only way around this is to pass a mutable object. Because both references refer to the same object, any changes to the object are reflected in both places.
def __init__(self):
self.variable = ['Original']
self.Change(self.variable)
def Change(self, var):
var[0] = 'Changed'
I found the other answers rather long and complicated, so I created this simple diagram to explain the way Python treats variables and parameters.
It is neither pass-by-value or pass-by-reference - it is call-by-object. See this, by Fredrik Lundh:
Call By Object
Here is a significant quote:
"...variables [names] are not objects; they cannot be denoted by other variables or referred to by objects."
In your example, when the Change method is called--a namespace is created for it; and var becomes a name, within that namespace, for the string object 'Original'. That object then has a name in two namespaces. Next, var = 'Changed' binds var to a new string object, and thus the method's namespace forgets about 'Original'. Finally, that namespace is forgotten, and the string 'Changed' along with it.
Think of stuff being passed by assignment instead of by reference/by value. That way, it is always clear, what is happening as long as you understand what happens during the normal assignment.
So, when passing a list to a function/method, the list is assigned to the parameter name. Appending to the list will result in the list being modified. Reassigning the list inside the function will not change the original list, since:
a = [1, 2, 3]
b = a
b.append(4)
b = ['a', 'b']
print a, b # prints [1, 2, 3, 4] ['a', 'b']
Since immutable types cannot be modified, they seem like being passed by value - passing an int into a function means assigning the int to the function's parameter. You can only ever reassign that, but it won't change the original variables value.
There are no variables in Python
The key to understanding parameter passing is to stop thinking about "variables". There are names and objects in Python and together they
appear like variables, but it is useful to always distinguish the three.
Python has names and objects.
Assignment binds a name to an object.
Passing an argument into a function also binds a name (the parameter name of the function) to an object.
That is all there is to it. Mutability is irrelevant to this question.
Example:
a = 1
This binds the name a to an object of type integer that holds the value 1.
b = x
This binds the name b to the same object that the name x is currently bound to.
Afterward, the name b has nothing to do with the name x anymore.
See sections 3.1 and 4.2 in the Python 3 language reference.
How to read the example in the question
In the code shown in the question, the statement self.Change(self.variable) binds the name var (in the scope of function Change) to the object that holds the value 'Original' and the assignment var = 'Changed' (in the body of function Change) assigns that same name again: to some other object (that happens to hold a string as well but could have been something else entirely).
How to pass by reference
So if the thing you want to change is a mutable object, there is no problem, as everything is effectively passed by reference.
If it is an immutable object (e.g. a bool, number, string), the way to go is to wrap it in a mutable object.
The quick-and-dirty solution for this is a one-element list (instead of self.variable, pass [self.variable] and in the function modify var[0]).
The more pythonic approach would be to introduce a trivial, one-attribute class. The function receives an instance of the class and manipulates the attribute.
Effbot (aka Fredrik Lundh) has described Python's variable passing style as call-by-object: http://effbot.org/zone/call-by-object.htm
Objects are allocated on the heap and pointers to them can be passed around anywhere.
When you make an assignment such as x = 1000, a dictionary entry is created that maps the string "x" in the current namespace to a pointer to the integer object containing one thousand.
When you update "x" with x = 2000, a new integer object is created and the dictionary is updated to point at the new object. The old one thousand object is unchanged (and may or may not be alive depending on whether anything else refers to the object).
When you do a new assignment such as y = x, a new dictionary entry "y" is created that points to the same object as the entry for "x".
Objects like strings and integers are immutable. This simply means that there are no methods that can change the object after it has been created. For example, once the integer object one-thousand is created, it will never change. Math is done by creating new integer objects.
Objects like lists are mutable. This means that the contents of the object can be changed by anything pointing to the object. For example, x = []; y = x; x.append(10); print y will print [10]. The empty list was created. Both "x" and "y" point to the same list. The append method mutates (updates) the list object (like adding a record to a database) and the result is visible to both "x" and "y" (just as a database update would be visible to every connection to that database).
Hope that clarifies the issue for you.
Technically, Python always uses pass by reference values. I am going to repeat my other answer to support my statement.
Python always uses pass-by-reference values. There isn't any exception. Any variable assignment means copying the reference value. No exception. Any variable is the name bound to the reference value. Always.
You can think about a reference value as the address of the target object. The address is automatically dereferenced when used. This way, working with the reference value, it seems you work directly with the target object. But there always is a reference in between, one step more to jump to the target.
Here is the example that proves that Python uses passing by reference:
If the argument was passed by value, the outer lst could not be modified. The green are the target objects (the black is the value stored inside, the red is the object type), the yellow is the memory with the reference value inside -- drawn as the arrow. The blue solid arrow is the reference value that was passed to the function (via the dashed blue arrow path). The ugly dark yellow is the internal dictionary. (It actually could be drawn also as a green ellipse. The colour and the shape only says it is internal.)
You can use the id() built-in function to learn what the reference value is (that is, the address of the target object).
In compiled languages, a variable is a memory space that is able to capture the value of the type. In Python, a variable is a name (captured internally as a string) bound to the reference variable that holds the reference value to the target object. The name of the variable is the key in the internal dictionary, the value part of that dictionary item stores the reference value to the target.
Reference values are hidden in Python. There isn't any explicit user type for storing the reference value. However, you can use a list element (or element in any other suitable container type) as the reference variable, because all containers do store the elements also as references to the target objects. In other words, elements are actually not contained inside the container -- only the references to elements are.
A simple trick I normally use is to just wrap it in a list:
def Change(self, var):
var[0] = 'Changed'
variable = ['Original']
self.Change(variable)
print variable[0]
(Yeah I know this can be inconvenient, but sometimes it is simple enough to do this.)
(edit - Blair has updated his enormously popular answer so that it is now accurate)
I think it is important to note that the current post with the most votes (by Blair Conrad), while being correct with respect to its result, is misleading and is borderline incorrect based on its definitions. While there are many languages (like C) that allow the user to either pass by reference or pass by value, Python is not one of them.
David Cournapeau's answer points to the real answer and explains why the behavior in Blair Conrad's post seems to be correct while the definitions are not.
To the extent that Python is pass by value, all languages are pass by value since some piece of data (be it a "value" or a "reference") must be sent. However, that does not mean that Python is pass by value in the sense that a C programmer would think of it.
If you want the behavior, Blair Conrad's answer is fine. But if you want to know the nuts and bolts of why Python is neither pass by value or pass by reference, read David Cournapeau's answer.
You got some really good answers here.
x = [ 2, 4, 4, 5, 5 ]
print x # 2, 4, 4, 5, 5
def go( li ) :
li = [ 5, 6, 7, 8 ] # re-assigning what li POINTS TO, does not
# change the value of the ORIGINAL variable x
go( x )
print x # 2, 4, 4, 5, 5 [ STILL! ]
raw_input( 'press any key to continue' )
Python’s pass-by-assignment scheme isn’t quite the same as C++’s reference parameters option, but it turns out to be very similar to the argument-passing model of the C language (and others) in practice:
Immutable arguments are effectively passed “by value.” Objects such as integers and strings are passed by object reference instead of by copying, but because you can’t change immutable objects in place anyhow, the effect is much like making a copy.
Mutable arguments are effectively passed “by pointer.” Objects such as lists
and dictionaries are also passed by object reference, which is similar to the way C
passes arrays as pointers—mutable objects can be changed in place in the function,
much like C arrays.
In this case the variable titled var in the method Change is assigned a reference to self.variable, and you immediately assign a string to var. It's no longer pointing to self.variable. The following code snippet shows what would happen if you modify the data structure pointed to by var and self.variable, in this case a list:
>>> class PassByReference:
... def __init__(self):
... self.variable = ['Original']
... self.change(self.variable)
... print self.variable
...
... def change(self, var):
... var.append('Changed')
...
>>> q = PassByReference()
['Original', 'Changed']
>>>
I'm sure someone else could clarify this further.
There are a lot of insights in answers here, but I think an additional point is not clearly mentioned here explicitly. Quoting from Python documentation What are the rules for local and global variables in Python?
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a new value anywhere within the function’s body, it’s assumed to be a local. If a variable is ever assigned a new value inside the function, the variable is implicitly local, and you need to explicitly declare it as ‘global’.
Though a bit surprising at first, a moment’s consideration explains this. On one hand, requiring global for assigned variables provides a bar against unintended side-effects. On the other hand, if global was required for all global references, you’d be using global all the time. You’d have to declare as global every reference to a built-in function or to a component of an imported module. This clutter would defeat the usefulness of the global declaration for identifying side-effects.
Even when passing a mutable object to a function this still applies. And to me it clearly explains the reason for the difference in behavior between assigning to the object and operating on the object in the function.
def test(l):
print "Received", l, id(l)
l = [0, 0, 0]
print "Changed to", l, id(l) # New local object created, breaking link to global l
l = [1, 2, 3]
print "Original", l, id(l)
test(l)
print "After", l, id(l)
gives:
Original [1, 2, 3] 4454645632
Received [1, 2, 3] 4454645632
Changed to [0, 0, 0] 4474591928
After [1, 2, 3] 4454645632
The assignment to an global variable that is not declared global therefore creates a new local object and breaks the link to the original object.
As you can state you need to have a mutable object, but let me suggest you to check over the global variables as they can help you or even solve this kind of issue!
http://docs.python.org/3/faq/programming.html#what-are-the-rules-for-local-and-global-variables-in-python
example:
>>> def x(y):
... global z
... z = y
...
>>> x
<function x at 0x00000000020E1730>
>>> y
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
>>> z
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'z' is not defined
>>> x(2)
>>> x
<function x at 0x00000000020E1730>
>>> y
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
>>> z
2
Here is the simple (I hope) explanation of the concept pass by object used in Python.
Whenever you pass an object to the function, the object itself is passed (object in Python is actually what you'd call a value in other programming languages) not the reference to this object. In other words, when you call:
def change_me(list):
list = [1, 2, 3]
my_list = [0, 1]
change_me(my_list)
The actual object - [0, 1] (which would be called a value in other programming languages) is being passed. So in fact the function change_me will try to do something like:
[0, 1] = [1, 2, 3]
which obviously will not change the object passed to the function. If the function looked like this:
def change_me(list):
list.append(2)
Then the call would result in:
[0, 1].append(2)
which obviously will change the object. This answer explains it well.
Aside from all the great explanations on how this stuff works in Python, I don't see a simple suggestion for the problem. As you seem to do create objects and instances, the Pythonic way of handling instance variables and changing them is the following:
class PassByReference:
def __init__(self):
self.variable = 'Original'
self.Change()
print self.variable
def Change(self):
self.variable = 'Changed'
In instance methods, you normally refer to self to access instance attributes. It is normal to set instance attributes in __init__ and read or change them in instance methods. That is also why you pass self as the first argument to def Change.
Another solution would be to create a static method like this:
class PassByReference:
def __init__(self):
self.variable = 'Original'
self.variable = PassByReference.Change(self.variable)
print self.variable
#staticmethod
def Change(var):
var = 'Changed'
return var
I used the following method to quickly convert some Fortran code to Python. True, it's not pass by reference as the original question was posed, but it is a simple workaround in some cases.
a = 0
b = 0
c = 0
def myfunc(a, b, c):
a = 1
b = 2
c = 3
return a, b, c
a, b, c = myfunc(a, b, c)
print a, b, c
There is a little trick to pass an object by reference, even though the language doesn't make it possible. It works in Java too; it's the list with one item. ;-)
class PassByReference:
def __init__(self, name):
self.name = name
def changeRef(ref):
ref[0] = PassByReference('Michael')
obj = PassByReference('Peter')
print obj.name
p = [obj] # A pointer to obj! ;-)
changeRef(p)
print p[0].name # p->name
It's an ugly hack, but it works. ;-P
Since it seems to be nowhere mentioned an approach to simulate references as known from e.g. C++ is to use an "update" function and pass that instead of the actual variable (or rather, "name"):
def need_to_modify(update):
update(42) # set new value 42
# other code
def call_it():
value = 21
def update_value(new_value):
nonlocal value
value = new_value
need_to_modify(update_value)
print(value) # prints 42
This is mostly useful for "out-only references" or in a situation with multiple threads / processes (by making the update function thread / multiprocessing safe).
Obviously the above does not allow reading the value, only updating it.
Given the way Python handles values and references to them, the only way you can reference an arbitrary instance attribute is by name:
class PassByReferenceIsh:
def __init__(self):
self.variable = 'Original'
self.change('variable')
print self.variable
def change(self, var):
self.__dict__[var] = 'Changed'
In real code you would, of course, add error checking on the dict lookup.
Since your example happens to be object-oriented, you could make the following change to achieve a similar result:
class PassByReference:
def __init__(self):
self.variable = 'Original'
self.change('variable')
print(self.variable)
def change(self, var):
setattr(self, var, 'Changed')
# o.variable will equal 'Changed'
o = PassByReference()
assert o.variable == 'Changed'
Since dictionaries are passed by reference, you can use a dict variable to store any referenced values inside it.
# returns the result of adding numbers `a` and `b`
def AddNumbers(a, b, ref): # using a dict for reference
result = a + b
ref['multi'] = a * b # reference the multi. ref['multi'] is number
ref['msg'] = "The result: " + str(result) + " was nice!"
return result
number1 = 5
number2 = 10
ref = {} # init a dict like that so it can save all the referenced values. this is because all dictionaries are passed by reference, while strings and numbers do not.
sum = AddNumbers(number1, number2, ref)
print("sum: ", sum) # the returned value
print("multi: ", ref['multi']) # a referenced value
print("msg: ", ref['msg']) # a referenced value
You can merely use an empty class as an instance to store reference objects because internally object attributes are stored in an instance dictionary. See the example.
class RefsObj(object):
"A class which helps to create references to variables."
pass
...
# an example of usage
def change_ref_var(ref_obj):
ref_obj.val = 24
ref_obj = RefsObj()
ref_obj.val = 1
print(ref_obj.val) # or print ref_obj.val for python2
change_ref_var(ref_obj)
print(ref_obj.val)
While pass by reference is nothing that fits well into Python and should be rarely used, there are some workarounds that actually can work to get the object currently assigned to a local variable or even reassign a local variable from inside of a called function.
The basic idea is to have a function that can do that access and can be passed as object into other functions or stored in a class.
One way is to use global (for global variables) or nonlocal (for local variables in a function) in a wrapper function.
def change(wrapper):
wrapper(7)
x = 5
def setter(val):
global x
x = val
print(x)
The same idea works for reading and deleting a variable.
For just reading, there is even a shorter way of just using lambda: x which returns a callable that when called returns the current value of x. This is somewhat like "call by name" used in languages in the distant past.
Passing 3 wrappers to access a variable is a bit unwieldy so those can be wrapped into a class that has a proxy attribute:
class ByRef:
def __init__(self, r, w, d):
self._read = r
self._write = w
self._delete = d
def set(self, val):
self._write(val)
def get(self):
return self._read()
def remove(self):
self._delete()
wrapped = property(get, set, remove)
# Left as an exercise for the reader: define set, get, remove as local functions using global / nonlocal
r = ByRef(get, set, remove)
r.wrapped = 15
Pythons "reflection" support makes it possible to get a object that is capable of reassigning a name/variable in a given scope without defining functions explicitly in that scope:
class ByRef:
def __init__(self, locs, name):
self._locs = locs
self._name = name
def set(self, val):
self._locs[self._name] = val
def get(self):
return self._locs[self._name]
def remove(self):
del self._locs[self._name]
wrapped = property(get, set, remove)
def change(x):
x.wrapped = 7
def test_me():
x = 6
print(x)
change(ByRef(locals(), "x"))
print(x)
Here the ByRef class wraps a dictionary access. So attribute access to wrapped is translated to a item access in the passed dictionary. By passing the result of the builtin locals and the name of a local variable, this ends up accessing a local variable. The Python documentation as of 3.5 advises that changing the dictionary might not work, but it seems to work for me.
Pass-by-reference in Python is quite different from the concept of pass by reference in C++/Java.
Java and C#: primitive types (including string) pass by value (copy). A reference type is passed by reference (address copy), so all changes made in the parameter in the called function are visible to the caller.
C++: Both pass-by-reference or pass-by-value are allowed. If a parameter is passed by reference, you can either modify it or not depending upon whether the parameter was passed as const or not. However, const or not, the parameter maintains the reference to the object and reference cannot be assigned to point to a different object within the called function.
Python:
Python is “pass-by-object-reference”, of which it is often said: “Object references are passed by value.” (read here). Both the caller and the function refer to the same object, but the parameter in the function is a new variable which is just holding a copy of the object in the caller. Like C++, a parameter can be either modified or not in function. This depends upon the type of object passed. For example, an immutable object type cannot be modified in the called function whereas a mutable object can be either updated or re-initialized.
A crucial difference between updating or reassigning/re-initializing the mutable variable is that updated value gets reflected back in the called function whereas the reinitialized value does not. The scope of any assignment of new object to a mutable variable is local to the function in the python. Examples provided by #blair-conrad are great to understand this.
I am new to Python, started yesterday (though I have been programming for 45 years).
I came here because I was writing a function where I wanted to have two so-called out-parameters. If it would have been only one out-parameter, I wouldn't get hung up right now on checking how reference/value works in Python. I would just have used the return value of the function instead. But since I needed two such out-parameters I felt I needed to sort it out.
In this post I am going to show how I solved my situation. Perhaps others coming here can find it valuable, even though it is not exactly an answer to the topic question. Experienced Python programmers of course already know about the solution I used, but it was new to me.
From the answers here I could quickly see that Python works a bit like JavaScript in this regard, and that you need to use workarounds if you want the reference functionality.
But then I found something neat in Python that I don't think I have seen in other languages before, namely that you can return more than one value from a function, in a simple comma-separated way, like this:
def somefunction(p):
a = p + 1
b = p + 2
c = -p
return a, b, c
and that you can handle that on the calling side similarly, like this
x, y, z = somefunction(w)
That was good enough for me and I was satisfied. There isn't any need to use some workaround.
In other languages you can of course also return many values, but then usually in the from of an object, and you need to adjust the calling side accordingly.
The Python way of doing it was nice and simple.
If you want to mimic by reference even more, you could do as follows:
def somefunction(a, b, c):
a = a * 2
b = b + a
c = a * b * c
return a, b, c
x = 3
y = 5
z = 10
print(F"Before : {x}, {y}, {z}")
x, y, z = somefunction(x, y, z)
print(F"After : {x}, {y}, {z}")
which gives this result
Before : 3, 5, 10
After : 6, 11, 660
Alternatively, you could use ctypes which would look something like this:
import ctypes
def f(a):
a.value = 2398 ## Resign the value in a function
a = ctypes.c_int(0)
print("pre f", a)
f(a)
print("post f", a)
As a is a c int and not a Python integer and apparently passed by reference. However, you have to be careful as strange things could happen, and it is therefore not advised.
Use dataclasses. Also, it allows you to apply type restrictions (aka "type hints").
from dataclasses import dataclass
#dataclass
class Holder:
obj: your_type # Need any type? Use "obj: object" then.
def foo(ref: Holder):
ref.obj = do_something()
I agree with folks that in most cases you'd better consider not to use it.
And yet, when we're talking about contexts, it's worth to know that way.
You can design an explicit context class though. When prototyping, I prefer dataclasses, just because it's easy to serialize them back and forth.
There are already many great answers (or let's say opinions) about this and I've read them, but I want to mention a missing one. The one from Python's documentation in the FAQ section. I don't know the date of publishing this page, but this should be our true reference:
Remember that arguments are passed by assignment in Python. Since
assignment just creates references to objects, there’s no alias
between an argument name in the caller and callee, and so no
call-by-reference per se.
If you have:
a = SOMETHING
def fn(arg):
pass
and you call it like fn(a), you're doing exactly what you do in assignment. So this happens:
arg = a
An additional reference to SOMETHING is created. Variables are just symbols/names/references. They don't "hold" anything.

Why are there different behaviors for the ways of addressing GString keys in maps?

While studying the Groovy (2.4.4) syntax in the official documentation, I came across the special behavior concerning maps with GStrings as identifiers. As described in the documentation, GStrings are a bad idea as (hash)map identifiers, because the hashcodes of a non-evaluated GString-object differs from a regular String-object with the same representation as the evaluated GString.
Example:
def key = "id"
def m = ["${key}": "value for ${key}"]
println "id".hashCode() // prints "3355"
println "${key}".hashCode() // prints "3392", different hashcode
assert m["id"] == null // evaluates true
However, my intuitive expectation was that using the actual GString identifier to address a key in the map will in fact deliver the value - but it does not.
def key = "id"
def m = ["${key}": "value for ${key}"]
assert m["${key}"] == null // evaluates also true, not expected
That made me curious. So I had several suggestions concerning this issue and did some experiments.
(pls keep in my mind that I am new to Groovy and I was just brainstorming on the fly - continue to Suggestion #4 if you do not want to read how I tried to examine the cause of the issue)
Suggestion #1. hashcode for GString objects works/is implemented somewhat non-deterministic for whatever reason and delivers different results depending on the context or the actual object.
That turned out to be nonsense quite fast:
println "${key}".hashCode() // prints "3392"
// do sth else
println "${key}".hashCode() // still "3392"
Suggestion #2. The actual key in the map or the map item does not have the expected representation or hashcode.
I took a closer look at the item in the map, the key, and its hashcode.
println m // prints "[id:value for id]", as expected
m.each {
it -> println key.hashCode()
} // prints "3355" - hashcode of the String "id"
So the hashcode of the key inside the map is different from the GString hashcode. HA! or not. Though it is nice to know, it is actually not relevant because I still do know the actual hashcodes in the map index. I just rehashed a key that has been transformed to a string after being put into the index. So what else?
Suggestion #3. The equals-method of a GString has an unknown or non- implemented behavior.
No matter whether two hashcodes are equal, they may not represent the same object in a map. That depends on the implementation of the equals method for the class of the key-object. If the equals-method is, for instance, not implemented, two objects are not equal even if the hashcode is identical and therefore the desired map key cannot be adressed properly. So I tried:
def a = "${key}"
def b = "${key}"
assert a.equals(b) // returns true (unfortunate but expected)
So two representations of the same GString are equal by default.
I skip some others ideas I tried and continue with the last thing I tried just before I was going to write this post.
Suggestion #4. The syntax of access matters.
That was a real killer of understanding. I knew before: There are syntactically different ways two access map values. Each way has its restrictions, but I thought the results stay the same. Well, this came up:
def key = "id"
def m = ["${key}": "value for ${key}"]
assert m["id"] == null // as before
assert m["${key}"] == null // as before
assert m.get("${key}") == null // assertion fails, value returned
So if I use the get-method of a map, I get the actual value in the way I expected it to in the first place.
What is the explanation for this map access behavior concerning GStrings? (or what kind of rookie mistake is hidden here?)
Thanks for your patience.
EDIT: I am afraid that my actual question is not clearly stated, so here is the case in short and concise:
When I have a map with a GString as a key like this
def m = ["${key}": "value for ${key}"]
why does this return the value
println m.get("${key}")
but that does not
println m["${key}"]
?
You can look at this matter with a very different approach. A map is supposed to have immutable keys (at least for hashcode and equals), because the map implementation depends on this. GString is mutable, thus not really suited for map keys in general. There is also the problem of calling String#equals(GString). GString is a Groovy class, so we can influence the equals method to equal to a String just fine. But String is very different. That means calling equals on a String with a GString will always be false in the Java world, even if hashcode() would behave the same for String and GString. And now imagine a map with String keys and you ask the map for a value with a GString. It would always return null. On the other hand a map with GString keys queried with a String could return the "proper" value. This means there will always be a disconnection.
And because of this problem GString#hashCode() is not equal to String#hashCode() on purpose.
It is in no way non-deterministic, but a GString hashcode can change, if the participating objects change their toString representation:
def map = [:]
def gstring = "$map"
def hashCodeOld = gstring.hashCode()
assert hashCodeOld == gstring.hashCode()
map.foo = "bar"
assert hashCodeOld != gstring.hashCode()
Here the toString representation of map will change for Groovy and GString, thus the GString will produce a different hashcode

Question related to string

I have two statements:
String aStr = new String("ABC");
String bStr = "ABC";
I read in book that in first statement JVM creates two bjects and one reference variable, whereas second statement creates one reference variable and one object.
How is that? When I say new String("ABC") then It's pretty clear that object is created.
Now my question is that for "ABC" value to we do create another object?
Please clarify a bit more here.
Thank you
You will end up with two Strings.
1) the literal "ABC", used to construct aStr and assigned to bStr. The compiler makes sure that this is the same single instance.
2) a newly constructed String aStr (because you forced it to be new'ed, which is really pretty much non-sensical)
Using a string literal will only create a single object for the lifetime of the JVM - or possibly the classloader. (I can't remember the exact details, but it's almost never important.)
That means it's hard to say that the second statement in your code sample really "creates" an object - a certain object has to be present, but if you run the same code in a loop 100 times, it won't create any more objects... whereas the first statement would. (It would require that the object referred to by the "ABC" literal is present and create a new instance on each iteration, by virtue of calling the constructor.)
In particular, if you have:
Object x = "ABC";
Object y = "ABC";
then it's guaranteed (by the language specification) than x and y will refer to the same object. This extends to other constant expressions equal to the same string too:
private static final String A = "a";
Object z = A + "BC"; // x, y and z are still the same reference...
The only time I ever use the String(String) constructor is if I've got a string which may well be backed by a rather larger character array which I don't otherwise need:
String x = readSomeVeryLargeString();
String y = x.substring(5, 10);
String z = new String(y); // Copies the contents
Now if the strings that y and x refer to are eligible for collection but the string that z refers to isn't (e.g. it's passed on to other methods etc) then we don't end up holding all of the original long string in memory, which we would otherwise.

EachWithIndex groovy statement

I am new to groovy and I've been facing some issues understanding the each{} and eachwithindex{} statements in groovy.
Are each and eachWithIndex actually methods? If so what are the arguments that they take?
In the groovy documentation there is this certain example:
def numbers = [ 5, 7, 9, 12 ]
numbers.eachWithIndex{ num, idx -> println "$idx: $num" } //prints each index and number
Well, I see that numbers is an array. What are num and idx in the above statement? What does the -> operator do?
I do know that $idx and $num prints the value, but how is it that idx and num are automatically being associated with the index and contents of the array? What is the logic behind this? Please help.
These are plain methods but they follow quite a specific pattern - they take a Closure as their last argument. A Closure is a piece of functionality that you can pass around and call when applicable.
For example, method eachWithIndex might look like this (roughly):
void eachWithIndex(Closure operation) {
for (int i = 0; this.hasNext(); i++) {
operation(this.next(), i); // Here closure passed as parameter is being called
}
}
This approach allows one to build generic algorithms (like iteration over items) and change the concrete processing logic at runtime by passing different closures.
Regarding the parameters part, as you see in the example above we call the closure (operation) with two parameters - the current element and current index. This means that the eachWithIndex method expects to receive not just any closure but one which would accept these two parameters. From a syntax prospective one defines the parameters during closure definition like this:
{ elem, index ->
// logic
}
So -> is used to separate arguments part of closure definition from its logic. When a closure takes only one argument, its parameter definition can be omitted and then the parameter will be accessible within the closure's scope with the name it (implicit name for the first argument). For example:
[1,2,3].each {
println it
}
It could be rewritten like this:
[1,2,3].each({ elem ->
println elem
})
As you see the Groovy language adds some syntax sugar to make such constructions look prettier.
each and eachWithIndex are, amongst many others, taking so called Closure as an argument. The closure is just a piece of Groovy code wrapped in {} braces. In the code with array:
def numbers = [ 5, 7, 9, 12 ]
numbers.eachWithIndex{ num, idx -> println "$idx: $num" }
there is only one argument (closure, or more precisely: function), please note that in Groovy () braces are sometime optional. num and idx are just an optional aliases for closure (function) arguments, when we need just one argument, this is equivalent (it is implicit name of the first closure argument, very convenient):
def numbers = [ 5, 7, 9, 12 ]
numbers.each {println "$it" }
References:
http://groovy.codehaus.org/Closures
http://en.wikipedia.org/wiki/First-class_function
Normally, if you are using a functional programing language such as Groovy, you would want to avoid using each and eachWithIndex since they encourage you to modify state within the closure or do things that have side effects.
If possible, you may want to do your operations using other groovy collection methods such as .collect or .inject or findResult etc.
However, to use these for your problem, i.e print the list elements with their index, you will need to use the withIndex method on the original collection which will transform the collection to a collection of pairs of [element, index]
For example,
println(['a', 'b', 'c'].withIndex())
EachWithIndex can be used as follows:
package json
import groovy.json.*
import com.eviware.soapui.support.XmlHolder
def project = testRunner.testCase.testSuite.project
def testCase = testRunner.testCase;
def strArray = new String[200]
//Response for a step you want the json from
def response = context.expand('${Offers#Response#$[\'Data\']}').toString()
def json = new JsonSlurper().parseText(response)
//Value you want to compare with in your array
def offername = project.getPropertyValue("Offername")
log.info(offername)
Boolean flagpresent = false
Boolean flagnotpresent = false
strArray = json.Name
def id = 0;
//To find the offername in the array of offers displayed
strArray.eachWithIndex
{
name, index ->
if("${name}" != offername)
{
flagnotpresent= false;
}
else
{
id = "${index}";
flagpresent = true;
log.info("${index}.${name}")
log.info(id)
}
}

Resources