I've noticed that many operations on lists that modify the list's contents will return None, rather than returning the list itself. Examples:
>>> mylist = ['a', 'b', 'c']
>>> empty = mylist.clear()
>>> restored = mylist.extend(range(3))
>>> backwards = mylist.reverse()
>>> with_four = mylist.append(4)
>>> in_order = mylist.sort()
>>> without_one = mylist.remove(1)
>>> mylist
[0, 2, 4]
>>> [empty, restored, backwards, with_four, in_order, without_one]
[None, None, None, None, None, None]
What is the thought process behind this decision?
To me, it seems hampering, since it prevents "chaining" of list processing (e.g. mylist.reverse().append('a string')[:someLimit]). I imagine it might be that "The Powers That Be" decided that list comprehension is a better paradigm (a valid opinion), and so didn't want to encourage other methods - but it seems perverse to prevent an intuitive method, even if better alternatives exist.
This question is specifically about Python's design decision to return None from mutating list methods like .append. Novices often write incorrect code that expects .append (in particular) to return the same list that was just modified.
For the simple question of "how do I append to a list?" (or debugging questions that boil down to that problem), see Why does "x = x.append([i])" not work in a for loop?.
To get modified versions of the list, see:
For .sort: How can I get a sorted copy of a list?
For .reverse: How can I get a reversed copy of a list (avoid a separate statement when chaining a method after .reverse)?
The same issue applies to some methods of other built-in data types, e.g. set.discard (see How to remove specific element from sets inside a list using list comprehension) and dict.update (see Why doesn't a python dict.update() return the object?).
The same reasoning applies to designing your own APIs. See Is making in-place operations return the object a bad idea?.
The general design principle in Python is for functions that mutate an object in-place to return None. I'm not sure it would have been the design choice I'd have chosen, but it's basically to emphasise that a new object is not returned.
Guido van Rossum (our Python BDFL) states the design choice on the Python-Dev mailing list:
I'd like to explain once more why I'm so adamant that sort() shouldn't
return 'self'.
This comes from a coding style (popular in various other languages, I
believe especially Lisp revels in it) where a series of side effects
on a single object can be chained like this:
x.compress().chop(y).sort(z)
which would be the same as
x.compress()
x.chop(y)
x.sort(z)
I find the chaining form a threat to readability; it requires that the
reader must be intimately familiar with each of the methods. The
second form makes it clear that each of these calls acts on the same
object, and so even if you don't know the class and its methods very
well, you can understand that the second and third call are applied to
x (and that all calls are made for their side-effects), and not to
something else.
I'd like to reserve chaining for operations that return new values,
like string processing operations:
y = x.rstrip("\n").split(":").lower()
There are a few standard library modules that encourage chaining of
side-effect calls (pstat comes to mind). There shouldn't be any new
ones; pstat slipped through my filter when it was weak.
I can't speak for the developers, but I find this behavior very intuitive.
If a method works on the original object and modifies it in-place, it doesn't return anything, because there is no new information - you obviously already have a reference to the (now mutated) object, so why return it again?
If, however, a method or function creates a new object, then of course it has to return it.
So l.reverse() returns nothing (because now the list has been reversed, but the identfier l still points to that list), but reversed(l) has to return the newly generated list because l still points to the old, unmodified list.
EDIT: I just learned from another answer that this principle is called Command-Query separation.
One could argue that the signature itself makes it clear that the function mutates the list rather than returning a new one: if the function returned a list, its behavior would have been much less obvious.
If you were sent here after asking for help fixing your code:
In the future, please try to look for problems in the code yourself, by carefully studying what happens when the code runs. Rather than giving up because there is an error message, check the result of each calculation, and see where the code starts working differently from what you expect.
If you had code calling a method like .append or .sort on a list, you will notice that the return value is None, while the list is modified in place. Study the example carefully:
>>> x = ['e', 'x', 'a', 'm', 'p', 'l', 'e']
>>> y = x.sort()
>>> print(y)
None
>>> print(x)
['a', 'e', 'e', 'l', 'm', 'p', 'x']
y got the special None value, because that is what was returned. x changed, because the sort happened in place.
It works this way on purpose, so that code like x.sort().reverse() breaks. See the other answers to understand why the Python developers wanted it that way.
To fix the problem
First, think carefully about the intent of the code. Should x change? Do we actually need a separate y?
Let's consider .sort first. If x should change, then call x.sort() by itself, without assigning the result anywhere.
If a sorted copy is needed instead, use y = x.sorted(). See How can I get a sorted copy of a list? for details.
For other methods, we can get modified copies like so:
.clear -> there is no point to this; a "cleared copy" of the list is just an empty list. Just use y = [].
.append and .extend -> probably the simplest way is to use the + operator. To add multiple elements from a list l, use y = x + l rather than .extend. To add a single element e wrap it in a list first: y = x + [e]. Another way in 3.5 and up is to use unpacking: y = [*x, *l] for .extend, y = [*x, e] for .append. See also How to allow list append() method to return the new list for .append and How do I concatenate two lists in Python? for .extend.
.reverse -> First, consider whether an actual copy is needed. The built-in reversed gives you an iterator that can be used to loop over the elements in reverse order. To make an actual copy, simply pass that iterator to list: y = list(reversed(x)). See How can I get a reversed copy of a list (avoid a separate statement when chaining a method after .reverse)? for details.
.remove -> Figure out the index of the element that will be removed (using .index), then use slicing to find the elements before and after that point and put them together. As a function:
def without(a_list, value):
index = a_list.index(value)
return a_list[:index] + a_list[index+1:]
(We can translate .pop similarly to make a modified copy, though of course .pop actually returns an element from the list.)
See also A quick way to return list without a specific element in Python.
(If you plan to remove multiple elements, strongly consider using a list comprehension (or filter) instead. It will be much simpler than any of the workarounds needed for removing items from the list while iterating over it. This way also naturally gives a modified copy.)
For any of the above, of course, we can also make a modified copy by explicitly making a copy and then using the in-place method on the copy. The most elegant approach will depend on the context and on personal taste.
As we know list in python is a mutable object and one of characteristics of mutable object is the ability to modify the state of this object without the need to assign its new state to a variable. we should demonstrate more about this topic to understand the root of this issue.
An object whose internal state can be changed is mutable. On the other hand, immutable doesn’t allow any change in the object once it has been created. Object mutability is one of the characteristics that makes Python a dynamically typed language.
Every object in python has three attributes:
Identity – This refers to the address that the object refers to in the computer’s memory.
Type – This refers to the kind of object that is created. For example integer, list, string etc.
Value – This refers to the value stored by the object. For example str = "a".
While ID and Type cannot be changed once it’s created, values can be changed for Mutable objects.
let us discuss the below code step-by-step to depict what it means in Python:
Creating a list which contains name of cities
cities = ['London', 'New York', 'Chicago']
Printing the location of the object created in the memory address in hexadecimal format
print(hex(id(cities)))
Output [1]: 0x1691d7de8c8
Adding a new city to the list cities
cities.append('Delhi')
Printing the elements from the list cities, separated by a comma
for city in cities:
print(city, end=', ')
Output [2]: London, New York, Chicago, Delhi
Printing the location of the object created in the memory address in hexadecimal format
print(hex(id(cities)))
Output [3]: 0x1691d7de8c8
The above example shows us that we were able to change the internal state of the object cities by adding one more city 'Delhi' to it, yet, the memory address of the object did not change. This confirms that we did not create a new object, rather, the same object was changed or mutated. Hence, we can say that the object which is a type of list with reference variable name cities is a MUTABLE OBJECT.
While the immutable object internal state can not be changed. For instance, consider the below code and associated error message with it, while trying to change the value of a Tuple at index 0
Creating a Tuple with variable name foo
foo = (1, 2)
Changing the index 0 value from 1 to 3
foo[0] = 3
TypeError: 'tuple' object does not support item assignment
We can conclude from the examples why mutable object shouldn't return anything when executing operations on it because it's modifying the internal state of the object directly and there is no point in returning new modified object. unlike immutable object which should return new object of the modified state after executing operations on it.
First of All, I should tell that what I am suggesting is without a doubt, a bad programming practice but if you want to use append in lambda function and you don't care about the code readability, there is way to just do that.
Imagine you have a list of lists and you want to append a element to each inner lists using map and lambda. here is how you can do that:
my_list = [[1, 2, 3, 4],
[3, 2, 1],
[1, 1, 1]]
my_new_element = 10
new_list = list(map(lambda x: [x.append(my_new_element), x][1], my_list))
print(new_list)
How it works:
when lambda wants to calculate to output, first it should calculate the [x.append(my_new_element), x] expression. To calculate this expression the append function will run and the result of expression will be [None, x] and by specifying that you want the second element of the list the result of [None,x][1] will be x
Using custom function is more readable and the better option:
def append_my_list(input_list, new_element):
input_list.append(new_element)
return input_list
my_list = [[1, 2, 3, 4],
[3, 2, 1],
[1, 1, 1]]
my_new_element = 10
new_list = list(map(lambda x: append_my_list(x, my_new_element), my_list))
print(new_list)
What's the difference between tuples/lists and what are their advantages/disadvantages?
Apart from tuples being immutable there is also a semantic distinction that should guide their usage. Tuples are heterogeneous data structures (i.e., their entries have different meanings), while lists are homogeneous sequences. Tuples have structure, lists have order.
Using this distinction makes code more explicit and understandable.
One example would be pairs of page and line number to reference locations in a book, e.g.:
my_location = (42, 11) # page number, line number
You can then use this as a key in a dictionary to store notes on locations. A list on the other hand could be used to store multiple locations. Naturally one might want to add or remove locations from the list, so it makes sense that lists are mutable. On the other hand it doesn't make sense to add or remove items from an existing location - hence tuples are immutable.
There might be situations where you want to change items within an existing location tuple, for example when iterating through the lines of a page. But tuple immutability forces you to create a new location tuple for each new value. This seems inconvenient on the face of it, but using immutable data like this is a cornerstone of value types and functional programming techniques, which can have substantial advantages.
There are some interesting articles on this issue, e.g. "Python Tuples are Not Just Constant Lists" or "Understanding tuples vs. lists in Python". The official Python documentation also mentions this
"Tuples are immutable, and usually contain an heterogeneous sequence ...".
In a statically typed language like Haskell the values in a tuple generally have different types and the length of the tuple must be fixed. In a list the values all have the same type and the length is not fixed. So the difference is very obvious.
Finally there is the namedtuple in Python, which makes sense because a tuple is already supposed to have structure. This underlines the idea that tuples are a light-weight alternative to classes and instances.
Difference between list and tuple
Literal
someTuple = (1,2)
someList = [1,2]
Size
a = tuple(range(1000))
b = list(range(1000))
a.__sizeof__() # 8024
b.__sizeof__() # 9088
Due to the smaller size of a tuple operation, it becomes a bit faster, but not that much to mention about until you have a huge number of elements.
Permitted operations
b = [1,2]
b[0] = 3 # [3, 2]
a = (1,2)
a[0] = 3 # Error
That also means that you can't delete an element or sort a tuple.
However, you could add a new element to both list and tuple with the only difference that since the tuple is immutable, you are not really adding an element but you are creating a new tuple, so the id of will change
a = (1,2)
b = [1,2]
id(a) # 140230916716520
id(b) # 748527696
a += (3,) # (1, 2, 3)
b += [3] # [1, 2, 3]
id(a) # 140230916878160
id(b) # 748527696
Usage
As a list is mutable, it can't be used as a key in a dictionary, whereas a tuple can be used.
a = (1,2)
b = [1,2]
c = {a: 1} # OK
c = {b: 1} # Error
If you went for a walk, you could note your coordinates at any instant in an (x,y) tuple.
If you wanted to record your journey, you could append your location every few seconds to a list.
But you couldn't do it the other way around.
The key difference is that tuples are immutable. This means that you cannot change the values in a tuple once you have created it.
So if you're going to need to change the values use a List.
Benefits to tuples:
Slight performance improvement.
As a tuple is immutable it can be used as a key in a dictionary.
If you can't change it neither can anyone else, which is to say you don't need to worry about any API functions etc. changing your tuple without being asked.
Lists are mutable; tuples are not.
From docs.python.org/2/tutorial/datastructures.html
Tuples are immutable, and usually contain an heterogeneous sequence of
elements that are accessed via unpacking (see later in this section)
or indexing (or even by attribute in the case of namedtuples). Lists
are mutable, and their elements are usually homogeneous and are
accessed by iterating over the list.
This is an example of Python lists:
my_list = [0,1,2,3,4]
top_rock_list = ["Bohemian Rhapsody","Kashmir","Sweet Emotion", "Fortunate Son"]
This is an example of Python tuple:
my_tuple = (a,b,c,d,e)
celebrity_tuple = ("John", "Wayne", 90210, "Actor", "Male", "Dead")
Python lists and tuples are similar in that they both are ordered collections of values. Besides the shallow difference that lists are created using brackets "[ ... , ... ]" and tuples using parentheses "( ... , ... )", the core technical "hard coded in Python syntax" difference between them is that the elements of a particular tuple are immutable whereas lists are mutable (...so only tuples are hashable and can be used as dictionary/hash keys!). This gives rise to differences in how they can or can't be used (enforced a priori by syntax) and differences in how people choose to use them (encouraged as 'best practices,' a posteriori, this is what smart programers do). The main difference a posteriori in differentiating when tuples are used versus when lists are used lies in what meaning people give to the order of elements.
For tuples, 'order' signifies nothing more than just a specific 'structure' for holding information. What values are found in the first field can easily be switched into the second field as each provides values across two different dimensions or scales. They provide answers to different types of questions and are typically of the form: for a given object/subject, what are its attributes? The object/subject stays constant, the attributes differ.
For lists, 'order' signifies a sequence or a directionality. The second element MUST come after the first element because it's positioned in the 2nd place based on a particular and common scale or dimension. The elements are taken as a whole and mostly provide answers to a single question typically of the form, for a given attribute, how do these objects/subjects compare? The attribute stays constant, the object/subject differs.
There are countless examples of people in popular culture and programmers who don't conform to these differences and there are countless people who might use a salad fork for their main course. At the end of the day, it's fine and both can usually get the job done.
To summarize some of the finer details
Similarities:
Duplicates - Both tuples and lists allow for duplicates
Indexing, Selecting, & Slicing - Both tuples and lists index using integer values found within brackets. So, if you want the first 3 values of a given list or tuple, the syntax would be the same:
>>> my_list[0:3]
[0,1,2]
>>> my_tuple[0:3]
[a,b,c]
Comparing & Sorting - Two tuples or two lists are both compared by their first element, and if there is a tie, then by the second element, and so on. No further attention is paid to subsequent elements after earlier elements show a difference.
>>> [0,2,0,0,0,0]>[0,0,0,0,0,500]
True
>>> (0,2,0,0,0,0)>(0,0,0,0,0,500)
True
Differences: - A priori, by definition
Syntax - Lists use [], tuples use ()
Mutability - Elements in a given list are mutable, elements in a given tuple are NOT mutable.
# Lists are mutable:
>>> top_rock_list
['Bohemian Rhapsody', 'Kashmir', 'Sweet Emotion', 'Fortunate Son']
>>> top_rock_list[1]
'Kashmir'
>>> top_rock_list[1] = "Stairway to Heaven"
>>> top_rock_list
['Bohemian Rhapsody', 'Stairway to Heaven', 'Sweet Emotion', 'Fortunate Son']
# Tuples are NOT mutable:
>>> celebrity_tuple
('John', 'Wayne', 90210, 'Actor', 'Male', 'Dead')
>>> celebrity_tuple[5]
'Dead'
>>> celebrity_tuple[5]="Alive"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
Hashtables (Dictionaries) - As hashtables (dictionaries) require that its keys are hashable and therefore immutable, only tuples can act as dictionary keys, not lists.
#Lists CAN'T act as keys for hashtables(dictionaries)
>>> my_dict = {[a,b,c]:"some value"}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
#Tuples CAN act as keys for hashtables(dictionaries)
>>> my_dict = {("John","Wayne"): 90210}
>>> my_dict
{('John', 'Wayne'): 90210}
Differences - A posteriori, in usage
Homo vs. Heterogeneity of Elements - Generally list objects are homogenous and tuple objects are heterogeneous. That is, lists are used for objects/subjects of the same type (like all presidential candidates, or all songs, or all runners) whereas although it's not forced by), whereas tuples are more for heterogenous objects.
Looping vs. Structures - Although both allow for looping (for x in my_list...), it only really makes sense to do it for a list. Tuples are more appropriate for structuring and presenting information (%s %s residing in %s is an %s and presently %s % ("John","Wayne",90210, "Actor","Dead"))
It's been mentioned that the difference is largely semantic: people expect a tuple and list to represent different information. But this goes further than a guideline; some libraries actually behave differently based on what they are passed. Take NumPy for example (copied from another post where I ask for more examples):
>>> import numpy as np
>>> a = np.arange(9).reshape(3,3)
>>> a
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
>>> idx = (1,1)
>>> a[idx]
4
>>> idx = [1,1]
>>> a[idx]
array([[3, 4, 5],
[3, 4, 5]])
The point is, while NumPy may not be part of the standard library, it's a major Python library, and within NumPy lists and tuples are completely different things.
Lists are for looping, tuples are for structures i.e. "%s %s" %tuple.
Lists are usually homogeneous, tuples are usually heterogeneous.
Lists are for variable length, tuples are for fixed length.
The values of list can be changed any time but the values of tuples can't be change.
The advantages and disadvantages depends upon the use. If you have such a data which you never want to change then you should have to use tuple, otherwise list is the best option.
Difference between list and tuple
Tuples and lists are both seemingly similar sequence types in Python.
Literal syntax
We use parenthesis () to construct tuples and square brackets [ ] to get a new list. Also, we can use call of the appropriate type to get required structure — tuple or list.
someTuple = (4,6)
someList = [2,6]
Mutability
Tuples are immutable, while lists are mutable. This point is the base the for the following ones.
Memory usage
Due to mutability, you need more memory for lists and less memory for tuples.
Extending
You can add a new element to both tuples and lists with the only difference that the id of the tuple will be changed (i.e., we’ll have a new object).
Hashing
Tuples are hashable and lists are not. It means that you can use a tuple as a key in a dictionary. The list can't be used as a key in a dictionary, whereas a tuple can be used
tup = (1,2)
list_ = [1,2]
c = {tup : 1} # ok
c = {list_ : 1} # error
Semantics
This point is more about best practice. You should use tuples as heterogeneous data structures, while lists are homogenous sequences.
Lists are intended to be homogeneous sequences, while tuples are heterogeneous data structures.
As people have already answered here that tuples are immutable while lists are mutable, but there is one important aspect of using tuples which we must remember
If the tuple contains a list or a dictionary inside it, those can be changed even if the tuple itself is immutable.
For example, let's assume we have a tuple which contains a list and a dictionary as
my_tuple = (10,20,30,[40,50],{ 'a' : 10})
we can change the contents of the list as
my_tuple[3][0] = 400
my_tuple[3][1] = 500
which makes new tuple looks like
(10, 20, 30, [400, 500], {'a': 10})
we can also change the dictionary inside tuple as
my_tuple[4]['a'] = 500
which will make the overall tuple looks like
(10, 20, 30, [400, 500], {'a': 500})
This happens because list and dictionary are the objects and these objects are not changing, but the contents its pointing to.
So the tuple remains immutable without any exception
The PEP 484 -- Type Hints says that the types of elements of a tuple can be individually typed; so that you can say Tuple[str, int, float]; but a list, with List typing class can take only one type parameter: List[str], which hints that the difference of the 2 really is that the former is heterogeneous, whereas the latter intrinsically homogeneous.
Also, the standard library mostly uses the tuple as a return value from such standard functions where the C would return a struct.
As people have already mentioned the differences I will write about why tuples.
Why tuples are preferred?
Allocation optimization for small tuples
To reduce memory fragmentation and speed up allocations, Python reuses old tuples. If a
tuple no longer needed and has less than 20 items instead of deleting
it permanently Python moves it to a free list.
A free list is divided into 20 groups, where each group represents a
list of tuples of length n between 0 and 20. Each group can store up
to 2 000 tuples. The first (zero) group contains only 1 element and
represents an empty tuple.
>>> a = (1,2,3)
>>> id(a)
4427578104
>>> del a
>>> b = (1,2,4)
>>> id(b)
4427578104
In the example above we can see that a and b have the same id. That is
because we immediately occupied a destroyed tuple which was on the
free list.
Allocation optimization for lists
Since lists can be modified, Python does not use the same optimization as in tuples. However,
Python lists also have a free list, but it is used only for empty
objects. If an empty list is deleted or collected by GC, it can be
reused later.
>>> a = []
>>> id(a)
4465566792
>>> del a
>>> b = []
>>> id(b)
4465566792
Source: https://rushter.com/blog/python-lists-and-tuples/
Why tuples are efficient than lists? -> https://stackoverflow.com/a/22140115
The most important difference is time ! When you do not want to change the data inside the list better to use tuple ! Here is the example why use tuple !
import timeit
print(timeit.timeit(stmt='[1,2,3,4,5,6,7,8,9,10]', number=1000000)) #created list
print(timeit.timeit(stmt='(1,2,3,4,5,6,7,8,9,10)', number=1000000)) # created tuple
In this example we executed both statements 1 million times
Output :
0.136621
0.013722200000000018
Any one can clearly notice the time difference.
A direction quotation from the documentation on 5.3. Tuples and Sequences:
Though tuples may seem similar to lists, they are often used in different situations and for different purposes. Tuples are immutable, and usually contain a heterogeneous sequence of elements that are accessed via unpacking (see later in this section) or indexing (or even by attribute in the case of namedtuples). Lists are mutable, and their elements are usually homogeneous and are accessed by iterating over the list.
In other words, TUPLES are used to store group of elements where the contents/members of the group would not change while LISTS are used to store group of elements where the members of the group can change.
For instance, if i want to store IP of my network in a variable, it's best i used a tuple since the the IP is fixed. Like this my_ip = ('192.168.0.15', 33, 60). However, if I want to store group of IPs of places I would visit in the next 6 month, then I should use a LIST, since I will keep updating and adding new IP to the group. Like this
places_to_visit = [
('192.168.0.15', 33, 60),
('192.168.0.22', 34, 60),
('192.168.0.1', 34, 60),
('192.168.0.2', 34, 60),
('192.168.0.8', 34, 60),
('192.168.0.11', 34, 60)
]
First of all, they both are the non-scalar objects (also known as a compound objects) in Python.
Tuples, ordered sequence of elements (which can contain any object with no aliasing issue)
Immutable (tuple, int, float, str)
Concatenation using + (brand new tuple will be created of course)
Indexing
Slicing
Singleton (3,) # -> (3) instead of (3) # -> 3
List (Array in other languages), ordered sequence of values
Mutable
Singleton [3]
Cloning new_array = origin_array[:]
List comprehension [x**2 for x in range(1,7)] gives you
[1,4,9,16,25,36] (Not readable)
Using list may also cause an aliasing bug (two distinct paths
pointing to the same object).
Just a quick extension to list vs tuple responses:
Due to dynamic nature, list allocates more bit buckets than the actual memory required. This is done to prevent costly reallocation operation in case extra items are appended in the future.
On the other hand, being static, lightweight tuple object does not reserve extra memory required to store them.
Lists are mutable and tuples are immutable.
Just consider this example.
a = ["1", "2", "ra", "sa"] #list
b = ("1", "2", "ra", "sa") #tuple
Now change index values of list and tuple.
a[2] = 1000
print a #output : ['1', '2', 1000, 'sa']
b[2] = 1000
print b #output : TypeError: 'tuple' object does not support item assignment.
Hence proved the following code is invalid with tuple, because we attempted to update a tuple, which is not allowed.
Lists are mutable. whereas tuples are immutable. Accessing an offset element with index makes more sense in tuples than lists, Because the elements and their index cannot be changed.
List is mutable and tuples is immutable. The main difference between mutable and immutable is memory usage when you are trying to append an item.
When you create a variable, some fixed memory is assigned to the variable. If it is a list, more memory is assigned than actually used. E.g. if current memory assignment is 100 bytes, when you want to append the 101th byte, maybe another 100 bytes will be assigned (in total 200 bytes in this case).
However, if you know that you are not frequently add new elements, then you should use tuples. Tuples assigns exactly size of the memory needed, and hence saves memory, especially when you use large blocks of memory.