Related
I am working on a Python script which is connected to a server. Every x min, server returns two list but the length of these list is not same. For ex:
a = [8, 10, 1, 34]
b = [4, 6, 8]
As you can see above that a is of length 4 and b is of length 3. Similarly, sometimes it returns
a = [3, 6, 4, 5]
b = [8, 3, 5, 2, 9, 3]
I have to write a logic where I have to check if length of these two list is not same, then add the 0 at the end of the list which is smaller than other list. So for ex, if input is:
a = [3, 6, 4, 5]
b = [8, 3, 5, 2, 9, 3]
then output will be:
a = [3, 6, 4, 5, 0, 0]
b = [8, 3, 5, 2, 9, 3]
What can I try to achieve this?
def pad(list1, list2):
# make copies of the existing lists so that original lists remain intact
list1_copy = list1.copy()
list2_copy = list2.copy()
len_list1 = len(list1_copy)
len_list2 = len(list2_copy)
# find the difference in the element count between the two lists
diff = abs(len_list1 - len_list2)
# add `diff` number of elements to the end of the list
if len_list1 < len_list2:
list1_copy += [0] * diff
elif len_list1 > len_list2:
list2_copy += [0] * diff
return list1_copy, list2_copy
a = [3, 6, 4, 5]
b = [8, 3, 5, 2, 9, 3]
# prints: ([3, 6, 4, 5, 0, 0], [8, 3, 5, 2, 9, 3])
print(pad(a, b))
a = [8, 10, 1, 34]
b = [4, 6, 8]
# prints: ([8, 10, 1, 34], [4, 6, 8, 0])
print(pad(a, b))
For now, I can suggest this solution:
a = [3, 6, 4, 5]
b = [8, 3, 5, 2, 9, 3]
# Gets the size of a and b.
sizeA, sizeB = len(a), len(b)
# Constructs the zeros...
zeros = [0 for _ in range(abs(sizeA-sizeB))]
# Determines whether a or b needs to be appended with 0,0,0,0...
if sizeA < sizeB:
a += zeros
else:
b += zeros
print(a,b)
You should use extend instead of append. This is the way to add a list to another list in Python. The list here is the list of zeros.
a = [3, 6, 4, 5, 9, 3]
b = [8, 3, 5, 2]
lenA, lenB = len(a), len(b)
diff=abs(len(a)-len(b))
if lenA < lenB:
a.extend([0]*diff)
else:
b.extend([0]*diff)
print(a)
print(b)
You could also try to use more_itertools padded() method:
It's prob. more elegant and adaptable for future Use cases.
Notes: just need to do pip install more_itertools first.
# simple example to demo it:
from more_itertools import padded
print(list(padded([1, 2, 3], 0, 5))) # last num: 5 is the numbers of 0 to be padded to make the total length to be 5. (needs 2 zeros)
# [1, 2, 3, 0, 0]
# more examples:
>>> L = [1, 2, 3]
>>> K = [3, 4, 5, 6, 8, 9]
>>> gap = len(K) - len(L)
# 3
# shorter list is L
>>>list(padded(L, 0, len(L) + gap))
[1, 2, 3, 0, 0, 0]
Suppose I have the following tensor: y = torch.randint(0, 3, (10,)). How would you go about counting the 0's 1's and 2's in there?
The only way I can think of is by using collections.Counter(y) but was wondering if there was a more "pytorch" way of doing this. A use case for example would be when building the confusion matrix for predictions.
You can use torch.unique with the return_counts option:
>>> x = torch.randint(0, 3, (10,))
tensor([1, 1, 0, 2, 1, 0, 1, 1, 2, 1])
>>> x.unique(return_counts=True)
(tensor([0, 1, 2]), tensor([2, 6, 2]))
I would like to modify a list that has 3 different numbers [0, 1, 2]
Each 0 should be replaced with either the last 1 or 2 value depending on which was most recent during the iteration.
Is it possible to create the new list using a list comprehension?
I know I can use a for loop and just record the last 1 or 2 and append the values to the new list but I prefer the most pythonic way.
list = [1, 1, 2, 1, 0, 0, 1, 0, 2, 0, 0]
new_list = [1, 1, 2, 1, 1, 1, 1, 2, 2, 2]
I was using this but then realised that after 2 0s in a sequence it would start recording 0s again.
new_list = [list[index-1] if list[index] == 0 else value for index,value in enumerate(list)]
Starting in python 3.8 you now have the walrus operator := which can assign values as part of an expression and works in list comprehensions. You just need to decide what the first value will be if the list starts with 0 since there is no previous value:
alist = [1, 1, 2, 1, 0, 0, 1, 0, 2, 0, 0]
j = 0
[j:= i if i else j for i in alist]
# [1, 1, 2, 1, 1, 1, 1, 1, 2, 2, 2]
Perform that just with a list comprehension could be a little weird, so here my solution (without creating a new list):
my_list = [1, 1, 2, 1, 0, 0, 1, 0, 2, 0, 0]
last_not_zero = 0
for index, number in enumerate(my_list):
if number!=0:
last_not_zero = number
else:
my_list[index] = last_not_zero
print(my_list)
And you'll get:
[1, 1, 2, 1, 1, 1, 1, 1, 2, 2, 2]
If you have a large list or you are using Pandas in your code,
import pandas as pd
s = pd.Series(list)
s.replace(0, pd.np.nan).ffill().to_list()
Output
[1, 1, 2, 1, 1, 1, 1, 1, 2, 2, 2]
NOTE: If you are not using pandas/numpy in your code, then basic for loop would be the best way to do this. Advise using the above code only for large arrays with multiple manipulations.
How about this
last_value = 0
new_list = []
for value in list:
if value != 0:
last_value = value
new_list.append(value)
else:
new_list.append(last_value)
Depends on the condition.
In most cases, you can place the condition within the while loop in the if-else criteria statements.
If, say, you want to replace the content of a list if the previous value was
something,
size = len(list)
while True:
index = index + 1
if index!=0 and index<size:
if list[index-1]==something:
list[index] = value to be replaced
if index == (size-1):
a = 0
for a in (0, (size-1)):
print(a,":",list(a))
break
Could someone help me how to perform bitwise AND operations on two tensors in Pytorch 1.4?
Apparently I could only find NOT and XOR operations in official document
I don't see them in the docs, but it looks like &, |, __and__, __or__, __xor__, etc are bit-wise:
>>> torch.tensor([1, 2, 3, 4]).__xor__(torch.tensor([1, 1, 1, 1]))
tensor([0, 3, 2, 5])
>>> torch.tensor([1, 2, 3, 4]) | torch.tensor([1, 1, 1, 1])
tensor([1, 3, 3, 5])
>>> torch.tensor([1, 2, 3, 4]) & torch.tensor([1, 1, 1, 1])
tensor([1, 0, 1, 0])
>>> torch.tensor([1, 2, 3, 4]).__and__(torch.tensor([1, 1, 1, 1]))
tensor([1, 0, 1, 0])
See https://github.com/pytorch/pytorch/pull/1556
Check this. There is no bitwise and/or operation for tensors in Torch. There are element-wise operations implemented in Torch, but not bite-wise ones.
However, if you could convert each bit as a separate Tensor dimension, you can use element-wise operation.
For an example,
a = torch.Tensor{0,1,1,0}
b = torch.Tensor{0,1,0,1}
torch.cmul(a,b):eq(1)
0
1
0
0
[torch.ByteTensor of size 4]
torch.add(a,b):ge(1)
0
1
1
1
[torch.ByteTensor of size 4]
Hope this will help you.
why is the output same every time?
a = torch.tensor([0, 1, 2, 3, 4])
a[-2:] = torch.tensor([[[5, 6]]])
a
tensor([0, 1, 2, 5, 6])
a = torch.tensor([0, 1, 2, 3, 4])
a[-2:] = torch.tensor([[5, 6]])
a
tensor([0, 1, 2, 5, 6])
a = torch.tensor([0, 1, 2, 3, 4])
a[-2:] = torch.tensor([5, 6])
a
tensor([0, 1, 2, 5, 6])
Pytorch is following Numpy here which allows assignment to slices as long as the shapes are compatible meaning that the two sides have the same shape or the right hand side is broadcastable to the shape of the slice. Starting with trailing dimensions, two arrays are broadcastable if they only differ in dimensions where one of them is 1. So in this case
a = torch.tensor([0, 1, 2, 3, 4])
b = torch.tensor([[[5, 6]]])
print(a[-2:].shape, b.shape)
>> torch.Size([2]) torch.Size([1, 1, 2])
Pytorch will perform the following comparisons:
a[-2:].shape[-1] and b.shape[-1] are equal so the last dimension is compatible
a[-2:].shape[-2] does not exist, but b.shape[-2] is 1 so they are compatible
a[-2:].shape[-3] does not exist, but b.shape[-3] is 1 so they are compatible
All dimensions are compatible, so b can be broadcasted to a
Finally, Pytorch will convert b to tensor([5, 6]) before performing the assignment thus producing the result:
a[-2:] = b
print(a)
>> tensor([0, 1, 2, 5, 6])