I have a list with 10 numerical values. I want to return all possible combination of this list such that each element can take value +/- element value.
The approach I had in mind was to take a binary variable which takes in value from 0 to 1023. 1 in this variable corresponds to positive d[i] and 0 to negative d[i].
e.g. for bin(8) = 0000001000 implies that d7 will take value -d7 and rest will be positive. Repeat this for all 0 to 1023 to get all combinations.
For example, if D = [d1,d2,...d10], we will have 1024 (2^10) combinations such that:
D1 = [-d1,d2,d3,....d10]
D2 = [-d1,-d2,d3,....d10]
D3 = [d1,-d2,d3,....d10] ...
D1024 = [-d1,-d1,-d3,....-d10]
Thank You!
you can just use the builtin itertools.product to make all combinations of positive and negative values.
from itertools import product
inputs = list(range(10)) # [1,2,3,4,5,6,7,8,9]
outputs = []
choices = [(x,-x) for x in inputs]
for item in product(*choices):
outputs.append(item)
print(outputs[:3])
print(len(outputs))
# [(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 7, 8, -9), (0, 1, 2, 3, 4, 5, 6, 7, -8, 9)]
# 1024
in a compressed form:
outputs = [item for item in product(*[(x,-x) for x in inputs])]
Related
This is a pretty specific usage case, but I'm hoping someone out there is more familiar with PyTorch tensors than I am and can help me speed this up.
I'm working on implementing a custom similarity metric for a neural network and have successfully gotten it to work, but it is incredibly slow to calculate. Each epoch takes about a minute to run, which simply isn't going to work with how I wanted to compare it with other metrics. So, I've been trying to utilize PyTorch tensors more effectively to speed things up, but haven't had much success.
Basically, I need to sum up the integers in the 'counts' tensor between the min and max indices specified in the 'min' and 'max' tensors for each sample and cluster combination.
As mentioned, my original implementation using loops took about a minute per epoch to run, but I did manage to reduce that to about 18-20 seconds using list comprehensions:
# counts has size (16, 100), max and min have size (2708, 7, 16)
data_mass = torch.sum(torch.tensor([[[torch.pow(torch.sum(counts[k][min[i][j][k]:max[i][j][k]+1]) / divisor, 2) for k in range(len(counts))] for j in range(len(min[i]))] for i in range(len(min))]), 2)
This feels super janky, and I've seen some clever things done with PyTorch functions, but I haven't been able to find anything yet that addresses quite what I want to do. Thanks in advance! I'm happy to clarify anything that may not be clear, I understand the use case is a bit convoluted.
EDIT: I'll try and break down the code snippet above and provide a minimal example. Examples of minimal inputs might look like the following:
'min' and 'max' are both 3-dimensional tensors of shape (num_samples, num_clusters, num_features), such as this one of size (2, 3, 4)
min = tensor([[[1, 2, 3, 1],
[2, 1, 1, 2],
[1, 2, 2, 1]],
[[2, 3, 2, 1],
[3, 3, 1, 2],
[1, 0, 2, 1]]])
max = tensor([[[3, 3, 4, 4],
[3, 2, 3, 4],
[2, 4, 3, 2]],
[[4, 4, 3, 3],
[4, 4, 2, 3],
[2, 1, 3, 2]]])
'counts' is a 2-dimensional tensor of size(num_features, num_bins),
so for this example we'll say size (4, 5)
counts = tensor([[1, 2, 3, 4, 5],
[2, 5, 3, 1, 1],
[1, 2, 3, 4, 5],
[2, 5, 3, 1, 1]])
The core part of the code snippet given above is the summation of the counts tensor between the values given by the min and max tensors for each pair of indices given at each index in max/min. For the first sample/cluster combo above:
mins = [1, 2, 3, 1]
maxes = [3, 3, 4, 4]
#Starting with feature #1 (leftmost element of min/max, top row of counts),
we sum the values in counts between the indices specified by min and max:
min_value = mins[0] = 1
max_value = maxes[0] = 3
counts[0] = [1, 2, 3, 4, 5]
subset = counts[0][mins[0]:maxes[0]+1] = [2, 3, 4]
torch.sum(subset) = 9
#Second feature
min_value = mins[1] = 2
max_value = maxes[1] = 3
counts[1] = [2, 5, 3, 1, 1]
subset = counts[0][mins[0]:maxes[0]+1] = [3, 1]
torch.sum(subset) = 4
In my code snippet, I perform a few additional operations, but if we ignore those and just sum all the index pairs, the output will have the form
pre_sum_output = tensor([[[9, 4, 9, 10],
[7, 8, 9, 5]
[5, 5, 7, 8]],
[[12, 2, 7, 9],
[9, 2, 5, 4],
[5, 7, 7, 8]]])
Finally, I sum the output one final time along the third dimension:
data_mass = torch.sum(pre_sum_output, 2) = torch.tensor([[32, 39, 25],
[30, 20, 27]])
I then need to repeat this for every pair of mins and maxes in 'min' and 'max' (each [i][j][k]), hence the list comprehension above iterating through i and j to get each sample and cluster respectively.
By noticing that torch.sum(counts[0][mins[0]:maxes[0]+1]) is equal to cumsum[maxes[0]] - cumsum[mins[0]-1] where cumsum = torch.cumsum(counts[0]), you can get rid of the loops like so:
# Dim of sample, clusters, etc.
S, C, F, B = range(4)
# Copy min and max over bins
min = min.unsqueeze(B)
max = max.unsqueeze(B)
# Copy counts over samples and clusters
counts = counts.reshape(
1, # S
1, # C
*counts.shape # F x B
)
# Number of samples, clusters, etc.
ns, nc, nf, nb = min.size(S), min.size(C), min.size(F), counts.size(B)
# Calculate cumulative sum and copy over samples and clusters
cum_counts = counts.cumsum(dim=B).expand(ns, nc, nf, nb)
# Prevent index error when min index is 0
is_zero = min == 0
lo = (min - 1).masked_fill(is_zero, 0)
# Compute the contiguous sum from min to max (inclusive)
lo_sum = cum_counts.gather(dim=B, index=lo)
hi_sum = cum_counts.gather(dim=B, index=max)
sum_counts = torch.where(is_zero, hi_sum, hi_sum - lo_sum)
pre_sum_output = sum_counts.squeeze(B)
You can then sum over the 2nd dim to get data_mass.
I have a list like this:
my_list = [-1, 5, 6, -7, -3, 7, 9, -8, 4, -12, ....., N]
Using Python 3.x, I would like to sum numbs one by one and check at each step if the sum is equal to zero. If it is not continue, otherwise break and store values contributing to the sum (when sum = 0) in a new list.
For now, I'm not considering problems regarding performance.
Can you please help me?
Really appreciate your help!
If I understood correctly, you want to sum up your items until the sum is different from 0 and also save this numbers in another list.
You can do it like this
my_list = [-1, 5, 6, -7, -3, 7, 9, -8, 4, -12]
numbers_that_sum_zero = []
total = 0
for i in range(len(my_list)):
total += my_list[i]
numbers_that_sum_zero.append(my_list[i])
if total == 0:
break
print(numbers_that_sum_zero)
This will return
[-1, 5, 6, -7, -3]
An option using numpy:
import numpy as np
my_list = [-1, 5, 6, -7, -3, 7, 9, -8, 4, -12]
a = np.array(my_list)
result = a[:a.cumsum().tolist().index(0)+1]
# or result = a[:np.where(a.cumsum()==0)[0][0]+1]
Output:
array([-1, 5, 6, -7, -3])
This returns the first subset of number in which the sum is 0. If there's no values in the list that sums to 0 at all, it raises an error.
You can handle it to return NaN or any other output with:
try:
result = a[:a.cumsum().tolist().index(0)+1]
except:
result = np.nan
I have to multiply the same index of two lists and then find the sum of that.
Please help me out! Thank you.
Try this,
>>> A=[2, 3, -6, 7, 10, 11]
>>> B=[1, 2, 3, 4, 5, 6]
>>> sum([x * y for x, y in zip(A, B)])
134
Let me explain what I did in my answer, I used zip() python Built-in Function and this is what documentation mention about it.
Make an iterator that aggregates elements from each of the iterables.
Returns an iterator of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The iterator stops when the shortest input iterable is exhausted. With a single iterable argument, it returns an iterator of 1-tuples. With no arguments, it returns an empty iterator.
Little bit confusing, right? Check the below example:-
>>> A=[2, 3, -6, 7, 10, 11]
>>> B=[1, 2, 3, 4, 5, 6]
>>> zip(A,B)
<zip at 0x1fde73e6f88> # it display something like this (zip object)
>>> list(zip(A,B)) # for visualization purpose, convert zip object to list
[(2, 1), (3, 2), (-6, 3), (7, 4), (10, 5), (11, 6)]
I think you can get clear idea what happen inside zip() function. Then multiply each and every value in zip object using python List Comprehensions to answer your question more pythonic. So zip object now created for us a new value series using A and B list. We assigned those values for x and y, multiply them and save in list.
>>> [x * y for x, y in zip(A, B)]
[2, 6, -18, 28, 50, 66]
After all the steps we used sum() to calculate the Sum a list of numbers in Python.
>>> sum([2, 6, -18, 28, 50, 66])
134
That's all, if you didn't get anything please add a comment to this answer.
AB = [A[i] * B[i] for i in range(len(A))]
sum(AB)
Alternatively, try
AB = [value_ * B[i] for i, value_ in enumerate(A)]
sum(AB)
I believe you want this:
def lists(A,B):
C = 0
for i in range(len(A)):
C += (A[i] * B[i])
return C
Now you can call your method lists with lists A and B like this:
A=[2, 3, -6, 7, 10, 11]
B=[1, 2, 3, 4, 5, 6]
lists(A,B)
Which will return 134. Your code was wrong because of your indentation. You had put your return statement inside the for loop, so your code would return C value in the first iteration, which was 0 + 2*1.
with list comprehension:
A = [2, 3, -6, 7, 10, 11]
B = [1, 2, 3, 4, 5, 6]
print (sum([A[i]*B[i] for i in range(len(A))]))
output:
134
your code:
def lists(A, B):
C = 0
for i in range(len(A)):
C += (A[i] * B[i])
return C # <-----
A = [2, 3, -6, 7, 10, 11]
B = [1, 2, 3, 4, 5, 6]
print (lists(A,B))
NOTE: you need to put your return statement out of the for loop.
If return statement is reached during the execution of a function, it will exit from function, what does it mean in your case if you have return in your for loop, in first iteration return function will be reached and exit (you got result 2 because in first iteration yuo have 2*1)
I have two equally long 1D arrays: "Number_of_Data" and "Value". With these two arrays, I would like to create a third array ("Final_Array") where each of the element values in "Value" appears as many times as specified by the corresponding element value in "Number_of_Data".
Let me give you an example:
import numpy as np
# Create the Arrays
Number_of_Data = np.asarray([2, 4, 1, 2, 3, 6, 3])
Value = np.arange(0, 7, 1)
# Then, somehow, I want Final_Array to be:
Final_Array
array([0, 0, 1, 1, 1, 1, 2, 3, 3, 4, 4, 4, 5, 5, 5, 5, 5, 5, 6, 6, 6])
Is there a python function to accomplish this? (In reality, my arrays are of course much longer than the length of "Value" and the maximum number of data is much higher than the highest value in "Number_of_Data")
I am not aware of any numpy integrated function. The easiest approach is to create a simple function that takes the two arrays and gives you back the desired vector exploiting two for loops.
import numpy as np
def final(numbers, values):
n_values = len(values)
vector = np.zeros(np.sum(numbers))
counter = 0
for k in range(n_values):
for j in range(numbers[k]):
vector[counter] = values[k]
counter += 1
return vector
Number_of_Data = np.asarray([2, 4, 1, 2, 3, 6, 3])
Value = np.arange(0, 7, 1)
Final_Array = final(Number_of_Data, Value)
print(Final_Array)
This gives back
[ 0. 0. 1. 1. 1. 1. 2. 3. 3. 4. 4. 4. 5. 5. 5. 5. 5. 5. 6. 6. 6.]
I am trying to write a python 3 function that finds all numbers in a list (unspecified length) that are not part of a pair.
For example, given the list [1, 2, 1, 3, 2], the function will return 3; and given the list [0, 1, 1, 7, 8, 3, 9, 3, 9], the function will return 0, 7, and 8.
Thanks for your help!
You can use the following function :
>>> def find(l):
... return (i for i in l if l.count(i)==1)
>>> l= [0, 1, 1, 7, 8, 3, 9, 3, 9]
>>> list(find(l))
[0, 7, 8]
This function will return a generator that is contain the elements in list which those count is equal to 1.
I can tell you how I would do it. What does it mean a "pair"?
You should say, find all the numbers repeated oddly in the array.
First plan: (more efficient!)
Sort the list and then a single loop through your list should be enough to find how many numbers of each there are inside and you can generate awhile another list that you will return.
Second plan (nicer in python, but also more expensive because of the number of evaluations though the hole list):
Try the solution of Kasra. 'count' function from 'list' type helps our code but not our efficiency. It counts the number of times that appears the value 'i' on the list 'l', obviously.
If the pair need to be "closed pair" I mean, if you have three 1 (ones), do you have one pair and one single 1? or do you have all the 1 paired? If the second one, the solution of Kasra is Ok. Else you should compare:
if l.count(i) % 2 == 1
This can be easily and efficiently done in 3 lines with collections.Counter.
from collections import Counter
def unpaired(numbers):
for key, count in Counter(numbers).items():
if count % 2:
yield key
print(list(unpaired([1, 2, 1, 3, 2])))
# [3]
print(list(unpaired([0, 1, 1, 7, 8, 3, 9, 3, 9])))
# [0, 7, 8]
My answer comport if you have three equals numbers or if you have one pair and one single number without pair.
def name(array):
o = sorted(array)
c = []
d = []
for i in o:
if o.count(i) % 2 == 1:
c.append(i)
for j in c:
if j not in d:
d.append(j)
return d
or do not use for j in c and use directly:
return list(set(c))
for example:
array = [0, 1, 1, 7, 8, 3, 9, 3, 9, 9]
output: [0, 7, 8, 9]