Troubles with simple accumulation with Theano scan function - theano

I am trying to accumulate matrix values while multiplying with the step:
res = sum_i(i * a). My code looks like this:
import numpy as np
from theano import function, scan
import theano.tensor as T
x = T.lmatrix()
results, updates = scan(
lambda res, step, x: res + step * x,
non_sequences=x,
sequences=T.arange(2),
outputs_info=T.zeros_like(x))
f = function([x], results)
a = np.array([[0, 0], [2, 2]], 'int64')
print(f(a))
This outputs:
[[[0 0]
[0 0]]
[[1 1]
[1 1]]]
While I would expect this:
[[[0 0]
[0 0]]
[[0 0]
[2 2]]]


The output is (perhaps unsurprising?) correct. The reason you get this output is as follows:
In the first iteration,
res = 0
step = [[0, 0], [0, 0]]
x = [[0, 0], [2, 2]]
and so
res + step * x = 0 + [[0, 0], [0, 0]] * [[0, 0], [2, 2]]
= 0 + [[0, 0], [0, 0]]
= [[0, 0], [0, 0]]
In the second iteration,
res = 1
step = [[0, 0], [0, 0]]
x = [[0, 0], [2, 2]]
and so
res + step * x = 1 + [[0, 0], [0, 0]] * [[0, 0], [2, 2]]
= 1 + [[0, 0], [0, 0]]
= [[1, 1], [1, 1]]
Note that the 1 is broadcasted to same shape as the matrix that results from the elementwise multiplication of step and x.

Related

Multiclass vs. multilabel fitting

In scikit-learn tutorials, I found the following paragraphs in the section 'Multiclass vs. multilabel fitting'.
I couldn't understand why the following codes generate the given results.
First
from sklearn.svm import SVC
from sklearn.multiclass import OneVsRestClassifier
from sklearn.preprocessing import LabelBinarizer
X = [[1, 2], [2, 4], [4, 5], [3, 2], [3, 1]]
y = [0, 0, 1, 1, 2]
classif = OneVsRestClassifier(estimator=SVC(random_state=0))
classif.fit(X, y).predict(X)
array([0, 0, 1, 1, 2])
y = LabelBinarizer().fit_transform(y)
classif.fit(X, y).predict(X)
array([[1, 0, 0],
[1, 0, 0],
[0, 1, 0],
[0, 0, 0],
[0, 0, 0]])
Next
from sklearn.preprocessing import MultiLabelBinarizer
y = [[0, 1], [0, 2], [1, 3], [0, 2, 3], [2, 4]]
y = MultiLabelBinarizer().fit_transform(y)
classif.fit(X, y).predict(X)
array([[1, 1, 0, 0, 0],
[1, 0, 1, 0, 0],
[0, 1, 0, 1, 0],
[1, 0, 1, 0, 0],
[1, 0, 1, 0, 0]])

Label binarization in scikit-learn will transform your targets and represent them in a label indicator matrix. This label indicator matrix has the shape (n_samples, n_classes) and is composed as follows:
each row represents a sample
each column represents a class
each element is 1 if the sample is labeled with the class and 0 if not
In your first example, you have a target collection with 5 samples and 3 classes. That's why transforming y with LabelBinarizer results in a 5x3 matrix. In your case, [1, 0, 0] corresponds to class 0, [0, 1, 0] corresponds to class 1 and so forth. Notice that in each row there is only one element set to 1, since each sample can have one label only.
In your next example, you have a target collection with 5 samples and 5 classes. That's why transforming y with MultiLabelBinarizer results in a 5x5 matrix. In your case, [1, 1, 0, 0, 0] corresponds to the multilabel [0, 1], [0, 1, 0, 1, 0] corresponds to the multilabel [1, 3] and so forth. The key difference to the first example is that each row can have multiple elements set to 1, because each sample can have multiple labels/classes.
The predicted values you get follow the very same pattern. They are however not equivalent to the original values in y since your classification model has obviously predicted different values. You can check this with the inverse_transform() of the binarizers:
from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
y = np.array([[0, 1], [0, 2], [1, 3], [0, 2, 3], [2, 4]])
y_bin = mlb.fit_transform(y)
# direct transformation
[[1 1 0 0 0]
[1 0 1 0 0]
[0 1 0 1 0]
[1 0 1 1 0]
[0 0 1 0 1]]
# prediction of your classifier
y_pred = np.array([[1, 1, 0, 0, 0],
[1, 0, 1, 0, 0],
[0, 1, 0, 1, 0],
[1, 0, 1, 0, 0],
[1, 0, 1, 0, 0]])
# inverting the binarized values to the original classes
y_inv = mlb.inverse_transform(y_pred)
# output
[(0, 1), (0, 2), (1, 3), (0, 2), (0, 2)]

Why Python 2D array access modifies the whole column

I had a confusion where I initialize a 2D array in this way:
>>> a = [[0] * 3] * 3
>>> a
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> a[0][0] = 1
>>> a
[[1, 0, 0], [1, 0, 0], [1, 0, 0]]
while I was expecting it to be
[[1, 0, 0], [0, 0, 0], [0, 0, 0]]
I also tried
>>> a = [[0 for _ in range(3)] for _ in range(3)]
>>> a
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
>>> a[0][0] = 1
>>> a
[[1, 0, 0], [0, 0, 0], [0, 0, 0]]
Which worked as expected
Wonder what is the reason caused this?

With the outter *3 you are making a shallow copy of a list.
This can be easily verifier by printing ids. An id is unique to each item.
a = [[0] * 3] * 3
print(*map(id, a))
# Same ID
Instead, you absolutely need to generate new lists
a = [[0] * 3 for _ in range(3)]
print(*map(id, a))
# Different ids
If you want more information you can check this question : What exactly is the difference between shallow copy, deepcopy and normal assignment operation?

Split list into sublists by delimiter

I have a list of lists:
[[0, 0], [0, 0], [0, 0], [0, 1, 0], [0, 0]]
I want to split it into what comes before the list [0,1,0] and what comes after like so:
[[0, 0], [0, 0], [0, 0]], [[0, 0]]
If I had a list:
[[0, 0], [0, 0], [0, 0], [0, 1, 0], [0, 0], [0, 1, 0], [0, 0]]
I would want to split it into a list like this:
[[0, 0], [0, 0], [0, 0]], [[0, 0]], [[0, 0]]
I am really stuck with this while loop, which does not seem to reset the temporary list at the right place:
def count_normal_jumps(jumps):
_temp1 = []
normal_jumps = []
jump_index = 0
while jump_index <= len(jumps) - 1:
if jumps[jump_index] == [0,0]:
_temp1.append(jumps[jump_index])
else:
normal_jumps.append(_temp1)
_temp1[:] = []
jump_index += 1
return normal_jumps
Why does this not work and is there a better approach?

You can use a for loop to append the sublists in the list to the last sublist in a list of lists, and append a new sublist to the list of lists when the input sublist is equal to [0, 1, 0]:
def split(lst):
output = [[]]
for l in lst:
if l == [0, 1, 0]:
output.append([])
else:
output[-1].append(l)
return output
or you can use itertools.groupby:
from itertools import groupby
def split(lst):
return [list(g) for k, g in groupby(lst, key=[0, 1, 0].__ne__) if k]
so that:
print(split([[0, 0], [0, 0], [0, 0], [0, 1, 0], [0, 0]]))
print(split([[0, 0], [0, 0], [0, 0], [0, 1, 0], [0, 0], [0, 1, 0], [0, 0]]))
outputs:
[[[0, 0], [0, 0], [0, 0]], [[0, 0]]]
[[[0, 0], [0, 0], [0, 0]], [[0, 0]], [[0, 0]]]

You can do something like this:
myList = [[0, 0], [0, 0], [0, 0], [0, 1, 0], [0, 0]]
toMatch = [0, 1, 0]
allMatches = []
currentMatches = []
for lst in myList:
if lst == toMatch:
allMatches.append(currentMatches)
currentMatches = []
else:
currentMatches.append(lst)
#push leftovers when end is reached
if currentMatches:
allMatches.append(currentMatches)
print(allMatches)

Elements in a list are overwritten

I tried to program a function which creates the linear span of a list of independent vectors, but it seems that the last calculated vector overwrites all other elements. I'd be nice if someone could help me fixing it.
def span_generator(liste,n):
"""function to generate the span of a list of linear independent
vectors(in liste) in the n-dimensional vectorspace of a finite
field with characteristic 2, returns a list of all elements which
lie inside the span"""
results=[]
blank=[]
for i in range(n):
blank.append(0)
a=blank
if len(liste)>1:
listenwert=liste[-1]
liste.pop(-1)
values=span_generator(liste,n)
for i in range(2):
for j in range(len(values)):
for k in range(n):
a[k]=(i*listenwert[k]+values[j][k])%2
results.append(a)
else:
for i in range(2):
for j in range(n):
a[j]=(i*liste[0][j])
results.append(a)
print(results)
return results
print(span_generator([[1,0],[0,1]],2)) gives following results
[[1, 0], [1, 0]]
[[1, 1], [1, 1], [1, 1], [1, 1]]
[[1, 1], [1, 1], [1, 1], [1, 1]]
instead of the expected: [[0,0],[1,0],[0,1],[1,1]]
Edit: I tried to simplify the program with itertools.product, but it didn't solve the problem.
def span_generator(liste):
n=len(liste[0])
results=[]
coeff=list(itertools.product(range(2), repeat=n))
blank=[]
for i in range(n):
blank.append(0)
for i in range(len(coeff)):
a=blank
for j in range(len(coeff[0])):
for k in range(n):
a[k]=(a[k]+coeff[i][j]*liste[j][k])%2
results.append(a)
return results
Output: span_generator([[0,1],[1,0]])
[[0, 0], [0, 0], [0, 0], [0, 0]]
But it should give [[0,0],[0,1],[1,0],[1,1]]
Another example: span_generator([[0,1,1],[1,1,0]]) should give [[0,0,0],[0,1,1],[1,1,0],[1,0,1]] (2=0 since i'm calculating modulo 2)

Coefficients
You can use itertools.product to generate the coefficients:
n = len(liste[0])
coefficients = itertools.product(range(2), repeat=len(liste))
yields an iterator with this content:
[(0, 0), (0, 1), (1, 0), (1, 1)]
Linear combinations
You can then selectively multiply the results with the transpose of your liste (list(zip(*liste)))
for coeff in coefficients:
yield [sum((a * c) for a, c in zip(transpose[i], coeff)) for i in range(n)]
which take for each dimensionality (for i in range(n)) the sum of the products
def span_generator3(liste):
n = len(liste[0])
transpose = list(zip(*liste))
coefficients = itertools.product(range(2), repeat=len(liste))
for coeff in coefficients:
yield [sum((a * c) for a, c in zip(transpose[i], coeff)) % 2 for i in range(n)]
this produces an iterator. If you want the result in a list-form, just can list() on the iterator
Result
list(span_generator3([[1,2],[4,8]]))
output:
[[0, 0], [4, 8], [1, 2], [5, 10]]
Higher dimensions
list(sorted(span_generator3([[1,2, 4],[8, 16, 32], [64, 128, 256]])))
output:
[[0, 0, 0],
[1, 2, 4],
[8, 16, 32],
[9, 18, 36],
[64, 128, 256],
[65, 130, 260],
[72, 144, 288],
[73, 146, 292]]
Modulo 2
If you want the result modulo 2, that's just adding 2 characters in the right place
def span_generator3_mod2(liste):
n = len(liste[0])
transpose = list(zip(*liste))
coefficients = itertools.product(range(2), repeat=len(liste))
# print(list(itertools.product(range(2), repeat=len(liste))))
for coeff in coefficients:
yield [sum((a * c) for a, c in zip(transpose[i], coeff)) % 2 for i in range(n)]
list(span_generator3_mod2([[0,1,1],[1,1,0]])) gives
[[0, 0, 0], [1, 1, 0], [0, 1, 1], [1, 0, 1]]

NumPy doesn't recognize well array shape

I have a code which is as follows:
data = np.array([[[i, j], i * j] for i in range(10) for j in range(10)])
print(data)
x = np.array(data[:,0])
x1 = x[:,0]
x2 = x[:,1]
print(x)
data correctly outputs [[[0,0],0],[[0,1],0],[[0,2],0],...,[[9,9],81]] which is, by the way, the multiplication table and it's results.
So, the first column of the data (which is x) must be separated into x1 and x2, which are the first and last column of it respectively. Which I think I did it right but it raises an error saying too many indices for array. What am I doing wrong?

data.dtype is object because the elements of [[i,j],k] are not homogeneous. A workaround for you :
data = np.array([(i, j, i * j) for i in range(10) for j in range(10)])
print(data)
x1 = data[:,:2]
x2 = data[:,2]
data.shape is now (100,3), data.dtype is int and x1 and x2 what you want.

Because of the mix of list lengths, this produces an object array:
In [97]: data = np.array([[[i, j], i * j] for i in range(3) for j in range(3)])
In [98]: data
Out[98]:
array([[[0, 0], 0],
[[0, 1], 0],
[[0, 2], 0],
[[1, 0], 0],
[[1, 1], 1],
[[1, 2], 2],
[[2, 0], 0],
[[2, 1], 2],
[[2, 2], 4]], dtype=object)
In [99]: data.shape
Out[99]: (9, 2)
One column contains numbers (but is still object dtype), the other lists. Both have (9,) shape
In [100]: data[:,1]
Out[100]: array([0, 0, 0, 0, 1, 2, 0, 2, 4], dtype=object)
In [101]: data[:,0]
Out[101]:
array([[0, 0], [0, 1], [0, 2], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1],
[2, 2]], dtype=object)
The easiest way of turning that column into a numeric arrays is via .tolist
In [104]: np.array(data[:,0].tolist())
Out[104]:
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[2, 2]])
In [105]: _.shape
Out[105]: (9, 2)
The [i, j, i * j] elements as suggested in the other answer are easier to work with.
A structured array approach to generating such a 'table':
In [113]: dt='(2)int,int'
In [114]: data = np.array([([i, j], i * j) for i in range(3) for j in range(3)],
...: dtype=dt)
In [115]: data
Out[115]:
array([([0, 0], 0), ([0, 1], 0), ([0, 2], 0), ([1, 0], 0), ([1, 1], 1),
([1, 2], 2), ([2, 0], 0), ([2, 1], 2), ([2, 2], 4)],
dtype=[('f0', '<i4', (2,)), ('f1', '<i4')])
In [116]: data['f0']
Out[116]:
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[2, 2]])
In [117]: data['f1']
Out[117]: array([0, 0, 0, 0, 1, 2, 0, 2, 4])

Resources