My input in a tensorflow graph comes as a vector which contains multiple overlapping windows. How can I create this array using only tensorflow operations?
input = [1,2,3,4,5,6,7,8]
shift = 2
window_width = 4
count = (len(input) - window_width) // 2 + 1 = 3
output = [[1,2,3,4],
[3,4,5,6],
[5,6,7,8]]
In numpy I would use stride_tricks, but something similar isn't available in tensorflow. How should I approach this?
TensorFlow doesn't have stride_tricks. The following would work for your particular use case.
>>> b=tf.concat(0, [tf.reshape(input[i:i+4], [1, window_width]) for i in range(0, len(input) - window_width + 1, shift)])
>>> with tf.Session(""): b.eval()
...
array([[1, 2, 3, 4],
[3, 4, 5, 6],
[5, 6, 7, 8]], dtype=int32)
If your input is large, you may also want to look at slice_input_producer.
In case somebody still needs this, there is a function tf.signal.frame().
For the example given in the question, the code would be:
output = tf.signal.frame(input, window_width, shift)
You can use tf.map_fn() to accomplish this:
input = [1,2,3,4,5,6,7,8]
shift = 2
window_width = 4
limit = len(input) - window_width + 1
input_tensor = tf.placeholder(tf.int32, shape=(8,))
output_tensor = tf.map_fn(lambda i: input_tensor[i:i+window_width], elems=tf.range(start=0, limit=limit, delta=shift))
with tf.Session() as sess:
answer_test = sess.run(output_tensor, feed_dict = {input_tensor:input})
print(answer_test)
: [[1 2 3 4]
[3 4 5 6]
[5 6 7 8]]
Related
I am trying to roll only the first n elements from my numpy axis instead of all. However, I am at a loss on how to accomplish this.
import numpy as np
foo = np.random.rand(32,3,16,16)
#Foo is a batch of 32 images, with 3 channels and a height, width of 16
print("Foo Shape = ", foo.shape)
#Foo Shape = (32, 3, 16, 16)
I would like to roll each first element of the second axis by 1 step. Basically roll the first channel of each image by 1.
np.roll(foo, 1, 1)
The code above rolls all the elements of the second axis (channel dimension) by 1, instead of just rolling the first element. I couldn't find any numpy functionality that helps with this issue.
Select only the elements you want using a 2D slice:
>>> import numpy as np
>>> arr = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
>>> print(arr)
[[[1 2]
[3 4]]
[[5 6]
[7 8]]]
>>> arr[:, 0] = np.roll(arr[:, 0], shift=1, axis=1)
>>> print(arr)
[[[2 1]
[3 4]]
[[6 5]
[7 8]]]
>>>
I begin with an n number of arrays. I'm going to define a few as an example.
X = ([1,2,3,4})
Y = ([5,6,7,8])
Z = ([3,6,7,8])
How do I create another array that will have each of my beginning arrays (X, Y, and Z) as columns? The final array will look like this:
1 5 3
2 6 6
3 7 7
4 8 8
Perhaps you can try like this:
import numpy as np
X = np.array([1,2,3,4])
Y = np.array([5,6,7,8])
Z = np.array([3,6,7,8])
a = np.concatenate((X[:,np.newaxis],Y[:,np.newaxis],Z[:,np.newaxis]),axis=1)
I'm not big fan of third party libraries, so this is an answer that doesn't use numpy:
>>> X = ([1,2,3,4])
>>> Y = ([5,6,7,8])
>>> Z = ([3,6,7,8])
# timeit.timeit 0.30220915000000004
>>> output = [[X[i], Y[i], Z[i]] for i in range(len(X))]
# timeit.timeit 1.677441058
>>> print(output)
[[1, 5, 3], [2, 6, 6], [3, 7, 7], [4, 8, 8]]
>>> columns = "\n".join([" ".join([str(n) for n in i]) for i in output])
# timeit.timeit 5.729952549999999
>>> print(columns)
1 5 3
2 6 6
3 7 7
4 8 8
>>> for i in range(len(X)):
... print(X[i], Y[i], Z[i])
1 5 3
2 6 6
3 7 7
4 8 8
# timeit.timeit 1.2027191299999984 without print
Using numpy:
>>> X = np.array([1,2,3,4])
>>> Y = np.array([5,6,7,8])
>>> Z = np.array([3,6,7,8])
# timeit.timeit 4.819555767999999
>>> output = np.concatenate((X[:, np.newaxis], Y[:, np.newaxis], Z[:, np.newaxis]), axis=1)
# timeit.timeit 4.175194263
>>> print(output)
[[1 5 3]
[2 6 6]
[3 7 7]
[4 8 8]]
>>> columns = "\n".join([" ".join([str(n) for n in i]) for i in output])
# timeit.timeit 22.564187487
>>> print(columns)
1 5 3
2 6 6
3 7 7
4 8 8
On comments (#) I wrote the time it takes to execute each section of the code using timeit.timeit so you could take your own conclusions.
There is an in-built function called column_stack for this purpose.
np.column_stack((X, Y, Z))
# array([[1, 5, 3],
# [2, 6, 6],
# [3, 7, 7],
# [4, 8, 8]])
Alternatively you can use vstack and then transpose
arr = np.vstack((X,Y,Z)).T
I have two tensors - one with bin specification and the other one with observed values. I'd like to count how many values are in each bin.
I know how to do this in either NumPy or bare Python, but I need to do this in pure TensorFlow. Is there a more sophisticated version of tf.histogram_fixed_width with an argument for bin specification?
Example:
# Input - 3 bins and 2 observed values
bin_spec = [0, 0.5, 1, 2]
values = [0.1, 1.1]
# Histogram
[1, 0, 1]
This seems to work, although I consider it to be quite memory- and time-consuming.
import tensorflow as tf
bins = [-1000, 1, 3, 10000]
vals = [-3, 0, 2, 4, 5, 10, 12]
vals = tf.constant(vals, dtype=tf.float64, name="values")
bins = tf.constant(bins, dtype=tf.float64, name="bins")
resh_bins = tf.reshape(bins, shape=(-1, 1), name="bins-reshaped")
resh_vals = tf.reshape(vals, shape=(1, -1), name="values-reshaped")
left_bin = tf.less_equal(resh_bins, resh_vals, name="left-edge")
right_bin = tf.greater(resh_bins, resh_vals, name="right-edge")
resu = tf.logical_and(left_bin[:-1, :], right_bin[1:, :], name="bool-bins")
counts = tf.reduce_sum(tf.to_float(resu), axis=1, name="count-in-bins")
with tf.Session() as sess:
print(sess.run(counts))
I have a numpy array X, and I'd like to return another array Y whose entries are the indices of the n largest values of X i.e. suppose I have:
a =np.array[[1, 3, 5], [4, 5 ,6], [9, 1, 7]]
then say, if I want the first 5 "maxs"'s indices-here 9, 7 , 6 , 5, 5 are the maxs, and their indices are:
b=np.array[[2, 0], [2 2], [ 2 1], [1 1], [0 , 2])
I've been able to find some solutions and make this work for a one dimensional array like
c=np.array[1, 2, 3, 4, 5, 6]:
def f(a,N):
return np.argsort(a)[::-1][:N]
But have not been able to generate something that works in more than one dimension. Thanks!
Approach #1
Get the argsort indices on its flattened version and select the last N indices. Then, get the corresponding row and column indices -
N = 5
idx = np.argsort(a.ravel())[-N:][::-1] #single slicing: `[:N-2:-1]`
topN_val = a.ravel()[idx]
row_col = np.c_[np.unravel_index(idx, a.shape)]
Sample run -
# Input array
In [39]: a = np.array([[1,3,5],[4,5,6],[9,1,7]])
In [40]: N = 5
...: idx = np.argsort(a.ravel())[-N:][::-1]
...: topN_val = a.ravel()[idx]
...: row_col = np.c_[np.unravel_index(idx, a.shape)]
...:
In [41]: topN_val
Out[41]: array([9, 7, 6, 5, 5])
In [42]: row_col
Out[42]:
array([[2, 0],
[2, 2],
[1, 2],
[1, 1],
[0, 2]])
Approach #2
For performance, we can use np.argpartition to get top N indices without keeping sorted order, like so -
idx0 = np.argpartition(a.ravel(), -N)[-N:]
To get the sorted order, we need one more round of argsort -
idx = idx0[a.ravel()[idx0].argsort()][::-1]
I am using Python3 and numpy with matplotlib on a project to get Jupiter's Mass from observational telescope astrometry. I want to take an array of numbers, say from 1 to 10, and multiply only a few of them in order, say 1 to 4, by -1.
So 1 to 4 is now negative and 5 to 10 is still positive. I imagine out put like this:
L = [1,2,3,4,5,6,7,8,9,10]
array_L = np.array(L)
>>>array_L
array([1,2,3,4,5,6,7,8,9,10])
neg = array_L[0:4]
>>>neg
array([1,2,3,4])
Neg = neg * -1
>>>Neg
array([-1,-2,-3,-4])
Now I need a way of combining neg and array_L into a new final array that would output like this:
# pseudo code: Neg + array_L(minus elements 0 to 4) = New_L
New_L = array([-1,-2,-3,-4, 5, 6, 7, 8, 9, 10])
Also I know it may be possible to do a limited element iteration over just the elements I want and not the whole array. I can do some of these operations on the list vs the array if it would make it easier.
You are almost there! Try this:
L = array([1,2,3,4,5,6,7,8,9,10])
L[0:4] *= -1
print(L)
Just like with regular python lists, you can do operations on slices of NumPy arrays to change them in-place:
>>> import numpy
>>> L = [1,2,3,4,5,6,7,8,9,10]
>>> array_L = numpy.array(L)
>>> array_L
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
>>> array_L[0:4] *= -1
>>> array_L
array([-1, -2, -3, -4, 5, 6, 7, 8, 9, 10])