I have a tensor t of dim n x 3. When I apply torch.linalg.norm it returns one single value. What I need is a batch-wise norm function which will return a tensor with n norms, one for each vector in t.
Thanks for your help.
It seems the most relevant documentation place is:
https://pytorch.org/docs/stable/generated/torch.linalg.norm.html
In the terminal you could try: python3 and then the following python commands:
>>> from torch import linalg as LA
>>> c = torch.tensor([[1., 2., 3.],
... [-1, 1, 4]])
>>> LA.norm(c, dim=0)
tensor([1.4142, 2.2361, 5.0000])
>>> LA.norm(c, dim=1)
tensor([3.7417, 4.2426])
Conclusion:
In your specific case you will need to do:
torch.linalg.norm(t,dim=1)
Related
I'm confused as to why there are double parantheses instead of just torch.randn(1,5).
Is torch.randn(1,5) the same thing as torch.randn((1,5))?
You should check the definition of this function here.
size (int...) – a sequence of integers defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.
>>> import torch
>>> a = torch.randn(1,5)
>>> b = torch.randn((1,5))
>>> a.shape == b.shape
True
Therefore, you can use either a or b since they have the same shape.
You can use both variants: (1, 2) and 1, 2. Because of python asterisk magics:
torch.randn(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
the first *size captures all positional arguments passed to the function, when passing 1, 2 function will pack it to (1, 2).
the second * turns any parameters that follow it to be keyword-only parameters, to avoid situations like this: randn(1, 2 None, torch.strided, "cuda", True), forcing you to randn(1, 2, out=None, dtype=None, layout=torch.strided, device="cuda", requires_grad=True)
I have a simple model with a pipelie using ColumnTransformer
I am able to train the model and save the model as pickle
When I load the pickle and predict on the real-time data, I received the following error regarding ColumnTransformer
Column ordering must be equal for fit and for transform when using the remainder keyword
The training data and the data used for prediction has exact the same number of column , e.g., 50. I am not sure how the "ordering" of the column could have changed.
Why ordering of the column is important for columntransformer?
How to fix this? Is there a way to ensure the "ordering" after running a column transformer?
Thanks.
pipeline = Pipeline([
('RepalceInf', ReplaceInf()),
('impute_30_100', ColumnTransformer(
[
('oneStdNorm', OneStdImputer(), self.cont_feature_strategy_dict['FEATS_30_100']),
],
remainder='passthrough'
)),
('regress_impute', IterativeImputer(random_state=0, estimator=self.cont_estimator)),
('replace_outlier', OutlierReplacer(quantile_range=(1, 99))),
('scaler', StandardScaler(with_mean=True))
])
class OneStdImputer(TransformerMixin, BaseEstimator):
def __init__(self):
"""
Impute the missing data with random value in the range of mean +/- one standard deviation
This is a simplified implementation without sparse/dense fit and check.
"""
self.mean = None
self.std = None
def fit(self, X, y=None):
self.mean = X.mean()
self.std = X.std()
return self
def transform(self, X):
# X_imp = X.fillna(np.random.randint()*2*self.std+self.mean-self.std)
for col in X:
self._fill_randnorm(X[col], col)
return X
def _fill_randnorm(self, df, col):
val = df.values
mask = np.isnan(df)
mu, sigma = self.mean[col], self.std[col]
val[mask] = np.random.normal(mu, sigma, size=mask.sum())
return df
You can use df_new =pd.DataFrame(df_origin, columns=df_train.columns to make sure the data to predict have same columns with training data.
And from the given example, it's obviously that ColumnTransformer will take the order number of a chosen column as a mark to process.(Although you can use exactly name to choose a column, but I think it will transform to number too)
>>> import numpy as np
>>> from sklearn.compose import ColumnTransformer
>>> from sklearn.preprocessing import Normalizer
>>> ct = ColumnTransformer(
... [("norm1", Normalizer(norm='l1'), [0, 1]),
... ("norm2", Normalizer(norm='l1'), slice(2, 4))])
>>> X = np.array([[0., 1., 2., 2.],
... [1., 1., 0., 1.]])
>>> # Normalizer scales each row of X to unit norm. A separate scaling
>>> # is applied for the two first and two last elements of each
>>> # row independently.
>>> ct.fit_transform(X)
array([[0. , 1. , 0.5, 0.5],
[0.5, 0.5, 0. , 1. ]])
PyTorch's torch.transpose function only transposes 2D inputs. Documentation is here.
On the other hand, Tensorflow's tf.transpose function allows you to transpose a tensor of N arbitrary dimensions.
Can someone please explain why PyTorch does not/cannot have N-dimension transpose functionality? Is this due to the dynamic nature of the computation graph construction in PyTorch versus Tensorflow's Define-then-Run paradigm?
It's simply called differently in pytorch. torch.Tensor.permute will allow you to swap dimensions in pytorch like tf.transpose does in TensorFlow.
As an example of how you'd convert a 4D image tensor from NHWC to NCHW (not tested, so might contain bugs):
>>> img_nhwc = torch.randn(10, 480, 640, 3)
>>> img_nhwc.size()
torch.Size([10, 480, 640, 3])
>>> img_nchw = img_nhwc.permute(0, 3, 1, 2)
>>> img_nchw.size()
torch.Size([10, 3, 480, 640])
Einops supports verbose transpositions for arbitrary number of dimensions:
from einops import rearrange
x = torch.zeros(10, 3, 100, 100)
y = rearrange(x, 'b c h w -> b h w c')
x2 = rearrange(y, 'b h w c -> b c h w') # inverse to the first
(and the same code works for tensorfow as well)
In the Theano Tutorial:
http://deeplearning.net/software/theano/tutorial/gradients.html
or
>>> import theano
>>> import theano.tensor as T
>>> x = T.dvector('x')
>>> y = x ** 2
>>> J, updates = theano.scan(lambda i, y,x : T.grad(y[i], x), sequences=T.arange(y.shape[0]), non_sequences=[y,x])
>>> f = theano.function([x], J, updates=updates)
>>> f([4, 4])
array([[ 8., 0.],
[ 0., 8.]])
"theano.scan" is used to compute the Jacobian. I understand how to compute the Jacobian by hand but I do not quite understand how the function scan is calculating it iteratively; specifically, I don't understand how the "zeroes" are obtained in the Jacobian.
Can someone show me, for each iteration, how this is calculated? Thanks!
How can I print sparse L and U matrices calculated by splu, which uses SuperLU?
My MWE:
>>> import scipy
>>> import scipy.sparse
>>> import scipy.sparse.linalg
>>> from numpy import array
>>> M = scipy.array([ [19,0,21,21,0],[12,21,0,0,0],[0,12,16,0,0],[0,0,0,5,21],[12,12,0,0,18] ])
>>> cscM = scipy.sparse.csc_matrix(M)
>>> lu_obj = scipy.sparse.linalg.splu(cscM)
>>> b = array([1, 2, 3, 4, 5])
>>> lu_obj.solve(b)
array([ 0.01245301, 0.08812209, 0.12140843, -0.08505639, 0.21072771])
You can use
lu_obj = scipy.sparse.linalg.splu(A)
L,R = lu_obj.L, lu_obj.R
in the current scipy version, which returns the matrices in csc format (scipy docs).
Glancing through the scipy docs and source, scipy.sparse.linalg.splu does indeed use SuperLU. It looks like SuperLU may not explicitly calculate L or U. L & U are apt to be more dense than your original sparse matrix, so it makes sense to avoid storing them if they are not needed. If it is any consolation, your lu_obj does contain the permutaion info for L & U: lu_obj.perm_c, lu_obj.perm_r.
To get L & U, the path of least work is to use scipy.linalg.lu to get the LU matrixes. You'll have to convert your sparse matrixes to dense ones, though. ie
P, L, U = scipy.linalg.lu(cscM.todense())