The program I am inspecting uses pytorch to load weights and cuda code to do the computations with the weights. My understanding of THC library is how tensors are implemented in the backend of pytorch ( and torch, maybe? ).
how is THC implemented ( I would really appreiciate some details if possible )?
what does THCudaTensor_data( THC_state, THCudaTensor* ) do? ( from the way it is used in the code, it seems like it is used to convert pytorch's tensor to an array in cuda. if this is the case, then would the function preserve all elements and the length of the array?)
I am still not exactly sure of the inner-workings of THCudaTensor_data, but the behaviour that was tripping me up was: for n-dimensional tensor, THCudaTensor_data returns a flattened 1D array of the tensor.
Hope this helps
Related
I have a bit lengthier ODE function which was simulated by using Scipy solve_ivp function. During this simulation I calculated many parameters but as the output, I am taking out put only some other parameters in return function.
But now I need to get some values of some other variables I used during the intermediate steps. So is there a way I can do that?.
Note-: When I Tried to return those parameters with the main ode outputs, it says "The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part."
Highly appreciate any valuable suggestions. Better if you can show the way by using a sample code.
Currently I have pytorch tensors with shape (batch_size, height, width, channel_size) and I want to convert it to a mini-batch described here. My current idea is to convert each example from tensor representation to graph representation separately and group them together. I want to do all these without involving file save/load as it will surely hinder the speed (I notice that Creating “In Memory Datasets” does it this way).
Yet I didn't find any function for the example grouping part. Could anyone please help give a plausible workflow for it, and is there any smarter way for this convertion, from tensor to mini-batch for pytorch-geometric?
I think I'm experiencing a similar question with you. If I understand your question correctly, your want to commit following transformation
Input: Tensor = [#batch,#vertex,#feature]
Output: torch_geometric.data.BatchData = Large tensor
My implementation is:
x = DataLoader([Data(x,edge_index=edges,num_node=#node) for x in x],batch_size=#batch)
data = next(iter(x))
I am trying to use a customized loss function for my NN. I've implemented all operations in torch and I have complex numbers among my data.
I get the error while training a NN:
RuntimeError: _th_addr_out not supported on CPUType for ComplexFloat
Do you know any possible solution to deal with it?
Well it seems Complex Autograd in PyTorch is currently in a prototype state, and the backward functionality for some of function is not included.
For example: torch.sign, which is used in the backward computation of torch.abs, is not defined for complex tensors. same for torch.mv. So I debugged my code line by line to find the functions which are not included, and replaced them with a customized function :)
Hope for a lot more functions to be included in the next release of PyTorch.
I am minimizing a function with scipy.optimize.fmin_cg. In "args" one can supply additional fixed parameters that are needed to completely specify the functions and it's derivative. I want to use a sparse representation, scipy.sparse.csr_matrix, of some arguments. I have the same function coded both with the sparse matrix as well as a dense ndarray. I have noticed differences in the solution.
Does anyone know whether scipy.optimize.fmin_cg accepts sparse matrices in "args"?
Thanks!
I want to know if the filters' weights in a, for example, 2D convolution layer in Keras are shared along the spatial dimensions by default. If yes, is there any way to have not shared weights?
I found that LocallyConnected2D does what I am looking for.
The LocallyConnected2D layer works similarly to the Conv2D layer, except that weights are unshared, that is, a different set of filters is applied at each different patch of the input.
I'm not clear on what your asking but:
The weights in the a single convolutional layer are shared. That is, the filters share the same weights with each stride.
However The weights between two convolutonal layers are not shared by default in keras.
There is no getting around shared wiegths in the filters within the conv layer. Since the execution of the convolution if offloaded to C++ libraries.
See this answer for further reference, in particular:
The implementation of tf.nn.conv2d() is written in C++, which invokes
optimized code using either Eigen (on CPU) or the cuDNN library (on
GPU). You can find the implementation here.