I'm trying to understand actual implementation of RNN forward , which I expect at some point has to have a loop over time step(sequence) that uses rnn cell.
While checking actual c++ code on github at got stuck at this point.
I'm wondering where is the torch::rnn_tanh code?
Related
I understand the iteration training process of finding the optimized weight to explain the dataset.
However, before studying about learning iteration, I used to apply simple code like lm(Y~x) or LinearRegression().fit(X, y) without multiple loops of training process.
Does this simple code already contain iteration of training process?
What is the difference between simple algorithm without learning iteration and the one with iteration?
Thanks
I am attempting to make a sudoku solver using a CNN. I am wondering if there is a way I can calculate the loss between the output puzzle and the without using a fully connected layer? If yes how?
In this example, sparse_categorical_crossentropy have been proposed as a loss function for learning to solve sudoku in a supervised manner.
It has been also stated that in "SCC loss, you don't need to provide a one-hot encoded target vector." So you will provide the output and the target as-is.
Shoutout to all Pytorch core developers!
I would like to implement a custom version of the typical LSTM cell as it is implemented in Pytorch, say, change one of the activation functions at a gate. For this, I would like to see how the LSTM is implemented in Pytorch at the moment. I can find some code here, but unfortunately, I cannot find the exact LSTM computations there, including the implementation of the gate formulas as they are also specified in the docstrings.
Related posts can for example be found here, but all they delivered me is that nobody has found the LSTM cell code in github.
Where is the LSTM cell implemented in the Pytorch Github exactly?
I am trying to implement the AntisymmetricRNN described in this paper: https://arxiv.org/abs/1902.09689.
Working in Keras, I guess I have to implement my own layer so I have read https://keras.io/layers/writing-your-own-keras-layers/. Instead of starting from a plain layer as explained there, I reckon the best would probably be to extend one of the existing RNN, but Keras has
RNN
SimpleRNNCell
SimpleRNN
The documentation isn't verbose enough for someone my level about what these classes do/are, and consequently I am having a hard time figuring out what should be my starting point.
Any help, both in terms of where to start and what to actually look out for, and all sorts of suggestions are greatly appreciated. Thank you.
In Keras, all recurrent layers are RNN layers with a certain Cell.
The definition is RNN(cell=someCell)
So, the LSTM layer follows the same principle, an LSTM(units=...) layer is equal to an RNN(cell=LSTMCell(units=...), ...) layer.
That said, to implement your recurrent layer (if it doesn't break the recurrent flow step by step or jump steps), you need to implement your own cell. You can study what is happening in the LSTMCell code, compare it with the papers and adjust the weights and formulas to your need.
So you will have your own RNN(cell=yourCell).
I just began my Deep learning journey with Keras along with Tenserflow. I followed a tutorial that used a feed forward model on MNIST dataset. The strange part is that I used the same complied code, yet, I got a higher accuracy rate than the exact same code. I'm looking to understand why or how can this happen?