since keras with tensorflow backend can not reproduce results, i switch to theano. However, when i run the exactly same lstm model with exactly the same data,with theano backend is almost three times slower than with tensorflow backend. any suggestions?
I solved this by running the command "conda install -y mkl"
Related
I met a strange problem. I trained my model with one GPU(RTX Titan), and it doesn't converge. However, it worked well on two same GPUs with the same settings. There is nothing to do with the batch size. And I use the torch.fft and torch.Transformer layer. I use Python 3.8, Pytorch 1.71 and Cuda 10.1.
It looks like tf.keras is suggested if you're using a Tensorflow 2 backend, but what about using Theano or CNTK as a backend? I have never used Keras or any DL library.
Keras has officially decided to drop support of CNTK and Theano moving forward. Therefore, if you are using keras with tensorflow as the backend, you should use tf.keras.
For older versions for keras, you can use all three backend with no syntax change in your keras code.
Keras 2.2.5 was the last release of Keras implementing the 2.2.* API.
It was the last release to only support TensorFlow 1 (as well as
Theano and CNTK).
The current release is Keras 2.3.0, which makes significant API
changes and add support for TensorFlow 2.0. The 2.3.0 release will be
the last major release of multi-backend Keras. Multi-backend Keras is
superseded by tf.keras.
You can find the above information here.
Since I am kind of new in this field I tried following the official tutorial from tensorflow for predicting time series. https://www.tensorflow.org/tutorials/structured_data/time_series
Following problem occurs:
-When training a multivariate model, after 2 or 3 epochs the kernel dies and restarts.
However this doesn't happen with a simpler univariate model, which has only one LSTM layer (not really sure if this makes a difference).
Second however, this problem just happened today. Yesterday the training of the multivariate model was possible and error-free.
As can be seen in the tutorial in the link below the model looks like this:
multi_step_model = tf.keras.models.Sequential()
multi_step_model.add(tf.keras.layers.LSTM(32,return_sequences=True,input_shape=x_train_multi.shape[-2:]))
multi_step_model.add(tf.keras.layers.LSTM(16, activation='relu'))
multi_step_model.add(tf.keras.layers.Dense(72))
multi_step_model.compile(optimizer=tf.keras.optimizers.RMSprop(clipvalue=1.0), loss='mae')
And the kernel dies after executing the following cell (usually after 2 or 3 epochs).
multi_step_history = multi_step_model.fit(train_data_multi, epochs=10,
steps_per_epoch=300,
validation_data=val_data_multi,
validation_steps=50)
I have uninstalled and reinstalled tf, restarted my laptop, but nothing seems to work.
Any ideas?
OS: Windows 10
Surface Book 1
Problem was a too big batch size. Reducing it from 1024 to 256 solved the crashing problem.
Solution taken from the comment of rbwendt on this thread on github.
I was wondering, what's the difference between importing keras from tensorflow using import tensorflow.keras or just pip installing keras alone and importing it using import keras as both seemed to work nicely so far, the only difference I noticed is that i get Using TensorFlow backend. in the command line every time I execute the one using keras.
Tensorflow.keras is an version of Keras API implemented specifically for use with Tensorflow. It is a part of Tensorflow repo and from TF version 2.0 will become main high level API replacing tf.layers and slim.
The only reason to use standalone keras is to maintain framework-agnostic code, i.e. use it with another backend.
How you can program keras or tensorflow to partitionate training on multiple GPU, let's say you are in an amaozn ec2 instance that has 8 GPU's and you want to use all of them to train faster, but your code is just for a single cpu or GPU ?
Yes, can run Keras models on multiple GPUs. This is only possible with the TensorFlow backend for the time being, because the Theano feature is still rather new. We are looking at adding support for multi-gpu in Theano in the near future (it should be fairly straightforward).
With the TensorFlow backend, you can achieve this the same way as you would in pure TensorFlow: by using the with tf.device(d) scope when defining Keras layers.
Originally from here