I am trying to compile pytorch transformer to run it in C++:
from torch.nn import TransformerEncoder, TransformerEncoderLayer
encoder_layers = TransformerEncoderLayer(1000, 8, 512, 0.1)
transf = TransformerEncoder(encoder_layers, 6)
sm = torch.jit.script(transf)
But I am getting an error:
RuntimeError: Expected a default value of type Tensor on parameter
"src_mask": File "C:\Program Files (x86)\Microsoft Visual
Studio\Shared\Python36_64\lib\site-packages\torch\nn\modules\transformer.py",
line 271
def forward(self, src, src_mask=None, src_key_padding_mask=None):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~... <--- HERE
r"""Pass the input through the encoder layer.
It looks like something wrong with pytorch transformer module.
Is there any way to run pytorch transformer in C++ ?
You need to upgrade to PyTorch 1.5.0, older versions did not support converting Transformers to TorchScript (JIT) modules.
pip install torch===1.5.0 -f https://download.pytorch.org/whl/torch_stable.html
In 1.5.0 you will see some warnings about the parameters being declared as constants, such as:
UserWarning: 'q_proj_weight' was found in ScriptModule constants, but it is a non-constant parameter. Consider removing it.
These can be safely ignored.
Related
I have Python 3.9.7, TensowFlow == 2.5, NVIDIA-SMI 497.09, Driver Version: 497.09, CUDA Version: 11.5. On trying to define a LSTM model as follows:
n_steps = 500
n_features = 1
# Univariate multi-step time series prediction-
model = Sequential()
model.add(LSTM(units=50, activation='relu', return_sequences=True, input_shape=(n_steps, n_features)))
It first gives a warning:
WARNING:tensorflow:Layer lstm_4 will not use cuDNN kernels since it
doesn't meet the criteria. It will use a generic GPU kernel as
fallback when running on GPU.
And then:
NotImplementedError: Cannot convert a symbolic Tensor
(lstm_4/strided_slice:0) to a numpy array. This error may indicate
that you're trying to pass a Tensor to a NumPy call, which is not
supported
Changing the activation to 'tanh' doesn't change anything.
Check your numpy version. Is it above 1.20? Here's issue #14687 with the same problem, and they point to issue #47691, where they fixed this in tensorflow 2.6.
So, either downgrade to numpy 1.19.2, pip install numpy==1.19.2 or upgrade tensorflow, pip install --upgrade tensorflow. This should hopefully fix the issue!
SOLUTION at the bottom!
I want to do Object Detection with this tutorial:
https://towardsdatascience.com/building-your-own-object-detector-pytorch-vs-tensorflow-and-how-to-even-get-started-1d314691d4ae
Although I have compatible versions of Pytorch, Torchvision and Cuda:
conda list torch gives me:
I get the following RunTime Error at the bottom:
RuntimeError: Couldn't load custom C++ ops. This can happen if your
PyTorch and torchvision versions are incompatible, or if you had
errors while compiling torchvision from source. For further
information on the compatible versions, check
https://github.com/pytorch/vision#installation for the compatibility
matrix. Please check your PyTorch version with torch__version__ and
your torchvision version with torchvision__version__ and verify if
they are compatible, and if not please reinstall torchvision so that
it matches your PyTorch install.
when running:
num_epochs = 10
for epoch in range(num_epochs):
train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq=10)#.to_fp16()
lr_scheduler.step()
evaluate(model, data_loader_test, device=device)
Is it really an error resulting from incompatibility of pytorch and torchvision?
Thank you very much.
SOLUTION:
I imported torchvision from the wrong directory. I found out using following:
import torchvision
print(torchvision.__path__)
I am getting this warning when running h2o AutoML. I have version 3.32.1.2 installed, and running it on python 3.8.
AutoML progress: |
11:30:52.773: AutoML: XGBoost is not available; skipping it.
CODE:
import h2o
h2o.init()
h2o_df = h2o.H2OFrame(df)
train, test = h2o_df.split_frame(ratios=[.75])
# Identify predictors and response
x = train.columns
y = "TERM_DEPOSIT"
x.remove(y)
from h2o.automl import H2OAutoML
aml = H2OAutoML(max_runtime_secs=600,
#exclude_algos=['DeepLearning'],
seed=1,
#stopping_metric='logloss',
#sort_metric='logloss',
balance_classes=False,
project_name='Completed'
)
%time aml.train(x=x, y=y, training_frame=train)
XGBoost is not supported on Windows, see the limitations in the H2O documentation.
If you are not using Windows and you didn't find another reason in the documentation mentioned above, you can try to reinstall h2o, e.g.,
pip install --force-reinstall https://h2o-release.s3.amazonaws.com/h2o/rel-zipf/2/Python/h2o-3.32.1.2-py2.py3-none-any.whl
I think I found the answer to this warning. I am running a windows machine.
https://twitter.com/ledell/status/1148512129659625472?lang=en
If you're on Windows, XGBoost is not supported 😿 so the parts of the tutorial that use XGBoost can be replaced by h2o.gbm(). The AutoML process will also exclude XGBoost models.
I need to use Tensorflow 2.0(TF2.0) and Keras but I don't know if it's necessary to install both seperately or just TF2.0 (assuming TF2.0 has Keras bundled inside it). If I need to install TF2.0 only, will installing in a Python 3.7.1 be acceptable?
This is for Ubuntu 16.04 64 bit.
In Tensorflow 2.0 there is strong integration between TensorFlow and the Keras API specification (TF ships its own Keras implementation, that respects the Keras standard), therefore you don't have to install Keras separately since Keras already comes with TF in the tf.keras package.
I notice densenet has been added to keras (https://github.com/keras-team/keras/tree/master/keras/applications)and I want to apply it in my project but when I tried to import it in jupyter anaconda, I got an error saying:
module 'keras.applications' has no attribute 'densenet'
it seems like densenet has not been incorporated into current version of keras.
Any idea how can I add it myself?
Densenet was added in keras version 2.1.3. What version of keras are you running?
Have you tried to update keras with pip install keras --upgrade since January?