data_root = os.path.join(os.getcwd(), "data")
transform = transforms.Compose(
[
transforms.ToTensor(),
transforms.Normalize([0.5], [0.5]),
]
)
fashion_mnist_dataset = FashionMNIST(data_root, download = True, train = True, transform = transform)
Error Message
/usr/local/lib/python3.7/dist-packages/torchvision/datasets/mnist.py in read_sn3_pascalvincent_tensor(path, strict)
524 # we need to reverse the bytes before we can read them with torch.frombuffer().
525 needs_byte_reversal = sys.byteorder == "little" and num_bytes_per_value > 1
--> 526 parsed = torch.frombuffer(bytearray(data), dtype=torch_type, offset=(4 * (nd + 1)))
527 if needs_byte_reversal:
528 parsed = parsed.flip(0)
AttributeError: module 'torch' has no attribute 'frombuffer'
what can i do for this err in Colab
I tried your code in my Google Colab by adding the codes (to import the libraries) below, but it works well without errors.
import os
from torchvision import transform
from torchvision.datasets import FashionMNIST
I used
torchvision 0.13.0+cu113
google-colab 1.0.0
Runtime GPU (when I set "None," it also works)
Do you get errors when you also use the same codes above? Do you use another versions?
Related
I'm getting started with machine learning, and I'm trying to run the code in this link but I got stuck in these lines:
import torchtext
agnews_train, agnews_test = torchtext.datasets.AG_NEWS(root='./datasets', split=('train', 'test')) # this line added for completeness
VOCAB_SIZE = len(trainset.get_vocab())
EMBED_DIME = 100
HIDDEN_DIME = 64
NUM_OUTPUTS = len(trainset.get_labels())
The error I got is:
AttributeError: 'MapperIterDataPipe' object has no attribute 'get_vocab
I also get this error when I write the last line at first:
AttributeError: 'MapperIterDataPipe' object has no attribute 'get_labels
I tried to replace get_vocab() with this code taken from this link:
tokenizer = get_tokenizer("basic_english")
def build_vocab(datasets):
for dataset in datasets:
for _, text in dataset:
tok = tokenizer(text)
yield tok
vocab = build_vocab_from_iterator(build_vocab([agnews_train, agnews_test]), specials=["<UNK>"])
vocab.set_default_index(vocab["<UNK>"])
VOCAB_SIZE = len(vocab.get_itos())
It runs but I don't know if I'm doing it right because I don't know which value get_vocab() returns.
I think the problem is the library version. I have pytorch 1.11.0+cpu and torchtext 0.12.0, while the code I'm trying to reproduce uses torchtext 0.4.
How can I replace get_vocab() and get_labels() functions?
The function detect_faces() fails in JupiterLab:
image = Image.open(filename)
imageRGB = image.convert('RGB')
pixels = asarray(imageRGB)
detector = MTCNN()
results = detector.detect_faces(pixels)
mtcnn version 0.1.0
The error:
AbortedError: Operation received an exception:Status: 2, message:
could not create a descriptor for a softmax forward propagation
primitive, in file tensorflow/core/kernels/mkl/mkl_softmax_op.cc:306
[[node model/softmax/Softmax (defined at
/home/rikkatti/anaconda3/envs/poi/lib/python3.9/site-packages/mtcnn/mtcnn.py:342)
]] [Op:__inference_predict_function_828]
Function call stack: predict_function
that's probably due to the conflict of keras and tensorflow version.
I solved it by doing these steps:
Uninstall tensorflow from anaconda venv
pip install tensorflow==2.9.0
Try it this way using cv2
from mtcnn import MTCNN
import os
import cv2
detector = MTCNN()
dest_dir=r'C:\Temp\people\storage\cropped' # specify where to save the images
filename=r'C:\Temp\people\storage\34.png' # specify the file name full path
try:
img=cv2.imread(filename) # filename must be full path to the image
shape=img.shape # will cause an exception if image was not read properly
data=detector.detect_faces(img)
if data ==[]:
print ('no faces were detected for file ', filename)
else:
for i, faces in enumerate(data):
box= faces['box']
if box != []:
box[0]= 0 if box[0]<0 else box[0]
box[1]= 0 if box[1]<0 else box[1]
cropped_img=img[box[1]: box[1]+box[3],box[0]: box[0]+ box[2]]
fname=os.path.split(filename)[1]
index=fname.rfind('.')
fileid=fname[:index]
fext=fname[index:]
fname=fileid + '-' +str(i) + fext
save_path=os.path.join(dest_dir,fname )
cv2.imwrite(save_path, cropped_img)
except:
print(' an error occurred')
This will detect all faces in the image and store them as cropped images in the dest_dir. Tested it with an image with multiple faces and it works fine
I am trying to convert a pre-trained torch model to ONNX, but recive the following error:
RuntimeError: step!=1 is currently not supported
I'm trying this on a pre-trained colorization model: https://github.com/richzhang/colorization
Here is the code I ran in Google Colab:
!git clone https://github.com/richzhang/colorization.git
cd colorization/
import colorizers
model = colorizer_siggraph17 = colorizers.siggraph17(pretrained=True).eval()
input_names = [ "input" ]
output_names = [ "output" ]
dummy_input = torch.randn(1, 1, 256, 256, device='cpu')
torch.onnx.export(model, dummy_input, "test_converted_model.onnx", verbose=True,
input_names=input_names, output_names=output_names)
I appreciate any help :)
UPDATE 1: #Proko suggestion solved the ONNX export issue. Now I have a new possibly related problem when I try to convert the ONNX to TensorRT. I get the following error:
[TensorRT] ERROR: Network must have at least one output
Here is the code I used:
import torch
import pycuda.driver as cuda
import pycuda.autoinit
import tensorrt as trt
import onnx
TRT_LOGGER = trt.Logger()
def build_engine(onnx_file_path):
# initialize TensorRT engine and parse ONNX model
builder = trt.Builder(TRT_LOGGER)
builder.max_workspace_size = 1 << 25
builder.max_batch_size = 1
if builder.platform_has_fast_fp16:
builder.fp16_mode = True
network = builder.create_network()
parser = trt.OnnxParser(network, TRT_LOGGER)
# parse ONNX
with open(onnx_file_path, 'rb') as model:
print('Beginning ONNX file parsing')
parser.parse(model.read())
print('Completed parsing of ONNX file')
# generate TensorRT engine optimized for the target platform
print('Building an engine...')
engine = builder.build_cuda_engine(network)
context = engine.create_execution_context()
print("Completed creating Engine")
return engine, context
ONNX_FILE_PATH = 'siggraph17.onnx' # Exported using the code above
engine,_ = build_engine(ONNX_FILE_PATH)
I tried to force the build_engine function to use the output of the network by:
network.mark_output(network.get_layer(network.num_layers-1).get_output(0))
but it did not work.
I appropriate any help!
Like I have mentioned in a comment, this is because slicing in torch.onnx supports only step = 1 but there are 2-step slicing in the model:
self.model2(conv1_2[:,:,::2,::2])
Your only option as for now is to rewrite slicing to be some other ops. You can do it by using range and reshape to obtain proper indices. Consider the following function "step-less-arange" (I hope it is generic enough for anyone with similar problem):
def sla(x, step):
diff = x % step
x += (diff > 0)*(step - diff) # add length to be able to reshape properly
return torch.arange(x).reshape((-1, step))[:, 0]
usage:
>> sla(11, 3)
tensor([0, 3, 6, 9])
Now you can replace every slice like this:
conv2_2 = self.model2(conv1_2[:,:,self.sla(conv1_2.shape[2], 2),:][:,:,:, self.sla(conv1_2.shape[3], 2)])
NOTE: you should optimize it. Indices are calculated for every call so it might be wise to pre-compute it.
I have tested it with my fork of the repo and I was able to save the model:
https://github.com/prokotg/colorization
What works for me was to add the opset_version=11 on torch.onnx.export
First I had tried use opset_version=10, but the API suggest 11 so it works.
So your function should be:
torch.onnx.export(model, dummy_input, "test_converted_model.onnx", verbose=True,opset_version=11,
input_names=input_names, output_names=output_names)
I started working on this about two months ago on Google Colab for a midterm project and everything worked perfectly. Now I am modifying it for a final project and keep getting the error 'RuntimeError: Trying to create tensor with negative dimension -1: [-1, 768]'. It looks like pytorch recently pushed a new version 1.5, so I downgraded to version 1.4 and still got the same error. Same with 1.3, and I know I wasn't using anything lower since that came out last year. I checked it with my midterm code and still got the same error, so I don't know what's going on. Here is the chunk of code related to downloading and using the model.
train_inputs, validation_inputs, train_labels, validation_labels = train_test_split(inputIds,
labels,
random_state=2020,
test_size=0.2)
train_masks, validation_masks, _, _ = train_test_split(attention_masks, inputIds, random_state=2020,
test_size=0.2)
# Turn data into torch tensors
train_inputs = torch.tensor(train_inputs)
validation_inputs = torch.tensor(validation_inputs)
train_labels = torch.tensor(train_labels)
validation_labels = torch.tensor(validation_labels)
train_masks = torch.tensor(train_masks)
validation_masks = torch.tensor(validation_masks)
# Create Iterators of the datasets
train_data = TensorDataset(train_inputs, train_masks, train_labels)
train_sampler = RandomSampler(train_data)
train_dataloader = DataLoader(train_data, sampler=train_sampler, batch_size=batch_size)
validation_data = TensorDataset(validation_inputs, validation_masks, validation_labels)
validation_sampler = SequentialSampler(validation_data)
validation_dataloader = DataLoader(validation_data, sampler=validation_sampler, batch_size=batch_size)
model = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased', num_labels=2)
# Loads model into GPU memory
model.cuda()
param_optimizer = list(model.named_parameters())
no_decay = ['bias','gamma','beta']
optimizer_grouped_parameters = [
{'params':[p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],
'weight_decay_rate':0.01},
{'params':[p for n, p in param_optimizer if any(nd in n for nd in no_decay)],
'weight_decay_rate':0.0}
]
optimizer = AdamW(optimizer_grouped_parameters, lr=2e-5)
The error happens on the line model = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased', num_labels=2). The packages I am using:
from pandas import to_datetime
import torch
from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler
from keras.preprocessing.sequence import pad_sequences
from sklearn.model_selection import train_test_split
# MUST INSTALL PYTORCH-TRANSFORMERS
from pytorch_transformers import XLNetTokenizer, XLNetForSequenceClassification, AdamW
from tqdm import trange
from numpy import argmax, sum
import nltk
nltk.download('punkt')
Thank you to anyone who tries to help.
You can try transformers instead of pytorch_transformers.
! pip install transformers (Google Colab)
In terminal,
pip install transformers
import torch
from transformers import XLNetForSequenceClassification
model = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased', num_labels=2)
model.cuda()
param_optimizer = list(model.named_parameters())
no_decay = ['bias','gamma','beta']
optimizer_grouped_parameters = [
{'params':[p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],
'weight_decay_rate':0.01},
{'params':[p for n, p in param_optimizer if any(nd in n for nd in no_decay)],
'weight_decay_rate':0.0}
]
Here's the code without any error in google colab: https://colab.research.google.com/drive/1A8edGYyFuE7d1Z-ZJusz5zU_7yCevF2L
I am running the following code on colab taken from the example here: https://huggingface.co/transformers/model_doc/albert.html#albertformaskedlm
import os
import torch
import torch_xla
import torch_xla.core.xla_model as xm
assert os.environ['COLAB_TPU_ADDR']
dev = xm.xla_device()
from transformers import AlbertTokenizer, AlbertForMaskedLM
import torch
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
model = AlbertForMaskedLM.from_pretrained('albert-base-v2').to(dev)
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0) # Batch size 1
data = input_ids.to(dev)
outputs = model(data, masked_lm_labels=data)
loss, prediction_scores = outputs[:2]
I haven't done anything to the example code except move input_ids and model onto the TPU device using .to(dev). It seems everything is moved to the TPU no problem as when I input data I get the following output: tensor([[ 2, 10975, 15, 51, 1952, 25, 10901, 3]], device='xla:1')
However when I run this code I get the following error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-5-f756487db8f7> in <module>()
1
----> 2 outputs = model(data, masked_lm_labels=data)
3 loss, prediction_scores = outputs[:2]
9 frames
/usr/local/lib/python3.6/dist-packages/transformers/modeling_albert.py in forward(self, hidden_states, attention_mask, head_mask)
277 attention_output = self.attention(hidden_states, attention_mask, head_mask)
278 ffn_output = self.ffn(attention_output[0])
--> 279 ffn_output = self.activation(ffn_output)
280 ffn_output = self.ffn_output(ffn_output)
281 hidden_states = self.full_layer_layer_norm(ffn_output + attention_output[0])
RuntimeError: Unknown device
Anyone know what's going on?
Solution is here: https://github.com/pytorch/xla/issues/1909
Before calling model.to(dev), you need to call xm.send_cpu_data_to_device(model, xm.xla_device()):
model = AlbertForMaskedLM.from_pretrained('albert-base-v2')
model = xm.send_cpu_data_to_device(model, dev)
model = model.to(dev)
There are also some issues with getting the gelu activation function ALBERT uses to work on the TPU, so you need to use the following branch of transformers when working on TPU: https://github.com/huggingface/transformers/tree/fix-jit-tpu
See the following colab notebook (by https://github.com/jysohn23) for full solution: https://colab.research.google.com/gist/jysohn23/68d620cda395eab66289115169f43900/getting-started-with-pytorch-on-cloud-tpus.ipynb