Related
I want to use a model pre-trained by a friend. However, when I load it:
checkpoint_path = "../Models/ckpt_camembert.ckpt"
py_dict = torch.load(checkpoint_path)
model.load_state_dict(py_dict)
I get:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-19-be1591e802b7> in <module>
1 # If you already finetuned a model you can load it by the following rows
2 checkpoint_path = "../Models/ckpt_camembert.ckpt"
----> 3 py_dict = torch.load(checkpoint_path)
4 model.load_state_dict(py_dict)
c:\Programs\Anaconda\lib\site-packages\torch\serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
703 # reset back to the original position.
704 orig_position = opened_file.tell()
--> 705 with _open_zipfile_reader(opened_file) as opened_zipfile:
706 if _is_torchscript_zip(opened_zipfile):
707 warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"
c:\Programs\Anaconda\lib\site-packages\torch\serialization.py in __init__(self, name_or_buffer)
240 class _open_zipfile_reader(_opener):
241 def __init__(self, name_or_buffer) -> None:
--> 242 super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
243
244
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
Is it because my .ckpt file hasn't been properly downloaded/corrupted?
I have the Cuda version 11.2.2_461.33 with the Nvidia driver 11.2.109, cudnn version cudnn-11.2-windows-x64-v8.1.1.33 for Windows 10. I am running tensorflow version 2.8.0 in Jupyter notebook with Python 3.9 through anaconda.
I have GPU enabled successfully.
so I am getting errors when I try to fit this model
# Model Definition with Conv1D
model_conv = tf.keras.Sequential([
tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
tf.keras.layers.Conv1D(filters, kernel_size, activation='relu'),
tf.keras.layers.GlobalMaxPooling1D(),
tf.keras.layers.Dense(dense_dim, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
the error is
Epoch 1/10
---------------------------------------------------------------------------
UnimplementedError Traceback (most recent call last)
Input In [6], in <cell line: 4>()
1 NUM_EPOCHS = 10
3 # Train the model
----> 4 history_conv = model_conv.fit(training_padded, training_labels, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels))
File ~\.conda\envs\tf-gpu\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~\.conda\envs\tf-gpu\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is not None:
UnimplementedError: Graph execution error:
Detected at node 'sequential/conv1d/Conv1D' defined at (most recent call last):
File "C:\Users\me\.conda\envs\tf-gpu\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
...
Node: 'sequential/conv1d/Conv1D'
DNN library is not found.
[[{{node sequential/conv1d/Conv1D}}]] [Op:__inference_train_function_842]
I have installed cuda and the drivers following these instructions for my tensorflow version (cuda 11.2, cuDNN 11.2 for tensorflow 2.8.0):
https://www.tensorflow.org/install/source_windows
(I have not installed bazel though as I use anaconda)
and this guide here, step by step:
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installcuda-windows
Also I get this error when running a bidirectional layer:
InternalError: Graph execution error:
Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]: 2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 64, 64, 1, 1551, 256, 64]
[[{{node CudnnRNN}}]]
[[sequential/bidirectional/backward_lstm/PartitionedCall]] [Op:__inference_train_function_5897]
So it looks like my installation have not gone smoothly after all.
hopefully, someone can advise something.
I am following this article to find the text similarity.
The code I have is this:
from sentence_transformers import SentenceTransformer
from tqdm import tqdm
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
import pandas as pd
documents = [
"Vodafone Wins ₹ 20,000 Crore Tax Arbitration Case Against Government",
"Voda Idea shares jump nearly 15% as Vodafone wins retro tax case in Hague",
"Gold prices today fall for 4th time in 5 days, down ₹6500 from last month high",
"Silver futures slip 0.36% to Rs 59,415 per kg, down over 12% this week",
"Amazon unveils drone that films inside your home. What could go wrong?",
"IPHONE 12 MINI PERFORMANCE MAY DISAPPOINT DUE TO THE APPLE B14 CHIP",
"Delhi Capitals vs Chennai Super Kings: Prithvi Shaw shines as DC beat CSK to post second consecutive win in IPL",
"French Open 2020: Rafael Nadal handed tough draw in bid for record-equaling 20th Grand Slam"
]
model = SentenceTransformer('sentence-transformers/bert-base-nli-mean-tokens')
I get an error when running the above code:
Full:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~\anaconda3\envs\py3_nlp\lib\tarfile.py in nti(s)
188 s = nts(s, "ascii", "strict")
--> 189 n = int(s.strip() or "0", 8)
190 except ValueError:
ValueError: invalid literal for int() with base 8: 'ld_tenso'
During handling of the above exception, another exception occurred:
InvalidHeaderError Traceback (most recent call last)
~\anaconda3\envs\py3_nlp\lib\tarfile.py in next(self)
2298 try:
-> 2299 tarinfo = self.tarinfo.fromtarfile(self)
2300 except EOFHeaderError as e:
~\anaconda3\envs\py3_nlp\lib\tarfile.py in fromtarfile(cls, tarfile)
1092 buf = tarfile.fileobj.read(BLOCKSIZE)
-> 1093 obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
1094 obj.offset = tarfile.fileobj.tell() - BLOCKSIZE
~\anaconda3\envs\py3_nlp\lib\tarfile.py in frombuf(cls, buf, encoding, errors)
1034
-> 1035 chksum = nti(buf[148:156])
1036 if chksum not in calc_chksums(buf):
~\anaconda3\envs\py3_nlp\lib\tarfile.py in nti(s)
190 except ValueError:
--> 191 raise InvalidHeaderError("invalid header")
192 return n
InvalidHeaderError: invalid header
During handling of the above exception, another exception occurred:
ReadError Traceback (most recent call last)
~\anaconda3\envs\py3_nlp\lib\site-packages\torch\serialization.py in _load(f, map_location,
pickle_module, **pickle_load_args)
594 try:
--> 595 return legacy_load(f)
596 except tarfile.TarError:
~\anaconda3\envs\py3_nlp\lib\site-packages\torch\serialization.py in legacy_load(f)
505
--> 506 with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as
tar, \
507 mkdtemp() as tmpdir:
~\anaconda3\envs\py3_nlp\lib\tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
1590 raise CompressionError("unknown compression type %r" % comptype)
-> 1591 return func(name, filemode, fileobj, **kwargs)
1592
~\anaconda3\envs\py3_nlp\lib\tarfile.py in taropen(cls, name, mode, fileobj, **kwargs)
1620 raise ValueError("mode must be 'r', 'a', 'w' or 'x'")
-> 1621 return cls(name, mode, fileobj, **kwargs)
1622
~\anaconda3\envs\py3_nlp\lib\tarfile.py in __init__(self, name, mode, fileobj, format, tarinfo, dereference, ignore_zeros, encoding, errors, pax_headers, debug, errorlevel, copybufsize)
1483 self.firstmember = None
-> 1484 self.firstmember = self.next()
1485
~\anaconda3\envs\py3_nlp\lib\tarfile.py in next(self)
2310 elif self.offset == 0:
-> 2311 raise ReadError(str(e))
2312 except EmptyHeaderError:
ReadError: invalid header
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
~\anaconda3\envs\py3_nlp\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1210 try:
-> 1211 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1212 except Exception:
~\anaconda3\envs\py3_nlp\lib\site-packages\torch\serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
425 pickle_load_args['encoding'] = 'utf-8'
--> 426 return _load(f, map_location, pickle_module, **pickle_load_args)
427 finally:
~\anaconda3\envs\py3_nlp\lib\site-packages\torch\serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
598 # .zip is used for torch.jit.save and will throw an un-pickling error here
--> 599 raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))
600 # if not a tarfile, reset file offset and proceed
RuntimeError: C:\Users\user1/.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens\pytorch_model.bin is a zip archive (did you mean to use torch.jit.load()?)
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-3-bba56aac60aa> in <module>
----> 1 model = SentenceTransformer('sentence-transformers/bert-base-nli-mean-tokens')
~\anaconda3\envs\py3_nlp\lib\site-packages\sentence_transformers\SentenceTransformer.py in __init__(self, model_name_or_path, modules, device, cache_folder)
88
89 if os.path.exists(os.path.join(model_path, 'modules.json')): #Load as SentenceTransformer model
---> 90 modules = self._load_sbert_model(model_path)
91 else: #Load with AutoModel
92 modules = self._load_auto_model(model_path)
~\anaconda3\envs\py3_nlp\lib\site-packages\sentence_transformers\SentenceTransformer.py in _load_sbert_model(self, model_path)
820 for module_config in modules_config:
821 module_class = import_from_string(module_config['type'])
--> 822 module = module_class.load(os.path.join(model_path, module_config['path']))
823 modules[module_config['name']] = module
824
~\anaconda3\envs\py3_nlp\lib\site-packages\sentence_transformers\models\Transformer.py in load(input_path)
122 with open(sbert_config_path) as fIn:
123 config = json.load(fIn)
--> 124 return Transformer(model_name_or_path=input_path, **config)
125
126
~\anaconda3\envs\py3_nlp\lib\site-packages\sentence_transformers\models\Transformer.py in __init__(self, model_name_or_path, max_seq_length, model_args, cache_dir, tokenizer_args, do_lower_case, tokenizer_name_or_path)
27
28 config = AutoConfig.from_pretrained(model_name_or_path, **model_args, cache_dir=cache_dir)
---> 29 self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=config, cache_dir=cache_dir)
30 self.tokenizer = AutoTokenizer.from_pretrained(tokenizer_name_or_path if tokenizer_name_or_path is not None else model_name_or_path, cache_dir=cache_dir, **tokenizer_args)
31
~\anaconda3\envs\py3_nlp\lib\site-packages\transformers\models\auto\auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
393 if type(config) in cls._model_mapping.keys():
394 model_class = _get_model_class(config, cls._model_mapping)
--> 395 return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
396 raise ValueError(
397 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
~\anaconda3\envs\py3_nlp\lib\site-packages\transformers\modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1212 except Exception:
1213 raise OSError(
-> 1214 f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
1215 f"at '{resolved_archive_file}'"
1216 "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "
OSError: Unable to load weights from pytorch checkpoint file for 'C:\Users\user1/.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens\' at 'C:\Users\user1/.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens\pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
Short:
OSError: Unable to load weights from pytorch checkpoint file for 'C:\Users\user1/.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens' at 'C:\Users\user1/.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens\pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
I do have the pytorch_model.bin in the '.cache\torch\sentence_transformers\sentence-transformers_bert-base-nli-mean-tokens' folder.
Why am I getting this error?
The reason for the error seems to be that the pre-trained model weight files are not available or loadable.
You can try that one to load pretrained model weight file:
from transformers import AutoModel
model = AutoModel.from_pretrained('sentence-transformers/bert-base-nli-mean-tokens')
Reference: https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens
Also, the model's hugging face page says:
This model is deprecated. Please don't use it as it produces sentence embeddings of low quality. You can find recommended sentence embedding models here: SBERT.net - Pretrained Models
Maybe you might want to take a look.
You may need to use the model without sentence_transformers.
The following code is tweaked from https://www.sbert.net/examples/applications/computing-embeddings/README.html
As I understand it, from the exception you need to pass from_tf=True to AutoModel.
from transformers import AutoTokenizer, AutoModel
import torch
#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
token_embeddings = model_output[0] #First element of model_output contains all token embeddings
input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1)
sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9)
return sum_embeddings / sum_mask
#Sentences we want sentence embeddings for
sentences = ['This framework generates embeddings for each input sentence',
'Sentences are passed as a list of string.',
'The quick brown fox jumps over the lazy dog.']
#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/bert-base-nli-mean-tokens')
model = AutoModel.from_pretrained('sentence-transformers/bert-base-nli-mean-tokens',from_tf=True)
#Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt')
#Compute token embeddings
with torch.no_grad():
model_output = model(**encoded_input)
#Perform pooling. In this case, mean pooling
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
I used two methods to decode one jpeg image file, one is tensorflow(abbreviated to tf), the other is opencv(abbreviated to cv2). But cv2 is right and tf got some errors. The code is as following:
import tensorflow as tf
import cv2
path = 'one jpeg file path'
img_tf = tf.image.decode_jpeg(tf.io.read_file(path))
img_cv2 = cv2.imread(path)
img_cv2 got right, but img_tf got errors:
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-26-e44ad973b8b3> in <module>
----> 1 img = tf.image.decode_jpeg(tf.io.read_file(path))
//anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_image_ops.py in decode_jpeg(contents, channels, ratio, fancy_upscaling, try_recover_truncated, acceptable_fraction, dct_method, name)
1176 try_recover_truncated=try_recover_truncated,
1177 acceptable_fraction=acceptable_fraction, dct_method=dct_method,
-> 1178 name=name, ctx=_ctx)
1179 except _core._SymbolicException:
1180 pass # Add nodes to the TensorFlow graph.
//anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_image_ops.py in decode_jpeg_eager_fallback(contents, channels, ratio, fancy_upscaling, try_recover_truncated, acceptable_fraction, dct_method, name, ctx)
1259 "acceptable_fraction", acceptable_fraction, "dct_method", dct_method)
1260 _result = _execute.execute(b"DecodeJpeg", 1, inputs=_inputs_flat,
-> 1261 attrs=_attrs, ctx=_ctx, name=name)
1262 _execute.record_gradient(
1263 "DecodeJpeg", _inputs_flat, _attrs, _result, name)
//anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
65 else:
66 message = e.message
---> 67 six.raise_from(core._status_to_exception(e.code, message), None)
68 except TypeError as e:
69 if any(ops._is_keras_symbolic_tensor(x) for x in inputs):
//anaconda3/lib/python3.7/site-packages/six.py in raise_from(value, from_value)
InvalidArgumentError: Expected image (JPEG, PNG, or GIF), got unknown format starting with 'II*\000\226%\026\000\177wgjvo_r' [Op:DecodeJpeg]
Some people know something about this, thanks.
Are you sure that the file you are providing is actually in JPEG format? According to Wikipedia II* is signature of TIFF file (the extension of your file might be wrong).
Could you try another JPEG image?
You are not getting error with OpenCV since it can read TIFF images, while tf.io.decode_jpeg can not.
I am using hyperas module to tune my Keras model and return the error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 4785: ordinal not in range(128)
The error occurred at calling place, the syntax of trials:
if __name__ == '__main__':
best_run, best_model = optim.minimize(model=create_model,
data=data,
algo=tpe.suggest,
max_evals=20,
trials=Trials())
and I think the origin of the problem is due to my loaded numpy .npy file which is a ascii encoding format data.
So, how can I change the ascii format to utf-8 format?
I saw some solution like this by adding the encoding='latin1' but it doesn't work.
label =np.load(os.getcwd()+'/Simu_Sample_label_1000.npy',encoding="latin1")
sample=np.load(os.getcwd()+'/Training_Sample_1000.npy',encoding="latin1")
Add my whole traceback here:
In [3]: %run 1dCNN.py
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
~/subg_ps/cnn_train/1dCNN.py in <module>()
127 algo=tpe.suggest,
128 max_evals=20,
--> 129 trials=Trials())
130 trX, trY, teX, teY = data()
131 print("Evalutation of best performing model:")
~/anaconda3/lib/python3.6/site-packages/hyperas/optim.py in minimize(model, data, algo, max_evals, trials, functions, rseed, notebook_name, verbose, eval_space, return_space, keep_temp)
67 notebook_name=notebook_name,
68 verbose=verbose,
---> 69 keep_temp=keep_temp)
70
71 best_model = None
~/anaconda3/lib/python3.6/site-packages/hyperas/optim.py in base_minimizer(model, data, functions, algo, max_evals, trials, rseed, full_model_string, notebook_name, verbose, stack, keep_temp)
96 model_str = full_model_string
97 else:
---> 98 model_str = get_hyperopt_model_string(model, data, functions, notebook_name, verbose, stack)
99 temp_file = './temp_model.py'
100 write_temp_files(model_str, temp_file)
~/anaconda3/lib/python3.6/site-packages/hyperas/optim.py in get_hyperopt_model_string(model, data, functions, notebook_name, verbose, stack)
184 calling_script_file = os.path.abspath(inspect.stack()[stack][1])
185 with open(calling_script_file, 'r') as f:
--> 186 source = f.read()
187
188 cleaned_source = remove_all_comments(source)
~/anaconda3/lib/python3.6/encodings/ascii.py in decode(self, input, final)
24 class IncrementalDecoder(codecs.IncrementalDecoder):
25 def decode(self, input, final=False):
---> 26 return codecs.ascii_decode(input, self.errors)[0]
27
28 class StreamWriter(Codec,codecs.StreamWriter):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 4785: ordinal not in range(128)
I think I'd better put all traceback here, and all the code as follow:
https://github.com/MinghaoDu1994/MyPythonFunctions/blob/master/1Dcnn
I think that the problem is due to the funciton Trials in hyperopt, but I don't find any related question like mine.
The problem has been solved.
When calling the optim.minimize function, we must first define two functions named as data and model, rather than what I named create_model or anything else. It is a very strict limitation.
I can recreate your error by converting a unicode string (PY3 default) to bytestring, and then trying to decode it:
In [347]: astr = 'abc'+chr(0xe8)+'xyz'
In [348]: astr
Out[348]: 'abcèxyz'
In [349]: astr.encode('latin1')
Out[349]: b'abc\xe8xyz'
In [350]: astr.encode('latin1').decode('ascii')
---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
<ipython-input-350-1825a76f5d5b> in <module>
----> 1 astr.encode('latin1').decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe8 in position 3: ordinal not in range(128)
hyperas reading some sort of script file in get_hyperopt_model_string(). I can't tell what variable is controlling this read, maybe it's the notebook. I don't think the arrays that you loaded from npy files have anything to do with this problem. It's decoding a large string (position 4785), not some element of an array.
In short, this is a hyperas model problem, not a npy file one.