TypeError Traceback (most recent call last)
<ipython-input-9-cf2b62201e08> in <module>
----> 1 from pyspark import SparkContext
2
3 sc = SparkContext.getOrCreate()
4 sc
4 frames
/content/spark-2.3.1-bin-hadoop2.7/python/pyspark/cloudpickle.py in _make_cell_set_template_code()
125 )
126 else:
--> 127 return types.CodeType(
128 co.co_argcount,
129 co.co_kwonlyargcount,"
TypeError: an integer is required (got type bytes)
I am trying to build a Pyspark project using Google colab notebooks, though this problem is not limited to colab, it could happen in any other IDE.
I tried resolving the issue using the latest versions of Spark and Hadoop, but this error comes in every new notebook.
I am expecting this work without any errors.
Related
I am running my code in a Databricks notebook environment.
Error 1:
To get a tokenizer returned, I am running the following code:
tokenizer = T5Tokenizer.from_pretrained("t5-base")
But I get an error: TypeError: 'NoneType' object is not callable.
When I ran this code on Google Colab, I installed the sentencepiece library and restarted the runtime and the error was resolved. I attempted to do the same and then restart the cluster of the Databricks notebook but this error persisted. How can I fix this?
Error 2:
To get a model returned, I am running the following code:
t5_model = T5ForConditionalGeneration.from_pretrained('t5-base')
But I get an error:
ImportError Traceback (most recent call last)
<command-3778897587544926> in <module>
2 from transformers import T5Tokenizer, T5ForConditionalGeneration
3 import torch
----> 4 t5_model = T5ForConditionalGeneration.from_pretrained('t5-base')
/databricks/python/lib/python3.8/site-packages/transformers/utils/import_utils.py in __getattr__(cls, key)
786 if key.startswith("_"):
787 return super().__getattr__(cls, key)
--> 788 requires_backends(cls, cls._backends)
789
790
/databricks/python/lib/python3.8/site-packages/transformers/utils/import_utils.py in requires_backends(obj, backends)
774 failed = [msg.format(name) for available, msg in checks if not available()]
775 if failed:
--> 776 raise ImportError("".join(failed))
777
778
ImportError:
T5ForConditionalGeneration requires the PyTorch library but it was not found in your environment. Checkout the instructions on the
installation page: https://pytorch.org/get-started/locally/ and follow the ones that match your environment.
I have installed the torch library using the instructions on the installation page:
pip3 install torch torchvision torchaudio
and imported it as well using import torch.
But, this error persists. It ran without any errors in the Colab notebook.
torch: 1.7.1
transformers: 3.3.0
tokenizers: 0.8.1rc2
sentencepiece: 0.1.96
How can I fix these two errors?
AttributeError Traceback (most recent call last)
in
14 sys.path.append(ROOT_DIR) # To find local version of the library
15 from mrcnn import utils
---> 16 import mrcnn.model as modellib
17 from mrcnn import visualize
18 # Import COCO config
~\ComputerVisionProject\Mask_RCNN_CustomDataset\Mask_RCNN-master\Mask_RCNN-master\mrcnn\model.py in
253
254
--> 255 class ProposalLayer(KE.Layer):
256 """Receives anchor scores and selects a subset to pass as proposals
257 to the second stage. Filtering is done based on anchor scores and
AttributeError: module 'keras.engine' has no attribute 'Layer'
!pip install tensorflow==1.15 keras==2.0.8
Install the above version then restart the run time it will work
I am having a problem as i upload my model in drive and mount it in Colab module imports are working fine in previous cell as i gave the root path and sys.append() it , I am using (tf.compat.v1) you can see it in ouptut, but still getting this error also tried (tensorflow v2) and used tf.io.gfile but still same error. Kindly help me
<module 'tensorflow_core.compat.v1.version' from '/usr/local/lib/python3.6/dist-packages/tensorflow_core/_api/v2/compat/v1/version/__init__.py'>
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-7-e851eb463895> in <module>()
1 print(tf.version)
----> 2 category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name=True)
2 frames
/content/drive/My Drive/models/research/object_detection/utils/label_map_util.py in load_labelmap(path)
136 a StringIntLabelMapProto
137 """
--> 138 with tf.gfile.GFile(path, 'r') as fid:
139 label_map_string = fid.read()
140 label_map = string_int_label_map_pb2.StringIntLabelMap()
AttributeError: module 'tensorflow' has no attribute 'gfile'
Try to downgrade TF 1.15 from TF 2.x, your issue will be resolved.
Please refer code as shown below
!pip uninstall tensorflow
!pip install tensorflow==1.15
import tensorflow as tf
If you are looking for simple solution for Colab, please use %tensorflow_version 1.x magic command, which does for you without un-installation.
%tensorflow_version 1.x
import tensorflow as tf
I am currently trying to import some Python package(matplotlib, pandas & numpy) using Jupyter Notebook. However, when I try to do so, this error appears:
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-2-0aa0b027fcb6> in <module>
----> 1 import numpy as np
~\AppData\Roaming\Python\Python37\site-packages\numpy\__init__.py in <module>
140 from . import _distributor_init
141
--> 142 from . import core
143 from .core import *
144 from . import compat
~\AppData\Roaming\Python\Python37\site-packages\numpy\core\__init__.py in <module>
21 # NOTE: would it change behavior to load ALL
22 # DLLs at this path vs. the name restriction?
---> 23 WinDLL(os.path.abspath(filename))
24 DLL_filenames.append(filename)
25 if len(DLL_filenames) > 1:
~\Anaconda3\lib\ctypes\__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
354
355 if handle is None:
--> 356 self._handle = _dlopen(self._name, mode)
357 else:
358 self._handle = handle
OSError: [WinError 193] %1 is not a valid Win32 application
I have already tried to install these packages using Anaconda prompt via conda install numpy, conda install pandas. I did not encounter such issues importing the package in IDLE using cmd. Any help is much appreciated.
Like you, I had done uninstall/reinstalls of Anaconda and numpy/pandas, uninstalled 32-bit versions, and various other suggestions that people made, but none of that worked. However, notice that the error is referencing ~\AppData\Roaming\Python\....
I was able to solve the problem by deleting ~\AppData\Roaming\Python.
I installed pytorch using pip3 command for my windows pc without GPU support.
But when I tried to import torch it is giving an error.
At first, there was a different error saying numpy version not matching and I updated the numpy to the latest version.
import torch
RuntimeError Traceback (most recent call last)
<ipython-input-10-c031d3dd82fc> in <module>()
----> 1 import torch
C:\Users\iamuraptha\Anaconda3\lib\site-packages\torch\__init__.py in <module>()
82 pass
83
---> 84 from torch._C import *
85
86 __all__ += [name for name in dir(_C)
RuntimeError: generic_type: cannot initialize type "TensorProtoDataType": an object with that name is already defined
I reinstalled anaconda and then created a virtual environment for pytorch.Now everything works fine
If you are running this in colab, remember to restart your runtime after installing your modules with pip.