Pymc3 theano import failure - theano

I have been unable to identify the issue with Pymc3's inability to correctly run Theano. I have also been unable to find a satisfactory answer anywhere else online. I am working in Mac OSX Mojave using Anaconda and Python 3.6. I am trying to run the code on a Jupyter Notebook from "Probabilistic Programming and Bayesian Methods for Hackers".
https://nbviewer.jupyter.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Ch1_Introduction_PyMC3.ipynb
Code breaks with the first attempt to utilize Pymc3:
import pymc3 as pm
import theano.tensor as tt
with pm.Model() as model:
alpha = 1.0/count_data.mean() # Recall count_data is the
# variable that holds our txt counts
lambda_1 = pm.Exponential("lambda_1", alpha)
lambda_2 = pm.Exponential("lambda_2", alpha)
tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data - 1)
Error code follows, below. Thanks in advance.
ImportError Traceback (most recent call last)
/anaconda3/lib/python3.6/site-packages/theano/gof/lazylinker_c.py in <module>
80 version,
---> 81 actual_version, force_compile, _need_reload))
82 except ImportError:
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
/anaconda3/lib/python3.6/site-packages/theano/gof/lazylinker_c.py in <module>
104 version,
--> 105 actual_version, force_compile, _need_reload))
106 except ImportError:
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True
During handling of the above exception, another exception occurred:
Exception Traceback (most recent call last)
/anaconda3/lib/python3.6/site-packages/theano/gof/vm.py in <module>
673 raise theano.gof.cmodule.MissingGXX('lazylinker will not be imported if theano.config.cxx is not set.')
--> 674 from . import lazylinker_c
675
/anaconda3/lib/python3.6/site-packages/theano/gof/lazylinker_c.py in <module>
139 cmodule.GCC_compiler.compile_str(dirname, code, location=loc,
--> 140 preargs=args)
141 # Save version into the __init__.py file.
/anaconda3/lib/python3.6/site-packages/theano/gof/cmodule.py in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols)
2398 raise Exception('Compilation failed (return status=%s): %s' %
-> 2399 (status, compile_stderr.replace('\n', '. ')))
2400 elif config.cmodule.compilation_warning and compile_stderr:
Exception: Compilation failed (return status=1): In file included from /Users/kthln/.theano/compiledir_Darwin-18.7.0-x86_64-i386-64bit-i386-3.6.9-64/lazylinker_ext/mod.cpp:1:. In file included from /anaconda3/include/python3.6m/Python.h:25:. /anaconda3/bin/../include/c++/v1/stdio.h:108:15: fatal error: 'stdio.h' file not found. #include_next <stdio.h>. ^~~~~~~~~. 1 error generated..
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
<ipython-input-48-f2e1580aa39f> in <module>
1 import pymc3 as pm
----> 2 import theano.tensor as tt
3
4
5 with pm.Model() as model:
/anaconda3/lib/python3.6/site-packages/theano/__init__.py in <module>
108 object2, utils)
109
--> 110 from theano.compile import (
111 SymbolicInput, In,
112 SymbolicOutput, Out,
/anaconda3/lib/python3.6/site-packages/theano/compile/__init__.py in <module>
10 from theano.compile.function_module import *
11
---> 12 from theano.compile.mode import *
13
14 from theano.compile.io import *
/anaconda3/lib/python3.6/site-packages/theano/compile/mode.py in <module>
9 import theano
10 from theano import gof
---> 11 import theano.gof.vm
12 from theano import config
13 from six import string_types
/anaconda3/lib/python3.6/site-packages/theano/gof/vm.py in <module>
681 except ImportError:
682 pass
--> 683 except (OSError, theano.gof.cmodule.MissingGXX) as e:
684 # OSError happens when g++ is not installed. In that case, we
685 # already changed the default linker to something else then CVM.
AttributeError: module 'theano' has no attribute 'gof'

Disregard. I noticed with an update I was running Python 3.7. Set up a separate conda env with Python 3.6 and it works fine.

Related

RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback): initialization failed

Goal: Run a GPT-2 model instance.
I am using the latest Tensorflow and Hugging Face 🤗 Transformers.
Tensorflow - 2.9.1
Transformers - 4.21.1
Notebook:
pip install tensorflow
pip install transformers
from transformers import pipeline, set_seed
generator = pipeline('text-generation', model='gpt2')
set_seed(42)
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
ImportError: numpy.core.multiarray failed to import
The above exception was the direct cause of the following exception:
SystemError Traceback (most recent call last)
SystemError: <built-in method __contains__ of dict object at 0x7f5b58a64d00> returned a result with an error set
The above exception was the direct cause of the following exception:
ImportError Traceback (most recent call last)
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/utils/import_utils.py in _get_module(self, module_name)
1001 try:
-> 1002 return importlib.import_module("." + module_name, self.__name__)
1003 except Exception as e:
~/anaconda3/envs/python3/lib/python3.8/importlib/__init__.py in import_module(name, package)
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)
128
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _gcd_import(name, package, level)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _find_and_load(name, import_)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _load_unlocked(spec)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap_external.py in exec_module(self, module)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/pipelines/__init__.py in <module>
36 from ..utils import HUGGINGFACE_CO_RESOLVE_ENDPOINT, http_get, is_tf_available, is_torch_available, logging
---> 37 from .audio_classification import AudioClassificationPipeline
38 from .automatic_speech_recognition import AutomaticSpeechRecognitionPipeline
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/pipelines/audio_classification.py in <module>
19 from ..utils import add_end_docstrings, is_torch_available, logging
---> 20 from .base import PIPELINE_INIT_ARGS, Pipeline
21
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/pipelines/base.py in <module>
33 from ..feature_extraction_utils import PreTrainedFeatureExtractor
---> 34 from ..modelcard import ModelCard
35 from ..models.auto.configuration_auto import AutoConfig
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/modelcard.py in <module>
43 )
---> 44 from .training_args import ParallelMode
45 from .utils import (
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/training_args.py in <module>
25 from .debug_utils import DebugOption
---> 26 from .trainer_utils import (
27 EvaluationStrategy,
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/trainer_utils.py in <module>
46 if is_tf_available():
---> 47 import tensorflow as tf
48
~/anaconda3/envs/python3/lib/python3.8/site-packages/tensorflow/__init__.py in <module>
36
---> 37 from tensorflow.python.tools import module_util as _module_util
38 from tensorflow.python.util.lazy_loader import LazyLoader as _LazyLoader
~/anaconda3/envs/python3/lib/python3.8/site-packages/tensorflow/python/__init__.py in <module>
36 from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow
---> 37 from tensorflow.python.eager import context
38
~/anaconda3/envs/python3/lib/python3.8/site-packages/tensorflow/python/eager/context.py in <module>
34 from tensorflow.python import tf2
---> 35 from tensorflow.python.client import pywrap_tf_session
36 from tensorflow.python.eager import executor
~/anaconda3/envs/python3/lib/python3.8/site-packages/tensorflow/python/client/pywrap_tf_session.py in <module>
18 from tensorflow.python import pywrap_tensorflow
---> 19 from tensorflow.python.client._pywrap_tf_session import *
20 from tensorflow.python.client._pywrap_tf_session import _TF_SetTarget
ImportError: initialization failed
The above exception was the direct cause of the following exception:
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_4924/2487422996.py in <cell line: 1>()
----> 1 from transformers import pipeline, set_seed
2
3 generator = pipeline('text-generation', model='gpt2')
4 set_seed(42)
~/anaconda3/envs/python3/lib/python3.8/importlib/_bootstrap.py in _handle_fromlist(module, fromlist, import_, recursive)
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/utils/import_utils.py in __getattr__(self, name)
990 value = self._get_module(name)
991 elif name in self._class_to_module.keys():
--> 992 module = self._get_module(self._class_to_module[name])
993 value = getattr(module, name)
994 else:
~/anaconda3/envs/python3/lib/python3.8/site-packages/transformers/utils/import_utils.py in _get_module(self, module_name)
1002 return importlib.import_module("." + module_name, self.__name__)
1003 except Exception as e:
-> 1004 raise RuntimeError(
1005 f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
1006 f" traceback):\n{e}"
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
initialization failed
def query(payload, multiple, min_tokens, max_tokens):
nlp_setup()
list_dict = generator(payload, min_length=min_tokens, max_new_tokens=max_tokens, num_return_sequences=multiple)
return [d['generated_text'].split(payload)[1].strip() for d in list_dict
output = query("Banking customer's needs:", 3000, 50, 50)
RunTime, SystemError and ImportError all occur during import of transformers:
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback): initialization failed
Changed Kernel: conda_tensorflow2_p38

installed anaconda and when I run pandas it won't work. Is there a fix to this? Do I need to uninstall and reinstall for this to work?

OSError Traceback (most recent call last)
<ipython-input-2-7dd3504c366f> in <module>
----> 1 import pandas as pd
~\AppData\Roaming\Python\Python38\site-packages\pandas\__init__.py in <module>
9 for dependency in hard_dependencies:
10 try:
---> 11 __import__(dependency)
12 except ImportError as e:
13 missing_dependencies.append(f"{dependency}: {e}")
~\AppData\Roaming\Python\Python38\site-packages\numpy\__init__.py in <module>
136
137 # Allow distributors to run custom init code
--> 138 from . import _distributor_init
139
140 from . import core
~\AppData\Roaming\Python\Python38\site-packages\numpy\_distributor_init.py in <module>
24 # NOTE: would it change behavior to load ALL
25 # DLLs at this path vs. the name restriction?
---> 26 WinDLL(os.path.abspath(filename))
27 DLL_filenames.append(filename)
28 if len(DLL_filenames) > 1:
~\Anaconda3\lib\ctypes\__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error,
winmode)
379
380 if handle is None:
--> 381 self._handle = _dlopen(self._name, mode)
382 else:
383 self._handle = handle
OSError: [WinError 193] %1 is not a valid Win32 application
I tried to "import pandas as pd" and this is the output in the jupyter notebook. what does the traceback(most recent call last) mean? Also why are there arrows pointing at specific lines?

Trouble importing modules into ipython

On a freshly installed Windows 10 machine, I installed the newest 64bit version of anaconda. Now when I try importing modules into ipython, I get errors. Some sample errors are reproduced below.
I am new to Python and Programming, so would appreciate a bit of help. This is a brand new Lenovo laptop, so there's no scope for any Python version conflict etc. (since nothing was installed on it earlier).
In [1]: import sklearn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-b7c74cbf5af0> in <module>
----> 1 import sklearn
C:\Software\anaconda3\lib\site-packages\sklearn\__init__.py in <module>
74 else:
75 from . import __check_build
---> 76 from .base import clone
77 from .utils._show_versions import show_versions
78
C:\Software\anaconda3\lib\site-packages\sklearn\base.py in <module>
11 import re
12
---> 13 import numpy as np
14
15 from . import __version__
C:\Software\anaconda3\lib\site-packages\numpy\__init__.py in <module>
138
139 # Allow distributors to run custom init code
--> 140 from . import _distributor_init
141
142 from . import core
C:\Software\anaconda3\lib\site-packages\numpy\_distributor_init.py in <module>
32
33 with RTLD_for_MKL():
---> 34 from . import _mklinit
35
36 del RTLD_for_MKL
ImportError: DLL load failed: The specified module could not be found.
In [2]: import matplotlib.pyplot as plt
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-2-a0d2faabd9e9> in <module>
----> 1 import matplotlib.pyplot as plt
C:\Software\anaconda3\lib\site-packages\matplotlib\__init__.py in <module>
136 # cbook must import matplotlib only within function
137 # definitions, so it is safe to import from it here.
--> 138 from . import cbook, rcsetup
139 from matplotlib.cbook import (
140 MatplotlibDeprecationWarning, dedent, get_label, sanitize_sequence)
C:\Software\anaconda3\lib\site-packages\matplotlib\cbook\__init__.py in <module>
29 from weakref import WeakMethod
30
---> 31 import numpy as np
32
33 import matplotlib
C:\Software\anaconda3\lib\site-packages\numpy\__init__.py in <module>
138
139 # Allow distributors to run custom init code
--> 140 from . import _distributor_init
141
142 from . import core
C:\Software\anaconda3\lib\site-packages\numpy\_distributor_init.py in <module>
32
33 with RTLD_for_MKL():
---> 34 from . import _mklinit
35
36 del RTLD_for_MKL
ImportError: DLL load failed: The specified module could not be found.
In [3]: import pystan
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-3-ca8618bb8e4a> in <module>
----> 1 import pystan
ModuleNotFoundError: No module named 'pystan'
In [4]:
After a lot of google searching, I found a few sites that touch upon some closely related issues (ex. https://github.com/stan-dev/pystan/issues/520). It so happens that one needs to add some anaconda directroties to the Windows path. I added anaconda/Library/bin, anaconda/Scripts to the path. Now everything is working as expected.

Error: While Using Gdal library with Python anaconda Distribution

I am using Python with anaconda distribution for converting a satellite image to array for further processing.But while using .ReadAsArray() function the below error is occurring.I have tried to search about the error but no proper results are coming.I have installed everything using conda command.
from osgeo import gdal
ds = gdal.Open('/home/alay/Desktop/Studies/2010/ndvi01jun2010.tif')
band = ds.GetRasterBand(1)
img= band.ReadAsArray()
The error occuring after executing the above code
ImportError Traceback (most recent call last)
~/.local/lib/python3.6/site-packages/osgeo/gdal_array.py in
swig_import_helper()
17 try:
---> 18 fp, pathname, description = imp.find_module('_gdal_array', [dirname(__file__)])
19 except ImportError:
~/anaconda3/lib/python3.6/imp.py in find_module(name, path)
296 else:
--> 297 raise ImportError(_ERR_MSG.format(name), name=name)
298
ImportError: No module named '_gdal_array'
During handling of the above exception, another exception occurred:
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-4-eae4e7981f3a> in <module>()
----> 1 img= band.ReadAsArray()
~/.local/lib/python3.6/site-packages/osgeo/gdal.py in ReadAsArray(self, xoff, yoff, win_xsize, win_ysize, buf_xsize, buf_ysize, buf_type, buf_obj, resample_alg, callback, callback_data)
2603 parameters should generally not be specified if buf_obj is specified. The array is returned"""
2604
-> 2605 from . import gdalnumeric
2606
2607 return gdalnumeric.BandReadAsArray( self, xoff, yoff,
~/.local/lib/python3.6/site-packages/osgeo/gdalnumeric.py in <module>()
----> 1 from osgeo.gdal_array import *
2 from numpy import *
~/.local/lib/python3.6/site-packages/osgeo/gdal_array.py in <module>()
26 fp.close()
27 return _mod
---> 28 _gdal_array = swig_import_helper()
29 del swig_import_helper
30 else:
~/.local/lib/python3.6/site-packages/osgeo/gdal_array.py in swig_import_helper()
18 fp, pathname, description = imp.find_module('_gdal_array', [dirname(__file__)])
19 except ImportError:
---> 20 import _gdal_array
21 return _gdal_array
22 if fp is not None:
ModuleNotFoundError: No module named '_gdal_array'

How do you run tensorflow on colaboratory?

I'm trying to use Google's Colaboratory. The first line of output was hopeful.
Using TensorFlow backend.
But then I got this error message.
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
That error apparently triggered another error. (Sorry for the long stack trace, but the error message itself suggested including it when asking for help.)
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
<ipython-input-6-0b846698ae4a> in <module>()
2 import numpy as np
3 import matplotlib.pylab as plt
----> 4 from keras.models import Sequential
5 from keras.layers import Dense, InputLayer
6
/usr/local/lib/python3.6/dist-packages/keras/__init__.py in <module>()
1 from __future__ import absolute_import
2
----> 3 from . import utils
4 from . import activations
5 from . import applications
/usr/local/lib/python3.6/dist-packages/keras/utils/__init__.py in <module>()
4 from . import data_utils
5 from . import io_utils
----> 6 from . import conv_utils
7
8 # Globally-importable utils.
/usr/local/lib/python3.6/dist-packages/keras/utils/conv_utils.py in <module>()
7 from six.moves import range
8 import numpy as np
----> 9 from .. import backend as K
10
11
/usr/local/lib/python3.6/dist-packages/keras/backend/__init__.py in <module>()
82 elif _BACKEND == 'tensorflow':
83 sys.stderr.write('Using TensorFlow backend.\n')
---> 84 from .tensorflow_backend import *
85 else:
86 raise ValueError('Unknown backend: ' + str(_BACKEND))
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in <module>()
3 from __future__ import print_function
4
----> 5 import tensorflow as tf
6 from tensorflow.python.training import moving_averages
7 from tensorflow.python.ops import tensor_array_ops
/usr/local/lib/python3.6/dist-packages/tensorflow/__init__.py in <module>()
22
23 # pylint: disable=g-bad-import-order
---> 24 from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
25 # pylint: disable=wildcard-import
26 from tensorflow.tools.api.generator.api import * # pylint: disable=redefined-builtin
/usr/local/lib/python3.6/dist-packages/tensorflow/python/__init__.py in <module>()
47 import numpy as np
48
---> 49 from tensorflow.python import pywrap_tensorflow
50
51 # Protocol buffers
/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
72 for some common reasons and solutions. Include the entire stack trace
73 above this error message when asking for help.""" % traceback.format_exc()
---> 74 raise ImportError(msg)
75
76 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long
ImportError: Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/usr/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Below that was a suggestion to consult https://www.tensorflow.org/install/install_sources#common_installation_problems.
But that page is about installing TensorFlow from source. I'm not trying to install from source. I just want a runnable version that I can call.
Any suggestions would be appreciated.
Thanks.

Resources