I am attempting to run some simple PyMC3 code in PyCharm on MacOS Big Sur but the execution fails and throws the following error:
Exception: ('Compilation failed (return status=1): ld: library not found for -lSystem. clang-10: error: linker command failed with exit code 1 (use -v to see invocation). ', '[Elemwise{add,no_inplace}(TensorConstant{1.0}, TensorConstant{1.0})]')
I am using PyCharm 2020.3.2. I am able to install pymc3 (version 3.8) and theano (1.0.4) into the environment. Code I am attempting to run follows:
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
import pymc3 as pm
np.random.seed(123)
n_experiments = 4
theta_real = 0.35
data = stats.bernoulli.rvs(p=theta_real, size=n_experiments)
with pm.Model() as our_first_model:
theta = pm.Beta('theta', alpha=1, beta=1)
y = pm.Bernoulli('y', p=theta, observed=data)
start = pm.find_MAP()
step = pm.Metropolis()
trace = pm.sample(1000, step=step, start=start)
burnin = 100
chain = trace[burnin:]
pm.traceplot(chain, lines={'theta':theta_real});
I disabled Apple System Integrity Protection on some speculation that maybe Apple hiding /usr/include might be the source of the problem; it made no difference. No clue how else to fix this.
From the information in comments it seems like your anaconda is broken.
Update anaconda and try again.
If that doesn't work, try to install PyMC3 in the system Python not the Anaconda one.
This will use the system clang and not the Anaconda clang.
Related
The following code is supposed to display thumbnails of different images that are available in the \data directory. However, something is wrong with it.
from os import listdir
from os.path import isfile, join
mypath = "data/"
file_names = [mypath+f for f in listdir(mypath) if isfile(join(mypath, f))]
number_files = len(file_names)
fig = plt.figure(figsize = (40,50))
fig.subplots()
axes = []
for i ,file_name in enumerate(file_names):
dataset = pydicom.dcmread(file_name)
axes.append(fig.add_subplot(int(math.sqrt(number_files))+1 , int(math.sqrt(number_files))+1, i+1))
plt.imshow(dataset.pixel_array, cmap=plt.cm.bone)
plt.show()
I have the following error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-73-3ed68a41a4d3> in <module>
10 dataset = pydicom.dcmread(file_name)
11 axes.append(fig.add_subplot(int(math.sqrt(number_files))+1 , int(math.sqrt(number_files))+1, i+1))
---> 12 plt.imshow(dataset.pixel_array, cmap=plt.cm.bone)
13 plt.show()
...
RuntimeError: The following handlers are available to decode the pixel data however they are missing required dependencies: GDCM (req. GDCM)
I've tried to do !conda install -c conda-forge gdcm -y but I've gotthe following error:
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- gdcm -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|3.4.*']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
Any idea how to fix the error?
One alternative to GDCM that'll be available in the next released version of pydicom is pylibjpeg (disclosure: I'm a contributor to pydicom and author of pylibjpeg). You can use pylibjpeg with pydicom v2.0 simply by importing it before calling Dataset.pixel_array:
Installation
pip install pylibjpeg pylibjpeg-libjpeg
Usage
from pydicom import dcmread
import pylibjpeg
ds = dcmread('path/to/file.dcm')
arr = ds.pixel_array
If don't absolutely need python 3.8, you could try to create a new environment with a python 3.7 interpreter by using the following command line:
conda create -n myenv python=3.7
You could also try to install the dependency with pip from your existing environment.
Would you help me to achieve reproducible results with Tensorflow 1.15 without restarting Python kernel. And why the output results in TF 2.0 and TF 1.5 are different with absolutely identical parameters and dataset? Is it possible to achieve identical output?
More details:
I tried to interpret model results in TF 2.0 by:
import shap
background = df3.iloc[np.random.choice(df3.shape[0], 100, replace=False)]
explainer = shap.DeepExplainer(model, background)
I recieved an error:
`get_session` is not available when using TensorFlow 2.0.`get_session` is not available when using TensorFlow 2.0.
According to the SO topic, I tried to setup TF 2.0 compatibility with TF 1 by using in the front of my code:
import tensorflow.compat.v1 as tf
But the error appeared again.
Following advice by many users, I downgraded TF2 to TF 1.15 it solved the problem, and shap module interprets the results but:
1) to make results reproducible now I have to change tf.random.set_seed(7) on tf.random.set_random_seed(7) and restart the Python kernel every time! In TF2 I didn't have to restart the kernel.
2) prediction results has been changed, especially, Economical efficiency (that is, TF1.5. wrongly classifies more important samples than TF2.0).
TF 2:
Accuracy: 94.95%, Economical efficiency = 64%
TF 1:
Accuracy: 94.85%, Economical efficiency = 56%
The code of the model is here
First, results differ from each other not only in TF1 and TF2 versions, but also in TF2.0 and TF2.2 versions. Probably, it depends on diffenent internal parameters in the packages.
Second, TensorFlow2 works with DeepExplainer in the following versions:
import tensorflow
import pandas as pd
import keras
import xgboost
import numpy
import shap
print(tensorflow.__version__)
print(pd.__version__)
print(keras.__version__)
print(xgboost.__version__)
print(numpy.__version__)
print(shap.__version__)
output:
2.2.0
0.24.2
2.3.1
0.90
1.17.5
0.35.0
But you will face some difficulties in updating the libraries.
In Python 3.5, running TF2.2, you will face the error 'DLL load failed: The specified module could not be found'.
It 100% can be solved by installing newer C++ package. See this:https://github.com/tensorflow/tensorflow/issues/22794#issuecomment-573297027
Link to download the package:https://support.microsoft.com/ru-ru/help/2977003/the-latest-supported-visual-c-downloads
In Python 3.7 you will not find the shap 0.35.0 version with whl extention. Only tar.gz extension which gives the error: "Install visual c++ package". But installation doesn't help.
Then download shap 0.35.0 for Python 3.7 here: https://anaconda.org/conda-forge/shap/files. Run Anaconda shell. Type: conda install -c conda-forge C:\shap-0.35.0-py37h3bbf574_0.tar.bz2.
I just got Ubuntu and I'm trying to experiment abit with emacs. Wrote a simple python code that was supposed to make a plot, but when I try to run the code I get this message;
ImportError: cannot import name '_imaging' from 'PIL' (/usr/lib/python3/dist-packages/PIL/init.py)
And also I get alot of other random stuff, but I think this is the actual error.
I generally want to run the code from the terminal window, and then get out the plot.
Here is the code:
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0,2.0,0.01)
s = 1 + np.sin(2*np.pi*t)
plt.plot(t,s)
plt.show()
I fixed the issue! Just used
pip install --upgrade --force-reinstall Pillow
I am running Theano with Anaconda on Windows. Ihave pretty much followed the steps in the comments here. I can import theano with no problems:
import theano
from theano import tensor as T
x = T.vector('x')
W = T.matrix('W')
dot = T.dot(x, W)
This works fine. But when I do
f = theano.function(inputs=[x, W], outputs=dot)
I get warnings:
WARNING (theano.gof.compilelock): Overriding existing lock by dead process '4844' (I am process '3908')
WARNING:theano.gof.compilelock:Overriding existing lock by dead process '4844' (I am process '3908')
and a long error, which ends with:
ImportError: ('The following error happened while compiling the node', CGemv{inplace}(AllocEmpty{dtype='float64'}.0, TensorConstant{1.0}, W.T, x, TensorConstant{0.0}), '\n', 'DLL load failed: The specified module could not be found.', '[CGemv{inplace}(<TensorType(float64, vector)>, TensorConstant{1.0}, W.T, x, TensorConstant{0.0})]')
Any ideas on how to fix this?
Theano on Windows need Theano development version and not last Theano version:
http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions
So just update Theano and it should work.
I'm not able to import the following modules in an ipython notebook:
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
The following error pops up
ImportError: cannot import name PolynomialFeatures
The same error also appears for make_pipeline.
I'm a newbie in scikit-learn, please help out.
I'm using the miniconda installation of python and the version number for scikit-learn is 0.14.1.
Polynomial Features is included for next version of scikit-learn and is not available in 0.14.1. Please update to 0.15-git if you want to use it. The same holds for make pipeline.
To get the bleeding edge version:
git clone git://github.com/scikit-learn/scikit-learn.git
python setup.py build_ext --inplace
Please read: http://scikit-learn.org/stable/developers/index.html#git-repo
You have to check your current version of scikit:
import sklearn
print sklearn.__version__
if it is less than 0.15.0, then you have to upgrade it. In addition to an excellent answer of Abhishek, you can follow official installation process (which is described for various OS).
If you are using pyCharm, it can be done even simpler: File -> Settings -> Project Interpreter and then select your package and click upgrade
(I selected another one, be cause my scikitlearn is the newest)