My OS is Ubuntu 22.04
I only install Anaconda from Anaconda3-2022.10-Linux-x86_64.sh
And then run conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
Then, import torch gives
File “/home/fenaux/anaconda3/lib/python3.9/site-packages/torch/init.py”, line 191, in
_load_global_deps()
File “/home/fenaux/anaconda3/lib/python3.9/site-packages/torch/init.py”, line 153, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File “/home/fenaux/anaconda3/lib/python3.9/ctypes/init.py”, line 382, in init
self._handle = _dlopen(self._name, mode)
OSError: libcufft.so.10: cannot open shared object file: No such file or directory
when I check in anaconda3/lib I only find libcufft.so.11.
I also tried with
conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia
and the result is exactly the same.
I have no other environments nor installations of PyTorch.
Thanks for helping me.
Problem was solved with a PR in PyTorch see here :
https://discuss.pytorch.org/t/import-torch-fails-with-fresh-conda-installation/168167
Related
Whenever I try to move a variable to cuda in pytorch (e.g. torch.zeros(1).cuda(), I get the error message "AssertionError: Torch not compiled with CUDA enabled". Besides,torch.cuda.is_available() returns False.
I have read several answers to approaching this error but for some reason several attempts to reinstall cuda and putorch didn't change anything. Here are some of the settings I used:
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
conda install pytorch torchvision cudatoolkit=11 -c pytorch-nightly
conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
Yet the same error remains. What could be the issue?
Some settings:
I'm using Ubuntu 20.04, GPU is RTX 2080, nvidia-smi works fine (NVIDIA-SMI 460.91.03, Driver Version: 460.91.03, (max possible) CUDA Version: 11.2)
Try installing with pip
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
You can go through this thread for detailed explanations
Pytorch for cuda 11.2
I am using jetson NX xavier kit having cuda 10.2.89, open Cv 4.1.1 and tensorRT 7.1.3 . Trying to install pytorch. Tried installing with this line
conda install pytorch torchvision cpuonly -c pytorch
but when i write this line
import torch
It throws an error by saying that module not installed.
How I can verify if pytorch has been installed correctly.
Try this one
conda install -c pytorch pytorch
After executing this command, you need to enter yes(if prompted in the command line) for installing all the related packages. If there is no conflict while installing the libraries, the PyTorch library will be installed.
To check if it is properly installed or not, type the command python in your command line and type import torch to check if it is properly installed or not.
I am trying to create an isolated environment using pip, the instructions from "Hands on Machine Learning With Ski-Kit and Tensor Flow" have me run these lines of code and this is the output I get. I've tried uninstalling pipenv and that creates another error where pipenv command not found. Not sure how to work around this, can anyone help? I'm using a Mac with python3.9
Darrens-MacBook-Air:~ odonned4$ cd $ML_PATH
Darrens-MacBook-Air:~ odonned4$ virtualenv env
File "/usr/local/bin/virtualenv", line 6, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3241, in <module>
#_call_aside
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3225, in _call_aside
f(*args, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 585, in _build_master
return cls._build_from_requirements(__requires__)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 598, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'platformdirs<3,>=2' distribution was not found and is required by virtualenv```
What is the problem?
You are trying to run Python 3.9, right? Well, according to your error message, it seems like you're invoking Python 2.7:
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3241, in <module>
^^^
This confusion happens because macOS added Python 2.7 to the distribution for... compatibility reasons. And, probably, you have virtualenv on that distribution, but you don't have platformdirs, so creating a virtualenv fails.
How to solve the problem?
There are many ways to clarify this confusion. A good way is using python3 -m virtualenv (instead of virtualenv). That should create a virtualenv with Python 3.
Also, you can get sure that the virtualenv package is available in your Python 3.9 installation. To identify that, see if virtualenv appears when typing python3 -m pip list. If not, run python3 -m pip install virtualenv.
The real problem here is that Mac is hard wired to use python2. When using virtualenv, you might have run the command:
$ /usr/bin/easy_install virtualenv
which should have added virtualenv to the /usr/bin/ for direct use, but to be used with the default python2 instead of python3.9 which you might have installed on your own.
A simple way around is to use
$ python3 -m virtualenv <env-name>
instead of
$ virtualenv <env-name>
Or in my case, add an alias in .bash_profile:
$ alias virtualenv="python3 -m virtualenv"
which is a duct tape solution but it works as well.
Try uninstalling setuptools with
pip uninstall -y setuptools
and reinstalling it with
pip install setuptools
OR
Upgrade setuptools with
pip install --upgrade setuptools
OR
For me i have faced this issue some before in python 3.8.
I fixed it by
sudo apt install --reinstall python3-pkg-resources python3-setuptools
OR
Uninstall virtualenv and setuptools first and find pip cache folder.
pip cache dir
Delete the cache and install virtualenv and setuptools.
I JUST SAID SOME STEPS .. TRY THIS . MAY HELP : )
I'm unable to install matplotlib through pip on my M1 Mac. I have Python 3.9.1 installed through Homebrew. I've tried the solution here: Pip install matplotlib fails on M1 Mac but it does not work for me.
I get this long error:
ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so, 2): no suitable image found. Did find:
/opt/homebrew/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so: mach-o, but wrong architecture
/opt/homebrew/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so: mach-o, but wrong architecture
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-install-mrz7uc56/matplotlib_20151701613b4e6fb6955488fa89e81e/setup.py", line 256, in <module>
setup( # Finally, pass this all along to distutils to do the heavy lifting.
File "/opt/homebrew/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/opt/homebrew/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/opt/homebrew/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
mm.run()
File "/opt/homebrew/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/opt/homebrew/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 577, in add_defaults
sdist.add_defaults(self)
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/sdist.py", line 228, in add_defaults
self._add_defaults_ext()
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/sdist.py", line 311, in _add_defaults_ext
build_ext = self.get_finalized_command('build_ext')
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/opt/homebrew/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/private/tmp/pip-install-mrz7uc56/matplotlib_20151701613b4e6fb6955488fa89e81e/setup.py", line 88, in finalize_options
self.distribution.ext_modules[:] = [
File "/private/tmp/pip-install-mrz7uc56/matplotlib_20151701613b4e6fb6955488fa89e81e/setup.py", line 91, in <listcomp>
for ext in package.get_extensions()
File "/private/tmp/pip-install-mrz7uc56/matplotlib_20151701613b4e6fb6955488fa89e81e/setupext.py", line 345, in get_extensions
add_numpy_flags(ext)
File "/private/tmp/pip-install-mrz7uc56/matplotlib_20151701613b4e6fb6955488fa89e81e/setupext.py", line 468, in add_numpy_flags
import numpy as np
File "/opt/homebrew/lib/python3.9/site-packages/numpy/__init__.py", line 145, in <module>
from . import core
File "/opt/homebrew/lib/python3.9/site-packages/numpy/core/__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
I am able to import matplotlib with python 2.7 on my Rosetta Terminal but I am trying to use matplotlib with python3 instead.
Any help would be appreciated!
November 2021 Update
Just run the following command:
$ pip3 install matplotlib
Worked on Mac mini (M1, 2020), Monterey 12.0.1. Python 3.9.7
Feb 2021 Answer Below
In order to install matplotlib with pip3 on an M1 mac, you first need to install the dependencies Cython, numpy and Pillow from source. The following method worked for me.
Install Cython
pip3 install Cython
Install numpy
$ git clone https://github.com/numpy/numpy.git
$ cd numpy
$ pip3 install . --no-binary :all: --no-use-pep517
Install Pillow
libjpeg is necessary for Pillow so you might want to install it via brew.
$ brew install libjpeg
$ git clone https://github.com/python-pillow/Pillow.git
$ cd Pillow
$ pip3 install . --no-binary :all: --no-use-pep517
Install matplotlib
$ git clone https://github.com/matplotlib/matplotlib.git
Then you need to manually download qhull and extract the archive. (http://www.qhull.org/download/qhull-2020-src-8.0.2.tgz on Qhull Downloads)
You will get a folder named qhull-2020.2. You then have to place the folder at matplotlib/build. The build folder may not exist so you might have to create it.
Lastly, the following command will install the matplotlib on your M1 mac.
$ cd matplotlib
$ pip3 install . --no-binary :all:
Tested for Python 3.9.1 on non-Rosetta terminal on M1 Mac mini (Big Sur 11.2.1).
I solved this by just uninstalling homebrew first and downloaded matplotlib with python3. After that i reinstalled homebrew using non-rosetta terminal. Idk how it works cause im very new to these stuff but what works works i guess.
I solved this by realizing that I had two seperate Python instances installed. When you run which python3 you get the path of your Python installation. Check to see if the path corresponds to the Python version you installed.
I fixed it by uninstalling the version of Python3 I had installed with homebrew with brew uninstall python3.
When I did which python3 again it showed a different Python installation path. I then did pip3 install Matplotlib and I was able to import Matplotlib.
1. Uninstall Following Package
pip uninstall pillow
pip uninstall numpy
pip uninstall wordcloud
pip uninstall matplotlib
pip uninstall pyparsing
pip uninstall kiwisolver
pip uninstall python-dateutil
pip uninstall cycler
pip uninstall packaging
pip uninstall fonttools
pip uninstall contourpy
pip uninstall six
2. Install matplotlib
pip install matplotlib
I originally installed python 3.9 using homebrew, but my matplotlib problem was fixed by uninstalling python and reinstalling using the download from the python website(because of architecture problems).
I found another nice way with pyenv,miniforge and conda.
pyenv install miniforge3-4.10
conda create -n env_new python=3.9
conda activate env_new
conda install poetry
peotry new project
poetry add matplotlib
Maybe it is not perfect, but it works for a variety of moduls and you can even choose python=3.8 and other python versions and everything is running on the M1.
Pip installing Pytorch 1.7.1 here on Gtx1660(latest drivers installed) doesn't recognise the installed Cuda toolkit 10.2 on my machine(Windows10).
As this is a personal project, I don't wish to use Anaconda.
How to resolve this?
#INSTALLATION (10.2 CUDA SUPPORT)
pip install torch===1.7.1 torchvision===0.8.2 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
#CODE
import torch
device = torch.device("cuda")
# device = torch.device("cuda:0") #ALSO NOT WORKING
cuda_available = torch.cuda.is_available()
cuda_init = torch.cuda.is_initialized()
print(f'device_current = {device},cuda_available = {cuda_available} and cuda_init = {cuda_init}\n')
torch.cuda.init()
print(f'device_current = {device} and cuda_init = {cuda_init}')
#TERMINAL
device_current = cuda:0,cuda_available = False and cuda_init = False
Traceback (most recent call last):
File "f:\Script\Ai\Pytorch\mod_test\test1.py", line 8, in <module>
torch.cuda.init()
File "C:\Users\User0\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\cuda\__init__.py", line 137, in init
_lazy_init()
File "C:\Users\User0\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\cuda\__init__.py", line 166, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
I could be wrong, but here is what I found while searching on their package registry.
For a PyTorch 1.7.1 pip install. The instructions on pytorch.org are:
## cu101
pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
## cu110
pip install torch===1.7.1+cu110 torchvision===0.8.2+cu110 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
Yet the command for CUDA 10.2 support is:
## cu102
pip install torch===1.7.1 torchvision===0.8.2 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
This seems off because the CUDA version is not stated anywhere in the command. Yet download.pytorch.org provides support for this particular PyTorch version with cu92, cu101, cu102, and cu110.
You can either use:
pip install torch==1.7.0 torchvision==0.8.1 -f https://download.pytorch.org/whl/cu102/torch_stable.html
or try this instead from the main Torch stable directory:
pip install torch==1.7.1+cu102 torchvision==0.8.2+cu102 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html