pip install cupy won't install cupy without cuda.
Linux: lubuntu v21.10
pip v3.10
conda v22.11.1
I have linux installed on an older laptop and it doesn't have any dedicated GPU.
I have a python framework I'm trying to test out and it requires cupy to run. I tried pip install cupy but it wouldn't run without any cuda installation.
I tried conda install cupy but the framework won't run when conda install cuda
https://pypi.org/project/cupy/ is "NumPy & SciPy for GPU" (emphasize mine — phd). You cannot install it without CUDA.
If you want to use pyVHR without GPU you need to switch to the branch pyVHR_CPU. There is pyVHR_CPU_env.yml to create CPU-ony conda environment. See the installation instructions.
Another approach would be to install an emulator (qemu, VirtualBox, etc.) and configure it to emulate a GPU. Not sure if it worth the trouble in terms of speed.
Related
I installed the transformers in the Macbook Pro M1 Max
Following this, I installed the tokenizers with
pip install tokenizers
It showed
Collecting tokenizers
Using cached tokenizers-0.12.1-cp39-cp39-macosx_12_0_arm64.whl
Successfully installed tokenizers-0.12.1
It seems to use the correct architecture for the whl file
When I import it I get
'/Users/myname/miniforge3/envs/tf/lib/python3.9/site-packages/tokenizers/tokenizers.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))
I see that this problem used to happen to others before. Any thoughts on how to fix this?
James Briggs method works but produces the following error
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
The Issue
After installing Rust and Cargo, we must source the environment file. This is the missing step in the previous answer.
The Solution
The workaround to solving this is to type the following in the terminal, right after installing Rust:
source "$HOME/.cargo/env"
Then, you can install transformers with the following code snippet:
pip install transformers
If using Anaconda we switch to a terminal window and create a new ARM environment like so:
CONDA_SUBDIR=osx-arm64 conda create -n ml python=3.9 -c conda-forge
now get in to ml envoriment
conda activate ml
run inside the env
conda env config vars set CONDA_SUBDIR=osx-arm64
needs to restart env
conda deactivate
get into to env
conda activate ml
PyTorch Installation
To get started we need to install PyTorch v1.12. For now, this is only available as a nightly release.
pip3 install -U --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu
Side note: The transformers library uses tokenizers built in Rust (it makes them faster). Because we are using this new ARM64 environment we may get ERROR: Failed building wheel for tokenizers. If so, we install Rust (in the same environment) with:
curl — proto ‘=https’ — tlsv1.2 -sSf https://sh.rustup.rs | sh
restart your env
conda deactivate
conda activate ml
than you can install transformer comes with tokenizers or only install tokenizers
pip install tokenizers or pip install transformer
thanks to James Briggs
You can try
conda install -c huggingface transformers
I got this error too. Solved it after a lot of trial & error.
The Problem: my brew was still running on Rosetta. Fixed that by uninstalling, cleaning and reinstalling. So everything seemed to run fine. Except this problem still kept cropping up
Until I discovered that pip is quite agressive in caching. So it caches the build even if the architecture changed. Solution: pip cache purge. Or remove the whole cache directory which you find with pip cache info
After testing most of the solutions provided I finally got it working by doing
brew install ffmpeg
sudo pip install tokenizers
🛠️🚀
I am running CNN on PyTorch. The torch.cuda.is_available() function returned false and no GPU is detected. However, I can run Keras model with GPU. Here is my system information:
OS: Ubuntu 18.04.3
Python 3.7.3 (Conda)
GPU: GTX1080Ti
Nvidia driver: 430.50
When I check nvidia-smi, the output said that the CUDA version is 10.1. However, the nvcc -V command tells me that it is CUDA 9.1.
I downloaded NVIDIA-Linux-x86_64-430.50.run from the official site and install it with command line. I installed CUDA 10.1 using these following command line recommended by the official site:
wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_418.87.00_linux.run
sudo sh cuda_10.1.243_418.87.00_linux.run
I installed PyTorch through pip install. What is wrong? Thanks in advance!
The default Pytorch 1.2 package depends on CUDA 10.0, but you have CUDA 9.1. The output of nvidia-smi just tells you the maximum CUDA version your GPU supports, nvcc gives the CUDA installed on your system. It seems that your installation of CUDA 10.1 was unsuccessful.
In addition to CUDA 10.0, Pytorch also supports CUDA 9.2 and I've found that the Pytorch package compiled for CUDA 10.0 also works with CUDA 10.1. So you can either upgrade your CUDA installation to 9.2 and install the Pytorch CUDA 9.2 package with
pip3 install torch==1.2.0+cu92 torchvision==0.4.0+cu92 -f https://download.pytorch.org/whl/torch_stable.html
Or get a working installation of CUDA 10.1. There are detailed Linux instructions here. (Note that you may have to remove previous installations of CUDA before installing a new one.)
FYI, this answer is a hack which could mess up your conda env, but may work more easily than installing a fresh env. A consistency-checking tool would be really helpful because of all the people having exactly this problem. Matching anaconda's CUDA version with the system driver and the actual hardware and the other system environment settings is challenging to say the least and almost an art.
I found that Anaconda improperly guesses the CUDA version to use frequently. So I have found the best way to fix this is to surgically uninstall and reinstall just pytorch with pip:
pip uninstall torch
pip install torch
Note that pip calls pytorch torch while conda calls it pytorch.
However, I also found that pip sometimes refuses to reinstall torch because it didn't get rid of the anaconda site package files. If that is the case you can very carefully remove them manually as:
rm -fr $HOME/miniconda3/envs/<ENV>/lib/python3.9/site-packages/torch/
rm -fr $HOME/miniconda3/envs/<ENV>/lib/python3.9/site-packages/torch-*.dist-info/
where should be replaced with your environment name and miniconda might be anaconda or something else depending on your installation.
Be very careful not to delete anything other than the torch files or you may mess something else up. Then you would be best served by installing yet another fresh environment.
After this pip install torch should work and torch.cuda.is_available() should return True. Unless there is another problem... YMMV.
Note that I recommend using miniconda because the full anaconda comes overloaded with packages and I find it quickly gets clogged and broken.
I am a little bit confused....
I installed anaconda on my computer (I have windows 10).
Normally, when I want to install a package I simply do "pip install package_name" or "conda install package_name" and it is done.
First question: what is the difference between pip and conda?
Now I tried to install xgboost and it was really complicated I tried lot of things nothings worked until I install something called miniconda.
There it works but now, when I do "conda install package_name" it install it in miniconda3/lib/site _package and I have to copy/paste it in Anaconda3/lib/site_package if I want it to work.
Second question: how can I ask to the computer that "conda install
package_name" install it directly in anaconda3 and not miniconda3?
Finally I tried to install the package "surprise" for recommended systems. Both "pip install" or "conda install" failed.
I went in github and got the file "surprise" from https://github.com/NicolasHug/Surprise
I tried to copy it in Anaconda3/lib/site_package but it doesn't work.
When I do from surprise import Reader I did not get the error "no module name surprise" anymore but I get "cannot import name 'Reader'"
Last question: how can I make it work? I think I have to build it but
I do not now how...
Thank you in advance for anyone that can explain all this for me :-)
Similarly to you, I had issues installing the surprise package.
I tried both pip install surprise and conda install surprise unsuccessfully.
conda install -c conda-forge scikit-surprise
conda install -c conda-forge/label/gcc7 scikit-surprise
conda install -c conda-forge/label/cf201901 scikit-surprise
I found those on the anconda website and the first one worked for me.
Hopefully this would help you as well
pip vs conda
pip is a package manager that facilitates installation, upgrade,
and uninstallation of python packages. It also works with virtual python environments.
conda is a package manager for any software (installation, upgrade and uninstallation).
It also works with virtual system environments.
Conda is a packaging tool and installr that aims to do more than what pip does;
handle library dependencies outside of the Python packages as well as the Python packages themselves.
Conda also creates a virtual environment, like virtualenv does.
For more see here
Anaconda vs miniconda
The open source version of Anaconda is an easy-to-install
high performance Python and R distribution with a package manager,
environment manager and collection of 720+ open source packages.
It also comes with the options to install RStudio.
The "lite" version of Anaconda without the collection of 720 packages.
The downside is that you need to type in command line commands,
"conda install PACKAGENAME"
And Last
To install this package with conda run:
conda install -c anaconda py-xgboost=0.60
Update for surprise
The easiest way is to use pip (you'll need numpy):
$ pip install numpy
$ pip install scikit-surprise
Or you can clone the repo and build the source (you'll need Cython and numpy):
$ git clone https://github.com/NicolasHug/surprise.git
$ python setup.py install
I am trying to install TensorFlow. The installation instruction for Windows (https://www.tensorflow.org/install/install_windows) have as first step to install Python 3.5.2. And I'm doing the 'TensorFlow with CPU support only'.
Python was successfully installed in my computer as I can run it via the Start menu.
However, when I try to do the 2nd step of the installation instructions in order to install TensorFlow, this step says to:
To install TensorFlow, start a terminal. Then issue the appropriate pip3 install command in that terminal. To install the CPU-only version of TensorFlow, enter the following command:
C:\> pip3 install --upgrade tensorflow
But I'm getting an error when I perform the above statement, the error is
'pip' is not recognized as an internal or external command, oprable program or batch file.
I looked at several postings in StackOverflow and tried the commands provided in one of the postings, but I would get the same type of error.
So, how is 'pip3' installed? from what I read, it is supposed to be installed together with the installation, but obviously that did not happen.
How do I install it? I need to install TensorFlow and it seems that it needs to be done via the pip3 installation tool.
Thank you in advance for your help!
Either set the system environment path variable to include the python 3.5.x path in it, or just cd into the correct python folder to run pip3 from there.
The folder in windows 10 should be something like this:
C:\Users\YOUR_USERNAME\AppData\Local\Programs\Python\Python35\Scripts
Open the terminal, cd to that path (change YOUR_USERNAME to the correct user) and then just run the following command:
pip3 install --upgrade tensorflow
and if you want the gpu version:
pip3 install --upgrade tensorflow-gpu
Pip3 is already installed when you install Python, so there is no need to do anything else.
Trying to install
pip install numpy
pip install scipy
pip install matplotlib
pip install scikit-learn
It failed with scipy, matplotlib and scikit-learn.
(from https://pypi.python.org/simple/scipy/) because it is not compatible with this Python
Skipping
My python version is 3.4 and pip version is 1.5.6
please help me install those above package
With pip 1.5.6 it will try to compile those projects from source which requires a lot of system dependencies (especially for scipy, you need gfortran and an optimized BLAS/LAPACK implementation).
I assume you are using the system provided version of pip under Linux. I would recommend to either use the latest version of pip (8.1 or later) in an a virtualenv (to avoid replacing the files of the system installed version of pip). Then you should be able to install manylinux wheels which do not require the compilation step.
Alternatively you can install miniconda and install those packages with the conda command line instead of pip.
Forget shitty pip, which is flawed beyond repair (static linking etc.)
Download IPython with the Anaconda Suite.... https://www.continuum.io/downloads
It brings most of the needed modules for scientific computing (as it is a crappy task if you have to download stuff to site-packages and run python setup.py install 3781 times..)
I wrote several programs using matplotlib, scipy, numpy etc with it..
Moreover it sports module package manager (comparable to Synaptic on Ubuntu..) if you are to lazy for the above mentioned task (and you are..).
Greets Dr. Cobra