I have just started using python and I have set up python3 to be installed in C:\Python37 dir. I have added python3 path to environment variables. When I run python3 -m pip install [package_name] it installs it in C:\Python37\Lib\site-packages. But when I try to install pylint with python3 -m pip install pylint it prints:
Requirement already satisfied: pylint in c:\users\radio\appdata\roaming\python\python37\site-packages (2.3.1)
It's installing it in above mentioned completely different location, and then VS Code complains how pylint is not installed. Why doesn't pip install it in C:\Python37\Lib\site-packages where it installs all the other packages?
It seems you might have multiple python installations on the computer or you are not running CMD with administrator priviledges when using pip.
I would refer you to this thread: windows pip installing libraries in wrong directory which seems to deal with a problem similar to the one you are having.
I would also try checking the environment variables to see the path set for Python if that doesn't work.
Related
I'm attempting to make a website with a few others for the first time, and have run into a weird error when trying to use Django/Python/VirtualEnv. I've found solutions to this problem for other operating systems, such as Ubuntu, but can't find any good solutions for Mac.
This is the relevant code being run:
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt
After running that block, I get the following errors:
AssertionError
Failed building wheel for django-toolbelt
Running setup.py bdist_wheel for psycopg2
...
AssertionError
Failed building wheel for psycopg2
Failed to build django-toolbelt psycopg2
I believe I've installed the "django-toolbelt" and "psycopg2", so I'm not sure why it would be failing.
The only difference I can think of is that I did not use the command
sudo apt-get install libpq-dev
as was instructed for Ubuntu usage as I believe that installing postgresql with brew took care of the header.
Thanks for any help or insight!
For MacOS users
After trying all the above methods (which did not work for me on MacOS 10.14), that one worked :
Install openssl with brew install openssl if you don't have it already.
add openssl path to LIBRARY_PATH :
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
install psycopg2 with pip pip3 install psycopg2
I had the same problem on Arch linux. I think that it's not an OS dependant problem. Anyway, I fixed this by finding the outdated packages and updating then.
pip uninstall psycopg2
pip list --outdated
pip install --upgrade wheel
pip install --upgrade setuptools
pip install psycopg2
I was also getting same error.
Using Python 3.7.3 and pip 19.1.1.
I used following command.
pip install psycopg2-binary==2.8.3
TDLR
If you aren't used to installing Python C-extensions, and psycopg2 isn't a core part of your work, try
pip install psycopg2-binary
Building Locally
psycopg2 is a C-extension, so it requires compilation when being installed by pip. The Build Prerequisites section of the docs explain what must be done to make installation via pip possible. In summary (for psycopg 2.8.5):
a C compiler must be installed on the machine
the Python header files must be installed
the libpq header files must be installed
the pg_config program must be installed (it usually comes with the libpq headers) and on $PATH.
With these prerequisites satisfied, pip install psycopg2 ought to succeed.
Installing pre-compiled wheels
Alternatively, pip can install pre-compiled binaries so that compilation (and the associated setup) is not required. They can be installed like this:
pip install psycopg2-binary
The docs note that
The psycopg2-binary package is meant for beginners to start playing with Python and PostgreSQL without the need to meet the build requirements.
but I would suggest that psycopg2-binary is often good enough for local development work if you are not using psycopg2 directly, but just as a dependency.
Concluding advice
Read the informative installation documentation, not only to overcome installation issues but also to understand the impact of using the pre-compiled binaries in some scenarios.
I had same problem and this appears to be a Mojave Issue, I was able to resolve with:
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
For Mac OS X users:
1. First check your postgresql path by running this command in terminal:
pg_config
If this fails lookup how to add pg_config to your path.
2. Next install Xcode Tools by running this command in terminal:
xcode-select --install
If you have both those sorted out now try to install psycopg2 again
For MacOS users, this question has the correct solution:
install command line tools if necessary:
xcode-select --install
then
env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install psycopg2
I was also facing the same after running all the above commands, but the following two commands worked for me:
Instead of pip, use this:
sudo apt-get install libpq-dev
then run this command:
pip install psycopg2
On OS X, I was able to solve this by simply upgrading wheel before installing psycopg2:
pip install --upgrade wheel
For OSX Sierra users, it seems that an xcode update is the solution: Can't install psycopg2 package through pip install... Is this because of Sierra?
I tried all the above solutions but they did not work for me. What I did was change the psycopg2 version in my requirements.txt file from psycopg2==2.7.4 to psycopg2==2.7.6
Is your error message complete? the most encountered reason for failing to install psycopg2 on mac from pip is pg_config is not in path.
by the way, using macports or fink to install psycopg2 is more recommended way, so you don't have to worry about pg_config, libpq-dev and python-dev.
plus, are using Python 3.5? then upgrage your wheel to > 0.25.0 using pip.
I faced the same issue, but the answers above didn't work for me.
So this is what I did in my requirements.txt
psycopg2-binary==2.7.6.1 and it worked fine
I had this issue on several packages, including psycopg2, numpy, and pandas. I simply removed the version from the requirements.txt file, and it worked.
So instead of psycopg2-binary==2.7.6.1 I just had psycopg2-binary.
I know you are asking for development environment but if you are deploying on server say, Heroku. Just add below line in the requirements.txt of your project.
django-heroku==0.3.1
As this package itself will install the required packages like psycopg2 on server deployment.So let the server(heroku) should take care of it.
sudo apt install libpq-dev python3.X-dev
where X is the sub version,
these should be followed by :
pip install --upgrade wheel
pip install --upgrade setuptools
pip install psycopg2
Enjoy !!!
I solved my problem by updating/installing vs_BuildTools. The link to the software was given in the error itself.
Error Image
Fixed by installing python3.7-dev: sudo apt install python3.7-dev, based on the link.
Python: 3.7
Ubuntu: 20.04.3 LTS
I have a 2-part question about conda vs. pip virtual environments. I found great information on the answers What is the difference between pip and conda? and Does Conda replace the need for virtualenv? but still have something unclear.
I have a given python project (say PR) that I need to install and further develop on a linux server (say S) where python is installed with anaconda. Now, the usage/installation instructions of PR tell me to use python to create virtual environment and pip to install all packages. That is,
python3 -m venv PR
pip install --editable . (the dot included at the end)
According to "pip install --editable ./" vs "python setup.py develop" the latter reads the file setup.py (included in PR) which contains a function setup(...) with option install_requires listing all the required packages and installs them automatically. I have tested this on my own computer (which does not have conda) and it works fine. At least no error messages.
Now I need to further develop PR on S. My question Part 1: can I use conda instead of pip to create and update virtual environment? If yes, what would be the conda command replacing pip install --editable . ? I'm positive I will later need to install other packages as well. I'm worried about conflicts between conda/pip.
On S, I have Spyder and no other python IDEs. I have never used Spyder but I'm very familiar with PyCharm (Windows) and VS Code (Linux) so I assume debugging with Spyder will be similar to those. My question Part 2 (tied to Part 1): if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
(Edit/update): Thank you Carlos for comments. I continue my question:
I created and activated the virtual environment (VE) with conda
conda create PR_venv
conda activate PR_venv
Installed pip with
conda install pip
(this upgraded pip and installed several other packages too, including newer version of python). Installed PR and its required packages with pip
pip install -e .
Now, if I run the PR package inside this active VE interactively from the terminal, everything works fine. I would like to do the same from within spyder, to get the IDE debugging abilities in my hand.
When I start spyder, open a python file to be run, click "Run" button, it crashes in the import statements.
Spyder cannot see the installed packages. It can see only the local package PR but none of the packages installed by pip for this VE.
I am not sure what is the correct question here; I'm confused how are conda VEs related to spyder/jupyter/ipython ? I cannot find information in the conda documents about this.
I cannot find from spyder documents anything about VEs. Do I have to somehow re-install the packages (how?) inside Spyder? It seems pointless because the packages are installed already.
(Edit/Update 2): The information on https://docs.spyder-ide.org/current/installation.html makes me even more confused: Spyder is presented as both a stand-alone program and as a python package. So do I have to re-install Spyder inside the VE(?!) with
conda activate PR_venv
conda install spyder
Any clarification would be appreciated. I have always thought that the IDEs are stand-alone programs and that's it. This Spyder setup twists my brains into pretzel.
(Spyder maintainer here) About your questions:
can I use conda instead of pip to create and update virtual environment?
Yes, you can. Please see here to learn about the functionality offered by conda for managing environments.
If yes, what would be the conda command replacing pip install --editable . ?
Conda doesn't offer a good replacement for that command. However, you can still use it in a conda environment, as long as all you've installed all your package dependencies with conda before running it. That would avoid mixing conda and pip packages, which usually leads to really bad results.
if I have to use pip to install packages, does Spyder see those? Or can it only see conda-installed packages?
Spyder can work with pip and conda packages without problems. Just make sure of not mixing them (as I said above) and you'll be fine. In addition, please read our documentation to learn how to connect a local Spyder instance to a remote server.
Part 1: yes I can use conda to create VE and pip to install packages
conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list
The last line shows which packages are installed by conda and which by pip (shown as pypi)
Part 2: spyder by default cannot see the packages. Need to do two things:
conda install spyder-kernels
Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]
Restart Spyder. Now it can see the packages.
(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder
I want to install google-cloud-pubsub via pip installation on Mac OS but I get an error: distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1. The command I run: pip install google-cloud-pubsub==2.1.0.
Here the complete error message.
Any suggestion? Thank you!
More info:
$ python -V
Python 3.9.0
$ pip -V
pip 20.2.4 from /.../lib/python3.9/site-packages/pip (python 3.9)
$ sw_vers
ProductName: macOS
ProductVersion: 11.0.1
BuildVersion: 20B29
I've seen other similar cases but them don't solve my issue. I tried:
Pip install error in Mac OS(error: command '/usr/bin/clang' failed with exit status 1)
Mac OS Mojave installation error - error: command 'clang' failed with exit status 1
Try to add these env var before
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install google-cloud-pubsub==2.1.0
If it does not work you can try with virtualenv:
pip install virtualenv
virtualenv my-test-env
source my-test-env/bin/activate
my-test-env/bin/pip install google-cloud-pubsub==2.1.0
Because is written on github:
Install this library in a virtualenv using pip. virtualenv is a tool to create isolated Python environments. The basic problem it addresses is one of dependencies and versions, and indirectly permissions.
With virtualenv, it's possible to install this library without needing system install permissions, and without clashing with the installed system dependencies.
I was using pyenv and facing the similar kind of issue. Then I did the following and it worked.
First, upgrade pip
pip3 install --upgrade pip
Then, update the setup tools:
python3 -m pip install --upgrade setuptools
I just had this same problem but I'm using homebrew to manage my Mac packages.
My error result was the same as yours but within the error message it was:
plyvel/_plyvel.cpp:632:10: fatal error: 'leveldb/db.h' file not found
#include "leveldb/db.h"
This can be fixed using homebrew by just installing leveldb:
brew install leveldb
This allowed the plyvel dependency to be satisified. I then manually installed plyvel just to be safe:
pip3 install plyvel
And lastly because I'm trying to install the airflow libraries that include pubsub:
pip3 install apache-airflow-providers-google
I ended up just using Python 3.8.13 instead, managed with pyenv and pyenv-virtualenv.
If you don't have pyenv:
brew install pyenv pyenv-virtualenv
add
export PATH="$HOME/.pyenv/bin:$PATH"
export PATH="$HOME/.pyenv/shims:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
to your ~/.zshrc if it's not there, and source ~/.zshrc
pyenv install 3.8.13
pyenv global 3.8.13
pyenv virtualenv 3.8.13 venv
pyenv activate venv
pip install grcpio
Are you attempting to compile on a new M1 processor? I ran into trouble installing the latest grpcio (1.34.0) too, with the same error message as you.
Without messing with compile flags and libraries, I found the best solution for me (setting up a new Flask environment to talk to Google Cloud on a new Big Sur Mac) was the tip offered here:
https://osxdaily.com/2020/11/18/how-run-homebrew-x86-terminal-apple-silicon-mac/
Duplicate your Terminal rename it to "Rosetta Terminal" or similar
Get Info on the new app, and check the box for "Open using Rosetta"
Do your work with this new terminal app instead
grpcio installed fine for me this way, and I don't anticipate any problems running my Web apps in this translated environment (until I presume this issue gets fixed in the source libraries).
I encountered a similarly-named clang issue when attempting to awsiotsdk on an M1 Mac with Python 3.8
Incase it's relevant to anyone reading this, the underlying issue was a dependency on awscrt. This particular library (on PyPi) did not contain Built Distributions for the combination of M1 ("xxx_universal2.whl") and Python 3.8.
The solution was to use pyenv with pipenv (a favourite pattern of mine) to force this project to use a supported combination (in our case: Python 3.9 with M1 )
if you are using Mac M1 chip, just do the below thing it worked for me.
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
Thanks for the below answer.
How can I install GRPCIO on an Apple M1 Silicon laptop?
Working solution in my case (Mac M1 Monterey OS):
brew install openssl re2
LDFLAGS="-L$(/opt/homebrew/bin/brew --prefix openssl)/lib -L$(/opt/homebrew/bin/brew --prefix re2)/lib" CPPFLAGS="-I$(/opt/homebrew/bin/brew --prefix openssl)/include -I$(/opt/homebrew/bin/brew --prefix re2)/include" GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_RE2=true GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio
Credits to https://github.com/grpc/grpc/issues/24677#issuecomment-862413344
I started having that issue. Following this comment,
with pyenv, I uninstalled my python 3.9.12 and reinstalled it again.
I am having an M1 and macOS 12.6, btw.
Deactivate your current virtualenv first, then:
pyenv uninstall 3.9.12
pyenv install 3.9.12
pyenv local 3.9.12 # to set your local python version to the newly installed python
poetry shell
pip install grpcio # or poetry install grpcio
I am a little bit confused....
I installed anaconda on my computer (I have windows 10).
Normally, when I want to install a package I simply do "pip install package_name" or "conda install package_name" and it is done.
First question: what is the difference between pip and conda?
Now I tried to install xgboost and it was really complicated I tried lot of things nothings worked until I install something called miniconda.
There it works but now, when I do "conda install package_name" it install it in miniconda3/lib/site _package and I have to copy/paste it in Anaconda3/lib/site_package if I want it to work.
Second question: how can I ask to the computer that "conda install
package_name" install it directly in anaconda3 and not miniconda3?
Finally I tried to install the package "surprise" for recommended systems. Both "pip install" or "conda install" failed.
I went in github and got the file "surprise" from https://github.com/NicolasHug/Surprise
I tried to copy it in Anaconda3/lib/site_package but it doesn't work.
When I do from surprise import Reader I did not get the error "no module name surprise" anymore but I get "cannot import name 'Reader'"
Last question: how can I make it work? I think I have to build it but
I do not now how...
Thank you in advance for anyone that can explain all this for me :-)
Similarly to you, I had issues installing the surprise package.
I tried both pip install surprise and conda install surprise unsuccessfully.
conda install -c conda-forge scikit-surprise
conda install -c conda-forge/label/gcc7 scikit-surprise
conda install -c conda-forge/label/cf201901 scikit-surprise
I found those on the anconda website and the first one worked for me.
Hopefully this would help you as well
pip vs conda
pip is a package manager that facilitates installation, upgrade,
and uninstallation of python packages. It also works with virtual python environments.
conda is a package manager for any software (installation, upgrade and uninstallation).
It also works with virtual system environments.
Conda is a packaging tool and installr that aims to do more than what pip does;
handle library dependencies outside of the Python packages as well as the Python packages themselves.
Conda also creates a virtual environment, like virtualenv does.
For more see here
Anaconda vs miniconda
The open source version of Anaconda is an easy-to-install
high performance Python and R distribution with a package manager,
environment manager and collection of 720+ open source packages.
It also comes with the options to install RStudio.
The "lite" version of Anaconda without the collection of 720 packages.
The downside is that you need to type in command line commands,
"conda install PACKAGENAME"
And Last
To install this package with conda run:
conda install -c anaconda py-xgboost=0.60
Update for surprise
The easiest way is to use pip (you'll need numpy):
$ pip install numpy
$ pip install scikit-surprise
Or you can clone the repo and build the source (you'll need Cython and numpy):
$ git clone https://github.com/NicolasHug/surprise.git
$ python setup.py install
So I've got a virtualenv I've created using pyvenv-3.3, which I thought set up pip to install things into the virtualenv's path. However, I get the following outputs after I've activated by virtualenv:
$ pip --version
pip 1.4.1 from /usr/local/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg (python 2.7)
pip-3.3 --version
pip 1.4.1 from /usr/local/lib/python3.3/site-packages/pip-1.4.1-py3.3.egg (python 3.3)
This is all fine and good, but then my sys.path is this:
['',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python33.zip',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/plat-darwin',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload',
'/Users/alexgolec/Documents/gutenberg/virtualenv/lib/python3.3/site-packages']
Most notably, the whole site-packages directory is outright missing. Furthermore, this output indicates to me that I'm probably using a python that installed through homebrew at some point. Most infuriatingly, none of the directories in my sys.path are compatible with pip.
Any thoughts on remedies? I won't manually edit my sys.path, but maybe there's some configuration with pip I can do?
Install virtualenvwrapper (makes using virtualenv so much easier):
pip install virtualenvwrapper
.. and then try this:
mkvirtualenv <your_env> -p /usr/local/bin/python3
Where homebrew sticks python is different from where your system has it. You can run the following to see what I'm talking about:
which python
which python3