Python - managing package installations between anaconda and base system [duplicate] - python-3.x

Is there any way to make pip play well with multiple versions of Python? For example, I want to use pip to explicitly install things to either my site 2.5 installation or my site 2.6 installation.
For example, with easy_install, I use easy_install-2.{5,6}.
And, yes — I know about virtualenv, and no — it's not a solution to this particular problem.

The current recommendation is to use python -m pip, where python is the version of Python you would like to use. This is the recommendation because it works across all versions of Python, and in all forms of virtualenv. For example:
# The system default python:
$ python -m pip install fish
# A virtualenv's python:
$ .env/bin/python -m pip install fish
# A specific version of python:
$ python-3.6 -m pip install fish
Previous answer, left for posterity:
Since version 0.8, Pip supports pip-{version}. You can use it the same as easy_install-{version}:
$ pip-2.5 install myfoopackage
$ pip-2.6 install otherpackage
$ pip-2.7 install mybarpackage
EDIT: pip changed its schema to use pipVERSION instead of pip-VERSION in version 1.5. You should use the following if you have pip >= 1.5:
$ pip2.6 install otherpackage
$ pip2.7 install mybarpackage
Check https://github.com/pypa/pip/pull/1053 for more details
References:
https://github.com/pypa/pip/issues/200
http://www.pip-installer.org/docs/pip/en/0.8.3/news.html#id4
https://pip.pypa.io/en/stable/news/#v0-8 or
https://web.archive.org/web/20140310013920/http://www.pip-installer.org:80/docs/pip/en/0.8.3/news.html#id4

On Windows, you can execute the pip module using a given Python version through the Python launcher, py.exe, if you chose to install it during Python 3 setup.
py -3 -m pip install packagename
py -2 -m pip install packagename
You can be even more specific and request an exact sub-version of Python:
py -3.6 -m pip install packagename
To get a list of all installed Python versions available through the launcher, run:
py --list
Alternatively, you can launch the desired Python executable directly:
C:/path/to/specific/python.exe -m pip install packagename

/path/to/python2.{5,6} /path/to/pip install PackageName doesn't work?
For this to work on any python version that doesn't have pip already installed you need to download pip and do python*version* setup.py install. For example python3.3 setup.py install. This resolves the import error in the comments. (As suggested by #hbdgaf)

I had python 2.6 installed by default (Amazon EC2 AMI), but needed python2.7 plus some external packages for my application. Assuming you already installed python2.7 alongside with default python (2.6 in my case). Here is how to install pip and packages for non-default python2.7
Install pip for your python version:
curl -O https://bootstrap.pypa.io/get-pip.py
python27 get-pip.py
Use specific pip version to install packages:
pip2.7 install mysql-connector-python --allow-external mysql-connector-python

It worked for me in windows this way:
I changed the name of python files python.py and pythonw.exe to python3.py pythonw3.py
Then I just ran this command in the prompt:
python3 -m pip install package

Other answers show how to use pip with both 2.X and 3.X Python, but does not show how to handle the case of multiple Python distributions (eg. original Python and Anaconda Python).
I have a total of 3 Python versions: original Python 2.7 and Python 3.5 and Anaconda Python 3.5.
Here is how I install a package into:
Original Python 3.5:
/usr/bin/python3 -m pip install python-daemon
Original Python 2.7:
/usr/bin/python -m pip install python-daemon
Anaconda Python 3.5:
python3 -m pip install python-daemon
or
pip3 install python-daemon
Simpler, as Anaconda overrides original Python binaries in user environment.
Of course, installing in anaconda should be done with conda command, this is just an example.
Also, make sure that pip is installed for that specific python.You might need to manually install pip. This works in Ubuntu 16.04:
sudo apt-get install python-pip
or
sudo apt-get install python3-pip

From here: https://docs.python.org/3/installing/
Here is how to install packages for various versions that are installed at the same time linux, mac, posix:
python2 -m pip install SomePackage # default Python 2
python2.7 -m pip install SomePackage # specifically Python 2.7
python3 -m pip install SomePackage # default Python 3
python3.4 -m pip install SomePackage # specifically Python 3.4
python3.5 -m pip install SomePackage # specifically Python 3.5
python3.6 -m pip install SomePackage # specifically Python 3.6
On Windows, use the py Python launcher in combination with the -m switch:
py -2 -m pip install SomePackage # default Python 2
py -2.7 -m pip install SomePackage # specifically Python 2.7
py -3 -m pip install SomePackage # default Python 3
py -3.4 -m pip install SomePackage # specifically Python 3.4

I ran into this issue myself recently and found that I wasn't getting the right pip for Python 3, on my Linux system that also has Python 2.
First you must ensure that you have installed pip for your python version:
For Python 2:
sudo apt-get install python-pip
For Python 3:
sudo apt-get install python3-pip
Then to install packages for one version of Python or the other, simply use the following for Python 2:
pip install <package>
or for Python 3:
pip3 install <package>

pip is also a python package. So the easiest way to install modules to a specific python version would be below
python2.7 /usr/bin/pip install foo
or
python2.7 -m pip install foo

So apparently there are multiple versions of easy_install and pip. It seems to be a big mess. Anyway, this is what I did to install Django for Python 2.7 on Ubuntu 12.10:
$ sudo easy_install-2.7 pip
Searching for pip
Best match: pip 1.1
Adding pip 1.1 to easy-install.pth file
Installing pip-2.7 script to /usr/local/bin
Using /usr/lib/python2.7/dist-packages
Processing dependencies for pip
Finished processing dependencies for pip
$ sudo pip-2.7 install django
Downloading/unpacking django
Downloading Django-1.5.1.tar.gz (8.0Mb): 8.0Mb downloaded
Running setup.py egg_info for package django
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
Installing collected packages: django
Running setup.py install for django
changing mode of build/scripts-2.7/django-admin.py from 644 to 755
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
changing mode of /usr/local/bin/django-admin.py to 755
Successfully installed django
Cleaning up...
$ python
Python 2.7.3 (default, Sep 26 2012, 21:51:14)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>>

On Linux, Mac OS X and other POSIX systems, use the versioned Python commands in combination with the -m switch to run the appropriate copy of pip:
python2.7 -m pip install SomePackage
python3.4 -m pip install SomePackage
(appropriately versioned pip commands may also be available)
On Windows, use the py Python launcher in combination with the -m switch:
py -2.7 -m pip install SomePackage # specifically Python 2.7
py -3.4 -m pip install SomePackage # specifically Python 3.4
if you get an error for py -3.4 then try:
pip install SomePackage

Installation of multiple versions of Python and respective Packages.
Python version on the same windows machine : 2.7 , 3.4 and 3.6
Installation of all 3 versions of Python :
Installed the Python 2.7 , 3.4 and 3.6 with the below paths
PATH for all 3 versions of Python :
Made sure the PATH variable ( in System Variables ) has below paths included - C:\Python27\;C:\Python27\Scripts;C:\Python34\;C:\Python34\Scripts;C:\Python36\;C:\Python36\Scripts\;
Renaming the executables for versions :
Changed the python executable name in C:\Python36 and C:\Python34 to python36 and python34 respectively.
Checked for the command prompt with all versions :
Installing the packages separately for each version

If you have multiple versions as well as multiple architectures (32 bit, 64 bit) you will need to add a -32 or -64 at the end of your version.
For windows, go to cmd and type py --list and it will produce the versions you have installed. The list will look like the following:
Installed Pythons found by py Launcher for Windows
-3.7-64 *
-3.7-32
-3.6-32
The full command as an example will be:
py -3.6-32 -m pip install (package)
If you want to get more indepth, to install a specific version of a package on a specific version of python, use ==(version) after the package. As an example,
py -3.6-32 -m pip install opencv-python==4.1.0.25

Here is my take on the problem. Works for Python3. The main features are:
Each Python version is compiled from source
All versions are installed locally
Does not mangle your system's default Python installation in any way
Each Python version is isolated with virtualenv
Prerequisites: If you are using some bare-bones thin client with no extra turf installed, you should run this first (in ubuntu 18.04 at least, extra packages added for convenience):
sudo apt-get update
sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install -y build-essential cmake
sudo apt-get install -y zlib1g zlib1g-dev libsqlite3-dev \
openssl libssl-dev libffi-dev unzip pciutils net-tools \
libblas-dev gfortran libblas3
The steps are as follows:
If you have several extra python versions installed in some other way, get rid of them, e.g., remove $HOME/.local/lib/python3.x, etc. (also the globally installed ones). Don't touch your system's default python3 version though.
Download source for different python versions under the following directory structure:
$HOME/
python_versions/ : download Python-*.tgz packages here and "tar xvf" them. You'll get directories like this:
Python-3.4.8/
Python-3.6.5/
Python-3.x.y/
...
At each "Python-3.x.y/" directory, do the following (do NOT use "sudo" in any of the steps!):
mkdir root
./configure --prefix=$PWD/root
make -j 2
make install
virtualenv --no-site-packages -p root/bin/python3.x env
At "python_versions/" create files like this:
env_python3x.bash:
#!/bin/bash
echo "type deactivate to exit"
source $HOME/python_versions/Python-3.x.y/env/bin/activate
Now, anytime you wish to opt for python3.x, do
source $HOME/python_versions/env_python3x.bash
to enter the virtualenv
While in the virtualenv, install your favorite python packages with
pip install --upgrade package_name
To exit the virtualenv and python version just type "deactivate"
UPDATE
It seems that --no-site-packages is deprecated. There's an easy fix for this: Once you have activated the virtualenv, just point the HOME env variable to somewhere else than your actual home directory, i.e.:
export HOME=some/where/else
A nice way to do this in general is:
Create virtualenv
Activate virtualenv
If you want to "recycle" existing libraries to your virtualenv, softlink them from your existing install, i.e.
ln -s $HOME/.local/lib/python3.6/site-packages/numpy $PWD/venv/lib/python3.6/site-packages/
Do export PYTHONPATH=, export HOME=/some/other/dir
Now you should have custom-isolated virtualenv.
UPDATE 2 / SUDO
Wan't to force sudo to use your virtualenv?
Defaults secure_path="/home/USENAME/Python-3.x.y/env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"
Defaults env_keep += "VIRTUAL_ENV"
Defaults env_keep += "PYTHONPATH"
Now try "sudo python3 --version" and magic should happen
UPDATE 3 / DOCKER
Enable virtualenv inside your docker (of course, you have built it in your docker image):
ENV VIRTUAL_ENV=/home/USER/Python-3.x.y/env
ENV PYTHONPATH=
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

You can use one of the following commands:
pip2 install SomePackage
pip3 install SomePackage
python2 -m pip install SomePackage
python3 -m pip install SomePackage
And of course, make sure that you have the correct version of pip installed
sudo apt-get install python-pip
sudo apt-get install python3-pip
I haven't used these commands myself but, some answers above suggests using them to specify exactly the version of python you want to use
pip-2.7 install SomePackage
python-3.6 -m pip install SomePackage

For python 3 and Windows OS, I always use this syntax to install packages on different version:
First I always use Git Bash Command Prompt.
Here an example installing urllib package.
Default Python version:(The normal pip command)
pip install urllib3
For the other versions
py -3.8 -m pip install urllib3
py => for python
-3.8 => for the version (I'm using the 3.8.7 version) but if you're using the 3.7.7 version it will be "-3.7"
-m : just because or for modify
pip install urllib3 : the normal pip command

Most of the answers here address the issue but I want to add something what was continually confusing me with regard to creating an alternate installation of python in the /usr/local on CentOS 7. When I installed there, it appeared like pip was working since I could use pip2.7 install and it would install modules. However, what I couldn't figure out was why my newly installed version of python wasn't seeing what I was installing.
It turns out in CentOS 7 that there is already a python2.7 and a pip2.7 in the /usr/bin folder. To install pip for your new python distribution, you need to specifically tell sudo to go to /usr/local/bin
sudo /usr/local/bin/python2.7 -m ensurepip
This should get pip2.7 installed in your /usr/local/bin folder along with your version of python. The trick is that when you want to install modules, you either need to modify the sudo $PATH variable to include /usr/local/bin or you need to execute
sudo /usr/local/bin/pip2.7 install <module>
if you want to install a new module. It took me forever to remember that sudo wasn't immediately seeing /usr/local/bin.

Simple and recent
On windows
1- Supposed that you have a different version of Python installed in your system. To check use the following command to check:
> py --list
-3.10-64 *
-3.7-64
2- Set your preferred default version:
by setting the PY_PYTHON environment variable (e.g. PY_PYTHON=3.7).
by settting the py.ini file usually located on C:\Users\<your user name>\AppData\Local if not create one. For example, setting PY_PYTHON=3 and PY_PYTHON3=3.7 environment variables are equivalent to the INI file containing:
[defaults]
python=3
python3=3.7
3- check by typing again py --list:
> py --list
-3.10-64
-3.7-64 *
4- If you would like to run virtual environment with spesific version of python and pip see this post.
On Linux (Ubuntu)
An easy approach for managing different python versions on Linux is update-alternatives command. This command giving us the ability to switch between many versions of the same software easily.
command format:update-alternatives --install link name path priority, the name is the generic name for the master link, the link is the name of its symlink, the path is the alternative being introduced for the master link, and priority is the priority of the alternatives group.
Usage: Suppose you installed two versions of python (python3.10 , python3.7). Now by running this command you will link the command name (python3) to different versions of python and assign a priority number. A higher priority number means a higher priority.
$ update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 1
$ update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 2
List installed versions of python with this command:
$ update-alternatives --list python3
/usr/bin/python3.7
/usr/bin/python3.10
Switching between versions: Just manually select the priority number of the desired python version after running following command.
$ update-alternatives --config python3
There are 2 choices for the alternative python3 (providing /usr/bin/python3).
Selection Path Priority Status
------------------------------------------------------------
0 /usr/bin/python3.10 2 auto mode
* 1 /usr/bin/python3.7 1 manual mode
2 /usr/bin/python3.10 2 manual mode
Press <enter> to keep the current choice[*], or type selection number: 2

Context: Archlinux
Action:
Install python2-pip:
sudo pacman -S python2-pip
You now have pip2.7:
sudo pip2.7 install boto
Test (in my case I needed 'boto'):
Run the following commands:
python2
import boto
Success: No error.
Exit: Ctrl+D

for example, if you set other versions (e.g. 3.5) as default and want to install pip for python 2.7:
download pip at https://pypi.python.org/pypi/pip (tar)
unzip tar file
cd to the file’s directory
sudo python2.7 setup.py install

If you have both python3.6 and python3.7 installed and want to use pip with python3.7 by default, here's what you should do:
First make sure you have pip installed for python3.7
python3.7 -m pip install -U pip
Now pip3.7 must be available, so we edit .bashrc
nano ~/.bashrc
adding the following line to it
alias pip=pip3.7
In order for the changes to take effect type in the shell:
source ~/.bashrc
Now if you type:
pip --version
you should get:
pip 20.1.1 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)
which means, if you use, for example:
pip install <package>
it would install the <package> for python3.7

Another possible way could be using conda and pip. Some time you probably want to use just one of those, but if you really need to set up a particular version of python I combine both.
I create a starting conda enviroment with the python I want. As in here https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html. Alternatively you could set up the whole enviroment just using conda.
conda create -n myenv python=3.6.4
Then activate your enviroment with the python you like. This command could change depending on the OS.
source activae myenv
Now you have your python active then you could continue using conda but if you need/want to use pip:
python -m pip -r requirements.txt
Here you have a possible way.

You can go to for example C:\Python2.7\Scripts and then run cmd from that path. After that you can run pip2.7 install yourpackage...
That will install package for that version of Python.

This is probably the completely wrong thing to do (I'm a python noob), but I just went in and edited the pip file
#!/usr/bin/env python3 <-- I changed this line.
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())

To use multiple versions of pip, just type
pip{version} and run command
Example: for python 3.10
pip3.10
pip3.10 list
pip3.10
for Python 3.7
pip3.7
pip3.7 list

For windows specifically:
\path\to\python.exe -m pip install PackageName works.

for Blender:
/usr/bin $ python3.7 -m pip install irc

Some useful information for debugging this is the pip debug command. It shows the location of the python interpreter that it is attached to in the 1st line (after the warning).
$ pip debug
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 21.2.4 from /data/akshay/anaconda3/lib/python3.9/site-packages/pip (python 3.9)
sys.version: 3.9.12 (main, Apr 5 2022, 06:56:58)
sys.executable: /data/akshay/anaconda3/bin/python

Related

Status of Python 3.7 in Cygwin

Does anyone have a way to install python 3.7 in latest stable Cygwin 32 or 64 bit that works out of the box without hacking? I've noticed that 3.6 works fine but 3.7 libraries don't behave and are missing key functionality.
I have usecases to have Cygwin for various scripts but want to use 3.7 for its improvements with type annotations. The new Linux runtime is not available on my servers so Cygwin is the only decent posix environment I can run in my windows servers.
[Update - 2022-03-30] I recently have successfully gotten Python 3.9 working on Cygwin. It does create proper python3 and pip3 executables out of the box. Only issues need to install cryptography==3.3.2 and pyopenssl<=21.0.0 due to rust dependency.
Although Timothy's answer is correct, the cleaner way to do this is to ensure Cygwins 'alternative' package is installed and run the following commands in order to have your Python versions switchable through the alternative system. This will also switch to the correct pip versions.
/usr/sbin/update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 0 --slave /usr/bin/pip3 pip3 /usr/bin/pip3.6
/usr/sbin/update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 10 --slave /usr/bin/pip3 pip3 /usr/bin/pip3.7
/usr/sbin/update-alternatives --install /usr/bin/python python /usr/bin/python3 10 --slave /usr/bin/pip pip /usr/bin/pip3
/usr/sbin/update-alternatives --set python3 /usr/bin/python3.7
/usr/sbin/update-alternatives --set python /usr/bin/python3
In case you still need deprecated Python 2.x versions you can add these accordingly as an alternative for 'python' via a 'python2' group.
The python37 packages will install correctly but will create an executable called python3.7 instead of python3 which can be confusing. I would guess that the Cygwin dev's wanted to allow two versions of python3 to be installed at the same time.
However, as its not recommended to have two versions of python3 installed at the same time, you can simply create the executable python3 as follows to correct this:
ln -s /usr/bin/python3.7 /usr/bin/python3
The same goes for python37-pip:
ln -s /usr/bin/pip3.7 /usr/bin/pip3
These instructions above will likely apply to other non-standard version of python on Cygwin although I have only tested Python37.
For those who need to install pyopenssl package, you will need the following cygwin packages: gcc-g++, libffi-devel, libssl-devel, python37-devel
For pandas, you will need python37-numpy package and be sure to upgrade it in pip before installing pandas as there is a known bug in numpy package version in cygwin:
pip3 install --upgrade numpy
pip3 install pandas
python3 defaults to python 3.6 but python 3.7 is available in packages. Once installed you can run using:
$ python3.7 -V
Python 3.7.3

Can’t run pip3 due to import error after upgrade [duplicate]

Whenever I am trying to install any package using pip, I am getting this import error:
guru#guru-notebook:~$ pip3 install numpy
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in <module>
from pip import main
ImportError: cannot import name 'main'
guru#guru-notebook:~$ cat `which pip3`
#!/usr/bin/python3
# GENERATED BY DEBIAN
import sys
# Run the main entry point, similarly to how setuptools does it, but because
# we didn't install the actual entry point from setup.py, don't use the
# pkg_resources API.
from pip import main
if __name__ == '__main__':
sys.exit(main())
It was working fine earlier, I am not sure why it is throwing this error.
I have searched about this error, but can't find anything to fix it.
Please let me know if you need any further detail, I will update my question.
You must have inadvertently upgraded your system pip (probably through something like sudo pip install pip --upgrade)
pip 10.x adjusts where its internals are situated. The pip3 command you're seeing is one provided by your package maintainer (presumably debian based here?) and is not a file managed by pip.
You can read more about this on pip's issue tracker
You'll probably want to not upgrade your system pip and instead use a virtualenv.
To recover the pip3 binary you'll need to sudo python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall.
If you want to continue in "unsupported territory" (upgrading a system package outside of the system package manager), you can probably get away with python3 -m pip ... instead of pip3.
We can clear the error by modifying the pip file.
Check the location of the file:
$ which pip
path -> /usr/bin/pip
Go to that location(/usr/bin/pip) and open terminal
Enter: $ sudo nano pip
You can see:
import sys
from pip import main
if __name__ == '__main__':
sys.exit(main())
Change to:
import sys
from pip import __main__
if __name__ == '__main__':
sys.exit(__main__._main())
then ctrl + o write the changes and exit
Hope this will do!!
For Ubuntu family, Debian, Linux Mint users
Thanks to Anthony's explanation above, you can retain your original system pip (in /usr/bin/ and dist-packages/) and remove the manually-installed pip (in ~/.local/) to resolve the conflict:
$ python3 -m pip uninstall pip
Ubuntu/Debian pip v8.1.1 (16.04) from python3-pip debian package (see$ pip3 -V) shows the same search results as the latest pip v10.0.1, and installs latest modules from PyPI just fine. It has a working pip command (already in the $PATH), plus the nice --user option patched-in by default since 2016. Looking at pip release notes, the newer versions are mostly about use-case specific bug fixes and certain new features, so not everyone has to rush upgrading pip just yet. And the new pip 10 can be deployed to Python virtualenvs, anyway.
But regardless of pips, your OS allows to quickly install common Python modules (including numpy) with APT, without the need for pip, for example:
$ sudo apt install python3-numpy python3-scipy (with system dependencies)
$ sudo apt install python3-pip (Debian-patched pip, slightly older but it doesn't matter)
Quick apt syntax reminder (please see man apt for details):
$ sudo apt update (to resync Ubuntu package index files from up-to-date sources)
$ apt search <python-package-name> (full text-search on all available packages)
$ apt show <python-package-name> (displays the detailed package description)
$ sudo apt install <python-package-name>
Package names prefixed with python- are for Python 2; and prefixed with python3- are for Python 3 (e.g. python3-pandas). There are thousands, and they undergo integration testing within Debian and Ubuntu. Unless you seek to install at per-user level (pip install --user option) or within virtualenv/venv, apt could be what you needed. These system packages are accessible from virtual envs too, as virtualenv will gracefully fall back to using system libs on import if your envs don't have given copies of modules.
Your custom-installed (with pip --user) per-user modules in ~/.local/lib will override them too.
Note, since this is a system-wide installation, you'd rarely need to remove them (need to be mindful about OS dependencies). This is convenient for packages with many system dependencies (such as with scipy or matplotlib), as APT will keep track and provide all required system libs and C extensions, while with pip you have no such guarantees.
In fact, for system-wide Python packages (in contrast to per-user, home dir level, or lower), Ubuntu expects using the APT package manager (rather than sudo pip) to avoid breaking OS: sudo pip3 targets the very same /usr/lib/python3/dist-packages directory where APT stores OS-sensitive modules. Recent Debian/Ubuntu releases depend heavily on Python 3, so its pre-installed modules are managed by apt and shouldn't be changed.
So if you use pip3 install command, please ensure that it runs in an isolated virtual dev environment, such as with virtualenv (sudo apt install python3-virtualenv), or with Python3 built-in (-m venv), or at a per-user level (--user pip option, default in Ubuntu-provided pip since 2016), but not system-wide (never sudo pip3!), because pip interferes with the operation of the APT package manager and may affect Ubuntu OS components when a system-used python module is unexpectedly changed. Good luck!
P. S. All the above is for the 'ideal' solution (Debian/Ubuntu way).
If you still want to use the new pip3 v10 exclusively, there are 3 quick workarounds:
simply open a new bash session (a new terminal tab, or type bash) - and pip3 v10 becomes available (see pip3 -V). debian's pip3 v8 remains installed but is broken; or
the command $ hash -d pip3 && pip3 -V to refresh pip3 pathname in the $PATH. debian's pip3 v8 remains installed but is broken; or
the command $ sudo apt remove python3-pip && hash -d pip3 to uninstall debian's pip3 v8 completely, in favor of your new pip3 v10.
Note: You will always need to add --user flag to any non-debian-provided pip, unless you are in a virtualenv! (it deploys python packages to ~/.local/, default in debian/ubuntu-provided python3-pip and python-pip since 2016). Your use of pip 10 system-wide, outside of virtualenv, is not really supported by Ubuntu/Debian. Never sudo pip3!
Further details:
https://github.com/pypa/pip/issues/5221#issuecomment-382069604
https://github.com/pypa/pip/issues/5240#issuecomment-381673100
resolved in one step only.
I too faced this issue, But this can be resolved simply by 1 command without bothering around and wasting time and i have tried it on multiple systems it's the cleanest solution for this issue. And that's:
For python3:- sudo python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall.
By this , you can simply install packages using pip3. to check use pip3 --version.
For older versions, use : sudo python -m pip uninstall pip && sudo apt install python-pip --reinstall.
By this, now you can simply install packages using pip. to check use pip --version.
Use python -m pip install instead of pip install
Example:
python -m pip install --user somepackage
python3 -m pip install --user somepackage
The pip (resp. pip3) executable is provided by your distro (python-pip package on Ubuntu 16.04) and located at /usr/bin/pip.
Therefore, it is not kept up-to date with the pip package itself as you upgrade pip, and may break.
If you just use python -m pip directly, e.g. as in:
python -m pip install --user somepackage
python3 -m pip install --user somepackage
it goes through your Python path, finds the latest version of pip and executes that file.
It relies on the fact that file is executable through import, but that is a very standard type of interface, and therefore less likely to break than the hackier Debian script.
Then I recommend adding the following aliases to your .bashrc:
pip() ( python -m pip "$#" )
pip3() ( python3 -m pip "$#" )
The Ubuntu 18.04 /usr/bin/pip3 file does:
from pip import main
and presumably main was removed from pip at some point which is what broke things.
The breaking pip commit appears to be: 95bcf8c5f6394298035a7332c441868f3b0169f4 "Move all internal APIs to pip._internal" which went into pip 18.0.
Tested in Ubuntu 16.04 after an update from pip3 9.0.1 to 18.0.
pyenv
Ultimately however, for serious Python development I would just recommend that you install your own local Python with pyenv + virtualenv, which would also get around this Ubuntu bug: https://askubuntu.com/questions/682869/how-do-i-install-a-different-python-version-using-apt-get/1195153#1195153
You can resolve this issue by reinstalling pip.
Use one of the following command line commands to reinstall pip:
Python2:
python -m pip uninstall pip && sudo apt install python-pip --reinstall
Python3:
python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall
Check if pip has been cached on another path, to do so, call $ which pip and check that the path is different from the one prompted in the error, if that's the case run:
$ hash -r
When the cache is clear, pip will be working again.
reference: http://cheng.logdown.com/posts/2015/06/14/-usr-bin-pip-no-such-file-or-directory
I'm running on a system where I have sudo apt but no sudo pip. (And no su access.) I got myself into this same situation by following the advice from pip:
You are using pip version 8.1.1, however 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
None of the other fixes worked for me, because I don't have enough admin privileges. However, a few things stuck with me from reading up on this:
I shouldn't have done this. Sure, pip told me to. It lied.
Using --user solves a lot of issues by focusing on the user-only directory.
So, I found this command line to work to revert me back to where I was. If you were using a different version than 8.1.1, you will obviously want to change that part of the line.
python -m pip install --force-reinstall pip==8.1.1 --user
That's the only thing that worked for me, but it worked perfectly!
I met the same problem on my Ubuntu 16.04 system. I managed to fix it by re-installing pip with the following command:
curl https://bootstrap.pypa.io/get-pip.py | sudo python3
Recover with python3 -m pip install --user pip==9.0.1 (or the version that worked)
Same thing happened to me on Pixelbook using the new LXC (strech). This solution is very similar to the accepted one, with one subtle difference, whiched fixed pip3 for me.
sudo python3 -m pip install --upgrade pip
That bumped the version, and now it works as expected.
I found it here ... Python.org: Ensure pip is up-to-date
The commands above didn't work for me but those were very helpful:
sudo apt purge python3-pip
sudo rm -rf '/usr/lib/python3/dist-packages/pip'
sudo apt install python3-pip
cd
cd .local/lib/python3/site-packages
sudo rm -rf pip*
cd
cd .local/lib/python3.5/site-packages
sudo rm -rf pip*
sudo pip3 install jupyter
In ubuntu 18.04.1 Bionic Beaver, you need to log out and log back in (restart not necessary) to get the proper environment.
$ sudo apt install python-pip
$ pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ pip install --upgrade pip
$ pip --version
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
from pip import main
ImportError: cannot import name main
$ exit
<login>
$ pip --version
pip 18.1 from /home/test/.local/lib/python2.7/site-packages/pip (python 2.7)
I use sudo apt remove python3-pip then pip works.
~ sudo pip install pip --upgrade
[sudo] password for sen:
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
from pip import main
ImportError: cannot import name 'main'
➜ ~ sudo apt remove python3-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libexpat1-dev libpython3-dev libpython3.5-dev python-pip-whl python3-dev python3-wheel
python3.5-dev
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
python3-pip
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 569 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 215769 files and directories currently installed.)
Removing python3-pip (8.1.1-2ubuntu0.4) ...
Processing triggers for man-db (2.7.5-1) ...
➜ ~ pip
Usage:
pip <command> [options]
For Python version 2.7 #Anthony solution works perfect, by changing python3 to python as follows:
sudo python -m pip uninstall pip && sudo apt install python-pip --reinstall
What worked for me to fix the error with using pip3 was:
sudo cp -v /usr/local/bin/pip3 /usr/bin/pip3
Everything works:
demon#UbuntuHP:~$ pip -V
pip 10.0.1 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
demon#UbuntuHP:~$ pip2 -V
pip 10.0.1 from /home/demon/.local/lib/python2.7/site-packages/pip (python 2.7)
demon#UbuntuHP:~$ pip3 -V
pip 10.0.1 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
Maybe the new 10.0.1 version of pip doesn't update the binary in /usr/bin ? (which seems it does not)
EDIT: the same issue occurs in Ubuntu 18.04. The best solution I've found is to symlink the pip binaries from /home/<user/.local/bin to /usr/local/bin or /usr/bin (depending on your preference), as follows:
ln -sv /home/<user>/.local/bin/pip /usr/local/bin/pip
ln -sv /home/<user>/.local/bin/pip2 /usr/local/bin/pip2
ln -sv /home/<user>/.local/bin/pip2.7 /usr/local/bin/pip2.7
ln -sv /home/<user>/.local/bin/pip3 /usr/local/bin/pip3
ln -sv /home/<user>/.local/bin/pip3.6 /usr/local/bin/pip3.6
NOTE: replace <user> with your current running user
The associated versions (latest) are in:
Version 3.6:
/home/demon/.local/lib/python3.6/site-packages/pip (python 3.6)
Version 2.7:
/home/demon/.local/lib/python2.7/site-packages/pip (python 2.7)
Trick and works too
sudo -H pip install lxml
I had this same error, but python -m pip was still working, so I fixed it with the nuclear option sudo python -m pip install --upgrade pip. It did it for me.
For what it's worth, I had the problem with pip (not pip2 or pip3):
$ pip -V
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
from pip import main
ImportError: cannot import name main
$ pip2 -V
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ pip3 -V
pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)
Somehow (I can't remember how) I had python stuff installed in my ~/.local directory. After I removed the pip directory from there, pip started working again.
$ rm -rf /home/precor/.local/lib/python2.7/site-packages/pip
$ pip -V
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
Is something wrong with the packages, when it generating de file /usr/bin/pip,
you have to change the import:
from pip import main
to
from pip._internal import main
That solves the problem, I'm not sure why it generated, but it saids somthing in the following issue:
After pip 10 upgrade on pyenv "ImportError: cannot import name 'main'"
You can try this:
sudo ln -sf $( type -P pip ) /usr/bin/pip
I also run into this problem when I wanted to upgrade system pip pip3 from 9.0.1 to 19.2.3.
After running pip3 install --upgrade pip, pip version becomes 19.2.3. But main() has been moved in pip._internal in the latest version, which leaves pip3 broken.
So in file /usr/bin/pip3, replace line 9: from pip import main with from pip._internal import main. The issue will be fixed, works the same for python2-pip. (Tested on Ubuntu 18.04 distribution)
According to #Vincent H.'s answer
Please run the following commands to do the fix. After running python3 -m pip install --upgrade pip, please run the following command.
hash -r pip
Source: https://github.com/pypa/pip/issues/5221
you can simply fix the pip and pip3 paths using update-alternatives
first thing you should check is your current $PATH
run echo $PATH and see is you can find /usr/local/bin which is where pip3 and pip usually are
there is a change your system is looking here /bin/pip and /bin/pip3
so i will say fix the PATH by adding to your ~/.bash_profile file so it persists
export PATH=$PATH:/usr/local/bin
and then check is its fixed with which pip and which pip3
if not then use update-alternatives to fix it finally
update-alternatives --install /bin/pip3 pip3 /usr/local/bin/pip3 30
and if you want to point pip to pip3 then
update-alternatives --install /bin/pip pip /usr/local/bin/pip3 30
I have the same problem and solved it. Here is my solution.
First, when I run pip install something, the error came out like this:
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in <module>
from pip import main
ImportError: cannot import name 'main'
So, I cd into the file /usr/bin/ and cat pip3 to see the code in it. I see this in it:
#!/usr/bin/python3
# GENERATED BY DEBIAN
import sys
# Run the main entry point, similarly to how setuptools does it, but because
# we didn't install the actual entry point from setup.py, don't use the
# pkg_resources API.
from pip import main
if __name__ == '__main__':
sys.exit(main())
And then I think that it was not in the installation path. So I cd into the python3-pip, like this:
cd /.local/lib/python3.5/site-packages/pip
P.S.: you have to cd into the right directions in your computer
Then I cat the file to see the differences(you can use other operations to see the code):
cat __main__.py
And I saw this:
from __future__ import absolute_import
import os
import sys
# If we are running from a wheel, add the wheel to sys.path
# This allows the usage python pip-*.whl/pip install pip-*.whl
if __package__ == '':
# __file__ is pip-*.whl/pip/__main__.py
# first dirname call strips of '/__main__.py', second strips off '/pip'
# Resulting path is the name of the wheel itself
# Add that to sys.path so we can import pip
path = os.path.dirname(os.path.dirname(__file__))
sys.path.insert(0, path)
from pip._internal import main as _main # isort:skip # noqa
if __name__ == '__main__':
sys.exit(_main())
So, can you see the difference? I can figure out that I have to make the file the same as the file in /usr/bin/pip3
So, I copied the code in /.local/lib/python3.5/site-packages/pip to replace the code in /usr/bin/pip3
and the problem disappear!
P.S.: pip3 or pip have no difference in this problem.
I will be happy if my solution solve your problem!
This Worked for me !
hash -r pip # or hash -d pip
Now, uninstall the pip installed version and reinstall it using the following commands.
python -m pip uninstall pip # sudo
sudo apt install --reinstall python-pip
If pip is broken, use:
python -m pip install --force-reinstall pip
Hope it helps!
I used the following code to load a module that might need install, thus avoiding this error (which I also got) - using the latest Python and latest pip with no problem
try
from colorama import Fore, Back, Style
except:
!pip install colorama
from colorama import Fore, Back, Style
import main from pip._internal
from pip._internal import main
Edit the pip code from
sudo nano /usr/bin/pip3
As #cryptoboy said - check what pip/python version you have installed
demon#UbuntuHP:~$ pip -V
demon#UbuntuHP:~$ pip2 -V
demon#UbuntuHP:~$ pip3 -V
and then check for no-needed libraries in your .local/lib/ folder.
I did backup of settings when I was migrating to newer Kubuntu and in had .local/lib/python2.7/ folder in my home directory. Installed python 3.6. I just removed the old folder and now everything works great!
On Debian you will need to update apt first....
sudo apt-get update -qq
sudo apt-get install python-pip -qq
sudo pip install pip --upgrade --quiet
sudo pip2 install virtualenv --quiet
If you skip 'sudo apt-get update -qq' your pip will become corrupt and display the 'cannot find main' error.

How to install xlrd in python3 library

I am trying to install xlrd to read Excel files in python.
I have tried this: pip install -U pip setuptools. My macOS Mojave 10.4.3 has Python 2.7 which is where the default install goes to. But I have also installed Python3.7. How do I get pip install to my 3.7 directory?
I am on Mac machine(Catalina -version 10.15.5) and below pip3 command worked for me.
pip3 install xlrd
python version : 3.7.6
OS : Mac-Catalina(10.15.5)
Thanks to #Tapan Hegde, pip3 install xlrd worked from me, after installing the pip3, like this:
sudo apt update
apt install python3-pip
pip3 install xlrd
I reckon the easiest/cleanest solution would be to use a tool that isolates your python environment, such as virtualenv
Once installed, create a virtual env by specifying which version of python you want to use:
$> virtualenv -p python3 env
Note: puttin python3 directly works only for mac, with linux, you must specify the absolute path or your python binary.
And then 'activate' your environment:
$> source env/bin/activate
From here, any python or pip command you use will use python3.
$> pip install xlrd
Virtualenv has the advantage of not 'polluting' your local python installation, your can manage your pip modules installed more easily.
If you want more detail on how it works and the other alternatives, check this post
When pip install xlrd not work and in computer is still old version, then try do it with current version, for example pip install xlrd==2.0.1.
The current versions are here

How to install pip for Python 3.6 on Ubuntu 16.10?

I'd like to start by pointing out that this question may seem like a duplicate, but it isn't. All the questions I saw here were regarding pip for Python 3 and I'm talking about Python 3.6. The steps used back then don't work for Python 3.6.
I got a clear Ubuntu 16.10 image from the official docker store.
Run apt-get update
Run apt-get install python3.6
Run apt-get install python3-pip
Run pip3 install requests bs4
Run python3.6 script.py
Got ModuleNotFoundError below:
Traceback (most recent call last):
File "script.py", line 6, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
Python's and pip's I have in the machine:
python3
python3.5
python3.5m
python3.6
python3m
python3-config
python3.5-config
python3.5m-config
python3.6m
python3m-config
pip
pip3
pip3.5
Let's suppose that you have a system running Ubuntu 16.04, 16.10, or 17.04, and you want Python 3.6 to be the default Python.
If you're using Ubuntu 16.04 LTS, you'll need to use a PPA:
sudo add-apt-repository ppa:jonathonf/python-3.6 # (only for 16.04 LTS)
Then, run the following (this works out-of-the-box on 16.10 and 17.04):
sudo apt update
sudo apt install python3.6
sudo apt install python3.6-dev
sudo apt install python3.6-venv
wget https://bootstrap.pypa.io/get-pip.py
sudo python3.6 get-pip.py
sudo ln -s /usr/bin/python3.6 /usr/local/bin/python3
sudo ln -s /usr/local/bin/pip /usr/local/bin/pip3
# Do this only if you want python3 to be the default Python
# instead of python2 (may be dangerous, esp. before 2020):
# sudo ln -s /usr/bin/python3.6 /usr/local/bin/python
When you have completed all of the above, each of the following shell commands should indicate Python 3.6.1 (or a more recent version of Python 3.6):
python --version # (this will reflect your choice, see above)
python3 --version
$(head -1 `which pip` | tail -c +3) --version
$(head -1 `which pip3` | tail -c +3) --version
In at least in ubuntu 16.10, the default python3 is python3.5. As such, all of the python3-X packages will be installed for python3.5 and not for python3.6.
You can verify this by checking the shebang of pip3:
$ head -n1 $(which pip3)
#!/usr/bin/python3
Fortunately, the pip installed by the python3-pip package is installed into the "shared" /usr/lib/python3/dist-packages such that python3.6 can also take advantage of it.
You can install packages for python3.6 by doing:
python3.6 -m pip install ...
For example:
$ python3.6 -m pip install requests
$ python3.6 -c 'import requests; print(requests.__file__)'
/usr/local/lib/python3.6/dist-packages/requests/__init__.py
This answer assumes that you have python3.6 installed. For python3.7, replace 3.6 with 3.7. For python3.8, replace 3.6 with 3.8, but it may also first require the python3.8-distutils package.
Installation with sudo
With regard to installing pip, using curl (instead of wget) avoids writing the file to disk.
curl https://bootstrap.pypa.io/get-pip.py | sudo -H python3.6
The -H flag is evidently necessary with sudo in order to prevent errors such as the following when installing pip for an updated python interpreter:
The directory '/home/someuser/.cache/pip/http' or its parent directory
is not owned by the current user and the cache has been disabled.
Please check the permissions and owner of that directory. If executing
pip with sudo, you may want sudo's -H flag.
The directory
'/home/someuser/.cache/pip' or its parent directory is not owned by the
current user and caching wheels has been disabled. check the
permissions and owner of that directory. If executing pip with sudo,
you may want sudo's -H flag.
Installation without sudo
curl https://bootstrap.pypa.io/get-pip.py | python3.6 - --user
This may sometimes give a warning such as:
WARNING: The script wheel is installed in '/home/ubuntu/.local/bin'
which is not on PATH. Consider adding this directory to PATH or, if
you prefer to suppress this warning, use --no-warn-script-location.
Verification
After this, pip, pip3, and pip3.6 can all be expected to point to the same target:
$ (pip -V && pip3 -V && pip3.6 -V) | uniq
pip 18.0 from /usr/local/lib/python3.6/dist-packages (python 3.6)
Of course you can alternatively use python3.6 -m pip as well.
$ python3.6 -m pip -V
pip 18.0 from /usr/local/lib/python3.6/dist-packages (python 3.6)
This website contains a much cleaner solution, it leaves pip intact as-well and one can easily switch between 3.5 and 3.6 and then whenever 3.7 is released.
http://ubuntuhandbook.org/index.php/2017/07/install-python-3-6-1-in-ubuntu-16-04-lts/
A short summary:
sudo apt-get install python python-pip python3 python3-pip
sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2
Then
$ pip -V
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
$ pip3 -V
pip 8.1.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)
Then to select python 3.6 run
sudo update-alternatives --config python3
and select '2'. Then
$ pip3 -V
pip 8.1.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)
To update pip select the desired version and
pip3 install --upgrade pip
$ pip3 -V
pip 9.0.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)
Tested on Ubuntu 16.04.
Some of the solutions above using the script get-pip.py worked until a couple of weeks ago.
The latest version of this script now requires python3.7 throwing the following error
ERROR: This script does not work on Python 3.6
The minimun supported Python version is 3.7.
Please use https://bootstrap.pypa.io/pip/3.6/get-pip.py instead.
So making the corresponding change works now.
wget https://bootstrap.pypa.io/pip/3.6/get-pip.py
sudo python3.6 get-pip.py

Pip won't install into python 3 when using homebrew python?

So I've got a virtualenv I've created using pyvenv-3.3, which I thought set up pip to install things into the virtualenv's path. However, I get the following outputs after I've activated by virtualenv:
$ pip --version
pip 1.4.1 from /usr/local/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg (python 2.7)
pip-3.3 --version
pip 1.4.1 from /usr/local/lib/python3.3/site-packages/pip-1.4.1-py3.3.egg (python 3.3)
This is all fine and good, but then my sys.path is this:
['',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python33.zip',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/plat-darwin',
'/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload',
'/Users/alexgolec/Documents/gutenberg/virtualenv/lib/python3.3/site-packages']
Most notably, the whole site-packages directory is outright missing. Furthermore, this output indicates to me that I'm probably using a python that installed through homebrew at some point. Most infuriatingly, none of the directories in my sys.path are compatible with pip.
Any thoughts on remedies? I won't manually edit my sys.path, but maybe there's some configuration with pip I can do?
Install virtualenvwrapper (makes using virtualenv so much easier):
pip install virtualenvwrapper
.. and then try this:
mkvirtualenv <your_env> -p /usr/local/bin/python3
Where homebrew sticks python is different from where your system has it. You can run the following to see what I'm talking about:
which python
which python3

Resources