how include a python package in my current project direct under linux? [duplicate] - python-3.x

I know the obvious answer is to use virtualenv and virtualenvwrapper, but for various reasons I can't/don't want to do that.
So how do I modify the command
pip install package_name
to make pip install the package somewhere other than the default site-packages?

The --target switch is the thing you're looking for:
pip install --target=d:\somewhere\other\than\the\default package_name
But you still need to add d:\somewhere\other\than\the\default to PYTHONPATH to actually use them from that location.
-t, --target <dir>
Install packages into <dir>. By default this will not replace existing files/folders in <dir>.
Use --upgrade to replace existing packages in <dir> with new versions.
Upgrade pip if target switch is not available:
On Linux or OS X:
pip install -U pip
On Windows (this works around an issue):
python -m pip install -U pip

Use:
pip install --install-option="--prefix=$PREFIX_PATH" package_name
You might also want to use --ignore-installed to force all dependencies to be reinstalled using this new prefix. You can use --install-option to multiple times to add any of the options you can use with python setup.py install (--prefix is probably what you want, but there are a bunch more options you could use).

Instead of the --target or --install-options options, I have found that setting the PYTHONUSERBASE environment variable works well (from discussion on a bug regarding this very thing):
PYTHONUSERBASE=/path/to/install/to pip install --user
(Or set the PYTHONUSERBASE directory in your environment before running the command, using export PYTHONUSERBASE=/path/to/install/to)
This uses the very useful --user option but tells it to make the bin, lib, share and other directories you'd expect under a custom prefix rather than $HOME/.local.
Then you can add this to your PATH, PYTHONPATH and other variables as you would a normal installation directory.
Note that you may also need to specify the --upgrade and --ignore-installed options if any packages upon which this depends require newer versions to be installed in the PYTHONUSERBASE directory, to override the system-provided versions.
A full example
PYTHONUSERBASE=/opt/mysterypackage-1.0/python-deps pip install --user --upgrade numpy scipy
..to install the scipy and numpy package most recent versions into a directory which you can then include in your PYTHONPATH like so (using bash and for python 2.6 on CentOS 6 for this example):
export PYTHONPATH=/opt/mysterypackage-1.0/python-deps/lib64/python2.6/site-packages:$PYTHONPATH
export PATH=/opt/mysterypackage-1.0/python-deps/bin:$PATH
Using virtualenv is still a better and neater solution!

To pip install a library exactly where I wanted it, I navigated to the location I wanted the directory with the terminal then used
pip install mylibraryName -t .
the logic of which I took from this page: https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/download

Installing a Python package often only includes some pure Python files. If the package includes data, scripts and or executables, these are installed in different directories from the pure Python files.
Assuming your package has no data/scripts/executables, and that you want your Python files to go into /python/packages/package_name (and not some subdirectory a few levels below /python/packages as when using --prefix), you can use the one time command:
pip install --install-option="--install-purelib=/python/packages" package_name
If you want all (or most) of your packages to go there, you can edit your ~/.pip/pip.conf to include:
[install]
install-option=--install-purelib=/python/packages
That way you can't forget about having to specify it again and again.
Any excecutables/data/scripts included in the package will still go to their default places unless you specify addition install options (--prefix/--install-data/--install-scripts, etc., for details look at the custom installation options).

Tested these options with python3.5 and pip 9.0.3:
pip install --target /myfolder [packages]
Installs ALL packages including dependencies under /myfolder. Does not take into account that dependent packages are already installed elsewhere in Python. You will find packages from /myfolder/[package_name]. In case you have multiple Python versions, this doesn't take that into account (no Python version in package folder name).
pip install --prefix /myfolder [packages]
Checks if dependencies are already installed. Will install packages into /myfolder/lib/python3.5/site-packages/[packages]
pip install --root /myfolder [packages]
Checks dependencies like --prefix but install location will be /myfolder/usr/local/lib/python3.5/site-packages/[package_name].
pip install --user [packages]
Will install packages into $HOME:
/home/[USER]/.local/lib/python3.5/site-packages
Python searches automatically from this .local path so you don't need to put it to your PYTHONPATH.
=> In most of the cases --user is the best option to use.
In case home folder can't be used because of some reason then --prefix.

pip3 install "package_name" -t "target_dir"
source - https://pip.pypa.io/en/stable/reference/pip_install/
-t switch = target

Nobody seems to have mentioned the -t option but that the easiest:
pip install -t <direct directory> <package>

pip install packageName -t pathOfDirectory
or
pip install packageName --target pathOfDirectorty

Just add one point to #Ian Bicking's answer:
Using the --user option to specify the installed directory also work if one wants to install some Python package into one's home directory (without sudo user right) on remote server.
E.g.,
pip install --user python-memcached
The command will install the package into one of the directories that listed in your PYTHONPATH.

Newer versions of pip (8 or later) can directly use the --prefix option:
pip install --prefix=$PREFIX_PATH package_name
where $PREFIX_PATH is the installation prefix where lib, bin and other top-level folders are placed.

To add to the already good advice, as I had an issue installing IPython when I didn't have write permissions to /usr/local.
pip uses distutils to do its install and this thread discusses how that can cause a problem as it relies on the sys.prefix setting.
My issue happened when the IPython install tried to write to '/usr/local/share/man/man1' with Permission denied. As the install failed it didn't seem to write the IPython files in the bin directory.
Using "--user" worked and the files were written to ~/.local. Adding ~/.local/bin to the $PATH meant I could use "ipython" from there.
However I'm trying to install this for a number of users and had been given write permission to the /usr/local/lib/python2.7 directory. I created a "bin" directory under there and set directives for distutils:
vim ~/.pydistutils.cfg
[install]
install-data=/usr/local/lib/python2.7
install-scripts=/usr/local/lib/python2.7/bin
then (-I is used to force the install despite previous failures/.local install):
pip install -I ipython
Then I added /usr/local/lib/python2.7/bin to $PATH.
I thought I'd include this in case anyone else has similar issues on a machine they don't have sudo access to.

If you are using brew with python, unfortunately, pip/pip3 ships with very limited options. You do not have --install-option, --target, --user options as mentioned above.
Note on pip install --user
The normal pip install --user is disabled for brewed Python. This is because of a bug in distutils, because Homebrew writes a distutils.cfg which sets the package prefix.
A possible workaround (which puts executable scripts in ~/Library/Python/./bin) is:
python -m pip install --user --install-option="--prefix=" <package-name>
You might find this line very cumbersome. I suggest use pyenv for management.
If you are using
brew upgrade python python3
Ironically you are actually downgrade pip functionality.
(I post this answer, simply because pip in my mac osx does not have --target option, and I have spent hours fixing it)

With pip v1.5.6 on Python v2.7.3 (GNU/Linux), option --root allows to specify a global installation prefix, (apparently) irrespective of specific package's options. Try f.i.,
$ pip install --root=/alternative/prefix/path package_name

I suggest to follow the documentation and create ~/.pip/pip.conf file. Note in the documentation there are missing specified header directory, which leads to following error:
error: install-base or install-platbase supplied, but installation scheme is incomplete
The full working content of conf file is:
[install]
install-base=$HOME
install-purelib=python/lib
install-platlib=python/lib.$PLAT
install-scripts=python/scripts
install-headers=python/include
install-data=python/data
Unfortunatelly I can install, but when try to uninstall pip tells me there is no such package for uninstallation process.... so something is still wrong but the package goes to its predefined location.

pip install /path/to/package/
is now possible.
The difference with this and using the -e or --editable flag is that -e links to where the package is saved (i.e. your downloads folder), rather than installing it into your python path.
This means if you delete/move the package to another folder, you won't be able to use it.

system` option, that will install pip package-bins to /usr/local/bin thats accessible to all users. Installing without this option may not work for all users as things go to user specific dir like $HOME/.local/bin and then it is user specific install which has to be repeated for all users, also there can be path issues if not set for users, then bins won't work. So if you are looking for all users - yu need to have sudo access:
sudo su -
python3 -m pip install --system <module>
logout
log back in
which <module-bin> --> it should be installed on /usr/local/bin/

Sometimes it works only works with Cache argument
-m pip install -U pip --target=C:\xxx\python\lib\site-packages Pillow --cache-dir C:\tmp

Related

python3: can't install psycopg2 [duplicate]

I cannot figure this out for the life of me.
When I pip install django-tenant-schemas it tries to install the dependency psycopg2 which requires the Python headers and gcc. I have all this installed and still keep getting this error!
./psycopg/psycopg.h:35:10: fatal error: libpq-fe.h: No such file or directory
So to install libpq-fe-h I need to sudo apt-get install libpq-dev..
..which returns..
libpq-dev is already the newest version (10.10-0ubuntu0.18.04.1).
Then when I sudo find / libpq-fe.h it doesn't seem to be in my OS.
I am lost at this point. If anyone can help I would highly appreciate it.
For some reason, the file is missing on the system.
As you're using apt-get, the system is dpkg based, presumably Debian or it's derivative. You can try the Ubuntu's package search to get which package contains a file with name ending in libpq-fe.h.
I found the package is libpq-dev and file's absolute path is /usr/include/postgresql/libpq-fe.h.
FWIW, on a dpkg based system, you can check which package gives a file if you know the file's absolute path:
% dpkg -S /usr/include/postgresql/libpq-fe.h
libpq-dev: /usr/include/postgresql/libpq-fe.h
Also, unlike find, locate keeps a cache of found files (mlocate.db) that is created everyday via cron; so if the file happens to be removed after the last run, you can run locate libfq-fe.h to get the absolute path to the file without needing to check the Ubuntu package search online.
So the package is libpq-dev. Now, reinstalling it will get everything to the default state i.e. all relevant files will be copied to the right places. As it is only a library package, no user/system level configurations will be overridden (and dpkg will prompt you for action for any package that does that).
To reinstall the package:
sudo apt-get install --reinstall libpq-dev
For me, I realized it was trying to use the deprecated setup.py so I installed wheel (pip install wheel) and that sorted it all out.
Well after installing these libraries
sudo dnf install python-virtualenv openssl-devel gcc libffi-devel libxslt-devel issue was not gone.
I used mlocate to find where libpq-fe.h file is located. On my system (Fedora 32) it was located at /usr/pgsql-10/include/libpq-fe.h
yum install mlocate
sudo updateb
locate libpq-fe.h
After all added this line to ~/.bash_profile
nano ~/.bash_profile
export PATH=/usr/pgsql-10/bin/:$PATH
Works fine, I can easily install psycopg2 without any trouble.
You need to create a LD_LIBRARY_PATH that indicates the path of your library /user/pgsql-11/lib
Source: The 3rd point of build prerequisites at https://www.psycopg.org/docs/install.html#build-prerequisites

installed python 3x on mac but terminal still show version still 2x

I installed python 3x via home brew, the process was successful.
However when I check version, it shows 2x
here is the terminal output
➜ ~ brew install python
Warning: python 3.7.2 is already installed, it's just not linked
You can use `brew link python` to link this version.
➜ ~ brew link python
Linking /usr/local/Cellar/python/3.7.2... Error: Permission denied # dir_s_mkdir - /usr/local/Frameworks
➜ ~ sudo brew link python
Error: Running Homebrew as root is extremely dangerous and no longer supported.
As Homebrew does not drop privileges on installation you would be giving all
build scripts full access to your system.
➜ ~ python -V
Python 2.7.10
I think I need to modify some kind of path. And do some magic then pray...lol
Anyway anyone know how to get this work on my machine? I'm mac latest
First, solve your permission problem by running the official command from the Homebrew Documentation, Troubleshooting page
cd /usr/local && sudo chown -R $(whoami) bin etc include lib sbin share var opt Cellar Caskroom Frameworks
Then run brew link python
And finally run echo $PATH and check that your /usr/local/bin has precedence over other directories.
you could do brew uninstall python3 and install it from the python website here https://www.python.org/
When you go install a module you do pip3 install packageName
And when you run a program you could run it from your IDE (I recommend VSCode) or run it from the terminal with python3 drag_python_file_here
you have to use python3 instead of python on Mac so like python3 path/to/file.py and pip is now pip3 so like pip3 install pillow

google-cloud-sdk installation not finding right Python 2.7 version in CentOS /usr/local/bin

Our server OS is CentOS 6.8, I was trying to install google-cloud-sdk, even though I installed
python 2.7 in /usr/local/bin
, it is still looking at old version of
python 2.6 in /usr/bin
. I tried giving export PATH=/usr/local/bin:$PATH to first look at /usr/local/bin than /usr/bin but still the problem persists. please suggest a way to fix.
The way I have solved this (and I know it works) is to first install Python 2.7 in whatever way you'd like, then install pip using Python 2.7 which will give you pip2.7. You can then use pip2.7 to install the google_compute_engine module so that it ends up in the right modules folder.
# get pip2.7
wget https://bootstrap.pypa.io/get-pip.py
python2.7 get-pip.py
# install the gcloud module
pip2.7 install google_compute_engine
You can then add this to your $HOME/.bashrc
export CLOUDSDK_PYTHON=/usr/local/bin/python2.7
This is the best repeatable way I know of
Go to the google-cloud-sdk folder and open the install.sh file.
Change the CLOUDSDK_PYTHON="python" value to CLOUDSDK_PYTHON="python2.7"
Rerun the install with the command:
./install.sh
Or you could install it using yum:
https://cloud.google.com/sdk/downloads#yum
If you are on Windows This is a simple solution that worked for me:
open Powershell as administrator and run this to add your Python folder to your environment's PATH:
$env:Path += ";C:\python27_x64\"
Then re-run the command that gave you the original error. It should work fine.
Alternatively you could run that original (error-causing) command within the Cloud SDK Shell. That also worked for me.
I found a CLOUDSDK_PYTHON inside my ~/.bash_profile (or ~/.zshenv).
I removed it, and went back into my google-cloud-sdk directory and reinstalled it.
./install.sh
This fixed the issue for me.

Step by step: Installing Python 3.3, Lighttpd & Pymongo on Ubuntu 12.04

I'm currently migrating to new computer and I need to reinstall the software I am using which are:
Python 3.3,
Lighttpd (newest version),
Pymongo (newest version),
Ubuntu 12.04 Desktop (The System I'm using)
I started to install Python 3.3 by downloading it from the its official website (in tar.bz2 file) and by following this tutorial. Afterwards I installed Lighttpd and changed the lighttpd.conf for Python by following this tutorial, too.
I tried several paths for my cgi.assign, none of them worked. Especially /opt/python3.3/bin/python3.3 should be working, but it shows 500 - internal Server error all the time with a "hello world" test script.
Now regardless to this problem I have no clue on installing Pymongo. If I try to intall pip OR easy_install python3.3 I have to manually download it and execute the setup.py with my python3.3 executable, right? Because this always fails with an error:
`Error missing zlib on a bundle called distribute-0.7.3 (is this even the right tool I need, because it seems to be a legacy wrapper !?) or unknown url type: https for pymongo2.6.2 itself.`
I'm getting crazy with this setup. Why is this so difficult to handle? Other programs are just a few clicks to install even on a system like Ubuntu, but these particular development tools seem to be really difficult to install.If anybody has an idea on how to install all three together or has information on a better solution please help me out.
The system is used to program Python scripts in Eclipse and trying them out directly on the system (lighttpd). The database used is MongoDB. Python and MongoDB are communicating over the Pymongo driver. I am planning to use the system on a Server distribution on release and it has to be nicely scalable on a high amount of excecutions.
Thanks for your time,
It's easiest to use the Ubuntu repositories:
sudo apt-get update
sudo apt-get install python3 python3-pip lightppd python-pymongo
Or if that only installs the python2.x pymongo, use pip, which you've just installed:
sudo pip-3.3 install pymongo
Or better yet, use a virtualenv with the help of virtualenvwrapper (docs)
sudo pip install virtualenvwrapper
... # follow instructions for installing virtualenvwrapper
mkvirtualenv --python=/usr/bin/python3 -i pymongo mongoppd
workon mongoppd
... which will segregate the environment I've called 'mongoppd' from the rest of your system so you can't cause any trouble. Then you don't need sudo to pip-3.3 install things, just workon mongoppd then pip-3.3 install [...]. Or after the -i flag when you create the virtualenv to get it installed straight away.
In general, on Ubuntu, you should hardly ever have to install something manually. Your first attempt should be using sudo apt-get install (use tab-complete to see what's available or just google "ubuntu 12.04 packages [...]" and you'll find the list of packages). Then for python use pip install or pip-3.3 install as appropriate. You'll only need to run python setup.py install if you need to install a development version of a package or something obscure that's not on pip. I don't think there's a good reason to ever use easy_install these days.

apt-get for Cygwin?

Is there any apt-get-like program for use with Cygwin?
I already tried cyg-apt but when I try I get this error:
cyg-apt: downloading: http://cygwin.mirrors.pair.com/setup-2.bz2
cyg-apt: downloading: http://cygwin.mirrors.pair.com/setup-2.ini
cyg-apt: bad URL http://cygwin.mirrors.pair.com/setup-2.ini, exiting.
Best I have ever used:
apt-cyg package manager
You can do this using Cygwin’s setup.exe from Windows command line. Example:
cd C:\cygwin64
setup-x86_64 -q -P wget,tar,gawk,bzip2,subversion,vim
For a more convenient installer, you may want to use the
apt-cyg package manager. Its syntax is
similar to apt-get, which is a plus. For this, follow the above steps and then
use Cygwin Bash for the following steps:
wget rawgit.com/transcode-open/apt-cyg/master/apt-cyg
install apt-cyg /bin
Now that apt-cyg is installed. Here are a few examples of installing some
packages:
apt-cyg install nano
apt-cyg install git
apt-cyg install ca-certificates
Update: you can read the more complex answer, which contains more methods and information.
There exists a couple of scripts, which can be used as simple package managers. But as far as I know, none of them allows you to upgrade packages, because it’s not an easy task on Windows since there is not possible to overwrite files in use. So you have to close all Cygwin instances first and then you can use Cygwin’s native setup.exe (which itself does the upgrade via “replace after reboot” method, when files are in use).
apt-cyg
The best one for me. Simply because it’s one of the most recent. It works correctly for both platforms - x86 and x86_64. There exists a lot of forks with some additional features. For example the kou1okada fork is one of improved versions.
Cygwin’s setup.exe
It has also command line mode. Moreover it allows you to upgrade all installed packages at once.
setup.exe-x86_64.exe -q --packages=bash,vim
Example use:
setup.exe-x86_64.exe -q --packages="bash,vim"
You can create an alias for easier use, for example:
alias cyg-get="/cygdrive/d/path/to/cygwin/setup-x86_64.exe -q -P"
Then you can for example install the Vim package with:
cyg-get vim
you can always make a bash alias to setup*.exe files in $home/.bashrc
cygwin 32bit
alias cyg-get="/cygdrive/c/cygwin/setup-x86.exe -q -P"
cygwin 64bit
alias cyg-get="/cygdrive/c/cygwin64/setup-x86_64.exe -q -P"
now you can install packages with
cyg-get <package>
This got it working for me:
curl https://raw.githubusercontent.com/transcode-open/apt-cyg/master/apt-cyg > \
apt-cyg && install apt-cyg /bin
No. The only officially supported tool for downloading and updating Cygwin packages is the setup.exe file you used for the initial install, although that can be invoked with command line arguments to help the process.
From that same page:
The basic reason for not having a more full-featured package manager is that such a program would need full access to all of Cygwin's POSIX functionality. That is, however, difficult to provide in a Cygwin-free environment, such as exists on first installation. Additionally, Windows does not easily allow overwriting of in-use executables so installing a new version of the Cygwin DLL while a package manager is using the DLL is problematic.
You can use Chocolatey to install cyg-get and then install your packages with it.
For example:
choco install cyg-get
Then:
cyg-get install my-package

Resources