My goal is to serve a hello world Django app that uses postgres on an EC2 instance running Ubuntu. I logged in via ssh and cloned a git repo containing a Django project with this requirements.txt:
Django==1.8.2
djangorestframework==3.1.2
psycopg2==2.6
I created a virtualenv and then, when I ran (ec2_deploy_test)ubuntu#ip-172-31-22-100:~/ec2-deploy-test$ pip install -r requirements.txt, this exception was thrown:
Collecting psycopg2==2.6 (from -r requirements.txt (line 3))
/home/ubuntu/Envs/ec2_deploy_test/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. For more information, see
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning Downloading psycopg2-2.6.tar.gz (367kB)
100% |████████████████████████████████| 368kB 785kB/s Building wheels for collected packages: psycopg2 Exception: Traceback (most
recent call last): File
"/home/ubuntu/Envs/ec2_deploy_test/local/lib/python2.7/site-packages/pip/basecommand.py",
line 223, in main
status = self.run(options, args) File "/home/ubuntu/Envs/ec2_deploy_test/local/lib/python2.7/site-packages/pip/commands/install.py",
line 291, in run
wb.build(autobuilding=True) File "/home/ubuntu/Envs/ec2_deploy_test/local/lib/python2.7/site-packages/pip/wheel.py",
line 753, in build
ensure_dir(output_dir) File "/home/ubuntu/Envs/ec2_deploy_test/local/lib/python2.7/site-packages/pip/utils/init.py", line 70, in ensure_dir
os.makedirs(path) File "/home/ubuntu/Envs/ec2_deploy_test/lib/python2.7/os.py", line 150, in
makedirs
makedirs(head, mode) File "/home/ubuntu/Envs/ec2_deploy_test/lib/python2.7/os.py", line 150, in
makedirs
makedirs(head, mode) File "/home/ubuntu/Envs/ec2_deploy_test/lib/python2.7/os.py", line 150, in
makedirs
makedirs(head, mode) File "/home/ubuntu/Envs/ec2_deploy_test/lib/python2.7/os.py", line 157, in
makedirs
mkdir(name, mode) OSError: [Errno 13] Permission denied: '/home/ubuntu/.cache/pip/wheels/ab'
Then I ran (ec2_deploy_test)ubuntu#ip-172-31-22-100:~/ec2-deploy-test$ sudo pip install -r requirements.txt and psycopg2 installed successfully.
Why would I need root permissions to install a python package in my virtual environment? I am new to Linux and sysadmin in general so all advice is welcome. Thanks in advance.
For some reason you don't have an access to create a directory inside the /home/ubuntu/.cache/pip/wheels/ab. Normally this problem shouldn't appear; anyway, since it happened, just change the rights of the .cache directory recursively. I guess that the problem is an ownership, so try to launch the command sudo chown -R <USERNAME> ~/.cache/pip, where the <USERNAME> supposed to be the name of your user.
An advice — try to not launch an apps from root without a real need in this. Most probably that the directory to which you have no an access was created by some application being run with root rights — and now an ownership messed.
I assume you used virtualenv -p /usr/bin/python3.4 env,
I had the same problem
Its python 3 and might have nothing to do with permissions or root if you tried Angles recommendation and it does not work. You need to install python3-dev ... Or just stick with python2. hope that helps
heres the link: Trouble with psycopg2 in virtualenv python3 for use with Django
Related
I am trying to use virtualenv to install some python modules for deep learning. I'm on Ubuntu 18.04 where there was python2.7 intalled. Somehow I installed python 3 and made a virtualenv called dl4cv where I can work on deep learning projects. Now that I'm trying to make a new virtualenv mkvirtualenv temp after installing virtualenvwrapper, I got an error as shown below :
hdafa#hdafa-HP-ProBook-450-G5:~$ mkvirtualenv temp
Running virtualenv with interpreter /usr/bin/python2
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/virtualenv.py", line 2375, in <module>
main()
File "/usr/lib/python3/dist-packages/virtualenv.py", line 724, in main
symlink=options.symlink)
File "/usr/lib/python3/dist-packages/virtualenv.py", line 946, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "/usr/lib/python3/dist-packages/virtualenv.py", line 1157, in install_python
mkdir(lib_dir)
File "/usr/lib/python3/dist-packages/virtualenv.py", line 329, in mkdir
os.makedirs(path)
File "/usr/lib/python2.7/os.py", line 150, in makedirs
makedirs(head, mode)
File "/usr/lib/python2.7/os.py", line 150, in makedirs
makedirs(head, mode)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/home/hdafa/.virtualenvs/temp'
any idea of how I can fix it?
You've been using sudo too much. Please stop.
First, fix permissions of your home:
sudo chown -R hdafa /home/hdafa
From now on please use only virtual environments and avoid sudo except for system administration tasks.
It looks like your OS is denying permission to make a directory. Try running the command with root permissions: sudo mkvirtualenv temp. The keyword sudo uses root permissions, and so it should be able to create the directory that it needs to.
I am unable to install packages in my intel dev cloud. I tried installing packages using pip from bash shell. I also created a new conda environment and tried installing packages there. It didn't work either. I think the problem is with permissions in file system. The error says "read-only file system" as you can see.
Any suggestions can greatly help.
P.S: I am trying to install keras and sklearn
please check the error log below.
Installing collected packages: keras
Exception:
Traceback (most recent call last):
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/wheel.py", line 316, in clobber
ensure_dir(destdir)
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/glob/intel-python/versions/2018u2/intelpython3/lib/python3.6/site-packages/docs'
I think the problem is with permissions in file system. The error says "read-only file system"
Yes, you correctly diagnosed the problem.
You need to choose a writable filesystem to write the installed files to.
You may find the df and mount commands helpful. Start with man df.
With the guidance of #J_H I found out the answer.
pip usually tries to install packages globally, which is not allowed on shared platforms like Intel DevCloud. Hence we need to install packages for single user that is you.
Hence
pip install --user keras
installs keras in your home directory and you are not restricted to write in your home directory. Problem Solved!
I am having a heck of a time installing an IDE, python and pip to play well across user accounts.
I have started a new job, where I am allowed to work on school/programming stuff at work. I've even been given the go-ahead to install everything I need on my work computer (two nice big monitors!).
I have also been given administrative login, as I can't install anything on my regular account.
So, I log in to the admin account and followed the documentation on how to install Python and Pip for all users:
Check option for install for all uses
Check Add Python 3.8 to path
Select Customize installation
Again select Install for all users
Ensure the installation path is not in a user directory but rather in C:/Programs/Python...
Install PyCharm IDE
Great. Done. I reboot my machine and login to my humble user account and clone a project from GitHub.
Ok, so I want to add a package using pip... First thing I notice is that pip install <package> doesn't work at all. I get a type error.
Next, I try py pip install <package> and that gets things going, but of course fails after downloading the package. I don't have access to install it.
So I read the solution is to use py pip install <package> --user, which proceeds to tell me that installation is complete and successful with one warning:
WARNING: The script f2py.exe is installed in 'C:\Users\WNeill\AppData\Roaming\Python\Python38\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
I find this interesting, since that's not where I installed Python. Also I find it interesting, because I had the installer add Python to the PATH. I verify my python installation is indeed in my machine's "Program Files" and satisfied that it is, I look up the f2py.script and find that it is part of the package I just tried to install.
A little wierded out, I type my import statement. Sure enough. No dice. "no module named numpy"
At this point, I'm a bit tired of running back and forth between two different user accounts repeatedly installing things and uninstalling them. I've been at it for about 6 hours. Is there a way to make python, my IDE and pip all play nicely together or should I just go back to bringing my personal laptop to work and forget my dreams of coding on one big screen with documentation up on the other?
Edit 1: I don't think this matters, but I'm no expert. I'm doing all of this from the PyCharm native terminal which is running Git Bash.
Edit 2: I decided to check my environment variables on my user account (even though they were added to the PATH variable from the admin account. The PATH variable was not set, so I added the the python38 path and he python38/Scripts. Now I am getting a whole new set of errors which, among other things in a two page error report, is telling me that I have to download Microsoft Visual C++. Is this for real?
Edit 3: I have also tried pip install --upgrade setuptools at another user's suggestion. This too fails miserably with error:
Traceback (most recent call last):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\Scripts\pip-script.py", line 11, in <module>
load_entry_point('pip==19.0.3', 'console_scripts', 'pip')()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\__init__.py", line 78, in main
return command.main(cmd_args)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\cli\base_command.py", line 225, in main
session = self._build_session(
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\cli\base_command.py", line 87, in _build_session
session = PipSession(
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\download.py", line 344, in __init__
self.headers["User-Agent"] = user_agent()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\download.py", line 134, in user_agent
setuptools_version = get_installed_version("setuptools")
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_internal\utils\misc.py", line 902, in get_installed_version
working_set = pkg_resources.WorkingSet()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 565, in __init__
self.add_entry(entry)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 621, in add_entry
for dist in find_distributions(entry, True):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1886, in find_eggs_in_zip
if metadata.has_metadata('PKG-INFO'):
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1405, in has_metadata
return self.egg_info and self._has(self._fn(self.egg_info, name))
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1760, in _has
return zip_path in self.zipinfo or zip_path in self._index()
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1637, in zipinfo
return self._zip_manifests.load(self.loader.archive)
File "C:\Users\WNeill\PycharmProjects\data-science-at-home\venv\lib\site-packages\pip-19.0.3-py3.8.egg\pip\_vendor\pkg_resources\__init__.py", line 1594, in load
mtime = os.stat(path).st_mtime
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\WNeill\\PycharmProjects\\data-science-at-home\\venv\\lib\\site-packages\\setuptools-40.8.0-py3.8.egg'
I receive a funny error when trying to install virtualenv:
$ pip install virtualenv
yields the following error:
Collecting virtualenv
Using cached virtualenv-15.0.2-py2.py3-none-any.whl
Installing collected packages: virtualenv
Exception:
Traceback (most recent call last):
File "/home/matt/.local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/matt/.local/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/home/matt/.local/lib/python2.7/site-packages/pip/req/req_set.py", line 742, in install
**kwargs
File "/home/matt/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 831, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/home/matt/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 1032, in move_wheel_files
isolated=self.isolated,
File "/home/matt/.local/lib/python2.7/site-packages/pip/wheel.py", line 346, in move_wheel_files
clobber(source, lib_dir, True)
File "/home/matt/.local/lib/python2.7/site-packages/pip/wheel.py", line 324, in clobber
shutil.copyfile(srcfile, destfile)
File "/usr/lib/python2.7/shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/virtualenv.py'
I was wondering if anybody had some insight as to fixing this? I read that there may be reference in usr/bin to older versions of software, this is possible, but this computer is on the newer side. Is the best bet to go in and clear all references to 'virtualenv' (all that I can find that is
)?
Thank you!
The problem is that you're trying to install a package to a location that typically requires root privileges. You can do as #Vince West suggested and run sudo pip install virtualenv, or you can do pip install --user virtualenv, which will install packages to ~/.local/lib/python<version>/site-packages. If you go that route, you'll likely want to make sure $HOME/.local/bin is included in your $PATH.
I am experiencing a problem when using virtualenv in openSuSE 12.3 with Python3:
I installed the python3 and python3-devel packages.
Then I installed the newest distribute and pip and finally virtualenv using pip.
When I try to create a virtualenv I get the following error:
$ virtualenv-3.3 venv01
Using base prefix '/usr'
New python executable in venv01/bin/python3.3
Also creating executable in venv01/bin/python
Installing distribute.........................................................................................................................................................................................................................................................................................................................................................................................................done.
Installing pip....
Complete output from command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz:
/home/user/venv01/bin/python3.3: can't open file '/home/user/venv01/bin/easy_install': [Errno 2] No such file or directory
----------------------------------------
...Installing pip...done.
Traceback (most recent call last):
File "/usr/local/bin/virtualenv-3.3", line 9, in <module>
load_entry_point('virtualenv==1.9.1', 'console_scripts', 'virtualenv-3.3')()
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 979, in main
no_pip=options.no_pip)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1094, in create_environment
install_pip(py_executable, search_dirs=search_dirs, never_download=never_download)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 667, in install_pip
filter_stdout=_filter_setup)
File "/usr/local/lib/python3.3/site-packages/virtualenv.py", line 1057, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /home/user/venv01/bin/python3.3 -x /home/user/venv01/bin/easy_install /usr/local/lib/pytho...ort/pip-1.3.1.tar.gz failed with error code 2
Running it with -vvv yields some interesting output that shows that distribute
is extracted to <venv>/local/lib/python3.3/site-packages/.
I wonder why distribute is not installed into <venv>/lib/python3.3/site-packages/?
Does anyone have an idea why this happens and how I can use virtualenv in
openSuSE without compiling python3 myself?
[now fixed in latest patches from opensuse]
[oh! i just realised you are the same person as the original link. sorry. but i will leave this as it is a top result for google search on this issue (was searching myself for any update) so it may help others.]
this is a known issue, discussed at https://forums.opensuse.org/english/get-technical-help-here/applications/484475-using-virtualenv-python-3-a.html and with an open bug at https://bugzilla.novell.com/show_bug.cgi?id=809831
the only work-around i know of is to install everything yourself. it's not so hard, and i describe what is necessary at http://www.acooke.org/cute/GettingPyt0.html
basically:
install python 3.3 from source (do an "altinstall" to install as /usr/local/bin/python3.3)
fix the lib issue (link lib_dynload from /usr/local/lib64/python3.3 to /usr/local/lib/python3.3)
install distutils
install virtualenv
then you can use python3.3 and virtual-env-3.3 etc as expected.