I recently reinstalled ubuntu and did upgrade to 16.04 and cannot use python:
$ python manage.py runserver
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Aborted
At this point, python itself doesn't work
$ python
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Aborted
Even this suggestion is no longer working:
unset PYTHONHOME
unset PYTHONPATH
Every every I fix it one way, it comes back again. Several answers help to fix it temporarily, but not for good. I have reinstalled python and python3 several times. What can I do from here?
For Python-3 try removing virtual environment files. And resetting it up.
rm -rf venv
virtualenv -p /usr/bin/python3 venv/
source venv/bin/activate
pip install -r requirements.txt
https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3
edit fo
For Windows10 User.
I was using python3.4 on Windows10. I installed python3.5. I couldn't find PYTHONPATH, PYTHONHOME env variable. If I command python in CMD console, It kept using python3.4. I deleted python3.4. Whenever I command python in CMD console, it starts showing an error like below.
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
I searched to figure out my problem. Solution was simple. When you install python3.5, you can custom install and check Add Python to environment variables in Advanced Options.
I just leave here for case that someone have similar issues visit here so that they don't waste their precious time much to figure out.
I was facing the same problem under Windows7. The error message looks like that:
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
Current thread 0x000011f4 (most recent call first):
I have installed python 2.7(uninstalled now), and I checked "Add Python to environment variables in Advanced Options" while installing python 3.6. It comes out that the Environment Variable "PYTHONHOME" and "PYTHONPATH" is still python2.7.
Finally I solved it by modify "PYTHONHOME" to python3.6 install path and remove variable "PYTHONPATH".
For the same issue on Windows7
You will see an error like this if your environment variables/ system variables are incorrectly set:
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'
Current thread 0x00001db4 (most recent call first):
Fixing this is really simple:
When you download Python3.x version, and run the .exe file, it gives you an option to customize where in your system you want to install Python. For example, I chose this location: C:\Program Files\Python36
Then open system properties and go to "Advanced" tab (Or you can simply do this: Go to Start > Search for "environment variables" > Click on "Edit the system environment variables".) Under the "Advanced" tab, look for "Environment Variables" and click it. Another window with name "Environment Variables" will pop up.
Now make sure your user variables have the correct Python path listed in "Path Variable". In my example here, you should see C:\Program Files\Python36. If you do not find it there, add it, by selecting Path Variable field and clicking Edit.
Last step is to double-check PYTHONHOME and PYTHONPATH fields under System Variables in the same window. You should see the same path as described above. If not add it there too.
Then click OK and go back to CMD terminal, and try checking for python. The issue should now be resolved. It worked for me.
I had this error during migration to Ubuntu 17.10, and this solved the problem :
sudo dpkg-reconfigure python3
Maybe you will have to close your session and reconnect.
Look at /lib/python3.5 and you will see broken links to python libraries. Recreate it to working directory.
Next error -
./script/bin/pip3
Failed to import the site module
Traceback (most recent call last):
File "/home/script/script/lib/python3.5/site.py", line 703, in <module>
main()
File "/home/script/script/lib/python3.5/site.py", line 683, in main
paths_in_sys = addsitepackages(paths_in_sys)
File "/home/script/script/lib/python3.5/site.py", line 282, in addsitepackages
addsitedir(sitedir, known_paths)
File "/home/script/script/lib/python3.5/site.py", line 204, in addsitedir
addpackage(sitedir, name, known_paths)
File "/home/script/script/lib/python3.5/site.py", line 173, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "/home/script/script/lib/python3.5/types.py", line 166, in <module>
import functools as _functools
File "/home/script/script/lib/python3.5/functools.py", line 23, in <module>
from weakref import WeakKeyDictionary
File "/home/script/script/lib/python3.5/weakref.py", line 12, in <module>
from _weakref import (
ImportError: cannot import name '_remove_dead_weakref'
fixed like this - https://askubuntu.com/questions/907035/importerror-cannot-import-name-remove-dead-weakref
cd my-virtualenv-directory
virtualenv . --system-site-packages
I had deleted rm -r /usr/lib/python*
dpkg -S '/usr/lib/python3*' | grep encodings
And then found I needed libpython3.10-minimal
sudo apt-get install --reinstall libpython3.10-minimal
I finally ran
sudo apt-get install --reinstall $(dpkg -S '/usr/lib/python3*' | cut -d ':' -f1 | cut -d ',' -f1 | sort | uniq | tr '\n' ' ')
sudo apt-get install --reinstall $(dpkg -S '/usr/lib/python2*' | cut -d ':' -f1 | cut -d ',' -f1 | sort | uniq | tr '\n' ' ')
To re-install all the packages I deleted files for
Had the same problem when updating my mac to macOS Catalina, while using pipenv. Pipenv creates and manages a virtualenv for you, so the earlier suggestion from #Anoop-Malav is the same, just using pipenv to remove the virtual environment based on the current dir and reset it:
pipenv --rm
pipenv shell # recreate a virtual env with your current Pipfile
I was facing this issue "ModuleNotFoundError: No module named 'encodings" after updating to macOS Catalina.
I was having multiple versions of Python installed in my system.
Removing all the python versions(2.7 and 3.7.4) from macOS system and reinstalling the latest python 3.8 worked for me.
To remove a python from macOS, I've followed the instructions from here How to uninstall Python 2.7 on a Mac OS X 10.6.4?
The above link is for python 2.7 and but you can use the same for 3.7 also.
In my case just changing the permissions of anaconda folder worked:
sudo chmod -R u=rwx,g=rx,o=rx /path/to/anaconda
I got this error when try to launch MySql Workbench 8.0 on my macOS Catalina 10.15.3.
I solved this issue by installing Python 3.7 on my system.
I guess in future, when Workbench will have version greater than 8, it will require newer version of Python. Just look at the library path in the error and you will find required version.
I tried everything, deleted PYTHONPATH and PYTHONHOME - the error didn't go away. I had overlooked that I had specified a python-home path under WSGIDaemonProcess in the wsgi configuration of a VirtualHost. I removed that and the error was gone.
I had a similar issue. I had both anaconda and python installed on my computer and my python dependencies were from the Anaconda directory. When I uninstalled Anaconda, this error started popping. I added PYTHONPATH but it still didn't go.
I checked with python -version and go to know that it was still taking the anaconda path.
I had to manually delete Anaconda3 directory and after that python started taking dependencies from PYTHONPATH.
Issue Solved!
Because this is the first result in google I just want to add the following information for anybody else having problems with jails:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f079b16d740 (most recent call first):
Aborted (core dumped)
When attempting to import python into your jail you both need to link the dependencies and /usr/lib/pythonX.Y to [JAIL]/usr/lib/. Hope this helps.
I was facing same issue on a Debian server and for me the problem was that I've put my project inside the /root/ folder for which mod_wsgi didn't have the rights to access. So I simply moved my project into /var/www/html/ and update my VirtualHost configuration.
In my case, what worked was the following: I changed the env path in the conf from /home/vinzee/.pyenv/versions/envname/bin/ to /home/vinzee/.pyenv/versions/envname/ and it worked.
It seems the environment path needs to be the path to the root folder of the virtual environment (and not the path to ROOT_VENV/bin/).
You can additionally try without the trailing slash as others have suggested, but in my case, it worked both with and without it.
It seems this error can happen for a large variety of reasons (given the number of different answers on here).
For me, the issue was related to the fact that my app was deployed in a virtualenv, and the pyvenv.cfg file had the wrong permissions. It must be writable by the user uwsgi is running as (which in my case was http).
Just go to File -> Settings -> select Project Interpreter under Project tab -> click on the small gear icon -> Add -> System Interpreter -> select the python version you want in the drop down menu
this seemed to work for me
I could also fix this. PYTHONPATH and PYTHONHOME were in cause.
run this in a terminal
touch ~/.bash_profile
open ~/.bash_profile
and then delete all useless parts of this file, and save.
I do not know how recommended it is to do that !
I am trying to make a Nextflow script that utilizes a python script. My python script imports a number of modules but within Nextflow python3 does not find two (cv2 and matplotlib) of 7 modules and crashes. If I call the script directly from bash it works fine. I would like to avoid creating a docker image to run this script.
Error executing process > 'grab_images (1)'
Caused by:
Process `grab_images (1)` terminated with an error exit status (1)
Command executed:
python3 --version
echo 'processing image-1.npy'
python3 /home/hq/cv_proj/k_means2.py image-1.npy
Command exit status:
1
Command output:
Python 3.7.3
processing image-1.npy
Command error:
Traceback (most recent call last):
File "/home/hq/cv_proj/k_means2.py", line 5, in <module>
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'
Work dir:
/home/hq/cv_proj/work/7f/b787c62ec420b2b5eb490603ef913f
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
I think there is a path issue as modules like numpy, sys, re, time are successfully loaded. How can I fix?
Thanks in advance
UPDATE
To assist other who may have problems using python in nextflow scripts... Make sure your shebang is correct. I was using
#!/usr/bin/python
instead of
#!/usr/bin/python3
Since all of my packages were installed with pip3 and I exclusively use python3 you need to have the right shebang.
Best to avoid absolute paths to your script(s) in your process declarations. This section of the docs is worth taking some time to read: https://www.nextflow.io/docs/latest/sharing.html#manage-dependencies, particularly the subsection on how to manage third party scripts:
Any third party script that does not need to be compiled (Bash,
Python, Perl, etc) can be included in the pipeline project repository,
so that they are distributed with it.
Grant the execute permission to these files and copy them into a
folder named bin/ in the root directory of your project repository.
Nextflow will automatically add this folder to the PATH environment
variable, and the scripts will automatically be accessible in your
pipeline without the need to specify an absolute path to invoke them.
Then the problem is how to manage your Python dependencies. You mentioned Docker is not an option. Is Conda also not an option? The config for Conda might look something like:
name: myenv
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- conda-forge::matplotlib-base=3.4.3
- conda-forge::numpy=1.21.2
- conda-forge::opencv=4.5.2
Then if the above is in a file called environment.yml, create the environment with:
conda env create
See also the best practices for using Conda.
can i list the python packages that are actually required for running exiting python program running in linux.
I tried running following commands.
pip3 freeze
pip3 list
To find the modules used by a single python script, you can try to use ModuleFinder:
Create a new python script to analyze the modules your script is using:
New Script:
from modulefinder import ModuleFinder
finder = ModuleFinder()
finder.run_script('MultiProcess.py')
print('Loaded modules:')
for name, mod in finder.modules.items():
print(('%s: ' % name))
print((','.join(list(mod.globalnames.keys())[:3])))
print(('-'*50))
print('Modules not imported:')
print(('\n'.join(iter(finder.badmodules.keys()))))
The output is very verbose and detailed
reference:
https://docs.python.org/2/library/modulefinder.html
I'm still a newbie at bash programming, but trying to run a program with little script. Reducing the problem to the error message, I have
cd /full/path/to/program
python3 -m krop
that is the command working when the actual folder is the /full/path/to/program
but if I run the same from root it doesn't work.
cd /another/path
python3 -m /full/path/to/program/krop
/usr/bin/python3: Error while finding module specification for '/full/path/to/program/krop'
(ModuleNotFoundError: No module named 'krop-0')
I tried lot of variants, but always the same output with errors. I do not have a clue of why the library "python3" adds the "-0" at the end of the name of the file.
What should I put to run the program from root?
python -m command expects a module name, similarly to the syntax you would import in a python program. So if your modules lies in ./directory and directory is a valid python module, you can do python -m directory.krop
You can't however index python modules from file system root. You have either to make your bash script run it in the good directory so you make a local import; or you have to package and install your module system-wide to make a global import that would be invoked with python -m krop from anywhere.
More information on packaging and installing modules: https://packaging.python.org/tutorials/packaging-projects/
Problem solved!,
It was a matter of managing the python import paths, as #hiroprotagonist replied. The list that contains all of directories python will use to search for modules, is available in a variable named sys.path.
So, if somebody wants to run a program (a 'library module as a script', according to python help) through python's command, from a directory different from the 'pwd' one, should write in the command line:
export PYTHONPATH='/full/path/to/program/'
python3 -c "import sys; print(sys.path)"
python3 -m krop
The second line is actually to print on screen, but the first one is the only necessary (export PYTHONPATH).
Thank you for the keywords and help!
Ps. May be should be edited the question title to "problem with a python command to run a program from command line on linux" or something like that.
Reference: python --help :)
I'm working with a Jupyter Notebook written in Python 3, and I would like to run Python 2 scripts from within that Notebook. I was wondering if it's possible to run Shell commands from within the Notebook, and have these Shell commands running under a different environment.
For example, if env2 is a Conda environment that runs Python 2, and env3 runs Python 3, and my Jupyter Notebook runs in env3, maybe I could write within my Notebook:
! source activate env2
! script_that_uses_python2.py
and then continue with python 3 code that is in the Notebook (and uses the output of script_that_uses_python2.py).
I tried it and it didn't work (! conda info --envs showed that env3 is still running). Any advice on how to change the environment in the middle of the Notebook and then go back to the original environment?
As far as I know, you cannot activate another environment and have it work like that. What you can do is run that Python explicitly, something like
!/path/to/anaconda/envs/python2env/bin/python script_that_uses_python2.py
If I run
!/path/to/anaconda/envs/python2env/bin/python -c "import sys; print sys.path"
on my system, it only shows Python 2 directories, so it would probably find the correct imports. However, the variables from that script won't be available in your notebook. You could have the Python 2 file write out a Pickle file and try to read that, maybe...
This worked for me:
! source /home/ubuntu/miniconda/etc/profile.d/conda.sh && conda activate envname && python run.py
Note: It only works if you run all commands in one line connecting them with &&