I have a python file that I need to run. I have made sure to make it executable. I have added a shebang to environment python3 and I have been having no such file or directory error when trying to run this code.
I have tried to convert the line endings to unix format using dos2unix and the error still occurs. The works fine when i run it directly without shebang using python3 file_name.py command.
#!/usr/bin/env python3
import tensorflow as tf
import numpy as np
import roslib
roslib.load_manifest('joint_states_listener')
roslib.load_manifest('spider_control')
import pylab as plt
This python file a custom plugin that has to run with ros. I have made sure to source the catkin environment. Do you think that shebang expression error is caused because I have sourced the terminal to another catkin directory where my ros packages are located ?
I have launched my robot in a terminal using
roslaunch spider_gazebo spider_world.launch
this command says ros to launch the spider_world launch file from spider_gazebo package. This has started all my controllers. Later i opened terminal and I exported my ros environment paths and sourced that terminal to catkin workspace and then used this command
rosrun spider_control control.py
where control.py is my python node and spider_control is the package. the error is
/usr/bin/env: python3.6.7: No such file or directory
Related
I am trying to make a Nextflow script that utilizes a python script. My python script imports a number of modules but within Nextflow python3 does not find two (cv2 and matplotlib) of 7 modules and crashes. If I call the script directly from bash it works fine. I would like to avoid creating a docker image to run this script.
Error executing process > 'grab_images (1)'
Caused by:
Process `grab_images (1)` terminated with an error exit status (1)
Command executed:
python3 --version
echo 'processing image-1.npy'
python3 /home/hq/cv_proj/k_means2.py image-1.npy
Command exit status:
1
Command output:
Python 3.7.3
processing image-1.npy
Command error:
Traceback (most recent call last):
File "/home/hq/cv_proj/k_means2.py", line 5, in <module>
import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'
Work dir:
/home/hq/cv_proj/work/7f/b787c62ec420b2b5eb490603ef913f
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
I think there is a path issue as modules like numpy, sys, re, time are successfully loaded. How can I fix?
Thanks in advance
UPDATE
To assist other who may have problems using python in nextflow scripts... Make sure your shebang is correct. I was using
#!/usr/bin/python
instead of
#!/usr/bin/python3
Since all of my packages were installed with pip3 and I exclusively use python3 you need to have the right shebang.
Best to avoid absolute paths to your script(s) in your process declarations. This section of the docs is worth taking some time to read: https://www.nextflow.io/docs/latest/sharing.html#manage-dependencies, particularly the subsection on how to manage third party scripts:
Any third party script that does not need to be compiled (Bash,
Python, Perl, etc) can be included in the pipeline project repository,
so that they are distributed with it.
Grant the execute permission to these files and copy them into a
folder named bin/ in the root directory of your project repository.
Nextflow will automatically add this folder to the PATH environment
variable, and the scripts will automatically be accessible in your
pipeline without the need to specify an absolute path to invoke them.
Then the problem is how to manage your Python dependencies. You mentioned Docker is not an option. Is Conda also not an option? The config for Conda might look something like:
name: myenv
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- conda-forge::matplotlib-base=3.4.3
- conda-forge::numpy=1.21.2
- conda-forge::opencv=4.5.2
Then if the above is in a file called environment.yml, create the environment with:
conda env create
See also the best practices for using Conda.
I installed the library ase. Since then I can't execute python3 on terminal neither run its python files. I found out with the command which python3 command that with python3 is located at '/usr/local/bin/python3'. The libraries, including ase, are located at '/usr/local/lib/python3.9/site-packages' apparently.
See:
pip3 install --upgrade git+https://gitlab.com/ase/ase.git#master
WARNING: The scripts ase, ase-build, ase-db, ase-gui, ase-info and ase-run are installed in '/usr/local/Cellar/python#3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
This is the error I got when I try to run python3 on terminal:
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Python runtime state: core initialized
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/ase/io/__init__.py", line 1, in <module>
ModuleNotFoundError: No module named 'ase'
I think that something has to be wrong at my .bash_profile file however, I'm not getting the error. Here is my edited .bash_profile:
export PYTHONPATH=/usr/local/lib/python3.9/site-packages/ase:$PYTHONPATH
export PATH=/usr/local/Cellar/python#3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin:$PATH
Hope you can help me, thank you!
Update:
I changed my PYTHONPATH from:
export PYTHONPATH=/usr/local/lib/python3.9/site-packages/ase:$PYTHONPATH
to:
export PYTHONPATH=/usr/local/lib/python3.9/site-packages:$PYTHONPATH
Now I can run python3 on terminal again, however I try to execute ase directly on the terminal it seems that the command is not found.
-bash: ase: command not found
I have already checked out the requires (matplotlib, scipy, numpy) and they are all installed. Seems that everything is wrong with the PATH. I don't know what could be.
The binaries files related to ase package are not anymore into this directory "/usr/local/Cellar/python#3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/bin". I don't know where they are since the installation was successfully done.
According to https://unix.stackexchange.com/questions/240037/why-did-pip-install-a-package-into-local-bin I redefined my paths to:
export PYTHONPATH=/usr/local/lib/python3.9/site-packages:$PYTHONPATH
export PATH=/usr/local/bin:$PATH
and seems to be working, I already ran a program containing that package, however when I enter the folder '/usr/local/bin' I don't find those binary files.
If you can help me again I will be grateful.
I have installed the opensource version of pymol on windows (https://github.com/schrodinger/pymol-open-source).
To run pymol, I can open a python3.8 command prompt on windows and type the following:
import pymol
pymol.finish_launching()
This launches the pymol gui.
I'd now like to create a simple python script so that I can click the script (or convert to a windows executible using for example pyinstaller), however if I put the above commands in a script pymol.py and then run with:
python pymol.py
where python points to python3.8 I get the following error:
AttributeError: partially initialize module 'pymol' has no attribute 'finish_launching' most likely due to a circular import.
Why does this work on the command line python but not in a script and how can I fix this so the script runs and can be converted to a windows executible?
I have 2 python tools that I have to run via the windows cmd line. One is written in python2.7 while the other requires python3.6.
I have installed the newest Anaconda python3.7 version and created two new environments in 'C:\ProgramData\Anaconda3\envs' called 'python27' and 'python36'. For some reason I had to manually install numpy and scipy using conda install -n env_name numpy scipy for each of the new environments.
The reason I have to run both tools using the windows cmd line is that I have integrated them into a workflow environment (RCE by the DLR in case this is relevant), which executes integrated tools in this way. Which means I cannot simply use the Anaconda Prompt instead.
I cannot simply add the python installation to the PATH environment variable because of each tool requiring a different python version (and the file being called 'python.exe' in all versions), so I tried to create aliases for the cmd prompt as suggested by "roryhewitt" in this thread Aliases in Windows command prompt.
my 'python27.bat' file:
#echo off
echo.
C:\ProgramData\Anaconda3\envs\python27\python.exe %*
The problem with this approach is that python encounters an error when trying to import numpy:
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\envs\python27\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\ProgramData\Anaconda3\envs\python27\lib\site-packages\numpy\core\__init__.py", line 71, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect (you're using C:\ProgramData\Anaconda3\envs\python27\python.exe),
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
Does anyone know a better way to run python scripts with a specific python environment via the windows cmd line, or what is causing the import error when I use the alias?
TLDR: I have 2 python tools that require python2.7 and python3.6 respectively. I have to run these tools using the windows cmd line and using aliases to the 'python.exe' file in the specific anaconda environments results in an import error of numpy. Is there a better way to handle two python environments via the cmd line or an easy fix for the import error?
It seems that the ImportError when using python2.7 via the alias in the cmd line was due to the anaconda environment not being activated. This caused the module to be unable to load properly.
I have added the conda functionality to be used in the cmd line as instructed in the post by "Simba" here: Conda command is not recognized on Windows 10. This has fixed the numpy ImportError.
I can now execute the desired tools from the cmd line using the correct python version by activating the correct anaconda environment first, and then calling the alias
Example:
C:\Users\user>conda activate python27
(python27) C:\Users\user>python27 main.py
I want to run a python script in cmd via a batch file. The script fails with dll loading error for numpy imports. The same python script works in pycharm (2019 Anaconda Version) and in the anaconda command prompt without errors.
For all I use the same anaconda enviorment.
For Error Reproduction
Install Anaconda, don't set any path variables
Create an enviorment stored not in the default folder
conda create --prefix
conda activate
conda install numpy flask
6.Code for testing. test-numpy.py
import numpy as np
print(np.abs([1,2-4,6]))
create batch script
run batch script
Running in cmd a python script that has only system imports or no imports works fine.
Setting the pythonpath Variable in Windows is no option.
What I did so far:
Checking if there is an PATH error:
I printed the sys.path, both via Pycharm and the same python script run via CMD gave the same results: it points to my anaconda enviorment with the folders:
...\ (the enviorment itself)
...\python37.zip
...\DLLs
...\lib
...\lib\site-packages
Edit
Tested in Visual Studio Code, gives same Error as running via cmd
Found out Visual Studio Coda until now, can't handle conda enviorments, that where installed with -p or -prefix and are not stored in the default path
Uninstalled and reinstalled numpy
Tested to import another module: Flask also fails with DDL load error but works fine while run in Pycharm.
Tested on another PC with differnt anaconda version.
Another person could reproduce the same error I get.
My guess is my code fails in cmd, becaust anacona enviorments needs somehow to be activated before hand and doesn't work as a stand alone.
The Error Messeag I get running my script in CMD (... points to my anaconda enviormant path):
...\lib\site-packages\numpy\core__init__.py", line 40, in
from . import multiarray ...\lib\site-packages\numpy\core\multiarray.py", line 12, in
from . import overrides ...\lib\site-packages\numpy\core\overrides.py", line 6, in
from numpy.core._multiarray_umath import ( ImportError: DLL load failed: Das angegebene Modul wurde nicht gefunden.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): "TestNumpy.py", line 14, in
import numpy as np ...\lib\site-packages\numpy__init__.py", line 142, in
from . import core ...\lib\site-packages\n umpy\core__init__.py", line 71, in
raise ImportError(msg) ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the multiarray numpy extension module failed. Mostlikely
you are trying to import a failed build of numpy. Here is how to
proceed:
- If you're working with a numpy git repository, try git clean -xdf (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed: your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
1. Check that you are using the Python you expect
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy versions you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: Das angegebene Modul wurde nicht
gefunden.
I found not other solution it seems like it is needed to really activate the conda enviorment before calling the script via conda:
call <file_path>/Anaconda/Scripts/activate.bat <file_path/Anaconda_enviorment> && python <file_path/pythonscript.py>