I have python 3.7.10, conda 4.12.0, and Spyder 5.0.5 installed on Windows 10, and I can see the list of environments with conda env list.
However, when I run conda activate <environment>, CommandNotFoundError appears:
In[1]: conda activate <env>
Note: you may need to restart the kernel to use updated packages.
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
If using 'conda activate' from a batch script, change your
invocation to 'CALL conda.bat activate'.
[...]
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
I like to change the environment without running Anaconda navigator (or close Spyder every time I change the environment).
Is there a command that I can run on Spyder console to change the virtual environment? Is it also possible to run different environments on different Spyder consoles?
To change environments you will need to change the interpreter preference and restart the console or create a new one. To change the interpreter preference you can go to Tools > Preferences > Python Interprer > Use the following Python interpreter and select there the path for the python.exe of the env you want to use.
For more info regarding how to use existing enviroments with Spyder: https://docs.spyder-ide.org/current/faq.html#using-existing-environment
For some reason the jupyter notebooks on my VM are in the wrong environment (ie stuck in (base)). Furthermore, I can change the environment in the terminal but not in the notebook. Here is what happens when I attempt !conda activate desired_env in the notebook:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
# conda environments:
#
base * /anaconda
azureml_py36 /anaconda/envs/azureml_py36
azureml_py38 /anaconda/envs/azureml_py38
azureml_py38_pytorch /anaconda/envs/azureml_py38_pytorch
azureml_py38_tensorflow /anaconda/envs/azureml_py38_tensorflow
I tried the answers here (e.g., first running !source /anaconda/etc/profile.d/conda.sh).
I also tried activating the environment using source rather than 'conda activate': !source /anaconda/envs/desired_env/bin/activate. This runs but doesn't actually do anything when I see the current environment in conda env list
Edit: also adding that if I install a package in the (base) environment in the terminal, I still don't have access to it in jupyter notebook.
I'm the PM that released AzureML Notebooks, you can't activate a Conda env from a cell, you have to create a new kernel will the Conda Env. Here are the instructions: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-access-terminal#add-new-kernels
This question already has answers here:
Activate conda environment using subprocess
(1 answer)
Conda command working in command prompt but not in bash script
(4 answers)
Closed 1 year ago.
I want to run the following bash script from inside a python script.
condo activate bne
CUDA_VISIBLE_DEVICES=1 python bne.py --model models/BNE_SGsc --fe ./embeddings/BioEmb/Emb_SGsc.txt --fi names.txt --fo output_BNE_SGsc.txt
My python script/function to do this is as follows -
def do_tensorflow_routine(path_name_file):
os.chdir(os.path.join("../","bne_resources/"))
os.system("conda activate bne")
bne_command = "python bne.py --model models/BNE_SGsc --fe ./embeddings/BioEmb/Emb_SGsc.txt --fi names.txt --fo names_bne_SGsc.txt"
subprocess.run(bne_command,shell=True)
pass
However, I am getting the following error -
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
I am using a cluster and my system specifications are as follows -
Static hostname: lab1
Icon name: computer-server
Chassis: server
Machine ID: 104265a0ea5b48c1a3c5a9802294af66
Boot ID: 4277780744ae448292d66a9ff39c76e2
Operating System: Ubuntu 20.04.1 LTS
Kernel: Linux 5.8.0-36-generic
Architecture: x86-64
Please let me know if I am missing something. My basic question is how to run another python script from a given python script, especially when we need to activate and deactivate virtual environments, since the parent python script and child python script require 2 different virtual environments due to some dependency conflicts.
Kindly help me.
Thanks in advance,
Megh
If the terminal does not show (base) after running conda activate bne, then try running:
conda init
In the bash script at the top you need the shebang line #!/usr/bin/env bash and env is used for portability for different distributions of Linux.
I'm trying to deploy a flask application to an ec2 instance using (1) nginx (2) gunicorn, (3) git, and (4) supervisor. I've set up nginx, git, gunicorn, but I've having trouble writing the supervisor script.
I'm unable to get supervisor to launch gunicorn within the context of the virtualenv.
When I run gunicorn run:app outside of the virtualenv it returns
ImportError: No module named flask
When I run the same command within the virtualenv it works just fine.
When I run the same command outside of the virtualenv but specify the gunicorn in the virtual env (i.e /var/www/sitename/env/bin/gunicorn run:app) it works just fine again.
That's a problem that I couldn't figure out, but I figured if I could just have supervisor run gunicorn inside the virtualenv it wouldn't be a problem, but I'm not able to do that either.
I've tried adding two programs in the supervisor script, one to launch the virtual environment and the other for gunicorn, added the two commands together using quotes which one similar SO answer suggested, using && to combine activating the virtualenv and launching gunicorn, declaring an environment=PATH= variable, and a number of other options; I just can't get it to work.
I have no idea what I'm doing wrong or what else to try; does anyone know what I can do at this point?
I'm running python3 - I read that supervisor is limited to v2 but someone else mentioned in an answer that it's just a task manager and it shouldn't matter.
You were on the right track, but it's simpler than you were making it. To run a Flask app named run with entry point app via gunicorn under supervisor with the path you gave:
/etc/supervisor/conf.d/run.conf
[program:run]
command = /var/www/sitename/env/bin/gunicorn run:app -b localhost:8000
directory = /var/www/sitename
user = siteuser
You can provide the environment argument to set stuff like production mode or whatever, but this is all you need to have the virtual environment version of gunicorn, running python 3 if it's a python 3 venv, run your flask app in the same virtual environment.
I'm trying to set up a recurring Python task through windows task scheduler.
I have had success when I input the path to 'python.exe' and provide the script's path as a parameter to windows task scheduler (see screenshot below)
However, I want to be able to choose a particular virtual environment in which to run the script. I don't have much knowledge of venv, and I typically use it by opening cmd and running Scripts\activate.bat in the desired virtual environment directory.
How can I accomplish 'run task x in venvxxx every 24 hours' using windows task scheduler?
Create batch file with these commands:
c:\__full_path_to_virtualenv__\Scripts\activate.bat && python __full_path_to_python_script__.py
&& means run command2 if command1 completed successfully.
Then set that batch file as script to run. You don't need to set any additional arguments in task scheduler (or you can set them in batch file anyway) and can set Start in if script has to read/write from specific directory and uses relative paths.
Though the answer by mx0 above seems to work, I have set up Task Scheduler to run a flask web app on bootup. In this case, manual starting works fine, but manual ending does not. Ending the task kills the cmd.exe task that sets up the virtual environment, but the python.exe continues to run.
The solution that I found worked was from this reddit post which skips the virtual environment activation to call the python executable directly:
path\to\venv\Scripts\python.exe path\to\script.py
I'm not sure how robust this will be, but at least this way ending the task will end the python.exe
This is more verbose but very easy to understand, and - I found the most important - much easier than using Windows Task Scheduler settings when you have lots of scripts. To create another you just copy the .bat file and change one line.
Save this as a .bat file and point to it under Actions > Start a Program > Program/Script:, with no arguments or "Start in" necessary.
set original_dir=%CD%
set venv_root_dir="C:\Python-Venvs\env-name"
cd %venv_root_dir%
call %venv_root_dir%\Scripts\activate.bat
python your_script.py <arg1> <arg2>
call %venv_root_dir%\Scripts\deactivate.bat
cd %original_dir%
exit /B 1
For an installed command-line program, you can replace python your_script.py <arg1> <arg2> ... with <program name> <arg1> <arg2> ....
In addition it's simple to add another script on the following line, rather than attempting to parse sequential scripts into a one-liner for Task Scheduler.
I tried with mx0's answer and it works fine as long as your script does not take too long to finish.
I use a different approach in the task scheduler instead using batch files:
In "Program/script" textbox you set the path to Python executable (in my case is inside the virtualenv folder).
"Add arguments" => Just the name of your Python script (name.ppy).
"Start in" => The full path of your Python script (without the name.py).
This way the script runs and wait until the end.
My solution is almost identical to mx0, but I've added an extra step to ensure environment parameters each time. Edit the path/to/app for the app_path variable.
It may be a little redundant to check the environment setup every time, but I like ensuring my environment is set.
Simply schedule the execute_app.bat file or run in the cmd prompt. Deactivate command is not needed unless running from an Anaconda prompt. If you use a full path for path/to/app this file can be executed from any directory. I also have a Linux solution using execute_app.sh file below from a terminal.
This answer has been edited to simplify, and to use variables to make this easier to adapt to new projects.
App structure:
app/bin/app.py
app/bin/execute_app.bat
app/env/requirements.txt
# execute_app.bat file
# windows solution
SETLOCAL
set app_path=path/to/app
set env_path=%app_path%/env
set activ=%env_path%/Scripts/activate.bat
set req=%env_path%/requirements.txt
set app=%app_path%/bin/app.py
py -m venv %env_path% && %activ% && python -m pip install --upgrade pip && pip install -r %req% && python %app%
ENDLOCAL
#!/bin/bash
# execute_app.sh file
# Linux solution
app_path='path/to/app'
env_path=$app_path'/env'
activ=$env_path'/bin/activate'
req=$env_path'/requirements.txt'
app=$app_path'/bin/app.py'
python3 -m venv $env_path &&
. $activ &&
python3 -m pip install --upgrade pip &&
pip install -r $req &&
python $app &&
deactivate
The selected answer for this question is not correct. If you review the comments, you'll see the problem.
My answer builds off of #Nick P's answer (the #2 answer currently). His batch file will work, but you'll want to change the exit code from 1 to 0 if you want Windows Task Scheduler to report the task completed successfully. Also, simply calling the .bat file on the "Program/Script" line will not work. Instead, you need to put the name of your shell as the "Program/Script" to run (for instance, cmd), then put "/c name-of-batch-file.bat" goes in the "Add arguments (optional):" field. And finally, put the path to the batch file (minus the file name) in the "Start in (optional):" field.
It should look something like this:
REM Windows batch script to run 1+ Python program/scripts, sequentially, within
REM their virtual environment. This can be called from Windows Task Scheduler.
set original_dir=%CD%
set venv_root_dir="C:\Users\myUsername\myProjects\nameOfProject"
cd %venv_root_dir%
call %venv_root_dir%\Scripts\activate.bat
python nameOfPythonProgram.py
call %venv_root_dir%\Scripts\deactivate.bat
cd %original_dir%
exit /B 1
Copied this from nmpowell on github and it works fine. Run from task scheduler
Solution
Patching the path is all necessary.
The quick way would be a bat script that source environment activation script at the beginning
#call PATH_TO_MY_VENV/bin/activate.bat
python app.py
Proceeding Problem
Later you will realize that the python job starts fine, but won't stop when Windows Scheduler stop it.
When taskengine.exe decides to stop the job, the intermediate cmd.exe (bat script) process is killed and the Python.exe will be left straw. since the cmd.exe(bat script) won't signal python.exe to stop on exit.
Final Solution
Let Windows Task Scheduler or taskengine.exe launch python.exe directly without a middle-man script.
Previous answers launch python.exe directly with py script, this works for simple modules, but not for some binary module in conda environment.
For binary modules in conda to work, you can create a utility module named e.g. patch_conda_path to patch PATH variable in os.environ based on sys.base_exec_prefix. Copy the patching work that activate.bat does, just in python.
below example has been tested for conda virtual environment:
import is, sys
conda_base = sys.base_exec_prefix
subps = [";", "library\\mingw-w64\\bin;", "library\\usr\\bin;", "library\\bin;", "scripts;", "bin;", "condabin;"]
conda_paths = ""
for p in subps :
_p = os.path.join(conda_base, p)
if _p in os.environ["PATH"]:
continue
else:
conda_paths += _p
os.environ["PATH'"] = conda_paths + os.environ["PATH"]
Import this module at the beginning of your main script.
import patch_conda_path
... original main script ...
In task scheduler set
program to ... conda environment path...\python.exe,
arguments to your py script file name and
start in to your py script folder.