I would like to ssh into a remote server and change the conda environment - conda won't let me do that.
Either the conda command is not found and if I explicitely give the path to conda or add the path manually, it wants me too re-initialize the shell with conda init bash. Re-initialization of course does not change the error message. What can I do to use conda via ssh.
(Doing the steps by hand does not lead to an error).
sshpass -p 'passwort' ssh root#999.999.999.999 "export PATH='/path/to/anaconda3/bin:$PATH'; conda activate main"
# yields:
#CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
#To initialize your shell, run $ conda init <SHELL_NAME>
#Although it is working very well when manually ssh-ing to the remote server
sshpass -p 'passwort' ssh root#999.999.999.999 "/path/to/anaconda3/bin/conda activate main"
# Same error
sshpass -p 'passwort' ssh root#999.999.999.999 "conda activate main"
# bash: conda: command not found
There is a similiar question
python - Conda command not found - Stack Overflow
but it does not cover the problem of re-initialization
Related
I recently got a new Mac with M1 processor. From the terminal I installed miniconda through home-brew. Then, I tried to create a new conda environment as usual using conda create --name my_env. Then, I try to activate the environment using conda activate my_env, but I get the following error:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
I obey of course, and I run conda init bash and close and restart my shell. However, when I then try to activate my environment I receive the above error again (and rerunning conda init bash does not fix the problem because it simply says 'no action taken'.
Questions:
Does anyone know why I still can't run conda activate?
Is it smart to even use miniconda for Python without Rosetta?
Many thanks
Mac changed the shell from Bash to Zsh a couple years ago. So, you need to do
conda init zsh
I want to create a bash script to install a new virtual environment "ABC" in conda. But before I go ahead and run a command to create this env, I want to check if conda is already installed on the machine. If not installed, I want to install miniconda and then create the env "ABC". If conda is already installed then I would just go ahead and create the environment. (All this should happen within the same script)
I just want to know if it is possible to check the existence of conda within a bash script and then proceed with the rest of the installations?
'''
#!/bin/bash
<code_to_check_existence_of_conda_env_here ?>
//If it does not exist, I will run the below code
mkdir -p miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O
miniconda3/miniconda.sh
bash miniconda3/miniconda.sh -b -u -p ~/miniconda3
conda env update -f my_env.yml
'''
The main issue here is that Conda has multiple components. Which components are loaded depends on the how Conda is installed and what user the BASH script is executing under. I'll try describing the components and hopefully you can decide what is suitable to verify.
Conda Components
1: Conda Package
The first is a Python package conda, which is installed in the Anaconda/Miniconda/Miniforge base environment. If the base environment is activated, one could test
python -m conda
which will give something like
/path/to/python: No module named conda
if it isn't there. Otherwise, it outputs the conda entrypoint's documentation.
2: Conda Entrypoint
The entrypoint conda, which acts as a CLI, is located under the condabin directory of the Anaconda/Miniconda/Miniforge installation. When a user runs conda init, a managed section is added to their shell initialization file (.bashrc for Linux BASH), that includes code to add the condabin to PATH. This is most likely what OP wants to identify, however, running with the shebang /bin/bash will not load the .bashrc file. Instead, one should probably be using
#!/bin/bash -l
or
#!/usr/bin/env bash -l
Then the entrypoint can be located with
which conda
3: Conda Activate
Finally, Conda also includes some shell-only functions, which are defined in the aforementioned shell initialization code. This sets up a middleman shell function, also called conda (essentially an alias), which can be viewed with
type conda
This function serves to determine whether the conda (de)?activate commands are being requested, which are pure shell functions, or something that needs to be forwarded to the entrypoint.
Recommendation
Were I designing this, I would write an interactive script that checks for #2 (which conda) and if that comes up blank then prompt the user to either provide the PATH to the Conda entrypoint (maybe they installed it in a weird place or didn't run conda init) or install Miniforge.1
I also would not use the base environment to install stuff - that is a bad idea for an end-user, let alone a third-party - and instead create a new environment. I would prompt the user with a specific default environment name, but also provide an option for them to customize.
[1] Yes, Miniforge, not Miniconda. Commercial use of the Anaconda defaults channels now requires a paid license, so better to use the free Miniforge.
I'm not really good at bash scripting but I would execute a command:
pip3 freeze | grep conda
and based on the output (if exit status was an error or not) install it or directly create the environment.
run the command conda list on your machine, the output will of be different when its installed vs not installed, then run if statement on the output.
In bash, if we want to check if a software is installed or not, the following check is used.
if dpkg -l $SOFTWARE; then
<Do stuff>
ensureconda
This sounds like the intended problem that the experimental tool ensureconda was designed to solve. This, however, requires a Python installation with pip:
pip install ensureconda
Here are the command options:
$ ensureconda --help
Usage: ensureconda [OPTIONS]
Ensures that a conda/mamba is installed.
Options:
--mamba / --no-mamba search for mamba
--micromamba / --no-micromamba search for micromamba, install if not
present
--conda / --no-conda search for conda
--conda-exe / --no-conda-exe search for conda.exe / conda-standalone,
install if not present
--no-install don't install conda/mamba if no version can
be discovered
--min-conda-version VERSIONNUMBER
minimum version of conda to accept (defaults
to 4.8.2)
--min-mamba-version VERSIONNUMBER
minimum version of mamba/micromamba to
accept (defaults to 0.7.3)
--help Show this message and exit.
Type conda activate
or conda activate env_name
The first command will activate the base environment if it is already installed. The second command will do activate the user created environment if it is installed.
I want to install miniconda in databricks environment
I run the following code:
%sh
/dbfs/FileStore/Miniconda3_latest_Linux_x86_64.sh
But I can't interact with command line, when it asks me to press Enter.
How do I pass Enter in databricks notebook ?
You can try installing Miniconda in silent mode, which will not prompt you to press any keys to accept the license, etc.
%sh
/dbfs/FileStore/Miniconda3_latest_Linux_x86_64.sh -b
You'll need to follow the other instructions in the link in order to set up the conda environment properly. Maybe something like
%sh
eval "$(/path/to/miniconda/bin/conda shell.bash hook)"
conda init
I am building a build server on AWS as part of my pipeline. I am using a buildspec file, which runs the following commands after my code is copied over to the linux instance, using the image (aws/codebuild/amazonlinux2-x86_64-standard:2.0):
- echo 'Install Conda to run python'
- echo 'Download conda from web'
- wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
- echo 'Install miniconda'
- sh ~/miniconda.sh -b -p $HOME/miniconda
- eval "$(~/miniconda/bin/conda shell.bash hook)"
- echo 'cd to outbound and list'
- cd outbound
- echo 'create conda environment'
- conda env create -f environment_droplet.yml
- conda env list
- bash
- conda init bash
- echo "conda activate calme" >> ~/.bash_profile
- echo ". ~/miniconda/etc/profile.d/conda.sh" >> ~/.bash_profile
- echo "export PATH=~/miniconda/bin:$PATH" >> ~/.bash_profile
- . ~/.bash_profile
- exec bash
- conda init bash
- conda activate calme
However, the last command 'conda activate is not successful and results in the error:
[Container] 2020/08/30 20:38:14 Running command conda init bash
no change /root/miniconda/condabin/conda
no change /root/miniconda/bin/conda
no change /root/miniconda/bin/conda-env
no change /root/miniconda/bin/activate
no change /root/miniconda/bin/deactivate
no change /root/miniconda/etc/profile.d/conda.sh
no change /root/miniconda/etc/fish/conf.d/conda.fish
no change /root/miniconda/shell/condabin/Conda.psm1
no change /root/miniconda/shell/condabin/conda-hook.ps1
no change /root/miniconda/lib/python3.8/site-packages/xontrib/conda.xsh
no change /root/miniconda/etc/profile.d/conda.csh
no change /root/.bashrc
No action taken.
[Container] 2020/08/30 20:38:14 Running command conda activate calme
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
[Container] 2020/08/30 20:38:14 Command did not exit successfully conda activate calme exit status 1
[Container] 2020/08/30 20:38:14 Phase complete: PRE_BUILD State: FAILED
[Container] 2020/08/30 20:38:14 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: conda activate calme. Reason: exit status 1
As you can see, from my buildspec snippet, I am reloading the .bash_profile, applying conda init bash and creating a new bash instance too. I believed all the above commands would help reset the existing terminal, so conda activate is a recognised command - but that has not been the case.
I do not have this problem when creating a standalone EC2 instance, as I can restart the terminal / close the connection and reconnect. However, I need to be able to activate conda via the buildspec and the ec2 instance created for the build of the project. Any help is appreciated
I'm trying to write a bash script that includes deactivating and removing a conda environment. Here's an example, remove_env.sh:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Deactivate environment
conda deactivate
# Remove environment
conda remove --name my_env --all --yes
The environment must be deactivated in order to remove it.
Unfortunately, this doesn't work. I perform this in the terminal:
$ conda activate my_env
$ ./remove_env.sh
CondaEnvironmentError: cannot remove current environment. deactivate and run conda remove again
I think the issue has to do with forking - essentially, the environment gets deactivated in one process, but then the remove call is run in another process, which doesn't have the environment deactivated. But I'm not entirely sure.
Some notes:
I can't use source remove_env.sh - I must be able to use ./remove_env.sh
I've tried this with no success:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Deactivate and remove environment
conda deactivate && conda remove --name my_env --all --yes
I call the command conda activate my_env in my ~./bashrc
I can't use aliases - it must be a bash script
Thanks Jonathan for the answer in the comments. You're totally right, I completely overlooked that blue note in the conda manual. I was able to do this:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Activate the environment
conda activate my_env
# Deactivate environment
conda deactivate
# Remove environment
conda remove --name my_env --all --yes
I think it works whether you conda activate with or without arguments.
Alternatively, using Conda's run tool, one could avoid manual activation. That is, something like
#!/usr/bin/env conda run bash
conda env remove -n my_env -y