I am building a build server on AWS as part of my pipeline. I am using a buildspec file, which runs the following commands after my code is copied over to the linux instance, using the image (aws/codebuild/amazonlinux2-x86_64-standard:2.0):
- echo 'Install Conda to run python'
- echo 'Download conda from web'
- wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh
- echo 'Install miniconda'
- sh ~/miniconda.sh -b -p $HOME/miniconda
- eval "$(~/miniconda/bin/conda shell.bash hook)"
- echo 'cd to outbound and list'
- cd outbound
- echo 'create conda environment'
- conda env create -f environment_droplet.yml
- conda env list
- bash
- conda init bash
- echo "conda activate calme" >> ~/.bash_profile
- echo ". ~/miniconda/etc/profile.d/conda.sh" >> ~/.bash_profile
- echo "export PATH=~/miniconda/bin:$PATH" >> ~/.bash_profile
- . ~/.bash_profile
- exec bash
- conda init bash
- conda activate calme
However, the last command 'conda activate is not successful and results in the error:
[Container] 2020/08/30 20:38:14 Running command conda init bash
no change /root/miniconda/condabin/conda
no change /root/miniconda/bin/conda
no change /root/miniconda/bin/conda-env
no change /root/miniconda/bin/activate
no change /root/miniconda/bin/deactivate
no change /root/miniconda/etc/profile.d/conda.sh
no change /root/miniconda/etc/fish/conf.d/conda.fish
no change /root/miniconda/shell/condabin/Conda.psm1
no change /root/miniconda/shell/condabin/conda-hook.ps1
no change /root/miniconda/lib/python3.8/site-packages/xontrib/conda.xsh
no change /root/miniconda/etc/profile.d/conda.csh
no change /root/.bashrc
No action taken.
[Container] 2020/08/30 20:38:14 Running command conda activate calme
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
[Container] 2020/08/30 20:38:14 Command did not exit successfully conda activate calme exit status 1
[Container] 2020/08/30 20:38:14 Phase complete: PRE_BUILD State: FAILED
[Container] 2020/08/30 20:38:14 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: conda activate calme. Reason: exit status 1
As you can see, from my buildspec snippet, I am reloading the .bash_profile, applying conda init bash and creating a new bash instance too. I believed all the above commands would help reset the existing terminal, so conda activate is a recognised command - but that has not been the case.
I do not have this problem when creating a standalone EC2 instance, as I can restart the terminal / close the connection and reconnect. However, I need to be able to activate conda via the buildspec and the ec2 instance created for the build of the project. Any help is appreciated
Related
I am trying to install go in a docker image with the below commands:
- cd /usr/local
- wget -L "https://golang.org/dl/go1.20.1.linux-amd64.tar.gz"
- tar -xzvf go1.20.1.linux-amd64.tar.gz
- rm -f go1.20.1.linux-amd64.tar.gz
- echo "export PATH=$PATH:/usr/local/go/bin" >> /etc/profile
- source /etc/profile
- /usr/local/go/bin version
- /usr/local/go/bin install cuelang.org/go/cmd/cue#latest
These commands are passed in a .gitlab-ci.yml in one of the stages and it throws an error at the end by saying go not found when I try to check the version.
What am I missing here? my ultimate goal is to install CUE but I want to install it via golang.
I would like to ssh into a remote server and change the conda environment - conda won't let me do that.
Either the conda command is not found and if I explicitely give the path to conda or add the path manually, it wants me too re-initialize the shell with conda init bash. Re-initialization of course does not change the error message. What can I do to use conda via ssh.
(Doing the steps by hand does not lead to an error).
sshpass -p 'passwort' ssh root#999.999.999.999 "export PATH='/path/to/anaconda3/bin:$PATH'; conda activate main"
# yields:
#CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
#To initialize your shell, run $ conda init <SHELL_NAME>
#Although it is working very well when manually ssh-ing to the remote server
sshpass -p 'passwort' ssh root#999.999.999.999 "/path/to/anaconda3/bin/conda activate main"
# Same error
sshpass -p 'passwort' ssh root#999.999.999.999 "conda activate main"
# bash: conda: command not found
There is a similiar question
python - Conda command not found - Stack Overflow
but it does not cover the problem of re-initialization
I recently got a new Mac with M1 processor. From the terminal I installed miniconda through home-brew. Then, I tried to create a new conda environment as usual using conda create --name my_env. Then, I try to activate the environment using conda activate my_env, but I get the following error:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
I obey of course, and I run conda init bash and close and restart my shell. However, when I then try to activate my environment I receive the above error again (and rerunning conda init bash does not fix the problem because it simply says 'no action taken'.
Questions:
Does anyone know why I still can't run conda activate?
Is it smart to even use miniconda for Python without Rosetta?
Many thanks
Mac changed the shell from Bash to Zsh a couple years ago. So, you need to do
conda init zsh
I'm trying to write a bash script that includes deactivating and removing a conda environment. Here's an example, remove_env.sh:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Deactivate environment
conda deactivate
# Remove environment
conda remove --name my_env --all --yes
The environment must be deactivated in order to remove it.
Unfortunately, this doesn't work. I perform this in the terminal:
$ conda activate my_env
$ ./remove_env.sh
CondaEnvironmentError: cannot remove current environment. deactivate and run conda remove again
I think the issue has to do with forking - essentially, the environment gets deactivated in one process, but then the remove call is run in another process, which doesn't have the environment deactivated. But I'm not entirely sure.
Some notes:
I can't use source remove_env.sh - I must be able to use ./remove_env.sh
I've tried this with no success:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Deactivate and remove environment
conda deactivate && conda remove --name my_env --all --yes
I call the command conda activate my_env in my ~./bashrc
I can't use aliases - it must be a bash script
Thanks Jonathan for the answer in the comments. You're totally right, I completely overlooked that blue note in the conda manual. I was able to do this:
#!/bin/bash
# Get the conda command available in bash
eval "$(conda shell.bash hook)"
# Activate the environment
conda activate my_env
# Deactivate environment
conda deactivate
# Remove environment
conda remove --name my_env --all --yes
I think it works whether you conda activate with or without arguments.
Alternatively, using Conda's run tool, one could avoid manual activation. That is, something like
#!/usr/bin/env conda run bash
conda env remove -n my_env -y
When I try to install nvm and test that it's installed in my vagrant shell provisioning script, using:
sudo wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.31.3/install.sh | bash
source ~/.bashrc
nvm
I get:
==> default: => Downloading nvm as script to '/root/.nvm'
==> default: => Appending source string to /root/.bashrc
==> default: => You can now install Node.js by running `nvm install`
==> default: /tmp/vagrant-shell: line 7: nvm: command not found
First I don't want it to install as root, I want to install it as the vagrant user in /home/vagrant to avoid permission issues.
Second why is the command not found when I ran source .bashrc as specified in the nvm installation instructions here https://github.com/creationix/nvm?
I have a solution which I'm adding now.
So first if you want to install somethnig as the vagrant user during a bootstrap.sh script, the easiest way to do that is to add another provisioner script to your Vagrantfile which is run in unprivileged mode:
config.vm.provision "ScriptRunAsRoot", type:"shell", path: "Vagrantdata/rootUserBootstrap.sh"
config.vm.provision "ScriptRunAsVagrantUser", privileged: false, type:"shell", path: "Vagrantdata/vagrantUserBootstrap.sh"
Then to get the nvm command in this situation, you have to be sure to run the nvm.sh script directly, ie not via the .bashrc file as stated in the nvm installation instructions. This is because .bashrc has a line which exits early if it is being run in a non interactive shell, which I guess vagrant is. From .bashrc:
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
So instead of the source ~/.bashrc, you need:
source ~/.nvm/nvm.sh