Why does `exec bash` not work in a CI pipeline? - python-3.x

Question

I have written a github workflow file. I want to run a python program in github actions to validate few changes. I have one environment.yml file which contains all conda environment dependencies required by this program. The thing is, actual program is not running at all, and my workflow is completed with success.
Following is jobs section of workflow.yml file
jobs:
build-linux:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout#v2
with:
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- name: Set up Python 3.8
uses: actions/setup-python#v2
with:
python-version: 3.8
- name: Cache conda
uses: actions/cache#v2
env:
# Increase this value to reset cache if etc/example-environment.yml has not changed
CACHE_NUMBER: 0
with:
path: ~/conda_pkgs_dir
key:
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{hashFiles('**/environment.yml') }}
- uses: conda-incubator/setup-miniconda#v2
with:
activate-environment: test-env
environment-file: environment.yml
use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
- name: Test
run: |
export PATH="./:$PATH"
conda init bash
exec bash
conda activate test-env
echo "Conda prefix: $CONDA_PREFIX"
python test.py
shell: bash
I also tried removing shell:bash in the last step, but this is also giving me the same result.
The logs in last step looks like this:
Run export PATH="./:$PATH"
export PATH="./:$PATH"
conda init bash
exec bash
conda activate test-env
echo "Conda prefix: $CONDA_PREFIX"
python test.py
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.8.11/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.11/x64/lib
CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir
no change /usr/share/miniconda/condabin/conda
no change /usr/share/miniconda/bin/conda
no change /usr/share/miniconda/bin/conda-env
no change /usr/share/miniconda/bin/activate
no change /usr/share/miniconda/bin/deactivate
no change /usr/share/miniconda/etc/profile.d/conda.sh
no change /usr/share/miniconda/etc/fish/conf.d/conda.fish
no change /usr/share/miniconda/shell/condabin/Conda.psm1
no change /usr/share/miniconda/shell/condabin/conda-hook.ps1
no change /usr/share/miniconda/lib/python3.9/site-packages/xontrib/conda.xsh
no change /usr/share/miniconda/etc/profile.d/conda.csh
modified /home/runner/.bashrc
==> For changes to take effect, close and re-open your current shell. <==
As we can clearly see, the line echo "Conda prefix: $CONDA_PREFIX" is not getting executed at all, and the workflow terminates with success. We should expect it to either run or fail the job, but nothing happens. The workflow simply ignores these commands and marks the workflow as success.

Answer 1

Your CI script contains the line:
exec bash
When this line is executed, the shell process is replaced with a new one, and the new shell process has no idea it should continue executing the script the previous process was: all the execution state is lost. GitHub Actions passes the script to execute as a command-line argument to the initial shell process and sets standard input to /dev/null; as the new shell process is started with an empty command line and an empty file on standard input, it simply exits immediately. The fact that this works well with an interactive shell is something of a lucky coincidence.
The reason the installer directs you to restart your shell is to apply the environment variable changes added to the shell’s initialisation file. As such, it should probably be enough to replace the exec bash line with
source "$HOME/.bashrc"
However, even with this line the environment modifications will not be applied in subsequent steps, as the documentation of the setup-miniconda action warns:
Bash shells do not use ~/.profile or ~/.bashrc so these shells need to be
explicitely declared as shell: bash -l {0} on steps that need to be properly
activated (or use a default shell). This is because bash shells are executed
with bash --noprofile --norc -eo pipefail {0} thus ignoring updated on bash
profile files made by conda init bash. See
Github Actions Documentation
and
thread.
Based on this advice, I think the best course of action is to end the actions step at the point where you would put exec bash, and apply the shell: setting to all further steps (or at least those which actually need it):
- name: Set up Conda environment
run: |
export PATH="./:$PATH"
conda init bash
- name: Perform Conda tests
shell: bash -l {0}
run: |
export PATH="./:$PATH"
conda activate test-env
echo "Conda prefix: $CONDA_PREFIX"
python test.py

Answer 2

As #user3840170 mentioned, bash shells do not use ~/.profile or ~/.bashrc. Then, a way to make it work would be to run what conda initialization would run. On GitHub Actions the path to the conda installation is on the variable $CONDA, so you can use it to run the initialization on each step needing conda activate. The following code worked for me on GitHub Actions (the one provided above didn't work in my case).
- name: Set up Conda environment
run: |
echo "${HOME}/$CONDA/bin" >> $GITHUB_PATH
conda init --all --dry-run
- name: On the steps you want to use
shell: bash
run: |
source $CONDA/etc/profile.d/conda.sh
conda activate test-env
python test.py