Gitlab CI PIP AWSCLI Avoid Install Again - gitlab

Question

I'm working with GitLab CE, Runners Docker and AWS ECS for Deployment.
We created a script that do what we need but we separate the stages and jobs for Development.
The problem is that we need to run this scripts to connect to AWS and let us to register containers and deploy our resources:
services:
- docker:dind
before_script:
- apk add build-base python3-dev python3 libffi-dev libressl-dev bash git gettext curl
- apk add py3-pip
- pip install six awscli awsebcli
- $(aws ecr get-login --no-include-email --region "${AWS_DEFAULT_REGION}")
- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"
The problem is that the script runs every time with the Job, this will not be a problem is the scripts avoid the reinstallation of the dependencies:
So we need to know if we can avoid this behavior in order to run the script only once becase every job take so long on finish for this.
Our complete script .gitlab-ci.yml:
image: docker:latest
stages:
- build
- tag
- push
- ecs
- deploy
variables:
REPOSITORY_URL: OUR_REPO
REGION: OUR_REGION
TASK_DEFINITION_NAME: task
CLUSTER_NAME: default
SERVICE_NAME: service
services:
- docker:dind
before_script:
- apk add build-base python3-dev python3 libffi-dev libressl-dev bash git gettext curl
- apk add py3-pip
- pip install six awscli awsebcli
- $(aws ecr get-login --no-include-email --region "${AWS_DEFAULT_REGION}")
- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"
build:
stage: build
tags:
- deployment
script:
- docker build -t $REPOSITORY_URL:latest .
only:
- refactor/ca
tag_image:
stage: tag
tags:
- deployment
script:
- docker tag $REPOSITORY_URL:latest $REPOSITORY_URL:$IMAGE_TAG
only:
- refactor/ca
push_image:
stage: push
tags:
- deployment
script:
- docker push $REPOSITORY_URL:$IMAGE_TAG
only:
- refactor/ca
task_definition:
stage: ecs
tags:
- deployment
script:
- TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition "$TASK_DEFINITION_NAME" --region "${REGION}")
- NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | python3 $CI_PROJECT_DIR/update_task_definition_image.py $REPOSITORY_URL:$IMAGE_TAG)
- echo "Registering new container definition..."
- aws ecs register-task-definition --region "${REGION}" --family "${TASK_DEFINITION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"
only:
- refactor/ca
register_definition:
stage: ecs
tags:
- deployment
script:
- aws ecs register-task-definition --region "${REGION}" --family "${TASK_DEFINITION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"
only:
- refactor/ca
deployment:
stage: deploy
tags:
- deployment
script:
- aws ecs update-service --region "${REGION}" --cluster "${CLUSTER_NAME}" --service "${SERVICE_NAME}" --task-definition "${TASK_DEFINITION_NAME}"
only:
- refactor/ca

Answer 1

One workaround will be to define before_script only for the stage that runs aws ecs register-task-definition and aws ecs update-service.
Additionally, as push and ecs stages access the IMAGE_TAG variable it is convenient to store it on an build.env artifact so that this stages can access it by specifiying a dependency on the image_tag stage.
image: docker:latest
stages:
- build
- tag
- push
- ecs
variables:
REPOSITORY_URL: OUR_REPO
REGION: OUR_REGION
TASK_DEFINITION_NAME: task
CLUSTER_NAME: default
SERVICE_NAME: service
services:
- docker:dind
build:
stage: build
tags:
- deployment
script:
- docker build -t $REPOSITORY_URL:latest .
only:
- refactor/ca
tag_image:
stage: tag
tags:
- deployment
script:
- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"
- echo "IMAGE_TAG=$IMAGE_TAG" >> build.env
- docker tag $REPOSITORY_URL:latest $REPOSITORY_URL:$IMAGE_TAG
only:
- refactor/ca
artifacts:
reports:
dotenv: build.env
push_image:
stage: push
tags:
- deployment
script:
- docker push $REPOSITORY_URL:$IMAGE_TAG
only:
- refactor/ca
dependencies:
- tag_image
task_definition:
stage: ecs
tags:
- deployment
before_script:
- apk add build-base python3-dev python3 libffi-dev libressl-dev bash git gettext curl
- apk add py3-pip
- pip install six awscli awsebcli
- $(aws ecr get-login --no-include-email --region "${AWS_DEFAULT_REGION}")
script:
- TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition "$TASK_DEFINITION_NAME" --region "${REGION}")
- NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | python3 $CI_PROJECT_DIR/update_task_definition_image.py $REPOSITORY_URL:$IMAGE_TAG)
- echo "Registering new container definition..."
- aws ecs register-task-definition --region "${REGION}" --family "${TASK_DEFINITION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"
- aws ecs update-service --region "${REGION}" --cluster "${CLUSTER_NAME}" --service "${SERVICE_NAME}" --task-definition "${TASK_DEFINITION_NAME}"
only:
- refactor/ca
dependencies:
- tag_image