GitHub Action - AWS CLI - aws-cli

Recently the following GitHub Action has been deprecated with a deletion date already established at end of month (2019-12-31). The issue is, there is no "official" alternative yet (should be here). My questions are:
Does someone know if the "official" action will be released before 2019-12-31?
Is there an alternative?

aws-cli package is available in GitHub-hosted virtual environments. (aws-cli/1.16.266 Python/2.7.12 Linux/4.15.0-1057-azure botocore/1.13.2)
Make sure to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in environment variables. You can use Github secrets to store these credentials securely.
- name: Upload to S3
run: |
aws s3 sync ./build s3://test-bucket
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: 'ap-south-1'

From GitHub documentation the aws-cli is already available directly on the host image.
It would be nice if this information were available on the deprecation notice
¯\_(ツ)_/¯

The AWS CLI will come preinstalled on GitHub Actions environments. More information can be found in the actions/virtual-environments repository. In my case I needed the latest possible version of the CLI. I followed the AWS CLI Install documentation and added the following step to a workflow running on ubuntu/latest:
- name: Install AWS CLI v2
run: |
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
unzip -q /tmp/awscliv2.zip -d /tmp
rm /tmp/awscliv2.zip
sudo /tmp/aws/install --update
rm -rf /tmp/aws/

An alternative to default awscli, or using third party actions is to configure python and install the awscli at the time of the build:
name: Sync to S3 bucket
on: [push]
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-python#v2
with:
python-version: '3.7'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install awscli
- run: aws s3 sync builddir s3://foobar --region eu-west-1 --cache-control max-age=0 --acl public-read --delete
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
"Github Actions > Building and testing Python" docs on Github https://docs.github.com/en/actions/guides/building-and-testing-python

The repo was updated yesterday with the following new deprecation notice:
This action has been deprecated in favor of
https://github.com/aws-actions. This repo has been archived and will
be made private on 12/31/2019

Related

Github action pipeline keeps failing with Error: Az CLI Login failed. Please check the credentials and make sure az is installed on the runner

I am pretty new to github actions. the pipeline is supposed to deploy the app to azure. it runs three jobs, the first two are successful, but the last job which is the deploy phase keeps failing. it keeps bringing out this output Error:
Az CLI Login failed. Please check the credentials and make sure az is installed on the runner. For more information refer https://aka.ms/create-secrets-for-GitHub-workflows
since both the first two jobs runs perfectly, I'll only be showing the last job here with the env
name: myapp
env:
AZURE_WEBAPP_PACKAGE_PATH: '.'
DOTNET_VERSION: '6.0.x' # set this to the .NET Core version to use
on:
push:
branches: [ "master" ]
workflow_dispatch:
defaults:
run:
working-directory: app/myapp
permissions:
contents: read
deploy:
defaults:
run:
working-directory: app/myapp
permissions:
contents: none
runs-on: ubuntu-latest
needs: [build,access]
if: needs.access.outputs.secrets-valid == 'true'
environment:
name: 'Test-QA'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v3
with:
name: .net-app
- name: Login to Azure
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
enable-AzPSSession: true
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ secrets.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZURE_WEBAPP_PUBLISH_PROFILE }}
package: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}
artifact download was successful, but it was throwing the this error message.
Az CLI Login failed. Please check the credentials and make sure az is installed on the runner. For more information refer https://aka.ms/create-secrets-for-GitHub-workflows
I had to make changes by installing azure cli before the login session like o:
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v3
with:
name: .net-app
- name: Install Azure cli
run: |
sudo apt-get install ca-certificates curl apt-transport-https lsb-release gnupg
curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
AZ_REPO=$(lsb_release -cs)
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
sudo apt-get update
sudo apt-get install azure-cli
- name: Login to Azure
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
enable-AzPSSession: true
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ secrets.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZURE_WEBAPP_PUBLISH_PROFILE }}
package: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}
I got a different error displaying:
Error: An error occurred trying to start process '/usr/bin/bash' with working directory '/runner/_work/...dir/app/myapp'. No such file or directory
I really can't place where the problem could be. is there something I ought to do that I am missing?

Download file from s3 then store the files into the github repository

I have files stored in an AWS S3 bucket. I would like to use GitHub actions to download those files and put them into my GitHub repository. Furthermore, I am able to download the files, but I cannot seem to get the files to then go into my repository. Here are the attempts I have made.
steps:
- name: Download from S3
run: |
aws s3 cp --recursive aws-bucket myDirectoryIWantTheFilesIn
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: 'us-east-1'
I have tried as well with the aws-s3-github-actions
- name: copy sitemaps
uses: keithweaver/aws-s3-github-action#v1.0.0
with:
command: cp
source: awsS3Bucket
destination: myDirectoryIWantTheFilesIn
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws_region: us-east-1
flags: --recursive
I needed to include the action's checkout and then commit it.
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
# This is a basic workflow to help you get started with Actions
name: Fetch data.
# Controls when the workflow will run
on:
schedule:
# Runs "at hour 6 past every day" (see https://crontab.guru)
- cron: '00 6 * * *'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: keithweaver/aws-s3-github-action#v1.0.0 # Verifies the recursive flag
name: cp folder
with:
command: cp
source: myBucket
destination: myDestination
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws_region: us-east-1
flags: --recursive
- name: Commit changes
run: |
git config --local user.email "action#github.com"
git config --local user.name "GitHub Action"
git add .
git diff-index --quiet HEAD || git commit -m "MyCommitMessage" -a
git push origin master

Github Workflow deploying Python app to Azure App Service

I have a requirements.txt with internal dependencies in private Github repositories. I've setup the build step of the workflow to use webfactory/ssh-agent#v0.5.4 to provide the SSH authentication which works perfectly during the build phase. The deployment phase is failing to authenticate because of SSH issues, but I can't find a similar way to get SSH working when Azure Oryx is handling the dependency building during the deploy.
The error:
Python Version: /opt/python/3.7.12/bin/python3.7
Creating directory for command manifest file if it doesnot exist
Removing existing manifest file
Python Virtual Environment: antenv
Creating virtual environment...
Activating virtual environment...
Running pip install...
"2022-09-12 15:13:31"|ERROR|ERROR: Command errored out with exit status 128: git clone -q
'ssh://****#github.com/Murphy-Hoffman/IBMi-MHC.git' /tmp/8da94d13f03a38b/antenv/src/ibmi-mhc-
db2 Check the logs for full command output. | Exit code: 1 | Please review your
requirements.txt | More information: https://aka.ms/troubleshoot-python
\n/bin/bash -c "oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform python --
platform-version 3.7 -i /tmp/8da94d13f03a38b --compress-destination-dir -p
virtualenv_name=antenv --log-file /tmp/build-debug.log | tee /tmp/oryx-build.log ; exit
$PIPESTATUS "
Generating summary of Oryx build
Parsing the build logs
Found 1 issue(s)
Build Summary :
===============
Errors (1)
1. ERROR: Command errored out with exit status 128: git clone -q
'ssh://****#github.com/Murphy-Hoffman/IBMi-MHC.git' /tmp/8da94d13f03a38b/antenv/src/ibmi-mhc-
db2 Check the logs for full command output.
- Next Steps: Please review your requirements.txt
- For more details you can browse to https://aka.ms/troubleshoot-python
My requirements.txt file
autopep8==1.7.0
ibm-db==2.0.9
-e git+ssh://git#github.com/Murphy-Hoffman/IBMi-
MHC.git#57085a5e1f5637bfdd815397b45ba1b2dfd9b52c#egg=IBMi_MHC_db2&subdirectory=utility/db2
-e git+ssh://git#github.com/Murphy-Hoffman/IBMi-
MHC.git#57085a5e1f5637bfdd815397b45ba1b2dfd9b52c#egg=IBMi_MHC_UNIT&subdirectory=IBMi/_UNIT
itoolkit==1.7.0
pycodestyle==2.9.1
pyodbc==4.0.32
toml==0.10.2
Finally, the Github Action yml that succeeds during the build phase but fails in deployment
# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions
# More info on Python, GitHub Actions, and Azure App Service: https://aka.ms/python-webapps-
actions
name: Build and deploy Python app to Azure Web App - mhc-customers
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: '3.7'
- name: Create and start virtual environment
run: |
python -m venv venv
source venv/bin/activate
- name: Setup SSH for Private Repos
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: |
${{ secrets.IBMI_MHC_SECRET }}
- name: Install Dependencies
run: |
pip install -r requirements.txt
# Optional: Add step to run tests here (PyTest, Django test suites, etc.)
- name: Upload artifact for deployment jobs
uses: actions/upload-artifact#v2
with:
name: python-app
path: |
.
!venv/
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Setup SSH for Private Repos
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: |
${{ secrets.IBMI_MHC_SECRET }}
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: python-app
path: .
- name: 'Deploy to Azure Web App'
uses: azure/webapps-deploy#v2
id: deploy-to-webapp
with:
app-name: 'mhc-customers'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_89B81B4839F24A7589B3A4D5D845DA59 }}
I've got this working - sort of. After reading up on the Oryx automated build platform https://github.com/microsoft/Oryx I added a appsvc.yaml in the application root that ran this config:
version: 1
pre-build: |
git config --global url."https://{secret}#github".insteadOf https://github
The problem is that we have to put our actual Github secret in the config yaml (in replace of "secret"). This isn't ideal but works to get Oryx using the correct credentials.

sync python code on GitHub repo and deploy on Azure function

I would like to sync the code (in an azure functions folder). I followed the doc to create a github actions cd. However in azure portal in my functions sections functions, my folder does not appear. Anyone have any idea what the problem is. I have no errors in my workflow.
I have an error when i'm trying to Redeploy/sync my code: it's not found...
My Worflow File
Just post how I deploy a function from GitHub repo.
My file structure:
https://github.com/Paprika-a11y/pythonfunc.git
Navigate to the Deployment Center page, configure the settings:
After saving the settings, check the deployment process on GitHub Action:
If your function project is correct locally, then it should appear on portal:
If you need the workflow, here is the file generated automatically:
# Docs for the Azure Web Apps Deploy action: https://github.com/azure/functions-action
# More GitHub Actions for Azure: https://github.com/Azure/actions
name: Build and deploy Python project to Azure Function App - pyfunctemp
on:
push:
branches:
- main
workflow_dispatch:
env:
AZURE_FUNCTIONAPP_PACKAGE_PATH: '.' # set this to the path to your web app project, defaults to the repository root
PYTHON_VERSION: '3.7' # set this to the python version to use (supports 3.6, 3.7, 3.8)
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: 'Checkout GitHub Action'
uses: actions/checkout#master
- name: Setup Python ${{ env.PYTHON_VERSION }} Environment
uses: actions/setup-python#v1
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: 'Resolve Project Dependencies Using Pip'
shell: bash
run: |
pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
python -m pip install --upgrade pip
pip install -r requirements.txt --target=".python_packages/lib/site-packages"
popd
- name: 'Run Azure Functions Action'
uses: Azure/functions-action#v1
id: fa
with:
app-name: 'pyfunctemp'
slot-name: 'production'
package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
publish-profile: ${{ secrets.AzureAppService_PublishProfile_xxxxxx }}

Failed to register application revision in bitbucket pipeline

I've used bitbucket pipeline for auto-deploy my laravel application with AWS codedeploy by follow this tutorial https://medium.com/technext/bitbucket-to-aws-ec2-continuous-deployment-pipeline-using-aws-code-deploy-for-php-application-e39004243cd9 , but still give me error (Failed to register application revision) at step aws deploy register-application-revision in build
need help . please.....
bitbucket-pipelines.yml
image: atlassian/default-image:2
pipelines:
default:
- step:
script:
- apt-get update
- apt-get install -y zip
- zip -r application1.zip .
- pipe: atlassian/aws-code-deploy:0.2.5
variables:
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
APPLICATION_NAME: $APPLICATION_NAME
S3_BUCKET: $S3_BUCKET
COMMAND: 'upload'
ZIP_FILE: 'application1.zip'
VERSION_LABEL: 'my-app-1.0.0'
- pipe: atlassian/aws-code-deploy:0.2.5
variables:
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
APPLICATION_NAME: $APPLICATION_NAME
DEPLOYMENT_GROUP: $DEPLOYMENT_GROUP
S3_BUCKET: $S3_BUCKET
COMMAND: 'deploy'
WAIT: 'true'
VERSION_LABEL: 'my-app-1.0.0'
IGNORE_APPLICATION_STOP_FAILURES: 'true'
FILE_EXISTS_BEHAVIOR: 'OVERWRITE'
appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/my-app1
This could be happening due to a couple of reasons and one easier way to pinpoint the root cause is to enable 'debug' in your pipelines. It can be done by adding the following line in pipe variables along with the AWS variables added:
DEBUG: 'true'
The issue in my case was that the AWS user with which I was running the pipelines (i.e. the user corresponding to AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values defined) did not have enough privileges to create a revision in CodeDeploy. This was clearly indicated by the following line which showed up only after I enabled the debug mode:
An error occurred (AccessDeniedException) when calling the RegisterApplicationRevision operation: User: arn:aws:iam::XXXXXXXXXXXX:user/bitbucket-deployer is not authorized to perform: codedeploy:RegisterApplicationRevision on resource: arn:aws:codedeploy:$AWS_DEFAULT_REGION:XXXXXXXXXXXX:application:$APPLICATION_NAME'

Resources