I started working on my first CodePipeline with node.js app which is hosted on github. I would like to create simple pipe as follow:
Github repo triggers pipe
Test env (Elastic Beanstalk app) is built from S3 .zip file
Test env runs npm test and npm lint
If everything is OK then QA env (another EB app) is built
For above pipe I've created .config files under .ebextensions directory:
I would like to use npm install --production for QA and PROD env, but it seems that EC2 can't find node nor npm. I checked logs and EC2 triggered npm install by default in temporary folder, then it fails on my first script and app catalogue is always empty.
container_commands:
install-dev:
command: "npm install"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
install-prod:
command: "npm install --production"
test: "[ \"$NODE_ENV\" != \"TEST\" ]"
ignoreErrors: false
Is it posible to run unit tests and linting without jenkins?
container_commands:
lint:
command: "npm run lint"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
test:
command: "npm run test"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
I set NODE_ENV for each Elastic Beanstalk instance. No matter what I will do every time my pipe fails because of npm is not recognized, but how is it possible if I'm running 64bit Amazon Linux with node.js ? What's more I cannot find any examples about CodePipeline with node.js in AWS Docs. Thanks in advance!
If you're using AWS for CI/CD, you can use CodeBuild. However, Github provides a great feature called Actions for running Unit Tests, which I find much simpler than AWS. Anyway, I will walk you through both examples:
Using AWS for running Unit Tests
Essentially, you could create a new stage into your CodePipeline, and configure the CodeBuild for running Unit Tests, e.g.
First, add a buildspec.yml file in the root folder of your app so you can use the following example:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- echo Installing Mocha globally...
- npm install -g mocha
pre_build:
commands:
- echo Installing dependencies...
- npm install
- npm install unit.js
build:
commands:
- echo Build started on `date`
- echo Run Unit Tests and so on
- npm run test
- npm run lint
post_build:
commands:
- echo Build completed on `date`
# THIS IS OPTIONAL
artifacts:
files:
- app.js
- package.json
- src/app/*
- node_modules/**/*
You can find everything you need in the BackSpace Academy, this course is for free:
AWS DevOps CI/CD - CodePipeline, Elastic Beanstalk and Mocha
Using Github for running Unit Tests
You could create your custom actions using Github, it will automatically set up everything you need in your root folder, e.g.
After choosing the appropriate workflow, it will automatically generate a folder/file ".github > workflow > nodejs.yml".
So it will look like this:
name: Node CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm test
env:
CI: true
I hope you could find everything you need in this answer. Cheers
Have you incorporated CodeBuild into your pipeline?
You should
1) Create a pipeline whose source is your github account. Go through the setup procedure so that commits on a particular branch trigger the Codepipeline
2) Create a test stage in your Codepipeline which leverages the CodeBuild service. In order to run your Node tests, you might need to provide a configured build environment. And you probably also need to provide a build spec file that specifies the tests to run etc.
3) Assuming that the test stage passes, you can determine if the pipeline continues to another stage which is linked to an elasticbeanstalk app environment which supports the Node platform. These environments are purely for artifacts that have passed testing, so I see no need to have the .ebextensions commands written above.
Have a read of what CodeBuild can do to help you run tests for Node,
https://aws.amazon.com/codebuild/
Good luck!
Related
I am using yarn workspaces (yarn version 1.22.19) and I would like to run tests for all workspaces, without stopping even if tests fail for one of the workspaces.
This is so I can collect all failing tests across all workspaces in one run. I'm running the tests on a github action.
I am running the following command:
yarn workspaces run test --passWithNoTests
All workspaces have a test script in the package.json that runs the tests with Jest.
Jest returns an exit code of 1 when tests fail. This causes the yarn workspaces run command to fail and stop. I would like it to continue and fail only after running tests for all workspaces.
How can I make the yarn workspaces run continue even if tests fail for one of the workspaces, yet still have it fail at the end?
Edit:
I am running bash.
Using workarounds like set -e or || true might help swallow the error, but I do want the command to fail ultimately, I just want it to fail after running all tests.
For example:
Say I have 3 workspaces - workspace a, workspace b and workspace c. All of them have the following script in their package.json:
test: "jest"
Say tests pass for workspace a and workspace c, but fail for workspace b. My desired result is that running yarn workspaces run test will run tests for all workspaces (and not stop after tests fail for workspace b) but for it to fail after running all tests.
Here is my github workflow. It just installs dependencies and runs the test script which runs the command yarn workspaces run test --passWithNoTests.
name: Run All Tests
on:
pull_request:
branches: ['develop']
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Run tests
run: yarn test
For future reference, this is what I ended up doing:
Add a test:ci script to all package.json file with the following defintion:
"test:ci": "jest --ci --reporters=jest-junit --reporters=default --passWithNoTests || true"
This makes it so the command passes even if tests fails.
Use a jest-junit test reporter to output an xml with test results.
In the action, run yarn workspaces run test which runs tests for all workspaces (packages).
Use the dorny/test-reporter#v1 to collect all test result xml files into a nice view.
Set fail-on-error: 'true' for the action dorny/test-reporter#v1 which will make the step fail if any test failed.
Here's the full github workflow:
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs
name: Build & Test
on:
pull_request:
branches: ['develop']
# cancel any previous runs that are still in progress if a new commit is pushed
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Run tests
run: yarn test:ci
- name: Unlink all symbolic links # so we don't go over the same file twice
if: success() || failure() # run this step even if previous step failed
run: find node_modules -type l -exec unlink {} \;
- name: Test Report
uses: dorny/test-reporter#v1
if: success() || failure() # run this step even if previous step failed
with:
name: Jest Test Results # Name of the check run which will be created
path: '**/jest-junit.xml'
reporter: jest-junit # Format of test results
list-suites: 'failed'
list-tests: 'failed'
fail-on-error: 'true'
This achieves everything I was looking for - running all tests for all workspaces, even if some fail, while still failing the workflow.
I have set up a cron job for my cypress tests however I want to run it on a specific test alone.
This is my cron job:
name: Cypress Tests
on: [push]
jobs:
cypress-run:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action#v4.x.x # use the explicit version number
with:
build: npm run build
start: npm start
I just want to run this one spec file:
Is there a way to run this spec file alone?
Try this out!
Add new command in script section of package.json. Use file or folder here.
"scripts": {
"cypress:spec-run": "cypress run --browser chrome --spec test.spec.js"
},
Run the commond in respective crone job
npm run cypress:spec-run
I've got a simple Vue SPA that builds and can be served without problem locally, but when I try and build and deploy via GitHub Actions to an Azure App Service, it just results in the ':( Application Error' page when launching.
Below is the pretty much default workflow .yml, the app service configuration, and the error log from trying to build the app.
I assume the files are built from the /dist folder at /home/site/wwwroot, where node_modules are installed and package.json is generated.. but it doesn't seem like it is (there are no files when checking wwwroot, so build failed?)
Any help would be appreciated, I've been stuck on this for a whole day and will happily provide more info. I've also deployed the NodeJS backend to an app service without too much hassle, so I'm familiar with the process -- just can't get this frontend up!
name: Build and deploy Node.js app to Azure Web App - shelf-library
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '16.x'
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: dist/
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'shelf-library'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_11D7C84BF0CE47B68181C49B9ED47D19 }}
package: .
Check the below steps to create VueJS and deploy to Azure App Service using Git Hub Actions.
Thanks #Anthony Salemo for the clear steps.
In Command prompt, run the below command to create a Vue App.
vue create myvueapp
Navigate to the root directory of the application cd myvueapp and run
yarn serve
or
npm run serve
Run npm run build command for production build.dist folder will be created.
Push the application to GitHub Repository.
You can check the code available in my GitHub Repository.
My GitHub folder structure
Create an Azure App service
Navigate to your App Service =>Deployment Center and select the code repo from your GitHub.
Initially I got the below content page when I tried to access the App.
Add the Startup Command in Configuration => General Settings.
pm2 serve /home/site/wwwroot/dist --spa --no-daemon
Initially even I got the same Application Error.
In Git Hub I can see the build and deploy action is not yet completed.
Wait for the build to deploy successfully.
My deployed folder structure in KUDU Console
Now Iam able to access the Application without any issues.
I saw you are running your stages with different agents, so I suppose that your issue could be resulted from that the configuration in the build stage agent could not be inherited into the deploy stage agent.
So you could test to add the npm task to install the vue-cli-service module in your deploy agent again.
==========================================================
Updated on 12/22
I suppose that in deploy stage, you could add the npm and node installation with below.
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '16.x'
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
- name: npm install, build, and test
run: |
npm install -g #vue/cli#latest
npm i -g #vue/cli-service-global
echo '<template><h1>Hello!</h1></template>' > App.vue
vue serve
I've been trying to setup a CI/CD Pipeline on my Repo which runs common tasks like linting/tests etc. I've successfully setup a Gitlab-Runner which is working fine. The only part I'm stuck is the "deploy" part.
When I run my build, how do I actually get the files into my /var/www/xyz folder.
I get that everything is running in a Docker Container and I can't just magically copy paste my files there, but i don't get how I get the files on my actual server-directory. I've been searching for days for good docs / explanations, so as always, StackOverflow is my last resort for help.
I'm running on a Ubuntu 20.04 LTS VPS and a SaaS GitLab-Repository if that info is needed. That's my .gitlab-ci.yml:
image: timbru31/node-alpine-git
before_script:
- git fetch origin
stages:
- setup
- test
- build
- deploy
#All Setup Jobs
Install Dependencies:
stage: setup
interruptible: true
script:
- npm install
- npm i -g #nrwl/cli
artifacts:
paths:
- node_modules/
# All Test Jobs
Lint:
stage: test
script: npx nx run nx-fun:lint
Tests:
stage: test
script: npx nx run nx-fun:test
Deploy:
stage: build
script:
- ls /var/www/
- npx nx build --prod --output-path=dist/
- cp -r dist/* /var/www/html/neostax/
only:
refs:
- master
Normally I would ssh into my server, run the build, and then copy the build to the corresponding web-directory.
TL;DR - How do I get files from a GitLab-Runner to an actual directory on the server?
I'm trying to deploy my webapp to firebase hosting through a bitbucket pipeline, It's not deploying correctly in the pipeline but in the console it works no problem. This is what I do in the console:
npm run build
firebase login:ci
firebase deploy --project $PROJECT_NAME
In the pipeline I'm running this YAML script:
image: node:10.15.3
pipelines:
default:
- step:
name: Install and Build App
caches:
- node
script:
- npm install
- CI=false npm run build
artifacts:
- build/
- step:
name: Deploy App to Firebase
deployment: production
script:
- pipe: atlassian/firebase-deploy:0.6.0
variables:
KEY_FILE: $KEY_FILE
PROJECT_ID: $PROJECT_ID
I think it might have to do with the .firebaserc but I'm not sure. this is the .firebaserc:
firebase target:apply hosting $PROJECT_ID $DOMAIN
Maybe someone can shed some light on why this isn't working, I'm new to pipeline scripts and I don't really see the issue, it succeeds in deploying to firebase hosting but It's not working at all on the actual domain.
When you run the command firebase login:ci that should generate a TOKEN, you add that token in Bitbucket in your Repository Settings > Repository Variables. What ever name you choose should match your pipeline. In my example I use FIREBASE_TOKEN_CI. When I commit my changes to bitbucket, it runs the pipeline, builds and deploys.
You can always modify your script in your package.json so in your cli you can run npm run build:prod like you would run npm run start, etc and use the build:prod in the yml.
here is an example:
"scripts": {
"ng": "ng",
"start": "ng serve",
"build:prod": "ng build --prod=true"
}
CODE BELOW is a pipeline.yml I use for Ionic/Angular
NOTE: Artifacts is the folder your build files are generated after running build. Angular is called dist, so you might use dist/. My example uses www/** that is Ionics build output. You have some CI=False in your example, I have not seen that nor use that and my project builds and deploys. My second script is for cloud functions
- cd functions
- npm install
- cd ..
you can omit that part if you don't have functions. I have recently had a error about OAuth and I had to generate a new token with login:ci and replace my token, and it was working again for deploy. Hope this helps anyone. I had problems at first also and found a working format that I can adapt to other frameworks.
image: node:10.15.3
pipelines:
default:
- step:
name: Install, Build
caches:
- node
deployment: test
script:
- npm install
- npm run build:prod
artifacts:
- www/**
- step:
name: Deploy to Firebase
deployment: production
script:
- cd functions
- npm install
- cd ..
- pipe: atlassian/firebase-deploy:0.3.4
variables:
FIREBASE_TOKEN: '$FIREBASE_TOKEN_CI'