I would like to deploy my MERN stack app in DigitalOcean Droplet via CircleCI as CI/CD tool. I read that it needs ssh into the Droplet and run commands for building and deploying the app, but I have tried a several ways to achieve it yet still ends up no luck.
Below is my config.yml for CircleCI
version: 2.1
orbs:
node: circleci/node#4.1.0
jobs:
express_deploy:
docker:
- image: "cimg/node:current"
steps:
- run: node --version
- add_ssh_keys:
fingerprints:
- "$ssh_key_prints"
- run: ssh -oStrictHostKeyChecking=no $username#$ip "cd $directory \ npm install \ node app.js"
workflows:
init:
jobs:
- express_deploy
error: bash: line 0: cd: too many arguments
Some of the others attempts I have tried:
steps:
- run: node --version
- add_ssh_keys:
fingerprints:
- "$ssh_key_prints"
- run: ssh -oStrictHostKeyChecking=no $username#$ip "cd $directory"
- run: npm install
- run: node app.js"
error: module $directory/app.js not found
I wanna know if this approach is wrong or is there a way to fix it?
notes: $ are credential variables that I blurred here, but I tested them independently and they worked fine
Related
I have installed gitlab runner in my server and binded with the gitlab repo and pipelines.
here's my gitlab-ci.yml
image:
name: node:14.9.0
stages:
- install
- deploy
install for production:
image: node:14.9.0
stage: install
tags:
- mytag
only:
- master
script:
- node -v
- /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm i -s
deploy to production server:
image: node:14.9.0
stage: deploy
only:
- master
script:
- cp -r * /the/folder/backend/
- cd /the/folder/backend/
- /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm run migrate
- node_modules/pm2/bin/pm2 reload all --update-env -- DB_OPERATOR_USERNAME=$DB_OPERATOR_USERNAME DB_OPERATOR_EMAIL=$DB_OPERATOR_EMAIL DB_OPERATOR_PASSWORD=$DB_OPERATOR_PASSWORD DB_ADMIN_USERNAME=$DB_ADMIN_USERNAME DB_ADMIN_EMAIL=$DB_ADMIN_EMAIL DB_ADMIN_PASSWORD=$DB_ADMIN_PASSWORD DB_USERNAME=$DB_USERNAME DB_DATABASE=$DB_DATABASE DB_PASSWORD=$DB_PASSWORD
after pushing the master branch and let the pipeline run, it's not loading the 14.9.0 node version.
Checking out b795f45a as master...
Removing node_modules/
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ node -v
v8.10.0
$ /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm i -s
if I ssh into my server, the node version that nvm loads (the default one) is the 14.9.0
I have 2 runners in this machine. The first one for the frontend (and it loads the correct default version of node, which is 14.9.0) and the second one for the backend that has the above problem. The node versions are the same. I don't have docker at the moment and the node -v command inside sudo su gitlab-runner user shows 14.9.0
Why the gitlab runner is loading the 8.10.0 version? how can I set it to 14.9.0?
Thanks in advance
Try to run the docker image inside the gitlab runner and check the global version of node, the problem does not seem to be the runner.
I am trying to deploy a nodejs app from github to a remote ubuntu server via ssh. Here is my main.yml:
name: Node Github CI
on:
push:
branches:
- master
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Node Js
uses: actions/setup-node#v1
- name: SSH and deploy node app
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.SERVER_IP }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.SERVER_KEY }}
script: |
service myservice stop
cd leancrm-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
When I run this, I get this error:
======CMD======
service myservice stop
cd myapp-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
======END======
err: fatal: Unable to look up myuser#github.com (port 9418) (Name or service not known)
err: bash: line 3: npm: command not found
==============================================
Screenshot:
Since you are connected to your server I assume you already have the repo there, so you only need to execute git pull.
Also you should add these lines at the beginning of the script:
export NVM_DIR=~/.nvm
source ~/.nvm/nvm.sh
My yml file looks like these at the end:
script: |
git pull
export NVM_DIR=~/.nvm
source ~/.nvm/nvm.sh
npm install
npm run start_server
Reason:
I use nvm for the server node environment, and nvm will not install the node environment in the /usr/local/bin/ directory, so that sudo can't find the corresponding instructions, and finally create a soft connection to solve
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/node" "/usr/local/bin/node"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/npm" "/usr/local/bin/npm"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/pm2" "/usr/local/bin/pm2"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/yarn" "/usr/local/bin/yarn"
you can test "sudo npm -v"
Your first step
name: Node Js
uses: actions/setup-node#v1
sets up Node.js on the GitHub build runner. Your second step however...
name: SSH and deploy node app
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.SERVER_IP }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.SERVER_KEY }}
script: |
service myservice stop
cd leancrm-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
... SSHs to your server and then runs script instructions there. You're also attempting to check out your source code repo there.
What you probably wanna do is check out your repo on the GitHub build runner...
- name: Checkout repo
uses: actions/checkout#v2
.. then run npm install there, then scp the output to your server, and finally ssh to that machine and restart your service.
Check if you did install npm on your remote ubuntu.
npm also needs to be installed on the remote server for the deployment.
I am using pm2 on my remote ubuntu server and CircleCI for CI, I've got the following configuration files:
version: 2.1
orbs:
node: circleci/node#1.1.6
jobs:
deploy-prod:
docker:
# specify the version you desire here (you might not want node)
- image: circleci/node:7.10
steps:
- checkout
- run: ssh -oStrictHostKeyChecking=no -v $DROPLET_USER#$DROPLET_IP ./deploy_project.sh $MICROSERVICE_NAME
workflows:
build-and-test:
jobs:
- deploy-prod:
filters:
branches:
only:
- master
In my deploy script I do the following:
cd /var/www/nodejs/$1
git pull git#github.com:DevandScorp/hippocrates_authorizationmicroservice.git
cd ..
pm2 restart ecosystem.config.js --only $1
But I've got the following error:
./deploy_project.sh: line 4: pm2: command not found
Is it possible to run my server's pm2 in CircleCI config or can I reload my microservice automatically in another way?
So, if you want to make anything on your server using CircleCI, it's just a waste of time. CircleCI provides a virtual environment, where you can, for example, make some tests. Also you can push changes on your remote server, but CircleCI will not have any access to your server's system. So if we speak about pm2, you can enable watch mode and relaunch your microservice everytime CircleCI push changes to it
Add your deploy SSH keys to Github (or whatever your remote source control is), link it to CircleCI to allow Circle CI to SSH into your remote server and have it run pm2 there.
Adding your SSH Keys to Github
You'd follow similar steps for Bitbucket, etc
https://github.com/settings/ssh/new
Add your SSH fingerprint to your CI steps:
- add_ssh_keys:
fingerprints:
- "bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb:bb"
CI SSHes into your remote server and runs pm2:
steps:
- checkout
- run:
name: Deploy over SSH
command: ssh -p your_port_number your_user#your_host "cd ../path/to/your/project; git pull; pm2 start hello_sts";
I followed this guide: https://scribe.rip/#blueish/how-to-setup-ci-cd-with-circleci-and-deploy-your-nodejs-project-to-a-remote-server-275a31d1f6c4
although it's for Bitbucket, not Github.
I am using gitlab-ci to publish react native app using expo cli but the pipeline gets stuck at the tunnel connected stage. How do I fix this issue ?
cache:
paths:
- node_modules/
stages:
- deploy
before_script:
- npm ci
expo-deployments:
stage: deploy
script:
- echo "EXPO_USERNAME=""$EXPO_USERNAME" >> .env
- echo "EXPO_PASSWORD=""$EXPO_PASSWORD" >> .env
- npx expo login -u "$EXPO_USERNAME" -p "$EXPO_PASSWORD"
- cat /proc/sys/fs/inotify/max_user_watches
- echo fs.inotify.max_user_watches=524288 | tee -a /etc/sysctl.conf && sysctl -p
- npx expo publish --non-interactive
Here is the pipeline screenshot.
Not really sure what might be the issue. Tried to search but didn't find any concrete solution specific to this problem.
I started working on my first CodePipeline with node.js app which is hosted on github. I would like to create simple pipe as follow:
Github repo triggers pipe
Test env (Elastic Beanstalk app) is built from S3 .zip file
Test env runs npm test and npm lint
If everything is OK then QA env (another EB app) is built
For above pipe I've created .config files under .ebextensions directory:
I would like to use npm install --production for QA and PROD env, but it seems that EC2 can't find node nor npm. I checked logs and EC2 triggered npm install by default in temporary folder, then it fails on my first script and app catalogue is always empty.
container_commands:
install-dev:
command: "npm install"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
install-prod:
command: "npm install --production"
test: "[ \"$NODE_ENV\" != \"TEST\" ]"
ignoreErrors: false
Is it posible to run unit tests and linting without jenkins?
container_commands:
lint:
command: "npm run lint"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
test:
command: "npm run test"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
I set NODE_ENV for each Elastic Beanstalk instance. No matter what I will do every time my pipe fails because of npm is not recognized, but how is it possible if I'm running 64bit Amazon Linux with node.js ? What's more I cannot find any examples about CodePipeline with node.js in AWS Docs. Thanks in advance!
If you're using AWS for CI/CD, you can use CodeBuild. However, Github provides a great feature called Actions for running Unit Tests, which I find much simpler than AWS. Anyway, I will walk you through both examples:
Using AWS for running Unit Tests
Essentially, you could create a new stage into your CodePipeline, and configure the CodeBuild for running Unit Tests, e.g.
First, add a buildspec.yml file in the root folder of your app so you can use the following example:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- echo Installing Mocha globally...
- npm install -g mocha
pre_build:
commands:
- echo Installing dependencies...
- npm install
- npm install unit.js
build:
commands:
- echo Build started on `date`
- echo Run Unit Tests and so on
- npm run test
- npm run lint
post_build:
commands:
- echo Build completed on `date`
# THIS IS OPTIONAL
artifacts:
files:
- app.js
- package.json
- src/app/*
- node_modules/**/*
You can find everything you need in the BackSpace Academy, this course is for free:
AWS DevOps CI/CD - CodePipeline, Elastic Beanstalk and Mocha
Using Github for running Unit Tests
You could create your custom actions using Github, it will automatically set up everything you need in your root folder, e.g.
After choosing the appropriate workflow, it will automatically generate a folder/file ".github > workflow > nodejs.yml".
So it will look like this:
name: Node CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm test
env:
CI: true
I hope you could find everything you need in this answer. Cheers
Have you incorporated CodeBuild into your pipeline?
You should
1) Create a pipeline whose source is your github account. Go through the setup procedure so that commits on a particular branch trigger the Codepipeline
2) Create a test stage in your Codepipeline which leverages the CodeBuild service. In order to run your Node tests, you might need to provide a configured build environment. And you probably also need to provide a build spec file that specifies the tests to run etc.
3) Assuming that the test stage passes, you can determine if the pipeline continues to another stage which is linked to an elasticbeanstalk app environment which supports the Node platform. These environments are purely for artifacts that have passed testing, so I see no need to have the .ebextensions commands written above.
Have a read of what CodeBuild can do to help you run tests for Node,
https://aws.amazon.com/codebuild/
Good luck!