I am trying to deploy a nodejs app from github to a remote ubuntu server via ssh. Here is my main.yml:
name: Node Github CI
on:
push:
branches:
- master
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Node Js
uses: actions/setup-node#v1
- name: SSH and deploy node app
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.SERVER_IP }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.SERVER_KEY }}
script: |
service myservice stop
cd leancrm-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
When I run this, I get this error:
======CMD======
service myservice stop
cd myapp-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
======END======
err: fatal: Unable to look up myuser#github.com (port 9418) (Name or service not known)
err: bash: line 3: npm: command not found
==============================================
Screenshot:
Since you are connected to your server I assume you already have the repo there, so you only need to execute git pull.
Also you should add these lines at the beginning of the script:
export NVM_DIR=~/.nvm
source ~/.nvm/nvm.sh
My yml file looks like these at the end:
script: |
git pull
export NVM_DIR=~/.nvm
source ~/.nvm/nvm.sh
npm install
npm run start_server
Reason:
I use nvm for the server node environment, and nvm will not install the node environment in the /usr/local/bin/ directory, so that sudo can't find the corresponding instructions, and finally create a soft connection to solve
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/node" "/usr/local/bin/node"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/npm" "/usr/local/bin/npm"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/pm2" "/usr/local/bin/pm2"
sudo ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/yarn" "/usr/local/bin/yarn"
you can test "sudo npm -v"
Your first step
name: Node Js
uses: actions/setup-node#v1
sets up Node.js on the GitHub build runner. Your second step however...
name: SSH and deploy node app
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.SERVER_IP }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.SERVER_KEY }}
script: |
service myservice stop
cd leancrm-backend
git pull git://myuser#github.com/mycmp/myapp-backend.git master
npm install
service myservice start
... SSHs to your server and then runs script instructions there. You're also attempting to check out your source code repo there.
What you probably wanna do is check out your repo on the GitHub build runner...
- name: Checkout repo
uses: actions/checkout#v2
.. then run npm install there, then scp the output to your server, and finally ssh to that machine and restart your service.
Check if you did install npm on your remote ubuntu.
npm also needs to be installed on the remote server for the deployment.
Related
I have a requirements.txt with internal dependencies in private Github repositories. I've setup the build step of the workflow to use webfactory/ssh-agent#v0.5.4 to provide the SSH authentication which works perfectly during the build phase. The deployment phase is failing to authenticate because of SSH issues, but I can't find a similar way to get SSH working when Azure Oryx is handling the dependency building during the deploy.
The error:
Python Version: /opt/python/3.7.12/bin/python3.7
Creating directory for command manifest file if it doesnot exist
Removing existing manifest file
Python Virtual Environment: antenv
Creating virtual environment...
Activating virtual environment...
Running pip install...
"2022-09-12 15:13:31"|ERROR|ERROR: Command errored out with exit status 128: git clone -q
'ssh://****#github.com/Murphy-Hoffman/IBMi-MHC.git' /tmp/8da94d13f03a38b/antenv/src/ibmi-mhc-
db2 Check the logs for full command output. | Exit code: 1 | Please review your
requirements.txt | More information: https://aka.ms/troubleshoot-python
\n/bin/bash -c "oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform python --
platform-version 3.7 -i /tmp/8da94d13f03a38b --compress-destination-dir -p
virtualenv_name=antenv --log-file /tmp/build-debug.log | tee /tmp/oryx-build.log ; exit
$PIPESTATUS "
Generating summary of Oryx build
Parsing the build logs
Found 1 issue(s)
Build Summary :
===============
Errors (1)
1. ERROR: Command errored out with exit status 128: git clone -q
'ssh://****#github.com/Murphy-Hoffman/IBMi-MHC.git' /tmp/8da94d13f03a38b/antenv/src/ibmi-mhc-
db2 Check the logs for full command output.
- Next Steps: Please review your requirements.txt
- For more details you can browse to https://aka.ms/troubleshoot-python
My requirements.txt file
autopep8==1.7.0
ibm-db==2.0.9
-e git+ssh://git#github.com/Murphy-Hoffman/IBMi-
MHC.git#57085a5e1f5637bfdd815397b45ba1b2dfd9b52c#egg=IBMi_MHC_db2&subdirectory=utility/db2
-e git+ssh://git#github.com/Murphy-Hoffman/IBMi-
MHC.git#57085a5e1f5637bfdd815397b45ba1b2dfd9b52c#egg=IBMi_MHC_UNIT&subdirectory=IBMi/_UNIT
itoolkit==1.7.0
pycodestyle==2.9.1
pyodbc==4.0.32
toml==0.10.2
Finally, the Github Action yml that succeeds during the build phase but fails in deployment
# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions
# More info on Python, GitHub Actions, and Azure App Service: https://aka.ms/python-webapps-
actions
name: Build and deploy Python app to Azure Web App - mhc-customers
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: '3.7'
- name: Create and start virtual environment
run: |
python -m venv venv
source venv/bin/activate
- name: Setup SSH for Private Repos
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: |
${{ secrets.IBMI_MHC_SECRET }}
- name: Install Dependencies
run: |
pip install -r requirements.txt
# Optional: Add step to run tests here (PyTest, Django test suites, etc.)
- name: Upload artifact for deployment jobs
uses: actions/upload-artifact#v2
with:
name: python-app
path: |
.
!venv/
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Setup SSH for Private Repos
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: |
${{ secrets.IBMI_MHC_SECRET }}
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: python-app
path: .
- name: 'Deploy to Azure Web App'
uses: azure/webapps-deploy#v2
id: deploy-to-webapp
with:
app-name: 'mhc-customers'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_89B81B4839F24A7589B3A4D5D845DA59 }}
I've got this working - sort of. After reading up on the Oryx automated build platform https://github.com/microsoft/Oryx I added a appsvc.yaml in the application root that ran this config:
version: 1
pre-build: |
git config --global url."https://{secret}#github".insteadOf https://github
The problem is that we have to put our actual Github secret in the config yaml (in replace of "secret"). This isn't ideal but works to get Oryx using the correct credentials.
I am using gitlab ci cd pipe line to deploy my application to ubuntu server. I have different .env file for local and for dev env and its not a part of git repo (included in gitignore) how to get env variables in my app when deployed to ubuntu server
my gitlab-ci.yml
stages:
- deploy
cache:
paths:
- node_modules/
deploy:
stage: deploy
script:
- npm install
- sudo pm2 delete lknodeapi || true
- sudo pm2 start server.js --name lknodeapi
I guess you are looking for this -Create Variables Gitlab.You can create your environment variables in the ui and then change your gitlab-ci.yml like below
stages:
- deploy
cache:
paths:
- node_modules/
deploy:
stage: deploy
script:
- echo "NGINX_REPO_KEY"=$NGINX_REPO_KEY >> ".env"
- npm install
- sudo pm2 delete lknodeapi || true
- sudo pm2 start server.js --name lknodeapi
This will create a .env file in root folder and put your variables in it.
I would like to deploy my MERN stack app in DigitalOcean Droplet via CircleCI as CI/CD tool. I read that it needs ssh into the Droplet and run commands for building and deploying the app, but I have tried a several ways to achieve it yet still ends up no luck.
Below is my config.yml for CircleCI
version: 2.1
orbs:
node: circleci/node#4.1.0
jobs:
express_deploy:
docker:
- image: "cimg/node:current"
steps:
- run: node --version
- add_ssh_keys:
fingerprints:
- "$ssh_key_prints"
- run: ssh -oStrictHostKeyChecking=no $username#$ip "cd $directory \ npm install \ node app.js"
workflows:
init:
jobs:
- express_deploy
error: bash: line 0: cd: too many arguments
Some of the others attempts I have tried:
steps:
- run: node --version
- add_ssh_keys:
fingerprints:
- "$ssh_key_prints"
- run: ssh -oStrictHostKeyChecking=no $username#$ip "cd $directory"
- run: npm install
- run: node app.js"
error: module $directory/app.js not found
I wanna know if this approach is wrong or is there a way to fix it?
notes: $ are credential variables that I blurred here, but I tested them independently and they worked fine
I have installed gitlab runner in my server and binded with the gitlab repo and pipelines.
here's my gitlab-ci.yml
image:
name: node:14.9.0
stages:
- install
- deploy
install for production:
image: node:14.9.0
stage: install
tags:
- mytag
only:
- master
script:
- node -v
- /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm i -s
deploy to production server:
image: node:14.9.0
stage: deploy
only:
- master
script:
- cp -r * /the/folder/backend/
- cd /the/folder/backend/
- /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm run migrate
- node_modules/pm2/bin/pm2 reload all --update-env -- DB_OPERATOR_USERNAME=$DB_OPERATOR_USERNAME DB_OPERATOR_EMAIL=$DB_OPERATOR_EMAIL DB_OPERATOR_PASSWORD=$DB_OPERATOR_PASSWORD DB_ADMIN_USERNAME=$DB_ADMIN_USERNAME DB_ADMIN_EMAIL=$DB_ADMIN_EMAIL DB_ADMIN_PASSWORD=$DB_ADMIN_PASSWORD DB_USERNAME=$DB_USERNAME DB_DATABASE=$DB_DATABASE DB_PASSWORD=$DB_PASSWORD
after pushing the master branch and let the pipeline run, it's not loading the 14.9.0 node version.
Checking out b795f45a as master...
Removing node_modules/
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ node -v
v8.10.0
$ /home/gitlab-runner/.nvm/versions/node/v14.9.0/bin/npm i -s
if I ssh into my server, the node version that nvm loads (the default one) is the 14.9.0
I have 2 runners in this machine. The first one for the frontend (and it loads the correct default version of node, which is 14.9.0) and the second one for the backend that has the above problem. The node versions are the same. I don't have docker at the moment and the node -v command inside sudo su gitlab-runner user shows 14.9.0
Why the gitlab runner is loading the 8.10.0 version? how can I set it to 14.9.0?
Thanks in advance
Try to run the docker image inside the gitlab runner and check the global version of node, the problem does not seem to be the runner.
I am running gitlab-runner on my server, I am not using docker for deployment. I am trying to achieve the deployment on a remote server by doing ssh to the server. This is my .gitlab-ci.yml file -
stages:
- deploy
pre-staging:
stage: deploy
environment:
name: Gitlab CI/CD for pre-staging deployment
url: "$REMOTE_SERVER"
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- mkdir -p ~/.ssh
- eval $(ssh-agent -s)
- 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- ssh-add <(echo "$REMOTE_PVT_KEY")
- ssh ubuntu#"$REMOTE_SERVER" "cd deployment_container; npm --version ;rm -rf static; source deploy.sh"
- echo "Deployment completed"
only:
- megre_requests
- pre-staging
tags:
- auto-deploy
My pipeline is failing with error npm: command not found. I have proper environment for npm on my ssh-ed server. I am trying to deploy the Django-react application.
I have already tried using image: node:latest.
npm is installed using nvm
Can somebody help me resolve this?
Try and replace the ssh step with:
ssh ubuntu#"$REMOTE_SERVER" "pwd; cd deployment_container; echo $PATH"
If this "deployment" (which won't do anything) completes, it means npm is not accessible in the default PATH defined in the SSH session.
In this we have to give npm access to all users by executing below command,
n=$(which node);n=${n%/bin/node}; chmod -R 755 $n/bin/*; sudo cp -r $n/{bin,lib,share} /usr/local
This resolved my issue of npm: command not found
You can try this one.
stages:
- build
- deploy
deploy-prod:
image: node:12.13.0-alpine
stage: deploy
script:
- npm i -g firebase-tools