I try to build for one of my Laravel Project a simple pipeline script for continuous integration with Jenkins which involves couple of simple steps.
build
compile assets
test
deploy
the process works fine till is not starting to compile the assets, on assets compile to hole process hangs and is not getting executed to the end.
As background I'm using Centos 7 and for the assets compile node-10.0.0 and here is the jenkins pipeline snippet
node {
stage('Install dependencies') {
// Run Composer
sh 'rm -rf vendor'
sh 'composer install'
//sh 'cp .env.example .env'
sh 'php artisan key:generate'
}
stage('Compile Assets') {
env.NODE_ENV = "test"
print "Environment will be : ${env.NODE_ENV}"
sh 'rm -rf node_modules'
sh 'node -v'
sh 'yarn install --ignore-engines'
}
stage('Run PHP Tests') {
sh "vendor/bin/phpunit"
}
}
Related
I'm running docker compose file in ec2 instance, this file contains mysql, jenkins images. Also running nodejs app using pm2 command, when I run nodejs server manually in ec2-instance everything is working properly.
But When I try to deploy nodejs app using jenkins container, latest code is not deployed, i tried to debug why it is not deployed, I found one interesting thing
When i try to run pipeline all commands executed inside jenkins container workspace with jenkins user(container path : /var/jenkins_home/workspace/main)
So my question is my actual nodejs app placed in /home/ubuntu/node-app. But when try to deploy code using jenkins pipeline, pipeline is running in different path(/var/jenkins_home/workspace/main).
Now i have question, is this possible to execute pipeline deployment command for /home/ubuntu/node-app path? not docker container path?
if changing path is not possible, how to point jenkins docker container to ec2 public ip?
I shared jenkinsfile script and docker compose image code for reference
Jenkinsfile code:
stages {
stage('Build') {
steps {
sh 'npm install && npm run build'
}
}
stage('Deploy') {
steps {
sh "pwd"
sh 'git pull origin main'
sh 'pm2 stop server || true'
sh 'npm install'
sh 'npm run build'
sh 'pm2 start build/server.js '
}
}
}
Jenkins Docker Image code:
jenkins:
image: 'jenkins/jenkins:lts'
container_name: 'jenkins'
restart: always
ports:
- '8080:8080'
- '50000:50000'
volumes:
- jenkins-data:/etc/gitlab
- /var/run/docker.sock:/var/run/docker.sock
Edit 1:
I tried to change the path following ways to in jenkinsfile
cd /home/ubuntu/node-app
I'm getting following error
/var/jenkins_home/workspace/main#tmp/durable-44039790/script.sh: 1: cd: can't cd to /home/ubuntu/node-app
Note : this path(/var/jenkins_home/workspace/main) is only visible in ec2 machine after exec following command, normally this path is not exist in ec2 machine
docker exec -it jenkins bash
Try with following fix code
stage('Deploy') {
steps {
sh "cd /home/ubuntu/node-app"
sh 'git pull origin main'
sh 'pm2 stop server || true'
sh 'npm install'
sh 'npm run build'
sh 'pm2 start build/server.js '
}
}
Finally I found solution for this issue.
The actual issues is I didn't create any slave agent for jenkins pipeline, that's why jenkins pipeline jobs are running in master agent location, here master agent location was jenkins docker container space, that's why pipeline jobs are strored into /var/jenkins_home/workspace/main this path
Now I added slave agent and mentioned the customWorkspace path( i mentioned customWorkspace path is 'home/ubuntu/node-app') in jenkinsfile. Now my jenkins pipeline is working under custom workspace that is /home/ubuntu/node-app
My updated jenkinsfile code:
pipeline {
agent {
node {
label 'agent1'
customWorkspace '/home/ubuntu/node-app'
}
}
stages {
stage('Build') {
steps {
sh 'npm install && npm run build'
}
}
stage('Deploy') {
steps {
sh 'pm2 restart server'
}
}
}
}
I am using gulp to run my js app on the docker container from an image built using a dockerfile.
I am an advanced beginner :) in Jenkins.
dockerfile
FROM node:10.11.0-alpine
RUN apk update && apk add --no-cache git curl python py-pip bzip2 alpine-sdk && \
pip install --upgrade awscli && \
rm -rf /tmp/* /var/lib/apt/lists/* /var/cache/apk/*
WORKDIR /app
COPY . .
ADD . /usr/src/app/.
RUN chmod -R 777 /usr/src/app
WORKDIR /usr/src/app
jenkinsFile - I am using sequential stages. The first stage is Initialize where I am setting up the docker container. The docker image is hosted internally, I am pulling that down and running the shell commands. To keep it simple I am adding only the relevant stages here.
pipeline {
agent none
options {
timeout(time: timeoutSeconds, unit: 'SECONDS')
disableConcurrentBuilds()
}
stage('Initialize') {
agent {
docker {
image 'js-docker-image'
registryUrl 'https://my-artifactory-url'
registryCredentialsId artifactoryCredentialsId
args '-u root -v /var/run/docker.sock:/var/run/docker.sock'
label 'docker'
}
}
stages {
stage('Install Dependencies') {
steps {
script {
def command = '''
npm install
'''
sh command
}
}
}
stage('Build') {
steps {
script {
def command = '''
./node_modules/.bin/gulp build_only
'''
sh command
}
}
}
...
Here are the relevant parts in the build output. I have removed some sensitive info.
...
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Initialize)
[Pipeline] node
Running on slave06 in /opt/jenkins/slave06/workspace/ntegrate-playground-573
[Pipeline] {
[Pipeline] checkout
...
$ docker run -t -d -u 133733063:133693953 -u root -v /var/run/docker.sock:/var/run/docker.sock -w /opt/jenkins/slave06/workspace/ntegrate-playground-573 -v /opt/jenkins/slave06/workspace/ntegrate-playground-573:/opt/jenkins/slave06/workspace/ntegrate-playground-573:rw,z -v /opt/jenkins/slave06/workspace/ntegrate-playground-573#tmp:/opt/jenkins/slave06/workspace/ntegrate-playground-573#tmp:rw,z -e ******** docker-internal-registry/js-docker-image:latest cat
...
...
$ docker top f6cddf731de8cd63c37e12462f1041db2f4a14486ad98e00dbb81d210711bc63
+ npm install
npm WARN jsdom#15.2.1 requires a peer of canvas#^2.5.0 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents#1.2.9 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents#1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
audited 901232 packages in 20.046s
found 16 vulnerabilities (4 low, 1 moderate, 11 high)
run `npm audit fix` to fix them, or `npm audit` for details
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ ./node_modules/.bin/gulp build_only
/opt/jenkins/slave06/workspace/ntegrate-playground-573#tmp/durable-8c5396a6/script.sh: line 1: ./node_modules/.bin/gulp: not found
Questions
Why is Jenkins trying to attach a new volume with #tmp?
What is durable-8c5396a6?
1) Why is Jenkins trying to attach a new volume with #tmp?
the #tmp is used to place temp artifacts, like the Shell scriptin sh, jenkins will
create a .sh file which includes the script of sh, then execute this .sh file.
Because this .sh file is not part of your source code, just temp file generated during stages running.
You can think of the files in #tmp are managed by Jenkins, not by user. User can't control it. It's a part of Jenkins pipeline design.
For source code, build/package artifact managed by use, they are not been placed in the #tmp, but in job workspace folder, in your case is /opt/jenkins/slave06/workspace/ntegrate-playground-573, which without the #tmp.
2) What is durable-8c5396a6?
Jenkins generated .sh file for each sh with same name script.sh.
In case there are more than one sh in your Jenkinsfile, jenkins put script.sh in different folder to avoid it for previous sh overwriten by next sh
To debug your issue, add two cmds pwd and ls -l before ./node_modules/.bin/gulp build_only. With that to check your current work dir and which files & folders under current work dir.
The most possible failure reason is you are in wrong work dir, secondly it's gulp is not added into project dependencies.
I am making a pipeline on Jenkins to test and deploy my node.js application using Docker containers. But I am getting my pipeline stuck because a test is failing. The behaviour I would expect is pipeline finishes without executing next stages but it will not get stuck.
Jenkinsfile:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''docker build --tag my-web:$BUILD_NUMBER .
docker stop my-web&& docker rm my-web
echo "Build step finished"'''
}
}
stage('Unit test') {
steps {
sh '''docker build -t my-web-test -f Dockerfile.test .
docker run --rm my-web-test
'''
}
}
stage('Run') {
steps {
sh '''docker run --name my-web -p 3000:3000 my-web:$BUILD_NUMBER node /var/www/index.js &
'''
echo 'RUNNING'
}
}
stage('End') {
steps {
echo 'End of pipeline'
}
}
}
}
Dockerfile.test:
FROM node:alpine
RUN mkdir /var/test
WORKDIR /var/test
COPY package.json /var/test/
RUN npm install && npm install -g mocha
COPY src/ /var/test/
CMD ["mocha", "tests/", "--recursive"]
When I trigger the pipeline:
If I remove Unit test stage from pipeline everything works OK and application begins running.
If I do not remove Unit test stage, testing stage begins and I get a result of 14 test passed and 1 failed but the pipeline hangs in this step so Run step never triggers and the pipeline keeps in Running status.
14 passing (2s)
1 failing
1) Checking user first-time-login
Should redirect to change-password page:
Error: expected "Location" of "/dashboard/change-password", got "/dashboard"
at Test._assertHeader (node_modules/supertest/lib/test.js:249:12)
at Test._assertFunction (node_modules/supertest/lib/test.js:283:11)
at Test.assert (node_modules/supertest/lib/test.js:173:18)
at localAssert (node_modules/supertest/lib/test.js:131:12)
at /var/test/node_modules/supertest/lib/test.js:128:5
at Test.Request.callback (node_modules/superagent/lib/node/index.js:728:3)
at IncomingMessage.<anonymous> (node_modules/superagent/lib/node/index.js:916:18)
at endReadableNT (_stream_readable.js:1154:12)
at processTicksAndRejections (internal/process/task_queues.js:77:11)
Newer versions of mocha need to exit, otherwise the server keeps running so next stage is never reached.
mocha --exit
I have a pipeline created in Jenkins with the following configuration for a Nodejs application:
pipeline {
agent any
stages {
stage('Build') {
steps {
nodejs(nodeJSInstallationName: 'Node8') {
sh 'npm install'
}
}
}
stage('Test') {
steps {
nodejs(nodeJSInstallationName: 'Node8') {
sh 'npm run test'
}
}
}
stage('Deploy') {
steps {
nodejs(nodeJSInstallationName: 'Node8') {
sh 'npm run start'
}
}
}
}
}
But the execution of 'npm' is failing, which gives me the following error:
+ npm install
/usr/bin/env:'node': No such file or directory
I installed Jenkins on my server using Docker (image jenkins/jenkins:lts) and I installed the NodeJS plugin in Jenkins.
I created the Node tooling in this way:
Do you know what may be happening?
A greeting.
Try this
sudo ln -sf "$(which node)" /usr/bin/node
sudo ln -s /var/lib/jenkins/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/node/bin/node /usr/bin/node
This is my first Jenkins pipeline project. I created a simple Node.js application, and I uploaded into hithub (public repo) and all I am trying to do with my Jenkinsfile is to "npm install" in my Build stage. I believe Jenkins is finding the Jenkinsfile but it is just not finding the npm. I am using jenkins official docker image to run my jenkins server. Here are the two plugging that I have installed
1) NodeJS Plugin and 2) Pipeline NPM Integration Plugin
and here is the file
pipeline {
agent any
stages {
stage ("Build") {
steps {
sh "npm install"
}
}
}
}
and this is the error I am getting when I run my 'Build Now'
[second project] Running shell script
+ npm install
/var/jenkins_home/workspace/second project#tmp/durable-ef33ffd4/script.sh: 2: /var/jenkins_home/workspace/second project#tmp/durable-ef33ffd4/script.sh:
npm: not found
can someone help?
I suppose, your npm binary isn't in PATH variable.
Try to specify full path to npm, usually it's /usr/bin
pipeline {
agent any
stages {
stage ("Build") {
steps {
sh "/usr/bin/npm install"
}
}
}
}
You can check npm path in console using command which npm
May be, you already figured this. Have you hosted your machine's docket socket in the container when you started the Jenkins container?
Specifically, you need use -v /var/run/docker.sock:/var/run/docker.sock on your docker run command.
Then in your pipeline, you need to run the npm on a docker container that is built it from official node docker image, such as node:10.11.0-alpine. Here is an example
pipeline {
agent {
docker {
image 'node:10.11.0-alpine'
}
}
stages {
stage ("Build") {
steps {
sh "npm install"
}
}
}
}
If you are on windows than try to run CMD as Administrator and then install NPM it will work for you