npm: not found on jenkins agent, but available through ssh - linux

I am trying to set up a jenkins pipeline that utilizes multiple agents. The agents are ubuntu instances living in a cloud tenancy (openstack). When trying to run some npm commands on some of the instances, I am getting the error npm: not found. I've read multiple other threads, but I am struggling to understand why npm might not be found. I set these instances up myself, and I know I installed all requirements, including node and npm.
Let's say I have 2 nodes - agent1 at IP1, and agent2 at IP2. They both have a user login with username cooluser1. When I do an ssh cooluser1#IP1 or ssh cooluser1#IP2, in either case, running npm -v gives me a proper node version (6.14.13). However, in my pipeline, npm is not found in the IP2 instance. Here's my pipline script:
pipeline {
agent {
node {
label 'agent1'
}
}
stages {
stage('Build'){
steps {
sh 'hostname -I'
sh 'echo "$USER"'
sh 'echo "$PATH"'
sh 'npm -v'
}
}
stage ('Run Tests'){
parallel {
stage('Running tests in parallel') {
agent {
node {
label 'agent2'
}
}
steps {
sh 'hostname -I'
sh 'echo "$USER"'
sh 'echo "$PATH"'
sh 'npm -v'
}
}
stage {
// more stuff running on another agent (agent3)
}
}
}
}
}
As you can see, in both the main agent agent1, and in the parallel stages, I run the same code, which checks the host IP, the username, the path, and the npm version. The IPs are as expected - IP1 and IP2. The $USER in both cases is indeed cooluser1. The path looks something like this:
// agent1
+ echo
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
// agent2
+ echo
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
A bit strange, but identical in both cases.
However, when I get to npm --v, for agent1, I get a version number, and any npm commands I want to run are workoing. but in agent2, I get npm: not found, and the pipeline fails if I try to use any npm commands. The full error is here:
+ npm -v
/home/vine/workspace/tend-multibranch_jenkins-testing#tmp/durable-d2a0251e/script.sh: 1: /home/vine/workspace/tend-multibranch_jenkins-testing#tmp/durable-d2a0251e/script.sh: npm: not found
But I clearly saw with ssh cooluser1#IP2 that npm is available in that machine to that user.
What might be going wrong here?

I will propose to you to install nodejs plugin, configure any nodejs version you want in 'manage jenkins' -> 'global tools configurations' and set nodejs in pipeline:
pipeline {
agent any
tools {
nodejs 'NodeJS_14.17.1'
}
stages {
stage ('nodejs test') {
steps {
sh 'npm -v'
}
}
}
}

Related

Jenkins build not failing at pipeline stage where OWASP Dependency Checker finds vulnerabilities

I am trying to fail my Jenkins build at the pipeline stage when OWASP Dependency checker finds and reports out vulnerabilities found. But instead it is moving forward and executing all the subsequent stages even if a vulnerability is found. My Jenkinsfile looks like this :-
pipeline {
tools {
nodejs "nodejs"
}
stages {
stage('install') {
steps {
// 'ci' install node modules
sh 'npm ci'
}
}
stage('Dependency Check') {
steps {
sh 'npm prune --production'
sh "mkdir -p build/report"
sh "'$DEPENDENCY_PATH' --project demoProject --disableRetireJS --suppression 'dependency-check-suppressions.xml' --format XML --out 'build/report/dependency-check-report.xml' --scan ."
dependencyCheckPublisher pattern: 'build/report/dependency-check-report.xml', failedTotalCritical: '0', failedTotalHigh: '0', failedTotalLow: '0', failedTotalMedium: '0'
}
}
stage('Test Step') {
steps {
sh 'echo "Reaching test step"'
}
}
}
}
Env. variable '$DEPENDENCY_PATH' contains the location for dependency-check.bat file. OWASP Dependency Checker finds and reports vulnerabilities which I can see in the dependency-check-report.xml and at the end it fails the build also. But the last stage Test Step also gets executed which I do not want. I want Jenkins build to fail at the Dependency Check stage if any vulnerabilities are found. Where am I doing wrong here?
Less hacky way that does not require whitelisting:
dependencyCheckPublisher (
pattern: '**/build/reports/dependencyCheck/dependency-check-report.xml',
failedTotalLow: 1,
failedTotalMedium: 1,
failedTotalHigh: 1,
failedTotalCritical: 1
)
if (currentBuild.result == 'UNSTABLE') {
unstable('UNSTABLE: Dependency check')
} else if (currentBuild.result == 'FAILURE') {
error('FAILED: Dependency check')
}
It seems that DependencyCheckPublisher throws an error but Jenkins is unable to catch it at that point, but at the end it checks for the same and fails the build. To catch the error at the exact point where it is thrown by DependencyCheckPublisher, I had to introduce a rawBuild console output check which checks whether DependencyCheckPublisher has printed anything about exceeded count for vulnerabilities or not. After dependencyCheckPublisher step add -
if (currentBuild.rawBuild.getLog(50).contains('[DependencyCheck] Findings exceed configured thresholds')) {
error("Build failed due to vulnerabilities found during dependencyCheck")
}else{
sh 'echo "No vulnerabilities found during dependencyCheck"'
}
For this you also need to allow rawBuild and getLog invocation permission from Jenkins. You can do it from Jenkins -> Manage Jenkins -> In-process Script Approval and allow both of them.(If you haven't allowed them then Jenkins build will fail and in the console output of the failed build you can find details regarding this)

Jenkins docker.image().withRun() what host name do I use to connect

I have a Jenkins pipeline and I'm trying to run a Postgres container and connect to it for some nodejs integrations tests. My Jenkins file looks like this:
stage('Test') {
steps {
script {
docker.image('postgres:10.5-alpine').withRun('-e "POSTGRES_USER=fred" -e "POSTGRES_PASSWORD="foobar" -e "POSTGRES_DB=mydb" -p 5432:5432') { c->
sh 'npm run test'
}
}
}
What hostname should I use to connect to the postgres database inside of my nodejs code? I have tried localhost but I get a connection refused exception:
ERROR=Error: connect ECONNREFUSED 127.0.0.1:5432
ERROR:Error: Error: connect ECONNREFUSED 127.0.0.1:5432
Additional Details:
I've added a sleep for 30 seconds for the container to start up. I know there are better ways to do this but for now I just want to solve the connection problem.
I run docker logs on the container to see if it is ready to accept connections, and it is.
stage('Test') {
steps {
script {
docker.image('postgres:10.5-alpine').withRun('-e "POSTGRES_USER=fred" -e "POSTGRES_PASSWORD="foobar" -e "POSTGRES_DB=mydb" -p 5432:5432') { c->
sleep 60
sh "docker logs ${c.id}"
sh 'npm run test'
}
}
}
tail of docker logs command:
2019-09-02 12:30:37.729 UTC [1] LOG: database system is ready to accept connections
I am running Jenkins itself in a docker container, so I am wondering if this is a factor?
My goal is to have a database startup with empty tables, run my integration tests against it, then shut down the database.
I can't run my tests inside of the container because the code I'm testing lives outside the container and triggered the jenkins pipeline to begin with. This is part of a Jenkins multi-branch pipeline and it's triggered by a push to a feature branch.
your code sample is missing a closing curly bracket and has an excess / mismatched quote. That way it is not clear whether you actually did (or wanted to) run your sh commands, inside or outside the call.
Depending on where the closing bracket was, the container might already have shut down.
Generally, the Postgres connection is fine with that fixed syntax issues:
pipeline {
agent any
stages {
stage('Test') {
steps {
script {
docker.image('postgres:10.5-alpine')
.withRun('-e "POSTGRES_USER=fred" '+
'-e "POSTGRES_PASSWORD=foobar" '+
'-e "POSTGRES_DB=mydb" '+
'-p 5432:5432'
){
sh script: """
sleep 5
pg_isready -h localhost
"""
//sh 'npm run test'
}
}
}
}
}
}
results in:
pg_isready -h localhost
localhost:5432 - accepting connections

Jenkins Pipeline - Switch node version dynamically

Is there a way we can set NodeJS version dynamically in Jenkins Pipeline. I am not able to make available nvm as well inside the pipeline.
sh 'export NVM_DIR=~/.nvm'
sh 'source ~/.nvm/nvm.sh
script.sh: line 2: nvm: command not found
None of this helped. Multiple teams using the pipeline need specific version of nodeJS. Earlier with non-pipeline jobs, this used to work using nvm.
You can create a choice parameter and use it in tools declaration.
pipeline {
agent any
parameters {
choice(name: 'NODE_VERSION', choices: ['NodeJS 9.6.1', 'NodeJS 7.7.0'], description: '')
}
tools {
nodejs params.NODE_VERSION
}
stages{
stage("Run"){
steps{
sh 'node --version'
}
}
}
}

Jenkins - env: ‘node’: No such file or directory

I have a jenkins server that is configured using
https://github.com/shierro/jenkins-docker-examples/tree/master/05-aws-ecs
I am running a blue ocean pipeline using a simple Jenkinsfile and the jenkins NodeJS plugin
pipeline {
agent any
tools {
nodejs 'node10'
}
stages {
stage ('Checkout Code') {
steps {
checkout scm
}
}
stage ('Install dependencies') {
steps {
sh "echo $PATH"
sh "npm install"
}
}
}
}
I made sure to add the node10 global tool as well w/c is used above
When the pipeline gets to the script sh "npm install" i am running through this error
this is the output of the command echo $PATH
so i think it's not a path issue
Also, it also wasn't able to add the global package
More info that might help:
Docker Jenkins server: FROM jenkins/jenkins:2.131-alpine
Blue ocean version: 1.7.0
NodeJS Plugin: 1.2.6
Multiple server restarts already
Any ideas why the jenkins server does not know where node is?
Big thanks in advance!
Thanks to #JoergS for some insight! The culprit in this case is: using alpine image as the docker base. So switching from jenkins/jenkins:2.131-alpine to jenkins/jenkins:2.131 solved the NodeJS plugin issue.
I have faced the same issue with jenkinsci/blueocean. I have resolved this by installing nodejs with below command(inside docker) not as jenkins plugin
apk add nodejs
I have faced the same issue with jenkinsci/blueocean. No jenkins nodejs plugin needed.
pipeline {
agent any
stages {
stage ('Checkout Code') {
steps {
checkout scm
}
}
stage ('Install dependencies') {
steps {
sh "apk add nodejs"
sh "echo $PATH"
sh "npm install"
}
}
}
}
Make a symbolic link like this:
sudo ln -s /var/lib/jenkins/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/node/bin/node /usr/bin/node
I want to highlight Mitch Downey's comment, it can't be just a comment because after spending 4 hours with no solution this comment helped me to resolve the solution
My issue ended up being with the jenkinsci/blueocean image. I was able
to just replace that image with jenkins/jenkins:lts and the NodeJS
plugin began working as expected

installing node on jenkins 2.0 using the pipeline plugin

I am running the following docker image jenkinsci/jenkins:2.0-rc-1 to try out jenkins 2.0, and the "pipeline" view.
I can't seem to install node. Here's my pipeline script:
node {
//tool([name: 'node-5.10.1', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'])
sh 'echo $(whoami)'
sh 'node -v'
}
The response when this runs is:
[ci] Running shell script
+ whoami
+ echo jenkins
jenkins
[Pipeline] sh
[ci] Running shell script
+ node -v
/../durable-3b0b1b07/script.sh: 2: /../durable-3b0b1b07/script.sh: node: not found
Here's what i've tried:
the jenkins NodeJS tool (which works correctly when used with a freestyle job)
logging into the docker container and installing node manually, for the same user:
UPDATE:
Building on Jesse Glick's answer below, i added the result to my scripts PATH:
node {
def nodeHome = tool name: 'node-5.10.1', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'
env.PATH = "${nodeHome}/bin:${env.PATH}"
sh 'npm install'
}
Either
node {
withEnv(["PATH+NODE=${tool name: 'node-5.10.1', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'}/bin"]) {
sh 'node -v'
}
}
or
node {
def nodeHome = tool name: 'node-5.10.1', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'
sh "${nodeHome}/bin/node -v"
}
should work. See JENKINS-28718 for further proposals.
By the way you can omit the type parameter and just use
tool 'node-5.10.1'
for brevity.
For me work next code:
node(){
def nodeHome = tool 'nodejs5'
env.PATH="${env.PATH}:${nodeHome}/bin"
...
sh 'npm install'
}
nodejs5 is the name of the tool specified in Jenkins configuration.
If anyone happens to deal with this issue on Jenkins running on Windows. Do the following:
def nodeHome = tool 'Node.js 6.9.5'
bat "\"${nodeHome}\"\\node.exe -v"
bat "\"${nodeHome}\"\\npm -v"

Resources