I want to create a simple job using NodeJS, Github and Jenkins.
There are an exchange what runs on two servers addresses:
for example, us.exchange.com and eu.exchange.com.
I created an environment variable named SERVERS_LOCATION,
browser.get(`http://${process.env.SERVERS_LOCATION}.exchange.com`);
and a Jenkins parameter named SERVERS_LOCATION_JEN which may takes two options - US and EU.
Also I created a pipeline in Jenkins where I want to run parameterized build by choose one or another option, for that I use pipeline script in jenkinsfile what looks like that:
pipeline{
agent any
options{
disableConcurrentBuilds()
}
stages{
stage("install npm"){
steps{
bat "npm install"
bat "npx webdriver-manager update --versions.chrome 76.0.3809.68"
}
}
stage("executing job"){
steps{
bat "SERVERS_LOCATION=%SERVERS_LOCATION_JEN% npx protractor config/conf.js"
}
}
}
}
The main idea is to take the choosen value from Jenkins variable SERVERS_LOCATION_JEN and put it to environment variable ${process.env.SERVERS_LOCATION}, which can be used in code for further calls.
But when I running this job I have an error:
'SERVERS_LOCATION' is not recognized as an internal or external command,operable program or batch file.
P.S. running that job from git-bash works fine. (Win10 Chrome browser)
Could you point me please what I am doing wrong?
You have to use "set" command to assign a value to a variable in batch, so please use the below code:-
bat "set SERVERS_LOCATION=%SERVERS_LOCATION_JEN% npx protractor config/conf.js"
Related
I am trying to build a shell script which can be called via a cronjob to trigger an npm nodejs application.
This is my start.sh shell script
#!/bin/bash
#!/usr/local/bin/npm
cd /home/lharby/sites/mysite
npm run start
If I cd to this folder and execute ./start.sh the command appears to run. (Path to bash and npm are both correct after checking which npm).
My cron job looks like this:
*/5 * * * * /home/lharby/sites/mysite/start.sh >> /home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
This is throwing an error and additionally I was lead to believe that using >> would append to the file, it seems to overwrite it each time.
My guess is that trying to run this command via cron it cannot access certain environment variables that are set up in my index.js
For example:
const config = {
access_token: process.env.NEXT_MASTODON_ACCESS_TOKEN,
client_key: process.env.NEXT_MASTODON_CLIENT_KEY,
client_secret: process.env.NEXT_MASTODON_CLIENT_SECRET,
timeout_ms: 60 * 1000,
api_url: 'https://botsin.space/api/v1/',
};
const M = new Mastodon(config);
I believe I see the same issue when I try to run node index.js from /home/sites/lharby/mysite/src/
As my package.json has this configuration:
"scripts": {
"start": "node ./src/index.js --experimental-modules",
"temp": "node ./src/temp.js --experimental-modules"
},
I was exploring looking at just trying to run the whole app passing in node index.js and passing in the argument flags but I need to be able to run the cron file invoking npm rather than node, as I guess that creates a wrapper and npm can access process.env variables.
From my cron-errors.txt file I am seeing this:
/home/lharby/sites/mysite/node_modules/mastodon-api/lib/mastodon.js:345
throw new Error('Mastodon config must include \'' + reqKey + '\' when using \'user_auth\'');
^
Error: Mastodon config must include 'access_token' when using 'user_auth'
at /home/lharby/sites/glyphbot/node_modules/mastodon-api/lib/mastodon.js:345:27
//
//
/home/lharby/sites/mysite/start.sh: line 4: npm: command not found
/bin/sh: 1: /home/sites/glyphbot/start.sh: not found
/bin/sh: 1: /home/sites/glyphbot/start.sh: not found
How can I ensure the crontab will invoke npm? I feel like I am doing everything correctly.
EDIT
My issues are:
How to run a node project using npm from a cron job?
Can the cronjob access environment variables from the npm command?
Should >> cron-errors.txt 2>&1 append the file rather than replace the content each time.
UPDATE 19.01.23
So I added exports for my variables to the .bashrc file. And when checking echo $NEXT_MASTODON_ACCESS_TOKEN I am seeing my string value.
Updated my .sh file so it now looks like this:
#!/bin/bash
node ./src/index.js --experimental-modules
And updated my cronjob to read this:
*/5 * * * * /home/lharby/sites/mysite/start.sh >> /home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
And I tried this also trying to bypass the shell script
*/5 * * * * /home/lharby/sites/mysite/src && /usr/local/bin/node index.js --experimental-modules >>/home/lharby/sites/mysite/src/log/cron-errors.txt 2>&1
It still failed with the same message being logged to the cron-errors.txt file.
However I am now able to run this command invoking node with argument flags (rather than using the npm command) So in the terminal I can type
node index.js --experimental-modules
As well as just running the .sh file I just don't understand why it is not passing this information to my cronjob.
I don't understand how if my code reads:
process.env.NEXT_MASTODON_ACCESS_TOKEN,
Will this get replaced or read by the bash export value instead?
Where are your environment variables being set up? It looks like your cron is at least running as expected, and your only problem is to get those env variables into the script correctly. My guess is that you need to use "export" (see bullet point 3)
I would debug this in a few different ways.
you should verify that it is the case that your index.js is not able to read your environment variables. I would recommend adding console.log(JSON.stringify(config, undefined, 2)) to check this.
you should verify that the SHELL has access to those variables before it runs the script. for this just run echo $NEXT_MASTODON_ACCESS_TOKEN (and similar for each variable) to verify if that is the case.
environment variables are a little funny. Assuming you set these values in your .bashrc with NEXT_MASTODON_ACCESS_TOKEN="whateverTheValueIs", just setting variables like this only affects your current process. In order for sub-processes to have the variable you need to export: export NEXT_MASTODON_ACCESS_TOKEN="insertValueHere"
Hope this helps!
Edit:
It's a little unclear what your current issues are, so it might be helpful if you bullet point each issue you are trying to solve
I was trying to run the test from CI/CD gitlab runner file but it is causing issue while executing from gitlab.
I have sucessfully executed the test locally using the karate option
Working fine in Local Run:
mvn test -Dkarate.env=stg +-DKarate.options=--tags #Ui" -Dtest.run.mode=localtest -Dtest.run.group=OKCUtest -Dtest=OKCUtest -Dtest.gitlabRunner=false -DbuildDirectory=stg-target/OKCUtest -Dtest.run.testSource=localtest
There are 5 test feature files which were executed using the #Api tags and now I have identified that one should be #Ui and changed the respective feature file and created the new pipeline OKCU-UI and have updated the command line syntax to address #Ui tests.
can you try this command ?
mvn test -Dkarate.options="--tags ~#Ui"
if still not try same command with version 0.9.6.RC3
Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.
I have a nodejs application that I've built/packaged via teamcity as well as deploy to one of our servers (which has node installed) through the Octopus deploy portal.
Everything works ok until i come to the post deploy powershell script from within octopus.
In one of the code blocks I have npm commands that throws an error npm not recognised.When I run the power-shell scripts on the server, everything works but when I try to run this via the octopus post deploy script I get the error as stated above.
I know that node /npm are installed and that the environment variables re:nodejs is set correctly, unless there is something else i'm missing , it still isn't working.
a simple pseudo-code of what i'm trying to do is this:
$deploymentDir = 'D:\Apps\<appname>\<octopus-version-number>'
$name = "service"
cd $deploymentDir
if($name){
Write-Host "link node-windows..."
npm link node-windows //node-windows is installed globally
}
else{
}
Unless there is something I'm missing , how can I get this to run via octopus ?
There is a possibility that Octopus master/tentacle service runs on different user, check the environment variable for that user also.
If you unable to find the root cause, the alternate is define one Target scoped variable called "npmPath" ( value may be different based on target ) in octopus to store npm path
Now, you can use $npmPath variable in script.
I want to display colored output in jenkins which is produced by node.js
Both work separately, but not combined:
Node Script
My test script test.js:
console.log(require("chalk").red("Node Red"))
Calling the test script in the shell works:
node test.js => OK
Calling a colored shell script in jenkins works:
echo -e "\033[31mShell Red\033[0m" => OK
But calling the node script in jenkins does not display any colors:
node test.js => No Color, when executed in jenkins
For me it worked when putting
export FORCE_COLOR=1
at the top of my script.
See https://github.com/chalk/supports-color#info
The answer of Raphael pointed me in the right direction. Here my complete solution for a Jenkins Pipeline Script (Scripted Pipeline):
:
node {
ansiColor('xterm') {
withEnv(['FORCE_COLOR=3']) {
...
sh "some-node-script-using-chalk.js"
...
}
}
}
If you are using the Declarative Pipeline see https://jenkins.io/doc/pipeline/tour/environment/ how to set environment variables in a Declarative Pipeline Script.
I just found the problem in my case :
In The Job Configuration
Look at the Bindings
Check the checkbox named "Color ANSI Console Output"
And it works (for me...)