I'd really like to get yarn working with TeamCity or Jenkins, however
yarn install consumes all my build server's cpu & memory, even with
a tiny npm project. Any idea why?
Summary
yarn install, when run by TeamCity or Jenkins, consumes most CPU/memory, never finishes
running yarn install from the command line on the build server works (finishes in a few seconds)
npm install completes in a few seconds when run by TeamCity/Jenkins
build server: Windows Server 2012, 2 cores, 4GB mem
yarn version 1.6.0
Jenkins version 2.107.2
TeamCity version 2017.2.3
Have tried Node.js versions 8.11.1 and 9.11.1
Details
Steps:
Create a tiny npm project:
mkdir temp
cd temp
yarn init # all defaults
yarn add dummy-module # tiny test package
Commit this to git, set up a Jenkins or TeamCity build with a single command yarn install.
The build never finishes, with the last line of console output: yarn install v1.6.0.
At this time, the Node.js process is consuming most of the CPU and memory on the
build machine. Changing the command to npm install makes it run successfully.
Since it's common to Jenkins and TeamCity, it seems like the problem is to do with
running sub processes from java. I can't find any indication of what's going wrong
in either the TeamCity or Jenkins logs. When aborting the Jenkins build,
I see the following in jenkins.err.log:
Apr 22, 2018 8:59:11 AM hudson.model.Run execute
INFO: master-cake #9 aborted
java.lang.InterruptedException
at java.lang.ProcessImpl.waitFor(Unknown Source)
at hudson.Proc$LocalProc.join(Proc.java:324)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:155)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1727)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
I left some crucial information out of the original question - I'm also using yarn's offline mirror, which was stored on a network drive. It turns out that the TeamCity windows service can't access network drives: https://confluence.jetbrains.com/display/TCD10/Known+Issues#KnownIssues-AgentrunningasWindowsServiceLimitations
Relocating yarn's offline mirror to a local directory solved the issue for both TeamCity and Jenkins.
Related
I am trying to implement a faucet for my local testnet which consists of 5 node. To do that I followed the avalances github documentation as in the link,
https://github.com/ava-labs/avalanche-faucet
In short, basically, I run my 5 nodes and all nodes runs successfully using an Ubuntu Sandbox. Also, the bootstrap is finished. Then tried to run the below command in the Ubuntu,
Platform Versions: Ubuntu 20.04.2 LTS
node-js v16.2.0
npm 7.15.0
Yarn 1.22.5
Creating Faucet:
Clone the repository git clone https://github.com/ava-labs/avalanche-faucet.git
Go to the root directory cd avalanche-faucet
Install javascript dependencies with yarn install.
Create a .env file by copying .env.example
Install AvalancheGo, our Avalanche node client written in Golang to spin up a network (https://github.com/ava-labs/avalanchego).
Running Project:
Make sure you have installed and able to run an Avalanche node properly.
All environment variables are correct and your private key has funds in it.
Run the project with hot reloading yarn serve
Everything is goes ok but when I run yarn serve then the below Error pop up. "Assertion `thread_id_key != 0x7777' failed."
43% building 275/282 modules 7 active ...cet/node_modules/babel-loader/lib/index.js!/home/avalanchelocal1/Downloads/avalanche-faucet/node_modules/vuetify/lib/services/index.jsnode: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed.
Aborted (core dumped)
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
I have a Jenkins server on which I observe a private git repository for changes, which then triggers a pipeline script (the repository contains a nodejs app). In this pipeline script I need to do the following steps:
Install dependencies (npm install)
Build my application (npm run build, which creates a dist folder)
Build a docker container (docker build) and run the container (which runs a script in the dist folder)
Which of the following two options would be the recommended way to do this, and why?
Option A: Run npm install and npm run build in the jenkins pipeline and copy the dist folder to the docker container during the docker build. This would allow me to only install runtime dependencies in the docker container using npm install --only=production, therefore reducing the image size significantly.
Option B: Run npm install and npm run build during docker build (In the Dockerfile). This would allow me to run the docker container outside the CI server if I have to (I don't have a use case for it now, but it seems cleaner because it is more independent). However, the image size would significantly increase and I am not sure if this is the recommended way.
Any suggestions?
I would choose option B.
The reason behind it would be that there are some npm packages that runs a node-gyp, gcc, and other platform-dependent builds.
Look at the popular bcrypt package as an example.
Going with option A would mean that your docker and Jenkins machine need to hold the same infra for such builds which is not common, to say the least.
So I have a MEAN application up and running and Im looking into a continuous integration solution. I have successfully gotten Jenkins up and running with web hooks that grab my project from a bitbucket repo when a merge happens to master.
Right now I do not have any tests so Jenkins just runs some shell commands that 'deploys' the server. Which is great. My goal would be to have this run tests and fail a deploy if they fail.
So my problem is that the build never completes. My goal would be to when it completes it will keep the server running or deploy it and keep it running.
Here are the shell commands I run one the build is kicked off.
npm install
npm install bower
bower install
npm install grunt-cli
grunt prod
node server
And it successfully runs the server and such but it just hangs up after the node server command is executed
How do I make it so Jenkins sees this as successful and then deploys it? I have crawled the internet with no much luck.
EDIT:
So looking at some docs and such. I would need to configure my tests to run when the build gets ran. If there are not tests then it passes (by default)... So what I need is when that happens, jenkins needs to run a deploy script. After looking around in jenkins I am still unable to figure out how to do so.
EDIT #2
So moving those shell script out of the build allows it to finish and is 'successful' since no tests are present. I see that jenkins keeps the project in a workspace directory. Is there a way to get jenkins to deploy from there or some kind of other application to deploy that build in that workspace?
Thanks
So what I ended up doing was something simple but im not sure if its best practice..
Jenkins has the webhook to my bitbucket repo and watches for pulls into master. This then kicks off my jenkins build which it runs
npm install bower
bower install
npm install grunt-cli
grunt prod
which builds the project. Then I installed nodemon which watches the last "successful build" folder run the server from there. When it gets refreshed the server restarts. This seems to run pretty smoothly so far.
The application I want to implement continuous deployment on Bamboo has node modules and bower component dependencies. On the bamboo server nodejs, npm have been installed.
There are only three tasks on default job:
Source Code Checkout
Build dependencies:
npm install
bower install
Deploy to the staging server
The problem is on the second task, bamboo fails with the message "No failed tests found, a possible compilation error occurred." I don't even run any tests.
The log file is not explanatory at all:
Starting task 'Build dependencies' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'
Failing task since return code of [/bin/sh /home/ubuntu/bamboo-installation/temp/WEB-WEB-JOB1-8-ScriptBuildTask-4430338079602360707.sh] was 1 while expected 0
Ok, I solved the problem. The issue was the wrong node (which obviously messed things up) was installed on the bamboo server. Uninstalled the wrong one and everything worked as expected.
Good to see you solved it.
There is a setup I use and which could prevent further problems with CI:
export npm_config_prefix=.npm/
export PATH=.npm/bin:$PATH
export CI=true
npm install -g bower
bower install
npm install
This installs bower (and others like grunt-cli if you want) in your project folder so you can e.g. have a specific version, sets CI=true as advised in bower docs, and then installs all dependencies.
Bamboo AMI originally have npm version 1.4.28 installed and you are probably using a more recent version on you development environment. I had the same issue and resolved it by creating a script task to update npm version on the very beginning of my build process. Here is the script:
# update npm
curl -O -L https://npmjs.org/install.sh
chmod +x install.sh
sudo PATH=$PATH:/opt/node-0.10/bin ./install.sh
I've setup Jenkins v1.550 on Windows Server 2008 R2. It runs as a service at http://localhost:8080 for now. I'm logged into the machine as an Administrator. I've installed Node.js and can run "npm" from the command line.
I've also installed the NodeJS plugin v0.2.1 for Jenkins. I then went into the Configure System section of Jenkins, scrolled down to NodeJS installations, clicked on Add NodeJS button, gave "NodeJS" as the name, and "C:\Program Files\nodejs" as the path to the installation directory. I didn't check the "Install automatically" option as I read on the plugin page that it is only available to Linux.
I then created a new job, clicked the checkbox that said "Provide Node & npm bin/ folder to PATH", created a new build step for "Execute Windows batch command" and typed in "node --version" and "grunt --version" and saved it.
I ran the job and this is the output -
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_1.0
[Test_1.0] $ cmd /c call C:\Windows\TEMP\hudson1381541243088903083.bat
C:\Program Files (x86)\Jenkins\workspace\Test_1.0>node --version
v0.10.24
C:\Program Files (x86)\Jenkins\workspace\Test_1.0>grunt --version
'grunt' is not recognized as an internal or external command,
operable program or batch file.
C:\Program Files (x86)\Jenkins\workspace\Test_1.0>exit 9009
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
It looks like it's unable to find the grunt-cli for the user account Jenkins is running under (System). I tried to installing grunt cli globally (npm install -g grunt-cli) and also grunt locally (npm install grunt). No luck.
Can someone please help?
for nice easy to configure self-installed nodejs on the machine, i have to recommend the excellent -> http://wiki.jenkins-ci.org/display/JENKINS/NodeJS+Plugin
it will install nodejs and grunt on the machine, through easy to use web front end no shell required
jenkins jobs can then simply run nodejs build steps, hey presto
steps involved :
a) install this on your jenkins instance -> http://wiki.jenkins-ci.org/display/JENKINS/NodeJS+Plugin
b) create a nodejs installation on jenkins
go to
http://URL_OF_JENKINS/jenkins/configure
NodeJS- > NodeJS installations -> Add NodeJS -> Name = "NodeJS 0.11.10", tick "Install automatically", select "Install from nodejs.org", add "grunt-cli" to globally installed packages
c) create a job with "execute NodeJS script" build task
var sys = require('sys');
sys.puts('NodeJS Test');
sys.puts('***************');
sys.puts('helloworld');
volia :)
run the job and see the nodejs script run,
from their the world is your oyster you can use grunt by ticking "Provide Node/npm bin folder to PATH" and running a "execute shell" build task
npm update
grunt
grunt --force reporting
You will need to restart the Jenkins service after installing node, presumably to cause it to refresh its cached copy of your PATH environment variable
I have grunt doing some tasks for me in Jenkins, but I went the npm script route. Grunt and grunt-cli are dev dependencies, and I have the following defined in my package.json file:
"scripts": {
"test": "node node_modules/grunt-cli/bin/grunt test"
},
In Jenkins (running on Windows), I added two post-build tasks:
npm install
npm test
We just installed NodeJs normally on the Jenkins server.
Another solution that worked for me on Windows is to use the full path to the grunt exec file, which can be found by writing "where grunt" in the command shell. I used the full path in the regular bat-file.
Had the same issue on Windows. When I manually installed node and ran npm install -g grunt-cli from command line, jenkins could not recognize the grunt command. So uninstall node, reinstall it but dont run npm install. Then restart the jenkins slave. Then from the jenkins job that runs on your specific jenkins slave, make it run a Windows batch command that runs npm install -g grunt-cli After that again restart the jenkins service. Then from the job run npm install. Then everything worked for me. If issues still persist, then uninstall the slave, and reinstall it, then everything works fine immediately.