Consider the following builds:
3rd build without npm cache
3rd build with npm cache
These two repositories are almost identical, with the only difference being the latter repository caches npm via the setup-node GitHub Action, whereas the former one does not. In other words, the only difference between the repositories is at the .github/workflows/main file:
name: Build Pipeline
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '11'
# Following line is present only on the latter repository
cache: 'npm'
- run: npm install
- run: npm run build
Although the build at "setup-node-with-cache" successfully uses npm cache (as evident by the output of the Run actions/setup-node#v2 step), run time of Run npm install step is almost the same as the corresponding step of the build at "setup-node-without-cache".
Isn’t the run time of Run npm install step of the build at "setup-node-with-cache" supposed to be significantly shorter than the corresponding step of the build at "setup-node-without-cache", since it is supposed to use the cached npm packages? Am I missing something here?
It really does not seem to be, at least on GitHub Actions. The workaround that I found is caching the actual node_modules folder, even though it is "not recommended". Caching the actual node_modules folder does speed up npm install.
Related
I have a bitbucket pipelines yml that I have step for running my test script and a step to run a serverless deploy script. Do I need to npm install at each step or will the first npm install carry through and suffice for each subsequent step. Further than that, what is happening under the hood? I know Docker container is created; does each step just update the container?
- step:
name: Test and Build
script:
- npm install --no-package-lock
- npm run test
- step:
name: Deploy Serverless
script:
- npm i serverless -g
- npm install --no-package-lock
- npm run deploy
Can you implement it like the documentation: https://confluence.atlassian.com/bitbucket/caching-dependencies-895552876.html
The functionality is there. Let me know if it doesn't work for you still.
Each step in the pipe creates a separate docker container which pulls in your branch. Using the cache option will allow your pipe to skip the install when building the container for the second step by pulling node_modules from the cache. You must still include the npm install line in each step to tell the pipe to use the cache if it exists.
For some reason Circle is not able to use ts-mocha installed with npm install in a previos step in the building process.
It used to work, but for some reason it doesn't anymore out of a sudden.
This is the CircleCI build job result:
All tests are running fine locally:
This is the script in the package.json that I run with npm test:
"test": "env NODE_ENV=test ts-mocha ./test/**/*.spec.ts --timeout 10000"
The package version is "ts-mocha": "^6.0.0",
This is my CircleCI job configuration (which hasn't changed in a month):
jobs:
build:
docker:
- image: circleci/node:10.13.0
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package-lock.json" }}
- v1-dependencies-
- run: npm install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package-lock.json" }}
- run: npm test
- run: npx tsc
It seems that something broke from the Circle side, as there were no changes in the code.
Even though I tried rerunning old successful builds, they fail with this same error.
Things I tried:
Runningnpm ci && npm test in the same step but it yields the same result.
Removing package-lock.json
Updating package-lock.json
Running npm install --no-package-lock
Updating npm
Running npm update
Running npm audit fix
Cleaning npm cache
Also tried using npx instead of relying on the previously installed ts-mocha package and this is the result:
I noticed that the CircleCI NODE_ENV environment variable was set to production, therefore any devDependencies were not getting installed (even with npm install --save, because it was already listed as a devDependency in the package.json).
I don't know when the environment variable was changed to that value, but the odd thing is that it started breaking from one day to another (although it should've been breaking since the moment that env variable was set) so it was extremely hard to debug, but it was a simple fix: changing the NODE_ENV environment variable in CircleCI to something different than production.
I am trying to setup CI on Azure DevOps Pipelines, on Hosted Ubuntu agent for a nodejs project with auto incrementing version numbers.
I am getting the errors on the incrementing the version using npm version patch
[command]/opt/hostedtoolcache/node/8.12.0/x64/bin/npm run release-it
npm ERR! Git working directory not clean.
npm ERR! M package-lock.json
> identity#2.0.2 release-it /home/vsts/work/1/s
> npm version patch && git push --follow-tags
npm ERR! A complete log of this run can be found in:
The error persists even if I don't push to git.
As a part of the CI, I have cleaned the repo
resources:
- repo: self
clean: true
trigger:
batch: true
branches:
include:
- '*'
pr:
autoCancel: true
branches:
include:
- 'master'
jobs:
- job: identity_release
timeoutInMinutes: 20
workspace:
clean: all
pool:
vmImage: 'ubuntu-16.04'
demands:
- npm
condition: or(eq(variables['Build.Reason'], 'Manual'), eq(variables['Build.Reason'], 'Schedule'))
steps:
- script: echo The build reason is $(Build.Reason) and branch is $(Build.SourceBranch)
- template: release-build-steps.yml
I have cleaned the build sources directory using shell script in CI with no avail.
Any direction will be helpful.
As per npm-version docs the command will fail if the working directory is not clean. I can see from your error log that the package-lock.json file is modified, causing the npm version command to fail.
In order to fix this, either commit the package-lock.json file or use the force flag like this
npm version patch --force
I'm so curious about trigger input on gitlab CI pipeline *cmiiw.
So, the main problem when there's any prompt on Node.js like this,
Ubuntu
But when i'm trying to implementation into Gitlab CI, there's any error something like this
Gitlab.CI
This is my gitlab.ci.yml script
image: node:latest
cache:
paths:
- node_modules/
all_tests:
script:
- npm install
- npm run setup
- John Doe \n
- npm run test
First, CI best-practices suggest you create a --force or --no-interactive variant of your installer, to omit interactive input in case of automated deploys.
A workaround could be to use the yes unix util. This util lets you feed a string to interactive input like this (in your case):
image: node:latest
cache:
paths:
- node_modules/
all_tests:
script:
- npm install
- yes 'Gitlab CI' | npm run setup
- npm run test
This will answer 'Gitlab CI' to all questions asked so its rather limited.
Btw; I think you mean .gitlab-ci.yml instead of travis.ci.yml in your question?
I think my question is related to these ones but the solution did not work in my case: How to run Node.js and Ruby tests within one project on Travis CI and Is it possible to set up travis to run tests for several languages?
I have a github repo that contains a little Golang application with a tiny Angularjs frontend. I want to run both go-tests and Karma-tests.
I see two options:
Run two travis-ci builds for one repo (I could not figure out how to
do that)
Run both tests in one build (did not work since the node version in travis-ci is too old (0.10).
The build runs for 60+ minutes and then stops with "FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory". The build shows tons of warnings like this one: npm WARN engine escodegen#1.8.0: wanted: {"node":">=0.12.0"} (current: {"node":"0.10.36","npm":"1.4.28"})
I tried to run the build as node_js but then the "go get xyz" does not work.
my .travis.yml file:
language: go
go:
- 1.5
env:
- TRAVIS_NODE_VERSION="0.12"
install:
- export PATH=$HOME/gopath/bin:$PATH
- go get golang.org/x/tools/cmd/cover
- go get -v github.com/axw/gocov
- go install github.com/axw/gocov/gocov
- go get github.com/GeertJohan/go.rice
# we do not need the rice tool!
- go get github.com/xeipuuv/gojsonschema
- go get github.com/finklabs/ttime
- go get github.com/finklabs/graceful
- go get github.com/gorilla/mux
before_script:
- npm install bower
- npm install --dev
- bower install
script:
- gocov test | gocov report
- npm test
sometimes the problem sits in front of the machine... I found a great workaround and I want to share it with you.
I created an account on https://drone.io/ in the settings tab I selected a build for "Go1" and added the following commands:
# install gogrinder dependencies
go get ./...
# install test dependencies
go get golang.org/x/tools/cmd/cover
go get -v github.com/axw/gocov
go install github.com/axw/gocov/gocov
# install nodejs requirements
npm -d install
./node_modules/bower/bin/bower install
# run the tests on the go code
gocov test | gocov report
# run the tests on the Angularjs frontend
npm test
Now I am back on track with a CI server that runs both Golang and Karma tests.
I gave a solution to this on http://entulho.fiatjaf.alhur.es/guias/how-to-use-node-along-with-other-language-on-travis-ci/ a long time ago, but it still works today:
Basically you just add
install:
- . $HOME/.nvm/nvm.sh
- nvm install stable
- nvm use stable
- npm install
to your .travis.yml, which must be set to a different language, and it will install node and npm.