CircleCI: $ npm test fails and doesn't find installed package - node.js

For some reason Circle is not able to use ts-mocha installed with npm install in a previos step in the building process.
It used to work, but for some reason it doesn't anymore out of a sudden.
This is the CircleCI build job result:
All tests are running fine locally:
This is the script in the package.json that I run with npm test:
"test": "env NODE_ENV=test ts-mocha ./test/**/*.spec.ts --timeout 10000"
The package version is "ts-mocha": "^6.0.0",
This is my CircleCI job configuration (which hasn't changed in a month):
jobs:
build:
docker:
- image: circleci/node:10.13.0
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package-lock.json" }}
- v1-dependencies-
- run: npm install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package-lock.json" }}
- run: npm test
- run: npx tsc
It seems that something broke from the Circle side, as there were no changes in the code.
Even though I tried rerunning old successful builds, they fail with this same error.
Things I tried:
Runningnpm ci && npm test in the same step but it yields the same result.
Removing package-lock.json
Updating package-lock.json
Running npm install --no-package-lock
Updating npm
Running npm update
Running npm audit fix
Cleaning npm cache
Also tried using npx instead of relying on the previously installed ts-mocha package and this is the result:

I noticed that the CircleCI NODE_ENV environment variable was set to production, therefore any devDependencies were not getting installed (even with npm install --save, because it was already listed as a devDependency in the package.json).
I don't know when the environment variable was changed to that value, but the odd thing is that it started breaking from one day to another (although it should've been breaking since the moment that env variable was set) so it was extremely hard to debug, but it was a simple fix: changing the NODE_ENV environment variable in CircleCI to something different than production.

Related

Does the npm cache speed up `npm install`?

Consider the following builds:
3rd build without npm cache
3rd build with npm cache
These two repositories are almost identical, with the only difference being the latter repository caches npm via the setup-node GitHub Action, whereas the former one does not. In other words, the only difference between the repositories is at the .github/workflows/main file:
name: Build Pipeline
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '11'
# Following line is present only on the latter repository
cache: 'npm'
- run: npm install
- run: npm run build
Although the build at "setup-node-with-cache" successfully uses npm cache (as evident by the output of the Run actions/setup-node#v2 step), run time of Run npm install step is almost the same as the corresponding step of the build at "setup-node-without-cache".
Isn’t the run time of Run npm install step of the build at "setup-node-with-cache" supposed to be significantly shorter than the corresponding step of the build at "setup-node-without-cache", since it is supposed to use the cached npm packages? Am I missing something here?
It really does not seem to be, at least on GitHub Actions. The workaround that I found is caching the actual node_modules folder, even though it is "not recommended". Caching the actual node_modules folder does speed up npm install.

yarn command not found in gitlab ci

I am trying to configure my gitlab-ci to use yarn install instead of npm install
My current gitlab-ci.yml looks like:
image: node:6.9.4
cache:
paths:
- node_modules/
- .yarn
before_script:
- apt-get update -qq && apt-get install -qy libelf1
stages:
- test
test_core:
stage: test
script:
- yarn config set cache-folder .yarn
- yarn install
- npm run build
- npm run test
tags:
- 2gb
But the build fails with the error:
/bin/bash: line 48: yarn: command not found
is there something I am missing?
I tried installing yarn with:
curl -o- -L https://yarnpkg.com/install.sh | bash
this gave me same error, probably because I need to reload the bash environment for the yarn command to become available.
The above config works perfect with npm install.
Please help me resolve this. If there is something missing in my config file or there is something wrong with the gitlab-ci.
Thanks.
Solved it by using the latest official node docker image.
Since image: 6.10.0, yarn is installed by default in the image.
But if you need node-gyp to build any package, it needs to be installed by adding a line to the script:
yarn global add node-gyp
Add the following to your ci script after yarn got installed:
export PATH=$HOME/.yarn/bin:$PATH
I use image:node:latest and sometimes it prompts the error. Clear Runner Caches do the job for me. Maybe the runner did not recover to the correct state after doing other jobs.
I solved it by using npx (package runner). It's better then extending docker-image only for this purpose
npx yarn someCommand

Docker Compose w/ Gulp - Local gulp not found

I am attempting to use gulp inside a Docker container.
I have the following Dockerfile
FROM golang:alpine
RUN apk --update add --no-cache git nodejs
RUN npm install --global gulp
ENV GOPATH=/go PATH=$PATH:/go/bin
VOLUME ["/go/src/github.com/me/sandbox", "/go/pkg","/go/bin"]
WORKDIR /go/src/github.com/me/sandbox
CMD ["gulp"]
and I have the following docker-compose.yml
version: '2'
services:
service:
build: ./service
volumes:
- ./service/src/:/go/src/github.com/me/sandbox
docker-compose build builds successfully, but when I run docker-compose up, I get the following error message
Recreating sandbox_service_1
Attaching to sandbox_service_1
service_1 | [22:03:40] Local gulp not found in /go/src/github.com/me/sandbox
service_1 | [22:03:40] Try running: npm install gulp
I have tried several different things to try to fix it.
Tried also installing gulp-cli globally and locally
Tried installing gulp locally with npm install gulp
Tried moving the npm install --global gulp after the WORKDIR
Tried different paths for volumes.
My guess is that it has something to do with the volumes, because when I get rid of anything having to do with a volume, it doesn't complain.
Mr project structure is shown in screenshot below:
This is what worked for me.
RUN npm install -g gulp
RUN npm link gulp
You need a local version of gulp as well as a global one.
Adding this line should fix your issue
RUN npm i gulp

Travis CI: Karma testing passing but always get errored build

I've been trying yo integrate travis for CI in my free-source project.
When I push to my repository, all looks ok , but after execute karma start --single-run, the console of the travis don't stop to execute the karma start task.
How to fix this?
.travis.yml
language: node_js
sudo: false
node_js:
- 0.10
script: karma start -–single-run
before_install:
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
before_script:
- npm install -g bower
- npm install
- bower install
Travis Console
Updated Question :
Why in the travis process when i use in the console karma start --single-run the option no overwrite the option singleRun: false in the karma.conf.js? In my local environment this works fine.
Finally I solved, the problem was in my karma.conf.js
singleRun: true,
I change this option from false to true. Frequently when i develop, use singleRun option in my karma configuration file to false for use auto watch option.
The weird thing it's that in my local machine when i run karma with the option explicity in the command line (karma start --single-run), the option in the karma.conf.js is overwritten, but in travis this it's not posible.

How to run Golang and Karma tests within one project on Travis CI

I think my question is related to these ones but the solution did not work in my case: How to run Node.js and Ruby tests within one project on Travis CI and Is it possible to set up travis to run tests for several languages?
I have a github repo that contains a little Golang application with a tiny Angularjs frontend. I want to run both go-tests and Karma-tests.
I see two options:
Run two travis-ci builds for one repo (I could not figure out how to
do that)
Run both tests in one build (did not work since the node version in travis-ci is too old (0.10).
The build runs for 60+ minutes and then stops with "FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory". The build shows tons of warnings like this one: npm WARN engine escodegen#1.8.0: wanted: {"node":">=0.12.0"} (current: {"node":"0.10.36","npm":"1.4.28"})
I tried to run the build as node_js but then the "go get xyz" does not work.
my .travis.yml file:
language: go
go:
- 1.5
env:
- TRAVIS_NODE_VERSION="0.12"
install:
- export PATH=$HOME/gopath/bin:$PATH
- go get golang.org/x/tools/cmd/cover
- go get -v github.com/axw/gocov
- go install github.com/axw/gocov/gocov
- go get github.com/GeertJohan/go.rice
# we do not need the rice tool!
- go get github.com/xeipuuv/gojsonschema
- go get github.com/finklabs/ttime
- go get github.com/finklabs/graceful
- go get github.com/gorilla/mux
before_script:
- npm install bower
- npm install --dev
- bower install
script:
- gocov test | gocov report
- npm test
sometimes the problem sits in front of the machine... I found a great workaround and I want to share it with you.
I created an account on https://drone.io/ in the settings tab I selected a build for "Go1" and added the following commands:
# install gogrinder dependencies
go get ./...
# install test dependencies
go get golang.org/x/tools/cmd/cover
go get -v github.com/axw/gocov
go install github.com/axw/gocov/gocov
# install nodejs requirements
npm -d install
./node_modules/bower/bin/bower install
# run the tests on the go code
gocov test | gocov report
# run the tests on the Angularjs frontend
npm test
Now I am back on track with a CI server that runs both Golang and Karma tests.
I gave a solution to this on http://entulho.fiatjaf.alhur.es/guias/how-to-use-node-along-with-other-language-on-travis-ci/ a long time ago, but it still works today:
Basically you just add
install:
- . $HOME/.nvm/nvm.sh
- nvm install stable
- nvm use stable
- npm install
to your .travis.yml, which must be set to a different language, and it will install node and npm.

Resources