I am trying to configure my gitlab-ci to use yarn install instead of npm install
My current gitlab-ci.yml looks like:
image: node:6.9.4
cache:
paths:
- node_modules/
- .yarn
before_script:
- apt-get update -qq && apt-get install -qy libelf1
stages:
- test
test_core:
stage: test
script:
- yarn config set cache-folder .yarn
- yarn install
- npm run build
- npm run test
tags:
- 2gb
But the build fails with the error:
/bin/bash: line 48: yarn: command not found
is there something I am missing?
I tried installing yarn with:
curl -o- -L https://yarnpkg.com/install.sh | bash
this gave me same error, probably because I need to reload the bash environment for the yarn command to become available.
The above config works perfect with npm install.
Please help me resolve this. If there is something missing in my config file or there is something wrong with the gitlab-ci.
Thanks.
Solved it by using the latest official node docker image.
Since image: 6.10.0, yarn is installed by default in the image.
But if you need node-gyp to build any package, it needs to be installed by adding a line to the script:
yarn global add node-gyp
Add the following to your ci script after yarn got installed:
export PATH=$HOME/.yarn/bin:$PATH
I use image:node:latest and sometimes it prompts the error. Clear Runner Caches do the job for me. Maybe the runner did not recover to the correct state after doing other jobs.
I solved it by using npx (package runner). It's better then extending docker-image only for this purpose
npx yarn someCommand
Related
I have a project that requires npm and gradle for build, and docker for building and pushing the image.
At first I thought that I should create my own ubuntu image with gradle and npm setup, but I found out that is not what docker images are for.
So I hoped to run official Gradle and node images as a service so that my script can call those commands, but that is not happening for some reason.
My .gitlab-ci.yml:
variables:
IMAGE_NAME: my.registry.production/project
IMAGE_TAG: $CI_COMMIT_BRANCH
GIT_SUBMODULE_STRATEGY: recursive
stages:
- build
- deploy
build_project:
stage: build
image: ubuntu:jammy
services:
- name: node:12.20
alias: npm
- name: gradle:6.3.0-jre8
alias: gradle
before_script:
- git submodule init && git submodule update --remote --recursive
script:
- cd project-server && npm install && gradle clean build -Pprod -Pwar -x test -x integrationTest
deploy_image:
stage: deploy
image: docker:20.10.17
services:
- name: docker:20.10.17-dind
alias: docker
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
script:
- docker login -u $REGISTRY_USER -p $REGISTRY_PASSWORD my.registry.production
- docker build -t $IMAGE_NAME:$IMAGE_TAG .
- docker push $IMAGE_NAME:$IMAGE_TAG
If anyone has any info on how to solve this I would greatly appreciate it, since I’m a novice DevOps.
Edit 1:
My Dockerfile for custom image with Gradle and Node installed.
FROM ubuntu:jammy
LABEL key=DevOps
SHELL ["/bin/bash", "--login", "-i", "-c"]
RUN apt update && apt upgrade -y && apt install curl -y
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
RUN source /root/.bashrc && nvm install 12.14.1
RUN nvm install 12.20.0
RUN apt install zip unzip
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
RUN sdk install java 8.0.302-open
RUN sdk install gradle 3.4.1
SHELL ["/bin/bash", "--login", "-c"]
CMD [ "bin/bash" ]
After I run it, it says that npm is not found in $PATH, I tried Java, Gradle as well but they weren't found in the path as well.
I don't know why since I installed them as you can tell from the Dockerfile.
As I know, a docker image is equal to one build. So if you have multiple services you need to build each one into docker image then you can encapsulate all images into docker-compose.yml file.
I think you can do the following:
Build the npm project into a docker image
Build the Gradle project into a docker image
Write the docker-compose.yml file and put both images.
Once you have done it, the pipeline calls the docker-compose.yml file.
I hope this will be helpful.
Consider a few suggestions based on the fundamental concepts about the deployment in your CI/CD pipeline:
Remove the services keyword. Reference GitLab's official documents on what the services keyword inside gitlab-ci.yaml file is not for. The feature is used
to provide network accessable services to your job runtime (like
a database): https://docs.gitlab.com/ee/ci/services/index.html
Your project uses npm as a dependency management system, Gradle is
a build tool. Both of these pieces of software are more than
appropriate to run on the host operating system of the container
runtime inside GitLab's Pipeline job. You need these tools to assemble some build artifact as a result of the job on the same host your code has been downloaded on in the Runner.
Think about the overall size of the base image in your build_project job and consider how time to download the image over the network on to the Runner will impact your job and overall pipeline duration. If performance can be improved by baking build dependencies into a custom Dockerfile do this. If your image is too large, instead use shell commands inside the script keyword block to download them at the runtime of the job. There can be pros and cons for both.
Break shell scripts to one command per line for easier troubleshooting of failures in your scripts. You will be able to see the line number of the command which returned a non-zero exit code in your job logs:
...
script:
- cd project-server
- npm install
- gradle clean build -Pprod -Pwar -x test -x integrationTest
...
It's recommended to use the Gradle wrapper (gradlew) most of the time instead of the gradle executable directly. Configure this within your project and check the configuration files for the wrapper into your version control system and this will simplify your build dependency: https://docs.gradle.org/current/userguide/gradle_wrapper.html
For some reason Circle is not able to use ts-mocha installed with npm install in a previos step in the building process.
It used to work, but for some reason it doesn't anymore out of a sudden.
This is the CircleCI build job result:
All tests are running fine locally:
This is the script in the package.json that I run with npm test:
"test": "env NODE_ENV=test ts-mocha ./test/**/*.spec.ts --timeout 10000"
The package version is "ts-mocha": "^6.0.0",
This is my CircleCI job configuration (which hasn't changed in a month):
jobs:
build:
docker:
- image: circleci/node:10.13.0
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package-lock.json" }}
- v1-dependencies-
- run: npm install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package-lock.json" }}
- run: npm test
- run: npx tsc
It seems that something broke from the Circle side, as there were no changes in the code.
Even though I tried rerunning old successful builds, they fail with this same error.
Things I tried:
Runningnpm ci && npm test in the same step but it yields the same result.
Removing package-lock.json
Updating package-lock.json
Running npm install --no-package-lock
Updating npm
Running npm update
Running npm audit fix
Cleaning npm cache
Also tried using npx instead of relying on the previously installed ts-mocha package and this is the result:
I noticed that the CircleCI NODE_ENV environment variable was set to production, therefore any devDependencies were not getting installed (even with npm install --save, because it was already listed as a devDependency in the package.json).
I don't know when the environment variable was changed to that value, but the odd thing is that it started breaking from one day to another (although it should've been breaking since the moment that env variable was set) so it was extremely hard to debug, but it was a simple fix: changing the NODE_ENV environment variable in CircleCI to something different than production.
I'm sure this is a common issue but I can't seem to find a definitive answer.
I have a node application which in order to build requires some devDependencies such as babel. In order to run my tests also requires devDependencies such as jest. But when CI runs in production environment, it obviously doesn't install any devDependencies so I get errors where the package isn't found.
What is the best practice for running builds and tests in prod without devDependencies?
If it helps, I am running my build in GitLab Pipelines:
image: node:8.11.2
stages:
- prepare
- test
- deploy
install_and_build:
stage: prepare
script:
- npm install yarn
- yarn
- yarn build
only:
- master
test:
stage: test
script:
- yarn test
only:
- master
deploy_production:
type: deploy
stage: deploy
image: ruby:latest
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=app-name --api-key=$HEROKU_API_KEY
only:
- master
From this answer,
You will want to follow have your process to the effect of the
following:
First, you need to "install with all dependencies".
npm install
Then do your tests.
npm test
Then "prune" your dev dependencies as below, as detailed in the
docs doing this
"will remove the packages specified in your devDependencies".
npm prune --production
I am attempting to use gulp inside a Docker container.
I have the following Dockerfile
FROM golang:alpine
RUN apk --update add --no-cache git nodejs
RUN npm install --global gulp
ENV GOPATH=/go PATH=$PATH:/go/bin
VOLUME ["/go/src/github.com/me/sandbox", "/go/pkg","/go/bin"]
WORKDIR /go/src/github.com/me/sandbox
CMD ["gulp"]
and I have the following docker-compose.yml
version: '2'
services:
service:
build: ./service
volumes:
- ./service/src/:/go/src/github.com/me/sandbox
docker-compose build builds successfully, but when I run docker-compose up, I get the following error message
Recreating sandbox_service_1
Attaching to sandbox_service_1
service_1 | [22:03:40] Local gulp not found in /go/src/github.com/me/sandbox
service_1 | [22:03:40] Try running: npm install gulp
I have tried several different things to try to fix it.
Tried also installing gulp-cli globally and locally
Tried installing gulp locally with npm install gulp
Tried moving the npm install --global gulp after the WORKDIR
Tried different paths for volumes.
My guess is that it has something to do with the volumes, because when I get rid of anything having to do with a volume, it doesn't complain.
Mr project structure is shown in screenshot below:
This is what worked for me.
RUN npm install -g gulp
RUN npm link gulp
You need a local version of gulp as well as a global one.
Adding this line should fix your issue
RUN npm i gulp
The application I want to implement continuous deployment on Bamboo has node modules and bower component dependencies. On the bamboo server nodejs, npm have been installed.
There are only three tasks on default job:
Source Code Checkout
Build dependencies:
npm install
bower install
Deploy to the staging server
The problem is on the second task, bamboo fails with the message "No failed tests found, a possible compilation error occurred." I don't even run any tests.
The log file is not explanatory at all:
Starting task 'Build dependencies' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'
Failing task since return code of [/bin/sh /home/ubuntu/bamboo-installation/temp/WEB-WEB-JOB1-8-ScriptBuildTask-4430338079602360707.sh] was 1 while expected 0
Ok, I solved the problem. The issue was the wrong node (which obviously messed things up) was installed on the bamboo server. Uninstalled the wrong one and everything worked as expected.
Good to see you solved it.
There is a setup I use and which could prevent further problems with CI:
export npm_config_prefix=.npm/
export PATH=.npm/bin:$PATH
export CI=true
npm install -g bower
bower install
npm install
This installs bower (and others like grunt-cli if you want) in your project folder so you can e.g. have a specific version, sets CI=true as advised in bower docs, and then installs all dependencies.
Bamboo AMI originally have npm version 1.4.28 installed and you are probably using a more recent version on you development environment. I had the same issue and resolved it by creating a script task to update npm version on the very beginning of my build process. Here is the script:
# update npm
curl -O -L https://npmjs.org/install.sh
chmod +x install.sh
sudo PATH=$PATH:/opt/node-0.10/bin ./install.sh