I'm trying to setup GitLab CI for a mono repository.
For the sake of the argument, lets say I want to process 2 JavaScript packages:
app
cli
I have defined 3 stages:
install
test
build
deploy
Because I'm reusing the files from previous steps, I use the GitLab cache.
My configuration looks like this:
stages:
- install
- test
- build
- deploy
install_app:
stage: install
image: node:8.9
cache:
policy: push
paths:
- app/node_modules
script:
- cd app
- npm install
install_cli:
stage: install
image: node:8.9
cache:
policy: push
paths:
- cli/node_modules
script:
- cd cli
- npm install
test_app:
image: node:8.9
cache:
policy: pull
paths:
- app/node_modules
script:
- cd app
- npm test
test_cli:
image: node:8.9
cache:
policy: pull
paths:
- cli/node_modules
script:
- cd cli
- npm test
build_app:
stage: build
image: node:8.9
cache:
paths:
- app/node_modules
- app/build
script:
- cd app
- npm run build
deploy_app:
stage: deploy
image: registry.gitlab.com/my/gcloud/image
only:
- master
environment:
name: staging
url: https://example.com
cache:
policy: pull
paths:
- app/build
script:
- gcloud app deploy app/build/app.yaml
--verbosity info
--version master
--promote
--stop-previous-version
--quiet
--project "$GOOGLE_CLOUD_PROJECT"
The problem is in the test stage. Most of the time the test_app job fails, because the app/node_modules directory is missing. Sometimes a retry works, but mostly not.
Also, I would like to use two caches for the build_app job. I want to pull app/node_modules and push app/build. I can't find a way to accomplish this. This makes me feel like I don't fully understand how the cache works.
Why are my cache files gone? Do I misunderstand how GitLab CI cache works?
The cache is provided on a best-effort basis, so don't expect that the cache will be always present.
If you have hard dependencies between jobs, use artifacts and dependencies.
Anyway, if it is just for node_modules, I suggest you to install it in every step, instead of using artifacts - you will not save much time with artifacts.
Related
I've been trying to setup a CI/CD Pipeline on my Repo which runs common tasks like linting/tests etc. I've successfully setup a Gitlab-Runner which is working fine. The only part I'm stuck is the "deploy" part.
When I run my build, how do I actually get the files into my /var/www/xyz folder.
I get that everything is running in a Docker Container and I can't just magically copy paste my files there, but i don't get how I get the files on my actual server-directory. I've been searching for days for good docs / explanations, so as always, StackOverflow is my last resort for help.
I'm running on a Ubuntu 20.04 LTS VPS and a SaaS GitLab-Repository if that info is needed. That's my .gitlab-ci.yml:
image: timbru31/node-alpine-git
before_script:
- git fetch origin
stages:
- setup
- test
- build
- deploy
#All Setup Jobs
Install Dependencies:
stage: setup
interruptible: true
script:
- npm install
- npm i -g #nrwl/cli
artifacts:
paths:
- node_modules/
# All Test Jobs
Lint:
stage: test
script: npx nx run nx-fun:lint
Tests:
stage: test
script: npx nx run nx-fun:test
Deploy:
stage: build
script:
- ls /var/www/
- npx nx build --prod --output-path=dist/
- cp -r dist/* /var/www/html/neostax/
only:
refs:
- master
Normally I would ssh into my server, run the build, and then copy the build to the corresponding web-directory.
TL;DR - How do I get files from a GitLab-Runner to an actual directory on the server?
since migrating our test from bitbucket to gitlab the video is no longer recorded during runs in the pipeline. has anyone encountered a similar problem? cypress version 7.3.0
stages:
- build
- test
variables:
npm_config_cache: "$CI_PROJECT_DIR/.npm"
CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .cache/*
- cache/Cypress
- node_modules
- build
image: cypress/browsers:node14.15.0-chrome86-ff82
stage: build
script:
- yarn install
- npx cypress cache path
- npx cypress cache list
phone-sanity-tests-development:
image: cypress/browsers:node14.15.0-chrome86-ff82
stage: test
parallel: 15
script:
- yarn cypress:run-phone-development-sanity
artifacts:
paths:
- cypress/screenshots/**
- cypress/videos/**
- cypress/reports/**
- cypress/projects/phone/puppeteer/videos/**
Here: - yarn cypress:run-phone-development-sanity you need to add --record.
In order to tell cypress to record and make screenshots you need to configure this on the run command in yml file.
This link is nice example how Cypress team configures their gitlab-ci.yml:
https://github.com/cypress-io/cypress-realworld-app/blob/develop/.gitlab-ci.yml
I have a pipeline that does 2 main things: 1) builds a static site using content from an external provider and 2) builds a docker container from that static site.
At the moment, I have these steps in 2 stages, and the build stage produces an artifact
stages:
- build
- package
build:
stage: build
image: node:12
script:
- npm ci
- npm run build
artifacts:
untracked: true
paths:
- folder/for/project
- folder/that/was/not/there/before/build/time
package:
stage: package
image: docker:stable
services:
- docker:dind
needs:
- build
script:
- export DOCKER_HOST=tcp://docker:2375/
- docker build -t my-project .
I can't get the package stage to see the built files though - the docker build -t my-project command will build a version where folder/for/project is present but folder/that/was/not/there/before/build/time is not. Downloading the artifact after the build step is completed does give me both folders, so clearly it is exporting the right stuff from that step and not importing it into the next.
The CI log for the package stage does say that it's downloading something but I can't tell where it goes or how to access it (someID and tokenHere match values seen in the upload artifacts bit of the build step)
Downloading artifacts for build (someID)...
Downloading artifacts from coordinator... ok id=someID responseStatus=200 OK token=tokenHere
How do I pass these files from one stage in my pipeline to the next?
I've now managed to get this working, hopefully this will help someone else in the future. The issue was that I had named my jobs the same names as my stages (build, test, etc) so GitLab must have been getting confused. Using the example in the question, the working solution would be:
stages:
- build
- package
build-job:
stage: build
image: node:12
script:
- npm ci
- npm run build
artifacts:
untracked: true
paths:
- folder/for/project
- folder/that/was/not/there/before/build/time
package:
stage: package
image: docker:stable
services:
- docker:dind
dependencies:
- build-job
script:
- export DOCKER_HOST=tcp://docker:2375/
- docker build -t my-project .
Notice I've changed the build job to build-job and updated the dependency name, and the stage names have remained the same.
Hi guys i was wondering if it is possible to make automatic deploys from gitlab to heroku (react app), right now this is my gitlab-ci
image: node:8.10.0-alpine
cache:
key: "alpine"
paths:
- node_modules/
before_script:
- npm install
build:
stage: build
artifacts:
paths:
- dist/
script:
- npm run build
only:
- master
tags:
- docker
is there a way to push the builded proyect to heroku?
I have problem with test scss-lint in my project on nodejs.
When tests reach scss-lint, it gives an error.
How to make sure that tests do not fall with the successful result of the test itself?
My gitlab-ci.yml
image: node:wheezy
cache:
paths:
- node_modules/
stages:
- build
- test
gem_lint:
image: ruby:latest
stage: build
script:
- gem install scss_lint
artifacts:
paths:
- node_modules/
only:
- dev
except:
- master
install_dependencies:
stage: build
script:
- npm install
artifacts:
paths:
- node_modules/
only:
- dev
except:
- master
scss-lint:
stage: test
script:
- npm run lint:scss-lint
artifacts:
paths:
- node_modules/
only:
- dev
except:
- master
You are doing it wrong.
Each job you define (gem_lint, install_dependencies, and scss-lint) is run with its own context.
So your problem here is that during the last step, it doesn't find the scss-lint gem you installed because it switched its context.
You should execute all the scripts at the same time in the same context :
script:
- gem install scss_lint
- npm install
- npm run lint:scss-lint
Of course for this you need to have a docker image that has both npm and gem installed maybe you can find one on docker hub), or you can choose one (for example : ruby:latest) and add as the first script another one that would install npm :
- curl -sL https://deb.nodesource.com/setup_6.x | bash -