Cannot get GitLab CI to build my Middleman/Gulp project - node.js

I am using Gitlab-CI to build my Middleman application which also uses some node stuff for the front end (Gulp).
Here is my .gitlab-ci.yml (mostly copied from here):
image: ruby:2.3
cache:
paths:
- vendor
- node_modules
before_script:
- apt-get update -yqqq
- apt-get install -y npm
- ln -s /usr/bin/nodejs /usr/bin/node
- npm install
- bundle install --path vendor
test:
script:
- bundle exec middleman build
except:
- master
pages:
script:
- bundle exec middleman build
artifacts:
paths:
- public
only:
- master
Everything goes alright apart from the vital problem that it seems to be using an old version of node when it's npm installing. I'm getting lots of this:
npm WARN engine gulp-babel#7.0.0: wanted: {"node":">=4"} (current: {"node":"0.10.29","npm":"1.4.21"})
Before finally failing on the "const path" SyntaxError.
I included a line to symlink the new nodejs with the old name (- ln -s /usr/bin/nodejs /usr/bin/node) but it seems to have no effect...?
Been banging my head for long enough, there's got to be someone out there who has made this work?

Debian Jessie ships with a fixed NodeJs major version, follow NodeSource instructions to install a specific version, this would fit in your gitlab-ci.yml like this (you probably need to install curl first since its not installed in the ruby:2.3 image):
before_script:
- apt-get update -q && apt-get -qqy install curl
- curl -sL https://deb.nodesource.com/setup_9.x | bash -
- apt-get update -q && apt-get -qqy install nodejs npm
- ln -s /usr/bin/nodejs /usr/bin/node
- npm install
- bundle install --path vendor

Related

How to install nodejs to install a npm package in a gitlab job?

.deploy: &deploy
before_script:
- apt-get update -y
script:
- cd source/
- npm install multi-file-swagger
- multi-file-swagger -o yaml temp.yml > swagger.yml
I want to install multi-file-swagger package to compile the temp.yml( which has been split into multiple files) into the swagger.yml. So before using npm I need to install nodejs. How can I do that?
As the image is Debian-based, you should be able to install the source repo from Node and install the package from there. The relevant section of your Gitlab file would look like this:
.deploy: &deploy
before_script:
- apt-get update -y
script:
- curl -sL https://deb.nodesource.com/setup_17.x | bash
- apt-get install nodejs -yq
- cd source/
- npm install multi-file-swagger
- multi-file-swagger -o yaml temp.yml > swagger.yml
Please note that these additional steps will add a significant amount of time to your build process. If you are executing them more frequently, consider creating your own build image derived from the one you’re using now, and adding these steps into the image itself.

How to run json-server in AWS?

My team has deployed an Angular application in AWS. We used json-server for making mock service and database for the application. But, we can't find a way to run json-server in aws. Can anyone help us in this regard?
This is the pipeline code-
version: 0.0
os: linux
version: 0.2
env:
variables:
S3_BUCKET: "initial-codedeploy-bucket-us-east-1-jready"
APP_NAME: "shopping-city"
BUILD_ENV : "prod"
phases:
install:
commands:
# Download and Install NodeJS 10.0
- curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
- sudo apt-get install -y nodejs
- echo Installing source NPM dependencies...
# Install http drivers for node
- sudo apt-get update -y
- sudo apt-get install -y apt-transport-https
# Install Yarn Package Manager (Replace the commands below if you using NPM).
# - curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
# - echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
- sudo apt-get update -y
- sudo apt-get install -y npm
# Install Angular CLI, If you are not using Angular 6, install angular/cli#1.7.0 or lower, confirm by running ng -v inside your project folder
- npm global add #angular/cli#6.0.8
# Install node dependancies.
- npm install
build:
commands:
# Builds Angular application. You can also build using custom environment here like mock or staging
- echo Build started on `date`
- ng build --${BUILD_ENV}
post_build:
commands:
# Clear S3 bucket.
- aws s3 rm s3://${S3_BUCKET} --recursive
- echo S3 bucket is cleared.
# Copy dist folder to S3 bucket, As of Angular 6, builds are stored inside an app folder in distribution and not at the root of the dist folder
- aws s3 cp dist s3://${S3_BUCKET}/${APP_NAME} --recursive
- echo Build completed on `date`
artifacts:
files:
- '/'
discard-paths: yes
base-directory: 'dist*'
If you want to run json-server on the AWS EÇ2, you need to run the following command:
nohup json-server --watch data/store.json --port
Then you can use curl to verify that the json-server is up and can respond to the request correctly.
If you want to use a port lower than 1024, run the command using sudo.
I can't call the api in the public network, maybe the issue of security group or some other permission isn't set properly. But I wish this can help.

gitlab.com CI - build a NodeJS app using docker in docker

I'm currently facing a problem with the gitlab.com shared-runners. What I'm trying to archieve in my pipeline is:
- NPM install and using grunt to make some uncss, minimize and compress tasks
- Cleaning up
- Building a docker container with the app included
- Moving the container to gitlab registry
Unfortunateley I don't get it running since a long time! I tried a lot of different gitlab.ci configs - without success. The problem is, that I have to use the "image: docker:latest" to have all the docker-tools running. But then I don't have node and grunt installed in the container.
Also the other way around is not working. I was trying to use image: centos:latest and install docker manually - but this is also not working as I always just get a Failed to get D-Bus connection: Operation not permitted
Does anyone has some more experience on the gitlab-ci using docker build commands in a docker shared runner?
Any help is highly appreciated!!
Thank you
Jannik
Gitlab can be a bit tricky :) I dont have an example based on CentOS, but I have one based on Ubuntu if that helps you. Here is some copy paste of a working gitlab pipeline of mine which uses gulp (you should be easily able to adjust it to work with your grunt).
The .gitlab-ci.yml looks like this (adjust the CONTAINER... variables at the beginning):
variables:
CONTAINER_TEST_IMAGE: registry.gitlab.com/psono/psono-client:$CI_BUILD_REF_NAME
CONTAINER_RELEASE_IMAGE: registry.gitlab.com/psono/psono-client:latest
stages:
- build
docker-image:
stage: build
image: ubuntu:16.04
services:
- docker:dind
variables:
DOCKER_HOST: 'tcp://docker:2375'
script:
- sh ./var/build-ubuntu.sh
- docker info
- docker login -u gitlab-ci-token -p "$CI_BUILD_TOKEN" registry.gitlab.com
- docker build -t $CONTAINER_TEST_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
in addition i have a this "./var/build-ubuntu.sh" which you can adjust a bit according to your needs, replace some ubuntu dependencies or switch gulp for grunt as needed:
#!/usr/bin/env bash
apt-get update && \
apt-get install -y libfontconfig zip nodejs npm git apt-transport-https ca-certificates curl openssl software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get install -y docker-ce && \
ln -s /usr/bin/nodejs /usr/bin/node && \
npm install && \
node --version && \
npm --version

Gitlab runner error "Build failed: exit code 1"

I'm trying to build Jekyll blog using gitlab runner (for gitlab pages). I get the following error: ERROR: Build failed: exit code 1. So far, everything worked. Link to project: https://gitlab.com/dash.plus/dashBlog
Just add
- apt-get update && apt-get install -y nodejs
And ofc
- bundle install
inside gitlab-cl.yaml
image: ruby:2.3
test:
stage: test
script:
- gem install jekyll
- bundle install
- apt-get update && apt-get install -y nodejs
- bundle exec jekyll -d test/
artifacts:
paths:
- test
except:
- master
pages:
stage: deploy
script:
- gem install jekyll
- bundle install
- apt-get update && apt-get install -y nodejs
- bundle exec jekyll -d public/
artifacts:
paths:
- public
only:
- master

test after build would run in new environment on gitlab-ci

I have the following configuration as .gitlab-ci.yml
but I found out after successfully pass build stage (which
would create a virtualenv called venv), it seems that
in test stage you would get a brand new environment(there's
no venv directory at all). So I wonder should I put setup
script in before_script therefor it would run in each phase(build/test/deploy). Is it a right way to do it ?
before_script:
- uname -r
types:
- build
- test
- deploy
job_install:
type: build
script:
- apt-get update
- apt-get install -y libncurses5-dev
- apt-get install -y libxml2-dev libxslt1-dev
- apt-get install -y python-dev libffi-dev libssl-dev
- apt-get install -y python-virtualenv
- apt-get install -y python-pip
- virtualenv --no-site-packages venv
- source venv/bin/activate
- pip install -q -r requirements.txt
- ls -al
only:
- master
job_test:
type: test
script:
- ls -al
- source venv/bin/activate
- cp crawler/settings.sample.py crawler/settings.py
- cd crawler
- py.test -s -v
only:
- master
adasd
Gitlab CI jobs supposed to be independent, because they could run on different runners. It is not issue. There two ways to pass files between stages:
The right way. Using artefacts.
The wrong way. Using cache. With cache key "hack". Still need same runner.
So yes, supposed by gitlab way to have everything your job depends on in before script.
Artifacts example:
artifacts:
when: on_success
expire_in: 1 mos
paths:
- some_project_files/
Cache example:
cache:
key: "$CI_BUILD_REF_NAME"
untracked: true
paths:
- node_modules/
- src/bower_components/
For correct running environment i suggest using docker with image containing apt-get dependencies. And use artefacts for passing job results between jobs. Note that artefact also uploaded to gitlab web interface and being able to download them. So if they are quite heavy use small expire_in time, for removing them after all jobs done.

Resources