This is my configuration:
language: node_js
node_js:
- '12'
cache: npm
script:
- npm test
- npm run build
after_success:
- sudo apt-get -y install lftp
- echo "set dns:order \"inet inet6\"" > ~/.lftprc
- lftp -e "mirror -eR ./app ~/tmp" -u ${USERNAME}:${PASSWORD} ftp://${FTP_SERVER}
And everything works fine. Except the last command. When I'm trying to upload files to the server using this command it takes 1-2 minutes, but travis cannot do it at all. It says that time is out and raises an error. Even if I increase timeout to 30 minutes nothing really changes.
I want to test, build and then deploy my site to the server using FTP protocol. And as I've already said I can do it from my machine using lftp.
How can I fix it?
I really tried to find the answer, but there's nothing. But istead I had found the solution - Github Actions. It works much better, then Travis.
Related
I have a project that requires npm and gradle for build, and docker for building and pushing the image.
At first I thought that I should create my own ubuntu image with gradle and npm setup, but I found out that is not what docker images are for.
So I hoped to run official Gradle and node images as a service so that my script can call those commands, but that is not happening for some reason.
My .gitlab-ci.yml:
variables:
IMAGE_NAME: my.registry.production/project
IMAGE_TAG: $CI_COMMIT_BRANCH
GIT_SUBMODULE_STRATEGY: recursive
stages:
- build
- deploy
build_project:
stage: build
image: ubuntu:jammy
services:
- name: node:12.20
alias: npm
- name: gradle:6.3.0-jre8
alias: gradle
before_script:
- git submodule init && git submodule update --remote --recursive
script:
- cd project-server && npm install && gradle clean build -Pprod -Pwar -x test -x integrationTest
deploy_image:
stage: deploy
image: docker:20.10.17
services:
- name: docker:20.10.17-dind
alias: docker
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
script:
- docker login -u $REGISTRY_USER -p $REGISTRY_PASSWORD my.registry.production
- docker build -t $IMAGE_NAME:$IMAGE_TAG .
- docker push $IMAGE_NAME:$IMAGE_TAG
If anyone has any info on how to solve this I would greatly appreciate it, since I’m a novice DevOps.
Edit 1:
My Dockerfile for custom image with Gradle and Node installed.
FROM ubuntu:jammy
LABEL key=DevOps
SHELL ["/bin/bash", "--login", "-i", "-c"]
RUN apt update && apt upgrade -y && apt install curl -y
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
RUN source /root/.bashrc && nvm install 12.14.1
RUN nvm install 12.20.0
RUN apt install zip unzip
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
RUN sdk install java 8.0.302-open
RUN sdk install gradle 3.4.1
SHELL ["/bin/bash", "--login", "-c"]
CMD [ "bin/bash" ]
After I run it, it says that npm is not found in $PATH, I tried Java, Gradle as well but they weren't found in the path as well.
I don't know why since I installed them as you can tell from the Dockerfile.
As I know, a docker image is equal to one build. So if you have multiple services you need to build each one into docker image then you can encapsulate all images into docker-compose.yml file.
I think you can do the following:
Build the npm project into a docker image
Build the Gradle project into a docker image
Write the docker-compose.yml file and put both images.
Once you have done it, the pipeline calls the docker-compose.yml file.
I hope this will be helpful.
Consider a few suggestions based on the fundamental concepts about the deployment in your CI/CD pipeline:
Remove the services keyword. Reference GitLab's official documents on what the services keyword inside gitlab-ci.yaml file is not for. The feature is used
to provide network accessable services to your job runtime (like
a database): https://docs.gitlab.com/ee/ci/services/index.html
Your project uses npm as a dependency management system, Gradle is
a build tool. Both of these pieces of software are more than
appropriate to run on the host operating system of the container
runtime inside GitLab's Pipeline job. You need these tools to assemble some build artifact as a result of the job on the same host your code has been downloaded on in the Runner.
Think about the overall size of the base image in your build_project job and consider how time to download the image over the network on to the Runner will impact your job and overall pipeline duration. If performance can be improved by baking build dependencies into a custom Dockerfile do this. If your image is too large, instead use shell commands inside the script keyword block to download them at the runtime of the job. There can be pros and cons for both.
Break shell scripts to one command per line for easier troubleshooting of failures in your scripts. You will be able to see the line number of the command which returned a non-zero exit code in your job logs:
...
script:
- cd project-server
- npm install
- gradle clean build -Pprod -Pwar -x test -x integrationTest
...
It's recommended to use the Gradle wrapper (gradlew) most of the time instead of the gradle executable directly. Configure this within your project and check the configuration files for the wrapper into your version control system and this will simplify your build dependency: https://docs.gradle.org/current/userguide/gradle_wrapper.html
Background Information
I'm trying to ensure that no matter how many times / when I run my gilab-ci.yml file, it will consistently download and install the EXACT same Azure Function deployment environment each time. I don't want to run the script today and have Azure CLI version 2.25 and then tomorrow when we trigger the pipeline, it will install / use version 2.26.
I recently came across an article that shows how to deploy an Azure Function. It's found here: https://dev.to/alandecastros/gitlab-ci-script-to-deploy-a-azure-function-3gc4
For ease of readability, I've copied and pasted the gitlab-ci.yml code here:
stages:
- deploy
deploy:
stage: deploy
image: mcr.microsoft.com/dotnet/core/sdk:3.1
script:
- curl -sL https://aka.ms/InstallAzureCLIDeb | bash
- apt-get install curl && curl -sL https://deb.nodesource.com/setup_12.x | bash -
- apt-get install nodejs
- npm install -g azure-functions-core-tools#3 --unsafe-perm true
- az login --service-principal -u $APPLICATION_ID -p $APPLICATION_SECRET --tenant $TENANT_ID
- func azure functionapp publish $FUNCTION_APP --csharp
only:
- master
QUESTIONS
From what I can tell, it feels like the first command under the scripts section will install the latest version of the Azure CLI. Is this correct? I reviewed the https://azurecliprod.blob.core.windows.net/$root/deb_install.sh file and it seems it's adding the necessary repositories to the Debian image and then runs
apt-get install -y azure-cli
In the case of nodejs, it seems it will always install major version 12... but the sub version can change. Is this correct?
How can I change this logic to control version numbers? One idea is to create my own docker image using this logic once, and then just keep reusing the custom image. I've tested it and its working.
But is there a way to install a very specific version of node? I tried to test like this:
# apt-get install curl && curl -sL https://deb.nodesource.com/setup_12.x | bash -
I can see it's installed 12.22.1
Unpacking nodejs (12.22.1-1nodesource1) ...
Setting up nodejs (12.22.1-1nodesource1) ...
I tried to follow up and do something like this:
# apt-get install nodejs12.22.1
and also
#apt-get install node_12.22.1
But in both cases I'm getting errors that it can't find these packages.
Thanks for reading / for the help.
Both the Azure CLI and Node.js offer a bash script for installation of the tools - with the drawback of always getting the latest release (then again for the majority of users this is probably a good thing). These scripts - as you figured out - do additional things like managing the repositories and trust.
Azure CLI
Microsoft offers a documentation on how to pin to a version: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#install-specific-version
In essence, you have to manually trust the signing key and add the repository as would be done by the script. Afterwards you can use the regular apt-get <package>=<version> syntax to specify a version.
Node.js
In case of Node, they are at least offering different scripts for each major release. But otherwise, it seems to be a bit more involved as is evident from https://github.com/nodesource/distributions/issues/33. I haven't tried the proposed workarounds from there as I personally am not interested in pinning Node.js to a minor release.
i am using pm2 in the shell and it works fine. But when i add it to .travis.yml, it shows me
$ pm2 restart index.js
No command 'pm2' found
pm2 is in /usr/local/bin and when i echo $PATH, it includes the path /usr/local/bin。i know nothing about it.
.travis.yml
language: node_js
node_js:
- 8.9.1
branchs:
only:
- master
cache:
apt: true
directories:
- node_modules
install:
- git pull
- rm -f package-lock.json && npm install
script:
- echo $PATH
- pm2 restart index.js
after_success:
- chmod 600 ~/.ssh/id_rsa
before_install:
- openssl aes-256-cbc -K $encrypted_a46a360c8512_key -iv $encrypted_a46a360c8512_iv
-in id_rsa.enc -out ~/.ssh/id_rsa -d
I think you have mixed up build container with the actual server where you want the final app to run.
Travis builds projects inside a VM/Container that ends when the Travis build ends.
PM2 is supposed to be installed and run on the actual web server that hosts the app.
So, from Travis ci you probably should:
Upload latest project files to the actual server via ssh.
Run pm2 on the actual server via ssh.
Here's something along these lines: https://oncletom.io/2016/travis-ssh-deploy/
I am trying to configure my gitlab-ci to use yarn install instead of npm install
My current gitlab-ci.yml looks like:
image: node:6.9.4
cache:
paths:
- node_modules/
- .yarn
before_script:
- apt-get update -qq && apt-get install -qy libelf1
stages:
- test
test_core:
stage: test
script:
- yarn config set cache-folder .yarn
- yarn install
- npm run build
- npm run test
tags:
- 2gb
But the build fails with the error:
/bin/bash: line 48: yarn: command not found
is there something I am missing?
I tried installing yarn with:
curl -o- -L https://yarnpkg.com/install.sh | bash
this gave me same error, probably because I need to reload the bash environment for the yarn command to become available.
The above config works perfect with npm install.
Please help me resolve this. If there is something missing in my config file or there is something wrong with the gitlab-ci.
Thanks.
Solved it by using the latest official node docker image.
Since image: 6.10.0, yarn is installed by default in the image.
But if you need node-gyp to build any package, it needs to be installed by adding a line to the script:
yarn global add node-gyp
Add the following to your ci script after yarn got installed:
export PATH=$HOME/.yarn/bin:$PATH
I use image:node:latest and sometimes it prompts the error. Clear Runner Caches do the job for me. Maybe the runner did not recover to the correct state after doing other jobs.
I solved it by using npx (package runner). It's better then extending docker-image only for this purpose
npx yarn someCommand
I think my question is related to these ones but the solution did not work in my case: How to run Node.js and Ruby tests within one project on Travis CI and Is it possible to set up travis to run tests for several languages?
I have a github repo that contains a little Golang application with a tiny Angularjs frontend. I want to run both go-tests and Karma-tests.
I see two options:
Run two travis-ci builds for one repo (I could not figure out how to
do that)
Run both tests in one build (did not work since the node version in travis-ci is too old (0.10).
The build runs for 60+ minutes and then stops with "FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory". The build shows tons of warnings like this one: npm WARN engine escodegen#1.8.0: wanted: {"node":">=0.12.0"} (current: {"node":"0.10.36","npm":"1.4.28"})
I tried to run the build as node_js but then the "go get xyz" does not work.
my .travis.yml file:
language: go
go:
- 1.5
env:
- TRAVIS_NODE_VERSION="0.12"
install:
- export PATH=$HOME/gopath/bin:$PATH
- go get golang.org/x/tools/cmd/cover
- go get -v github.com/axw/gocov
- go install github.com/axw/gocov/gocov
- go get github.com/GeertJohan/go.rice
# we do not need the rice tool!
- go get github.com/xeipuuv/gojsonschema
- go get github.com/finklabs/ttime
- go get github.com/finklabs/graceful
- go get github.com/gorilla/mux
before_script:
- npm install bower
- npm install --dev
- bower install
script:
- gocov test | gocov report
- npm test
sometimes the problem sits in front of the machine... I found a great workaround and I want to share it with you.
I created an account on https://drone.io/ in the settings tab I selected a build for "Go1" and added the following commands:
# install gogrinder dependencies
go get ./...
# install test dependencies
go get golang.org/x/tools/cmd/cover
go get -v github.com/axw/gocov
go install github.com/axw/gocov/gocov
# install nodejs requirements
npm -d install
./node_modules/bower/bin/bower install
# run the tests on the go code
gocov test | gocov report
# run the tests on the Angularjs frontend
npm test
Now I am back on track with a CI server that runs both Golang and Karma tests.
I gave a solution to this on http://entulho.fiatjaf.alhur.es/guias/how-to-use-node-along-with-other-language-on-travis-ci/ a long time ago, but it still works today:
Basically you just add
install:
- . $HOME/.nvm/nvm.sh
- nvm install stable
- nvm use stable
- npm install
to your .travis.yml, which must be set to a different language, and it will install node and npm.