I've been trying to setup a CI/CD Pipeline on my Repo which runs common tasks like linting/tests etc. I've successfully setup a Gitlab-Runner which is working fine. The only part I'm stuck is the "deploy" part.
When I run my build, how do I actually get the files into my /var/www/xyz folder.
I get that everything is running in a Docker Container and I can't just magically copy paste my files there, but i don't get how I get the files on my actual server-directory. I've been searching for days for good docs / explanations, so as always, StackOverflow is my last resort for help.
I'm running on a Ubuntu 20.04 LTS VPS and a SaaS GitLab-Repository if that info is needed. That's my .gitlab-ci.yml:
image: timbru31/node-alpine-git
before_script:
- git fetch origin
stages:
- setup
- test
- build
- deploy
#All Setup Jobs
Install Dependencies:
stage: setup
interruptible: true
script:
- npm install
- npm i -g #nrwl/cli
artifacts:
paths:
- node_modules/
# All Test Jobs
Lint:
stage: test
script: npx nx run nx-fun:lint
Tests:
stage: test
script: npx nx run nx-fun:test
Deploy:
stage: build
script:
- ls /var/www/
- npx nx build --prod --output-path=dist/
- cp -r dist/* /var/www/html/neostax/
only:
refs:
- master
Normally I would ssh into my server, run the build, and then copy the build to the corresponding web-directory.
TL;DR - How do I get files from a GitLab-Runner to an actual directory on the server?
Related
I had created a specific runner for my gitlab project,
its taking too long to run the pipeline.
Its is getting stuck in Cypress test mainly.
After "All Specs passed" it will not move forward.
- build
- test
build:
stage: build
image: gradle:jdk11
script:
- gradle --no-daemon build
artifacts:
paths:
- build/distributions
expire_in: 1 day
when: always
junit-test:
stage: test
image: gradle:jdk11
dependencies: []
script:
- gradle test
timeout: 5m
cypress-test:
stage: test
image: registry.gitlab.com/sahajsoft/gurukul2022/csv-parser-srijan:latestSrigin2
dependencies:
- build
script:
- unzip -q build/distributions/csv-parser-srijan-1.0-SNAPSHOT.zip -d build/distributions
- sh build/distributions/csv-parser-srijan-1.0-SNAPSHOT/bin/csv-parser-srijan &
- npm install --save-dev cypress-file-upload
- npx cypress run --browser chrome
A recommended approach is to try and replicate your script (from your pipeline) locally, from your computer.
It will allow to check:
how long those commands take
if there is any interactive step, where a command might expect a user entry, and wait for stdin.
The second point would explain why, in an unattended environment like a pipeline one, "it will not move forward".
I followed a tutorial on youtube Link to create a ci/cd pipeline for deployment on netlify.
The builds are successful but the deploy shows a 404 page.
Link to deployed site
I'm posting the yml commands below
stages:
- lint
- build
- deploy
lint project:
stage: lint
image: node:15
script:
- npm install
- npm run lint
build project:
stage: build
image: node:15
script:
- npm install
- npm run build
artifacts:
paths:
- .next
netlify:
stage: deploy
image: node:15
script:
- npm set prefix ~/.npm; path+=$HOME/.npm/bin
- path+=./node_modules/.bin
- npm install -g netlify-cli
- netlify deploy --dir=.next --prod
I'm trying to create a Next JS boilerplate to be used in my future projects & Through CI/CD pipeline i'm looking for automated deployment. Any help or link to resources would be appreciated.
I am attempting to create a CI/CD pipeline with Travis CI that tests the front-end, tests the back-end, and deploys. The front-end is using Node, the back-end is using Go.
My repository is structured as follows:
- client
- DockerFile
- ...(front-end code)
- server
- DockerFile
- ...(back-end code)
- .travis.yml
Would I be able to utilize the DockerFiles in some fashion to execute tests for both sides of the application and have Travis report their results properly?
I'm not well versed with either tools so I was hoping to get some input before I dig myself into a hole. I plan on using a combination of Travis stages and docker build/docker run commands. Something like this:
jobs:
include:
- stage: test client side
before_script:
- cd client
- docker build ...
script:
docker run image /bin/sh -c "run node tests"
after_script:
- cd ..
- stage: test server side
before_script:
- cd server
script:
docker run image /bin/sh -c "run go tests"
after_script:
- cd ..
- stage: deploy
script: skip
deploy:
- provider: s3
skip_cleanup: true
on:
branch: master
This doc page makes it looks promising, but the inclusion of language: ruby and script: - bundle exec rake test throws me off. I am not sure why Ruby is required if the tests are ran through docker (at least that's what it looks like).
Update 1
I believe I got it to work correctly with the client side of the application.
Here is what I got:
services:
- docker
jobs:
include:
- stage: test
before_script:
- docker pull node:12
script:
- docker run --rm -e CI=true -v $(pwd)/client:/src node:12 /bin/sh -c "cd src; npm install; npm test"
I'm trying to setup GitLab CI for a mono repository.
For the sake of the argument, lets say I want to process 2 JavaScript packages:
app
cli
I have defined 3 stages:
install
test
build
deploy
Because I'm reusing the files from previous steps, I use the GitLab cache.
My configuration looks like this:
stages:
- install
- test
- build
- deploy
install_app:
stage: install
image: node:8.9
cache:
policy: push
paths:
- app/node_modules
script:
- cd app
- npm install
install_cli:
stage: install
image: node:8.9
cache:
policy: push
paths:
- cli/node_modules
script:
- cd cli
- npm install
test_app:
image: node:8.9
cache:
policy: pull
paths:
- app/node_modules
script:
- cd app
- npm test
test_cli:
image: node:8.9
cache:
policy: pull
paths:
- cli/node_modules
script:
- cd cli
- npm test
build_app:
stage: build
image: node:8.9
cache:
paths:
- app/node_modules
- app/build
script:
- cd app
- npm run build
deploy_app:
stage: deploy
image: registry.gitlab.com/my/gcloud/image
only:
- master
environment:
name: staging
url: https://example.com
cache:
policy: pull
paths:
- app/build
script:
- gcloud app deploy app/build/app.yaml
--verbosity info
--version master
--promote
--stop-previous-version
--quiet
--project "$GOOGLE_CLOUD_PROJECT"
The problem is in the test stage. Most of the time the test_app job fails, because the app/node_modules directory is missing. Sometimes a retry works, but mostly not.
Also, I would like to use two caches for the build_app job. I want to pull app/node_modules and push app/build. I can't find a way to accomplish this. This makes me feel like I don't fully understand how the cache works.
Why are my cache files gone? Do I misunderstand how GitLab CI cache works?
The cache is provided on a best-effort basis, so don't expect that the cache will be always present.
If you have hard dependencies between jobs, use artifacts and dependencies.
Anyway, if it is just for node_modules, I suggest you to install it in every step, instead of using artifacts - you will not save much time with artifacts.
I have this pipeline file to unittest my project:
image: jameslin/python-test
pipelines:
default:
- step:
script:
- service mysql start
- pip install -r requirements/test.txt
- export DJANGO_CONFIGURATION=Test
- python manage.py test
but is it possible to switch to another docker image to deploy?
image: jameslin/python-deploy
pipelines:
default:
- step:
script:
- ansible-playbook deploy
I cannot seem to find any documentation saying either Yes or No.
You can specify an image for each step. Like that:
pipelines:
default:
- step:
name: Build and test
image: node:8.6
script:
- npm install
- npm test
- npm run build
artifacts:
- dist/**
- step:
name: Deploy
image: python:3.5.1
trigger: manual
script:
- python deploy.py
Finally found it:
https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html#Configurebitbucket-pipelines.yml-ci_stepstep(required)
step (required) Defines a build execution unit. Steps are executed in
the order in which they appear in the pipeline. Currently, each
pipeline can have only one step (one for the default pipeline and one
for each branch). You can override the main Docker image by specifying
an image in a step.
I have not found any information saying yes or no either so what I have assumed is that since this image can be configured with all the languages and technology you need I would suggest this method:
Create your docker image with all utilities you need for both default and deployment.
Use the branching method they show in their examples https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html#Configurebitbucket-pipelines.yml-ci_branchesbranches(optional)
Use shell scripts or other scripts to run specific tasks you need and
image: yourusername/your-image
pipelines:
branches:
master:
- step:
script: # Modify the commands below to build your repository.
- echo "Starting pipelines for master"
- chmod +x your-task-configs.sh #necessary to get shell script to run in BB Pipelines
- ./your-task-configs.sh
feature/*:
- step:
script: # Modify the commands below to build your repository.
- echo "Starting pipelines for feature/*"
- npm install
- npm install -g grunt-cli
- npm install grunt --save-dev
- grunt build