dastardly burp with gitlab - security

I want to use Burp dastardly which is the new DAST tool from portswigger.
actually I tried it in Gitlab CI/CD but I got an error! even I tried it in my server.
this is how I use it in Gitlab:
Burp_DAST:
stage: dast
image: docker:stable
script:
- |
docker run --user $(id -u):$(id -g) --rm -v $(pwd):/dastardly -e \
DASTARDLY_TARGET_URL=$TARGET_URL -e \
DASTARDLY_OUTPUT_FILE=/dastardly/$CI_PROJECT_NAME-dastardly-report.xml \
public.ecr.aws/portswigger/dastardly:latest
artifacts:
paths:
- "$CI_PROJECT_NAME-dastardly-report.xml"
when: always
and I have this error:
2022-11-01 12:03:09 INFO dastardly.EventLogPrinter - Nov 01 2022 11:52:22 INFORMATION Audit started.
2022-11-01 12:03:09 INFO dastardly.EventLogPrinter - Nov 01 2022 11:52:23 ERROR Could not start Burp's browser sandbox because you are running as root. Either switch to running as an unprivileged user or allow running without sandbox.
2022-11-01 12:03:09 ERROR dastardly.ScanFinishedHandler - Failing build as scanner identified issue(s) with severity higher than "INFO":
2022-11-01 12:03:09 ERROR dastardly.ScanFinishedHandler - Path: / Issue Type: Cross-origin resource sharing: arbitrary origin trusted Severity: HIGH
2022-11-01 12:03:09 ERROR dastardly.ScanFinishedHandler - Path: /robots.txt Issue Type: Cross-origin resource sharing: arbitrary origin trusted Severity: HIGH
2022-11-01 12:03:10 INFO bsee.BurpProcess.scan.scan-1 - Deleting temporary files - please wait ... done.
EDIT
I did try it in my server and found out it will correctly work if you run it with any sudoer user but root. this is my command that I used:
sudo docker run --user $(id -u):$(id -g) --rm -v $(pwd):/dastardly -e DASTARDLY_TARGET_URL=$TAGET_URL -e DASTARDLY_OUTPUT_FILE=/dastardly/dastardly-report.xml public.ecr.aws/portswigger/dastardly:latest
So I need how to do this in Gitlab since docker:dind run with root user and docker:dind-rootless not working well in gitlab?

I am running the script to run docker-entrypoint.sh
Here is the working CI that I implemented.
stages:
- dastardly
dastardly_burpsuit:
image:
name: public.ecr.aws/portswigger/dastardly:latest
entrypoint: [""]
stage: dastardly
variables:
# No need to clone the repo, we exclusively work on artifacts. See
# https://docs.gitlab.com/ee/ci/runners/README.html#git-strategy
GIT_STRATEGY: none
DASTARDLY_TARGET_URL: "https://ginandjuice.shop"
DASTARDLY_OUTPUT_FILE: "$CI_PROJECT_NAME-dastardly-report.xml"
artifacts:
paths:
- "$CI_PROJECT_NAME-dastardly-report.xml"
when: always
script:
- "/bin/bash /usr/local/bin/docker-entrypoint.sh dastardly"

Related

GitLab CI Pipeline: Pipeline cannot be run

I can't find out why the GitLab CI Pipelines for my Repo won't run. I have a .gitlab-ci.yml file and the feature enabled, but the pipeline won't run. Also if I try to trigger the pipeline manually I get the following error back.
Pipeline cannot be run.
Pipeline will not run for the selected trigger. The rules configuration prevented any jobs from being added to the pipeline.
The CI feature is enabled.
Here is my .gitlab-ci.yml file.
stages:
- build
- deploy
npm-run-build:
stage: build
image: node:19
only:
- main
cache:
key: ${CI_COMMIT_REF_SLUG}-build
paths:
- dist/
script:
- cp .env.example .env
- npm ci
- npm run build-only
deploy-dist:
stage: deploy
image: fedora:latest
only:
- main
environment:
name: production
url: https://example.com
needs:
- npm-run-build
cache:
key: ${CI_COMMIT_REF_SLUG}-build
paths:
- dist/
before_script:
- dnf install -y openssh-clients
- mkdir -p ~/.ssh
- echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- ssh-keyscan -t rsa example.com > ~/.ssh/known_hosts
script:
# create remote project dir if not available
- ssh thomas#example.com "mkdir -p /home/thomas/example.com"
# upload project files
- scp -prq . thomas#example.com:/home/thomas/example.com
# restart the container
- ssh thomas#example.com "cd /home/thomas/example.com && docker-compose down && docker-compose up -d"
Thank you! 😁
As D Malan pointed out in the comments, I have restricted the runs with only to the main branch. But the branch name is actually master 🤦
So I just changed the rule form main to master and now it is running 👌

How to use Kaniko inside Gitlab CICD?

I am trying to use Kaniko with Gitlab in order to get rid of the DinD flow.
So, I have this in my .gitlab-ci.yaml
kaniko:
stage: tagging
variables:
CI_REGISTRY: ${AZURE_REGISTRY_USERNAME_DEV}.azurecr.io
CI_REGISTRY_USER: ${AZURE_REGISTRY_USERNAME_DEV}
CI_REGISTRY_PASSWORD: ${AZURE_REGISTRY_PASS_DEV}
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
#
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/devops/Dockerfile"
--destination "${CI_REGISTRY}/kanikotest:bla"
--verbosity debug
tags: # select gitlab-runner based on this tag(s)
- docker
only:
refs:
- /^feat.*$/
but I keep getting this error in the logs
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "mysuperregistry.azurecr.io/kanikotest:bla": creating push check transport for mysuperregistry.azurecr.io failed: GET https://mysuperregistry.azurecr.io/oauth2/token?scope=repository%3Akanikotest%3Apush%2Cpull&service=mysuperregistry.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.
I am following this guide.
Fun fact... I have successfully deployed Kaniko inside Minikube by creating a secret with the same creds, and I successfully pushed to the same registry.
The syntax of the auth file seems good (I assume the creds are correct), so your code should work if you just set the DOCKER_CONFIG environment variable as following:
kaniko:
stage: tagging
variables:
CI_REGISTRY: ${AZURE_REGISTRY_USERNAME_DEV}.azurecr.io
CI_REGISTRY_USER: ${AZURE_REGISTRY_USERNAME_DEV}
CI_REGISTRY_PASSWORD: ${AZURE_REGISTRY_PASS_DEV}
DOCKER_CONFIG: "$CI_PROJECT_DIR/kanikotest/.docker"
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p $DOCKER_CONFIG
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > $DOCKER_CONFIG/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/devops/Dockerfile"
--destination "${CI_REGISTRY}/kanikotest:bla"
--digest-file "$CI_PROJECT_DIR/docker-content-digest-kanikotest"
--verbosity info
artifacts:
paths:
- docker-content-digest-kanikotest
Adding an extra directory (kanikotest) inside the DOCKER_CONFIG path will avoid concurrent builds to overwrite the same auth file (not required in your case example but a good practice in general).
The --digest-file option will permit also to save the image SHA for following CI jobs.

edit and execute pipeline *.yml template file from command line [duplicate]

If a GitLab project is configured on GitLab CI, is there a way to run the build locally?
I don't want to turn my laptop into a build "runner", I just want to take advantage of Docker and .gitlab-ci.yml to run tests locally (i.e. it's all pre-configured). Another advantage of that is that I'm sure that I'm using the same environment locally and on CI.
Here is an example of how to run Travis builds locally using Docker, I'm looking for something similar with GitLab.
Since a few months ago this is possible using gitlab-runner:
gitlab-runner exec docker my-job-name
Note that you need both docker and gitlab-runner installed on your computer to get this working.
You also need the image key defined in your .gitlab-ci.yml file. Otherwise won't work.
Here's the line I currently use for testing locally using gitlab-runner:
gitlab-runner exec docker test --docker-volumes "/home/elboletaire/.ssh/id_rsa:/root/.ssh/id_rsa:ro"
Note: You can avoid adding a --docker-volumes with your key setting it by default in /etc/gitlab-runner/config.toml. See the official documentation for more details. Also, use gitlab-runner exec docker --help to see all docker-based runner options (like variables, volumes, networks, etc.).
Due to the confusion in the comments, I paste here the gitlab-runner --help result, so you can see that gitlab-runner can make builds locally:
gitlab-runner --help
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
1.1.0~beta.135.g24365ee (24365ee)
AUTHOR(S):
Kamil Trzciński <ayufan#ayufan.eu>
COMMANDS:
exec execute a build locally
[...]
GLOBAL OPTIONS:
--debug debug mode [$DEBUG]
[...]
As you can see, the exec command is to execute a build locally.
Even though there was an issue to deprecate the current gitlab-runner exec behavior, it ended up being reconsidered and a new version with greater features will replace the current exec functionality.
Note that this process is to use your own machine to run the tests using docker containers. This is not to define custom runners. To do so, just go to your repo's CI/CD settings and read the documentation there. If you wanna ensure your runner is executed instead of one from gitlab.com, add a custom and unique tag to your runner, ensure it only runs tagged jobs and tag all the jobs you want your runner to be responsible of.
I use this docker-based approach:
Edit: 2022-10
docker run --entrypoint bash --rm -w $PWD -v $PWD:$PWD -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest -c 'git config --global --add safe.directory "*";gitlab-runner exec docker test'
For all git versions > 2.35.2. You must add safe.directory within the container to avoid fatal: detected dubious ownership in repository at.... This also true for patched git versions < 2.35.2. The old command will not work anymore.
Details
0. Create a git repo to test this answer
mkdir my-git-project
cd my-git-project
git init
git commit --allow-empty -m"Initialize repo to showcase gitlab-runner locally."
1. Go to your git directory
cd my-git-project
2. Create a .gitlab-ci.yml
Example .gitlab-ci.yml
image: alpine
test:
script:
- echo "Hello Gitlab-Runner"
3. Create a docker container with your project dir mounted
docker run -d \
--name gitlab-runner \
--restart always \
-v $PWD:$PWD \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
(-d) run container in background and print container ID
(--restart always) or not?
(-v $PWD:$PWD) Mount current directory into the current directory of the container - Note: On Windows you could bind your dir to a fixed location, e.g. -v ${PWD}:/opt/myapp. Also $PWD will only work at powershell not at cmd
(-v /var/run/docker.sock:/var/run/docker.sock) This gives the container access to the docker socket of the host so it can start "sibling containers" (e.g. Alpine).
(gitlab/gitlab-runner:latest) Just the latest available image from dockerhub.
4. Execute with
Avoid fatal: detected dubious ownership in repository at... More info
docker exec -it -w $PWD gitlab-runner git config --global --add safe.directory "*"
Actual execution
docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker test
# ^ ^ ^ ^ ^ ^
# | | | | | |
# (a) (b) (c) (d) (e) (f)
(a) Working dir within the container. Note: On Windows you could use a fixed location, e.g. /opt/myapp.
(b) Name of the docker container
(c) Execute the command "gitlab-runner" within the docker container
(d)(e)(f) run gitlab-runner with "docker executer" and run a job named "test"
5. Prints
...
Executing "step_script" stage of the job script
$ echo "Hello Gitlab-Runner"
Hello Gitlab-Runner
Job succeeded
...
Note: The runner will only work on the commited state of your code base. Uncommited changes will be ignored. Exception: The .gitlab-ci.yml itself does not have be commited to be taken into account.
Note: There are some limitations running locally. Have a look at limitations of gitlab runner locally.
I'm currently working on making a gitlab runner that works locally.
Still in the early phases, but eventually it will become very relevant.
It doesn't seem like gitlab want/have time to make this, so here you go.
https://github.com/firecow/gitlab-runner-local
If you are running Gitlab using the docker image there: https://hub.docker.com/r/gitlab/gitlab-ce, it's possible to run pipelines by exposing the local docker.sock with a volume option: -v /var/run/docker.sock:/var/run/docker.sock. Adding this option to the Gitlab container will allow your workers to access to the docker instance on the host.
The GitLab runner appears to not work on Windows yet and there is an open issue to resolve this.
So, in the meantime I am moving my script code out to a bash script, which I can easily map to a docker container running locally and execute.
In this case I want to build a docker container in my job, so I create a script 'build':
#!/bin/bash
docker build --pull -t myimage:myversion .
in my .gitlab-ci.yaml I execute the script:
image: docker:latest
services:
- docker:dind
before_script:
- apk add bash
build:
stage: build
script:
- chmod 755 build
- build
To run the script locally using powershell I can start the required image and map the volume with the source files:
$containerId = docker run --privileged -d -v ${PWD}:/src docker:dind
install bash if not present:
docker exec $containerId apk add bash
Set permissions on the bash script:
docker exec -it $containerId chmod 755 /src/build
Execute the script:
docker exec -it --workdir /src $containerId bash -c 'build'
Then stop the container:
docker stop $containerId
And finally clean up the container:
docker container rm $containerId
Another approach is to have a local build tool that is installed on your pc and your server at the same time.
So basically, your .gitlab-ci.yml will basically call your preferred build tool.
Here an example .gitlab-ci.yml that i use with nuke.build:
stages:
- build
- test
- pack
variables:
TERM: "xterm" # Use Unix ASCII color codes on Nuke
before_script:
- CHCP 65001 # Set correct code page to avoid charset issues
.job_template: &job_definition
except:
- tags
build:
<<: *job_definition
stage: build
script:
- "./build.ps1"
test:
<<: *job_definition
stage: test
script:
- "./build.ps1 test"
variables:
GIT_CHECKOUT: "false"
pack:
<<: *job_definition
stage: pack
script:
- "./build.ps1 pack"
variables:
GIT_CHECKOUT: "false"
only:
- master
artifacts:
paths:
- output/
And in nuke.build i've defined 3 targets named like the 3 stages (build, test, pack)
In this way you have a reproducible setup (all other things are configured with your build tool) and you can test directly the different targets of your build tool.
(i can call .\build.ps1 , .\build.ps1 test and .\build.ps1 pack when i want)
I am on Windows using VSCode with WSL
I didn't want to register my work PC as a runner so instead I'm running my yaml stages locally to test them out before I upload them
$ sudo apt-get install gitlab-runner
$ gitlab-runner exec shell build
yaml
image: node:10.19.0 # https://hub.docker.com/_/node/
# image: node:latest
cache:
# untracked: true
key: project-name
# key: ${CI_COMMIT_REF_SLUG} # per branch
# key:
# files:
# - package-lock.json # only update cache when this file changes (not working) #jkr
paths:
- .npm/
- node_modules
- build
stages:
- prepare # prepares builds, makes build needed for testing
- test # uses test:build specifically #jkr
- build
- deploy
# before_install:
before_script:
- npm ci --cache .npm --prefer-offline
prepare:
stage: prepare
needs: []
script:
- npm install
test:
stage: test
needs: [prepare]
except:
- schedules
tags:
- linux
script:
- npm run build:dev
- npm run test:cicd-deps
- npm run test:cicd # runs puppeteer tests #jkr
artifacts:
reports:
junit: junit.xml
paths:
- coverage/
build-staging:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build:stage
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-dev:
stage: deploy
needs: [build-staging]
tags: [linux]
only:
- schedules
# # - branches#gitlab-org/gitlab
before_script:
- apt-get update && apt-get install -y lftp
script:
# temporarily using 'verify-certificate no'
# for more on verify-certificate #jkr: https://www.versatilewebsolutions.com/blog/2014/04/lftp-ftps-and-certificate-verification.html
# variables do not work with 'single quotes' unless they are "'surrounded by doubles'"
- lftp -e "set ssl:verify-certificate no; open mediajackagency.com; user $LFTP_USERNAME $LFTP_PASSWORD; mirror --reverse --verbose build/ /var/www/domains/dev/clients/client/project/build/; bye"
# environment:
# name: staging
# url: http://dev.mediajackagency.com/clients/client/build
# # url: https://stg2.client.co
when: manual
allow_failure: true
build-production:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-client:
stage: deploy
needs: [build-production]
tags: [linux]
only:
- schedules
# - master
before_script:
- apt-get update && apt-get install -y lftp
script:
- sh deploy-prod
environment:
name: production
url: http://www.client.co
when: manual
allow_failure: true
The idea is to keep check commands outside of .gitlab-ci.yml. I use Makefile to run something like make check and my .gitlab-ci.yml runs the same make commands that I use locally to check various things before committing.
This way you'll have one place with all/most of your commands (Makefile) and .gitlab-ci.yml will have only CI-related stuff.
I have written a tool to run all GitLab-CI job locally without have to commit or push, simply with the command ci-toolbox my_job_name.
The URL of the project : https://gitlab.com/mbedsys/citbx4gitlab
Years ago I build this simple solution with Makefile and docker-compose to run the gitlab runner in docker, you can use it to execute jobs locally as well and should work on all systems where docker works:
https://gitlab.com/1oglop1/gitlab-runner-docker
There are few things to change in the docker-compose.override.yaml
version: "3"
services:
runner:
working_dir: <your project dir>
environment:
- REGISTRATION_TOKEN=<token if you want to register>
volumes:
- "<your project dir>:<your project dir>"
Then inside your project you can execute it the same way as mentioned in other answers:
docker exec -it -w $PWD runner gitlab-runner exec <commands>..
I recommend using gitlab-ci-local
https://github.com/firecow/gitlab-ci-local
It's able to run specific jobs as well.
It's a very cool project and I have used it to run simple pipelines on my laptop.

Masked variables when overriding service commands are empty

I want to use Testcontainers for my JUNIT tests and so I created this:
image: gitlab.registry.example:5005/my-custom-maven-image
variables:
MAVEN_CLI_OPTS: "--batch-mode -s $CI_PROJECT_DIR/.m2/settings.xml"
stages:
- test
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS clean test
services:
- name: docker:dind
alias: docker
command:
- /bin/sh
- -c
- "DOCKER_AUTH_CONFIG=`echo \"{\\\"auths\\\":{\\\"$CI_REGISTRY\\\":{\\\"username\\\":\\\"$CI_REGISTRY_USER\\\",\\\"password\\\":\\\"$CI_REGISTRY_PASSWORD\\\"}}}\"` && mkdir -p \"/root/.docker\" && echo \"${DOCKER_AUTH_CONFIG}\" > \"/root/.docker/config.json\" && cat /root/.docker/config.json && update-ca-certificates && dockerd-entrypoint.sh || exit"
variables:
# Instruct Testcontainers to use the daemon of DinD.
DOCKER_HOST: "tcp://docker:2375"
# Instruct Docker not to start over TLS.
DOCKER_TLS_CERTDIR: ""
DOCKER_TLS_VERIFY: 0
# Improve performance with overlayfs.
DOCKER_DRIVER: overlay2
This gives me the following output when the runner tries to spawn the dind container:
{"auths":{"gitlab.registry.example:5005":{"username":"gitlab-ci-token","password":""}}}
As you can see the password is empty. Printing the CI_REGISTRY_PASSWORD variable in a before_script shows me [masked] as I would expect.
I am about to create an issue in the gitlab-runner project but I wanted to make sure what I did is not wrong beforehand.
Update: Created an issue in the gitlab-runner project
It looks like a bug indeed: CI_REGISTRY_PASSWORD variable is not present at all in the container running DinD service, where it's properly set in job container.
I reproduced your issue by re-using your example in a simplified way:
test:
stage: test
script:
- echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
# - sleep 9999
services:
- name: docker:dind
alias: docker
command:
- /bin/sh
- -c
- echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD" && dockerd-entrypoint.sh || exit
This shows in Gitlab UI:
# Services logs (not always shown)
Registry registry.novadiscovery.net - User gitlab-ci-token - Password
# Script logs
$ echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
Registry registry.mycompany.com - User gitlab-ci-token - Password [MASKED]
At first I thought the variable was somehow hidden from Gitlab log UI, but instead of being shown as [masked] it was simply not shown at all. However, when inspecting underlying containers running jobs, we can see variable is indeed absent from DinD service:
# Running docker inspect command on machine running Gitlab Runner
# Inspect DinD service container
# CI_REGISTRY_PASSWORD does not exists
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-docker-0 | jq .[0].Config.Env | grep CI_REGISTRY
"CI_REGISTRY_USER=gitlab-ci-token",
"CI_REGISTRY=registry.mycompany.com",
"CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",
# Inspect job container
# CI_REGISTRY_PASSWORD is set
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-build-2 | jq .[0].Config.Env | grep CI_REGISTRY
"CI_REGISTRY_USER=gitlab-ci-token",
"CI_REGISTRY_PASSWORD=xxx",
"CI_REGISTRY=registry.mycompany.com",
"CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",
By comparing variables between job container and service container, it seems all secret or sensible pre-defined CI variables are missing from the services containers. From the above comparison, the following variables were missing (there may be others):
CI_JOB_TOKEN
CI_BUILD_TOKEN
CI_REGISTRY_PASSWORD
CI_REPOSITORY_URL
CI_DEPENDENCY_PROXY_PASSWORD
CI_JOB_JWT
Tested on Gitlab 13.11.3 and Gitlab Runner 13.2.1

jHipster App crashes propably because CloudFoundry activates cloud profile

The deployment of my small jhipster App "customerapp" fails and it is probably because cloud foundry sets the profile "cloud" in addition to the profile "dev". I am using several spaces in cloud foundry for the different stages of the development: dev, staging and prod.
I used the jhipster generator, added some entities customer, address and contacts. App is running locally without any issues.
I also use gitlab-ci to build, test and deploy my software. My .gitlab-ci.yml looks like this (I deleted some unecessary parts).
image: mydockerregistry.xxxxx.de/jutoro/jhipster_test/jhipster-dockerimage
services:
- docker:dind
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- node_modules
- .maven
before_script:
- chmod +x mvnw
- export MAVEN_USER_HOME=`pwd`/.maven
stages:
- build
- package
- deployToCF
mvn-build:
stage: build
only:
- dev
- prod
script:
- npm install
- ./mvnw compile -DskipTests -Dmaven.repo.local=$MAVEN_USER_HOME -
Dspring.profiles.active=dev
mvn-package-dev:
stage: package
only:
- dev
script:
- npm install
- ./mvnw package -Pdev -DskipTests -Dmaven.repo.local=$MAVEN_USER_HOME -Dspring.profiles.active=dev
artifacts:
paths:
- target/*.war
mvn-package-prod:
stage: package
only:
- prod
script:
- npm install
- ./mvnw package -Pprod -DskipTests -Dmaven.repo.local=$MAVEN_USER_HOME -Dspring.profiles.active=prod
artifacts:
paths:
- target/*.war
deployToCloudFoundry-dev:
image: pivotalpa/cf-cli-resource
stage: deployToCF
only:
- dev
cache:
paths:
- bin/
script:
- bash ci/scripts/deployToCloudFoundry.sh
deployToCloudFoundry-prod:
image: pivotalpa/cf-cli-resource
stage: deployToCF
only:
- prod
cache:
paths:
- bin/
script:
- bash ci/scripts/deployToCloudFoundry.sh
The DOCKERFILE (which is built and added to our docker repository also with gitlab-ci):
# DOCKER-VERSION 1.8.2
FROM openjdk:8
MAINTAINER Robert Zieschang
RUN apt-get install -y curl
# install node.js
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt-get install -y nodejs python g++ build-essential && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# install yeoman
RUN npm install -g yo
The deplpoyToCloudFoundry.sh shell script:
cf login -a $CF_API_ENDPOINT -u $CF_USER -p $CF_PASS -o "${CF_ORG^^}" -s ${CI_COMMIT_REF_NAME^^}
cf push -n $CI_PROJECT_NAME-$CI_COMMIT_REF_NAME
My manifest file:
---
applications:
- name: customerapp
memory: 1024M
#buildpack: https://github.com/cloudfoundry/java-buildpack#v3.19.2
path: target/customerapp-0.0.1-SNAPSHOT.war
services:
- postgresql
env:
#SPRING_PROFILES_ACTIVE: dev
#SPRING_PROFILES_DEFAULT: dev
#JAVA_OPTS: -Dspring.profiles.active=dev
The pipeline runs well, the app is packed into the war file and uploaded to cloud foundry as well, but it crashes and I assume it is because somehow cloud foundry still applies the profile 'cloud' and this overrides important configurations from jhipsters 'dev' profile.
[...]
2019-01-02T19:03:16.05+0100 [APP/PROC/WEB/0] OUT 2019-01-02 18:03:16.055 INFO 8 --- [ main] pertySourceApplicationContextInitializer : 'cloud' property source added
2019-01-02T19:03:16.05+0100 [APP/PROC/WEB/0] OUT 2019-01-02 18:03:16.056 INFO 8 --- [ main] nfigurationApplicationContextInitializer : Reconfiguration enabled
2019-01-02T19:03:16.06+0100 [APP/PROC/WEB/0] OUT 2019-01-02 18:03:16.064 INFO 8 --- [ main] com.jutoro.cco.CustomerappApp : The following profiles are active: cloud,dev,swagger
[...]
This later leads to:
2019-01-02T19:03:29.17+0100 [APP/PROC/WEB/0] OUT 2019-01-02 18:03:29.172 ERROR 8 --- [ main] com.jutoro.cco.CustomerappApp : You have misconfigured your application! It should not run with both the 'dev' and 'cloud' profiles at the same time.
[...]
After that cloud foundry stops the app.
2019-01-02T19:04:11.09+0100 [CELL/0] OUT Cell 83899f60-78c9-4323-8d3c-e6255086c8a7 stopping instance 74be1834-b656-4445-506c-bdfa
The generated application-dev.yml and bootstrap.yml was just modified in some places:
bootstrap.yml
uri: https://admin:${jhipster.registry.password}#url.tomy.jhipsterregistryapp/config
name: customerapp
profile: dev # profile(s) of the property source
label: config-dev
application-dev.yml
client:
service-url:
defaultZone: https://admin:${jhipster.registry.password}#url.tomy.jhipsterregistryapp/eureka/
What did I try to set the dev profile in cf:
added -Dspring.profiles.active=dev in gitlab-ci.yml in addition to -Pdev
added SPRING_PROFILES_ACTIVE: dev in the manifest env: section
added SPRING_PROFILES_DEFAULT: dev in the manifest env: section
added SPRING_APPLICATION_JSON: {"spring.cloud.dataflow.applicationProperties.stream.spring.profiles.active": "dev"} (as mentioned in https://github.com/spring-cloud/spring-cloud-dataflow/issues/2317)
added JAVA_OPTS: -Dspring.profiles.active=dev in the manifest env: section (cv env customerapp shows that it was set)
set the JAVA_OPTS -Dspring.profiles.active=dev with cf set-env and cf restage
Any help is appreciated.
Regards
Robert
Forget the answer before. Turns out deep down it was a datasource problem which made the app not respond to the heartbeats.
Uncomment
#hibernate.connection.provider_disables_autocommit: true
In the application properties fixed this.
Maybe any "future" person may stumble upon the same behaviour.
I was able to deploy my jhipster app to cloud foundry.
I somehow "fixed" it, but I am not aware of further consequences. Yet.
Turned out cloud foundry had a problem to monitor my jhipster app via the standard health-check-type http which should be "heartbeat"?
So I decided to switch the monitoring behaviour to a not heartbeat-ish way.
Just switch health-check-type to process in your manifest.yml file.
health-check-type: process
The app is now running.

Resources