Drone.IO – GitLab address with hosts file - gitlab

I have a GitLab instance on a fixed internal IP, and configured the hosts file on the clients if a hostname which points to that IP.
But we are unable to make Drone work when using the hostname. We have Gitlab on port 80 and Drone on port 8000. It does work when it’s configured to the fixed ip:
Our compose file:
version: '2'
services:
gitlab:
image: 'gitlab/gitlab-ce:latest'
restart: always
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://our.host.name.com'
gitlab_rails['gitlab_shell_ssh_port'] = 22
# Add any other gitlab.rb configuration here, each on its own line
ports:
- '80:80'
- '443:443'
- '22:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
drone-server:
image: drone/drone:0.7.3
links:
- "gitlab:our.host.name.com"
ports:
- "8000:8000"
volumes:
- /home/drone:/var/lib/drone/
- /home/drone/cache:/var/lib/drone/cache/
- /home/foresight/keys:/var/lib/drone/keys
environment:
DRONE_OPEN: "true"
DRONE_SERVER_ADDR: ":8000"
DRONE_HOST: "http://our.host.name.com:8000"
DRONE_ADMIN: <redacted>
DRONE_GITLAB: "true"
DRONE_GITLAB_URL: "http://our.host.name.com"
DRONE_GITLAB_CLIENT: <redacted>
DRONE_GITLAB_SECRET: <redacted>
DRONE_SECRET: <redacted>
drone-agent:
image: drone/drone:0.7.3
command: agent
depends_on:
- drone-server
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
DRONE_SERVER: ws://drone-server:8000/ws/broker
DRONE_DEBUG: "true"
DRONE_SECRET: <redacted>
When i configure it this way, drone doesn’t seem to accept triggers from GitLab.
Naturally, i reset the drone cache, updated the callback url to use the domain name also, and re-activated the repositories.
i couldn’t find any log file for drone that provides insight on why the trigger fails.

Related

Can't Access Traefik dashboard outside server

I am learning traefik and I am trying to connect to Ubuntu server hosted on a raspberry pi, Traefik is running on Docker. Currently I just want to access the dashboard, however am unable to access it outside my server but can ping it from inside the server itself,
This is how my traefik.yml file looks like
api:
dashboard: true
insecure: true
entryPoints:
http:
address: ":80"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
docker-compose file
services:
proxy:
image: traefik
networks:
- traefik
ports:
- "80:80"
- "8080:8080"
command:
- "--api.insecure=true"
- "--api.dashboard=true"
- "--providers.docker"
- "--log.level=DEBUG"
- "--configFile=/home/ubuntu/traefik/traefik.yml"
volumes:
- $PWD/traefik.yml:/home/ubuntu/traefik/traefik.yml
- $PWD/acme.json:/home/ubuntu/traefik/acme.json
- /var/run/docker.sock:/var/run/docker.sock
restart: unless-stopped
networks:
traefik:
external:
name: traefik
I have docker network registered called traefik
NETWORK ID NAME DRIVER SCOPE
...
ef66....... traefik bridge local

How to use Traefik to reverse proxy an already running node service?

I've got a node process running on port 3000 using pm2.
I want to configure Traefik so that it reverse proxies this service on port 80.
Following this excellent blog post, I was able to quickly start Traefik using docker compose and set up a skeleton config for the node-server.
However, that example assumes the node process is hosted inside a docker as well. I couldn't get this to work for my node process (*) so I just want to be able to configure Traefik by pointing to port 3000 in some way. Seems straightforward but couldn't get it to work.
I'm stuck with the following config (which is a mix of various blog-posts without actually knowing what I'm doing):
services:
reverse-proxy:
image: traefik:v2.4
container_name: "traefik"
command:
- "--api.insecure=true"
- "--api.dashboard=true"
- "--api.debug=true"
- "--providers.docker=true"
- "--log.LEVEL=DEBUG"
- "--entryPoints.web.address=:80"
- "--entryPoints.websecure.address=:443"
- "--providers.docker.exposedbydefault=false"
- "--certificatesresolvers.myresolver.acme.httpchallenge=true"
- "--certificatesresolvers.myresolver.acme.httpchallenge.entrypoint=web"
- "--certificatesresolvers.myresolver.acme.email=xxxx#xxx.com"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
ports:
- "443:443"
- "80:80"
- "8080:8080"
volumes:
- "./letsencrypt:/letsencrypt"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
node-server:
loadBalancer:
servers:
- url: http://127.0.0.1:3000/
labels:
- "traefik.enable=true"
- "traefik.http.routers.node-server.rule=Host(`xxxxxx.com`)"
- "traefik.http.routers.node-server.entrypoints=websecure"
- "traefik.http.routers.node-server.tls.certresolver=myresolver"
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
- "traefik.http.routers.redirs.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.redirs.entrypoints=web"
- "traefik.http.routers.redirs.middlewares=redirect-to-https"
This gives the error: 'Unsupported config option for services.node-server: 'loadBalancer'"
Long story short: how would I configure Traefik to just reverse proxy a service running on port 3000?
*) A total newbie to Docker and I couldn't get the situation to work, where the node process depends on custom javascript modules in a parent directory. Perhaps there's a way to do this and I could do it in the 'host node in docker' way instead. I'm all ears
A few months ago I have configured a reverse proxy, here you go my configuration:
version: '3'
services:
reverse-proxy:
image: traefik:v2.5
container_name: selling-point-reverse-proxy
ports:
- 80:80
- 8080:8080
volumes:
# Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
command:
# Enables the web UI
- --api.insecure=true
# Tells Traefik to listen to docker
- --providers.docker
# Creates a new entrypoint called web
- --entrypoints.web.address=:80
# Disable container exposition
- --providers.docker.exposedByDefault=false
# Traefik matches against the container's labels to determine whether to create any route for that container
- --providers.docker.constraints=Label(`traefik.scope`,`selling-point`)
# Enable tracing (using jaeger by default)
- --tracing=true
# Name of the tracing service on Jaeger
- --tracing.serviceName=reverse-proxy
# Host and port of the Jaeger agent
- --tracing.jaeger.localAgentHostPort=jaeger:6831
labels:
# Matcher for creating a route
- traefik.scope=selling-point
# Exposes container
- traefik.enable=true
# Creates circuit breaker middleware
- traefik.http.middlewares.latency.circuitbreaker.expression=LatencyAtQuantileMS(50.0) > 10000
# Creates a forward auth middleware
- traefik.http.middlewares.auth.forwardauth.address=http://auth:3000/auth/authorize
# Enables cross origin requests
- traefik.http.middlewares.cors.headers.accesscontrolalloworiginlist=*
# Enables forwarding of the request headers
- traefik.http.middlewares.cors.headers.accessControlAllowHeaders=*
networks:
- selling-point
api:
image: selling-point-api
container_name: selling-point-api
build:
context: ./selling-point-api
labels:
# Tells Traefik where to redirect the request if the url has the specified prefix
- traefik.http.routers.api.rule=PathPrefix(`/api`)
# Attaches a middleware for forwarding the authentication
- traefik.http.routers.api.middlewares=cors,auth,latency
# Attaches entrypoints
- traefik.http.routers.api.entrypoints=web
# Exposes container
- traefik.enable=true
# Matcher for creating a route
- traefik.scope=selling-point
# Creates a service called selling-point-api
- traefik.http.services.selling-point-api.loadbalancer.server.port=3000
# Attach the container to a service
- traefik.http.routers.api.service=selling-point-api
volumes:
- ./selling-point-api/src:/app/src
networks:
- selling-point
environment:
WAIT_HOSTS: mysql:3306
DATABASE_URL: mysql://root:huachinango#mysql:3306/selling_point
NODE_ENV: development
auth:
image: selling-point-auth
container_name: selling-point-auth
build:
context: ./selling-point-auth
labels:
# Tells Traefik where to redirect the request if the url has the specified prefix
- traefik.http.routers.auth.rule=PathPrefix(`/auth`)
# Attaches a circuit breaker middleware
- traefik.http.routers.auth.middlewares=cors,latency
# Attaches entrypoints
- traefik.http.routers.auth.entrypoints=web
# Exposes container
- traefik.enable=true
# Matcher for creating a route
- traefik.scope=selling-point
# Creates a service called selling-point-auth
- traefik.http.services.selling-point-auth.loadbalancer.server.port=3000
# Attach the container to a service
- traefik.http.routers.auth.service=selling-point-auth
environment:
WAIT_HOSTS: mysql:3306
IGNORE_ENV_FILE: 'true'
DATABASE_URL: mysql://root:huachinango#mysql:3306/selling_point
PASSWORD_SALT: $$2b$$10$$g0OI8KtIE3j6OQqt1ZUDte
NODE_ENV: development
volumes:
- ./selling-point-auth/src:/app/src
networks:
- selling-point
mysql:
image: mysql:5
container_name: selling-point-mysql
environment:
MYSQL_ROOT_PASSWORD: huachinango
MYSQL_DATABASE: selling_point
networks:
- selling-point
volumes:
- mysql-db:/var/lib/mysql
jaeger:
image: jaegertracing/all-in-one:1.29
container_name: selling-point-tracing
environment:
COLLECTOR_ZIPKIN_HOST_PORT: :9411
ports:
- 16686:16686
networks:
- selling-point
volumes:
mysql-db:
networks:
selling-point:
name: selling-point
driver: bridge

Jhipster registry not working on localhost

thanks in advance for this awesome stack platform that is jhipster.
I have a question, I am trying to run a microservice directly with:
./mvnw -Pdev -DskipTests
And I am getting (UnknownHostException -- 'http://admin:admin#jhipster-registry:8761/eureka/):
2021-09-16 10:06:26.225 INFO 6762 --- [ restartedMain] c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://admin:admin#jhipster-registry:8761/eureka/}, exception=I/O error on GET request for "http://admin:admin#jhipster-registry:8761/eureka/apps/": jhipster-registry: Name or service not known; nested exception is java.net.UnknownHostException: jhipster-registry: Name or service not known stacktrace=org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://admin:admin#jhipster-registry:8761/eureka/apps/": jhipster-registry: Name or service not known; nested exception is java.net.UnknownHostException: jhipster-registry: Name or service not known
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:711)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:602)
at org.springframework.cloud.netflix.eureka.http.RestTemplateEurekaHttpClient.getApplic
My doubt is, why is trying to use the domain jhipster-registry:8761 instead of what I have in the dev configurations, "localhost"?
eureka:
instance:
prefer-ip-address: true
client:
service-url:
defaultZone: http://admin:${jhipster.registry.password}#localhost:8761/eureka/
Right now I am using docker-compose in order to run the needed services, like the registry:
services:
jhipster-registry:
image: jhipster/jhipster-registry:v6.8.0
volumes:
- ./central-server-config:/central-config
# By default the JHipster Registry runs with the "dev" and "native"
# Spring profiles.
# "native" profile means the filesystem is used to store data, see
# http://cloud.spring.io/spring-cloud-config/spring-cloud-config.html
environment:
- _JAVA_OPTIONS=-Xmx512m -Xms256m
- JHIPSTER_SLEEP=20
- SPRING_PROFILES_ACTIVE=dev,oauth2
- SPRING_SECURITY_USER_PASSWORD=admin
- JHIPSTER_REGISTRY_PASSWORD=admin
- SPRING_CLOUD_CONFIG_SERVER_COMPOSITE_0_TYPE=native
- SPRING_CLOUD_CONFIG_SERVER_COMPOSITE_0_SEARCH_LOCATIONS=file:./central-config
# - SPRING_CLOUD_CONFIG_SERVER_COMPOSITE_0_TYPE=git
# - SPRING_CLOUD_CONFIG_SERVER_COMPOSITE_0_URI=https://github.com/jhipster/jhipster-registry/
# - SPRING_CLOUD_CONFIG_SERVER_COMPOSITE_0_SEARCH_PATHS=central-config
# For Keycloak to work, you need to add '127.0.0.1 keycloak' to your hosts file
- SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_OIDC_ISSUER_URI=http://keycloak:9080/auth/realms/jhipster
- SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_OIDC_CLIENT_ID=jhipster-registry
- SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_OIDC_CLIENT_SECRET=jhipster-registry
ports:
- 8761:8761
keycloak:
image: jboss/keycloak:12.0.4
command:
[
"-b",
"0.0.0.0",
"-Dkeycloak.migration.action=import",
"-Dkeycloak.migration.provider=dir",
"-Dkeycloak.migration.dir=/opt/jboss/keycloak/realm-config",
"-Dkeycloak.migration.strategy=OVERWRITE_EXISTING",
"-Djboss.socket.binding.port-offset=1000",
"-Dkeycloak.profile.feature.upload_scripts=enabled",
]
volumes:
- ./realm-config:/opt/jboss/keycloak/realm-config
environment:
- KEYCLOAK_USER=admin
- KEYCLOAK_PASSWORD=admin
- DB_VENDOR=h2
ports:
- 9080:9080
- 9443:9443
- 10990:10990
test-mysql:
container_name: test-mysql
restart: always
image: mysql:8.0.25
environment:
MYSQL_ROOT_PASSWORD: 'root'
ports:
# <Port exposed> : < MySQL Port running inside container>
- '3306:3306'
expose:
# Opens port 3306 on the container
- '3306'
volumes:
- test-datavolume:/var/lib/mysql
volumes:
test-datavolume:
I know that if I add into the /etc/hosts the entry "127.0.0.1 jhipster-registry" is going to work, but I cant find/understand why is trying to use jhipster-registry instead of localhost?
Thanks!

How to configure dns-proxy-server in docker-compose to bind to a non-default IP address

I am using mageddo/dns-proxy-server in my RoR environment to provide DNS resolution for the docker containers. This works like a treat until I connect to the company VPN. Unfortunately, dns-proxy-server binds to the port 172.17.0.2 which conflicts with the VPN. My other containers are using a different IP range. Here is the docker-compose file:
version: '3.7'
services:
jet-db:
image: postgres:latest
hostname: jet-db
ports:
- 5432:5432
environment:
POSTGRES_USER: '*'
POSTGRES_PASSWORD: '*'
volumes:
- database_data:/var/lib/postgresql/data
sidekiq:
build:
context: .
dockerfile: docker/app/ruby-on-rails.dockerfile
command: bundle exec sidekiq -v
environment:
POSTGRES_URL: 'postgres://db/'
RAILS_CACHE_URL: 'redis://rails-cache:6380/'
links:
- jet-db
- sidekiq-cache
- rails-cache
volumes:
- '.:/app'
sidekiq-cache:
image: redis
hostname: sidekiq-cache
command: redis-server /usr/local/etc/redis/redis.conf
ports:
- 6379:6379
volumes:
- ./docker/redis/sidekiq.conf:/usr/local/etc/redis/redis.conf
rails-cache:
image: redis
hostname: rails-cache
command: redis-server /usr/local/etc/redis/redis.conf
ports:
- 6380:6380
volumes:
- ./docker/redis/rails-cache.conf:/usr/local/etc/redis/redis.conf
mailcatcher:
image: zolweb/docker-mailcatcher:latest
ports:
- "1025:1025"
- "1080:1080"
dns-proxy-server:
image: defreitas/dns-proxy-server:latest
hostname: dns.mageddo
volumes:
- /opt/dns-proxy-server/conf:/app/conf
- /var/run/docker.sock:/var/run/docker.sock
- /etc/resolv.conf:/etc/resolv.conf
network_mode: bridge
ports:
- "5380:5380"
volumes:
database_data:
driver: local
tmp:
driver: local
networks:
default:
driver: bridge
ipam:
config:
- subnet: 169.10.0.0/16
Here is what /etc/resolv.conf looks like when the proxy is running normally without VPN:
:# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
# nameserver 8.8.8.8 # dps-comment
# nameserver 127.0.0.53 # dps-comment
nameserver 172.17.0.2 # dps-entry
I am wondering whether there is a way to get dns-proxy-server to bind to a different IP address to avoid the conflict with the VPN.

How to run docker-compose in Azure Container Service and deploy to agent rather than master?

I follow this article (https://blogs.msdn.microsoft.com/jcorioland/2016/04/25/create-a-docker-swarm-cluster-using-azure-container-service/#comment-1015) to setup a swarm docker host cluster. There are 1 master and 2 agents.The good points for this article is to use "-H 172.16.0.5:2375" which creates new containers on "agent" rather than "master" one.
My question is: if I want to make docker-compose.yml work with that, how could I do it? I have tried command like:
docker-compose -H 172.16.0.5:2375 up
But it doesn't work. If I just use:
docker-compose up
Then the containers will be created on master host and I couldn't even use public DNS to visit the website.
Here is the yml file I use for 1 magento & 1 mariadb containers:
version: '2'
services:
mariadb:
image: 'bitnami/mariadb:latest'
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- '3306:3306'
volumes:
- 'mariadb_data:/bitnami/mariadb'
magento:
image: 'bitnami/magento:latest'
environment:
- MAGENTO_HOST=172.16.0.5
- MARIADB_HOST=172.16.0.5
ports:
- '80:80'
volumes:
- 'magento_data:/bitnami/magento'
- 'apache_data:/bitnami/apache'
- 'php_data:/bitnami/php'
depends_on:
- mariadb
volumes:
mariadb_data:
driver: local
magento_data:
driver: local
apache_data:
driver: local
php_data:
driver: local
And this section is from my guess based on that article,
environment:
- MAGENTO_HOST=172.16.0.5
- MARIADB_HOST=172.16.0.5
but yml doesn't like port appended, e.g.
environment:
- MAGENTO_HOST=172.16.0.5:2375
- MARIADB_HOST=172.16.0.5:2375
Thanks a lot!

Resources