I have integrated my self hosted Gitlab with Hashicorp vault. I have followed the steps here https://docs.gitlab.com/ee/ci/examples/authenticating-with-hashicorp-vault/ and tried to run the pipeline.
I am receiving the certificate error while running the pipeline.
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
My .gitlab yml file -
Vault Client:
image:
name: vault:latest
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
script:
- export VAULT_ADDR=https:/vault.systems:8200/
- export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=staging jwt=$CI_JOB_JWT)"
- export PASSWORD="$(vault kv get -field=password kv/project/staging/db)"
- echo $PASSWORD
If i use -tls-skip-verify flag then it works fine.
Do i need to place the self signed server certificate somewhere on the vault server or gitlab server?
Please let me know if anyone has any ideas on this one?
The containers that are managed by the docker/kube executor must be configured to trust the self-signed cert(s). You can edit the config.toml for your runner to mount in the trusted certs/CA roots to GitLab CI job containers
For example, on Linux-based docker executors:
[[runners]]
name = "docker"
url = "https://example.com/"
token = "TOKEN"
executor = "docker"
[runners.docker]
image = "ubuntu:latest"
# Add path to your ca.crt file in the volumes list
volumes = ["/cache", "/path/to-ca-cert-dir/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro"]
See the docs for more info.
I was able to solve this by using this variable VAULT_CACERT in my gitlab.yml file :
- export VAULT_CACERT=/etc/gitlab-runner/certs/ca.crt. The certificate path here is the path of the mounted container which we specify during the start of container.
Posting this so if anyone is looking for it, this is the solution. :)
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
The error you're receiving is being returned from Vault, so it's Vault that you need to get to accept that certificate. There's a decent note on how to do it in the Deployment Guide. (I used to work for HashiCorp Vault so I knew where to dig it up.)
You can use -tls-skip-verify in your vault command vault kv get -tls-skip-verify -field=password kv/project/staging/db , or if you have vault's ca-cert you have to export CA CERT path by setting VAULT_CACERT to the right path .
Related
I'm really stuck here. I inherited a system which stores secrets in a Hashicorp vault, and I'm getting this error, Authentication failed: ldap operation failed: unable to retrieve user bind DN
I am not sure how to resolve this issue, and have been Googling for hours, and trying a lot of things.
I did see the post at ref. [A], but it isn't helpful.
Also the post at ref [B] gives some information about setting the binddn, but in the classic way to frustrate a new user, doesn't say where, how, or give any examples.
Hashicorp Vault v1.6.x
The vault is running on a docker container, on an AWS EC2.
... I have the .pem file, and am able to ssh into the EC2
. I am able to ssh into the docker container with root priv, like so:
... docker exec -it 123abc123abc sh
On the container, some vault commands work; e.g:
... vault version
--> Vault v1.6.0 (123asdf1234adsf1234adsf1234adsf13w4radsf1234asdff)
It is using ldap configuration
When trying to retrieve config and other info, I get this message:
... a. "* missing client token"
How to proceed?
I'm not an expert with this, and would appreciate clear, full, command-line examples.
Thanks for your help.
Sincerely,
Keith
DOCKER COMPOSE FILE
$ cat docker-compose.yml
version: '3'
services:
vault:
image: vault:1.6.0
cap_add:
- IPC_LOCK
environment:
- VAULT_ADDR=http://127.0.0.1:8200
command: vault server -config=/vault/config/config.json
ports:
- 80:8200
volumes:
- vault-data:/vault
- ./config.json:/vault/config/config.json
volumes:
vault-data:ubuntu#ip-192-0-2-1:/home/tarjan-docker
VAULT CONFIG
/vault/config # cat config.json
{
"backend": {
"file": {
"path": "/vault/data"
}
},
"listener": {
"tcp":{
"address": "0.0.0.0:8200",
"tls_disable": 1
}
},
"default_lease_ttl": "30m",
"max_lease_ttl": "30m",
"log_level": "info",
"ui": true
}
A. https://discuss.hashicorp.com/t/ldap-operation-failed-unable-to-retrieve-user-bind-dn/12926
B. https://support.hashicorp.com/hc/en-us/articles/5289574376083-Receiving-ldap-operation-failed-failed-to-bind-as-user-error-when-logging-in-via-LDAP-authentication-method
https://discuss.hashicorp.com/t/authentication-failed-ldap-operation-failed-unable-to-retrieve-user-bind-dn/50123
I want to use the Gitlab Docker registry. I am using GitLab CE 15.7
I created my own CA and signed a certificate. GitLab UI and GitLab runners are working fine!
When it comes to the Docker Registry I have some issues. I configured the gitlab.rb like this:
registry_external_url 'https://198.18.133.100:5000'
registry['enable'] = true
registry['username'] = "registry"
registry['group'] = "registry"
registry['registry_http_addr'] = "127.0.0.1:5000"
registry['debug_addr'] = "localhost:5001"
registry['env'] = {
'SSL_CERT_DIR' => "/etc/gitlab/ssl/"
}
registry['rootcertbundle'] = "/etc/gitlab/ssl/198.18.133.100.crt"
Which also confuses me are the options for registry and registry_nginx.
I am not sure if I configured it correctly and the documentation doesn't help me a lot. I didn't spin up any docker container for the registry or anything. I believe that this comes in the binary of the GitLab (if I am not mistaken). I port 5000 is available and I can telnet.
However, while pushing the image to the registry I get the following error:
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get "https://198.18.133.100:5000/v2/": x509: certificate signed by unknown authority
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1
Any ideas? Thanks a lot!
I tried already quite a lot of different configs and reconfigured the gitlab server.
It has been fixed with copying the ca at the following path:
mkdir -p /etc/docker/certs.d/<your_registry_host_name>:<your_registry_host_port>
As well as the right config in the gitlab.rb
registry_nginx['enable'] = true
registry_nginx['listen_https'] = true
registry_nginx['redirect_http_to_https'] = true
registry_external_url 'https://registry.YOUR_DOMAIN.gtld'
Thanks all for your help!
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -
Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -
Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "-s ci_settings.xml --batch-mode --errors --fail-at-end --show-version"
http_proxy: http://genproxy.myorg.com:8080
https_proxy: http://genproxy.myorg.com:8080
no_proxy: .myorg.com,localhost,127.0.0.1
deploy:
image: maven:3.6-jdk-11
before_script:
- 'gitlab-runner register --tls-ca-file xxxxx.pem'
script:
- 'mvn $MAVEN_CLI_OPTS deploy'
only:
- master
I have tried all the possible solutions with ssl_verify false, GIT_SSL_NO_VERIFY: "1". I am not using custom runner because i am not the admin. I need to add the cert in the default runner that gitlab ci invokes.
Currently I am getting a X509 PKIX error when I am trying to publish my jar to package registry of gitlab.
Update: I also tried with adding certs and update the ca certificates
before_script:
- 'cp gitlab.corp.myorg.com.crt /usr/local/share/ca-certificates'
- 'update-ca-certificates'
Gitlab runner added the cert but I still get the PKIX - 509 error.
Can anyone help?
You are on the right track in that you need to make the missing RootCA known. Please note that disabling TLS Verification will increase the attack surface, so you better avoid it.
Registering a runner in the before script is very unlikely to help you here. The command you run in your first attempt tries to register the current system as a new runner with the CA file to trust the GitLab Server. This has nothing to do with what you want to achieve.
Your update helps you by making the CA known to your system (in this case whatever linux distro the maven image is built on). I suggest you keep it to avoid further bugs even if it might not solve your issue here.
Maven depends on Java and as luck would have it, Java does it's own CA management. Hence you need to make the RootCA that signed you MVN-Reposerver known to Java. You seem to be using GitLab for MVN-Repos too, so that would be the RootCA that signed gitlab.corp.myorg.com.
You can find the right certificate in your browsers certificate details (lock-symbol next to the URL in most browsers) as the ISSUER.
Java uses keytool for managing it's trust anchor so with Java 11 you might want to execute
keytool -import -trustcacerts -alias <named-file> -file <certificate-file-path> -keystore "$JAVA_HOME/lib/security/cacerts"
in your before_script.
If your CI job needs to leverage a proxy you might want to trust the proxy's signing CA as well (same procedure).
Example:
before_script:
- cp /path/too/mycorp.rootca.pem /usr/local/share/ca-certificates
- update-ca-certificates
- keytool -import -trustcacerts -alias my-gitlab -file /path/too/mycorp.rootca.pem -keystore "$JAVA_HOME/lib/security/cacerts"
I just installed aws cli on my pc, and I want to run a terraform script to build an instance.
Here is the error message:
Error refreshing state: AccessDenied: Access Denied
status code: 403, request id:
I think it's a problem with the SSL certificate because when I do an aws s3 ls s3://MyBucketName --no-verify-ssl
How can I import the certificate?
I didn't see any documentation about how we can configure aws cli on windows.
Thanks,
John.
I doubt whether it's a ssl cert. problem. Try running
aws s3 ls
If it gives the following error -
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
Then it's a ssl cert problem. Otherwise try these steps -
Delete the .terraform directory
Place the access_key and secret_key under the backend block. like below given code
Run terraform init
backend "s3" {
bucket = "great-name-terraform-state-2"
key = "global/s3/terraform.tfstate"
region = "eu-central-1"
access_key = "<access-key>"
secret_key = "<secret-key>"
}
}
Should work.
In case it's an SLS cert problem,
If you want to use SSL and not have to specify the --no-verify-ssl option, then you need to set the AWS_CA_BUNDLE environment variable. e.g from PowerShell:
setx AWS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
The PEM file is a saved copy of the root certificate for the AWS endpoint you are trying to connect to. To generate it, first export the certificate in DER format (For details on how to do this, see here. Then run the following command to convert to the PEM format:
openssl x509 -inform der -in "C:\Users\UserX\Documents\RootCert.der" -out RootCert.pem
If you are using Powershell and not bash, then you will need to first install openssl.
For a full list of environment variables supported by the AWS CLI, see here
I'm trying to add my Azure AKS Kubernetes cluster to my GitLab CI/CD Kubernetes integration.
I can execute kubectl commands on the cluster from my pc, after I ran this command:
az aks get-credentials --resource-group <resource-group-name> --name <kubernetes-cluster-name>
It created a .kube/config file with a content like this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <some long base64 string here>
server: https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443
name: <kubernetes-cluster-name>
contexts:
- context:
cluster: <kubernetes-cluster-name>
user: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
name: <kubernetes-cluster-name>
current-context: <kubernetes-cluster-name>
kind: Config
preferences: {}
users:
- name: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
user:
client-certificate-data: <some long base64 string here>
client-key-data: <some long base64 string here>
token: <some secret string of hexadecimal chars here>
In GitLab form, I have to input these fields:
Kubernetes cluster name
API URL
CA Certificate - Certificate Authority bundle (PEM format)
Token
Project namespace (optional, unique)
I tried these values:
I put my <kubernetes-cluster-name> to match the name of the cluster on azure and the cluster name on the .kube/config file.
I put the url https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443 copied from the .kube/config file.
I tried first the certificate-authority-data from the .kube/config file, but didn't work and I already tried all three base64 strings from the .kube/config file, none worked.
I put the token from the .kube/config file.
Leave this empty, as it is optional.
In GitLab, When I try to hit the button Install to install the Helm Tiller, I got this error:
Something went wrong while installing Helm Tiller
Can't start installation process. nested asn1 error
And sometimes I get this error instead:
Kubernetes error: SSL_connect returned=1 errno=0 state=error: certificate verify failed
I'm trying to make this to work since yesterday, had google it a lot and doesn't find anything.
I think the problem is with this 3rd field, the CA Certificate, maybe there are some other way to get this content from the command line az or kubectl.
Are there someone here that already got this Kubernetes integration from GitLab to Azure AKS working?
I found out later that the base64 string in the certificate-authority-data of the .kube/config file that I was coping its content into the CA Certificate field of GitLab "Add Kubernetes cluster" form, it is the PEM format, but base64 encoded.
The PEM format already is a base64 encoded representation of the certificate bits, but it has some line breaks in the middle. This whole content is base64 encoded again before it goes to the .kube/config so it is turned into a big base64 single-line string.
I just had to base64 decode this big single-line string (I used the javascript atob("....") in the Chrome's Console window), what gave me something like this:
-----BEGIN CERTIFICATE-----
MIIEyDCCArCgAwIBAgIRAOL3N8oMIwWIxcFTZhTkfgMwDQYJKoZIhvcNAQELBQAw
...
...
...
5gP7yoL1peZ+AWjCgcUVZYiItqrBLpWYDgY9g8btYDUIiWlqkmC0+kBaPfwCtckx
cUp3vlwRITrv0mzrxiQjTLTUpEy7EcD+U6IecA==
-----END CERTIFICATE-----
Then I just copied this content into the GitLab "CA Certificate" field and it worked.