SSL validation failed when I'm using aws cli on windows - aws-cli

I just installed aws cli on my pc, and I want to run a terraform script to build an instance.
Here is the error message:
Error refreshing state: AccessDenied: Access Denied
status code: 403, request id:
I think it's a problem with the SSL certificate because when I do an aws s3 ls s3://MyBucketName --no-verify-ssl
How can I import the certificate?
I didn't see any documentation about how we can configure aws cli on windows.
Thanks,
John.

I doubt whether it's a ssl cert. problem. Try running
aws s3 ls
If it gives the following error -
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
Then it's a ssl cert problem. Otherwise try these steps -
Delete the .terraform directory
Place the access_key and secret_key under the backend block. like below given code
Run terraform init
backend "s3" {
bucket = "great-name-terraform-state-2"
key = "global/s3/terraform.tfstate"
region = "eu-central-1"
access_key = "<access-key>"
secret_key = "<secret-key>"
}
}
Should work.
In case it's an SLS cert problem,
If you want to use SSL and not have to specify the --no-verify-ssl option, then you need to set the AWS_CA_BUNDLE environment variable. e.g from PowerShell:
setx AWS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
The PEM file is a saved copy of the root certificate for the AWS endpoint you are trying to connect to. To generate it, first export the certificate in DER format (For details on how to do this, see here. Then run the following command to convert to the PEM format:
openssl x509 -inform der -in "C:\Users\UserX\Documents\RootCert.der" -out RootCert.pem
If you are using Powershell and not bash, then you will need to first install openssl.
For a full list of environment variables supported by the AWS CLI, see here

Related

GitLab CE 15.7 / Docker registry with Self Signed Certificate.. not working

I want to use the Gitlab Docker registry. I am using GitLab CE 15.7
I created my own CA and signed a certificate. GitLab UI and GitLab runners are working fine!
When it comes to the Docker Registry I have some issues. I configured the gitlab.rb like this:
registry_external_url 'https://198.18.133.100:5000'
registry['enable'] = true
registry['username'] = "registry"
registry['group'] = "registry"
registry['registry_http_addr'] = "127.0.0.1:5000"
registry['debug_addr'] = "localhost:5001"
registry['env'] = {
'SSL_CERT_DIR' => "/etc/gitlab/ssl/"
}
registry['rootcertbundle'] = "/etc/gitlab/ssl/198.18.133.100.crt"
Which also confuses me are the options for registry and registry_nginx.
I am not sure if I configured it correctly and the documentation doesn't help me a lot. I didn't spin up any docker container for the registry or anything. I believe that this comes in the binary of the GitLab (if I am not mistaken). I port 5000 is available and I can telnet.
However, while pushing the image to the registry I get the following error:
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get "https://198.18.133.100:5000/v2/": x509: certificate signed by unknown authority
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1
Any ideas? Thanks a lot!
I tried already quite a lot of different configs and reconfigured the gitlab server.
It has been fixed with copying the ca at the following path:
mkdir -p /etc/docker/certs.d/<your_registry_host_name>:<your_registry_host_port>
As well as the right config in the gitlab.rb
registry_nginx['enable'] = true
registry_nginx['listen_https'] = true
registry_nginx['redirect_http_to_https'] = true
registry_external_url 'https://registry.YOUR_DOMAIN.gtld'
Thanks all for your help!

Why am I having AWS credential errors in my AWS terminal setup?

Wanted to learn AWS and found the tutorial Build a Serverless Web Application. In my research the closest Q&A I could find for my issue was Unable to locate credentials aws cli.
My process has been:
Created a repo in Github
Navigated to IAM and created a user trainer. Tutorial didn't specify policies so chose AdministratorAccess. Per instructions went the Security credentials and Create access key. Downloaded the file locally.
Went to Configuration basics and did Importing a key pair via .CSV file with the command of:
aws configure import --csv file:///Users/path/to/file/aws-training.csv
params:
User name: trainer
Access key ID: ****57
Secret access key: *****1b
but then found that the file didn't contain region or format so did:
aws configure --profile trainer
and re-did all values based on the CSV (Quick Setup):
AWS Access Key ID: ****57
AWS Secret Access Key: *****1b
Default region name: us-east-1
Default output format: json
Made sure to reboot my terminal and locally in a directory I run the command:
aws s3 cp s3://wildrydes-us-east-1/WebApplication/1_StaticWebHosting/website ./ --recursive
The terminal has a delay then throws:
fatal error: Unable to locate credentials
Research
Q&As I've read through to try and see if I could diagnose the problem:
aws cli with shell script: upload failed: Unable to locate credentials
Bash with AWS CLI - unable to locate credentials
Unable to locate credentials aws cli
Unable to locate credentials in boto3 AWS
Get "fatal error: Unable to locate credentials" when I'm copying file from S3 to EC2 using aws cli
Unable to locate credentials when trying to copy files from s3-bucket to my ec2-instance
How can I resolve my error of Unable to locate credentials and what am I doing wrong or misunderstanding?
Per the comment:
Check the content of ~/.aws/credentials and ~/.aws/config
credentials
command:
nano ~/.aws/credentials
renders:
[training]
aws_access_key_id = *****57
aws_secret_access_key = ***1b
[trainer]
aws_access_key_id = *****57
aws_secret_access_key = ***1b
config
command:
nano ~/.aws/config
renders:
[profile training]
region = us-east-1
output = json
[profile trainer]
region = us-east-1
output = json
You've configured the profile with the name trainer. You didn't create a default profile, you created a named profile. You're getting the current error because the CLI tool is looking for a default profile, and you don't have one configured.
In order to use the trainer profile you either have to add --profile trainer to every aws command you run in the command line, or you need to set the AWS_PROFILE environment variable inside your command line environment:
export AWS_PROFILE=trainer
It looks like you also tagged this with nodejs, so I recommend going the environment variable route, which will also work with the nodeJS AWS SDK.

Running terraform plan through sops exec-file

I have an encrypted providers.tfvars.json, so running:
sops exec-file provider.tfvars.json 'terraform plan -var-file={} -out terraform.todo'
I expect terraform to work correctly, but I get an error from terraform:
Error: Argument or block definition required
Decrypting providers.tfvars.json, and running terraform plan -var-file=provider.tfvars.json -out terraform.todo works
I tried sops exec-file --no-fifo to no avail.
Any ideas?
Fixed in sops version 3.7.0 with PR #761
Terraform needs a filename extension for var files, sops didn’t provide one.

Gitlab integration with Hashicorp Vault

I have integrated my self hosted Gitlab with Hashicorp vault. I have followed the steps here https://docs.gitlab.com/ee/ci/examples/authenticating-with-hashicorp-vault/ and tried to run the pipeline.
I am receiving the certificate error while running the pipeline.
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
My .gitlab yml file -
Vault Client:
image:
name: vault:latest
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
script:
- export VAULT_ADDR=https:/vault.systems:8200/
- export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=staging jwt=$CI_JOB_JWT)"
- export PASSWORD="$(vault kv get -field=password kv/project/staging/db)"
- echo $PASSWORD
If i use -tls-skip-verify flag then it works fine.
Do i need to place the self signed server certificate somewhere on the vault server or gitlab server?
Please let me know if anyone has any ideas on this one?
The containers that are managed by the docker/kube executor must be configured to trust the self-signed cert(s). You can edit the config.toml for your runner to mount in the trusted certs/CA roots to GitLab CI job containers
For example, on Linux-based docker executors:
[[runners]]
name = "docker"
url = "https://example.com/"
token = "TOKEN"
executor = "docker"
[runners.docker]
image = "ubuntu:latest"
# Add path to your ca.crt file in the volumes list
volumes = ["/cache", "/path/to-ca-cert-dir/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro"]
See the docs for more info.
I was able to solve this by using this variable VAULT_CACERT in my gitlab.yml file :
- export VAULT_CACERT=/etc/gitlab-runner/certs/ca.crt. The certificate path here is the path of the mounted container which we specify during the start of container.
Posting this so if anyone is looking for it, this is the solution. :)
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
The error you're receiving is being returned from Vault, so it's Vault that you need to get to accept that certificate. There's a decent note on how to do it in the Deployment Guide. (I used to work for HashiCorp Vault so I knew where to dig it up.)
You can use -tls-skip-verify in your vault command vault kv get -tls-skip-verify -field=password kv/project/staging/db , or if you have vault's ca-cert you have to export CA CERT path by setting VAULT_CACERT to the right path .

How to add an Azure AKS Kubernetes Cluster self-signed CA to GitLab CI/CD Kubernetes integration?

I'm trying to add my Azure AKS Kubernetes cluster to my GitLab CI/CD Kubernetes integration.
I can execute kubectl commands on the cluster from my pc, after I ran this command:
az aks get-credentials --resource-group <resource-group-name> --name <kubernetes-cluster-name>
It created a .kube/config file with a content like this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <some long base64 string here>
server: https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443
name: <kubernetes-cluster-name>
contexts:
- context:
cluster: <kubernetes-cluster-name>
user: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
name: <kubernetes-cluster-name>
current-context: <kubernetes-cluster-name>
kind: Config
preferences: {}
users:
- name: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
user:
client-certificate-data: <some long base64 string here>
client-key-data: <some long base64 string here>
token: <some secret string of hexadecimal chars here>
In GitLab form, I have to input these fields:
Kubernetes cluster name
API URL
CA Certificate - Certificate Authority bundle (PEM format)
Token
Project namespace (optional, unique)
I tried these values:
I put my <kubernetes-cluster-name> to match the name of the cluster on azure and the cluster name on the .kube/config file.
I put the url https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443 copied from the .kube/config file.
I tried first the certificate-authority-data from the .kube/config file, but didn't work and I already tried all three base64 strings from the .kube/config file, none worked.
I put the token from the .kube/config file.
Leave this empty, as it is optional.
In GitLab, When I try to hit the button Install to install the Helm Tiller, I got this error:
Something went wrong while installing Helm Tiller
Can't start installation process. nested asn1 error
And sometimes I get this error instead:
Kubernetes error: SSL_connect returned=1 errno=0 state=error: certificate verify failed
I'm trying to make this to work since yesterday, had google it a lot and doesn't find anything.
I think the problem is with this 3rd field, the CA Certificate, maybe there are some other way to get this content from the command line az or kubectl.
Are there someone here that already got this Kubernetes integration from GitLab to Azure AKS working?
I found out later that the base64 string in the certificate-authority-data of the .kube/config file that I was coping its content into the CA Certificate field of GitLab "Add Kubernetes cluster" form, it is the PEM format, but base64 encoded.
The PEM format already is a base64 encoded representation of the certificate bits, but it has some line breaks in the middle. This whole content is base64 encoded again before it goes to the .kube/config so it is turned into a big base64 single-line string.
I just had to base64 decode this big single-line string (I used the javascript atob("....") in the Chrome's Console window), what gave me something like this:
-----BEGIN CERTIFICATE-----
MIIEyDCCArCgAwIBAgIRAOL3N8oMIwWIxcFTZhTkfgMwDQYJKoZIhvcNAQELBQAw
...
...
...
5gP7yoL1peZ+AWjCgcUVZYiItqrBLpWYDgY9g8btYDUIiWlqkmC0+kBaPfwCtckx
cUp3vlwRITrv0mzrxiQjTLTUpEy7EcD+U6IecA==
-----END CERTIFICATE-----
Then I just copied this content into the GitLab "CA Certificate" field and it worked.

Resources