How to copy paste Google's SSO certificate for connecting with dex? - base64

I keep getting the follwoing error in dex server -
failed to initialize server: server: Failed to open connector saml: failed to open connector: failed to create connector saml: parse cert: trailing data:
I'm copying the Google SSO certificate, converting it to base64 and pasting it . This is for configuring argocd with google sso login.( https://argo-cd.readthedocs.io/en/release-1.8/operator-manual/user-management/google/) I tried copying the certificate with \n , \r\n and without \n. Still the same error. I'm editing the argocd cm file and adding it. Is there a correct format of copying it?

1: Go to https://www.base64encode.org/ and paste your original cert there for encoding. The original in full format as:
-----BEGIN CERTIFICATE-----
MIIDdDDDD
XXXXXX
VVVVVVV
-----END CERTIFICATE-----
Copy the encoded result string end to end and be careful to have no extra characters.
2: Edit your config map and ensure the yaml formatting is right:
#kubectl -n argocd edit cm argocd-cm
Here is a sample config that worked:
---
#in argocd-cm
data:
url: https://argocd.int.domain.com
dex.config: |
logger:
level: debug
format: json
connectors:
- type: saml
id: saml
name: saml
config:
ssoURL: https://accounts.google.com/o/saml2/idp?idpid=XXXXXXXX
entityIssuer: https://argocd.int.domain.com/api/dex/callback
redirectURI: https://argocd.int.domain.com/api/dex/callback
ssoIssuer: https://accounts.google.com/o/saml2/idp?idpid=XXXXXXXXX
caData: |
LS0tLS1CRUdJTiBXXXXXXXXXXThe long BASE64EncodedString
usernameAttr: name
emailAttr: email
#etcetc
---
I hope this fixes your problem.
Note: Formatting characters in the configmap will likely break things by introducing yaml parse errors so ensure you are not seeing /n and such when you open up the config map after your edit is saved.
You should consider a restart of both the argocd-dex-server and argocd-server deployments and confirm that the logs in the new pods come up clean.
[taproot#ip-10-10-15-500 ~]# kubectl -n argocd rollout restart deployment argocd-dex-server
deployment.apps/argocd-dex-server restarted
[taproot#ip-10-10-15-500 ~]# kubectl -n argocd rollout restart deployment argocd-server
deployment.apps/argocd-server restarted
I had to do the above restart to get rid of prominent errors on the UI that read something like:
"unable to load data: grpc: the client connection is closing"
Ref: https://argoproj.github.io/argo-cd/operator-manual/user-management/google/

Related

Set up basic password authentication for JDBC connections in EMR using Presto

My use case is simple. I have an EMR cluster deployed through CDK running Presto using the AWS Data Catalog as the meta store. The cluster will be having just the default user running queries. By default, the master user is hadoop, which I can use to connect to the cluster via JDBC and run queries. However, I can establish said connection without a password. I have read the Presto docs and they mention LDAP, Kerberos and file based authentication. I just want this to behave like, say, a MySQL database, where I have to pass both username AND password to connect. However, for the life of me, I can't find what configuration to set the password on. These are the settings I have so far:
{
classification: 'spark-hive-site',
configurationProperties: {
'hive.metastore.client.factory.class': 'com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory',
},
},
{
classification: 'emrfs-site',
configurationProperties: {
'fs.s3.maxConnections': '5000',
'fs.s3.maxRetries': '200',
},
},
{
classification: 'presto-connector-hive',
configurationProperties: {
'hive.metastore.glue.datacatalog.enabled': 'true',
'hive.parquet.use-column-names': 'true',
'hive.max-partitions-per-writers': '7000000',
'hive.table-statistics-enabled': 'true',
'hive.metastore.glue.max-connections': '20',
'hive.metastore.glue.max-error-retries': '10',
'hive.s3.use-instance-credentials': 'true',
'hive.s3.max-error-retries': '200',
'hive.s3.max-client-retries': '100',
'hive.s3.max-connections': '5000',
},
},
Which setting can I use to set the hadoop password? Kerberos, LDAP and file based seem overly complicated for this simple use case. Am I missing something obvious?
EDIT
After reading countless pages of documentation and talking to AWS Support, i decided to move to Trino, but am running into more issues. These are the current configurations on my CDK deployment:
configurations: [
{
classification: 'spark-hive-site',
configurationProperties: {
'hive.metastore.client.factory.class': 'com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory',
},
},
{
classification: 'emrfs-site',
configurationProperties: {
'fs.s3.maxConnections': '5000',
'fs.s3.maxRetries': '200',
},
},
{
classification: 'presto-connector-hive',
configurationProperties: {
'hive.metastore.glue.datacatalog.enabled': 'true',
'hive.parquet.use-column-names': 'true',
'hive.max-partitions-per-writers': '7000000',
'hive.table-statistics-enabled': 'true',
'hive.metastore.glue.max-connections': '20',
'hive.metastore.glue.max-error-retries': '10',
'hive.s3.use-instance-credentials': 'true',
'hive.s3.max-error-retries': '200',
'hive.s3.max-client-retries': '100',
'hive.s3.max-connections': '5000',
},
},
{
classification: 'trino-config',
configurationProperties: {
'query.max-memory-per-node': `${instanceMemory * 0.15}GB`, // 25% of a node
'query.max-total-memory-per-node': `${instanceMemory * 0.5}GB`, // 50% of a node
'query.max-memory': `${instanceMemory * 0.5 * coreInstanceGroupNodeCount}GB`, // 50% of the cluster
'query.max-total-memory': `${instanceMemory * 0.8 * coreInstanceGroupNodeCount}GB`, // 80% of the cluster
'query.low-memory-killer.policy': 'none',
'task.concurrency': vcpuCount.toString(),
'task.max-worker-threads': (vcpuCount * 4).toString(),
'http-server.authentication.type': 'PASSWORD',
'http-server.http.enabled': 'false',
'internal-communication.shared-secret': 'abcdefghijklnmopqrstuvwxyz',
'http-server.https.enabled': 'true',
'http-server.https.port': '8443',
'http-server.https.keystore.path': '/home/hadoop/fullCert.pem',
},
},
{
classification: 'trino-password-authenticator',
configurationProperties: {
'password-authenticator.name': 'file',
'file.password-file': '/home/hadoop/password.db',
'file.refresh-period': '5s',
'file.auth-token-cache.max-size': '1000',
},
},
],
I started here:
https://trino.io/docs/current/security/tls.html
I am using this approach:
"Secure the Trino server directly. This requires you to obtain a valid certificate, and add it to the Trino coordinator’s configuration."
I have obtained an internal wildcard certificate from my company. This gets me:
A certificate text
A certificate chain
A private key
From here: https://trino.io/docs/current/security/inspect-pem.html
It seems i need to plug those 3 files into one, for which I do:
-----BEGIN RSA PRIVATE KEY-----
Content of private key
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
Content of certificate text
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
First content of chain
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Second content of chain
-----END CERTIFICATE-----
Then from a bootstrap action, i put the file in all nodes. That way i can fullfil this: https://trino.io/docs/current/security/tls.html#configure-the-coordinator with these configs:
'http-server.https.enabled': 'true',
'http-server.https.port': '8443',
'http-server.https.keystore.path': '/home/hadoop/fullCert.pem',
I know for sure the file is deployed to the nodes. THen I proceeded to do this: https://trino.io/docs/current/security/password-file.html
I also know that particular part works, because if I use the trino CLI directly on the master node with the wrong password, i get a credentials error.
Now, I'm currently stuck doing this:
[hadoop#ip-10-0-10-245 ~]$ trino-cli --server https://localhost:8446 --catalog awsdatacatalog --user hadoop --password --insecure
trino> select 1;
Query 20220701_201620_00001_9nksi failed: Insufficient active worker nodes. Waited 5.00m for at least 1 workers, but only 0 workers are active
From /var/log/trino/server.log I see:
2022-07-01T21:30:12.966Z WARN http-client-node-manager-51 io.trino.metadata.RemoteNodeState Error fetching node state from https://ip-10-0-10-245.ec2.internal:8446/v1/info/state: Failed communicating with server: https://ip-10-0-10-245.ec2.internal:8446/v1/info/state
2022-07-01T21:30:13.902Z ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 8.11ms. Next request will happen within 1000.00ms
2022-07-01T21:30:14.913Z ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 10.35ms. Next request will happen within 1000.00ms
2022-07-01T21:30:15.921Z ERROR Announcer-3 io.airlift.discovery.client.Announcer Service announcement failed after 8.40ms. Next request will happen within 1000.00ms
2022-07-01T21:30:16.930Z ERROR Announcer-0 io.airlift.discovery.client.Announcer Service announcement failed after 8.59ms. Next request will happen within 1000.00ms
2022-07-01T21:30:17.938Z ERROR Announcer-1 io.airlift.discovery.client.Announcer Service announcement failed after 8.36ms. Next request will happen within 1000.00ms
Also with this:
[hadoop#ip-10-0-10-245 ~]$ trino-cli --server https://localhost:8446 --catalog awsdatacatalog --user hadoop --password
trino> select 1;
Error running command: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
trino>
Even though I am following this to upload the .pem files as assets to S3:
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-encryption-enable.html#emr-encryption-certificates
Am I wrong in saying that something this simple shouldn't be this complicated? I really will appreciate any help here.
Based on the message you are getting from Trino, Insufficient active worker nodes, the authentication system is working, and you are now having problems with secure internal communication. Specifically, the machines are having problems talking to each other. I would start by disabling internal TLS, verifying that everything is working, and only then work on enabling that (assuming you need this in your environment). To disable TLS, use:
internal-communication.shared-secret=<secret>
internal-communication.https.required=false
discovery.uri=http://<coordinator ip address>:<http port>
Then restar all your machines. You should not see Service announcement failed. There might be a couple of these when the machines are starting up, but once they establish communication the error messages should stop.

Gitlab integration with Hashicorp Vault

I have integrated my self hosted Gitlab with Hashicorp vault. I have followed the steps here https://docs.gitlab.com/ee/ci/examples/authenticating-with-hashicorp-vault/ and tried to run the pipeline.
I am receiving the certificate error while running the pipeline.
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
My .gitlab yml file -
Vault Client:
image:
name: vault:latest
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
before_script:
script:
- export VAULT_ADDR=https:/vault.systems:8200/
- export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=staging jwt=$CI_JOB_JWT)"
- export PASSWORD="$(vault kv get -field=password kv/project/staging/db)"
- echo $PASSWORD
If i use -tls-skip-verify flag then it works fine.
Do i need to place the self signed server certificate somewhere on the vault server or gitlab server?
Please let me know if anyone has any ideas on this one?
The containers that are managed by the docker/kube executor must be configured to trust the self-signed cert(s). You can edit the config.toml for your runner to mount in the trusted certs/CA roots to GitLab CI job containers
For example, on Linux-based docker executors:
[[runners]]
name = "docker"
url = "https://example.com/"
token = "TOKEN"
executor = "docker"
[runners.docker]
image = "ubuntu:latest"
# Add path to your ca.crt file in the volumes list
volumes = ["/cache", "/path/to-ca-cert-dir/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro"]
See the docs for more info.
I was able to solve this by using this variable VAULT_CACERT in my gitlab.yml file :
- export VAULT_CACERT=/etc/gitlab-runner/certs/ca.crt. The certificate path here is the path of the mounted container which we specify during the start of container.
Posting this so if anyone is looking for it, this is the solution. :)
Error writing data to auth/jwt/login: Put "https://vault.systems:8200/v1/auth/jwt/login": x509: certificate signed by unknown authority
The error you're receiving is being returned from Vault, so it's Vault that you need to get to accept that certificate. There's a decent note on how to do it in the Deployment Guide. (I used to work for HashiCorp Vault so I knew where to dig it up.)
You can use -tls-skip-verify in your vault command vault kv get -tls-skip-verify -field=password kv/project/staging/db , or if you have vault's ca-cert you have to export CA CERT path by setting VAULT_CACERT to the right path .

Azure Kubernetes - No connection to Server

when I execute the following PowerShell command:
.\kubectl get nodes
I get no nodes in response. I noticed that the config file from kubectl is empty too:
apiVersion: v1
clusters:
- cluster:
server: ""
name: cl-kubernetes
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
When I enter the server address at the config file, I get the message that the connection was refused. I suspect that it is due to missing certificates. During another installation this (apparently) following information was created automatically, which is now missing:
certificate-authority-data,
contexts - cluster,
contexts - user,
current context,
users - name,
client-certificate-data,
client-key-data,
token,
Could that be it? If so, where do I get this information?
Many thanks for the help
You need to use the Azure CLI first to get the credentials. Run
az aks get-credentials
https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials

Ansible Lookup with azure_keyvault_secret Invalid Credentails

I'm attempting to retrieve a secret stored in Azure Key Vault with Ansible. I found and installed the azure.azure_preview_modules using ansible-galaxy. I've also updated the ansible.cfg to point to the lookup_plugins directory from the role. When Running the following playbook I get the error:
- hosts: localhost
connection: local
roles:
- { role: azure.azure_preview_modules }
tasks:
- name: Look up secret when ansible host is general VM
vars:
url: 'https://myVault.vault.azure.net/'
secretname: 'SecretPassword'
client_id: 'ServicePrincipalIDHere'
secret: 'ServicePrinipcalPassHere'
tenant: 'TenantIDHere'
debug: msg="the value of this secret is {{lookup('azure_keyvault_secret',secretname,vault_url=url, cliend_id=client_id, secret=secret, tenant_id=tenant)}}"
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'azure_keyvault_secret'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Invalid credentials provided."}
Using the same information I can connect to Azure using AZ PowerShell and AZCLI and retrieve the Azure Key Vault secrets at the commandline. However, those same credentails do not work within this task for the playbook using the lookup plug-in.
I had a similar error when using python sdk (which ansible is built on top of). try changing url to this:
url: 'https://myVault.vault.azure.net' # so remove the trailing slash
the error text is 101% misleading
After much toil I figured out the issue! The argument client_id is misspelled in the example and I didn't catch it which resulted in the error. cliend_id=client_id,
https://github.com/Azure/azure_preview_modules/blob/master/lookup_plugins/azure_keyvault_secret.py#L49
Corrected example below.
- name: Look up secret when ansible host is general VM
vars:
url: 'https://valueName.vault.azure.net'
secretname: 'secretName/version'
client_id: 'ServicePrincipalID'
secret: 'P#ssw0rd'
tenant: 'tenantID'
debug: msg="the value of this secret is {{lookup('azure_keyvault_secret',secretname,vault_url=url, client_id=client_id, secret=secret, tenant_id=tenant)}}"

How to add an Azure AKS Kubernetes Cluster self-signed CA to GitLab CI/CD Kubernetes integration?

I'm trying to add my Azure AKS Kubernetes cluster to my GitLab CI/CD Kubernetes integration.
I can execute kubectl commands on the cluster from my pc, after I ran this command:
az aks get-credentials --resource-group <resource-group-name> --name <kubernetes-cluster-name>
It created a .kube/config file with a content like this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <some long base64 string here>
server: https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443
name: <kubernetes-cluster-name>
contexts:
- context:
cluster: <kubernetes-cluster-name>
user: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
name: <kubernetes-cluster-name>
current-context: <kubernetes-cluster-name>
kind: Config
preferences: {}
users:
- name: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
user:
client-certificate-data: <some long base64 string here>
client-key-data: <some long base64 string here>
token: <some secret string of hexadecimal chars here>
In GitLab form, I have to input these fields:
Kubernetes cluster name
API URL
CA Certificate - Certificate Authority bundle (PEM format)
Token
Project namespace (optional, unique)
I tried these values:
I put my <kubernetes-cluster-name> to match the name of the cluster on azure and the cluster name on the .kube/config file.
I put the url https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443 copied from the .kube/config file.
I tried first the certificate-authority-data from the .kube/config file, but didn't work and I already tried all three base64 strings from the .kube/config file, none worked.
I put the token from the .kube/config file.
Leave this empty, as it is optional.
In GitLab, When I try to hit the button Install to install the Helm Tiller, I got this error:
Something went wrong while installing Helm Tiller
Can't start installation process. nested asn1 error
And sometimes I get this error instead:
Kubernetes error: SSL_connect returned=1 errno=0 state=error: certificate verify failed
I'm trying to make this to work since yesterday, had google it a lot and doesn't find anything.
I think the problem is with this 3rd field, the CA Certificate, maybe there are some other way to get this content from the command line az or kubectl.
Are there someone here that already got this Kubernetes integration from GitLab to Azure AKS working?
I found out later that the base64 string in the certificate-authority-data of the .kube/config file that I was coping its content into the CA Certificate field of GitLab "Add Kubernetes cluster" form, it is the PEM format, but base64 encoded.
The PEM format already is a base64 encoded representation of the certificate bits, but it has some line breaks in the middle. This whole content is base64 encoded again before it goes to the .kube/config so it is turned into a big base64 single-line string.
I just had to base64 decode this big single-line string (I used the javascript atob("....") in the Chrome's Console window), what gave me something like this:
-----BEGIN CERTIFICATE-----
MIIEyDCCArCgAwIBAgIRAOL3N8oMIwWIxcFTZhTkfgMwDQYJKoZIhvcNAQELBQAw
...
...
...
5gP7yoL1peZ+AWjCgcUVZYiItqrBLpWYDgY9g8btYDUIiWlqkmC0+kBaPfwCtckx
cUp3vlwRITrv0mzrxiQjTLTUpEy7EcD+U6IecA==
-----END CERTIFICATE-----
Then I just copied this content into the GitLab "CA Certificate" field and it worked.

Resources