I have a AKS and ACR, and attached the ACR successfully with my AKS using
az aks update -n <AKSNAME> -g <RESOURCE> --attach-acr <ACRNAME>
Yet, when I run the command below from this how-to-guide, I get a Error: failed pre-install: timed out waiting for the condition. Upon further investigating with kubectl get events, I find that the images I pull from ACR are failing due to authorization: failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized.
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace ingress-basic \
--set controller.replicaCount=2 \
--set controller.nodeSelector."kubernetes\.io/os"=linux \
--set controller.image.registry=$ACR_URL \
--set controller.image.image=$CONTROLLER_IMAGE \
--set controller.image.tag=$CONTROLLER_TAG \
--set controller.image.digest="" \
--set controller.admissionWebhooks.patch.nodeSelector."kubernetes\.io/os"=linux \
--set controller.admissionWebhooks.patch.image.registry=$ACR_URL \
--set controller.admissionWebhooks.patch.image.image=$PATCH_IMAGE \
--set controller.admissionWebhooks.patch.image.tag=$PATCH_TAG \
--set defaultBackend.nodeSelector."kubernetes\.io/os"=linux \
--set defaultBackend.image.registry=$ACR_URL \
--set defaultBackend.image.image=$DEFAULTBACKEND_IMAGE \
--set defaultBackend.image.tag=$DEFAULTBACKEND_TAG \
--set controller.service.loadBalancerIP=$STATIC_IP \
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-dns-label-name"=$DNS_LABEL
I find this very strange, cause I have the images in my ACR, and I have successfully authenticated ACR with AKS.
I run everything from the Azure CLI, Helm version 3.4.0. I found some related issues, which all use the attach-acr.
Related
I'm currently trying to deploy my helm charts through my private aks cluster. However, I'm unable to do anything since it can't find the path of my local directory.
This is the command that I'm running:
az aks command invoke \
--resource-group aharo-aks-appgateway01 \
--name aharo-aks02 \
--command "helm install haro ./haro_files_helm_chart"
This is the error message that I'm getting
command started at 2023-01-06 22:49:46+00:00, finished at 2023-01-06 22:49:46+00:00 with exitcode=1
Error: INSTALLATION FAILED: path "./haro_files_helm_chart" not found
To prove that this type of commands can work, I tried one from the Microsoft Documentation:
az aks command invoke \
--resource-group aharo-aks-appgateway01 \
--name aharo-aks02 \
--command "helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update && helm install my-release bitnami/nginx"
What else can I do to find the path of my directory? Do you know if I could be missing any configuration on my cluster?
When you are passing the helm install command to the AKS VMs, the VMs(nodes) will be looking for ./haro_files_helm_chart in their filesystem not the machine that is running the command, hence the path not found error.
In the example you shared, the node is installing a helm chart that it is downloading first.
To resolve the issue, you should attach the directory of the helm chart with the az aks command invoke as documented here. Below is the part you need:
You can also attach all files in the current directory. For example:
az aks command invoke \
--resource-group myResourceGroup \
--name myAKSCluster \
--command "kubectl apply -f deployment.yaml configmap.yaml -n default" \
--file .
For example, I created a chart called "test-chart" and installed it using helm create test-chart. The chart would be created in the current directory im in:
$ls
test-chart
Then run the same command shared above and just change the command (without changing the directory):
az aks command invoke \
--resource-group myResourceGroup \
--name myAKSCluster \
--command "helm install test-chart-override-name test-chart" \
--file .
The answer for this is the following:
az aks command invoke \
--resource-group aharo-aks-appgateway01 \
--name aharo-aks02 \
--command "helm install haro . "
Anoother workaround is uploading your helm charts to your container registry & then, you will have to download them and install them directly like the example from microsoft:
az aks command invoke \
--resource-group aharo-aks-appgateway01 \
--name aharo-aks02 \
--command "helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update && helm install my-release bitnami/nginx"
Kubernetes version : 1.22.6
previously deployed ingress controller through Terraform using chart https://kubernetes.github.io/ingress-nginx.
While starting the ingress controller we keep getting the below error message in job ingress-Nginx-admission-create and not starting the ingress controller
{
"err": "Get \"https://192.168.64.1:443/api/v1/namespaces/ingress-nginx/secrets/ingress-nginx-admission\": EOF",
"level": "fatal",
"msg": "error getting secret",
"source": "k8s/k8s.go:232",
"time": "2022-06-30T22:03:16Z"
}
I tried a couple of options to delete and re-create ingress-controller using the below command directly but no luck.
helm upgrade --install \
ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--debug \
--version="4.1.1" # tried with latest version too
Any help, please.
In order to resolve it, you should try the following:
Quote your values when you install it (TF vars. i assume)
Allow port 443 to the API server IP address/FQDN.
Reprovisioned your Service CIDR within the same network (as a subnet) as the nodes
Suggestion based upon your update
--set controller.hostNetwork=true
Add the above line to your command
helm upgrade \
ingress-nginx ingress-nginx \
--install \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx \
--create-namespace \
--debug \
--version="4.1.1" \
--set controller.hostNetwork=true
I deployed one ingress controller in a namespace by running the command:
helm install nginx-ingress ingress-nginx/ingress-nginx \
--namespace kk \
--set controller.replicaCount=2 \
--set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
--set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux
When I run the same command in another namespace I get the below error:
Error: Failed to download "ingress-nginx/ingress-nginx" (hint: running `helm repo update` may help)
I am pretty sure we can install multiple ingress controller in one Kubernetes cluster but I am unable to resolve this issue.
It's all about your internet connection. I've had the same problem.
Try using the specific version; that may help:
helm install ingress-nginx ingress-nginx/ingress-nginx --version 3.7.1 --debug
Unable to add spot node pool to Azure Kubernetes Cluster
AWS-CLI
azure-cli 2.12.0
core 2.12.0
telemetry 1.0.6
Extensions:
aks-preview 0.4.63
Following the instructions on Microsoft site: https://learn.microsoft.com/en-us/azure/aks/spot-node-pool
Ran the following
# Create a resource group in East US
az group create --name myResourceGroup --location westus2
# Create a basic single-node AKS cluster
az aks create \
--resource-group myResourceGroup \
--name myAKSCluster \
--vm-set-type VirtualMachineScaleSets \
--node-count 1 \
--generate-ssh-keys \
--load-balancer-sku standard
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
az aks nodepool add \
--resource-group myResourceGroup \
--cluster-name myAKSCluster \
--name spotnodepool \
--priority Spot \
--spot-max-price -1 \
--eviction-policy Delete \
--node-vm-size Standard_D2as_v4 \
--node-count 1
Got the following error:
Error code:
ValidationError: Operation failed with status: 'Bad Request'. Details: Provisioning of resource(s) for Agent Pool spotnodepool failed. Error: {
"code": "InvalidTemplateDeployment",
"message": "The template deployment failed with error: 'The resource with id: '/subscriptions/REDACTED/resourceGroups/MC_myResourceGroup_myAKSCluster_westus2/providers/Microsoft.Compute/virtualMachineScaleSets/aks-REDACTED-vmss' failed validation with message: 'The requested size for resource '/subscriptions/REDACTED/resourceGroups/MC_myResourceGroup_myAKSCluster_westus2/providers/Microsoft.Compute/virtualMachineScaleSets/aks-REDACTED-vmss' is currently not available in location 'westus2' zones '' for subscription 'REDACTED'. Please try another size or deploy to a different location or zones. See https://aka.ms/azureskunotavailable for details.'.'."
}
I have tried other AZ region, useast, uswest2, etc but with no luck
If I run the following it will work:
az aks nodepool add \
--resource-group myResourceGroup \
--cluster-name myAKSCluster \
--name mynodepool \
--node-count 1
Is spot node pool not supported? It is in the official docs: https://learn.microsoft.com/en-us/cli/azure/ext/aks-preview/aks/nodepool?view=azure-cli-latest
I just realised that this is not a AKS or Spot issue. This is a subscription issue.
In the docs (https://learn.microsoft.com/en-us/azure/virtual-machines/spot-vms#limitations) it says that only Pay-As-You-Go subscription are allowed to use spot instances. I just recently upgraded to Pay-As-You-Go model but my offerID is still stuck as "Free Trial" - I just need to be patient for it to change over
While installation of Velero on kubernetes using helm charts as below
helm install --namespace velero \
--set configuration.provider="Microsoft Azure" \
--set-file credentials.secretContents.cloud=<FULL PATH TO FILE> \
--set configuration.backupStorageLocation.name=azure \
--set configuration.backupStorageLocation.bucket=<BUCKET NAME> \
--set configuration.volumeSnapshotLocation.name=<PROVIDER NAME> \
--set configuration.volumeSnapshotLocation.config.region=<REGION> \
--set image.repository=velero/velero \
--set image.tag=v1.2.0 \
--set image.pullPolicy=IfNotPresent \
--set initContainers[0].name=velero-plugin-for-microsoft-azure:v1.0.0 \
--set initContainers[0].image=velero/velero-plugin-for-microsoft-azure:v1.0.0 \
--set initContainers[0].volumeMounts[0].mountPath=/target \
--set initContainers[0].volumeMounts[0].name=plugins \
stable/velero
I have configured the below environment variables in credential-velero file and path has been provided in above command.
credentials-velero file -
AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID}
AZURE_TENANT_ID=${AZURE_TENANT_ID}
AZURE_CLIENT_ID=${AZURE_CLIENT_ID}
AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}
AZURE_RESOURCE_GROUP=${AZURE_RESOURCE_GROUP}
AZURE_CLOUD_NAME=AzurePublicCloud
I am getting below error -
an error occurred: some backup storage locations are invalid: error getting backup store for location "default": rpc error: code = Unknown desc = unable to get all required environment variables: the following keys do not have values: AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_SUBSCRIPTION_ID
Could you please help with resolution of above error?
Your velero credential files should contain values for those, not placeholders.
cat << EOF > ./credentials-velero
AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID}
AZURE_TENANT_ID=${AZURE_TENANT_ID}
AZURE_CLIENT_ID=${AZURE_CLIENT_ID}
AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}
AZURE_RESOURCE_GROUP=${AZURE_RESOURCE_GROUP}
AZURE_CLOUD_NAME=AzurePublicCloud
EOF
https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure#setup
use this way
AZURE_SUBSCRIPTION_ID=XXXX-XXXXX-XXXXXXX-XXXXXXX
AZURE_TENANT_ID=XXXX-XXXXX-XXXXXXX-XXXXXXX
AZURE_CLIENT_ID=XXXX-XXXXX-XXXXXXX-XXXXXXX
AZURE_CLIENT_SECRET=XXXXXXXXXXXXXXXXXX
AZURE_RESOURCE_GROUP=MC_RESOURCE_GROUP_NAME_OF_AKS # this should be the MC resource group
AZURE_CLOUD_NAME=AzurePublicCloud
Also try using the master image
--set initContainers[0].image=velero/velero-plugin-for-microsoft-azure:master
Use this format
helm install velero vmware-tanzu/velero --namespace velero \
--set-file credentials.secretContents.cloud=./credentials-velero \
--set configuration.provider=azure \
--set configuration.backupStorageLocation.name=azure \
--set configuration.backupStorageLocation.bucket='velero' \
--set configuration.backupStorageLocation.config.resourceGroup=RESOURCE_GROUP_OF_STORAGEACCOUNT \
--set configuration.backupStorageLocation.config.storageAccount=STORAGE_ACCOUNT_NAME \
--set snapshotsEnabled=true \
--set deployRestic=true \
--set image.repository=velero/velero \
--set image.pullPolicy=Always \
--set initContainers[0].name=velero-plugin-for-microsoft-azure \
--set initContainers[0].image=velero/velero-plugin-for-microsoft-azure:master \
--set initContainers[0].volumeMounts[0].mountPath=/target \
--set initContainers[0].volumeMounts[0].name=plugins