I'm setting up RBAC in my AKS cluster which is integrated with Azure AD following the instructions here. I have created an AD group in my AAD tenant, added a user to it. Then the group is assigned "Cluster User role" in the AKS cluster as per the instructions. Created a Role and Rolebinding as shown below:
Role:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: development
name: restricted-role
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
RoleBinding:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: development
subjects:
- kind: Group
name: 308f50cb-e05a-4340-99d4-xxxxxxxb
apiGroup: rbac.authorization.k8s.io
namespace: development
roleRef:
kind: Role
name: restricted-role
apiGroup: rbac.authorization.k8s.io
I then tried login using the new user credentials:
az login --username kubeuser#xxx.onmicrosoft.com --password xxxx
az aks get-credentials --name mycluster --resource-group myrg --overwrite-existing
As per the documentation, I should be only allowed to do kubectl get pods on the development namespace. However, using this new user credentials, I see that I can do kubectl get pods --all-namespaces, kubectl get svc --all-namespaces etc. and view the results, as if the Rolebinding does not have any impact at all. I also have verified by checking that my cluster has
"enableRBAC": true
Can someone please tell me what is wrong with this configuration?
Using the command:
az aks show -g <rg> -n <clusterName> --query aadProfile
you can confirm if the cluster is AAD enabled. If enabled, the kubeconfig file you get from:
az aks get-credentials -g <rg_name> -n <aks_name>
should look like:
user:
auth-provider:
config:
apiserver-id: <appserverid>
client-id: <clientid>
environment: AzurePublicCloud
tenant-id: <tenant>
name: azure
Related
By using the reference of https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-nginx-tls this document, I'm trying to fetch the TLS secrets from AKV to AKS pods.
Initially I created and configured CSI driver configuration with using User Assigned Managed Identity.
I have performed the following steps:
Create AKS Cluster with 1 nodepool.
Create AKV.
Created user assigned managed identity and assign it to the nodepool i.e. to the VMSS created for AKS.
Installed CSI Driver helm chart in AKS's "kube-system" namespace. and completed all the requirement to perform this operations.
Created the TLS certificate and key.
By using TLS certificate and key, created .pfx file.
Uploaded that .pfx file in the AKV certificates named as "ingresscert".
Created new namespace in AKS named as "ingress-test".
Deployed secretProviderClass in that namespace are as follows.:
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: azure-tls
spec:
provider: azure
secretObjects: # secretObjects defines the desired state of synced K8s secret objects
- secretName: ingress-tls-csi
type: kubernetes.io/tls
data:
- objectName: ingresscert
key: tls.key
- objectName: ingresscert
key: tls.crt
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true"
userAssignedIdentityID: "7*******-****-****-****-***********1"
keyvaultName: "*****-*****-kv" # the name of the AKV instance
objects: |
array:
- |
objectName: ingresscert
objectType: secret
tenantId: "e*******-****-****-****-***********f" # the tenant ID of the AKV instance
Deployed the nginx-ingress-controller helm chart in the same namespace, where certificates are binded with application.
Deployed the Busy Box deployment are as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-one
labels:
app: busybox-one
spec:
replicas: 1
selector:
matchLabels:
app: busybox-one
template:
metadata:
labels:
app: busybox-one
spec:
containers:
- name: busybox
image: k8s.gcr.io/e2e-test-images/busybox:1.29-1
command:
- "/bin/sleep"
- "10000"
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store"
readOnly: true
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-tls"
---
apiVersion: v1
kind: Service
metadata:
name: busybox-one
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: busybox-one
Check secret is created or not by using command
kubectl get secret -n <namespaceName>
One thing to notice here is, if I attach shell with the busy box pod and go to the mount path which I provided to mount secrets I have seen that secrets are successfully fetched there. But this secrets are not showing in the AKS's secret list.
I have troubleshooted all the AKS,KV and manifest files but not found anything.
IF there is anything I have missed or anyone has solution for this please let me know.
Thanks in advance..!!!
i added this as a new answer, bcs the formatting was bad in the comments:
As you are using the Helm chart, you have to activate the secret sync in the values.yaml of the Helm Chart:
secrets-store-csi-driver:
syncSecret:
enabled: true
I would still recommend to use the csi-secrets-store-provider-azure as AKS Addon instead of the Helm-Chart
Your config looks good to me. One thing to consider is, that the User Assigned Managed Identity should not be the one you created for the AKS, it should be the managed identity from your nodepool (kubelet) and it also needs permission on the AKV.
I had the same issues while using the wrong Managed identity.
userAssignedIdentityID = Kubelet Client Id ( Nodepool Managed Idendity )
AZ CLI
export KUBE_ID=$(az aks show -g <resource group> -n <aks cluster name> --query identityProfile.kubeletidentity.objectId -o tsv)
export AKV_ID=$(az keyvault show -g <resource group> -n <akv name> --query id -o tsv)
az role assignment create --assignee $KUBE_ID --role "Key Vault Secrets Officer" --scope $AKV_ID
KEDA scaler not scales with scaled object defined with trigger using pod identity for authentication for service bus queue.
I'm following this KEDA service bus triggered scaling project.
The scaling works fine with the connection string, but when I try to scale using the pod identity for KEDA scaler the keda operator fails to get the azure identity bound to it with the following keda operator error message log:
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).isScaledObjectActive
/workspace/pkg/scaling/scale_handler.go:228
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
/workspace/pkg/scaling/scale_handler.go:211
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
/workspace/pkg/scaling/scale_handler.go:145
2021-10-10T17:35:53.916Z ERROR azure_servicebus_scaler error {"error": "failed to refresh token, error: adal: Refresh request failed. Status Code = '400'. Response body: {\"error\":\"invalid_request\",\"error_description\":\"Identity not found\"}\n"}
Edited on 11/09/2021
I opened a github issue at keda, and we did some troubleshoot. But it seems like an issue with AAD Pod Identity as #Tom suggests. The AD Pod Identity MIC pod gives logs like this:
E1109 03:15:34.391759 1 mic.go:1111] failed to update user-assigned identities on node aks-agentpool-14229154-vmss (add [2], del [0], update[0]), error: failed to update identities for aks-agentpool-14229154-vmss in MC_Arun_democluster_westeurope, error: compute.VirtualMachineScaleSetsClient#Update: Failure sending request: StatusCode=0 -- Original Error: Code="LinkedAuthorizationFailed" Message="The client 'fe0d7679-8477-48e3-ae7d-43e2a6fdb957' with object id 'fe0d7679-8477-48e3-ae7d-43e2a6fdb957' has permission to perform action 'Microsoft.Compute/virtualMachineScaleSets/write' on scope '/subscriptions/f3786c6b-8dca-417d-af3f-23929e8b4129/resourceGroups/MC_Arun_democluster_westeurope/providers/Microsoft.Compute/virtualMachineScaleSets/aks-agentpool-14229154-vmss'; however, it does not have permission to perform action 'Microsoft.ManagedIdentity/userAssignedIdentities/assign/action' on the linked scope(s) '/subscriptions/f3786c6b-8dca-417d-af3f-23929e8b4129/resourcegroups/arun/providers/microsoft.managedidentity/userassignedidentities/autoscaler-id' or the linked scope(s) are invalid."
Any clues how to fix it?
My scaler objects' definition is as below:
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: trigger-auth-service-bus-orders
spec:
podIdentity:
provider: azure
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: order-scaler
spec:
scaleTargetRef:
name: order-processor
# minReplicaCount: 0 Change to define how many minimum replicas you want
maxReplicaCount: 10
triggers:
- type: azure-servicebus
metadata:
namespace: demodemobus
queueName: orders
messageCount: '5'
authenticationRef:
name: trigger-auth-service-bus-orders
Im deploying the azure identity to the namespace keda where my keda deployment resides.
And installs KEDA with the following command to set the pod identity binding using helm:
helm install keda kedacore/keda --set podIdentity.activeDirectory.identity=app-autoscaler --namespace keda
Expected Behavior
The KEDA scaler should have worked fine with the assigned pod identity and access token to perform scaling
Actual Behavior
The KEDA operator could not be able to find the azure identity assigned and scaling fails
Scaler Used
Azure Service Bus
Steps to Reproduce the Problem
Create the azure identity and bindings for the KEDA
Install KEDA with the aadpodidentitybinding
Create the scaledobject and triggerauthentication using KEDA pod identity
The scaler fails to authenticate and scale
Unfortunately this looks like an issue with the identity itself and with AD Pod identities, they can be a bit flaky (based on my experiences)
First and foremost, I am using AKS with kubenet plugin.
By default
'AAD Pod Identity is disabled by default on Clusters with Kubenet starting from release v1.7.'
This is because of the Kubenet is vulnerable to ARP Spoofing.
Please read it here.
Even then you can have a workaround to enable the KEDA scaling in Kubenet powered AKS.(The script holds good for other CNI's also, except that you dont need to edit anything with the aad-pod-identity component nmi daemonset definition yaml, if it runs well with your cluster plugins.).
Below I'm adding an e2e script for the same.
Please visit the github issue for access to all the discussions.
# Define aks name and resource group
$aksResourceGroup = "K8sScalingDemo"
$aksName = "K8sScalingDemo"
# Create resource group
az group create -n $aksResourceGroup -l centralindia
# Create the aks cluster with default kubenet plugin
az aks create -n $aksName -g $aksResourceGroup
# Resourcegroup where the aks resources will be deployed
$resourceGroup = "$(az aks show -g $aksResourceGroup -n $aksName --query nodeResourceGroup -otsv)"
# Set the kubectl context to the newly created aks cluster
az aks get-credentials -n $aksName -g $aksResourceGroup
# Install AAD Pod Identity into the aad-pod-identity namespace using helm
kubectl create namespace aad-pod-identity
helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts
helm install aad-pod-identity aad-pod-identity/aad-pod-identity --namespace aad-pod-identity
# Check the status of installation
kubectl --namespace=aad-pod-identity get pods -l "app.kubernetes.io/component=mic"
kubectl --namespace=aad-pod-identity get pods -l "app.kubernetes.io/component=nmi"
# the nmi components will Crashloop, ignore them for now. We will make them right later
# Get Resourcegroup Id of our $ResourceGroup
$resourceGroup_ResourceId = az group show --name $resourceGroup --query id -otsv
# Get the aks cluster kubeletidentity client id
$aad_pod_identity_clientid = az aks show -g $aksResourceGroup -n $aksName --query identityProfile.kubeletidentity.clientId -otsv
# Assign required roles for cluster over the resourcegroup
az role assignment create --role "Managed Identity Operator" --assignee $aad_pod_identity_clientid --scope $resourceGroup_ResourceId
az role assignment create --role "Virtual Machine Contributor" --assignee $aad_pod_identity_clientid --scope $resourceGroup_ResourceId
# Create autoscaler azure identity and get client id and resource id of the autoscaler identity
$autoScaleridentityName = "autoscaler-aad-identity"
az identity create --name $autoScaleridentityName --resource-group $resourceGroup
$autoscaler_aad_identity_clientId = az identity show --name $autoScaleridentityName --resource-group $resourceGroup --query clientId -otsv
$autoscaler_aad_identity_resourceId = az identity show --name $autoScaleridentityName --resource-group $resourceGroup --query id -otsv
# Create the app azure identity and get client id and resource id of the app identity
$appIdentityName = "app-aad-identity"
az identity create --name app-aad-identity --resource-group $resourceGroup
$app_aad_identity_clientId = az identity show --name $appIdentityName --resource-group $resourceGroup --query clientId -otsv
$app_aad_identity_resourceId = az identity show --name $appIdentityName --resource-group $resourceGroup --query id -otsv
# Create service bus and queue
$servicebus = 'svcbusdemo'
az servicebus namespace create --name $servicebus --resource-group $resourceGroup --sku basic
$servicebus_namespace_resourceId = az servicebus namespace show --name $servicebus --resource-group $resourceGroup --query id -otsv
az servicebus queue create --namespace-name $servicebus --name orders --resource-group $resourceGroup
$servicebus_queue_resourceId = az servicebus queue show --namespace-name $servicebus --name orders --resource-group $resourceGroup --query id -otsv
# Assign Service Bus Data Receiver role to the app identity created
az role assignment create --role 'Azure Service Bus Data Receiver' --assignee $app_aad_identity_clientId --scope $servicebus_queue_resourceId
# Create a namespace for order app deployment
kubectl create namespace keda-dotnet-sample
# Create a yaml deployment configuration variable
$app_with_identity_yaml= #"
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentity
metadata:
name: $appIdentityName
annotations:
aadpodidentity.k8s.io/Behavior: namespaced
spec:
type: 0 # 0 means User-assigned MSI
resourceID: $app_aad_identity_resourceId
clientID: $app_aad_identity_clientId
---
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentityBinding
metadata:
name: $appIdentityName-binding
spec:
azureIdentity: $appIdentityName
selector: order-processor
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-processor
labels:
app: order-processor
spec:
selector:
matchLabels:
app: order-processor
template:
metadata:
labels:
app: order-processor
aadpodidbinding: order-processor
spec:
containers:
- name: order-processor
image: ghcr.io/kedacore/sample-dotnet-worker-servicebus-queue:latest
env:
- name: KEDA_SERVICEBUS_AUTH_MODE
value: ManagedIdentity
- name: KEDA_SERVICEBUS_HOST_NAME
value: $servicebus.servicebus.windows.net
- name: KEDA_SERVICEBUS_QUEUE_NAME
value: orders
- name: KEDA_SERVICEBUS_IDENTITY_USERASSIGNEDID
value: $app_aad_identity_clientId
"#
# Create the app deployment with identity bindings using kubectl apply
$app_with_identity_yaml | kubectl apply --namespace keda-dotnet-sample -f -
# Now the order processor app works with the pod identity and
# processes the queues
# You can refer the [project ](https://github.com/kedacore/sample-dotnet-worker-servicebus-queue/blob/main/pod-identity.md) for that.
# Now start installation of KEDA in namespace keda-system
kubectl create namespace keda-system
# Create a pod identity and binding for autoscaler azure identity
$autoscaler_yaml =#"
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentity
metadata:
name: $autoScaleridentityName
spec:
type: 0 # 0 means User-assigned MSI
resourceID: $autoscaler_aad_identity_resourceId
clientID: $autoscaler_aad_identity_clientId
---
apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentityBinding
metadata:
name: $autoScaleridentityName-binding
spec:
azureIdentity: $autoScaleridentityName
selector: $autoScaleridentityName
"#
$autoscaler_yaml | kubectl apply --namespace keda-system -f -
# Install KEDA using helm
helm install keda kedacore/keda --set podIdentity.activeDirectory.identity=autoscaler-aad-identity --namespace keda-system
# Assign Service Bus Data Owner role to keda autoscaler identity
az role assignment create --role 'Azure Service Bus Data Owner' --assignee $autoscaler_aad_identity_clientId --scope $servicebus_namespace_resourceId
# Apply scaled object definition and trigger authentication provider as `azure`
$aap_autoscaling_yaml = #"
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: trigger-auth-service-bus-orders
spec:
podIdentity:
provider: azure
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: order-scaler
spec:
scaleTargetRef:
name: order-processor
# minReplicaCount: 0 Change to define how many minimum replicas you want
maxReplicaCount: 10
triggers:
- type: azure-servicebus
metadata:
namespace: $servicebus
queueName: orders
messageCount: '5'
authenticationRef:
name: trigger-auth-service-bus-orders
"#
$aap_autoscaling_yaml | kubectl apply --namespace keda-dotnet-sample -f -
# Now the Keda is getting 401 unauthorized error as the AAD Pod Identity comnponent `nmi` is not runnig on the system
# To fix it edit the daemonset for `nmi` component
# add the container arg `--allow-network-plugin-kubenet=true` by editing the `daemonset.apps/aad-pod-identity-nmi`
kubectl edit daemonset.apps/aad-pod-identity-nmi -n aad-pod-identity
# the containe arg section should look like this after editing:
spec:
containers:
- args:
- --node=$(NODE_NAME)
- --http-probe-port=8085
- --enableScaleFeatures=true
- --metadata-header-required=true
- --operation-mode=standard
- --kubelet-config=/etc/default/kubelet
- --allow-network-plugin-kubenet=true
env:
# Now the KEDA is authenticated by aad-pod-identity metadata endpoint and the orderapp should scale up
# with the queue counts
# If the order app still falls back to errors please delete and redeploy it.
# And that's it you just scaled your app up using KEDA on Kubenet AKS cluster.
Note: Read this instruction before you run AAD Identity On a Kubenet powered AKS.
I created a deployment at AKS:
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: frontend
strategy:
type: Recreate
template:
metadata:
labels:
io.kompose.service: frontend
spec:
containers:
- image: app:latest
name: frontend
volumeMounts:
- mountPath: /app/db
name: db
- mountPath: /root/.aspnet/https
name: https
readOnly: true
env:
- name: ASPNETCORE_URLS
value: "https://+;http://+"
- name: ASPNETCORE_HTTPS_PORT
value: "443"
- name: ASPNETCORE_Kestrel__Certificates__Default__Path
value: "/root/.aspnet/https/cert.pfx"
- name: ASPNETCORE_Kestrel__Certificates__Default__Password
valueFrom:
secretKeyRef:
name: certificate-pass
key: pass
restartPolicy: Always
serviceAccountName: ""
volumes:
- name: db
persistentVolumeClaim:
claimName: db
- name: https
secret:
secretName: certificate
items:
- key: file
path: cert.pfx
and a service:
apiVersion: v1
kind: Service
metadata:
name: frontend-service
spec:
selector:
io.kompose.service: frontend
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
- name: https
protocol: TCP
port: 443
targetPort: 443
type: LoadBalancer
Service is created successfully. I can access it using provided External IP:
Now I want to make this IP static. There is an official docs which tells how to make it: Use a static public IP address and DNS label with the Azure Kubernetes Service (AKS) load balancer
There is also an article which technically duplicates the docs, but in a bit more details: Use a static public IP address outside of the node resource group with the Azure Kubernetes Service (AKS) load balancer
I am able to create an IP address, but when I reach az role assignment create command it fails ($GROUP here is just a placeholder for real Resource group literal):
$ CLIENT_ID=$(az aks show --resource-group Default --name k8s --query "servicePrinci
palProfile.clientId" --output tsv)
$ SUB_ID=$(az account show --query "id" --output tsv)
$ az role assignment create --assignee $CLIENT_ID --role "Network Contributor" --scope /subscriptions/$SUB_ID/resourceGroups/$GROUP
If the assignee is an appId, make sure the corresponding service principal is created with 'az ad sp create --id $CLIENT_ID
When I try proposed command if fails:
$ az ad sp create --id $CLIENT_ID
Another object with the same value for property servicePrincipalNames already exists.
I found similar issue at GitHub Azure/azure-cli repo, but there is no answer.
If I skip this step and set loadBalancerIP: XXX.XXX.XXX.XXX at config level, load balancer has a new property Load balancer IP, but External IP is not changed:
How to mitigate this error?
According to the messages you provide, I'm very confused about why you need to run the command az ad sp create --id $CLIENT_ID, both the docs do not show the necessity to run this command.
As I know, you only need to assign the "Network Contributor" role of the group other than the AKS node group to the service principal of the AKS. That's the right step you need to do. I think you need to read the docs again with more attention.
I had to use literal value instead of a variable $CLIENT_ID. That helped.
I have followed this documentation on providing ACR access to AKS : https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auth-aks but still i'm getting the authentication failed error. Can someone help me with this?
env:
- name: no_cpu
valueFrom:
resourceFieldRef:
containerName: proxy
resource: requests.cpu
imagePullSecrets:
- name: acr-auth
Make sure you have the acrpull role assigned to the AKS SP on the ACR resource (from the portal, on the IAM tab, or using the script:
#!/bin/bash
AKS_RESOURCE_GROUP=myAKSResourceGroup
AKS_CLUSTER_NAME=myAKSCluster
ACR_RESOURCE_GROUP=myACRResourceGroup
ACR_NAME=myACRRegistry
# Get the id of the service principal configured for AKS
CLIENT_ID=$(az aks show --resource-group $AKS_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query "servicePrincipalProfile.clientId" --output tsv)
# Get the ACR registry resource id
ACR_ID=$(az acr show --name $ACR_NAME --resource-group $ACR_RESOURCE_GROUP --query "id" --output tsv)
# Create role assignment
az role assignment create --assignee $CLIENT_ID --role acrpull --scope $ACR_ID
).
Then create pull secret via command line:
kubectl create secret docker-registry acr-auth --docker-server <acr-login-server> --docker-username <service-principal-ID> --docker-password <service-principal-password> --docker-email <email-address>
or
apiVersion: v1
kind: Secret
metadata:
name: acr-auth
type: docker-registry
data:
username: <base64encoded username>
password: <base64encoded password>
Both are equivalent.
you need to remove imagepullsecrets property from the pod\deployment definition. that way you will instruct kubernetes to use internal aks\acr auth
I'm trying to deploy a Kubernetes Pod in AKS (I'm new to Kubernetes, so at this stage, I just want to create a container, deploy to Kubernetes and connect to it).
My Yaml file is as follows:
apiVersion: v1
kind: Pod
spec:
containers:
- name: dockertest20190205080020
image: dockertest20190205080020.azurecr.io
ports:
- containerPort: 443
metadata:
name: my-test
I've created the image in Azure Container Registry and, according to the CLI, successfully deployed it to Kubernetes.
After deploying, I used the following command:
kubectl get service
And it tells me there is no External IP to connect to. I then tried:
kubectl describe pod my-test
Which gave the following errors:
Events:
Warning Failed 4m (x2221 over 8h) kubelet, aks-nodepool1-27401563-2 Error: ImagePullBackOff
Normal BackOff 0s (x2242 over 8h) kubelet, aks-nodepool1-27401563-2 Back-off pulling image "dockertest20190205080020.azurecr.io"
I then tried editing the deployment:
kubectl edit pods my-test
Which game me the error:
message: 'containers with unready status: [dockertest20190205080020]'
I'm not a little unsure what my next diagnostic step would be. I get the impression there's an issue with the container or the container registry, but I'm unsure how to determine what that may be.
What happens here (most likely) - your AKS doesnt have permissions to pull images frmo you ACR (that's the default behaviour). You need to grant those (link):
#!/bin/bash
AKS_RESOURCE_GROUP=myAKSResourceGroup
AKS_CLUSTER_NAME=myAKSCluster
ACR_RESOURCE_GROUP=myACRResourceGroup
ACR_NAME=myACRRegistry
# Get the id of the service principal configured for AKS
CLIENT_ID=$(az aks show --resource-group $AKS_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query "servicePrincipalProfile.clientId" --output tsv)
# Get the ACR registry resource id
ACR_ID=$(az acr show --name $ACR_NAME --resource-group $ACR_RESOURCE_GROUP --query "id" --output tsv)
# Create role assignment
az role assignment create --assignee $CLIENT_ID --role acrpull --scope $ACR_ID
Alternative is to just use a docker login secret (that article mentions that as well).
Example image in ACR:
image name would be
clrtacr.azurecr.io/dns:tag (or without tag for latest)
I'm not sure if you know there is something wrong in your yaml file, or it just shows as you want for security. But I would show you here:
apiVersion: v1
kind: Pod
spec:
containers:
- name: dockertest20190205080020
image: dockertest20190205080020.azurecr.io/image_name_and_version
ports:
- containerPort: 443
metadata:
name: my-test
Also, as the error that you got shows, you don't have the permission to pull the image from your ACR.
On my side, I would be better to use a secret for pulling all the image from the ACR. You can create a service principal to achieve it. The steps would be like here:
#!/bin/bash
ACR_NAME=myacrinstance
SERVICE_PRINCIPAL_NAME=acr-service-principal
# Populate the ACR login server and resource id.
ACR_LOGIN_SERVER=$(az acr show --name $ACR_NAME --query loginServer --output tsv)
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)
# Create acrpull role assignment with a scope of the ACR resource.
SP_PASSWD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role acrpull --scopes $ACR_REGISTRY_ID --query password --output tsv)
# Get the service principal client id.
CLIENT_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)
# Output used when creating Kubernetes secret.
echo "Service principal ID: $CLIENT_ID"
echo "Service principal password: $SP_PASSWD"
# Create the secret
kubectl create secret docker-registry acr-auth --docker-server <acr-login-server> --docker-username <service-principal-ID> --docker-password <service-principal-password>
Then you can change your yaml file like this:
apiVersion: v1
kind: Pod
spec:
containers:
- name: dockertest20190205080020
image: dockertest20190205080020.azurecr.io/image_name_and_version
ports:
- containerPort: 443
imagePullSecrets:
- name: acr-auth
metadata:
name: my-test