AKS. Can't pull image from an acr - azure

Question

I try to pull image from an ACR using a secret and I can't do it.
I created resources using azure cli commands:
az login
az provider register -n Microsoft.Network
az provider register -n Microsoft.Storage
az provider register -n Microsoft.Compute
az provider register -n Microsoft.ContainerService
az group create --name aksGroup --location westeurope
az aks create --resource-group aksGroup --name aksCluster --node-count 1 --generate-ssh-keys -k 1.9.2
az aks get-credentials --resource-group aksGroup --name aksCluster
az acr create --resource-group aksGroup --name aksClusterRegistry --sku Basic --admin-enabled true
After that I logged in and pushed image successfully to created ACR from local machine.
docker login aksclusterregistry.azurecr.io
docker tag jetty aksclusterregistry.azurecr.io/jetty
docker push aksclusterregistry.azurecr.io/jetty
The next step was creating a secret:
kubectl create secret docker-registry secret --docker-server=aksclusterregistry.azurecr.io --docker-username=aksClusterRegistry --docker-password=<Password from tab ACR/Access Keys> --docker-email=some#email.com
And eventually I tried to create pod with image from the ACR:
#pod.yml
apiVersion: v1
kind: Pod
metadata:
name: jetty
spec:
containers:
- name: jetty
image: aksclusterregistry.azurecr.io/jetty
imagePullSecrets:
- name: secret
kubectl create -f pod.yml
In result I have a pod with status ImagePullBackOff:
>kubectl get pods
NAME READY STATUS RESTARTS AGE
jetty 0/1 ImagePullBackOff 0 1m
> kubectl describe pod jetty
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m default-scheduler Successfully assigned jetty to aks-nodepool1-62963605-0
Normal SuccessfulMountVolume 2m kubelet, aks-nodepool1-62963605-0 MountVolume.SetUp succeeded for volume "default-token-w8png"
Normal Pulling 2m (x2 over 2m) kubelet, aks-nodepool1-62963605-0 pulling image "aksclusterregistry.azurecr.io/jetty"
Warning Failed 2m (x2 over 2m) kubelet, aks-nodepool1-62963605-0 Failed to pull image "aksclusterregistry.azurecr.io/jetty": rpc error: code = Unknown desc = Error response from daemon: Get https://aksclusterregistry.azurecr.io/v2/jetty/manifests/latest: unauthorized: authentication required
Warning Failed 2m (x2 over 2m) kubelet, aks-nodepool1-62963605-0 Error: ErrImagePull
Normal BackOff 2m (x5 over 2m) kubelet, aks-nodepool1-62963605-0 Back-off pulling image "aksclusterregistry.azurecr.io/jetty"
Normal SandboxChanged 2m (x7 over 2m) kubelet, aks-nodepool1-62963605-0 Pod sandbox changed, it will be killed and re-created.
Warning Failed 2m (x6 over 2m) kubelet, aks-nodepool1-62963605-0 Error: ImagePullBackOff
What's wrong? Why does approach with secret not work?
Please don't advice me approach with service principal, because I would like to understand why this aproach doesn't work. I think it must be working.

Answer 1

The "old" way with AKS was to do create secret as you mentioned. That is no longer recommended.
The "new" way is to attach the container registry. This article explains the "new" way to attach ACR, and also provides a link to the old way to clear up confusion. When you create your cluster, attach with:
az aks create -n myAKSCluster -g myResourceGroup --attach-acr $MYACR
Or if you've already created your cluster, update it with:
az aks update -n myAKSCluster -g myResourceGroup --attach-acr $MYACR
Notes:
$MYACR is just the name of your registry without the .azurecr.io. Ex: MYACR=foobar not MYACR=foobar.azurecr.io.
After you attach your ACR, it will take a few minutes for the ImagePullBackOff to transition to Running.

Answer 2

This looks good to me as well. That said, the recommendation is not to use the admin account, rather a service principle. With the SP you gain some granular control over access rights to the ACR instance (read, contributor, owner).
This doc includes two methods for authentication between AKS and ACR using service principles.
https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auth-aks

Answer 3

It's not exactly the question case. But I was having similar issue with utilization of Attach ACR approach. My problem was with Upper case characters in the registry name. Below warning was being generated by az cli.
Uppercase characters are detected in the registry name. When using its server url in docker commands, to avoid authentication errors, use all lowercase
So ensure to use all lowercases in ACR urls on Docker commands.