AKS timeout container kubectl Rollout status check failed - azure

Question

I have a sporadic issue that I am struggling to understand - azure pipeline on promote fails due to kubectl rollout status Deployment/name --timeout 120s --namespace xyz
I have tried to increase the progressDeadlineSeconds, but I know it may not take, I have tried to update the replicaSets to 2 so it can take but it still does not apply. I am not fully understanding this error and there is a roll out issue.
Yaml file
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: #{KubeComponentName}#
namespace: #{Namespace}#
spec:
selector:
matchLabels:
app: #{KubeComponentName}#
progressDeadlineSeconds: 600
replicas: #{ReplicaCount}#
template:
metadata:
labels:
app: #{KubeComponentName}#
annotations:
spec:
securityContext:
runAsUser: 999
serviceAccountName: #{KubeComponentName}#
containers:
- name: #{KubeComponentName}#
image: #{ImageRegistry}#/datahub/#{KubeComponentName}#:latest
#command: ["/bin/bash", "-c", "--"]
#args: [ "while true; do sleep 30; done;" ]
volumeMounts:
ports:
env:
- name: NodeName
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: PodName
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: PodNamespace
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: PodIp
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: PodServiceAccount
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
- name: ComponentInfo__ComponentName
value: #{KubeComponentName}#
- name: ComponentInfo__ComponentHost
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: ComponentInfo__ServiceUser
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
- name: MongoDbUserName
valueFrom:
secretKeyRef:
name: mongodb-xyz-username
key: secret-value
- name: MongoDbPassword
valueFrom:
secretKeyRef:
name: mongodb-xyz-password
key: secret-value
- name: MongoDbKubernetesHosts
value: #{MongoDbKubernetesHosts}#
- name: MongoDbScriptBasePath
value: #{MongoDbScriptBasePath}#
volumes:
My errors keep happening such that I get a timeout error waiting for rollout or exceeded progress deadline
----
/opt/vsts-agent/_work/_tool/kubectl/1.18.6/x64/kubectl rollout status Deployment/datahub-recon --timeout 120s --namespace xyz
Waiting for deployment "datahub-recon" rollout to finish: 0 of 1 updated replicas are available...
error: timed out waiting for the condition
##[error]Error: error: timed out waiting for the condition
/opt/vsts-agent/_work/_tool/kubectl/1.18.6/x64/kubectl describe Deployment datahub-recon --namespace xyz
Conditions:
Type Status Reason
---- ------ ------
Available False MinimumReplicasUnavailable
Progressing True ReplicaSetUpdated
OldReplicaSets: <none>
NewReplicaSet: datahub-recon-567c7d6958 (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 2m1s deployment-controller Scaled up replica set datahub-recon-567c7d6958 to 1
For more information, go to https://dev.azure.com/pbc/Premera/_environments/23
##[error]Rollout status check failed.
----
/opt/vsts-agent/_work/_tool/kubectl/1.18.6/x64/kubectl rollout status Deployment/datahub-recon --timeout 120s --namespace xyz
error: deployment "datahub-recon" exceeded its progress deadline
##[error]Error: error: deployment "datahub-recon" exceeded its progress deadline
/opt/vsts-agent/_work/_tool/kubectl/1.18.6/x64/kubectl describe Deployment datahub-recon --namespace xyz
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing False ProgressDeadlineExceeded
OldReplicaSets: datahub-recon-6bc6f85fc6 (2/2 replicas created)
NewReplicaSet: datahub-recon-bd7d9754 (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 13m deployment-controller Scaled up replica set datahub-recon-bd7d9754 to 1
For more information, go to https://dev.azure.com/pbc/Premera/_environments/23
##[error]Rollout status check failed.