Azure Devops kubectl task deployed image is with status ErrImagePull/ImagePullBackOff - azure

I have created a deployment pipeline using Azure Devops through which I am building and pushing an image to Azure Container Registry.
In the next step I am deploying from ACR to an AKS cluster which I have created and attached to ACR using the following command:
az aks create -n HealthCareAKSCluster -g AKSCICDRG --generate-ssh-keys --attach-acr HealthCareAKSCICDACR
Following is my yaml code:
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- master
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: 'XXXXXXXXXX'
imageRepository: 'patientservice'
containerRegistry: 'healthcareakscicdacr.azurecr.io'
dockerfilePath: './PatientService/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- stage: Deploy
displayName: Deploy
jobs:
- job: Deploy
displayName: Deploy
pool:
vmImage: $(vmImageName)
steps:
- task: Kubernetes#1
displayName: Deploy an image to AKS
inputs:
connectionType: 'Azure Resource Manager'
azureSubscriptionEndpoint: 'XXXXXXX'
azureResourceGroup: 'AKSCICDRG'
kubernetesCluster: 'HealthCareAKSCluster'
command: 'apply'
arguments: '-f patientservice.yaml'
secretType: 'dockerRegistry'
containerRegistryType: 'Azure Container Registry'
The image is getting pushed to ACR successfully and Build step is running fine.
Even deploy step is running fine but when I do 'kubectl get pods' in my AKS cluster I am getting the pod status as ImagePullBackOff or ErrImagePull.
When I did 'kubectl describe pod' I am getting the following message:
Failed to pull image "patientservice": rpc error: code = Unknown desc
= Error response from daemon: pull access denied for patientservice, repository does not exist or may require 'docker login': denied:
requested access to the resource is denied
Please help as to how can I deploy a proper image.

The issue was in patientservice.yaml file. I overlooked it. Previously image had only the name of the service.I added the following to the image in patientservice.yaml:
image: healthcareakscicdacr.azurecr.io/patientservice:latest
The deployment worked fine.

Related

Build and push image to ACR - ##[error]base name (${BUILD_IMAGE}) should not be blank

I am trying to build and push an image to Azure Container Registry using Azure DevOps pipeline, but I am getting the following error:
"base name (${BUILD_IMAGE}) should not be blank"
when trying to build the image.
I have checked my pipeline configuration file and I couldn't find the variable BUILD_IMAGE defined. From the error message, it seems that the variable is being used in the FROM statement in the Dockerfile but I don't know where it's supposed to be defined.
I have checked the pipeline configuration file, the script that triggers the pipeline, and the command-line but I couldn't find where the variable is defined.
I have also checked if the image aquariumcms with tag $(Build.BuildId) or 510914 exists in the registry rgaquariumtestimagewesteu.azurecr.io and if it's accessible from the pipeline but I couldn't find any issues.
My pipeline is using the Docker#2 and DockerCompose#0 tasks, and the container registry is rgaquariumtestimagewesteu.azurecr.io.
How can I add the variable BUILD_IMAGE so it's no longer blank and the pipeline can build and push the image to the container registry?
The pipeline
# Docker
# Build and push an image to Azure Container Registry
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- Initial-K8-Setup
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: '7739b21f-2572-4f36-ab8b-df15ab821756'
imageRepository: 'aquariumcms'
containerRegistry: 'rgAquariumTestImageWestEU.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- task: DockerCompose#0
inputs:
containerregistrytype: 'Azure Container Registry'
azureSubscription: 'HAWeb2-CFS-NonProd'
azureContainerRegistry: '{"loginServer":"rgAquariumTestImageWestEU.azurecr.io", "id" : "/subscriptions/5e2acfcb-7aeb-443d-9ea4-bc7553a51276/resourceGroups/rg-haweb-cfs-nonprod-web2-001/providers/Microsoft.ContainerRegistry/registries/rgAquariumTestImageWestEU"}'
dockerComposeFile: '**/docker-compose.yml'
additionalDockerComposeFiles: 'docker-compose.override.yml'
dockerComposeFileArgs: 'REGISTRY=rgAquariumTestImageWestEU.azurecr.io/'
action: 'Build services'
additionalImageTags: '$(Build.BuildNumber)-$(Build.SourceBranchName)'
includeLatestTag: true
- task: DockerCompose#0
inputs:
containerregistrytype: 'Azure Container Registry'
azureSubscription: 'HAWeb2-CFS-NonProd'
azureContainerRegistry: '{"loginServer":"rgAquariumTestImageWestEU.azurecr.io", "id" : "/subscriptions/5e2acfcb-7aeb-443d-9ea4-bc7553a51276/resourceGroups rg-haweb-cfs-nonprod-web2-001/providers/Microsoft.ContainerRegistry/registries/rgAquariumTestImageWestEU"}'
dockerComposeFile: '**/docker-compose.yml'
additionalDockerComposeFiles: 'docker-compose.override.yml'
dockerComposeFileArgs: 'REGISTRY=rgAquariumTestImageWestEU.azurecr.io/'
action: 'Push services'
additionalImageTags: '$(Build.BuildNumber)-$(Build.SourceBranchName)'
includeLatestTag: true
The expectation was that the pipeline would build and push the image to the Azure Container Registry specified in the pipeline configuration file. However, the pipeline is failing with the error message "base name (${BUILD_IMAGE}) should not be blank" when trying to build the image, indicating that the BUILD_IMAGE variable is empty or not defined.
Although I receive the following error log
Starting: Build and push an image to container registry
==============================================================================
Task : Docker
Description : Build or push Docker images, login or logout, start or stop containers, or run a Docker command
Version : 2.214.0
Author : Microsoft Corporation
Help : https://aka.ms/azpipes-docker-tsg
==============================================================================
/usr/bin/docker build -f /home/vsts/work/1/s/Dockerfile --label com.azure.dev.image.system.teamfoundationcollectionuri=https://dev.azure.com/dgsit/ --label com.azure.dev.image.system.teamproject=Sitecore --label com.azure.dev.image.build.repository.name=aquarium-cms --label com.azure.dev.image.build.sourceversion=dad7e8024ac8bba9b7b58bce934cdc96729da5cb --label com.azure.dev.image.build.repository.uri=https://dgsit#dev.azure.com/dgsit/Sitecore/_git/aquarium-cms --label com.azure.dev.image.build.sourcebranchname=Initial-K8-Setup --label com.azure.dev.image.build.definitionname=aquarium-cms (1) --label com.azure.dev.image.build.buildnumber=20230118.7 --label com.azure.dev.image.build.builduri=vstfs:///Build/Build/511111 -t ***/aquariumcms:511111 /home/vsts/work/1/s
Sending build context to Docker daemon 15.5MB
Step 1/29 : ARG BASE_IMAGE
Step 2/29 : ARG BUILD_IMAGE
Step 3/29 : FROM ${BUILD_IMAGE} AS prep
base name (${BUILD_IMAGE}) should not be blank
##[error]base name (${BUILD_IMAGE}) should not be blank
##[error]The process '/usr/bin/docker' failed with exit code 1
Finishing: Build and push an image to container registry
this is working for me, it overwrites the "latest" version in the repo as well as creates a new independent version based on the BuildID.
Under build added below tags.
tags: |
latest
$(Build.BuildId)
Here is my sample azure-pipeline.yml
# Docker
trigger:
- main
resources:
- repo: self
variables:
dockerRegistryServiceConnection: '...'
imageRepository: 'mycontainer'
containerRegistry: 'azuk.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/dockerfile'
tag: '$(Build.BuildId)'
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: 'Build and Push $(container.registry)'
inputs:
containerRegistry: '$(container.registry)'
repository: '$(container.repository)'
Dockerfile: '**/Dockerfile'
command: 'buildAndPush'
tags: |
latest
$(Build.BuildId)
refer to this github issue [8378](Docker push no longer pushes *:latest tag with includeLatestTag: true · Issue #8378 · microsoft/azure-pipelines-tasks (github.com)) for more information regardign similar issue

Azure devops Pipelines: Cannot find Docker Image when Pushing to Azure Registry, 'tag ***/appname'

We build our docker image with a custom .sh file. Then we would like to push it to our azure registry. We have the following step in our azure pipeline:
- task: Docker#2
inputs:
containerRegistry: 'someazureregistry'
repository: 'appname'
command: 'push'
tags: 'latest'
The build is successfull, and as can be seen below it is registered in docker as appname:latest , however for some reason azure does not find it, and asks for an image with the tag ***/appname.
/usr/bin/docker images
/usr/bin/docker push ***/appname:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
appname latest f1e11b4fc19c Less than a second ago 680MB
ubuntu 20.04 a0ce5a295b63 7 days ago 72.8MB
...
The push refers to repository [***/appname]
An image does not exist locally with the tag: ***/appname
##[error]An image does not exist locally with the tag: ***/appname
We have tried different combinations of tags. What is azure expecting here?
It looks like you are building the Docker image separately. Please note that the container image needs to be tag locally for the private container registry before it can be pushed. As Shayki Abramczyk mentioned you need to build the image with containerRegistry.
To resolve this issue you must Tag the image using the "docker tag" command with the specific private container registry.
For example:
docker tag image youoregistryhost:port/xxx/image-name:latest
Besides, you can build and push the Docker image together in Azure DevOps pipeline. The following YAML file for your reference:
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: 'testacr'
imageRepository: 'appname'
containerRegistry: 'xxx.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/app/Dockerfile'
tag: 'latest'
system.debug : true
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
Of course you can also build and push the image separately in Azure DevOps pipeline like this:
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build an image for container registry
inputs:
containerRegistry: '$(dockerRegistryServiceConnection)'
repository: '$(imageRepository)'
command: 'build'
Dockerfile: '$(dockerfilePath)'
tags: 'latest'
- task: Docker#2
displayName: Push an image to container registry
inputs:
containerRegistry: '$(dockerRegistryServiceConnection)'
repository: '$(imageRepository)'
command: 'push'
tags: 'latest'

Unable to find the pushed image in the azure container registry

I have pushed a new image in the azure container registry
But i am unable to find the image in azure container registry
Please find the deplyment yaml file
name: Dotnet Code Build and Push
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: ubuntu-18.04
steps:
- checkout: self
- task: Docker#0
displayName: Build an image
inputs:
azureSubscription: 'sc-***dp'
azureContainerRegistry:
loginServer: *****aksacr.azurecr.io
id: "/subscriptions/4f76bb2f-c521-45d1-b311-b87bf**747ff/resourceGroups/eus-***dp-rg/providers/Microsoft.ContainerRegistry/registries/*****aksacr"
imageName: ***dpacr.azurecr.io/ims-***dp/$(Build.Repository.Name):$(Build.BuildId)
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry:'sc-***dp'
- task: Docker#2
displayName: Push an image
inputs:
azureSubscription: 'sc-***dp'
repository: ims-***dp
azureContainerRegistry: '{"loginServer":"*****aksacr.azurecr.io", "id" : "/subscriptions/4f76bb2f-c521-45d1-b311-b87bf**747ff/resourceGroups/eus-***dp-rg/providers/Microsoft.ContainerRegistry/registries/*****aksacr"}'
action: Push an image
I would recommend to create a new service connection with the container registry if one does not exist.
and select Azure Container Registry.
Then you can push in your registry using the below task. Keep in mind that ${{container}} should have a notation as xxx.azurecr.io/imagename:tag
- task: Docker#2
displayName: pushing image ${{container}}
inputs:
containerRegistry: 'serviceConnectionName'
repository: '${{container}}'
command: 'push'
tags: |
mytag
Try to push the code in azure devops pipeline so you can directly push image in azure container registry
Here we can docker task it can build up and push the image with help of setting and it take the image from repository and then it can push into the azure container registry
task: Docker#2
displayName: Push an image
inputs:
command: buildAndpush
repository: $(imageRepository)
dockerfile: $(dockerfilepath)
ContainerRegistry: $(dockerRepositoryserviceconnectio)
tags:
$(tag)
Save and run.
Note : After running you can't see Repository please refresh again because in azure pipeline the job may be still running once the job is completed & successfully run and refresh you can able see image is pushed
Please refer this document by rajaniesh kaushikk has given detail information
Please check the repository and containerRegistry value in your Docker#2 task, make sure they points to target resource. My script for your reference below. Please note the parameters in Docker#2 task, no azureContainerRegistry and command needed.
variables:
dockerRegistryServiceConnection: acrconn1
imageRepository: 'acrlearn'
containerRegistry: 'test.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'windows-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)

How to run already deployed to azure app service container?

I use azure pipeline to build and deploy my docker container (ASP.Net core 3.1).
All steps finish successfully, I can see my container on Azure registry and App service portal pages. But seems like the instance is not running on App Service because I can see only "Starting page" which suggests me to deploy code and my swagger page returns 404.
So the question is: How to run already deployed by pipeline image?
My azure pipeline yml file:
# Docker
# Build a Docker image
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- master
resources:
- repo: self
variables:
tag: '$(Build.BuildId)'
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: ubuntu-latest
steps:
# run tests
- task: DotNetCoreCLI#2
displayName: Run tests
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: 'ProjectName'
dockerfile: '$(Build.SourcesDirectory)/ProjectName/API/Dockerfile'
buildContext: '$(Build.SourcesDirectory)/ProjectName/'
containerRegistry: AzureContainerRegistryConnection
tags: |
$(tag)
# Deploy stage to your App Service
- stage: Deploy
displayName: Deploy to App Service
jobs:
- job: Deploy
displayName: Deploy
pool:
vmImage: ubuntu-latest
steps:
- task: AzureWebAppContainer#1
displayName: Deploy to App Service
inputs:
azureSubscription: 'my_free_subscription(my-subscription-id)'
appName: 'app-service'
containers: 'projectnamecontainerregistry.azurecr.io/RepositoryName:$(tag)'
UPD
I dunno mb it's a trouble with app service settings in terraform
resource "azurerm_app_service_plan" "app_service_plab" {
name = "app-service-plan"
location = azurerm_resource_group.store_group.location
resource_group_name = azurerm_resource_group.store_group.name
kind = "Linux"
sku {
tier = "FREE"
size = "S1"
}
}
resource "azurerm_app_service" "app_service" {
name = "app-service"
location = azurerm_resource_group.store_group.location
resource_group_name = azurerm_resource_group.store_group.name
app_service_plan_id = azurerm_app_service_plan.app_service_plab.id
}
UPD2
Started to use new deployment task AzureRmWebAppDeployment#4 and it does do something I can even see logs on app service machine now.
- stage: Deploy
displayName: Deploy to App Service
jobs:
- job: Deploy
displayName: Deploy
pool:
vmImage: ubuntu-latest
steps:
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: 'free_subscription(id)'
appType: 'webAppContainer'
WebAppName: 'app-service'
DockerNamespace: 'containerregistry.azurecr.io'
DockerRepository: 'myRepository'
DockerImageTag: $(tag)
StartupCommand: 'az acr login --password $(REGISTRY_PASSWORD_UNSECRET) --username $(REGISTRY_USERNAME_UNSECRET) --name (MyRegistry)containerregistry.azurecr.io && docker run'
appSettings: |
-CONNECTION_STRING $(CONNECTION_STRING_UNSECRET)
But seems like az acr login doesn't work :(
Now I'm getting such error
2020-09-19T19:29:08.498Z ERROR - Image pull failed: Verify docker image configuration and credentials (if using private repository)
2020-09-19T19:34:00.826Z INFO - Pulling image from Docker hub: MyContainercontainerregistry.azurecr.io/MyRepo:85
2020-09-19T19:34:01.151Z ERROR - DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get https://MyContainercontainerregistry.azurecr.io/v2/MyRepo/manifests/85: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information."}
Use docker run cmd.
Add the task Docker->delete the field cmd content and add the cmd run

run spring boot application on azure container registry instance

I'm trying to run a spring boot application on azure. This is what I'm trying to accomplish:
local project > Azure Devops Git Repos > Azure container registry
From my local, I push the changes to Azure Devops Repos. Repository has a pipeline that does maven package, create docker image and push the image to Azure Container registry. And I'm running the container image using Azure container instance. Container instance is deploying successfully, but when I try to access a rest api defined in spring boot application, nothing happens. Page keeps on loading.
I try to access the GET api like this:
my_public_ip:8080/hello
DockerFile:
FROM openjdk:8
VOLUME /tmp
ADD target/command-center.jar command-center.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/command-center.jar"]
azure-pipeline.yaml
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- master
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: 'f4031dc5-3c27-4f41-be8c-2abdf064eaf2'
imageRepository: 'commandcenter'
containerRegistry: 'sapcemission.azurecr.io'
dockerfilePath: '**/command-center/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Maven#3
inputs:
mavenPomFile: 'pom.xml'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
mavenVersionOption: 'Default'
mavenAuthenticateFeed: false
effectivePomSkip: false
sonarQubeRunAnalysis: false
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
Azure container instance settings:
What am I doing wrong here?
Azure Container Instance doesnt support port mapping like docker. So in your case when you setup ACI, port property should be set to 8080 (not default 80).

Resources