I have a bitbucket pipeline like this:
definitions:
steps:
- step: &step1
name: step1
image: eu.gcr.io/$PROJECT/my_private_image:tag
script:
- ...
And I want to pull "my_private_image:tag" from my Google private gcr or artifact registry, Any help?
Thanks in advance.
Just Read the Fine Manual
To pull an image from GCR, you need to configure a service account for Pipelines with "Viewer" access in your GCP admin console. Download the generated JSON private key, and copy/paste it into a secure variable in Pipelines. The configured variable can then be used directly in the password field in your YAML image configuration [...]
definitions:
steps:
- step: &step1
name: step1
image:
name: eu.gcr.io/$PROJECT/my_private_image:tag
username: _json_key
password: '$GCR_JSON_KEY'
script:
- ...
Related
There are a lot of questions that seem tangentially related to this, but unfortunately, I've not been able to tie these disparate bits of information together to get to a solution.
I'm trying to use a custom docker image with an Azure function and have been following along with this guide.
I've been building up these steps inside a devops pipeline so I can accurately reproduce them, and I've got that working fine as far as provisioning is concerned.
However, I'm struggling when it comes to actually uploading the built image to the ACR.
I've found this yaml snippet in the documentation:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: "xxx"
dockerfile: "xxx"
containerRegistry: "xxx"
imageRepository: "xxx"
tags: "xxx"
However, when I fill this out with the url for our ACR as the container registry, I get an error about needing to use a service reference.
Now, I've looked through the docs pretty extensively and I can't quite work out how this is all meant to tie together in a devops pipeline...
I've tried to come up with something like this:
resources:
containers:
- container: "xxx" # name of the container (Alias)
type: ACR # type of registry
azureSubscription: arm-connection # name of the ARM service connection
resourceGroup: "$(resourceGroupName)" # Azure resource group with the container
registry: "$(dockerRegistryName)" # Azure container registry name
repository: "xxx" # name of the of container image collection
trigger:
tags:
- "$(dockerTag):$(dockerVersion)" # tag for the container image to use
But that looks like it's creating a pointer to a specific container rather than to the registry...
So yeah, I'm a bit stuck... Could someone please help by pointing out what I'm misunderstanding here and ideally providing an example of how I can build and push an image to the ACR?
Command called buildAndPush allows for build and push of images to a container registry in a single command, you can find an example in Azure DevOps documentation
You will need to create Service Connection for Azure Container Registry in your Azure DevOps to configure an access, please see doc
The following YAML snippet is an example of building and pushing an image to ACR
steps:
- task: Docker#2
displayName: Build and Push
inputs:
command: buildAndPush
containerRegistry: azureContainerRegistryServiceConnectionName
repository: repositoryName
Dockerfile: '**/Dockerfile'
buildContext: .
addPipelineData: true
tags: |
tag1
I'm trying to send a kubescape report via github action and unfortunetely, I have an error shown as below:
Scan results have not been submitted: Sign up for free:
https://portal.armo.cloud/account/sign-up
Of course I've created an account, and I try to submit the report like this:
kubescape:
runs-on: ubuntu-20.04
strategy:
matrix: { dir_kube: ['ionos/kubernetes/dev/*.yaml', 'azure/kubernetes/prod/*.yaml', 'ionos/kubernetes/prod/*.yaml']}
steps:
- name: Clone repo
uses: actions/checkout#master
- name: Install kubescape
run: curl -s https://raw.githubusercontent.com/armosec/kubescape/master/install.sh | /bin/bash
# Scanning cluster, specified by filter path
- name: Scan repository
run: kubescape scan --submit --account=${{ secrets.KUBESCAPE_REPORT }} ${{ matrix.dir_kube }}
The secret is corresponding to my key account.
What I've tried:
Replace the secret (In case I would have mistyped it)
add --verbose
add --logger debug
scan and send the report from my machine directly
(I manage to send the report correctly, but as you wonder, I don't want to do from my machine as it is a cron job.)
Is it even possible to do it from GA? Am I missing something?
Kubescape just released support for submitting file scans to the portal. Check it now with the latest version!
Set a github secret called KUBESCAPE_ACCOUNT.
Then, add the following code to your workflow (copied from here):
name: Kubescape scanning for misconfigurations
on: [push, pull_request]
jobs:
kubescape:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- uses: kubescape/github-action#main
continue-on-error: true
with:
format: sarif
outputFile: results.sarif
# Specify the Kubescape cloud account ID
account: ${{secrets.KUBESCAPE_ACCOUNT}}
- name: Upload Kubescape scan results to Github Code Scanning
uses: github/codeql-action/upload-sarif#v2
with:
sarif_file: results.sarif
I am trying to automate my deployment process by creating a pipeline in Azure DevOps that does the following
Build my project, create a docker image, then pushed the image to a private Azure Registry service.
Deploy the image on a slot called staging in Azure Web Service.
Here is the .yaml file that I am using
trigger:
- master
resources:
- repo: self
variables:
dockerRegistryServiceConnection: 'MyPrivateRegistry'
imageRepository: 'MyPrivateRepositoryName'
containerRegistry: 'MyPrivateRepositoryName.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
azureSubscription: 'MyPrivateSubscribtionName(5c4b9a4b-private-subscribtion-id-91503531e1a0)'
appName: 'private_appname'
resourceGroupName: 'PrivateResourceGroup'
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Push and Build
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: '$(dockerRegistryServiceConnection)'
tags: |
$(tag)
- job: DeployToStaging
displayName: Deploy To staging
dependsOn: Build
condition: succeeded()
pool:
vmImage: $(vmImageName)
steps:
- task: AzureWebAppContainer#1
inputs:
azureSubscription: $(azureSubscription)
appName: $(appName)
deployToSlotOrASE: true
resourceGroupName: $(resourceGroupName)
slotName: 'staging'
containers: '$(containerRegistry)/$(imageRepository):$(tag)'
The projects is built successfully and pushed to the private registry as expected. I can verify that the new image in pushed with a new tagId. However, my container fails to start with the following error
Image pull failed since Inspect image returned null: MyPrivateRepositoryName.azurecr.io/MyPrivateRepositoryName:151
Here is the suggestion I see
Please check the repository name, image name, and container definitions defined by DOCKER_REGISTRY_SERVER_USERNAME, DOCKER_REGISTRY_SERVER_URL, and DOCKER_REGISTRY_SERVER_PASSWORD.
When I go to the staging slot configuration I see the following and the values are all correct. I copied these values from the "Access keys" section in the Container Registry service after enabling the Admin user
What am I missing here? How can I get the slot to correctly pull the docker image from the contain registry?
Updated
Looking more at the logs gives me this error
2022-04-01T20:36:21.409Z ERROR - Pull image threw Exception: Input string was not in a correct format.
2022-04-01T20:36:21.411Z INFO - Pulling image from Docker hub: privateregistry.azurecr.io/privateimage:152
2022-04-01T20:36:21.594Z ERROR - DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get https://privateregistry.azurecr.io/v2/privateimage/manifests/152: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information."}
It sounds like something does not have permission to pull the docker image from the repository. Question is what object need to have this permission? Would I add permission to the private repository of would I add it to the Web Service?
Few things you can check if Enable Admin Access does not work.
Is your app service in the same subscription as your ACR? If not, try moving it to the same subscription.
In Azure Portal -> App Service -> Deployment Center, see if you have conflicting settings. If it's empty, try set up the container registry information here instead of passing in app settings as env variables.
If still having errors, I recommend you creating another credential to login to ACR instead of using the admin one. (For this, it won't involve system identity or managed identity)
Register a new app in Azure that will read from ACR
In ACR -> Access Control -> give this app ACR PULL permission
Replace your app service app setting with the following
DOCKER_REGISTRY_SERVER_USERNAME = Client Id of the created app
DOCKER_REGISTRY_SERVER_PASSWORD = Client Secret of the created app
Pushing and pulling of image to Azure Container Registry task in Azure DevOps pipeline fails. When tried to pull or push from local system, there's no problem but when tried to do it using the Azure Devops pipeline it fails. Docker login was successful but it fails when I want to pull the image from the ACR with the following result:
**Error response from daemon: Head "***/a2/abcd/manifest/latest": unauthorized: Invalid clientid or client secret.
##[error]Bash exited with code '1'.
##[debug]Processed: ##vso[task.issue type=error;]Bash exited with code '1'.
I checked all the service connections in Az Devops, and they all look correctly configured. Checked the associated service principals as well if they have AcrPull and AcrPush permissions, all of them are in place. Just couldn't understand what's going wrong.
My Yaml looks like this:
trigger: none
schedules:
- cron: "0 0 0 * *"
displayName: **** *
branches:
include:
- abcd
always: true
pool:
vmImage: 'ubuntu-latest'
variables:
- name: acrname
value: *****.azurecr.io
stages:
- stage: abcd
displayName: "pull images from acr"
jobs:
- job: abcdef
displayName: "abcdef"
pool:
vmImage: ubuntu-latest
steps:
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry: '*****.azurecr.io'
- bash: |
docker pull $(acrname)/abc-def:latest
docker pull $(acrname)/igh-jkl:latest
name: pull
displayName: 'pull acr images'
Can anyone help?
I had the same issue with the ACR service connection was expired. So I had to create a new service connection by using these steps.
Docker ID and Docker Password can be obtained from ACR --> Settings --> Access keys
Update your pipeline with this new service connection and you are good to go. Please rate if this solution helps you.
In my case, when I ran into this issue, the simple and clean resolution was to use the docker login. In your situation, it looks like this would be a good solution :
docker login $(acrname)
prior to your calls to get your images
docker pull $(acrname)/abc-def:latest
In my case, the docker login password expired. So I have to do the following:
Go azure and generate the new password for the docker app authentication.
Copy the newly generated password.
Go to your virtual machine where docker is running.
Try the below command
docker login blah.azurecr.io --username your-user-name-here --password yourhaspasswordhere~5Crf9b
Now you are good to go.
I have a very weird (and I suppose easy to fix) problem :) I am trying to have a working CI/CD pipeline in Azure. For this purpose, I have a repository in Azure devops and build and release pipeline created. I am publishing docker images to Azure Container Registry and during release, I am pulling this image (or at least - I am trying because it doesn't work) and I am trying to publish it on Webapp for containers. The "app" in my case it is SingalR hub on .NET Core 3.1 (but I don't suppose it makes a difference in the problem I am having)
If somebody wants to know in details how did i configure it - here is the tutorial i did use:
https://wikiazure.com/devops/azure-devops-automate-your-release-pipeline-to-provision-a-docker-container-to-azure-web-app-for-containers/
There were some doubts/differences in the tutorial (for example - why initially in the tutorial web app is being configured on Docker hub, when in fact it is using ACR. And why to connect to ACR the tutorial uses Azure Resource Manager connection (And not dedicated Docker container --> ACR connetion) And why later on in build pipeline there is some weird id set for dockerRegistryServiceConnection (i am giving in this place name of my ACR docker service connection)
But the whole build pipeline is working. It is publishing image to ACR. Everything is fine till this step.
The problem starts when I want to publish Azure WebApp with this image. The problem is with ... TAGS :) They are mismatching. I have automatic CI/CD - so when i push some change to the repo i see that release pipeline is working. It is creating the image in the ACR. Then i see, that release pipeline is running. Everything is "correct" - meaning no error are seen and the release is green.
But when i go to App service and Container settings i see from logs:
2020-04-21 18:02:28.321 INFO - Pulling image: myAcrName.azurecr.io/mobile/signalr:c7aead0c46b66afc4131935efc7e6a51280dfb1a
2020-04-21 18:02:28.761 ERROR - DockerApiException: Docker API responded with status code=NotFound, response={"message":"manifest for myAcrName.azurecr.io/mobile/signalr:c7aead0c46b66afc4131935efc7e6a51280dfb1a not found: manifest unknown: manifest unknown"}
2020-04-21 18:02:28.761 ERROR - Pulling docker image myAcrName.azurecr.io/mobile/signalr:c7aead0c46b66afc4131935efc7e6a51280dfb1a failed:
2020-04-21 18:02:28.762 INFO - Pulling image from Docker hub: myAcrName.azurecr.io/mobile/signalr:c7aead0c46b66afc4131935efc7e6a51280dfb1a
2020-04-21 18:02:28.867 ERROR - DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get https://myAcrName.azurecr.io/v2/mobile/signalr/manifests/c7aead0c46b66afc4131935efc7e6a51280dfb1a: unauthorized: authentication required"}
2020-04-21 18:02:28.870 ERROR - Image pull failed: Verify docker image configuration and credentials (if using private repository)
Very sophisticated error but the root cause is, that he is trying to get the image with non-existing tag, which is GIT COMMIT tag. And it suppose to get image by $(Build.BuildId) (this was my first attempt) or by $(Build.BuilNumber) (this was my second attempt)
Here is how this pipeline step (Deploy Azure App Service) looks like:
- task: AzureRmWebAppDeployment#4
displayName: 'Deploy Azure App Service'
inputs:
azureSubscription: mySubcsriptionARM
appType: webAppContainer
WebAppName: myProductsignalr
DockerNamespace: myAcrName.azurecr.io
DockerRepository: mobile/signalr
DockerImageTag: '$(Build.BuildNumber)'
When i go to Release pipeline logs as a "Deploy Azure App Service" log i see that
2020-04-21T18:41:01.6012767Z ##[section]Starting: Deploy Azure App Service
2020-04-21T18:41:01.6367124Z ==============================================================================
2020-04-21T18:41:01.6367787Z Task : Azure App Service deploy
2020-04-21T18:41:01.6368381Z Description : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2020-04-21T18:41:01.6368765Z Version : 4.163.5
2020-04-21T18:41:01.6369158Z Author : Microsoft Corporation
2020-04-21T18:41:01.6369603Z Help : https://aka.ms/azureappservicetroubleshooting
2020-04-21T18:41:01.6369976Z ==============================================================================
2020-04-21T18:41:03.8970184Z Got service connection details for Azure App Service:'myProductsignalr'
2020-04-21T18:41:04.5534864Z Trying to update App Service Configuration settings. Data: {"appCommandLine":null,"linuxFxVersion":"DOCKER|myAcrName.azurecr.io/mobile/signalr:1f283100"}
2020-04-21T18:41:05.5465725Z Updated App Service Configuration settings.
2020-04-21T18:41:05.5495890Z Trying to update App Service Application settings. Data: {"DOCKER_CUSTOM_IMAGE_NAME":"myAcrName.azurecr.io/mobile/signalr:1f283100"}
2020-04-21T18:41:06.2703349Z Updated App Service Application settings and Kudu Application settings.
2020-04-21T18:41:32.4715682Z Updated App Service Application settings and Kudu Application settings.
2020-04-21T18:41:33.4179962Z Successfully updated deployment History at https://myProductsignalr.scm.azurewebsites.net/api/deployments/111587494492765
2020-04-21T18:41:33.5945654Z App Service Application URL: http://myProductsignalr.azurewebsites.net
2020-04-21T18:41:33.6180118Z ##[section]Finishing: Deploy Azure App Service
What amazes me, that it is showing, that everything was ok - when it was far from "ok" :)
When i go to container settings after:
a) new code is published
b) build pipeline fires
c) release pipeline fires
i see it like this:
The tag is empty. If i would pick some tag manually:
And would choose: "SAVE" everything works correctly (SingalR is up and running correctly)
Clearly, I am missing something :/ Help me to see what;)
The root cause for me is that this fragment:
DockerImageTag: '$(Build.BuildNumber)'
should insert build number (as stated) and the info from container settings should be:
Pulling image: myAcrName.azurecr.io/mobile/signalr:20200421.09 (for BuildNumber 20200421.09) and it is inserting GIT COMMIT there as a tag and ends up with: Pulling image: myAcrName.azurecr.io/mobile/signalr:c7aead0c46b66afc4131935efc7e6a51280dfb1a Why o why?:)
[UPDATE 22.04 10:56]
I am posting build pipeline that i am using currently. I don't suppose it is important as it is working correctly, and the problem is more with deployment of correctly created docker image (on ACR), than with creating this image by the build pipeline. Nevertheless, here is the pipeline:
# Docker
# Build a Docker image
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- master
resources:
- repo: self
variables:
dockerRegistryServiceConnection: 'MyProductDockerACR'
imageRepository: 'mobile/signalr'
containerRegistry: 'myAcrName.azurecr.io'
dockerfilePath: '**/Dockerfile'
tag: '$(Build.BuildNumber)'
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push image to container registry
inputs:
containerRegistry: $(dockerRegistryServiceConnection)
repository: $(imageRepository)
command: 'buildAndPush'
Dockerfile: $(dockerfilePath)
tags: |
$(tag)
I saw the release you are using is configured by UI. It's work logic much different with the one which configured by YAML.
In fact, here what you received just be the different performance produced while the running reason of the release are different.
I guess this release has the artifact source which targeting to Repos, right? You can confirm by checking its icon.
While the release source is coming from Repos, then the Build.BuildNumber would be the short part of the commit id(8 characters). And the Build.BuildId is the complete commit id.
If you want the release keep using the Build.Buildnumber value which the corresponding build(created/pushed image) was using, you must make sure the release source is targeting to this build. Also, this build need has artifacts generated. According to the YAML you shared, obviously, you haven't done that.
Only the release triggered by build along with artifact, then the Build.BuildNumber can be like 20200422.1 which the build was using.
So, please go your release definition, and re-configure its source to make sure it is coming from build artifact instead of repository.
Yes. You are right. You have mismatch in tags.
In Docker#2 task you can define tags:
steps:
- task: Docker#2
displayName: Login to ACR
inputs:
command: login
containerRegistry: devopsmanual-acr
- task: Docker#2
displayName: Build and Push
inputs:
repository: $(imageName)
command: buildAndPush
Dockerfile: build-docker-image/SampleAppForDocker/DOCKERFILE
tags: |
$(Build.BuildNumber)
- task: Docker#2
displayName: Logout of ACR
inputs:
command: logout
containerRegistry: devopsmanual-acr
Your definition should be pretty much like this one. Whre devopsmanual-acr is connection to your ACR.
.
I recently mad a blog post about creating docker images on Azure DevOps so maybe this will be also helpful for you.
If this won't be enough to solve your issue, please edit your question and show how you create and push your images.