I am building an Azure DevOps pipleline using Terraform. The pipeline creates a Linux server and then logs into the Linux server to update packages and install Apache.
I am currently storing the private key in my BitBucket repo (I know, this is not best practice), which are then pulled down onto the build agent server and then I login to the new server with the following command:
ssh -f -q -o BatchMode=yes -o StrictHostKeyChecking=no -i ../private_key.pem ubuntu#$ip sudo apt update -y
What is the best way to store and then retrieve the private key within Azure DevOps?
Two options I can think of:
1) Create an ssh service connection in azure DevOps. Reference the service connection in your pipeline. https://medium.com/#sibeeshvenu/ssh-deployment-task-in-azure-pipelines-b0e2923bd7b4
2) Store the SSH key as an Azure Key Vault secret and then download the secret using the Azure CLI during the build.
az keyvault secret download --name mysshkey --vault-name mykeyvault --file ~/.ssh/id_rsa
Authenticate the Azure CLI using a service principal, and supply the credentials to the pipeline using a variable group.
I found that Azure DevOps provides you a feature to upload secret files are part of the build. You can see more information here:
https://learn.microsoft.com/en-us/azure/devops/pipelines/library/secure-files?view=azure-devops
Related
I am trying to clone a repo from Azure DevOps on Linux VM on windows 11 using SSH. It is asking me for a username and password for https://XX.visualstudio.com. The steps are below. Am I missing anything?
Created a VM Ubuntu 18.04 LTS using Hyper V
Installed git
Generated a key pair
added a public key in Azure DevOps
and git clone the repo
I was not able to create and edit ~/.ssh/config, it is not present, some solutions were suggested to edit it.
Please make sure we are using SSH URL when cloning.
Enter your passphrase for SSH key.
For more details about how to generate a new SSH key and set passphrase, you could refer to this doc: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent
I am trying to use Add-AzVMSshPublicKey cmdlet to update ssh key on existing Azure linux VM , command executed without an issue but key hasnt been updated on server.
Is there any other way available where i can able to achieve OR any Azure cli command to do same ?
The Add-AzVMSshPublicKey cmdlet is used to add the public keys for SSH for a virtual machine, when only creating the VM. If you use the command with Update-AzVM, the command will error. So it does not use to add the keys after VM is created.
The following example updates the SSH key for the user azureuser on the VM named myVM.
az vm user update \
--resource-group myResourceGroup \
--name myVM \
--username azureuser \
--ssh-key-value ~/.ssh/id_rsa.pub
You can use the above CLI commands to append the new public key text to the ~/.ssh/authorized_keys file for the admin user on the VM. This does not replace or remove any existing SSH keys. You can SSH into that VM to remove the old public key if you would like to remove it.
Read Manage administrative users, SSH, and check or repair disks on Linux VMs using the VMAccess Extension with the Azure CLI for more details.
Alternatively, you can reset the SSH key from the Azure portal---> reset password.
I have a linux system where I have installed dockers. I also have a registry on azure for which I have the user name and password. To get a list of docker images from a private registry we can simply use curl command like below:
curl localhost:5000/v2/_catalog
This command I tested when I installed private registry on my machine and it was giving me the list of images which I have in the registry. Now I have azure registry. I can login to it successfully but don't know what command I can run to get the list of docker images. Is this possible.? For example, if I run:
curl myregistry.azurecr.io/v2/_catalog
It shows:
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
How do I get the list of images stored in azure registry from my linux machine
Thanks
You can use the container registry cli for azure:
az acr repository list --name <acrName> --output table
https://learn.microsoft.com/en-us/azure/container-registry/container-registry-get-started-azure-cli
It is important to understand how docker lists the images in the registry.
Docker CLI provides command to pull/push/delete images from a private Azure Registry like myprivate.azurecr.io after the user authenticates itself using docker login command but the docker CLI does not provide any command to list the images in the private registry.
It is important to understand that the docker image ls only lists the images present on the local machine and not in a registry.
There are multiple answers that describe the Docker HTTP API V2 (Refer here) to list the images present in the registry. The HTTP v2 API v2/_catalog and other only work with local registry created on-premise but when user wants to list the images present in the Private Azure Registry one needs to use Azure CLI
What is Local Registry ?
The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license. Local Registry can be created to store and distribute images in house or on-premise.
Refer here : https://docs.docker.com/registry/ . One can create a private registry,push and pull image from there using Dokcker HTTP API V2.
Azure CR is a special type and inorder to list the images there is no other option to Azure CLI.
Use Case
- List the top three images present in the registry
The command for the same can be
az acr repository show-tags -n <RegistryName> --repository <RepositoryName> --orderby time_desc --output table | select -First 5
Not used to Azure I accidentally got stuck on the idea that I needed the Azure credentials to access the API, these answers strengthening that perception, but given you have the u/p you should be able to access it with curl in a simple:
curl -L --user <username>:<password> myregistry.azurecr.io/v2/_catalog
{"repositories":["name1", "name2", "nameN"]}
As yamenk said, you could use Azure CLI 2.0 to get your registry on azure.
Azure Cli 2.0 works on linux and docker, so I think it could work your linux machine.
Also, you could use Azure Rest APi to get registry on azure.
GET https://management.azure.com/subscriptions/<subscription id>/resourceGroups/<rg>/providers/Microsoft.ContainerRegistry/registries/<registry name>?api-version=2017-10-01
For get token, please refer to this link.
Using API like below:
curl -X "GET" "https://management.azure.com/subscriptions/********/resourceGroups/shuiapp/providers/Microsoft.ContainerRegistry/registries/shuitest?api-version=2017-10-01" \
-H "Authorization: Bearer $token" \
-H "Content-Type: application/json"
I've used this command to deploy a Kubernetes cluster in Azure:
az acs create -n acs-cluster -g acsrg1 -d applink789 --generate-ssh-keys
Everything is working- I can connect to the cluster with kubectl. Now I want to define an SSH step in a Continuous Delivery pipeline. The documentation indicates that this command created a public/private key pair. Where is the private key stored? I've looked in the .ssh, .kube, and .azure folders in my home directory but I can't tell if any of those values are the private key.
Figured it out- the documentation says the keys will be generated if they are missing. If the id_rsa keypair is present in the .ssh hidden directory, it is used. Connected with Putty using the azureuser default account.
I've tried absolutely everything I can think of to do a SSH reset of my user on my Linux VM (Hortonworks Sandbox to be precise).
The VMAccessForLinux will not install, it simply states that it fails to provision:
I've tried adding it as 1.*,1.1, 1.2 and now 1.4 as per https://github.com/Azure/azure-content/blob/master/articles/virtual-machines/virtual-machines-troubleshoot-ssh-connections.md
I can't access my SSH, and I can't do any of the Azure reset commands, either using Azure CLI or Azure PS.
The VM is a RM vm.
How can I resolve this?
In PS I get errors like:
I'm beyond tearing my hair out.
And before anyone suggest that I use the portal, this is what I'm offered there (thanks Azure):
I can't say if this is a universal fix, but I managed to resolve this issue, by using the following in the Azure CLI:
$ azure vm reset-access -n {VMNAME} -g {GROUPNAME} \
-u {SSH_USER} -p {SSH_PASS} -E 1.4 -vv --json
It did NOT work for my original user on the box though; I created ANOTHER user, and from there I did a password reset with a sudo on the box, then I could SSH into the box from that user.
Firstly, can you go through the instructions here if you've not already. The VM extension has changed recently and that is the latest doc to go through: https://azure.microsoft.com/en-us/blog/using-vmaccess-extension-to-reset-login-credentials-for-linux-vm/.
EDIT #1
Glad to see you resolved it by creating a new user with reset-access.
If azure vm reset-access should fail, the next step would have been to download this tool which can allow you to inspect the VHD when not logged onto the VM: https://github.com/paulmey/inspect-azure-vhd - and inspect Waagent log is /var/log/waagent.log (You can see extension updates here) and
extension.log in /var/log/azure/.