Sometimes installing new software takes a lot of time.
I would like to install software in the VM and reuse this software later in new VMs. There are two ways to do that.
create an image after installations as described here or
use the VHD disk with the OS and preinstalled software (e.g. C:)?
In the first case I would just create a new VM directly from that image, in the second case I would attach the disk after the new VM is created.
Which way is preferable and possible at all?
Let me try to address your two points below.
1) An image is really about creating a sys prepped [1] version from an existing Azure VM ( that presumably has all the software you need installed).
Once you have captured the image, you can then create additional VM(s) based on it.
You can also go Azure Marketplace or VM Depot [2] and choose from a set of pre-built images and easily spin VMs based on it. This is the quickest way to get up and running with a VM.
2)
A disk is created from an image when you provision a VM. By default an OS disk will be created. But if your image includes data disks those will also be added to the VM.
To your point, #2 allows you to attach additional data disks (number of disks you can attach depends on the size of the VM you are creating). Note that you can attach an empty or an existing disk.
[1]
https://technet.microsoft.com/library/bb457073.aspx
[2]
http://azure.microsoft.com/en-us/marketplace/virtual-machines/
Related
The default Disk Size when creating a VM in azure is 30GB. I only use around 3GB of it. Now, Azure offers a 4GB disk, at a sixteenth of the cost. They decided to make it SO complicated to make it 4GB or 8GB, instead of just offering as an option when creating a VM.
All I need to do is to create a new 4GB managed disk.
How do I get the VHD file for ubuntu so I can finally make the disk? I was looking everywhere but can't find the answer.
EDIT:
As mentioned in the comments, what you require is the Desktop version of Ubuntu and not the Server one. In addition to Ubuntu NOT providing any Azure-compatible Desktop version of Ubuntu, Azure only endorses the Ubuntu versions below:
Ubuntu Server and Pro. 16.x, 18.x, 20.x
Information about extended support for Ubuntu 12.04 and 14.04 can be found here: Ubuntu Extended Security Maintenance.
Additional distro support info here too.
For the images below, you rightly pointed out (apologies for overlooking) that it will still provide you with a 30GB image after conversion.
Also, because Ubuntu Server and Desktop editions share the same kernel, it should be possible for you to follow the same steps to download and create your custom image by following the same prep steps outlined for non-endorsed distros, convert it to VHD and then upload it. Even if this is successful, it's also possible that it will cause some other issues with underlying Azure services.
One last option I thought of is using Azure Migrate to move an existing on-premises Ubuntu VM to Azure. This option failed when we tried but I should disclose our VM was running a much older version of CentOS. When using Azure Migrate, Azure will analyse your machine and tell you whether your VM is supported.
Original Answer:
So here's the thing - you can only use images that are in an Azure Storage Account. Although it seems you've been through a lot already, I'm afraid it doesn't get any easier but steps are generally clear on how it can be done.
1 - Get the image
The Azure documentation on the topic actually recommends to download it from the official Ubuntu cloud repository.
Note
Before attempting to create your own custom Ubuntu image for Azure, please consider using the pre-built and tested images from https://cloud-images.ubuntu.com/ instead.
You can also just download one of the two versions listed on the documentation - this will provide you with a .VHDX file.
2 - Convert the VHDX to VHD
Azure does not support the .VHDX format. See:
The VHDX format is not supported in Azure, only fixed VHD. You can convert the disk to VHD format using Hyper-V Manager or the Convert-VHD cmdlet.
Here's an article which explains how to do this with both options.
3 - Upload the VHD to Blob
Once you have the file in VHD format, you can now upload it to Azure.
You can use AzCopy which is rather complicated when compared to a direct upload, but just mentioning that this can also be done. It also seems to be the recommended approach from Azure.
The easiest option is to upload manually through the Portal. See this article that goes into details - pick what's relevant to you.
(a) Choose your storage account, create/select a container you want to upload to
(b) Click Upload and select the Ubuntu VHD file to upload.
Ensure that the Blob type is set to Page Blob.
4 - Create a Managed Disk and VM
Once your Ubuntu VHD is uploaded, go to its properties page and get the blob URI.
Use that when creating the disk.
Create a VM from the managed disk.
That should be all - it's very painful but possible
I managed to go as low as to just 3 GB and have Ubuntu 20.04 LTS (Focal Fossa) running as an Azure VM.
Prerequisites are below mentioned images downloaded from https://cloud-images.ubuntu.com/focal/current/, a tool to convert VMDK to VHD (e.g. 2Tware Convert VHD) and Hyper-V Manager installed/turned on under local Windows features.
I couldn’t go that low when using the dedicated focal-server-cloudimg-amd64-azure.vhd.zip. When unpacked the file livecd.ubuntu-cpc.azure.vhd is 30-31 GB large. I didn’t manage to shrink it (only an expansion was possible), so the result would have been worse than the original (30GB) Ubuntu VMs from Azure Marketplace.
I have downloaded focal-server-cloudimg-amd64.vmdk instead. Here a conversion to VHD was needed with help of 2Tware Convert VHD. Unfortunately, if uploaded to Azure this image is not accepted as it is a Dynamic VHD.
I had to use Hyper-V Manager to convert it to fixed. Here however I faced one another issue. When in Hyper-V Manager going through Edit Disks…->Browse…->Convert->VHD->Fixed the file I was getting was 10 GB large (noticeable improvement, but still larger than the final 3 GB).
Instead of going through Edit Disks… I went for Quick Create… of a new Virtual Machine. I chose Local installation source and selected the VHD file converted from VMDK. Finally, I clicked Create Virtual Machine.
The VM is created, but there is no need to connect to it. In fact, it can be deleted right away, as what is really needed is the file (presumably) stored as New Virtual Machine.vhdx under C:\Users\Public\Documents\Hyper-V\Virtual hard disks\.
With help of Hyper-V Manager this time I managed to both convert the file to Fixed VHD as well as to shrink it to 3 GB.
Later the process is as described in #Ked Mardemootoo’s answer (the instruction from IBM). A new Storage account needs to be created in Azure. Then in the Storage Account a new blob Container needs to be created. The VHD file is to be uploaded to the Container. Later a new Image needs to be created from this Storage blob and from this image a new small VM can be successfully created.
The Azure portal web interface has several options for creating 'images' of a VM including:
snapshot creates a snapshot of the machine which can presumably be restored or copied (what I am trying to do without much success so far)
capture generalises a VM into an image that can be used to create multiple VMs (in theory)
The capture option makes the original VM unusable. In fact you are prompted about whether you want to keep it as it will no longer run (which indeed it can't).
Why is capture a destructive operation?
When you generalise an image using sysprep it will remove the customisation from your VM and that particular VM be of no use except as a golden image. This golden image then can be used as a template to spin more VM’s by passing on the missing parameters which sysprep removed.
If you like to keep the VM, you are using for capture it is recommended that you make a copy of it first and then use it for capture and sysprep process
Refer below for details
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/capture-image-resource
Snapshot is more of a VM state at a given point in time and. Mostly used for migration to another region or to capture a state of an application running the VM before an application upgrade or patch.
I am new to the cloud and have a very basic questions that I am having a hard time understanding.
I have created an Azure Virtual machine and now I am installing third party software using RDP. Example: BareTail, NotePad++, a Trading Software(TWS), the goal is to replace my own Desktop/PC with the one in cloud, to help me when I am travelling.
Question: How often will i have to re-install thee s/w ? Or is it a one and done ? I am hoping only one time, but not sure.
Thank You.
In Azure, by default, if you do not attach any data disk you will have a persistent system disk and temporary swap disk.
Just do not install / do not put any data on a temporary disk and all your data will persist until you pay for your subscription or you will remove your VM with OS Disk.
Even if you remove Virtual Machine resource, Azure will not remove you OS disk, so you will able to find your data but you need to use command line tools to create a Virtual Machine from existing OS disk to recover your data, so be careful. You can use Azure Locks to protect your resources from deletion.
If you want to protect data on your disks from corruption you have also Azure Backup.
Start / Stop operations do not have any impact on your data and software if it was not placed on a temporary disk.
I have my virtual machine running Linux. I've created it via new "Resource manager". Then I added data disk to it.
Then I created new Virtual Machine. And I want it to use the same data disk attached to the first one (at least in read-only mode).
When I try to "attach existing disk" to this new machine I get this error:
Failed to attach existing disk 'DISK-NAME.vhd' to the virtual machine 'MACHINE-NAME'. Error: Failed to acquire lease while creating disk 'DISK-NAME.vhd' using blob with URI https://BLOB-URI-disk1.vhd. Blob is already in use.
How do I attach existing data disk which is in use by another machine to my current machine?
Simply, you can't.
A disk in Azure can only be attached to a single VM at a time. in order to attach it to another VM you need to disconnect it from the first.
If you need to have data shared amongst many machines, you could use Azure File shares which provides SMB 2.1 and SMB 3.0. Most modern Linux versions can connect to this quite seamlessly.
If you need block storage, i.e. sharing an actual disk, you would need to spin up a separate VM and use a protocol like iscsi (or NFS) to share that disk amongst multiple machines.
Maybe the "StorSimple" Solution from Microsoft Azure could be a way to go? I would describe it as SAN on Azure.
I have not tested it today, but it should be possible to connect several virtual machines to it, and share the files.
You can find more information in the documentation:
Azure StorSimple
Extending Michael's answer a bit, based on your comments under his answer:
If the goal is to provide data access when a VM goes down for some reason, and the data is on an attached disk, then you can detach the disk from the downed VM and reattach it to another VM (the detach breaks the lease, and the reattach creates a new lease). But be aware: This is a time-consuming process - it might take a minute or two for each operation. But you can certainly do it, and you can do it programmatically.
Regarding disk replication: Yes, Azure disks are triple-replicated (or replicated 6 times, if you enable georeplication). But logically, it's a single disk; it's replicated for durability, not for you to attach to different replicas.
Michael mentioned Azure File Service. Maybe it wasn't clear what that was but... there's no Virtual Machine involved with File Service - it's a durable-storage SMB service, with its own SLA unrelated to your VMs. You may attach to it from multiple VMs and read/write files as you would a locally-attached disk (which seems to be the problem you're trying to tackle).
Regarding replication of data across VM's: If you choose to go this route, and make physical copies yourself, it's strictly up to you how you do it - there is no "best way." But this is the type of thing database engines are built for (and you can imagine how complex they are, dealing with replication, journaling, errors, etc.).
I have an Ubuntu 14 VM on Azure to host my developed web sites. (I do not think the OS matters in the point of view the question, but never know)
I've discovered the relatively new Capture button, so for the storage price of a disk size I regularly save a "snapshot" via the Capture function (I am not preparing the image for provisioning, I mean not checking the "I have run 'waagent -deprovision' on the virtual machine" checkbox). Be aware quickly becomes pretty addictive.
The result is an image what I can use when creating new machines, its there in My Images in the wizard. This can function as a backup/rollback worflow. If I delete the VM and create a new from one of resulting image of the previously captured "snapshots". (again, no provisioning will take place)
It is possible to initiate the Capture operation on a running VM. It is not clear for me, if the result will be an image what is a template for a new VM, and that VM will start up and boot, in what state the filesystem etc will be?
Is not it a similar state than sudden power lost? If yes, then it is strongly recommended to always shutdown the VM before capturing, however this such a pain and productivity killer, so no one (including me) wants to do unless it is mandatory.
Accidentally I've switched to the new Azure portal and there the Capture UI says:
Capturing a virtual machine while it's running isn't recommended.