I have some Selenium test code that I need to run in parallel. In order for Selenium to run effectively, certain configurations have to be done on the machine (I.E. zone settings, Chrome and Firefox installs, etc.) and these settings are hard (if not impossible) to apply via an automated approach. I've manually created a VM, done all the setup and created an image following the directions in Microsoft's documentation.
Now I need to setup my code so that I can specify a VM image to use when creating the nodes. I've searched as much as I can and not found any documentation that explains how I can go about doing this. The example in the DotNetTutorial sample doesn't seem to have any way to specify an image.
There is a feedback item here on this same topic and shows the request as started on Jun 1st 2015. I'm hoping this means that it's done now and that it just hasn't been documented well.
Q: How I can specify a custom VM image as the source for my Azure Batch nodes?
https://github.com/Azure/azure-sdk-for-net/blob/AutoRest/src/Batch/Client/changelog.md
• Added support for deploying nodes using custom VHDs, via the OSDisk property of VirtualMachineConfiguration. Note that the Batch account being used must have been created with PoolAllocationMode = UserSubscription to allow this.
Updated Answer on 2017-12-05:
Custom images are now supported through normal Batch accounts (i.e., Batch service pool allocation mode accounts). You will need to specify a valid ARM Image Id and use Azure Active Directory authentication to create custom images (shared key auth does not support custom images).
Updated Answer on 2017-03-17:
Custom images are now supported through "User Subscription" Batch accounts. You can create these types of accounts in Azure Portal or through the newest management SDKs for supported languages.
Previous Answer:
Currently, custom VM images are not supported. As you noted, this is a feature that is being worked on. In addition to uservoice, you can periodically check for product updates at this site.
Related
I would like to tweak some settings in AKS node group with something like userdata in AWS. Is it possible to do in AKS?
how abt using
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_machine_scale_set_extension
The underlying Virtual Machine Scale Set (VMSS) is an implementation detail, and one that you do not get to adjust outside of SKU and disk choice. Just like you cannot pick the image that goes on the VMSS; you also cannot use VM Extensions on that scale set, without being out of support. Any direct manipulation of those VMSSs (from an Azure resource provider perspective) behind your nodepools puts you out of support. The only supported affordance to perform host (node)-level actions is via deploying your custom script work in a DaemonSet to the cluster. This is fully supported, and will give you the ability to run (almost) anything you need at the host level. Examples being installing/executing custom security agents, FIM solutions, anti-virus.
From the support FAQ:
Any modification done directly to the agent nodes using any of the IaaS APIs renders the cluster unsupportable. Any modification done to the agent nodes must be done using kubernetes-native mechanisms such as Daemon Sets.
I'm going to make an solution offer to Azure Marketplace using this documents.
https://learn.microsoft.com/en-us/azure/marketplace/partner-center-portal/create-new-azure-apps-offer
I have already created the ARM templates. But inside my templates, they're referring to custom images in Share Images Gallery that I captured from Azure Portal. Do i need to create VMs offer for my Images separately?
If yes, Let say if I have 10 templates that are referring to 20 custom images in Share Images Gallery, then I need to create 20 VM offers for 20 custom images, don't I?
Sorry I very new on Azure Cloud.
Without knowing too much detail here is an answer based on some assumptions:
The Azure App is meant for public customers
The Azure App (ARM Template) is going to deploy Virtual Machine images
If that’s the right set of assumptions then: Using Shared Image Gallery would turn out to be a costly exercise because it would be someone’s Storage account. The preferred solution would be
Create a Azure Virtual Machine offer
Each offer can have many plans (selling motion)
Each plan can have up to 40 versions of an image
Keep the VM offer hidden, so users don’t buy that, instead publish an Azure App that manages the deployments
Another way to think about it is, if you have same product with many versions, you create a single plan
If you have same product with different tiers you can create different plans (and code your VM to query azure to find out what context (plan) its running in and behave appropriately)
You want to keep the same “offering” under the same “offer”. For example all Windows Server images are under the same offer (with plans for each major release)
If you are offering a banana in one image and a potato in the other, you want to put them in different offers.
We want to use the key vault extension to handle getting new certs onto our batch nodes. I was wondering if what's the preferred way to install Azure VM extensions onto azure batch nodes since the machines don't appear as regular VMs in the azure portal or cli that we can interact with. It seems using the key vault extension seems like the best way to update certificates on a machine? We are currently using an base image for the batch nodes but I read that it's a bad practice to have extensions pre-installed on base images?
What's the best way to go about this?
Thanks!
2021-02-17 Updated Answer:
Select VM extensions are now supported on Batch pools. For KeyVaultExtension, you will need Managed Identity support on Batch pools which is now in public preview in select regions. Please see this doc for extensions on Batch pools and this doc for managed identity.
Original Answer:
Currently, arbitrary VM extensions are not supported on Azure Batch Pools (thus compute nodes). As of now, to update certificates on Batch pool compute nodes, you will need to add a new certificate on the account, patch the Batch pool with the new certificate reference, and then reboot the corresponding active compute nodes.
For your specific ask of installing Key Vault based certificates, this has been requested in UserVoice.
I have created an Azure Batch pool that is configured to allocate VMs from a custom VM image that I have created.
The custom VM image is based on "PyTorch from NVIDIA" which requires an image plan.
When the pool tries to allocate nodes it fails on the following error:
Deployment allocation failed due to missing image plan information in the request. You need to create custom image based on first party image which does not require image plan.
It seems that it's possible to create Virtual Machine Scale Sets with an image plan as indicated by this link. Is it possible to create an Azure Batch pool with a custom VM image that requires an image plan? The error suggests that it's not possible but that seems very limiting and in contrast to VMSS in which it does seem possible.
You can only use a first-party Azure Marketplace image supported by Batch as the base image for your managed image, refer to this.
If your custom VM image is generated from an Azure Marketplace image, you can select the correct Publisher/Offer/Sku for your custom image when you create a pool from a custom image in the portal.
Edit
I also can produce this issue when I use a generalized PyTorch from NVIDIA VM as the base image. As the document stated, it's impossible to use a third-party image as the base image to create a pool at the moment.
Some VM images in the Azure Marketplace have an additional license and purchase terms that you must accept and provide purchase plan parameters before you can deploy them programmatically.
You can't use a third-party image that has additional license and
purchase terms as your base image. For information about these
Marketplace images, see the guidance for Linux or Windows VMs.
I have a very special requirement which is:
Two web roles accessing a local shared file location.
I am aware of the "Local Storage" role settings, but those are only accessible within each role scope.
Does anyone know another option to accomplish this?
------- EDIT --------
As suggested I will explain more clearly what I'm trying to achieve here.
I'm implementing Only Office which is a web editor for office files. Their product requires to have a file saved on the file system to be opened on the editor.
I don't want to mix their ASP.NET MVC open source project with my code, so that's why I want to deploy their website as a separate webrole.
-------- END EDIT ------------
Thanks
In your question, you state that (my emphasis):
I'm implementing Only Office which is a web editor for office files. Their product requires to have a file saved on the file system to be opened on the editor.
If Only Office's requirement is to have temporary file storage that is used while the document is being edited, you may be able to get away with this in a Cloud Service Web Role. This is assuming that your users wouldn't be too angry if the temp. working document was 'lost' during a role re-start.
Web (and Worker) Roles are non-durable and the Azure Service Fabric might bounce them if they need to patch the underlying host or they might just crash due to a fault (which is usually why you deploy them in pairs - fault-tolerance etc.) If you save something to the file system on a Web Role, you are not guaranteed that it will be there if the role is bounced.
If however you need durability, you will need to implement something based around Azure Blob Storage and possibly something based on Blob Leases. However I imagine that Only Office doesn't have an implementation for Azure....
Failing that, you could try running on Azure Web App Service, however I imagine you would have the same issue re. backing storage and would need to implement something on Blob Storage.
So, finally, if you want complete control and something akin to running on-premise, take a look at using an IaaS Virtual Machine where you have all of the file system to play-with as you please.
==UPDATE==
Taking a look at the Only Office website, there is a SaaS offering Only Office SaaS Hosting which is probably cheaper to run for a year than the time taken for me to write this answer!
Failing that, if you look at the requirements for Only Office Document Server there is no way you're going to run that on a Web Role. Go Azure IaaS VM's.
You basically have 2 options here, both mentioned in the comments. You can use BLOB storage, or you can use an SMB share using Azure Files, which I believe is in preview still. We have used Azure files to mount an SMB share on several linux boxes. One thing we have noticed is that it is not particularly fast. It is also built on top of blob storage. Here is a link to Azure Files https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/.
If you choose to use blob storage and you will need to consider concurrency.
https://azure.microsoft.com/en-us/blog/managing-concurrency-in-microsoft-azure-storage-2/
I would suggest to use Azure File Services, you could have a share like URI to be used.
take a look at this:
https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-files/