As described in https://learn.microsoft.com/en-us/azure/templates/microsoft.machinelearningservices/workspaces/computes?tabs=bicep there are the properties location, sku, tags and identity.
For me it is not clear whether these properties relate to the parent workspace or to the compute resource (e.g. as the there is also computeLocation or sku as far as I can see should have the same value as the workspace)...
It would be great when someone can clarify to which resource these properties and related values belong to (workspace vs. compute resource).
EDIT:
Also: which properties are actually required for attach versus create? E.g. do I need identity or computeLocation for attach, and if yes what is the purpose of it as the compute resource is being created in another context?
I also figured out that location as well as disableLocalAuth are required for the attach operation - why when the resource is being deployed in another context and only attached?
And why do I get unsupported compute type when checking for the compute resources via Azure CLI for the attached AKS?
{
"description": "Default AKS Cluster",
"id": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.MachineLearningServices/workspaces/xxx/computes/DefaultAKS",
"location": "westeurope",
"name": "DefaultAKS",
"provisioning_state": "Succeeded",
"resourceGroup": "xxx",
"resource_id": "/subscriptions/xxx/resourcegroups/xxx/providers/Microsoft.ContainerService/managedClusters/xxx",
"type": "*** unsupported compute type ***"
}
EDIT-2:
So based on the response from #SairamTadepalli-MT all the properties actually relate to the compute resource - what makes sense. Still, I don't understand the purpose of a few of these properties. For instance why is there a "location" and a "computeLocation" or what is the meaning of "sku" (e.g. I tried "AmlCompute" and provided the value "Basic" - but "Basic" is the "sku" of the workspace and for "AmlCompute" the size is actually defined by the "vmSize" or?...).
What brings me to the next point: the current documentation currently lacks a detailed description in which scenarios which properties can have which values respectively need to be provided (beside "properties").
This is also true for attach (i.e. providing a "resourceId") vs. create (i.e. providing "properties"): which properties are actually required for attach? For what I figured out it requires "location" and "disableLocalAuth" - why do I need these properties as I would assume "name" and "resourceId" (and maybe "computeType") should be sufficient to attach a compute resource? What is the purpose of properties like "sku", "tags" or "identity" when I attach an existing compute resource?
Finally regarding "unsupported compute type": not sure if your response really helps me. The AKS is successfully attached, so I don't understand why I get "unsupported compute type". This should be fixed.
Before identifying the properties and their uses, we need to understand a small difference between workspaces and computer resources
Workspaces: The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. The workspace keeps a history of all training runs, including logs, metrics, output, and a snapshot of your scripts. You use this information to determine which training run produces the best model.
https://learn.microsoft.com/en-us/azure/machine-learning/concept-workspace
on the other hand let's check about computer resources
Compute Resources: An Azure Machine Learning compute instance is a managed cloud-based workstation for data scientists. Compute instances make it easy to get started with Azure Machine Learning development as well as provide management and enterprise readiness capabilities for IT administrators.
https://learn.microsoft.com/en-us/azure/machine-learning/concept-compute-instance
Workspaces are used to work on machine learning resources and identify all the operations which are completed and in progress or even failed as a log.
Compute resources are the kind of administration operations which can handle the authority on the resource which was shared and created.
Moving on to the next part of the question. Regarding the properties.
Location: This is used to identify where the compute resources are created for ML operations. Comes under compute resources.
SKU: This is regarding the tier and price of each tier which we have chosen in our subscription. This is related to the computer resources.
Tags: Each resource created need to be recognized. Tags are compute resource identifiers.
Identity: Subscription details of the compute resource. This can be further need to be used with workspace which deploying ML application.
Location:: The location where the parent code is located. Where the output is stored will be defined
disableLocalAuth: Azure Automation provides Microsoft Azure Active Directory (Azure AD) authentication support for all Automation service public endpoints. This critical security enhancement removes certificate dependencies and gives organizations control to disable local authentication methods. This feature provides you with seamless integration when centralized control and management of identities and resource credentials through Azure AD is required.
unsupported compute type: Azure Kubernetes Service (AKS) clusters, you might occasionally come across problems. Refer the link mentioned below for better trouble shooting
https://learn.microsoft.com/en-us/azure/aks/troubleshooting
Related
I am using below link to create/assign Standard Azure policies for my Management Groups. The problem is my org. has already created Management Groups and Subscriptions manually using Azure Portal. Can i still create / apply policies using ESLZ TF code and apply to these manually created Management groups using TF code:
https://github.com/Azure/terraform-azurerm-caf-enterprise-scale
When i see the code archetype (policy) is very tightly coupled to MG creation ?
locals.management_groups.tf:
"${local.root_id}-landing-zones" = {
archetype_id = "es_landing_zones"
parameters = local.empty_map
access_control = local.empty_map
}
archetype_id is the policy.
The enterprise scale example follows what they call a "supermodule approach".
The example from locals.management_groups.tf that you posted there is just the configuration layer of the module. If you want CAF to adopt an existing management group instead of creating a new one, you have to terraform import it. That requires you to know the resource address, which you can derive from the the actual resource definition.
So you're looking for resource "azurerm_management_group" and that's in resources.management_groups.tf. That file declares multiple management group resources level_1 to level_6, and they're using for_each to generate the resources from the module's configuration model.
So after you've somehow traced your way through the modules' configuration model and determined where you want your existing management group to end up in the CAFs model, you can run an import as described in terraform's docs and the azurerm provider's docs like this:
terraform import azurerm_management_group.level_3["my-root-demo-corp"] providers/Microsoft.Management/managementGroups/group1
However, even though all of that is technically possible, I'm not sure this is a good approach. The enterprise scale supermodule configuration model is super idiosyncratic to trace through - it's stated goal is to replace "infrastructure as code" with "infrastructure as data". You may find one of the approaches described in Azure Landing Zone for existing Azure infrastructure more practical for your purposes.
You could import the deployed environment into the TF state and proceed from there.
However, do you see an absolute necessity ? IMO, setting up the LZ is a one-time activity. However, one can argue that ensuring compliance across active & DR LZs is a good use-case for a TF based LZ deployment.
This is probably an elemental question, but why would you want to ensure that resources are deployed to the same locations as their resource groups? I'm getting out of compliance issues with HIPPA and from CloudSploit about this and I'd like to get some more details around it as I know it deals with meta data but I'm not crystal clear.
Thanks in advance
Resources within a resource group need not always be deployed in the same location as the resource group. But there is a built-in Azure Policy available to audit this condition that one may choose to enable.
The resource group location serves two purposes:
First: The resource group stores metadata about the resources. When you specify a location for the resource group, you're specifying where that metadata is stored. Now, for compliance reasons, you may need to ensure that your data is stored within a particular region.
Second: If the resource group's region is temporarily unavailable, you can't update resources in the resource group because the metadata is unavailable. The resources in other regions will still function as expected, but you can't update them.
For knowing more about the factors to consider while defining your resrouce groups, check the Azure Resource Manager documentation.
We manage an Azure subscription operated by several countries. Each of them is quite independant about they can do (create/edit/remove resources). A guide of good practices has been sent to them, but we (security team) would like to ensure a set of NSG is systematically applied for every new subnet/vnet created.
Giving a look to Azure Triggers, I am not sure that subnet creation belongs to the auditable events. I also was told to give a look to Azure policy, but once again I am not sure this will match our expectations which are : For every new vnet/subnet, automatically apply a set of predefined NSG.
Do you have any idea about a solution for our need ?
I have done work like this in the past (not this exact issue) and the way I solved it was with an Azure Function that walked the subscription and looked for these kinds of issues. You could have the code run as a Managed Identity with Reader rights on the subscription to report issues, or as a Contributor to update the setting. Here's some code that shows how you could do this with PowerShell https://github.com/Azure/azure-policy/tree/master/samples/Network/enforce-nsg-on-subnet
You could consider using a Policy that has a DeployIfNotExists Action, to deploy an ARM template that contains all the data for the NSG. https://learn.microsoft.com/en-us/azure/governance/policy/samples/pattern-deploy-resources
You can get the ARM template by creating the NSG and getting the template:
GettingNSGTemplate
Note also that creating a subnet is audited, you can see it in the Activity Log for the VNet. See the screen shot.
AddingASubnet
I'm trying to build a small program to change the autoscale settings for our Azure WebApps, using the Microsoft.WindowsAzure.Management.Monitoring and Microsoft.WindowsAzure.Management.WebSites NuGet packages.
I have been roughly following the guide here.
However, we are interested in scaling WebApps / App Services rather than Cloud Services, so I am trying to use the same code to read the autoscale settings but providing a resource ID for our WebApp. I have already got the credentials required for making a connection (using a browser window popup for Active Directory authentication, but I understand we can use X.509 management certificates for non-interactive programs).
This is the request I'm trying to make. Credentials already established, and an exception is thrown earlier if they're not valid.
AutoscaleClient autoscaleClient = new AutoscaleClient(credentials);
var resourceId = AutoscaleResourceIdBuilder.BuildWebSiteResourceId(webspaceName: WebSpaceNames.NorthEuropeWebSpace, serverFarmName: "Default2");
AutoscaleSettingGetResponse get = autoscaleClient.Settings.Get(resourceId); // exception here
The WebApp (let's call it "MyWebApp") is part of an App Service Plan called "Default2" (Standard: 1 small), in a Resource Group called "WebDevResources", in the North Europe region. I expect that my problem is that I am using the wrong names to build the resourceId in the code - the naming conventions in the library don't map well onto what I can see in the Azure Portal.
I'm assuming that BuildWebSiteResourceId is the correct method to call, see MSDN documentation here.
However the two parameters it takes are webspaceName and serverFarmName, neither of which match anything in the Azure portal (or Google). I found another example which seemed to be using the WebApp's geo region for webSpaceName, so I've used the predefined value for North Europe where our app is hosted.
While trying to find the correct value for serverFarmName in the Azure Portal, I found the Resource ID for the App Service Plan, which looks like this:
/subscriptions/{subscription-guid}/resourceGroups/WebDevResources/providers/Microsoft.Web/serverfarms/Default2
That resource ID isn't valid for the call I'm trying to make, but it does support the idea that a 'serverfarm' is the same as an App Service Plan.
When I run the code, regardless of whether the resourceId parameters seem to be correct or garbage, I get this error response:
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/">
{"Code":"SettingNotFound","Message":"Could not find the autoscale settings."}
</string>
So, how can I construct the correct resource ID for my WebApp or App Service Plan? Or alternatively, is there a different tree I should be barking up to programatially manage WebApp scaling?
Update:
The solution below got the info I wanted. I also found the Azure resource explorer at resources.azure.com extremely useful to browse existing resources and find the correct names. For example, the name for my autoscale settings is actually "Default2-WebDevResources", i.e. "{AppServicePlan}-{ResourceGroup}" which I wouldn't have expected.
There is a preview service https://resources.azure.com/ where you can inspect all your resources easily. If you search for autoscale in the UI you will easily find the settings for your resource. It will also show you how to call the relevant REST Api endpoint to read or update that resorce.
It's a great tool for revealing a lot of details for your deployed resources and it will actually give you an ARM template stub for the resource you are looking at.
And to answer your question, you could programmatically call the REST API from a client with updated settings for autoscale. The REST API is one way of doing this, the SDK another and PowerShell a third.
The guide which you're following is based on the Azure Service Management model, aka Classic mode, which is deprecated and only exists mainly for backward compatibility support.
You should use the latest
Microsoft.Azure.Insights nuget package for getting the autoscale settings.
Sample code using the nuget above is as below:
using Microsoft.Azure.Management.Insights;
using Microsoft.Rest;
//... Get necessary values for the required parameters
var client = new InsightsManagementClient(new TokenCredentials(token));
client.AutoscaleSettings.Get(resourceGroupName, autoScaleSettingName);
Besides, the autoscalesettings is a resource under the "Microsoft.Insights" provider and not under the "Microsoft.Web" provider, which explains why you are not able to find it with your serverfarm resourceId.
See the REST API Reference below for getting the autoscale settings.
GET
https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.insights/autoscaleSettings/{autoscale-setting-name}?api-version={api-version}
I am trying to use Azure resource manager and Azure Compute Management APIs to get list of virtual machines in a resource group and all of virtual machine properties. It's failing with InvalidAPIVersion error even though I have updated to latest version. As per the error it expects older version no but i don't see any old library with such a version. Can someone please suggest what am I missing?
VirtualMachineGetResponse vm = m_computeClient.VirtualMachines.Get("/subscriptions/1f94c869-####-####-####-055e8ae15be3/resourceGroups/TestGroup", "TestMachine");
m_computeClint is an object of ComputeManagementClient class in name space Microsoft.Azure.Management.Compute
Is this because resource provider is Microsoft.ClassicCompute?
Error message:
InvalidApiVersionParameter: The api-version '2015-05-01-preview' is invalid. The supported versions are '2015-01-01,2014-04-01-preview,2014-04-01,2014-01-01,2013-03-01,2014-02-26,2014-04'.
If you're using the resource group management api, you're definitely going to get an invalidapiversionparameter using ClassicCompute. The resource management compute provider is "Microsoft.Compute"; ClassicCompute exists only to view VMs that were already created using the service management API.
Right now, you should keep the two APIs and resources separate. Did you create the VM using the preview portal (portal.azure.com) or using PowerShell? If the former, it has almost (but not quite) converted over to using the resource management api. Try following these steps: https://azure.microsoft.com/en-us/documentation/articles/resource-group-template-deploy/.
Hope that helps.