I understand API is in preview but maybe someone can help me out here .
I am trying to create Data container as its described in AML REST API
curl --location --request PUT 'https://management.azure.com/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroupName}}/providers/Microsoft.MachineLearningServices/workspaces/{{workspaceName}}/data/abc?api-version=2021-03-01-preview' \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
--data-raw '{"properties": { "description": "string",
"tags": { },
"properties": {}
}
}'
I receive 400 status code with message
Error setting value to 'Description' on 'Microsoft.MachineLearning.ManagementFrontEnd.Contracts.V20210301Preview.Assets.DataContainer'."
after removing Description which should be optional
DataContainers_CreateOrUpdate is not supported
How can I create Dataset with local files? I cannot use UI. There it works and I can receive it with GET list method.
If you want to create a dataset from local file, you can use az ml cli. Use following command from a powershell:
az ml dataset create --file data.yaml --resource-group "your-resource-group" --workspace-name "your-ws-name"
data.yaml
$schema: https://azuremlschemas.azureedge.net/latest/dataset.schema.json
name: local-file-example
description: Dataset created from local file.
local_path: data/titanic.csv
See az yaml schema and az dataset cli.
It seems like API version 2021-03-01-preview is not working properly for dataset operations. By reverse-engineering the az CLI command mentioned by qiax using the --debug option I could see that the endpoint used by the CLI is the following:
https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/datasets/{dataset-name}/versions/1/?api-version=2021-10-01
Here's an example of a valid request body for that endpoint (it seems like it's the same as the Create Or Update Data Version endpoint shown in the documentation):
{
"properties": {
"isAnonymous": false,
"paths": [
{
"file": "foo/"
}
],
"properties": {},
"tags": {}
}
Related
Quarkus Azure function apps currently rely on the Maven azure-functions-maven-plugin plugin to perform a deployment of the function. However, this means that I need to package the application source code and rebuild it with each deployment. This is not ideal, as I really want an immutable package that I can deploy, promote, and roll back without rebuilding.
Is there any way to deploy a prepackaged Quarkus app without the Maven plugin?
The reality is that Microsoft has done a poor job of supporting Java developers deploying to Azure functions. The requirement to recompile the app with each deployment, which is the only option available to you when using the Maven plugin, is an anti-pattern for repeatable and reliable deployments.
Digging through the source code of the Maven plugin eventually leads you to the RunFromBlobFunctionDeployHandler class, which uploads a ZIP file as a blob, creates a long lived SAS token, and then sets the WEBSITE_RUN_FROM_PACKAGE setting to the SAS URL.
To recreate this process, we first need a function.json file, which will redirect all requests to the Quarkus io.quarkus.azure.functions.resteasy.runtime.Function.run class. You can get this class by generating a sample Quarkus project, as documented here:
{
"scriptFile" : "../products-microservice-runner.jar",
"entryPoint" : "io.quarkus.azure.functions.resteasy.runtime.Function.run",
"bindings" : [ {
"type" : "httpTrigger",
"direction" : "in",
"name" : "req",
"route" : "{*path}",
"methods" : [ "GET", "POST", "HEAD", "PUT", "OPTIONS", "DELETE" ],
"dataType" : "binary",
"authLevel" : "ANONYMOUS"
}, {
"type" : "http",
"direction" : "out",
"name" : "$return"
} ]
}
You then need a host.json file, again generated by the sample Quarkus project:
{
"version": "2.0"
}
Once we build your Quarkus Azure function app, you'll have a self contained JAR file. In my case it was called products-microservice-runner.jar. The next step is to recreate the directory structure documented here:
rm -rf /tmp/octopubproductservice
mkdir /tmp/octopubproductservice
mkdir /tmp/octopubproductservice/octopubproductservice
cp target/products-microservice-runner.jar /tmp/octopubproductservice
cp azure-config/host.json /tmp/octopubproductservice
cp azure-config/function.json /tmp/octopubproductservice/octopubproductservice
pushd /tmp/octopubproductservice
zip -r $ZIP_FILE .
popd
This produces a ZIP file like this:
Now create the resource group, storage account, and function as documented here:
REGION=australiaeast
RESOURCE_GROUP=octopubproductservice
FUNCTION_NAME=octopubproductservice
STORAGE_ACCOUNT=octopubproductservice
STORAGE_SKU="Standard_LRS"
ZIP_FILE=product-service-azure.zip
CURRENT_DATE=$(date +%Y%m%d)
SAS_EXPIRY=$(date -d "$CURRENT_DATE +10 years" +%Y-%m-%d)
# Create a resource group
az group create --location $REGION --name $RESOURCE_GROUP
# Create a storage account
az storage account create --name $STORAGE_ACCOUNT --resource-group $RESOURCE_GROUP --sku $STORAGE_SKU
# Create a function app
az functionapp create \
--name $FUNCTION_NAME \
--resource-group $RESOURCE_GROUP \
--storage-account $STORAGE_ACCOUNT \
--consumption-plan-location $REGION \
--functions-version 4 \
--os-type linux \
--runtime java \
--runtime-version 11.0
Next upload the function package and generate an SAS token:
# Upload the function package
az storage blob upload \
--account-name $STORAGE_ACCOUNT \
--container-name java-functions-run-from-packages \
--name product-service-azure.zip \
--file /tmp/octopubproductservice/product-service-azure.zip \
--overwrite \
--auth-mode key
# Create a SAS key for the function package
URL=$(az storage blob generate-sas \
--account-name $STORAGE_ACCOUNT \
--container-name java-functions-run-from-packages \
--name product-service-azure.zip \
--permissions r \
--expiry $SAS_EXPIRY \
--auth-mode key \
--full-uri)
# The URL is quoted. We treat this as a JSON string, and use jq to return the raw string
FIXED_URL=$(echo $URL | jq -r '.')
Finally, set the WEBSITE_RUN_FROM_PACKAGE setting to the SAS URL, which is required when deploying Linux functions:
# The raw string is set as the WEBSITE_RUN_FROM_PACKAGE value, which indicates Azure
# must download the function from the URL.
az functionapp config appsettings set \
--name $FUNCTION_NAME \
--resource-group $RESOURCE_GROUP \
--settings "WEBSITE_RUN_FROM_PACKAGE=$FIXED_URL"
At that point your Quarkus app should be working.
I am trying to get the AML token using CLI.
I am able to get the token using the command -> token=$(az account get-access-token --subscription {subscri ID} --resource-type arm --query accessToken --output tsv)
but when I use this token to get the AMLToken I get below error, however it is working fine if I make this query using postman :
curl -d POST --header "Authorization: Bearer $token" "https://management.azure.com/subscriptions/{subcri id}/resourceGroups/{res_grup}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/onlineEndpoints/{endpoint}/token?api-version=2022-05-01"
the error which I get is below :
{
"error": {
"code": "UnsupportedApiVersion",
"message": "The HTTP resource that matches the request URI 'https://cert-eastus2.experiments.azureml.net/mferp/managementfrontend/subscriptions/{sub_id}/resourceGroups/{r_group}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/onlineEndpoints/{endpoint}/token' does not support the API version '2022-05-01'.",
"innerError": null
}
}
any helps or pointers please, why I am getting this error? Not able to find any documentation for it.
I tried to reproduce the same in my environment and got the below results
I ran the same commands as you and got the same error as below:
token=$(az account get-access-token --subscription subscriptionID --resource-type arm --query accessToken --output tsv)
url="https://management.azure.com/subscriptions/subscriptionID/resourceGroups/rgname/providers/Microsoft.MachineLearningServices/workspaces/workspace_name/onlineEndpoints/endpoint_name/token?api-version=2022-05-01"
curl -d POST --header "Authorization: Bearer $token" $url
Response:
When I did the same via Postman, I got the AML token successfully like below:
Alternatively, make use of below Azure CLI command to get AML token like below:
az ml online-endpoint get-credentials --name <endpoint_name> --resource-group <rg_name> --workspace-name <workspace_name>
I ran the same command and got AML token successfully like below:
When I decoded the token in jwt.ms, I got the claims same as token from Postman like below:
I am attempting to setup and install this reference app from Azure, created by the Microsoft patterns & practices team: https://github.com/mspnp/serverless-reference-implementation - the setup is via Azure CLI not the UI.
However it uses Azure AD Graph, which has just very recently been deprecated and replaced by Microsoft.Graph as documented here: https://learn.microsoft.com/en-gb/cli/azure/microsoft-graph-migration
This breaks the installation instructions in the app, specifically the az ad app create command:
export API_APP_ID=$(az ad app create --display-name $API_APP_NAME --oauth2-allow-implicit-flow true \
--native-app false --reply-urls http://localhost --identifier-uris "http://$API_APP_NAME" \
--app-roles ' [ { "allowedMemberTypes": [ "User" ], "description":"Access to device status", "displayName":"Get Device Status", "isEnabled":true, "value":"GetStatus" }]' \
--required-resource-accesses ' [ { "resourceAppId": "00000003-0000-0000-c000-000000000000", "resourceAccess": [ { "id": "e1fe6dd8-ba31-4d61-89e7-88639da4683d", "type": "Scope" } ] }]' \
--query appId --output tsv)
Fails with: ERROR: unrecognized arguments: --native-app false
If I try to update the command with the new parameters for Microsoft.Graph as per the migration guide above:
export API_APP_ID=$(az ad app create --display-name $API_APP_NAME --enable-access-token-issuance true \
--is-fallback-public-client false --web-redirect-uris http://localhost --identifier-uris "http://$TENANT_NAME.onmicrosoft.com/$API_APP_NAME" \
--app-roles ' [ { "allowedMemberTypes": [ "User" ], "description":"Access to device status", "displayName":"Get Device Status", "isEnabled":true, "value":"GetStatus" }]' \
--required-resource-accesses ' [ { "resourceAppId": "00000003-0000-0000-c000-000000000000", "resourceAccess": [ { "id": "e1fe6dd8-ba31-4d61-89e7-88639da4683d", "type": "Scope" } ] }]' \
--query appId --output tsv)
The create command works - but the app's manifest does not have the expected data:
export API_IMPERSONATION_PERMISSION=$(az ad app show --id $API_APP_ID --query "oauth2Permissions[?value == 'user_impersonation'].id" --output tsv)
This returns NULL - the new Microsoft.Graph API does not even create this field in the app's manifest. I manually confirmed this by looking at the manifest in the Azure UI. This means the following commands that rely on this field existing are unusable.
How can I fix this so I can install this reference app given that the install instructions refer to the no longer available Azure AD Graph API? What is the correct az ad app create command that lets Microsoft.Graph API create the API app with oauth?
I am trying to update my APIM with a new hostname using the Azure CLI.
When doing this
$hostConfiguration = '[{
"hostName": "myhostname.com",
"type": "Proxy",
"keyVaultId": "https://mykeyvault.vault.azure.net/secrets/mysecret"
}]'
az apim update --name "apim-name" --resource-group "rg-name" --set hostnameConfigurations=$hostConfiguration
I get this :
Unable to build a model: Cannot deserialize as [HostnameConfiguration] an object of type <class 'str'>, DeserializationError: Cannot deserialize as [HostnameConfiguration] an object of type <class'str'>
What is wrong with the way I do it?
You will have to convert the json to string before storing in a variable.
I made use of https://tools.knowledgewalls.com/jsontostring to convert your json to string
Updated Snippet
$hostConfiguration ='[{\"hostName\":\"myhostname.com\",\"type\":\"Proxy\",\"keyVaultId\":\"https:\/\/mykeyvault.vault.azure.net\/secrets\/mysecret\"}]'
az apim update --name "apim-name" --resource-group "rg-name" --set hostnameConfigurations=$hostConfiguration
I have an resource group ARM template that I use to create an application gateway that is configured for url routing. It sends traffic to different Web Apps in that resource group based on url path rules. I deploy the base resource group ARM template, and then each web app has its own ARM template that setups a Web App on an App Service Plan. I am trying to figure out how to add a rule to an existing Url Path Map on an Application Gateway without defining the whole application gateway in every template. This way, I can simply add web apps and have them "register" on to the application gateway with a particular path rule.
I considered doing a linked template where my base template would have all of the shared resources (databases, app service plan, and app gateway), but even with a linked template, I don't think I can add a rule to the application gateway.
Update
So I modified my template by adding a reference to the existing application gateway, and then adding variables for the new BackEndPoolAddress and new Path Rule. It ends up like this (abbreviated to only relevant parts):
"variables": {
"appGateway": "[reference(concat('Microsoft.Network/applicationGateways/', 'appGateWay-', uniqueString(resourceGroup().id)), '2017-06-01')]",
"pathRule": {
"name": "[concat(parameters('websiteName'), '- RoutingRule')]",
"properties": {
"paths": [
"[parameters('routingRule')]"
],
"backendAddressPool": {
"id": "[concat(variables('appGateway').id, '/backendAddressPools/',parameters('websiteName'), 'BackEndPool')]"
},
"backendHttpSettings": {
"id": "[variables('appGateway').backendHttpSettingsCollection[0]]"
}
}
},
"backendPool": {
"name": "[concat(parameters('websiteName'), 'BackEndPool')]",
"properties": {
"IpAddress": "[reference(variables('webSiteName')).defaultHostName]"
}
}
},
"resources": [
...
{
"apiVersion": "2017-06-01",
"name": "[variables('appGateway').name]",
"type": "Microsoft.Network/applicationGateways",
"location": "[resourceGroup().location]",
"properties": {
"backendAddressPools": "[concat(variables('appGateway').backendAddressPools, variables('backendPool'))]",
"urlPathMaps": [
{
"name": "[variables('appGateway').urlPathMaps[0]]",
"pathRules": "[concat(variables('appGateway').urlPathMaps[0].pathRules, variables('pathRule'))]"
}
]
}
}
],
However I get a template validation error saying I can't use the Reference function in the Variables section. If I don't add it in the variables section, how can I build the correct paths for the pool and pathRule in my variables section?
you can achieve this using the reference() function, array manipulation and nested templates (might work even without those, worst case you will need them). example:
"outputs": {
"httpListeners": {
"type": "array",
"value": "[reference('application_gateway_id', '2018-08-01', 'Full').properties.httpListeners]"
}
}
Will return you array or httpListeners. you can get all the relevant application gateway properties and add new (additional) properties with the concat() and assign the result to the property (properties):
"httpListeners": "[concat(reference('application_gateway_id', '2018-08-01', 'Full').properties.httpListeners, variables('newListener'))]"
you just need to make sure 2 deployments dont start at the same time, one might overwrite the other
Here is the solution I finally ended up with using the Azure CLI. This script is idempotent and runs during my release process.
echo "Logging into AKS Cluster"
az aks get-credentials --resource-group $RESOURCEGROUP_NAME --name $AKSNAME
echo "Get the created service's ip address"
SERVICEIP=$(kubectl get service --namespace $AKSNAMESPACE $APPNAME-service -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
echo "Creating backend pool - IP $SERVICEIP"
az network application-gateway address-pool create \
--gateway-name $APPGATEWAYNAME \
--resource-group $RESOURCEGROUP_NAME \
--name "$APPNAME-pool" \
--servers $SERVICEIP
echo "Creating probe"
az network application-gateway probe create \
--gateway-name $APPGATEWAYNAME \
--name "$APPNAME-probe" \
--path $APPPROBE \
--resource-group $RESOURCEGROUP_NAME \
--protocol Http \
--resource-group $RESOURCEGROUP_NAME \
--host-name-from-http-settings true
echo "Creating HTTP Settings"
az network application-gateway http-settings create \
--gateway-name $APPGATEWAYNAME \
--name "$APPNAME-settings" \
--port 80 \
--resource-group $RESOURCEGROUP_NAME \
--host-name-from-backend-pool \
--probe "$APPNAME-probe" \
--protocol Http
echo "Creating URL Path Map"
az network application-gateway url-path-map rule create \
--gateway-name $APPGATEWAYNAME \
--name "$APPNAME-rule" \
--paths $RULEPATH \
--path-map-name $RULENAME \
--resource-group $RESOURCEGROUP_NAME \
--http-settings "$APPNAME-settings" \
--address-pool "$APPNAME-pool"