How to include a health-check endpoint in a Azure Machine Learning Model deployed as an AKS Webservice? - azure-machine-learning-service

When we deploy a model using Azure Machine Learning Service, we need a scoring script with at least two functions init which is run once when a model is first deployed and a run function which is run against any data we want to be scored.
The endpoint for this is: http://<IP-Address>/api/v1/service/<Webservice-Name>/score
Is there a way to include a health-check function in the deployed web service? Maybe adding another function healthcheck() and a corresponding endpoint http://<IP-Address>/api/v1/service/<Webservice-Name>/health which when pinged sends us a custom JSON containing information about the health of our service. This may even include information about the Azure Health Check too:

Apparently, the Azure Machine Learning SDK has to show the properties of the class Webservice. I haven't found any mention to this issue online.
To reproduce fully the experiment, you have to install AML SDK:
pip install azureml-core
And get the workspace:
# get the workspace
subscription_id = '<your_subscription_id>'
resource_group = '<your_resource_group>'
workspace_name = '<your_workspace_name>'
ws = Workspace(subscription_id, resource_group, workspace_name)
Using the Webservice Class, it should work. I have 3 services, but I can't see the status:
# this should work, but it isn't returning the state
for service in Webservice.list(workspace= ws):
print('Service Name: {} - {}'.format(service.name, service.state))
>> Service Name: akstest - None
>> Service Name: regression-model-1 - None
>> Service Name: xgboost-classification-1 - None
Using an intermediate step using the _get() method, it works:
# this works
for service in Webservice.list(workspace= ws):
service_properties = Webservice._get(workspace= ws, name = service.name)
print('Service Name: {} - {}'.format(service_properties['name'], service_properties['state']))
>> Service Name: akstest - Failed
>> Service Name: regression-model-1 - Healthy
>> Service Name: xgboost-classification-1 - Healthy

Related

How to enable GCP service agent account via Terraform?

I understand there is a difference between a service account and a service agent for different services such as composer.
How do you enable a service agent via terraform?
What I'm trying to do is this :
# TODO : Maybe enable this service agent somehow via gcloud? It got enabled when trying to manually create the composer env from the console
# Step 4 (Src2) - Host project GKE service account: service-<some-numeric-id>#container-engine-robot.iam.gserviceaccount.com
# Need 'container.serviceAgent' in the host project
resource "google_project_iam_member" "dev-omni-orch-gke-project-lvl-roles-t2" {
provider = google.as_super_admin
for_each = toset([
"roles/container.serviceAgent",
])
role = each.value
member = "serviceAccount:service-<some-numeric-id>#container-engine-robot.iam.gserviceaccount.com"
# member = "serviceAccount:service-${google_project.main-shared-vpc-host.number}#container-engine-robot.iam.gserviceaccount.com"
# project = google_project.dev-main-code-base.project_id
project = google_project.main-shared-vpc-host.project_id
}
I get
Request `Create IAM Members roles/container.serviceAgent serviceAccount:service-<some-numeric-id>#container-engine-robot.iam.gserviceaccount.com for project "<shared-vpc-host-project-id>"` returned error: Batch request and retried single request "Create IAM Members roles/container.serviceAgent serviceAccount:service-<some-numeric-id>#container-engine-robot.iam.gserviceaccount.com for project \"<shared-vpc-host-project-id>\"" both failed. Final error: Error applying IAM policy for project "<shared-vpc-host-project-id>": Error setting IAM policy for project "<shared-vpc-host-project-id>": googleapi: Error 400: Service account service-<some-numeric-id>#container-engine-robot.iam.gserviceaccount.com does not exist., badRequest
But when I try to do it via the console manually, there is a prompt that asks me if I want to enable this service agent, which I do, but I want to be able to do this on terraform.
The said prompt :
The service-[PROJECT_ID]#cloudcomposer-accounts.iam.gserviceaccount.com service agent will only exist after the Cloud Composer API has been enabled.
This can be done in Terraform using the google_project_service resource, for example:
resource "google_project_service" "project" {
project = "your-project-id"
service = "composer.googleapis.com"
}
Once the API has been enabled, the service agent should exist and you should be able to grant it the required permissions.

How to have a health check against an AML endpoint?

I have a running AML endpoint deployed on AKS cluster, looking like http://[IP address]:80/api/v1/service/modelscoring36/score.
This is the POST endpoint to respond any incoming model scoring request. Is there any way to check if the relevant AML service or attached AKS cluster is healthy?
In Asp.Net we is used to some dummy GET method constructed in one controller for this health check purpose.
Does AML service provide such health check approach? or any other suggestion?
Thank you.
To get the health check of the end points
install Azure ML SDK using Command Prompt
pip install azureml-core
Get complete workspace details like subscription ID and other details. The requirements are mentioned below:
subscription_id = '<your_subscription_id>'
resource_group = '<your_resource_group>'
workspace_name = '<your_workspace_name>'
ws = Workspace(subscription_id, resource_group, workspace_name)
Using intermediate _get() method, we can get the health check details like below
for service in Webservice.list(workspace= ws):
service_properties = Webservice._get(workspace= ws, name = service.name)
print('Service Name: {} - {}'.format(service_properties['name'], service_properties['state']))
The results look like below:
Service Name: Model Name - Failed
Service Name: Model Name - Healthy
Service Name: Model name - Healthy

Azure Bicep - Connect Azure API Management (API) to Azure Function App

I can see within the Azure Management Console, specifically within the Azure API Management Service, via the GUI you are able to use Azure Functions to form an API.
I am trying to implement the same via Azure Bicep, but I do not see any options in the Bicep documentation for API Management - API Service.
In the GUI, I see something like this:
This allows me to specify my Function App:
However, within the Bicep Documentation, I don't see anything where I would expect to: Microsoft.ApiManagement service/apis
I have instead tried using the Microsoft.ApiManagement service/backends but that doesn't give the same experience and I haven't managed to get that to work.
So my question is, how do I connect my Azure API Management service to an Azure Site (app) which is set as a suite of Azure Functions?
You need to create backend and all api definitions manually. The portal gives you a nice creator and does all those REST calls for you. With bicep (and ARM) which is operating directly on the REST endpoints of each resource provider you need to build own solution.
Perhaps there’re somewhere some existing templates that can do this but personally I didn’t see any yet.
I added OpenAPI specifications to my functionApps to produce the sawgger \ -openAPI link (or file). Then leveraged the OpenAPI file to build the APIs.
// Create APIM Service
resource apimServiceRes 'Microsoft.ApiManagement/service#2021-08-01' = {
name: 'apim service name'
location: resourceGroup().location
sku:{
capacity: 0
name: 'select a sku'
}
identity:{
type: 'SystemAssigned'
}
properties:{
publisherName: 'your info'
publisherEmail: 'your info'
}
}
// Create the API Operations with:
resource apimApisRes 'Microsoft.ApiManagement/service/apis#2021-08-01' = {
name: '${apimServiceRes.name}/name-to-represent-your-api-set'
properties: {
format: 'openapi-link'
value: 'https://link to your swagger file'
path: ''
}
}

Azure Connection String with Spring profile

I am new to Azure Spring Boot, I have done working simple POC with using spring boot and azure with using connection-string in environment property.( https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-java-spring-app )
But I have the following requirements:
1) No need to set connection-string in environment property, in this case directly I want those connection-string bootstrap.yaml file.
Ex:(It is not working)
spring:
application:
name: azurespringappconfig
cloud:
azure:
appconfiguration:
stores:
- connection-string: 'Endpoint=https://azurespringconfig:azconfig:xxxxxxxxxxx'
# cloud:
# azure:
# appconfiguration:
# stores:
# - connection-string: ${CONNECTIONSTRING} -- this is working using environment
2) I need to apply profile concept: Here different profiles have different connection-string, so which profile active so going pick up the that connection-string.
Is any possibility to solve both the problems.
Thanks,
Amar

How to use Datadog agent in Azure App Service?

I'm running web apps as Docker containers in Azure App Service. I'd like to add Datadog agent to each container to, e.g., read the log files in the background and post them to Datadog log management. This is what I have tried:
1) Installing Datadog agent as extension as described in this post. This option does not seem to be available for App Service apps, only on VMs.
2) Using multi-container apps as described in this post. However, we have not found a simple way to integrate this with Azure DevOps release pipelines. I guess it might be possible to create a custom deployment task wrapping Azure CLI commands?
3) Including Datadog agent into our Dockerfiles by following how Datadog Dockerfiles are built. The process seems quite complicated and add lots of extra dependencies to our Dockerfile. We'd also not like to inherit our Dockerfiles from Datadog Dockerfile with FROM datadog/agent.
I'd assume this must be a pretty standard problem for Azure+Datadog users. Any ideas what's the cleanest option?
I doubt the Datadog agent will ever work on App Services web app as you do not have access to the running host, it was designed for VMs.
Have you tried this https://www.datadoghq.com/blog/azure-monitoring-enhancements/ ? They say they support AppServices
I have written a app service extension for sending Datadog APM metrics with .NET core and provided instructions for how to set it up here: https://github.com/payscale/datadog-app-service-extension
Let me know if you have any questions or if this doesn't apply to your situation.
Logs from App Services can also be sent to Blob storage and forwarded from there via an Azure Function. Unlike traces and custom metrics from App Services, this does not require a VM running the agent. Docs and code for the Function are available here:
https://github.com/DataDog/datadog-serverless-functions/tree/master/azure/blobs_logs_monitoring
If you want to use DataDog for logging from Azure Function of App Service you can use Serilog and DataDog Sink to the log files:
services
.AddLogging(loggingBuilder =>
loggingBuilder.AddSerilog(
new LoggerConfiguration()
.WriteTo.DatadogLogs(
apiKey: "REPLACE - DataDog API Key",
host: Environment.MachineName,
source: "REPLACE - Log-Source",
service: GetServiceName(),
configuration: new DatadogConfiguration(),
logLevel: LogEventLevel.Infomation
)
.CreateLogger())
);
Full source code and required NuGet packages are here:
To respond to your comment on wanting custom metrics, this is still possible without the agent at the same location. After installing the nuget package of datadog called statsdclient you can then configure it to send the custom metrics to an agent located elsewhere. Example below:
using StatsdClient;
var dogstatsdConfig = new StatsdConfig
{
StatsdServerName = "127.0.0.1", // Optional if DD_AGENT_HOST environment variable set
StatsdPort = 8125, // Optional; If not present takes the DD_DOGSTATSD_PORT environment variable value, else default is 8125
Prefix = "myTestApp", // Optional; by default no prefix will be prepended
ConstantTags = new string[1] { "myTag:myTestAppje" } // Optional
};
StatsdClient.DogStatsd.Configure(dogstatsdConfig);
StatsdClient.DogStatsd.Increment("fakeVisitorCountByTwo", 2); //Custom metric itself

Resources