I'm looking at the documentation of the rest API and I'm looking to get the costs of my resources.
Looking at both the REST API and SDK for .NET, there's a Consumption endpoint that does this, but on further testing, this only seems to provide the latest available billing cycle (24 hour window).
Example Code:
var filterString = $"properties/instanceName eq \'{vm.VirtualMachineName}\'";
var vmConsumptionData = await consumptionClient.Item1.UsageDetails.ListAsync(
scope: $"/subscriptions/{consumptionClient.subscriptionGuid}",
filter: filterString
);
consumptionClient is an instance of a consumptionClient returned as part of a Tuple elsewhere.
Result is yesterdays last billed amount.
Are there any other endpoints that would provide a more up to date cost overview of my resources in Azure? I.E. Per minute or hour
The only other way I believe this can be done is to retrieve the 'Usage quantity' of a resource and work this out based on the resources cost per hour.
As of Mar-06-2020 documentation on Microsoft.CostManagement/Query worked:
POST https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/Microsoft.CostManagement/query?api-version=2019-11-01&$top=10000
Content-Type: application/json
{
"type": "ActualCost",
"dataSet": {
"granularity": "None",
"aggregation": {
"totalCost": {
"name": "PreTaxCost",
"function": "Sum"
},
"totalCostUSD": {
"name": "PreTaxCostUSD",
"function": "Sum"
}
},
"filter": {
"Dimensions": {
"Name": "ResourceId",
"Operator": "In",
"Values": [
"{resourceId}"
]
}
}
},
"timeframe": "Custom",
"timePeriod": {
"from": "2020-02-29T00:00:00+00:00",
"to": "2020-03-06T23:59:59+00:00"
}
}
with response:
{
"id": "subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/Microsoft.CostManagement/query/XXXXXXXX-9ab4-XXXX-83fe-XXXXXXXXXXXX",
"name": "XXXXXXXX-9ab4-XXXX-83fe-XXXXXXXXXXXX",
"type": "Microsoft.CostManagement/query",
"location": null,
"sku": null,
"eTag": null,
"properties": {
"nextLink": null,
"columns": [
{
"name": "PreTaxCost",
"type": "Number"
},
{
"name": "Currency",
"type": "String"
}
],
"rows": [
[
1234.5678910,
"CAD"
]
]
}
}
I found this returns accurate value for my specific resource.
NOTE: Only concern is that this API endpoint is flagged as -legacy in the documentation. Not sure if it will be supported long term but works for now.
Related
WHAT I am trying to do:
I am trying to retrieve the Sign-In logs for the last 24 hours of all users and save it in a blob storage. After the first result set creates the blob, the next results sets would update the blob file with the next remaining results.
Thought of using a blob storage and MS Graph because the Graph output contains all the details that I want without having to jump through various hoops in Powershell to expand certain properties and because the result size is huge (over 1GB via Export-CSV in PowerShell).
HOW I'm trying to do it
A scheduled run that does an HTTP request with the Graph query filtered by the last 24h which creates a block with the HTTP Body as content. After creation of the Blob, I added a (Do) Until control that runs until the HTTP Body does not contain #odata.nextLink and updates the blob file.
ISSUES:
First issue is that the Until loop finishes in 6 seconds.
Second issue is that the blob file only contains the first result set and is usually 9.3MB in size. Which means the next results set is not accessed and appended to the existing blob file.
My research
I tried with pagination enabled & disabled, various pagination thresholds, custom functions, but nothing that would make sense (to me at least) and I'm trying to follow the KISS model.
I looked over and tried to apply in one shape or form the answers from the below S.O. questions:
Graph Pagination in Logic Apps | Pagination with oauth azure data factory | Microsoft graph, batch request's nextLink | https://learn.microsoft.com/en-us/graph/paging;
Code I am trying
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Create_blob": {
"inputs": {
"body": "#body('fRequest')",
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "post",
"path": "/datasets/default/files",
"queries": {
"folderPath": "/graph",
"name": "DoUntil",
"queryParametersSingleEncoded": true
}
},
"runAfter": {
"fRequest": [
"Succeeded"
]
},
"runtimeConfiguration": {
"contentTransfer": {
"transferMode": "Chunked"
}
},
"type": "ApiConnection"
},
"Until": {
"actions": {
"Update_blob": {
"inputs": {
"body": "#body('fRequest')",
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "put",
"path": "/datasets/default/files/#{encodeURIComponent(encodeURIComponent('/graph/DoUntil'))}"
},
"runAfter": {},
"type": "ApiConnection"
}
},
"expression": "#not(contains(body('fRequest'), '#odata.nextLink'))",
"limit": {
"count": 60,
"timeout": "PT1H"
},
"runAfter": {
"Create_blob": [
"Succeeded"
]
},
"type": "Until"
},
"fRequest": {
"inputs": {
"authentication": {
"audience": "https://graph.microsoft.com",
"clientId": "registered_app",
"secret": "app_secret",
"tenant": "tenant_id",
"type": "ActiveDirectoryOAuth"
},
"method": "GET",
"uri": "https://graph.microsoft.com/beta/auditLogs/signIns?$filter=createdDateTime gt #{addDays(utcNow(),-1)}"
},
"runAfter": {},
"runtimeConfiguration": {
"paginationPolicy": {
"minimumItemCount": 500
}
},
"type": "Http"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Week",
"interval": 7,
"schedule": {
"hours": [
"7"
],
"minutes": [
0
]
}
},
"type": "Recurrence"
}
}
},
"parameters": {
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/subscription_id/resourceGroups/Apps/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/subscription_id/providers/Microsoft.Web/locations/eastus/managedApis/azureblob"
}
}
}
}
}
What am I doing wrong or missing?
Thanks in advance!
I managed to increase the pagination threshold to 20000 and now my files are no longer 9MB, they reach 200MB in size. I also removed the "Do" loop. Now I only need to create a break to avoid the threshold and resume collecting the remaining pages of results.
I'm trying to add a query into a Workbook in Azure Monitor that queries for the month-to-date usage costs for my Azure subscription.
When I construct the query and run it, I receive the following error message:
BadRequest: Invalid query definition, Dataset is invalid or not supplied.
However, the documentation for this REST API call specifically indicates that the datasets property of the body is not required.
Docs: https://learn.microsoft.com/en-us/rest/api/cost-management/query/usage
Screenshot of docs:
Here's a screenshot of what my query looks like in the Workbook:
Question: How do I properly construct the body for this request, when the documentation doesn't properly explain it?
Actually, this api-doc provides many examples, you can click any of the example for the request body details.
Take BillingAccountQuery-Modern for example, the request body is as below(you can modify it as per your need):
{
"type": "Usage",
"timeframe": "MonthToDate",
"dataset": {
"granularity": "Daily",
"filter": {
"and": [
{
"or": [
{
"dimension": {
"name": "ResourceLocation",
"operator": "In",
"values": [
"East US",
"West Europe"
]
}
},
{
"tag": {
"name": "Environment",
"operator": "In",
"values": [
"UAT",
"Prod"
]
}
}
]
},
{
"dimension": {
"name": "ResourceGroup",
"operator": "In",
"values": [
"API"
]
}
}
]
}
}
}
I have created a REST API with a Swagger/OPEN API specification which I will like to consume trough a Azure API Management tenant in a Logic App.
When I download the specification it looks like this:
{
"swagger": "2.0",
"info": {
"title": "Leasing",
"version": "1.0"
},
"host": "ENDPOINT.azure-api.net",
"basePath": "/leasing",
"schemes": [
"http",
"https"
],
"securityDefinitions": {
"apiKeyHeader": {
"type": "apiKey",
"name": "Ocp-Apim-Subscription-Key",
"in": "header"
},
"apiKeyQuery": {
"type": "apiKey",
"name": "subscription-key",
"in": "query"
}
},
"security": [
{
"apiKeyHeader": []
},
{
"apiKeyQuery": []
}
],
"paths": {
"/{Brand}/groups": {
"get": {
"description": "Get a list of leasing groups on a brand",
"operationId": "GetGroups",
"parameters": [
{
"name": "Brand",
"in": "path",
"description": "Selection of possible brands",
"required": true,
"type": "string",
"enum": [
"Volkswagen",
"Audi",
"Seat",
"Skoda",
"VolkswagenErhverv",
"Porsche",
"Ducati"
]
}
],
"responses": {
"200": {
"description": "Returns a list of leasing groups",
"schema": {
"$ref": "#/definitions/GroupArray"
}
},
"400": {
"description": "If the brand is not valid",
"schema": {
"$ref": "#/definitions/Error"
}
}
},
"produces": [
"application/json"
]
}
}
},
"definitions": {
"Group": {
"type": "object",
"properties": {
"id": {
"format": "int32",
"type": "integer"
},
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"leasingModelCount": {
"format": "int32",
"type": "integer"
},
"lowestMonthlyFee": {
"format": "int32",
"type": "integer"
}
}
},
"Error": {
"type": "object",
"properties": {
"code": {
"enum": [
"NotValidBrand",
"NotValidGroupId"
],
"type": "string",
"x-ms-enum": {
"name": "ErrorCode",
"modelAsString": true
}
},
"message": {
"type": "string"
}
}
},
"GroupArray": {
"type": "array",
"items": {
"$ref": "#/definitions/Group"
}
}
}
}
When I add this in a Logic App with the connector HTTP + Swagger I only get to define the {Brand} query input but not the various ways of using the Subscriptions key (header or query) as defined in SecurityDefiniations.
The whole securityDefinitions and security section are automatically generated in the Azure API Management service, but not recognized in Logic App.
See image of missing subscription key field:
What am I doing wrong?
Update
I have tried the following:
Usage of the 'Authentication' field (but this field is limited to certain types of auths flows https://learn.microsoft.com/en-us/azure/connectors/connectors-native-http#authentication)
Change the Logic App 'Http + Swagger'-action in code to add the header parameter, but this action converts the action to a simple 'Http' action and therfore loosing the automatic schema generation from Swagger.
I think you need to specify this in the Authentication-field in a JSON format. Something like:
{
"apiKeyHeader" : "your Ocp-Apim-Subscription-Key",
"apiKeyQuery" : "your subscription key"
}
We are working on an app that allows our Azure administrator to monitor Azure Resource requests by people in organization.
There is a requirement where I want to get list of all possible SKUs and SKU capacities through some API if available. I could have hard coded the values but considering the frequency of changes being brought by Microsoft to Azure subscription plans, I wanted to make it more generic and dog feed by Azure itself.
I could not find any API endpoint under Azure Rest API that provides us list of SKUs and Capacities for requested resource type.
Has anyone figured it out already? if not, what is the alternative and better approach?
Rahul.
The pattern for the Azure SKUs API in Azure Resource Manager is HTTP GET {resourceId}/skus?api-version=...
As others have indicated, there doesn't seem to be a good way to discover which ARM resource types have implemented the SKUs API. If you are looking for a generic option today, I would consider the following approach --
1) Gather a list of ARM resource ids in the subscription(s). You can use the ARM resources API to get this list.
2) For each resource id, call the following to retrieve the SKUs: GET {resourceId}/skus?api-version=...
3) Keep track of which ARM resource types return non-2xx status codes. These resource types do not implement the SKUs API and your code should ignore these resource types today.
Here are a few examples of the SKUs API for two common ARM resource types:
armclient get /subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachineScaleSets/{name}/skus?api-version=2017-03-30
armclient get /subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Web/serverFarms/{name}/skus?api-version=2014-04-01
As 4c74356b41 mentioned some might have that feature and some not. We can get the Azure REST API from the official document. And we can get some List SKU REST APIs, detail please refer to screenshot. You also can give your feedback to the Azure team.
Looks like Azure has made this REST API available now:
Resource Skus - List
Get all the resources in a subscription.
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/skus?api-version=2021-07-01
Sample Response:
{
"value": [
{
"resourceType": "virtualMachines",
"locations": [
"westus"
],
"capabilities": [
{
"name": "MaxResourceVolumeMB",
"value": "20480"
},
{
"name": "OSVhdSizeMB",
"value": "1047552"
},
{
"name": "vCPUs",
"value": "1"
},
{
"name": "HyperVGenerations",
"value": "V1"
},
{
"name": "MemoryGB",
"value": "0.75"
},
{
"name": "MaxDataDiskCount",
"value": "1"
},
{
"name": "LowPriorityCapable",
"value": "False"
},
{
"name": "PremiumIO",
"value": "False"
},
{
"name": "vCPUsAvailable",
"value": "1"
},
{
"name": "ACUs",
"value": "50"
},
{
"name": "vCPUsPerCore",
"value": "1"
},
{
"name": "EphemeralOSDiskSupported",
"value": "False"
},
{
"name": "AcceleratedNetworkingEnabled",
"value": "False"
},
{
"name": "RdmaEnabled",
"value": "False"
},
{
"name": "MaxNetworkInterfaces",
"value": "2"
}
],
"locationInfo": [
{
"location": "westus",
"zones": [
"2",
"1"
],
"zoneDetails": [
{
"name": [
"2"
],
"capabilities": [
{
"name": "UltraSSDAvailable",
"value": "True"
}
]
}
]
}
],
"name": "Standard_A0",
"tier": "Standard",
"size": "A0",
"family": "standardA0_A7Family"
},
{
"resourceType": "virtualMachines",
"locations": [
"westus"
],
"capabilities": [
{
"name": "MaxResourceVolumeMB",
"value": "71680"
},
{
"name": "OSVhdSizeMB",
"value": "1047552"
},
{
"name": "vCPUs",
"value": "1"
},
{
"name": "HyperVGenerations",
"value": "V1"
},
{
"name": "MemoryGB",
"value": "1.75"
},
{
"name": "MaxDataDiskCount",
"value": "2"
},
{
"name": "LowPriorityCapable",
"value": "True"
},
{
"name": "PremiumIO",
"value": "False"
},
{
"name": "vCPUsAvailable",
"value": "1"
},
{
"name": "ACUs",
"value": "100"
},
{
"name": "vCPUsPerCore",
"value": "1"
},
{
"name": "EphemeralOSDiskSupported",
"value": "False"
},
{
"name": "AcceleratedNetworkingEnabled",
"value": "False"
},
{
"name": "RdmaEnabled",
"value": "False"
},
{
"name": "MaxNetworkInterfaces",
"value": "2"
}
],
"locationInfo": [
{
"location": "westus",
"zones": [
"1",
"2",
"3"
]
}
],
"name": "Standard_A1",
"tier": "Standard",
"size": "A1",
"family": "standardA0_A7Family"
}
],
"nextLink": null
}
Is context telemetry "grouped" during the sampling of request telemetry?
For example, the data below contains a request which has a sample count of 10 ("count": 10), meaning that it is being used to represent 9 other "similar" requests; 90% of the telemetry has actually been discarded.
Does Application Insights only sample data together when the context data is exactly the same for the requests? For example, can I assume that the other 9 requests were also from 41.191.204.0 and have a custom field company of value 22f0141f-b3dc-53e1-86b8-dd0727c14497?
{
"request": [
{
"id": "bs6o2dRoL/Q=",
"name": "GET /api/resources",
"count": 10,
"responseCode": 200,
"success": true,
"url": "https://example.com/api/resources",
"urlData": {
"base": "/api/resources",
"host": "example.com",
"hashTag": "",
"protocol": "https"
},
"durationMetric": {
"value": 1073743.0,
"count": 11.0,
"min": 97613.0,
"max": 97613.0,
"stdDev": 0.0,
"sampledValue": 97613.0
}
}
],
"internal": {
"data": {
"id": "8cbd12ec-9780-11e6-b38b-c5e9335e7642",
"documentVersion": "1.61"
}
},
"context": {
"application": {
"version": "1.0.16286.5"
},
"data": {
"eventTime": "2016-10-21T11:21:16.942Z",
"isSynthetic": false,
"samplingRate": 9.09090909090909
},
"device": {
"type": "PC",
"osVersion": "Windows 10",
"roleInstance": "RD0003FF727A10",
"deviceName": "Other",
"deviceModel": "Other",
"browser": "Chrome",
"browserVersion": "Chrome 53.0",
},
"user": {
"isAuthenticated": false
},
"session": {
"isFirst": false
},
"operation": {
"id": "bs6o2dRoL/Q=",
"parentId": "bs6o2dRoL/Q=",
"name": "GET Resources/GetResourceAsync [id]"
},
"location": {
"clientip": "41.191.204.0",
"continent": "Africa",
"country": "South Africa",
"province": "Eastern Cape"
},
"custom": {
"dimensions": [
{
"company": "22f0141f-b3dc-53e1-86b8-dd0727c14497"
},
{
"factor": "100"
}
]
}
}
}
Application Insights does not group telemetry events based on the context, but based on the Operation ID. This is synchronized between the SDK sampling and the server side sampling to make sure you will be able to navigate between related page views and requests.
So if you want to make sure some events are grouped together in sampling, set their OperationId to be the same.
See here for full details on how Application Insights implements it's sampling.
Hope this helps,
Asaf