I currently have an Azure Application Gateway that is configured for a minimum of 2 instances and a maximum of 10 instances. It's Tier is "WAF V2" and autoscaling is enabled.
If autoscaling is enabled, then theoretically there should be somewhere between 2 and 10 instances. So where can I go to check the current number of instances that the gateway has scaled up to? This seems like important information if you want to figure out if your gateway is overloaded.
I currently have been pointed out by Microsoft to this topic after asking them the same question.
My assumption on this that I take, and may not be acurate, is that I look at the Current Capacity Units metric, to see how many are in use for a certain moment. Since, documentation says one instance uses around 10 capacity units, I do the simple math to know how many instances we are using, and if we need to increase the max, or lower the minimum.
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-autoscaling-zone-redundant
"Each capacity unit is composed of at most: 1 compute unit, or 2500 persistent connections, or 2.22-Mbps throughput."
"Note
Each instance can currently support approximately 10 capacity units. The number of requests a compute unit can handle depends on various criteria like TLS certificate key size, key exchange algorithm, header rewrites, and in case of WAF incoming request size. We recommend you perform application tests to determine request rate per compute unit. Both capacity unit and compute unit will be made available as a metric before billing starts."
I've tried to obtain this value with Logs analytics.
Enable it and use this query:
AzureDiagnostics
| where TimeGenerated > ago(30m)
| summarize dcount(instanceId_s) by bin(TimeGenerated, 1m)
You will have the different request grouped by the distinct instance name every minute. Consider adding some additional filter to the query, since you may only be interested in certain types of events.
I think it can be a good approximation
I don't think it shows you the current number of instances (if you switch to manual it will show you the instance count under properties blade), because it doesn't make sense. That's what autoscale is for, you don't really care how many instances are running, what you care is request latency\failed requests. If you see those increase, you can increase the number of maximum Application Gateway instances.
Api gives the following response with autoscale enabled:
"sku": {
"name": "Standard_v2",
"tier": "Standard_v2"
},
And this without autoscale enabled:
"sku": {
"name": "Standard_v2",
"tier": "Standard_v2",
"capacity": 4
},
so I guess it hidden from the api, so no way to know it.
Related
I've red the documentation and searched the internet for a simple explanation on Azure application gateway auto-scaling and the above quoted line but failed.
It would be really helpful if you can explain/provide a explanation link related to the same for better understanding.
Thank you!
When you enable auto scaling you need to set a minimum and maximum instance count. How do you know how many instances you need to handle the minimum amount of traffic you want to be able to handle? That is where Capacity Units play a role:
Capacity Unit is the measure of capacity utilization for an Application Gateway across multiple parameters.
A single Capacity Unit consists of the following parameters:
2500 Persistent connections
2.22-Mbps throughput
1 Compute Unit
If any one of these parameters are exceeded, then another n capacity unit(s) are necessary, even if the other two parameters don’t exceed this single capacity unit’s limits. The parameter with the highest utilization among the three above will be internally used for calculating capacity units, which is in turn billed.
When configuring the minimum and maximum number of instances you can now calculate how many instances you need because a single instance can handle up to 10 Capacity Units, so for example a maximum number of 10 * 2500 = 25.000 persistent connections.
For example: if you expect to have to deal with 6000 persistent connections you will need at least 3 instances (3 * 2500 = up to 7500 persistent connections)
All the functions in our Function Apps seem to be reporting weirdly high resource consumption. From the docs:
Functions are billed based on observed resource consumption measured in gigabyte seconds (GB-s). Observed resource consumption is calculated by multiplying average memory size in gigabytes by the time in milliseconds it takes to execute the function. Memory used by a function is measured by rounding up to the nearest 128 MB
(I know it's also priced per million executions, but I'm investigating a single execution here).
The "monitor" blade for my function shows that the last execution had a duration of 47659ms, or 47.659 seconds. The "memory working set" metric says that my function was using 117MB at its peak, so round that up to 128MB, and my function's resource consumption should be:
0.128GB * 47.659s = 6.1 GB-s.
Problem is, for that execution the "Function Execution Units" metric shows that it actually used 5.94M (million) GB-s. I'm pretty sure M = Million because some tiny ones earlier on had a "k" suffix instead of an M (the function was failing to start back then and only ran for ~100ms).
I also have the daily execution quota set to 13,333 so the whole Function App should have immediately stopped for the day, but it hasn't, and one of the other functions executed a few minutes after. Attached is a screenshot with the execution in question indicated along with the other one. I've asked other people here to log in and check it out, and everyone sees the same kind of thing, on all function apps.
What am I missing here?
Edit: the metrics API shows the same weirdness - a function execution that runs for 591ms returns this:
az monitor metrics list --resource 'func-name' --resource-group 'rg-name' --resource-type 'Microsoft.Web/sites' --metrics 'FunctionExecutionUnits'
{
"average": null,
"count": null,
"maximum": null,
"minimum": null,
"timeStamp": "2020-07-06T06:55:00+00:00",
"total": 120064.0
}
Finally found some clarity on a different documentation page, Estimating Consumption plan costs, at the bottom of the "View Execution Data" section there's an example for how to generate a chart of Function Execution Units, and underneath their example chart, it briefly mentions the following:
This chart shows a total of 1.11 billion Function Execution Units consumed in a two-hour period, measured in MB-milliseconds. To convert to GB-seconds, divide by 1024000.
Would have been a lot clearer if the units were displayed in Azure Monitor and/or mentioned in the Functions Pricing docs. But my fault for not RTFM...
I got this error when i use the function ST_ISVALIDDETAILED:
This is a linestring object.
I can't find a documentation of limit or size. Is there a roadmap to increase these limits?
Is there some workaround to bypass this?
Again..
I use:
SELECT ST_ISVALIDDETAILED({ "type": "LineString", "coordinates": c.coordinate})
FROM c
WHERE (etc)
If the function returns false (for example because one or more points overlas) there is something (function, or procedure etc) to correct the object?
The 4096-point limit in the Service Quotas is a general points-per-polygon limit, not simply a limit applied to ST_ISVALIDDETAILED. As someone else said, we recommend submitting a request to support a greater number of points-per-polygon via Cosmos DB UserVoice if this is something you need.
In the new Azure portal under
Cloud services > Scaling blade > Scale Profile > choosing Recurrence
There is a range of instances to choose, if I understood correctly.
Can someone explain the meaning of a "range", since in certain hour I would expect to have a fixed number of instances.
New portal
The range defines the minimum and maximum number of instances that will be running at any given time. For example, if you're configuring a scale profile based on CPU load you will have CPU percentage rules in place that will add/remove instances based on CPU thresholds you define. The scale profile will add/remove instances as needed within the range you specified.
The Range in the new Portal has 2 sliders Minimum and Max Number of Instances
Depending on the trigger you will set for this to Happen
Manual Scale : The Nummber of Instances will be set based on your Choice
CPU : You set the Minimum and the Max for Instances and Besed on the CPU Untalization Azure will Set the Number of instances
Other Measures : Same as CPU but for Queuing, Traffic, Disk I/O Etc.
There is "Scale Profile" and also a sub scaling call "rule".
The rule determine the scale within the range of the "Scale Profile".
target range for the metric. For example, if you chose CPU percentage, you can set a target for the average CPU across all of the instances in your service. A scale out will happen when the average CPU exceeds the maximum you define, likewise, a scale in will happen whenever the average CPU drops below the minimum
https://learn.microsoft.com/en-us/azure/monitoring-and-diagnostics/insights-how-to-scale
The facts:
1 Azure SQL S0 instance
a few tables one of them containing ~ 8.6 Million Rows and 1 PK
Running a Count-query on this table takes nearly 30 minutes (!) to complete.
Upscaling the instance from S0 to S1 reduces the query time to 13 minutes:
Looking into Azure Portal (new version) the resource-usage-monitor shows the following:
Questions:
Does anyone else consider even 13 minutes as rediculos for a simple COUNT()?
Does the second screenshot meen that during the 100%-period my instance isn't responding to other requests?
Why are my metrics limited to 100% in both S0 and S1? (see look under "Which Service Tier is Right for My Database?" stating " These values can be above 100% (a big improvement over the values in the preview that were limited to a maximum of 100).") I'd expect the S0 to bee like on 150% or so if the quoted statement is true.
I'm interested in experiences regarding usage of databases with more than 1.000 records or so from other people. I don't see how a S*-scaled Azure SQL for 22 - 55 € per month could help me in upscaling-strategies at the moment.
Azure SQL Database editions provide increasing levels of DTUs from Basic -> Standard -> Premium levels (CPU,IO,Memory and other resources - see https://msdn.microsoft.com/en-us/library/azure/dn741336.aspx). Once your query reaches its limits of DTU (100%) in any of these resource dimensions, it will continue to receive these resources at that level (but not more) and that may increase the latency in completing the request. It looks like in your scenario above, the query is hitting its DTU limit (10 DTUs for S0 and 20 for S1). You can see the individual resource usage percentages (CPU, Data IO or Log IO) by adding these metrics to the same graph, or by querying the DMV sys.dm_db_resource_stats.
Here is a blog that provides more information on appropriately sizing your database performance levels. http://azure.microsoft.com/blog/2014/09/11/azure-sql-database-introduces-new-near-real-time-performance-metrics/
To your specific questions
1) As you have 8.6 million rows, database needs to scan the index entries to get the count back. So, it may be hitting the IO limit for the edition here.
2) If you have multiple concurrent queries running against your DB, they will be scheduled appropriately to not starve one request or the other. But latencies may increase further for all queries since you will be hitting the available resource limits.
3) For older Web/Business editions, you may be able to see the metric values going beyond 100% (they are normalized to the limits of an S2 level), as they don't have any specific limits and run in a resource-shared environment with other customer loads. For the new editions, metrics will never exceed 100%, because system guarantees you resources upto 100% of that edition's limits, but no more. This provides predictable, guaranteed amount of resources for your DB unlike Web/Business editions, where you may get very little or lot more resources at different times depending on other competing customer DB workloads running on the same machine.
Hope this helps.
-- Srini