We host a website on Microsoft Azure and we have Traffic Manager distributing traffic on two AppServices each with 15 instances and a maxpool size of 80 in the connection strings.
We also use P11 database which has a max connection count of 2400
From our understanding, the maximum possible connections would be:
Instance count * maxpool
30 * 80 = 2400
But we get errors regarding exceeding the maximum allowed number and to our surprise, running a query on the database to show us active connections yields 2600.
We don't have any webjobs running.
Can someone please explain what is happening?
Using the following query you can identity which program name have the most sessions, which login name creates the most sessions, which host creates the most sessions.
SELECT
c.session_id, c.net_transport, c.encrypt_option,
c.auth_scheme, s.host_name, s.program_name,
s.client_interface_name, s.login_name, s.nt_domain,
s.nt_user_name, s.original_login_name, c.connect_time,
s.login_time
FROM sys.dm_exec_connections AS c
JOIN sys.dm_exec_sessions AS s
ON c.session_id = s.session_id
The following statement shows you the maximum number of connections for the current tier.
SELECT ##MAX_CONNECTIONS AS 'Max Connections';
Hope this helps.
Regards,
Alberto Morillo
Related
I've red the documentation and searched the internet for a simple explanation on Azure application gateway auto-scaling and the above quoted line but failed.
It would be really helpful if you can explain/provide a explanation link related to the same for better understanding.
Thank you!
When you enable auto scaling you need to set a minimum and maximum instance count. How do you know how many instances you need to handle the minimum amount of traffic you want to be able to handle? That is where Capacity Units play a role:
Capacity Unit is the measure of capacity utilization for an Application Gateway across multiple parameters.
A single Capacity Unit consists of the following parameters:
2500 Persistent connections
2.22-Mbps throughput
1 Compute Unit
If any one of these parameters are exceeded, then another n capacity unit(s) are necessary, even if the other two parameters don’t exceed this single capacity unit’s limits. The parameter with the highest utilization among the three above will be internally used for calculating capacity units, which is in turn billed.
When configuring the minimum and maximum number of instances you can now calculate how many instances you need because a single instance can handle up to 10 Capacity Units, so for example a maximum number of 10 * 2500 = 25.000 persistent connections.
For example: if you expect to have to deal with 6000 persistent connections you will need at least 3 instances (3 * 2500 = up to 7500 persistent connections)
Is there a way to limit number of connections per host in cassandra cluster and based on what parameter this is calculated?
In some of cassandra node i can see established connections count for 9042 goes up to 1400+, is this something i need to worry about?
Thanks
Yes, you can limit the number of connections per host in the Cassandra cluster.
If you are using the C++ driver, check this out.
I visualize any query following the below path:
Client --> Session --> IO threads --> Connections --> Nodes
You can configure the number of IO threads(This is the number of threads that will handle query requests) associated with the session. In each IO thread, you can then configure the number of connections per host. If needed, the number of connections per host will increase based on certain parameters(The maximum count up to which this can be increased is also configurable).
So, at max, there can be x number of connections per host where,
x = number_of_sessions * number_of_IO_threads * max_number_of_connections_per_host
All 3 variables on RHS in the above equation are configurable.
Also check out:
https://www.datastax.com/dev/blog/4-simple-rules-when-using-the-datastax-drivers-for-cassandra
https://stackoverflow.com/a/28219086/5701173
If I have multiple instances on Azure and a webapp which has a connection string with maxpool:
the connection string max pool * instance count should equal the max allowed in the SQL tier , right? or should I set the maxpool to be equal to max for the tier
Example: SQL with max connections if 2400
Instance count is 10
Should the connection string for the app has sql connection string 240 or 2400 to be within limits?
Thank you
The pool size is defined per instance (actually per process, you might have multiple web jobs running on the same instance). So, if you want to guarantee that you are always below the limit, you should divide total size on instance/process count, 240 in your example.
How can i tell that i have reached the maximum number of queries that my cluster can handle at a time?
If the connection pool size is too small for the workload then we have to adjust the max_pooled_connections configuration variable, which controls the number of pooled connections between each pair of nodes.
However how can I tell how many pooled connections we have at a specific time ?
In memsql agregator status i can see following entries Aborted_connects is 11 - why do we abort those connection ? Also Max_used_connections is 41, while Connections is a number that increases constantly.
How can i tell that i have reached the maximum number of queries that
my cluster can handle at a time?
There isn't a hard limit on the number of queries you can send other than max_connections (100k), but at some point the cluster will not execute them all at once and will schedule/queue them up. Is your question about the former or the latter?
If the connection pool size is too small for the workload then we have
to adjust the max_pooled_connections configuration variable, which
controls the number of pooled connections between each pair of nodes.
However how can I tell how many pooled connections we have at a
specific time ?
show leaves will show how many connections are currently open from the current node to each leaf. So the current connection pool size is min(current open connections, max_pooled_connections). Note that this is per (node, node) pair.
In memsql agregator status i can see following entries
Aborted_connects is 11 - why do we abort those connection ? Also
Max_used_connections is 41, while Connections is a number that
increases constantly.
Aborted connects includes e.g. failed login authentications.
max_used_connections is max peak, connections is cumulative total.
The facts:
1 Azure SQL S0 instance
a few tables one of them containing ~ 8.6 Million Rows and 1 PK
Running a Count-query on this table takes nearly 30 minutes (!) to complete.
Upscaling the instance from S0 to S1 reduces the query time to 13 minutes:
Looking into Azure Portal (new version) the resource-usage-monitor shows the following:
Questions:
Does anyone else consider even 13 minutes as rediculos for a simple COUNT()?
Does the second screenshot meen that during the 100%-period my instance isn't responding to other requests?
Why are my metrics limited to 100% in both S0 and S1? (see look under "Which Service Tier is Right for My Database?" stating " These values can be above 100% (a big improvement over the values in the preview that were limited to a maximum of 100).") I'd expect the S0 to bee like on 150% or so if the quoted statement is true.
I'm interested in experiences regarding usage of databases with more than 1.000 records or so from other people. I don't see how a S*-scaled Azure SQL for 22 - 55 € per month could help me in upscaling-strategies at the moment.
Azure SQL Database editions provide increasing levels of DTUs from Basic -> Standard -> Premium levels (CPU,IO,Memory and other resources - see https://msdn.microsoft.com/en-us/library/azure/dn741336.aspx). Once your query reaches its limits of DTU (100%) in any of these resource dimensions, it will continue to receive these resources at that level (but not more) and that may increase the latency in completing the request. It looks like in your scenario above, the query is hitting its DTU limit (10 DTUs for S0 and 20 for S1). You can see the individual resource usage percentages (CPU, Data IO or Log IO) by adding these metrics to the same graph, or by querying the DMV sys.dm_db_resource_stats.
Here is a blog that provides more information on appropriately sizing your database performance levels. http://azure.microsoft.com/blog/2014/09/11/azure-sql-database-introduces-new-near-real-time-performance-metrics/
To your specific questions
1) As you have 8.6 million rows, database needs to scan the index entries to get the count back. So, it may be hitting the IO limit for the edition here.
2) If you have multiple concurrent queries running against your DB, they will be scheduled appropriately to not starve one request or the other. But latencies may increase further for all queries since you will be hitting the available resource limits.
3) For older Web/Business editions, you may be able to see the metric values going beyond 100% (they are normalized to the limits of an S2 level), as they don't have any specific limits and run in a resource-shared environment with other customer loads. For the new editions, metrics will never exceed 100%, because system guarantees you resources upto 100% of that edition's limits, but no more. This provides predictable, guaranteed amount of resources for your DB unlike Web/Business editions, where you may get very little or lot more resources at different times depending on other competing customer DB workloads running on the same machine.
Hope this helps.
-- Srini