I am running an application that will connect an azure SQL database. I am running into a problem that the application will connect to the database in multiple threads, thus causing race conditions. I want to run them sequentially.
I can't change the application, is there a way to set the database to allow a single connection only, if it doesn't then wait until it connects?
Single user mode is not available on Azure SQL Database.
However, maybe you should consider adding a retry logic on the application so it can retry X times and the interval between retries can be increased after a retry fails.
public void HandleTransients()
{
var connStr = "some database";
var _policy = RetryPolicy.Create < SqlAzureTransientErrorDetectionStrategy(
retryCount: 3,
retryInterval: TimeSpan.FromSeconds(5));
using (var conn = new ReliableSqlConnection(connStr, _policy))
{
// Do SQL stuff here.
}
}
Related
I have a singleton Cosmos DB Client running as a singleton with default options. I'm using a .NET 6.0 WebAPI project, running in an Azure app service with "Always-On" enabled. The App Service and Cosmos Account are in the same region, UE2. The API queries a Cosmos container and returns the result.
I've noticed that the latency of the first query is always slow (4-6 seconds), subsequent queries are much faster (-100ms) but also sometimes have random high latency. This is not a cold start scenario, the client has already been initialized by the DI pipeline. I'm also not being rate limited.
Here is my singleton client
public CosmosDbService(IConfiguration configuration)
{
var account = configuration.GetSection("CosmosDb")["Account"];
var key = configuration.GetSection("CosmosDb")["Key"];
var databaseName = configuration.GetSection("CosmosDb")["DatabaseName"];
var containerName = configuration.GetSection("CosmosDb")["Container"];
CosmosClient client = new (account, key);
_myContainer = client.GetContainer(databaseName, containerName);
}
Here is the meat of the query where a Linq query is being passed in:
public class RetrieveCarRepository : IRetrieveCarRepository
{
public async Task<List<CarModel>> RetrieveCars(IQueryable<CarModel> querydef)
{
var query = querydef.ToFeedIterator();
List<CarModel> cars = new ();
while (query.HasMoreResults)
{
var response = await query.ReadNextAsync();
foreach (var car in response)...do a thing
I've been through several Cosmos training videos and cosmos courses but still haven't been able to come to an idea of what is happening.
From the comments.
For query performance using the .NET SDK please see: https://learn.microsoft.com/en-us/azure/cosmos-db/performance-tips-query-sdk?tabs=v3&pivots=programming-language-csharp#use-local-query-plan-generation
Query Plan generation can affect latency and can be avoided if:
The query is reworked to be on a single partition (instead of cross-partition).
The workload runs on Windows, compiled as x64 and with the Nuget DLLs co-located. Which in turn would leverage local query plan generation through the ServiceInterop.dll
On both cases the Query Plan request should be removed and latency improved.
As a general rule, latency should be investigated on the P99 across 1h to understand how it is impacted. A couple of higher latency requests can always happen.
Keep also in mind that query latency will vary based on the type of query, volume of data to transfer, and number of pages. You can capture the Diagnostics and use: https://learn.microsoft.com/azure/cosmos-db/troubleshoot-dot-net-sdk-slow-request
I want to delete 20-30k items in bulk. Currently I am using below method to delete these items. But its taking 1-2 mins.
private async Task DeleteAllExistingSubscriptions(string userUUId)
{
var subscriptions = await _repository
.GetItemsAsync(x => x.DistributionUserIds.Contains(userUUId), o => o.PayerNumber);
if (subscriptions.Any())
{
List<Task> bulkOperations = new List<Task>();
foreach (var subscription in subscriptions)
{
bulkOperations.Add(_repository
.DeleteItemAsync(subscription.Id.ToString(), subscription.PayerNumber).CaptureOperationResponse(subscription));
}
await Task.WhenAll(bulkOperations);
}
}
Cosmos Client:As we can see I have already set AllowBulkExecution = true
private static void RegisterCosmosClient(IServiceCollection serviceCollection, IConfiguration configuration)
{
string cosmosDbEndpoint = configuration["CosmoDbEndpoint"];
Ensure.ConditionIsMet(cosmosDbEndpoint.IsNotNullOrEmpty(),
() => new InvalidOperationException("Unable to locate configured CosmosDB endpoint"));
var cosmosDbAuthKey = configuration["CosmoDbAuthkey"];
Ensure.ConditionIsMet(cosmosDbAuthKey.IsNotNullOrEmpty(),
() => new InvalidOperationException("Unable to locate configured CosmosDB auth key"));
serviceCollection.AddSingleton(s => new CosmosClient(cosmosDbEndpoint, cosmosDbAuthKey,
new CosmosClientOptions { AllowBulkExecution = true }));
}
Is there any way to delete these item in a batch with CosmosDB SDK 3.0 in less time?
Please check the metrics to understand if the volume of data you are trying to send is not getting throttled because your provisioned throughput is not enough.
Bulk just improves the client-side aspect of sending that data by optimizing how it flows from your machine to the account, but if your container is not provisioned to handle that volume of operations, then operations will get throttled and the time it takes to complete will be longer.
As with any data flow scenario, the bottlenecks are:
The source environment cannot process the data as fast as you want, which would show as a bottleneck/spike on the machine's CPU (processing more data would require more CPU).
The network's bandwidth has limitations, in some cases the network has limits on the amount of data it can transfer or even the amount of connections is can open. If the machine you are running the code has such limitations (for example, Azure VMs have SNAT, Azure App Service has TCP limits) and you are running into them, new connections might get delayed and thus increasing latency.
The destination has limits in the amount of operations it can process (in the form of provisioned throughput in this case).
I'm testing Azure SQL Serverless and from SSMS it seems to work fine, but from my ASP.NET Core application it never wakes up.
Using SSMS I can open a connection to a sleeping Serverless SQL database and after a delay the connection will go through.
Using my ASP.NET Core application I tried the same. From the login page I tried to login, which opens a connection to the database. After 10 or 11 seconds (I looked up the default timeout and its supposed to be 15 seconds but in this case it always seems to be about 10.5 seconds +/-0.5s). According to the docs, the first connection attempt may fail but subsequent ones should succeed, but I can send multiple queries to the database and it always fails with the following error:
Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myDb' on server
'MyDbSvr.database.windows.net' is not currently available. Please retry the connection later. If the
problem persists, contact customer support, and provide them the session tracing ID of
'{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}'.
If I wake the database up using SSMS then the login web page can connect to the database and succeeds.
I have added Connect Timeout=120; to the connection string.
The connection does happen during an HTTP request that is marked async on the Controller, thought I don't know if that makes any difference.
Am I doing something wrong or is there something additional I need to do to get the DB to wake?
[updte]
as an extra test wrote the following test
void Main()
{
SqlConnection con = new SqlConnection("Server=mydbsvr.database.windows.net;Database=mydb;User Id=abc;Password=xyz;Connect Timeout=120;");
Console.WriteLine(con.ConnectionTimeout);
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "select getdate();";
Console.WriteLine(cmd.ExecuteScalar());
}
and got the same error.
I figured it out and its the dumbest thing.
This Azure SQL Server instance was migrated from another subscription and the group that migrated it gave it a new name, but they did something that allowed the use of the old name also. I'm researching to figure out how that was done. I will update this answer when I find out what that was.
As it turns out, using the old name with an Serverless Database won't wake up the db. Don't know why. But if you change to use the new/real server name it works. you do have to add a retry to the connection as it may fail the first few times.
[Update]
The new server allows logins using the old name by using a Azure SQL Database Alias https://learn.microsoft.com/en-us/azure/sql-database/dns-alias-overview
I'm using a Logic App where the workflow calls at a certain point an Azure Function using the Webhook URL (as a workaround to Azure Functions Durable).
The goal of this function is to insert/update data into an Azure SQL Database with a SQL request
"MERGE INTO...USING...WHEN NOT MATCHED...WHEN MATCHED AND...".
In the logs of the Azure Function, i could see it failed and it seems to run 4 times (maybe due to the supposed Timeout, I don't know), but I don't understand since I increased the CommandTimeout to 50minutes and I set 1Hour to the Timeout of the action "Launch Webhook" in the LogicApp :S Here's the sample of the exception logged in the Azure Function :
Exception while executing function: XmlImport_DoWork
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: XmlImport_DoWork ---> System.Data.SqlClient.SqlException : Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated. ---> System.ComponentModel.Win32Exception : The wait operation timed out
The table actually have around 250,000 lines and it seems to be good when I launch the LogicApp (and so the Azure Function) to a table which is almost empty !
Any ideas about what's going on and how to fix it ? I tried to look at the "Query Performance Insight" in Azure SQL database component but there are nothing in "Recommendations" section
The Function App where are stored my Azure Functions is using an App Service Plan.
BTW the XML file I was trying to import in DB has a size of 20M but I tried with a lighter XML (9M) but it didn't work either
Azure Durable Function: V2 and .Net Core 2.2 - Timeout expired issue RESOLOVED
The activity function 'A_ValidateAndImportData' failed: "Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.". See the function execution logs for additional details.
Using DAPPER to call SQL Server stored procedure: Dapper not honoring "Connection Timeout" Property in the connection string
Solution: Use a connection timeout parameter to provide "0"(ZERO or increase timeout according to your need) to solve this problem
Example Code:
public async Task<int> ValidateAndImportData(string connectionString, int param1,
int databaseTimeOut = 0)
{
using (var connection = new SqlConnection(connectionString))
{
var param = new DynamicParameters();
param.Add("#param1", param1);
param.Add("#returnStatus", dbType: DbType.Int32, direction: ParameterDirection.Output);
await connection.ExecuteAsync("[dbo].[ValidateAndImportData]", param,
commandType: CommandType.StoredProcedure, commandTimeout: databaseTimeOut).ConfigureAwait(false);
return param.Get<int>("returnStatus");
}
}
I am writing a singleton class (Object in scala) which uses apache httpclient(4.5.2) to post some file content and return status to caller.
object HttpUtils{
protected val retryHandler = new HttpRequestRetryHandler() {
def retryRequest(exception: IOException, executionCount: Int, context: HttpContext): Boolean = {
//retry logic
true
}
}
private val connectionManager = new PoolingHttpClientConnectionManager()
// Reusing same client for each request that might be coming from different threads .
// Is it correct ????
val httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.setRetryHandler(retryHandler)
.build()
def restApiCall (url : String, rDD: RDD[SomeMessage]) : Boolean = {
// Creating new context for each request
val httpContext: HttpClientContext = HttpClientContext.create
val post = new HttpPost(url)
// convert RDD to text file using rDD.collect
// add this file as MultipartEntity to post
var response = None: Option[CloseableHttpResponse] // Is it correct way of using it ?
try {
response = Some(httpClient.execute(post, httpContext))
val responseCode = response.get.getStatusLine.getStatusCode
EntityUtils.consume(response.get.getEntity) // Is it require ???
if (responseCode == 200) true
else false
}
finally {
if (response.isDefined) response.get.close
post.releaseConnection() // Is it require ???
}
}
def onShutDown = {
connectionManager.close()
httpClient.close()
}
}
Multiple threads (More specifically from spark streaming context) are calling restApiCall method. I am relatively new to scala and apache httpClient. I have to make frequent connections to only few fixed server (i.e. 5-6 fixed URL's with different request parameters).
I went through multiple online resource but still not confident about it.
Is it the best way to use http client in multi-threaded environment?
Is it possible to keep live connections and use it for various requests ? Will it be beneficial in this case ?
Am i using/releasing all resources efficiently ? If not please suggest.
Is it good to use it in Scala or there exist some better library ?
Thanks in advance.
It seems the official docs have answers to all your questions:
2.3.3. Pooling connection manager
PoolingHttpClientConnectionManager is a more complex implementation
that manages a pool of client connections and is able to service
connection requests from multiple execution threads. Connections are
pooled on a per route basis. A request for a route for which the
manager already has a persistent connection available in the pool will
be serviced by leasing a connection from the pool rather than creating
a brand new connection.
PoolingHttpClientConnectionManager maintains a maximum limit of
connections on a per route basis and in total. Per default this
implementation will create no more than 2 concurrent connections per
given route and no more 20 connections in total. For many real-world
applications these limits may prove too constraining, especially if
they use HTTP as a transport protocol for their services.
2.4. Multithreaded request execution
When equipped with a pooling connection manager such as
PoolingClientConnectionManager, HttpClient can be used to execute
multiple requests simultaneously using multiple threads of execution.
The PoolingClientConnectionManager will allocate connections based on
its configuration. If all connections for a given route have already
been leased, a request for a connection will block until a connection
is released back to the pool. One can ensure the connection manager
does not block indefinitely in the connection request operation by
setting 'http.conn-manager.timeout' to a positive value. If the
connection request cannot be serviced within the given time period
ConnectionPoolTimeoutException will be thrown.