I am using jdbc to connect to presto server.
From the documentation, I am able to connect to the server and run queries.
https://prestodb.io/docs/current/installation/jdbc.html
I am facing issues while sending ClientTags (X-Presto-Client-Tags) in the connection/statement.
Here's the function to build a connection and run a query.
public void test() {
java.util.Properties info = new java.util.Properties();
if (PRESTO_USER != null) {
info.put("user", PRESTO_USER);
}
if (PRESTO_PASSWORD != null) {
info.put("password", PRESTO_PASSWORD);
}
info.put("ClientTags", "my,tag");
try (Connection connection = DriverManager.getConnection(PRESTO_URL, info);
Statement statement = connection.createStatement()) {
testBody(connection, statement);
} catch (Exception ex) {
ex.printStackTrace();
fail("Exception occured");
}
}
However, it fails with
java.sql.SQLException: Unrecognized connection property 'ClientTags'
at com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:316)
For pyhive, I was able to override the session and pass client tags
https://github.com/dropbox/PyHive/issues/283. Can this be done for jdbc driver too?
It's currently not possible to set ClientTags on the connection URL.
Please create a feature request # https://github.com/trinodb/trino/issues/
Currently, the only way is to use Connection method:
Connection connection = DriverManager.getConnection("....");
connection.unwrap(PrestoConnection.class)
.setClientInfo("ClientTags", "one,two,three");
Related
Using Spring integration with Spring mail to connect to IMAP inbox and read emails, I need to detect login failure and network connectivity issues, i cannot find any methods that would let me pull connection status, even errorChannel is not getting any messages nor the logs shows any network connectivity problems
#Bean(name = "getImapMailReceiver")
#Scope(scopeName = ConfigurableBeanFactory.SCOPE_PROTOTYPE)
ImapMailReceiver getImapMailReceiver(Properties imapConnectionProperties) {
ImapMailReceiver obj = new ImapMailReceiver(imapConnectionProperties.getProperty(IMAP_MAIL_URL_KEY));
obj.setMaxFetchSize(Integer.parseInt(imapConnectionProperties.getProperty(IMAP_MAX_FETCH_SIZE_KEY)));
obj.setJavaMailProperties(imapConnectionProperties);
obj.setShouldDeleteMessages(false);
obj.setAutoCloseFolder(false);
obj.setShouldMarkMessagesAsRead(true);
obj.setSimpleContent(false);
obj.setCancelIdleInterval(10000);
obj.setJavaMailAuthenticator(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(imapConnectionProperties.getProperty("mail.imap.user"),
imapConnectionProperties.getProperty("testpassword"));
}
});
return obj;
}
To catch exceptions from such an ImapIdleChannelAdapter async component, you need to add a even listener for the ImapIdleExceptionEvent.
See more info in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/mail.html#imap-idle-and-lost-connections
I am using Azure CosmosDB with Blazor WebAssembly (client-side).
I want to catch CosmosException When connection token expires, But not getting cosmos exception for that, (Null exception found)
I have also tried same code in console application, In that I am able to catch cosmos exception that showing token expired.
Code sample :
using (CosmosClient client = new CosmosClient(account, token))
{
Database db = null;
db = client.GetDatabase("databaseName");
Container orgContainer = client.GetContainer("databaseName","containerName");
try
{
ItemResponse<CosmosException> response = await orgContainer.ReadItemAsync<CosmosException>("test", new PartitionKey("test"));
var data = response.Resource;
}
catch(CosmosException ex)
{
}
catch(Exception ex)
{
}
}
Exception Details of Console Application
This is probably because the exception occurs outside the try block. Also, make sure you do not instantiate a new cosmos client on each request. This does not perform well. Use singleton and keep alive between requests.
I can never store a string value to an Azure hosted Redis Cache. Using StackExchange.Redis version 2.0.601 and vs2015 and vs2019. Code below has error in comments (basically even with successful ConnectionMultiplexer.Connect there no connection is established).
static bool Connect()
{
ConnectionMultiplexer redis;
try
{
ConfigurationOptions cfgOptions = new ConfigurationOptions
{
EndPoints =
{
{"RedisOnMyAzureServer", myPort}
},
AbortOnConnectFail = false,
Ssl = true,
ConnectRetry = 3,
ConnectTimeout = 10000,
SyncTimeout = 10000,
DefaultDatabase = 0,
Password = "myPassword"
};
redis = ConnectionMultiplexer.Connect(cfgOptions); // takes 10.5 seconds on average
}
catch
{ return false; } // never errors
// some diagnostics follow
if (redis.IsConnected)
Console.WriteLine("client connection open");
else
Console.WriteLine("client connection closed");
if (redis.GetDatabase().IsConnected(default(RedisKey)))
Console.WriteLine("database connection open");
else
Console.WriteLine("database connection closed");
// both connection are always closed.
try
{
IDatabase db = redis.GetDatabase();
db.StringSet("mykey", "value");
}
catch
{ return false; } // always errors
return true;
}
Errors at last try/catch on db.StringSet method. I get this message:
No connection is available to service this operation: SET mykey; A blocking operation was interrupted by a call to WSACancelBlockingCall; IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=2,Free=1021,Min=4,Max=1023), Local-CPU: n/a
I got same error when I set minimum TLS version to 1.2 on Azure Redis Cache, and it was caused by TLS mismatch. It was resolved by following measures.
Add sslprotocols=tls12 to the connection string
Add TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 to TLS cipher suites
Here is the details, Remove TLS 1.0 and 1.1 from use with Azure Cache for Redis
The CosmosDb provider is sending this message:
“Response status code does not indicate success: 503 Substatus: 0 Reason: (The request failed because the client was unable to establish connections to 3 endpoints across 1 regions. Please check for client resource starvation issues and verify connectivity between client and server.”
In my tests, it works (.net core 3.1):
Task.Run(async () =>
{
var endpoint = “test”;
var masterKey = “test”;
using (var client = new DocumentClient(new Uri(endpoint), masterKey))
{
//Insert new Document
Console.WriteLine("\r\n>>>>>>>>>>>>>>>> Creating Document <<<<<<<<<<<<<<<<<<<");
dynamic candidato = new
{
Id = 1,
Nome = "Test"
};
var document1 = await client.CreateDocumentAsync(
UriFactory.CreateDocumentCollectionUri("Test", "Test"),
candidato);
Console.ReadKey();
}
}).Wait();
It does not:
Task.Run(async () =>
{
using (var context = new StudentsDbContext())
{
context.Add(new FamilyContainer(2, "Test"));
await context.SaveChangesAsync();
}
}).Wait();
public class FamilyContainer
{
public int Id { get; set; }
public string Nome { get; set; }
public FamilyContainer(int id, string nome)
{
Id = id;
Nome = nome;
}
}
public class StudentsDbContext : DbContext
{
public DbSet<FamilyContainer> FamilyContainer { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseCosmos(
"test",
"test",
"FamilyDatabase",
options =>
{ }
);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<FamilyContainer>(x =>
{
x.ToContainer("FamilyContainer");
});
}
}
Packages
Can anyone help me? Thanks
fail: Microsoft.EntityFrameworkCore.Update[10000]
An exception occurred in the database while saving changes for context type '...'.
Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: Maximum number of retries (6) exceeded while executing database operations with 'CosmosExecutionStrategy'. See inner exception for the most recent failure.
---> Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: 503 Substatus: 0 Reason: (Microsoft.Azure.Documents.ServiceUnavailableException: Service is currently unavailable.ActivityId: 07fbf539-0d44-4e5a-89d0-cd46838ee605, {"RequestStartTimeUtc":"2020-02-21T16:34:09.1834993Z","RequestEndTimeUtc":"2020-02-21T16:34:41.3484203Z","RequestLatency":"00:00:32.1649210","IsCpuOverloaded":false,"NumberRegionsAttempted":1,"ResponseStatisticsList":[{"ResponseTime":"2020-02-21T16:34:11.5964152Z","ResourceType":2,"OperationType":0,"StoreResult":"StorePhysicalAddress: rntbd:.../, LSN: -1, GlobalCommittedLsn: -1, PartitionKeyRangeId: , IsValid: True, StatusCode: 410, SubStatusCode: 0, RequestCharge: 0, ItemLSN: -1, SessionToken: , UsingLocalLSN: False, TransportException: A client transport error occurred: Failed to connect to the remote endpoint. (Time: 2020-02-21T16:34:11.5298608Z, activity ID: 07fbf539-0d44-4e5a-89d0-cd46838ee605, error code: ConnectFailed [0x0005], base error: socket error ConnectionRefused [0x0000274D]...
--- End of inner exception stack trace ---
I was facing same issue.
What worked for me is changing ConnectionMode to ConnectionMode.Gateway while initializing CosmosClient like :
var options = new CosmosClientOptions() { ConnectionMode = ConnectionMode.Gateway };
var client = new CosmosClient(endpoint, key, options);
For more details on refer :
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.cosmosclientoptions?view=azure-dotnet
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.connectionmode?view=azure-dotnet
TransportException: A client transport error occurred: Failed to connect to the remote endpoint. (Time: 2020-02-21T16:34:11.5298608Z, activity ID: 07fbf539-0d44-4e5a-89d0-cd46838ee605, error code: ConnectFailed [0x0005], base error: socket error ConnectionRefused
This means that the Connection was refused.
Either your Cosmos DB account has Firewall/VPN enabled and the application is not able to establish a connection due not not being in a whitelisted IP/Network : Try checking your account configuration.
The environment you are executing the code is restricting connections (some corporate Firewall or network might be blocking port ranges): Try running the app in a different network, or use GatewayMode. If that works, then this is related to the network.
The machine might be running low on sockets or high on CPU.
My RCA for this is: Cosmos Partitions where served by individual processes on CosmosDB, each partition serving process has it's own TCP port. When client connects to 443 (Using TCP Direct Mode), CosmosDB Proxy sends partition ports back to client so that client can talk to server-partitions in parallel. Partition ports are random (11000 upwards afaik). Normal company firewall would allow outbound 443 (connection to cosmos works) but blocks the outbound random ports. So at the end, access fails.
Workarounds:
Open firewall
Use Gateway Mode. This uses https/443 only by forwarding internally instead of redirecting to other ports.
It is because Entity framework has a default connection mode of Direct. It worked for me after overriding it to Gateway.
{
optionsBuilder.UseCosmos(
"test",
"test",
"FamilyDatabase",
options =>
{ options.ConnectionMode(ConnectionMode.Gateway); }
);
}
I just want to add this because it wasted a lot of my time. The following code would instantly die with an error message that led me to this S.O. post:
var container = _client.Client.GetContainer(_databaseName, containername);
var result = await container.CreateItemAsync(dataitem, pk);
I disbelieved the error message because everything else has worked, upsert, read, etc. After messing with it for a while, I noticed the documentation shows a template type for CreateItemAsync.
var container = _client.Client.GetContainer(_databaseName, containername);
var result = await container.CreateItemAsync<T>(dataitem, pk);
Changing the code to that fixed it (inside of a templated function).
I wanted to add: if I had been catching exceptions, I would have gotten to the meat of the problem much sooner. The library I am working with is not meant to catch exceptions, they are handled by the layer above it.
I'm using IMobileServiceSyncTable from Azure Mobile App. In InsertAsync operation, on the backend server side, I had some validations for the data and, if that validations failure, I want throw Exception from the server side. I tried return InternalServerError(), throw HttpResponseException, but never worked on the client side. I debbuged the Post method in server side, the server throws the exception or return InternalServerError, but in the mobile client, don't occurs error.
Can anyone help me?
Here is my code on the client side:
public async Task<bool> AddPaciente(Paciente novoPaciente)
{
//other things
try
{
await _pacienteTable.InsertAsync(novoPaciente);
}
catch (Exception e)
{
Debug.WriteLine(e.Message);
Debug.WriteLine(e.StackTrace);
throw new WebException(AppResources.MensagemFalhaConexaoServidorAzure);
}
await SyncPaciente();
return true;
}
Here is my post method on the backend server side
// POST tables/Paciente
public async Task<IHttpActionResult> PostPaciente(Paciente novoPaciente)
{
//other things
if (paciente != null)
{
var responseMessage = new HttpResponseMessage(HttpStatusCode.BadRequest)
{
Content = new StringContent("Já existe um paciente com esse token cadastrado.")
};
//throw new HttpResponseException(responseMessage);
return InternalServerError(new Exception("Já existe um paciente com esse token cadastrado."));
}
}
You should listen to the response status code.
var response = await _pacienteTable.InsertAsync(novoPaciente);
if (response.IsSuccessStatusCode) {
var content = await response.Content.ReadAsStringAsync ();
}
else
{
//Here handle the error code
throw new WebException(AppResources.MensagemFalhaConexaoServidorAzure);
}
I'm using IMobileServiceSyncTable from Azure Mobile App. In InsertAsync operation, on the backend server side, I had some validations for the data and, if that validations failure, I want throw Exception from the server side.
For IMobileServiceSyncTable, you are dealing with An Offline Client which means that IMobileServiceSyncTable<T>.InsertAsync would directly insert data into the local SQLite store on your mobile client-side. Until you manually call MobileServiceClient.SyncContext.PushAsync(), then your local data store would be pushed to your mobile backend. For this approach, I would recommend you make sure that you need to validate the inputs before you saving them into the local data store, otherwise your push operations would fail, then you need to force your client user to adjust the existing inputs even after it has been successfully added before.
If you use An Online Client as follows, then both your approaches for throwing the exception would be immediately returned to your client.
var mobileClient= new MobileServiceClient("https://{your-mobile-app-name}.azurewebsites.net");
var _pacienteTable= mobileClient.GetTable<Paciente>();
await _pacienteTable.InsertAsync(novoPaciente);
Moreover, I used the following code line for catching the exception:
try
{
await table.InsertAsync(item);
}
catch (MobileServiceInvalidOperationException ex)
{
//TODO:
//await ex.Response.Content.ReadAsStringAsync(); //get detailed response content
}
catch (Exception e)
{
//TODO: other uncaught exception
}
Also, for the similar issue, you could leverage Fiddler to capture the network traces to narrow down this issue, make sure your client-side could correctly receive the relevant response.