If I have a JDBCAppender configured to throw log messages to MySQL
and, while my system is up I restart the database is it reconnect to DB?
I have had this use case occur over this past weekend. My database is hosted by Amazon AWS. It rolled over my log database and all of the instances logging to that database via the Log4j JDBC Appender stopped logging. I bounced one of the apps and it resumed logging.
So the answer to this question, through experience, appears to be No.
If the database goes down and comes back online, the JDBC appender does not reconnect automatically.
edit
JDBCAppender getConnection might be overridden to fix.
JDBCAppender in log4j 1.2.15 has following code
protected Connection getConnection() throws SQLException {
if (!DriverManager.getDrivers().hasMoreElements())
setDriver("sun.jdbc.odbc.JdbcOdbcDriver");
if (connection == null) {
connection = DriverManager.getConnection(databaseURL, databaseUser,
databasePassword);
}
return connection;
}
so if connection is not null, but is broken (needs reconnect) log4j will return broken connection to its logic, and executing statement which does logging to db will fail.
Not a workaround, but a proper solution is to replace log4j with logback: see related answer: Log to a database using log4j
Related
I have this code
#Bean
public CqlSession getCqlSession() {
return CqlSession.builder()
.addContactPoint(new InetSocketAddress(cassandraHost, cassandraPort))
.withAuthCredentials(cassandraUsername, cassandraPassword)
.build();
}
The connection is failing with this exception:
Failed to instantiate [com.datastax.oss.driver.api.core.CqlSession]: Factory method 'getCqlSession' threw
exception; nested exception is com.datastax.oss.driver.api.core.AllNodesFailedException: Could not reach
any contact point, make sure you've provided valid addresses (showing first 1 nodes, use getAllErrors()
for more): Node(endPoint=tinyurl-cassandra.cassandra.cosmos.azure.com/52.230.23.170:10350, hostId=null,
hashCode=237f706): [com.datastax.oss.driver.api.core.DriverTimeoutException: [s0|control|id: 0xb89dacff,
L:/192.168.0.101:59158 - R:tinyurl-cassandra.cassandra.cosmos.azure.com/52.230.23.170:10350] Protocol
initialization request, step 1 (OPTIONS): timed out after 5000 ms]
I am new to Cassandra and have tried the following:
Validated that the credentials are okay.
Try with csqlsh - could not connect as well.
Check there's no firewall setup in my machine. Can telnet to host and port.
Can open Cassandra Shell from Azure Data Explorer.
What am I missing? I am new to this. Any help will be appreciated.
Looks like you are using the v.4x version of the Java Driver. The default load balancing in this driver mandates that you provide local data center, e.g:
CqlSession.builder().withSslContext(sc)
.addContactPoint(new InetSocketAddress(cassandraHost, cassandraPort)).**withLocalDatacenter("UK South")**
.withAuthCredentials(cassandraUsername, cassandraPassword).build();
You could take a look at this getting started sample for further reference: https://github.com/Azure-Samples/azure-cosmos-db-cassandra-java-getting-started-v4
I'm testing Azure SQL Serverless and from SSMS it seems to work fine, but from my ASP.NET Core application it never wakes up.
Using SSMS I can open a connection to a sleeping Serverless SQL database and after a delay the connection will go through.
Using my ASP.NET Core application I tried the same. From the login page I tried to login, which opens a connection to the database. After 10 or 11 seconds (I looked up the default timeout and its supposed to be 15 seconds but in this case it always seems to be about 10.5 seconds +/-0.5s). According to the docs, the first connection attempt may fail but subsequent ones should succeed, but I can send multiple queries to the database and it always fails with the following error:
Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myDb' on server
'MyDbSvr.database.windows.net' is not currently available. Please retry the connection later. If the
problem persists, contact customer support, and provide them the session tracing ID of
'{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}'.
If I wake the database up using SSMS then the login web page can connect to the database and succeeds.
I have added Connect Timeout=120; to the connection string.
The connection does happen during an HTTP request that is marked async on the Controller, thought I don't know if that makes any difference.
Am I doing something wrong or is there something additional I need to do to get the DB to wake?
[updte]
as an extra test wrote the following test
void Main()
{
SqlConnection con = new SqlConnection("Server=mydbsvr.database.windows.net;Database=mydb;User Id=abc;Password=xyz;Connect Timeout=120;");
Console.WriteLine(con.ConnectionTimeout);
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "select getdate();";
Console.WriteLine(cmd.ExecuteScalar());
}
and got the same error.
I figured it out and its the dumbest thing.
This Azure SQL Server instance was migrated from another subscription and the group that migrated it gave it a new name, but they did something that allowed the use of the old name also. I'm researching to figure out how that was done. I will update this answer when I find out what that was.
As it turns out, using the old name with an Serverless Database won't wake up the db. Don't know why. But if you change to use the new/real server name it works. you do have to add a retry to the connection as it may fail the first few times.
[Update]
The new server allows logins using the old name by using a Azure SQL Database Alias https://learn.microsoft.com/en-us/azure/sql-database/dns-alias-overview
We have a redis cache hosted on Azure and we use the following as a service in our .net core application:
services.AddSingleton<IConnectionMultiplexer>(provider =>
ConnectionMultiplexer.Connect(Configuration.GetConnectionString("RedisConnection"))
);
Our connection string is similar to the following:
ourserver.redis.cache.windows.net:6380,password=******=,ssl=True,abortConnect=False,syncTimeout=4000
in getting from cache we don't face problems but if we add to cache we sometimes get the following error:
"StackExchange.Redis.RedisConnectionException: It was not possible to
connect to the redis server(s); to create a disconnected multiplexer,
disable AbortOnConnectFail. ConnectTimeout"
I have read solutions like setting abortconnect to false and increasing the timeout but I have implemented it like the connection string above.
If you have suggestions or you think that we have a problem with the connection string kindly advise.
Update: we have implemented a solution to solve connectivity issue but now we get this:
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation
What is your connectTimeout set to? Can you try increasing that to see if it helps?
I've enabled Caching in my Spring app and I use Redis to serve the purpose.
However, whenever a connection failure occurs, the app stops working whereas I think it had better
skip the Caching and go on with normal execution flow.
So, does anyone have any idea on how to gracefully do it in Spring ?
Here is the exception I got.
Caused by: org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
As from Spring Framework 4.1, there is a CacheErrorHandler that you can implement to handle such exceptions. Refer to the javadoc for more details.
You can register it by having your #Configuration class extends CachingConfigurerSupport (see errorHandler()).
You can use CacheErrorHandler as suggested by Stephane Nicoll. But you should make sure to make
RedisCacheManager transactionAware to false in your Redis Cache Config(to make sure the transaction is committed early when executing the caching part and the error is caught by CacheErrorHandler and don't wait until the end of the execution which skips CacheErrorHandler part). The function to set transactionAware to false looks like this:
#Bean
public RedisCacheManager redisCacheManager(LettuceConnectionFactory lettuceConnectionFactory) {
JdkSerializationRedisSerializer redisSerializer = new JdkSerializationRedisSerializer(getClass().getClassLoader());
RedisCacheConfiguration redisCacheConfiguration = RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofHours(redisDataTTL))
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(redisSerializer));
redisCacheConfiguration.usePrefix();
RedisCacheManager redisCacheManager = RedisCacheManager.RedisCacheManagerBuilder.fromConnectionFactory(lettuceConnectionFactory)
.cacheDefaults(redisCacheConfiguration)
.build();
redisCacheManager.setTransactionAware(false);
return redisCacheManager;
}
Similar to what Stephane has mentioned, I have done in by consuming the error in try catch block. Adding a fall back mechanism where if Redis is not up or may be the data is not present then I fetch the data from DB.(Later if I find one then I add the same data in Redis,if it is up to maintain consistency.)
Environment is Red Hat, Cassandra 2.1, Datastax Java driver 2.1.1.
I have developed custom authentication/authorization plugins for Cassandra, and they work beautifully when I try them with cqlsh - I can see my plugins being called, users are authenticated/authorized accordingly, etc. - bottom line, everything works exactly as expected.
Then I tried to test using the Datastax driver. I'm connecting to Cassandra with:
public class CassandraConnection {
private final Cluster cluster;
private final Session session;
public CassandraConnection(final String node, final int port) {
this.cluster = Cluster.builder()
.addContactPoint(node)
.withPort(port)
.withCredentials("someuser", "somepassword")
.build();
this.session = cluster.connect();
}
// Etc....
The call to cluster.connect() generates an exception:
Exception in thread "main" com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042] Cannot connect))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:196)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:80)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1145)
at com.datastax.driver.core.Cluster.init(Cluster.java:149)
at com.datastax.driver.core.Cluster.connect(Cluster.java:225)
at com.<company...packages...>.CassandraConnection.<init>(CassandraConnection.java:21)
Here is the puzzling part: although I can see my plugins being called when I test them using cqlsh, they are never accessed when I use the Datastax driver - I have added log messages in the beginning of each method, and they are never called. There are no errors in the logs indicating any sort of initialization problem, and I do see a message indicating that my plugins will be used.
That exact same client code works with no problem when:
I don't have my plugin running.
I use Cassadra's PasswordAuthenticator.
So, it looks like there is some problem with my plugins, but how can that be if 1) they work fine with cqlsh and 2) none of their methods are being called when the datastax driver is being used?
A couple of additional points - if I try to connect using Datastax's DevCenter, I see the same behavior as my client, with the exact same exception, so that rules out my (very simple) client code. I have also tried to:
cluster.getConfiguration().getSocketOptions().setReadTimeoutMillis(10000);
before calling connect() as suggested in other posts, but that didn't help either - when I step through the client with the debugger, I see the error as soon as I call cluster.connect(), so it's not a time out issue either.
Any help is appreciated.