We're currently getting an hazelcast IMap each time a request is incoming.
Sample code:
#GetMapping("/tipus-adreca")
public List<TipusAdrecaDTO> getTipusAdreca(){
IMap<String, String> hzMap = hzInstance.getMap(HesConstants.SISTEMA_TELECOMUNICACIONS);
//Do something
}
I'm running in my mind if it's too eager.
Is there any best practice in order to obtain IMap? I mean, Could I get IMap as a #Bean?
By other side... Is there any way to configure TTL of each IMap?hazelcast
By other side... Is there any way to configure TTL of each IMap?hazelcast
You can't define a TTL for an IMap, but you can for the entries of the IMap.
You should most certainly not be doing hz.getMap for every cluster access request. Its an expensive operation, so look to reuse the IMap instance instead of getting from the cluster.
Related
We want our Redis to be more scalable and we want to be able to add more read instances.
I am trying to use this new Reader endpoint: https://aws.amazon.com/about-aws/whats-new/2019/06/amazon-elasticache-launches-reader-endpoint-for-redis
However I dont see any easy or automated way for ioredis to use that approach where I can set up which endpoint will be for writes and which one for reads. Even here I can see the recommended approach at the end is to "manually split": https://github.com/luin/ioredis/issues/387
Do you know any existing solution or good approach where I can set up which endpoints will be used for writes and which one will be used for reads?
The most straightforward for me right now is some kind of "proxy" layer, where I will create two instances of Redis and I will send all writes to the primary endpoint and all reads to Reader endpoint. However I would prefer some better (or well tested) approach.
PS: I tried to "hack it" with Cluster functionality of ioredis, but even the simple connection without any functionality and one - primary endpint - fails with ClusterAllFailedError: Failed to refresh slots cache.
(To have Reader endpoint enabled - the Cluster mode must be off)
Just note about how it ended
We had two instances (or reused the same instance if URL was same)
redis = new Redis(RKT_REDIS_URL.href, redisOptions)
if (RKT_REDIS_READER_URL.href === RKT_REDIS_URL.href) {
redisro = redis
} else {
redisro = new Redis(RKT_REDIS_READER_URL.href, redisOptions)
}
And then used first for writes and other for reads.
redis.hmset(key, update)
redisro.hmget(key, field)
However after some time we have adopted clustered redis and it is much better and can recommend it. Also the ioredis npm module is capable of using it seemlessly (you dont have to configure anything, you just put there configure endpoint which i.e. AWS provides and thats it).
This was our configuration
redisOptions.scaleReads = 'master'
redis = new Redis.Cluster([RKT_REDIS_URL.href], redisOptions)
The options for scaleReads are
scaleReads is "master" by default, which means ioredis will never send
any queries to slaves. There are other three available options:
"all": Send write queries to masters and read queries to masters or
slaves randomly. "slave": Send write queries to masters and read
queries to slaves.
https://github.com/luin/ioredis
I know it is possible to store every entry on a Map at Hazelcast to backing DataStore via implementation of MapStore interface but in our application we don't want that, we only want that if our application is running on danger of getting OutOfMemory Exception, it evicts certain percentage of data in the memory (with LRU principle) but during the eviction it store evicted entries to data store and load them again if entry key asked again.
I know that the OutOfMemoryHandler interface exist to manage OutOfMemory situations, %25 Eviction policy exists and MapStore also.
What I don't know can I combine them all?
Thx for answers...
You can use IMap.putTransient() to put your entries in IMap but not trigger MapStore.store()
IMap.evict() (or auto eviction) will not trigger MapStore removal, i.e. it will just remove from the IMap.
However I don't suggest this approach because once the jvm gets OutOfMemoryError it can become unstable so applying such business logic would not be possible. Also OutOfMemoryHandler may not be called even if JVM throws OutOfMemoryError because the error may be thrown from an external (user) thread and Hazelcast may not be informed about it.
There is already a question on the difference between Hazelcast Instance and Hazelcast client.
And it is mentioned that
HazelcastInstance = HazelcastClient + AnotherFeatures
So is it right to say client just reads and writes to the cluster formed without getting involved in the cluster? i.e. client does not store the data?
This is important to know since we can configure JVM memory as per the usage. The instances forming the cluster will be allocated more than the ones that are just connecting as a client.
It is a little bit more complicated than that. The Hazelcast Lite Member is a full-blown cluster member, without getting partitions assigned. That said, it doesn't store any data but otherwise behaves like a normal member.
Clients on the other side are simple proxies that have to forward everything to one cluster member to get any operation done. You can imagine a Hazelcast client to be something like a JDBC client, that has just enough code to connect to the cluster and redirect requests / retrieve responses.
Does Redis emit any kind of event when the TTL expires for a particular key?
I am looking to have a count of keys added in Redis for my application at any given point of time. I am having a increment counter when I am generating the key, similarly I would like to have a decrement counter when the key expires (TTL expires).
I know I can acheive this by executing 'KEYS', but I am wondering if Redis generates some kind of event which I can capture when key expires.
I will use NodeJS to capture the event.
Thanks,
Raghu.
Do not use KEYS in production - it is a potentially long-running, RAM-consuming, service-denying operation.
Yes, as of v2.8.0 Redis does have what you're looking for. Read the Redis Keyspace Notifications page, specifically about setting up the x flag and subscribing to relevant channels.
Note that while this is a great way to use Redis, PubSub messages' delivery is not guaranteed so your counters could shift over time if messages are lost. In this case it would probably be good to periodically scan your database (using the SCAN command, not KEYS) to refresh them.
From what I'm seeing in the docs, it appears that Redis only lets you persist all keys, or don't persist at all (to disk).
What I'm trying to do is to only persist keys that don't have a TTL. That is, if I
setex some_key 60 "some data"
// or
set some_key "some data"
expire some_key 60
then don't persist those keys to disk -- ever!
In case this is not possible, I guess the next best solution is to use Memcached for those values, and Redis for what I'd like persisted, but it'd sure be nice if I don't have to go that far..
AFAIK What you are telling is correct it can either persist on can not persist. However, in this scenario Instead of using Memcache I would run two instance of redis one which can persist the keys and one with no persistence. As Creating the redis instance is easy.
Also, In future if there is a situation where you need few keys to be persist you can make the changes easily at application level if you use redis instead of Memcache.