In my nodejs app, I'm using Redis keys as channel names. I want a client to subscribe a channel only if the corresponding key exists. The problem is between a EXISTS command and a SUBSCRIBE command, another client may remove an existing key. I can't use WATCH-MULTI-EXEC to make it atomic because I can't use SUBSCRIBE within a MULTI-EXEC block. I can't use Lua script either.
If there any way to maintain atomicity in this case?
It seems impossible with current version of Redis. I switched to a different approach that do not require atomic subscribe.
Related
I am building a system with a main server and worker server. The main server uses MongoDB as the primary database. I want to communicate between servers in a none asynchronous manner.
My biggest fear is making an update to the database, and then failing to emit the corresponding event, or emitting the event, but then having an unsuccessful write operation.
Is there any way to ensure that I either succeed entirely or fail entirely. I have been looking at MongoDB Transactions, and they seem to be fine for saving multiple documents at once.
Is there any way I can use MongoDB transactions to ensure my write operation and my other package for event emitting are in sync.
I am using NodeJS and I haven't chosen an event emitter yet.
Any Ideas?
I am working on setting up socket.io in cluster mode using PM2.
I am using socket.io-redis package and it works fine in cluster mode.
But the problem arises when I want to access all connected sockets. Because processes don't know about socket connections in other processes in cluster mode.
I thought socket.io-redis keeps track of all the connected sockets and all its session info but it didn't.
Is there any way or solution to access all the socket connection existing in all processes in socket.io/Nodejs?
Socket.io-redis does keep track in a sense..
From their docs
"The Redis adapter extends the broadcast function of the in-memory adapter: the packet is also published to a Redis channel (see below for the format of the channel name).
Each Socket.IO server receives this packet and broadcasts it to its own list of connected sockets."
So basically, redis is used as the broker to tell each socket server to emit based on X channel etc. Allowing you to have a socket.io server in cluster mode work, but as you have mentioned it can fall short when you need to keep track of things outside of just an emit.
So where does this leave us.. Well you can use custom hooks via socket.io-redis but personally I found it to be really difficult to understand and use and had limited success personally. I think with the new version of socket.io and socket.io redis there were some tweaks to make this simpler however I have not tried them.
Instead, what we do is use redis hset and jget to store the socket and an ID of a users, then when we want to get all users online we can query redis to get the list of online users or users in a specific room etc.
What you will want to do is add the redis package and connect in additon to the regular pub / sub.
Then, when a user joins a room or your server for that matter you will do an hset. On the first join ours looks something like this
redis.hset([collection-name],[Field],[value])
So in code it looks like
redis.hset(decoded.cID,"socket-" + socket.id,socket.nickname)
This will set a value in redis, so the collection name is a value ( for us its a unique id of the channel ) then we stock the 'socket.id' for the Field along with a 'nick-name' for the value. This value is the users ID OR its anonymous if they are not logged in
Then, when we want to grab who is in a room we use the hget command
redis.HGETALL([collection-name],function(err,results){}
So inside of say the emit, we call the redis.HGETALL command to get all items inside a specific collection that we pass in and send that back to all connected users.
Aws elasticache creates a redis cluster by default.
Im using nodejs and using ioredis.
My question is if i call hgetall will it automatically query all nodes in the cluster?
Or is there something else i need to do?
You don't need to query all nodes. Use Redis.Cluster to connect to the cluster, and it will send the command to the right node.
A decent client library for Redis Cluster should implement the MOVED and ASK Redirection. The end user of the client library should NOT worry about where the key is located.
Update: I think what I said here is incorrect, see the other answer. For a command like HGETALL the client library knows where to send it.
From the docs, emphasis mine:
Every command will be sent to exactly one node. For commands containing keys, (e.g. ... HGETALL), ioredis sends them to the node that [sic] serving the keys, and for other commands not containing keys, (e.g. INFO, KEYS and FLUSHDB), ioredis sends them to a random node.
No, not automatically. You would need to send hgetall to each node in the cluster yourself. There is a nodes() utility function returns the array of nodes to facilitate this.
The documentation answers this explicitly and provides an example, see https://github.com/luin/ioredis#running-commands-to-multiple-nodes.
I'm trying to create a Flash app with some real time functionality, and would like to use Redis' pubsub functionality which is a perfect fit for what I need.
I know that connecting to a data store directly from client is almost always bad. What are the security implications of this (since I'm not an expert on Redis), and are there ways to work around them? From what I read, there is a possible exploit of doing config sets and changing the rdb file location and be able to arbitrary overwrite files. Is there anything else? (If I don't use that particular redis instance for anything at all, i.e. no data being stored)
I understand the alternative is to write some custom socket server program and have it act as the mediating layer for connecting to redis and issuing commands -- that's the work I'd like to avoid having to write, if possible.
** Edit **
Just learned about the rename-command configuration to disable commands. If I disable every single command on the redis instance and leave only SUBSCRIBE and PUBLISH open, would this be good enough to run on production?
I think it would be a bad idea to connect directly your client to Redis. Redis offers an authentication system for a unique user only. It expects this user to be your server app.
From my point of view, directly exposing Redis is always a bad idea. It would allow anybody to access all of your data. This is confirmed by the Redis doc.
So you won't avoid adding or developing the server side of your app.
I am implementing a TCP chat server using node js and redis, however i dont seem to be able to persist chat data on redis using Publish and Subscribe, and hence when i have left the chat room and reentered, i will not be updated on the newest messages, how should i implement something like this?
Publish is not meant to be stored in Redis, even if you chose the disk storage. When it recieves message, it just finds the connections with requested channels and forwards to each. So, it is not storing anything. Even if it did, It should continously try to forward messages (because it's a pub/sub model) which is not very effective. Instead, you should also push (by lpush the messages to a queue, so they can be stored. And when a client connects and has no messages, it can retrieve those messages from queue (without popping, so other newcomers can also use) and then subscribe to channel and recieve new messages.
By default, redis is in memory only. You have to enable persistence explicitly.
There are multiple options, AOF every query being the safest, but probably the slowest.
More details here: http://redis.io/topics/persistence