I'm trying to use redis on a node express server to store user sessions, by using the connect-redis(on Github) library. And I find this block of settings works well:
var app = express();
app.use(session({
secret: 'hahahahahahahahahaha',
cookie: { maxAge: 36000000 },
store: new redisStore(),
}))
Notice I didn't pass any params to new redisStore() and it also works for now (in its documentation a client is passed), guess it's using localhost and default port by default.
But I'm worrying, if I put my server on AWS EC2 in the future which is a shared server, will that be a problem if I don't specify a client? Say, will redis conflicts with other servers also hosting on that EC2?
You are using a redis client here. You just relinquished the configuration of the client though. That will be a problem in the future, as (presumably) you will be deploying on AWS in a cluster. The redis client needs to be common across instances.
If you don't define the port number and other details of the client, you basically are undoing the purpose of having state management. Your redis clients will be limited to your own instance.
Go through http://redis.io/topics/partitioning for more info.
The idea is to have one (or a cluster) of redis instances that run independent of your AWS instances, so that state is shared across all of your instances. That is why you need to have complete control of your redis client
Related
I am beginner to use redis-server in my nodejs application I am using redis server as a session-store for my app as:
var RedisStore = require('connect-redis')(express);
var admin_session = express.session({
key: 'admin_token',
store: new RedisStore({
host: 'localhost',
port: 6379,
db: 2
// pass: 'RedisPASS'
}),
secret: 'aersda##$32sfas2342'
});
This is working fine for single instance. But my query is for multiple instance serving from aws elb.
Actually I want redis server as a cluster which clear the session for all the instances if any changes is available inside any of the instance and cleared all the code level caching.
If its possible can anybody help me how to do this and whats the steps?
Thanks in advance,
Vijay
This is a fairly standard approach for sessions when you have multiple web servers behind a load balancer.
If you run a redis server on every web instance, then you need to enable sticky sessions on your load balancer. This will work fine until you start auto-scaling: as soon as one of your web instances is removed from the pool, anyone with a session on that instance will lose it.
So you want to run a shared redis (or memcached) server for your caching and/or sessions. With elasticache, you have the option of running a single node "cluster" and using the standard way of connecting, or running a true cluster and using the AWS library to connect to it.
For sessions, I would probably just use a single node and not bother with the elasticache client.
If you want to clear your caches when you do a deployment, you'll have to have a hook and write code to do this. Or you could simply spin up a new Elasticache server, update your configurations, and destroy the old one.
i'm building a web application on NodeJS using express; the session store is a Redis instance which i talk to using connect-redis. the usual bits look like, well, usual:
RedisStore = ( require 'connect-redis' ) express
express_options =
...
'session':
'secret': 'xxxxxxxx'
'store': new RedisStore host: '127.0.0.1', port: 6379, ttl: 2 * weeks
'cookie': maxAge: 2 * weeks
app = express()
# Middleware
...
app.use express.cookieParser 'yyyyyy'
app.use express.session express_options[ 'session' ]
...
this does work well as such. however, i have not demonized Redis yet. after starting the server (but not Redis) and re-issuing an HTTP request from the browser, the application (apparently, naturally) failed to recognize yesterday's session cookie. to be more precise, the point of failure was
request.session.regenerate =>
request.session.user = uid_hint
in a login view, and the message was TypeError: Cannot call method 'regenerate' of undefined. now the question is:
(1) is my impression true that express won't balk at me when i try to use a session middleware that is configured to ask for data on a specific port, and yet that port is not served at all? if so, why is there no error message?
(2) what is a good way to test for that condition? i'd like a helpful message at that point.
(3) given that a DB instance may become unavailable at any one time—especially when it is separated by a network from the app server—what are best practices in such a case? fall back to memory-based sessions? refuse to serve clients?
(4) let us assume we fall back on another session storage mechanism. now all existing sessions have become invalid, right? unless we can decide whether a given signed SID coming in from a client is computationally valid in the absence of an existing record. those sessions will still be devoid of data, so it's not clear how useful that would be. we might as well throw away the old session and start a new one. but how? request.session = new ( require 'express' ).session.Session(), maybe?
Bonus Points (i'm aware some people will scoff at me for asking so many different things, but i think a discussion centered on sessions & cookies should include the below aspect)
thinking it over, i'm somewhat unhappy i'm using Redis at all—not because it's Redis, but because i have yet another DB make in the app. a theoretical alternative to using a session DB could be a reasonably secure way to keep all session data (NOT the user ID data, NO credit card numbers—just general stuff like which page did you come from etc) within the cookie. that way, any one server process can accept a request and has all the session data at hand to respond properly. i'm aware that cookie storage space is limited (like 4kB), but that might prove enough still. any middleware to recommend here? or is the idea dumb / insecure / too 1990?
connect-reddis listens to redis for the error event
./lib/connect-redis.js
self.client.on('error', function () { self.emit('disconnect'); });
So after creating the store, listen to the disconnect event
var store = new RedisStore({
host: 'localhost',
port: 6379,
db: 2,
pass: 'RedisPASS'
});
store.on('disconnect', function(){
console.log('disconnect');
});
I have 3 Servers running NodeJs, and they are related each other with Redis (1 master, 2 slaves).
The issue i'm having is that running the system on a single server works fine, but when I scale it to 3 NodeJS servers, it starts missing messages and the system gets unstable.
My load balancer does not accept sticky sessions. So every time that the requests from the client arrives to it, they can go to a different server.
I'm pointing all the NodeJS servers to the Redis Master.
It looks like socket.io is storing information on each server and it is not being distributed with redis.
I'm using socket.io V9, I'm suspecting that I don't have any handshake code, could this be the reason?
My code to configure socket.io is:
var express = require('express');
var io = require('socket.io');
var redis = require('socket.io/node_modules/redis');
var RedisStore = require('socket.io/lib/stores/redis');
var pub = redis.createClient("a port", "an ip");
var sub = redis.createClient("a port", "an ip");
var client = redis.createClient("a port", "an ip");
var events = require('./modules/eventHandler');
exports.createServer = function createServer() {
var app = express();
var server = app.listen(80);
var socketIO = io.listen(server);
socketIO.configure(function () {
socketIO.set('store', new RedisStore({
redisPub: pub,
redisSub: sub,
redisClient: client
}));
socketIO.set('resource', '/chat/socket.io');
socketIO.set('log level', 0);
socketIO.set('transports', [, 'htmlfile', 'xhr-polling', 'jsonp-polling']);
});
// attach event handlers
events.attachHandlers(socketIO);
// return server instance
return server;
};
Redis only syncs from the master to the slaves. It never syncs from the slaves to the master. So, if you're writing to all 3 of your machines, then the only messages that will wind up synced across all three servers will be the ones hitting the master. This is why it looks like you're missing messages.
More info here.
Read only slave
Since Redis 2.6 slaves support a read-only mode that
is enabled by default. This behavior is controlled by the
slave-read-only option in the redis.conf file, and can be enabled and
disabled at runtime using CONFIG SET.
Read only slaves will reject all
the write commands, so that it is not possible to write to a slave
because of a mistake. This does not mean that the feature is conceived
to expose a slave instance to the internet or more generally to a
network where untrusted clients exist, because administrative commands
like DEBUG or CONFIG are still enabled. However security of read-only
instances can be improved disabling commands in redis.conf using the
rename-command directive.
You may wonder why it is possible to revert
the default and have slave instances that can be target of write
operations. The reason is that while this writes will be discarded if
the slave and the master will resynchronize, or if the slave is
restarted, often there is ephemeral data that is unimportant that can
be stored into slaves. For instance clients may take information about
reachability of master in the slave instance to coordinate a fail over
strategy.
I arrived to this post:
It can be a good idea to have a "proxy" between nodejs servers and the load balancer.
With this approach XHR-Polling can be used in load balancers without Sticky sessions.
Load balancing with node.js using http-proxy
using nodejs-http-proxy i can have custom routing route, ex. by adding a parameter on the "connect url" of socket.io.
Anyone tried this solution before?
I have a standalone Node.js app which has SocketIO server that listens on a certain port, e.g. 8888. Now I am trying to run this app in a cluster and because cluster randomly assigns workers to requests, SocketIO clients in XHR polling mode once handshaken and authorized with one worker get routed to another worker where they're not handshaken and the mess begins.
And because workers don't share anything, I can't find a workaround. Is there a known solution to this issue?
There is no "simple" solution. What you have to do is the following:
If a client connects to a worker, save the connection-id together with the worker-id and a potential additional identification-id in a global (=for all workers accessible) store (i.e. redis).
If a client gets routed to another worker, use the store to look up which worker is reponsible for this client (either with the connection-id or with the additional identification-id and then hand it over to that worker (either with the nodejs-worker-master-worker-communication or via redis-pub-sub)
I habe implemented such thing with sock.js and an additional degree of complexity: I have two node.js servers with four workers each, so I had to use redis-pub-sub for worker/worker communication, because it is not guaranteed that they are on the same machine.
Actually there is a simple solution: using Redis to store sockets states.
Everything is explained in Socket.IO documentation:
The default 'session' storage in Socket.IO is in memory (MemoryStore).
The MemoryStore only allows you to deploy socket.io on a single
process. If you want to scale to multiple process and / or multiple
servers you can use our RedisStore which uses the Redis NoSQL database
as man in the middle.
So in order to change the store instance to RedisStore we add this:
var RedisStore = require('socket.io/lib/stores/redis')
, redis = require('socket.io/node_modules/redis')
, pub = redis.createClient()
, sub = redis.createClient()
, client = redis.createClient();
// Needs to be done after 'listen()'
io.set('store', new RedisStore({
redisPub : pub
, redisSub : sub
, redisClient : client
}));
Of course you will need to have a redis server running.
First Problem
I'm trying to figure out sessions, stores, authorization, and redis. If it's important, I am using Express#3.0.0rc4. I understand that I have two options to put my RedisStore(). Which one do I use? Do I use both?
express.session({secret: 'secret', key: 'key', store: new RedisStore()});
io.set('store', new RedisStore());
I already have node.js, express, and socket.io running. So now I'm trying to implement redis but I don't know how to implement authorization using sessions/stores and I don't know how to write to the redis database. I haven't found any documentation on this. I found a site that talks about sessions and stores using socket.io and express but no redis, and another one that talks about sessions and stores using all three, but it doesn't use io.set('store', ...).
I also don't know if I should use two different stores, one for express and one for socket.io, or if I should just use one. Look at the example for clarification:
//Redis Variables
var redis = require('socket.io/node_modules/redis');
var RedisStore = require('socket.io/lib/stores/redis');
var pub = redis.createClient();
var sub = redis.createClient();
var client = redis.createClient();
var redis_store = new RedisStore({
redisPub: pub,
redisSub: sub,
redisClient: client
});
app.configure(function(){
//...code goes here...
app.use(express.session({
secret: 'secret',
key: 'key',
store: redis_store //Notice I'm using redis_store
}));
//...more code...
});
io.configure(function(){
io.set('store', redis_store); //Notice it's the same RedisStore() being used
});
Do I use the same RedisStore() for each? Do I create seperate ones for each? Do I just use express or socket.io? What I really want is to be able to authenticate clients (I assume that's done through sessions) and have them update the redis database when they connect - keeping a log of when people accessed my site. Which leads to my second problem.
Second Problem
So I have no idea how to access and edit the redis database from this point. I haven't been able to test this because of my first problem but I assume it would be something like this:
io.sockets.on('connection', function(socket){
var session = socket.handshake.session;
redis.put(session);
});
I also haven't seen any documentation on how to update a redis database from within node.js so I highly doubt that redis.put() is the correct terminology haha. I have visited redis's website but I can't find commands for node.js. Just commands for using regular redis from the command line. Anyways, if someone could at least point me in the right direction that would be great. Thanks. :)
Express and Socket.IO have their own integration with Redis for session management, as you've seen. It is designed as a blackbox integration, the idea being that the session store implementation is independent from the rest of your code. Since it's independent, that means you can't go in and use express or socket.io to access Redis directly. You'll need to add a regular redis client like node_redis. The benefit is you don't have to worry about making all those redis calls yourself, instead you'll be interacting with express or socket.io's session store interfaces.
So in your #1 case, you could pass in a single new instance of RedisStore, not two new ones as you've done. Or you could follow your second link and have socket.io listen through express. In that case it would integrate with express session management. That's why you don't see the extra io.set('store') call in that example.
It'll probably seem redundant to you, but try to think of RedisStore as a special client designed only for session management. Even thought RedisStore probably relies on something like node_redis, you shouldn't try to access it. You have to include another library for accessing your redis database directly, assuming you wanted to store other non-session items in redis in the first place.