Today I ran my Node.js application in "production" mode for the first time and got this warning:
Warning: connection.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and obviously only work within a single process.
I only need to run a single process, but what should I use instead? I want my sessions to reside in RAM for fast access. I also want to be able to discard all the sessions by simply shutting down the Node app.
It seems an overkill to install Redis, MongoDB or another database just for this simple task. I also don't understand why is MemoryStore included in Node when it should not really be used?
Ok, after talking to Connect developers, I got more information. There are two things considered memory leaks here:
problem with JSON parsing which is already fixed in recent versions
the fact that there is no cleanup of expired sessions if the users never access them (i.e. the only cleanup is on-access)
The solution seems to be rather simple, at least this is what I plan to do: use setInterval to periodically clean up the expired sessions. MemoryStore provides all() to get the list, and we can use get() to force reading and thus expire them. Pseudo-code:
function sessionCleanup() {
sessionStore.all(function(err, sessions) {
for (var i = 0; i < sessions.length; i++) {
sessionStore.get(sessions[i], function() {} );
}
});
}
Now just call sessionCleanup periodically via setInterval() and you have automatic garbage collection for expired sessions. No more memory leaks.
So the accepted answer to this is [edit: was] pretty much a hack, and the others are just recommending using a database which I think is overkill.
I had the same problem and just replaced express-session with cookie-session.
To do this simply install cookie-session:
npm install cookie-session
Then in your app.js, find where express-session is being used and replace with cookie-session.
app.use(require('cookie-session')({
// Cookie config, take a look at the docs...
}));
You may need to change some other things, for me is was a simple swap-out-bobs-your-uncle-no-harm-done.
MemoryStore is just for (rapid) development mode, because if your app restarts (process dies) you will lose all the session data (that resided in the memory of that process).
If you don't want to use a database, use encrypted cookie storage instead.
http://www.senchalabs.org/connect/cookieSession.html
This module was designed to deal with the memory leak issue.
https://www.npmjs.com/package/session-memory-store
The accepted answer may be fine. However, since this question shows up high in the list of search results I figured I would include this in case it helps anyone else.
I think the consensus around the web is that the right way would be to indeed use a DB for that, but if you're positive you don't want to do that, then suppress the warning -- the warning is not law.
However, since you and I both agree that the memory leak is a real problem, it is hard to justify saying redis is overkill, since it would solve your problem.
I also don't understand why is MemoryStore included in Node when it
should not really be used
that is a great point -- but to that I would say that node iself has only recently itself become production ready. Some people would not agree with the notion that it is at all.
The alternative is to use Redis or Mongo as a store. With Mongo you use the express-session-mongo module.
There is an advice to remove stale sessions with an indexing option:
var MongoStore = require('express-session-mongo');
app.use(express.session({ store: new MongoStore() }));
db.sessions.ensureIndex( { "lastAccess": 1 }, { expireAfterSeconds: 3600 } )
Since stale sessions are removed by the database itself, Express session doesn't need to handle the cleanup by itself.
EDIT: It seems like you need to have your own "lastAccess" field. When you access it you update that field yourself. Check the MongoDB documentation expire-data
http://docs.mongodb.org/manual/tutorial/expire-data/
EDIT2:
Now becomes db.sessions.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 3600 } )
The Mongo background thread to check this field runs every 60 seconds. So the timing to remove the document is not exact.
For those who are having trouble with Redis try the following - hope this helps.
I'm using Redis for DEV and PROD and targeting Express v4.
On Windows I'm using the lightweight MSOpenTech Redis v3.0 toolset, otherwise, I just use the Heroku Redis Addon. To get it to work via Node hasn't been too hard - so far...
var session = require('express-session');
. . .
var RedisStore = require('connect-redis')(session);
var redisClient = require('redis').createClient(process.env.REDIS_URL);
var redisOptions = {
client: redisClient,
no_ready_check: true,
ttl: 600,
logErrors: true
};
var redisSessionStore = new RedisStore(redisOptions);
app.use(session({
store: redisSessionStore,
secret: 'Some.Long.Series.of.Crazy.Words.and.Jumbled.letter.etc',
resave: true,
saveUninitialized: true
}));
Good luck!
ps. I only just re-read the original query and noticed this - sorry!
It seems an overkill to install Redis, MongoDB or another database just for this simple task.
If you use OSX use
brew install memcached
if linux
apt install memcached
solve session message, becose de app can connect to 127.0.0.1:11211 memcache service.
Use
import createMemoryStore from "memorystore";
...
// Memory store
const MemoryStore = createMemoryStore(session);
app.use(session({
...
store: new MemoryStore({
checkPeriod: 86400000 // prune expired entries every 24h
}),
...
}))
Fix the problem in my case
Related
i'm building a web application on NodeJS using express; the session store is a Redis instance which i talk to using connect-redis. the usual bits look like, well, usual:
RedisStore = ( require 'connect-redis' ) express
express_options =
...
'session':
'secret': 'xxxxxxxx'
'store': new RedisStore host: '127.0.0.1', port: 6379, ttl: 2 * weeks
'cookie': maxAge: 2 * weeks
app = express()
# Middleware
...
app.use express.cookieParser 'yyyyyy'
app.use express.session express_options[ 'session' ]
...
this does work well as such. however, i have not demonized Redis yet. after starting the server (but not Redis) and re-issuing an HTTP request from the browser, the application (apparently, naturally) failed to recognize yesterday's session cookie. to be more precise, the point of failure was
request.session.regenerate =>
request.session.user = uid_hint
in a login view, and the message was TypeError: Cannot call method 'regenerate' of undefined. now the question is:
(1) is my impression true that express won't balk at me when i try to use a session middleware that is configured to ask for data on a specific port, and yet that port is not served at all? if so, why is there no error message?
(2) what is a good way to test for that condition? i'd like a helpful message at that point.
(3) given that a DB instance may become unavailable at any one time—especially when it is separated by a network from the app server—what are best practices in such a case? fall back to memory-based sessions? refuse to serve clients?
(4) let us assume we fall back on another session storage mechanism. now all existing sessions have become invalid, right? unless we can decide whether a given signed SID coming in from a client is computationally valid in the absence of an existing record. those sessions will still be devoid of data, so it's not clear how useful that would be. we might as well throw away the old session and start a new one. but how? request.session = new ( require 'express' ).session.Session(), maybe?
Bonus Points (i'm aware some people will scoff at me for asking so many different things, but i think a discussion centered on sessions & cookies should include the below aspect)
thinking it over, i'm somewhat unhappy i'm using Redis at all—not because it's Redis, but because i have yet another DB make in the app. a theoretical alternative to using a session DB could be a reasonably secure way to keep all session data (NOT the user ID data, NO credit card numbers—just general stuff like which page did you come from etc) within the cookie. that way, any one server process can accept a request and has all the session data at hand to respond properly. i'm aware that cookie storage space is limited (like 4kB), but that might prove enough still. any middleware to recommend here? or is the idea dumb / insecure / too 1990?
connect-reddis listens to redis for the error event
./lib/connect-redis.js
self.client.on('error', function () { self.emit('disconnect'); });
So after creating the store, listen to the disconnect event
var store = new RedisStore({
host: 'localhost',
port: 6379,
db: 2,
pass: 'RedisPASS'
});
store.on('disconnect', function(){
console.log('disconnect');
});
Using connect-mongo to handle session storage, I am getting a lot of empty sessions in my mongo database (using mongolab), probably on every request (or very often). Here is an example:
{
"_id": "oThxLcz3PtztDWVxcKZvoADC",
"session": "{\"cookie\":{\"originalMaxAge\":null,\"expires\":null,\"httpOnly\":true,\"path\":\"/\"},\"passport\":{}}",
"expires": {
"$date": "2013-07-10T22:45:52.383Z"
}
}
As you can see, there is no user defined in the object. I read in another issue that it might be a difference of time between the server and the client so the cookies are expiring instantly but that's not the case, the sessions are working fine. Any idea how to have only a line every time somebody logs in or signs up? I don't want to pay extra storage on my database.
For info, I am using 2 dynos on Heroku and mongolab and my set up is:
app.use(express.session({ secret: 'secretkey',
store: new MongoStore({
url:'mongodb://db:user#link:27468/dbname'
})
}));
I can't help you debug that issue, but I can suggest using connect-redis instead. The Redistogo nano instance on Heroku is free, and should support automatically expiring unused sessions so that you won't need a pricier option.
i m creating application, using nodejs (0.8.15), express (>3.0) framework, socket.io and mongodb for register users.
1) Everyone knows that it is easy to create simple chat using only socket.io, without mongodb (or other). So, in this case where stores the messages? in session?
2) this second question is VERY IMPORTANT for me
i don't understand: why are MongoStore there for sessions? i read that it is "SessionStorage for connect's session middleware/Connect". As i know session destroys when user go away from site. So why do people store it and use require('connect-mongo') ??
var config = require('../config'),
express = require('express'),
MongoStore = require('connect-mongo'),
server = express.createServer();
server.configure(function() {
server.use(express.logger());
server.use(express.methodOverride());
server.use(express.static(config.staticPath));
server.use(express.bodyParser());
server.use(express.cookieParser());
server.use(express.session({
store: new MongoStore({
db: config.db
}),
secret: config.salt
}));
});
code i have just found. it seems it is for express 2.x
3) what is the difference between connect-mongo and connect-mongodb libs?
https://github.com/masylum/connect-mongodb
https://github.com/kcbanner/connect-mongo
4) do i have to make MongoStore for cookies and sockets?
thank you!
1) Nowhere? Server receives message, broadcasts it and forgets about it. Who sais anything about storing? Server can be just a proxy.
2) Because sessions don't have to be destroyed once a user leaves the site. For example consider a shop. You go to a shop, you put some things in your basket and then you close the browser. After 3 days you go back and you still see your items in the basket. This can only be done with sessions stored in DB ( at least in a reliable way ).
It really depends on your needs. In my case I hardly ever use DB based sessions. I use in-memory storages like Redis.
3) Who knows? Might be some subtle differences, I can't see any obvious.
4) No. As I said: it depends on your needs. If you want sessions to be active only when user is actually viewing the page, then you can stick with any in-memory storage. Or you might not need sessions at all, in which case you don't have to use it at all.
1) If you don't want to use a database for your chat, you should store the messages into a simple hash. But keep in mind, if you restart your node application, they will be lost.
2) MongoStore (or RedisStore) allows you to store the Express sessions into MongoDB/Redis instead of using the MemoryStore, which is not designed for a production environment.
(by the way the code you've found is for Express 2.x)
3) The two looks similar. Personnally I use connect-mongo.
4) No. Express handles the session cookies for you, and Socket.IO the sockets. If you want to work with MongoDB, you should try Mongoose http://mongoosejs.com/
First Problem
I'm trying to figure out sessions, stores, authorization, and redis. If it's important, I am using Express#3.0.0rc4. I understand that I have two options to put my RedisStore(). Which one do I use? Do I use both?
express.session({secret: 'secret', key: 'key', store: new RedisStore()});
io.set('store', new RedisStore());
I already have node.js, express, and socket.io running. So now I'm trying to implement redis but I don't know how to implement authorization using sessions/stores and I don't know how to write to the redis database. I haven't found any documentation on this. I found a site that talks about sessions and stores using socket.io and express but no redis, and another one that talks about sessions and stores using all three, but it doesn't use io.set('store', ...).
I also don't know if I should use two different stores, one for express and one for socket.io, or if I should just use one. Look at the example for clarification:
//Redis Variables
var redis = require('socket.io/node_modules/redis');
var RedisStore = require('socket.io/lib/stores/redis');
var pub = redis.createClient();
var sub = redis.createClient();
var client = redis.createClient();
var redis_store = new RedisStore({
redisPub: pub,
redisSub: sub,
redisClient: client
});
app.configure(function(){
//...code goes here...
app.use(express.session({
secret: 'secret',
key: 'key',
store: redis_store //Notice I'm using redis_store
}));
//...more code...
});
io.configure(function(){
io.set('store', redis_store); //Notice it's the same RedisStore() being used
});
Do I use the same RedisStore() for each? Do I create seperate ones for each? Do I just use express or socket.io? What I really want is to be able to authenticate clients (I assume that's done through sessions) and have them update the redis database when they connect - keeping a log of when people accessed my site. Which leads to my second problem.
Second Problem
So I have no idea how to access and edit the redis database from this point. I haven't been able to test this because of my first problem but I assume it would be something like this:
io.sockets.on('connection', function(socket){
var session = socket.handshake.session;
redis.put(session);
});
I also haven't seen any documentation on how to update a redis database from within node.js so I highly doubt that redis.put() is the correct terminology haha. I have visited redis's website but I can't find commands for node.js. Just commands for using regular redis from the command line. Anyways, if someone could at least point me in the right direction that would be great. Thanks. :)
Express and Socket.IO have their own integration with Redis for session management, as you've seen. It is designed as a blackbox integration, the idea being that the session store implementation is independent from the rest of your code. Since it's independent, that means you can't go in and use express or socket.io to access Redis directly. You'll need to add a regular redis client like node_redis. The benefit is you don't have to worry about making all those redis calls yourself, instead you'll be interacting with express or socket.io's session store interfaces.
So in your #1 case, you could pass in a single new instance of RedisStore, not two new ones as you've done. Or you could follow your second link and have socket.io listen through express. In that case it would integrate with express session management. That's why you don't see the extra io.set('store') call in that example.
It'll probably seem redundant to you, but try to think of RedisStore as a special client designed only for session management. Even thought RedisStore probably relies on something like node_redis, you shouldn't try to access it. You have to include another library for accessing your redis database directly, assuming you wanted to store other non-session items in redis in the first place.
I'm using Node's Express w/ Connect middleware. Connect's memory session store isn't fit for production:
Warning: connection.session() MemoryStore is not designed for a production environment, as it will leak memory, and obviously only work within a single process.
For larger deployments, mongo or redis makes sense.
But what is a good solution for a single-host app in production?
Spent the day looking into this. Here are the options I've discovered. Requests/second are performed via ab -n 100000 -c 1 http://127.0.0.1:9778/ on my local machine.
no sessions - fast (438 req/sec)
cookieSession: requires no external service, minor speed impact (311 req/sec) - fastest, sessions will expire with the cookie (customised by maxAge)
connect-redis: requires redis server, large speed impact (4 req/sec with redis2go and redisgreen) - faster than mongo, sessions will be deleted after a while (customised by ttl)
connect-mongo - requires mongodb server, large speed impact (2 req/sec with mongohq) - slower than redis, requires manual clear_interval to be set to cleanup sessions
Here is the coffeescript I used for cookieSession:
server.use express.cookieSession({
secret: appConfig.site.salt
cookie: maxAge: 1000*60*60
})
Here is the coffeescript I use for redis:
RedisSessionStore ?= require('connect-redis')(express)
redisSessionStore ?= new RedisSessionStore(
host: appConfig.databaseRedis.host
port: appConfig.databaseRedis.port
db: appConfig.databaseRedis.username
pass: appConfig.databaseRedis.password
no_ready_check: true
ttl: 60*60 # hour
)
server.use express.session({
secret: appConfig.site.salt
cookie: maxAge: 1000*60*60
store: redisSessionStore
})
Here is my coffeescript for mongo:
server.use express.session({
secret: appConfig.site.salt
cookie:
maxAge: 100*60*60
store: new MongoSessionStore({
db: appConfig.database.name
host: appConfig.database.host
port: appConfig.database.port
username: appConfig.database.username
password: appConfig.database.password
auto_reconnect: appConfig.database.serverOptions.auto_reconnect
clear_interval: 60*60 # hour
})
})
Now of course, the remote redis and mongo databases will be slower than their local equivalents. I just couldn't get the local equivalents working, especially considering the installation and maintenance time for me was far more than what I was willing to invest when compared with hosted remote alternatives, something I feel is true for others too hence why these hosted remote database services exist in the first place!
For local database benhmarks, see #Mustafa's answer.
Happy for someone to edit this answer to add their local database benchmarks to the mix.
Since the accepted answer is only connecting to remote hosts, it is obvious that it will be always slower than localhost. Even if it is the next computer in your home, it would take milliseconds to read from that computer, but local memory takes only nanoseconds. You should compare them by using locally installed servers.
Here are my results from my local pc: You see, redis is almost as fast as in-memory in under high load. You can clone my the repo that these test codes are available: https://github.com/mustafaakin/express-session-store-benchmark
Concurrency: 1
none 4484.86 [#/sec]
memory 2144.15 [#/sec]
redis 1891.96 [#/sec]
mongo 710.85 [#/sec]
Concurrency: 10
none 5737.21 [#/sec]
memory 3336.45 [#/sec]
redis 3164.84 [#/sec]
mongo 1783.65 [#/sec]
Concurrency: 100
none 5500.41 [#/sec]
memory 3274.33 [#/sec]
redis 3269.49 [#/sec]
mongo 2416.72 [#/sec]
Concurrency: 500
none 5008.14 [#/sec]
memory 3137.93 [#/sec]
redis 3122.37 [#/sec]
mongo 2258.21 [#/sec]
The session used pages are very simple pages;
app.get("/", function(req,res){
if ( req.session && req.session.no){
req.session.no = req.session.no + 1;
} else {
req.session.no = 1;
}
res.send("No: " + req.session.no);
});
Redis store config:
app.use(express.session({
store: new RedisStore({
host: 'localhost',
port: 6379,
db: 2,
}),
secret: 'hello'
}));
Mongo store config:
app.use(express.cookieParser());
app.use(express.session({
store: new MongoStore({
url: 'mongodb://localhost/test-session'
}),
secret: 'hello'
}));
Another good option is memcached. The session states are lost if memcached is restarted, but there is virtually never any reason to do that. You can leave the cache running all the time even when you restart your app server. Access to the session data is virtually instantaneous and memcached will run happily with whatever (appropriate) amount of memory you give it. And I've never seen memcached crash (on Linux).
https://github.com/elbart/node-memcache
Things to keep in mind about memcached generally:
Never have whitespace in your cache keys
Be aware that there is a maximum cache key length, including any namespace prefix you might use. If your cache key is too long, use a 1-way hash of it instead.
Neither of these should be an issue with session storage; just with generalized caching.
I've gone with a MongoDB session store using connect-mongo.
Install with npm install connect-mongo and replace the existing MemoryStore with
app.use(express.session({ store: new MongoStore({ db: 'some-database' }) }));
It manages the database side of sessions automatically.
I would still use Redis even for local development. This is helpful because it stores the session even when you restart the Node application, keeping your browser session logged in. Redis by default saves the session in memory, same as connect's memory store is simple to configure (I just run it in screen along with my node apps) can support multiple applications if you just use a different database or session value in the configuration.
I'm just exploring node.js myself, but if you don't need to store a lot of information in the session object -- you might want to explore secure cookies.
Secure cookies store session information as part of the cookie that the browser stores and forwards with each request. They are encrypted to prevent a user from forging a valid cookie.
The advantage is that you don't have to maintain state at the server -- this solution scales well and is simple to implement.
The disadvantage is that you can only store up to about 4KB and that data gets sent to the server on every request (But you can have multiple fictitious domains pointing at your server so you don't impose that baggage on publicly visible static content, for example).
Searching the web it seems like there are at least two implementations of secure cookies for node.js. Not sure how production ready they are, though:
https://github.com/benadida/node-client-sessions/blob/master/lib/client-sessions.js
https://github.com/caolan/cookie-sessions
I appreciate that this is an old question, but I came across it while searching for a solution to a similar problem. I had already decided to use memcached for session storage on Linux (with connect-memcached), but I also required the ability to run on Windows. I spent a while trying to find an in-memory session storage for a single-process node app. Redis and Memcached don't appear to be well-supported on Windows, and I didn't want the additional complexity of their installation.
I found session-memory-store in another Stack Overflow thread, which looks good but significantly increased the size of my dependencies.
Finally, I found memorystore in the documentation for express-session. I had missed it originally due to the similarly of its name to the default MemoryStore, but it's exactly what I was looking for:
express-session full featured MemoryStore module without leaks!
I'm now using connect-memcached when running in a cluster (on Linux only), and memorystore when running a single process (on Linux or Windows).
I thought it worth posting this as another answer, just in case anyone else makes the mistake of missing memorystore as I initially did.
Check out my benchmarks at https://github.com/llambda/express-session-benchmarks showing comparisons of different session implementations.