New to node coming over from asp.net so was curious about the best modules/frameworks to implement caching in node.js. I would like to cache some expensive db queries so looking for an easy way to cache db results or entire http page output to memory.
In .net it would be using either httpcache or page output caching. Is there an equivalent in the node world?
Should I just set up a local memcached or redis server to accomplish this?
If you are using express then there is a settings option for enabling view cache, also since express is built on top of connect you can try to look at connect-cache module. For caching static stuff you can take a look at node-static for example. Caching database queries in memcached or redis may be highly dependent on your data structure, but I would probably choose redis since it supports more advanced data structures which may be handy.
Related
What is Azure REDIS Cache? How to use it? When to use and when not to use it? Can I use it for a simple c# application which accesses data from MS-SQL server?
What is best source to get started with Azure Redis cache?
Azure Redis Cache offering is a Software as a service (SAAS) offering of the opensource Redis cache. You should be able to get started with the resource below:
https://azure.microsoft.com/en-in/services/cache/
https://azure.microsoft.com/en-in/documentation/services/redis-cache/
http://redis.io/documentation
There is good support for C# via 'StackExchange.Redis' nuget package. However, please explore if your application needs or will benefit from a cache solution before you get to the implementation.
Azure Cache for Redis provides an in-memory data store based on the Redis software. Redis improves the performance and scalability of an application that uses backend data stores heavily
In my application I have stored around 5 millions key/value data with each key containing around 800 records in json format under Redis cache. Called the Redis using stackexchange library(1.2.6) since the application does not support the latest version
You can go through the below links
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-overview
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-redis-samples
I'm looking into using gcloud node api to access the datastore api but was curious if it supported query caching in a similar manner to ndb? If not, what's the best way to make sure repeated queries are cached?
As far as I know, gcloud-node isn't planning to be a full-on ORM (like ndb is for Python). Also, as Patrick Costello noted in the comments above, NDB doesn't cache query results, but individual entities instead.
I think if you want caching of query results (or individual entities), you'd have to manually cache these by running your own Memcache server (http://memcached.org/) and interacting with it using memcached (https://www.npmjs.com/package/memcached)
I ended up using NsqlCache-datastore which is integrated into gstore-node. Guide: https://medium.com/google-cloud/how-to-add-a-cache-layer-to-the-google-datastore-in-node-js-ffb402cd0e1c
Looks like I can use the memcache app engine service accessible through this node library:
https://github.com/GoogleCloudPlatform/appengine-nodejs
I'm evaluating GridGain as a caching solution for my PHP based web sites. I'll be using the Memcached interface. It would be nice to be able to access different caches (i.e. a replicated cache and a local cache) on the same node. From what I can find, however, it is not possible to select a cache when using the Memcached interface; it just uses the default cache. The documentation on using the Memcached interface doesn't mention anything about which cache it uses or if a cache can be selected. Is there a way to specify the cache to use when using the Memcached interface that I have missed or can I only use the default cache?
I think the easiest way is to use the default cache.
By looking at code on GridGain server side, it looks like you could still specify cache-name in the "extra" space of memcached protocol, but it is not obvious how to achieve it from the memcached client side.
If you can switch to GridGain Client API, then you can freely use any cache you have configured.
I have a nodejs project that using couchbase as database.
Just wonder if I store the temporary data in
1.redis
or in
2.couchbase directly.
As I know there is socket delay for couchbase, I think store temporary data in redis while store the permanent data in couchbase is better.
Is there any person has the experience on this?
Your comment welcome
I'm a big Redis fan, but in this situation I would use Couchbase only.
Couchbase is rather efficient, and comparable to the performance of memcached when the working set of your data fits in memory. Most of the time, an extra caching layer on top of Couchbase is not useful.
That said, if you really need a caching layer, or simply some storage for temporary data, you can simply create a memcached bucket hosted in the Couchbase cluster. So you would have an "eventually persistent" bucket for your persistent data, and a memcached bucket for the temporary data.
The bucket types are described here:
http://docs.couchbase.com/couchbase-manual-2.5/cb-admin/#data-storage
In that context, adding Redis as a extra storage layer does not really make sense.
Couchbase has a managed cache built into it, even for Couchbase buckets. So it already has a caching layer and adding another one on top just sounds superfluous.
I am not sure what you mean by a socket delay in Couchbase. Can you perhaps explain more about that? That is not something I have ever seen before and sticks out as suspect to me. I would try and troubleshoot this and figure out what that is before looking to add redis to the mix and have yet another layer to manage and code against. Without know more about the socket delay, it is difficult to make more recommendations.
It's an old question, but I'll have my take at it as well, if nothing else then for the people coming across it via google, just as I did.
I agree with he accepted answer, in that CouchBase has the most recently used Documents in RAM. In that aspect, it does the same as Redis. The advantage of CouchBase is of course that the data can reliably spill over the RAM limit, and the server disk limit, automatically, by adding more nodes.
However, I have a project where I am considering using Redis along side CouchBase. It's basically thought as a caching server, but for the "calculated" items. Such as html-snippets or other things. CouchBase is a fantastic document store, but making lists and other structures, doesn't come that easy, especially not without a lot of views. So I'm thinking to use Redis as a temporary datastore for the ad-hoc data manipulation needed, and CouchBase as the main datastore.
I recently wanted to process some large data in nodejs and then put them into mongodb, at first I tried to process data in memory (javascript arrays),but when I decide to use node cluster, I figured out that by using node cluster it is not good way to use cache object. however, it seems impractical solution to store data in arrays even for short time. I think it would be better to store it in some memory databases such as redis.
so my first question is whether I can use redis via node cluster or not? and whether redis is the best solution or not?
thanks in advance.
When using the cluster module, an external data store is by far the simplest way of sharing state. Think of each worker as an individual client accessing the data store. Redis happens to be perfectly fit for the job, but the same can be said for just about any database. For short term storage, redis and memcached are both popular solutions. For arrays, redis beats memcached because it has an array like type that you can push to and pop from.
An extremely popular library for Node.js connecting to redis is node-redis. I've personally used it and highly recommend the API.