Concurrent writing to redis in node.js - node.js

Question

In my node.js application I read messages from AWS Kinesis stream, and I need store all messages, for last minute in cache (Redis). I run next code in one node worker:
var loopCallback = function(record) {
var nowMinute = moment.utc(record.Data.ts).minute();
//get all cached kinesis records
var key = "kinesis";
cache.get(key,function (err, cachedData) {
if (err) {
utils.logError(err);
} else {
if(!cachedData) {
cachedData = [];
} else {
cachedData = JSON.parse(cachedData);
}
//get records with the same minute
var filtered = _.filter(cachedData, function (item) {
return moment.utc(item.ts).minute() === nowMinute;
});
filtered.push(record.Data);
cache.set(key, JSON.stringify(filtered), function (saveErr) {
if (saveErr) {
utils.logError(saveErr);
}
//do other things with record;
});
}
});
};
Most of the records (few dozens) I receive exactly in the same moment. So when I try to save it, some records are not stored.
I uderstand it happen due to race condition.
Node reads old version of array from Redis and overwrites array while it writes another record to cache.
I have read about redis transactions, but as I understand it will not help me, because only one transaction will be completed, and other will be rejected.
There is way to save all records to cache in my case?
Thank you

Answer 1

You could use a sorted set, with the score being a Unix timestamp
ZADD kinesis <unixtimestamp> "some data to be cached"
To get the elements added less than one minute ago, create a timestamp for (now - 60 seconds) then use ZRANGEBYSCORE to get the oldest element first:
ZRANGEBYSCORE myzset -inf (timestamp
or ZREVRANGEBYSCORE if you want the newest element first:
ZRANGEBYSCORE myzset -inf (timestamp
To remove the elements older than one minute, create a timestamp for (now - 60 seconds) then use ZREMRANGEBYSCORE
ZREMRANGEBYSCORE myzset -inf (timestamp