How to use Redis cache for pagination? - node.js

Question

I want to do pagination with redis cache, i'am using Node.js as my back-end and i'am using npm
redis-scanner package for scanning through the keys .
Here i'am unable to go to a particular key and fetch next 15 keys from redis cache. How do i do it?

Answer 1

You can use ioredis like so:
const Redis = require("ioredis");
async getByCursor({ cursor }) {
const db = new Redis();
const stream = db.scanStream({ count: 5 }); // count = number of items to return per page
try {
const scan = () =>
new Promise((resolve, reject) => {
let items = [];
stream.on("data", (page) => {
items = [...items, page]; // page = ['key1', 'key2', etc.]
});
stream.on("end", () => {
resolve(items); // items = [['key1', 'key2'], ['key3', 'key4']]
});
});
const keys = await scan();
return Promise.all(
keys[cursor].map(async (key) => JSON.parse(await db.get(key)))
);
} catch (e) {
console.error(e);
return Promise.resolve([]);
}
}
or slightly faster:
async getByCursor({ cursor }) {
const stream = db.scanStream({ count: 5 });
try {
return new Promise((resolve, reject) => {
let pageCount = 0;
stream.on("data", (page) => {
pageCount = pageCount + 1;
if (pageCount == Number(cursor)) {
resolve(
Promise.all(
page.map(async (id) => unpack(await db.get(id)))
).catch((e) => {
console.error(e);
reject([]);
})
);
}
});
});
} catch (e) {
console.error(e);
}
Using cursors
Here cursor value 1 provided e.g. as param or query when calling your 'items-by-cursor' node route gives you the first 5 items, value of 2 the next page etc. Note that the count is just an estimate, i.e. you can't rely on it always returning strictly 5 keys (see the documentation link below for details). You can check https://react-query.tanstack.com/examples/load-more-infinite-scroll for a frontend solution to obtain the relevant cursors to be sent to the route.
Note on performance
Fetching the key-value pairs via a stream is non-blocking in comparison to getting all items via all keys by calling db.get('*') and/or filtering the result in your route. Therefore this implementation is recommended for production.
Further ioredis documentation
https://github.com/luin/ioredis#streamify-scanning