Using Redis Pattern Subscribe - node.js

Question

I am working on a NodeJS application.
I am new to redis, I just installed it yesterday, but I'd like to be able publish this data and subscribe to it from another process.
Suppose I have the following data:
var Exchanges = [
{
_id: 'tsx',
name: 'Toronto Stock Exchange',
data: {
instrument: [
{
symbol: 'MBT'
markPrice: 0,
},
{
symbol: 'ACQ'
markPrice: 0,
}
],
orderBooks: [
{
symbol: 'MBT',
bids: [],
asks: [],
},
{
symbol: 'ACQ',
bids: [],
asks: [],
}
],
trades: [
{
timestamp: "2014-11-06T20:53:00.000Z",
symbol: "MBT",
side: "Buy",
size: 0,
price: 352.80,
},
{
timestamp: "2014-11-06T20:53:00.000Z"
symbol: "ACQ",
side: "Sell",
size: 0,
price: 382.90,
}
],
},
},
{
_id: 'nyse',
name: 'New York Stock Exchange',
data: {
instrument: [
{
symbol: 'IBM'
markPrice: 0,
},
{
symbol: 'WMT'
markPrice: 0,
}
],
orderBooks: [
/* Orderbook Data Here */
],
trades: [
/* Trades Data Here */
],
},
}
];
I am saving this with something like:
exchange.websocket_conn.on('message', function (updateData) {
// Use 'updateData' (a diff) to update exchange.data object.
// ...
// Then
redisClient.hmset(exchange._id.toString(), exchange.data);
redisClient.publish(exchange._id.toString(), exchange.data);
});
This works and does publish the data, however I've been reading about 'PSUBSCRIBE' and I'm wondering if this can be broken down a bit further:
I'd like to be able to do something like:
someOtherRedisClient.subscribe('tsx');
// Receive All Data from the Exchange Whenever Anything Changes.
someOtherRedisClient.subscribe('tsx.instrument');
// Receive 'Instrument' array of All Instruments on Exchange when any Instrument Changes.
someOtherRedisClient.subscribe(tsx.instrument:MBT');
// Get Back Only the 'MBT' Instrument Whenever It Changes.
Can the 'Pattern Subscribe' function be used to achieve this?
Thanks,

Answer 1

I'd break down that one big JSON into many JSON's, one for each type of content, e.g.
Level 1 (e.g. last trade price, best bid/ask)
Order book
Trades
and create a seperate topic for each, e.g.
mktdata:tsx:level1:MBT would have the market price for MBT on the TSX exchange
mktdata:tsx:orderbook:MBT would be the order book for MBT on the TSX exchange
mktdata:tsx:trades:MBT could be all the trades but more likely, due to volume, would be better used as a notification to the client to make a seperate query to get the last N trades required from a list
You don't say how many instruments you're writing into Redis, or how many different client applications are consuming the data, but assuming you've not got a huge number of instruments you could indeed use PSUBSCRIBE to get all orderbook updates across the exchange, etc. Given a list of symbols you can also subscribe a long list of channels (e.g. mktdata:tsx:level1:MBT mktdata:tsx:orderbook:MBT mktdata:tsx:level1:ACQ), which can run to tens/hundreds without problem.