I'am new to redis, i need help to get notify to my running server if any chages/transaction happen in redis(either from bakend also). The server which is in running state is created using node.js
Use the keyspace notification system. This uses the pub sub layer of redis
Steps :
Set this parameter -notify-keyspace-events in redis.conf file
By default its empty
This will enable capture of almost all DML events
I'm not sure about how to get notification about transaction, but here is a working code (using ioredis) to get notified about any change in redis:
const redisClient = const redisClient = new Redis({
host: 'localhost',
port: 20000,
lazyConnect: true, // because i will try to connect manually in the next line
connectTimeout: 1000,
})
await redisClient.connect()
// "AKE" === listen to all events of all redis-operations
await redisClient.config('SET', 'notify-keyspace-events', 'AKE')
await redisClient.psubscribe([`__key*__:*`])
redisClient.on(`pmessage`, (_pattern, redisKey, redisOpeation) => {
const redisKey = redisKey.replace('__keyspace#0__:', '')
if(redisKey === 'cool-redis-key' && redisOpeation === 'set'){
console.log(`there was a "set" operation on "cool-redis-key" key`)
}
})
Addtional info:
example: https://stackoverflow.com/a/58687129/806963
activate notifications: https://redis.io/topics/notifications
Related
So, if I use my DeepStream server as the following
const {Deepstream} = require('#deepstream/server')
const server = new Deepstream({
server.start()
it's working just fine I can connect to it from my frontend app like the following
const {DeepstreamClient} = require('#deepstream/client')
const client = new DeepstreamClient('192.168.88.238:6020')
client.login()
but If I add MongoDB storage instance or RethinkDB
NPM - RethinkDB
const {Deepstream} = require('#deepstream/server')
const server = new Deepstream({
storage: {
name: 'rethinkdb',
options: {
host: 'localhost',
port: 28015
}
}
})
// start the server
server.start()
I get the following error message when trying to reach my ds server.
(I've also tried to connect via WSS:// instead of WS://)
So hi everybody who has the same problem as me...
I figured it out!
So first of all what the npm packages documentation says from the usage of the Mongo DB driver is completely out of data
so what they say how should u use the npm package :
var Deepstream = require( 'deepstream.io' ),
MongoDBStorageConnector = require( 'deepstream.io-storage-mongodb' ),
server = new Deepstream();
server.set( 'storage', new MongoDBStorageConnector( {
connectionString: 'mongodb://test:test#paulo.mongohq.com:10087/munchkin-dev',
splitChar: '/'
}));
server.start();
INSTEAD OF ALL THIS!
You ain't really need the 'deep stream.io-storage-MongoDB' because it's an old module (based on: ), and u don't really need to use this way...
The correct usage of the MongoDB connector :
const {Deepstream} = require('#deepstream/server');
const server = new Deepstream({
storage: {
name: "mongodb",
options: {
connectionString: MONGO_CONNECTION_STRING ,
splitChar: '/'
}
}
});
server.start();
or you can also create a config. yaml file from all this :
storage:
name: mongodb
options:
connectionString: 'MONGO_CONNECTION_STRING'
# optional database name, defaults to `deepstream`
database: 'DATABASE_NAME'
# optional table name for records without a splitChar
# defaults to deepstream_docs
defaultCollection: 'COLLECTION_NAME'
# optional character that's used as part of the
# record names to split it into a table and an id part
splitChar: '/'
and pass it to the deep stream constructor as below:
const {Deepstream} = require('#deepstream/server');
const server = new Deepstream('config.yaml');
server.start();
When i try to connect my Nodsjs application to RedisCloud on Heroku I am getting the following error
Redis::CannotConnectError: Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)
I have even tried to directly set the redis URL and port in the code to test it out as well. But still, it tried to connect to the localhost on Heroku instead of the RedisCloud URL.
const {Queue} = require('bullmq');
const Redis = require('ioredis');
const conn = new Redis(
'redis://rediscloud:mueSEJFadzE9eVcjFei44444RIkNO#redis-15725.c9.us-east-1-4.ec2.cloud.redislabs.com:15725'
// Redis Server Connection Configuration
console.log('\n==================================================\n');
console.log(conn.options, process.env.REDISCLOUD_URL);
const defaultQueue = () => {
// Initialize queue instance, by passing the queue-name & redis connection
const queue = new Queue('default', {conn});
return queue;
};
module.exports = defaultQueue;
Complete Dump of the Logs https://pastebin.com/N9awJYL9
set REDISCLOUD_URL on .env file as follows
REDISCLOUD_URL =redis://rediscloud:password#hostname:port
import * as Redis from 'ioredis';
export const redis = new Redis(process.env.REDISCLOUD_URL);
I just had a hard time trying to find out how to connect the solution below worked for me.
Edit----
Although I had been passed the parameters to connect to the Redis cloud, it connected to the local Redis installed in my machine. Sorry for that!
I will leave my answer here, just in case anyone need to connect to local Redis.
let express = require('express');
var redis = require('ioredis');
pwd = 'your_pwd'
url = 'rediss://host'
port = '1234'
redisConfig = `${url}${pwd}${port}`
client = redis.createClient({ url: redisConfig })
client.on('connect', function() {
console.log('-->> CONNECTED');
});
client.on("error", function(error) {
console.error('ERRO DO REDIS', error);
});
Just wanted to post my case in case someone has the same problem like me.
In my situation I was trying to use Redis with Bull, so i need it the url/port,host data to make this happened.
Here is the info:
https://devcenter.heroku.com/articles/node-redis-workers
but basically you can start your worker like this:
let REDIS_URL = process.env.REDISCLOUD_URL || 'redis://127.0.0.1:6379';
//Once you got Redis info ready, create your task queue
const queue = new Queue('new-queue', REDIS_URL);
In the case you are using local, meaning 'redis://127.0.0.1:6379' remember to run redis-server:
https://redis.io/docs/getting-started/
I'm trying to create an API using nodeJS, express and azure-mobile-apps to do some data synchronisation between an Ionic3 mobile app (which use an SQLite local database) and a Microsoft SQL Database.
My API has to create a synchronisation connection for each mobile application. Each application will be linked to a distant database. For example, if user_01 wants to synchronise his data, he's going to be linked to his client_01 database. So each time it'll have to, the API will create a new process running on a different port.
here is an example : https://zupimages.net/up/19/36/szhu.png
The problem is that i'm not able to create more than one connection with azure-mobile-apps. The first one always works, but the second, third etc are still using the first connection that i have instantiated. I've looked into the app stack and everything seems fine.
Is that an issue with azure-mobile-app, or did I misunderstand something with express ?
Thanks for your responses !
var azureMobileApps = require('azure-mobile-apps');
var express = require('express');
module.exports = {
async createConnection(client) {
try {
let app = express();
mobileApp = azureMobileApps({
homePage: true,
swagger: true,
data: {
server: '****',
user: client.User,
password: client.Password,
port: '1443',
database: client.Database,
provider: 'mssql',
dynamicSchema: false,
options: {
encrypt: false
}
}
});
await mobileApp.tables.import('./tables');
await mobileApp.tables.initialize();
app.listen(global.portCounter);
app.use(mobileApp);
console.log(app._router.stack);
console.log('Listening on port ',global.portCounter);
global.portCounter++;
} catch (error) {
console.log(error);
}
}
}
It's working now. The thing is, it's impossible to do multiple connection with the azure-mobile-apps SDK for nodeJS.
I had to use worker-thread which seems to isolate the memory in a sub-proccess.
Hope it can help somebody one day
We have a client which connects to the server. The server can be accessed both: http and https.
Https server contains CA certified SSL key.
Since the client is running behind a corporate proxy, we used the following command:
const events = this.proxy ? new EventSource(this.source, { proxy: this.proxy }) : new EventSource(this.source)
So, the current command works only when connecting to http://server.
If we try to connect to the https://server, the following error received:
Event { type: 'error', status: 400, message: 'Bad Request' }
So we tried to set:
const events = this.proxy ? new EventSource(this.source, {https: {proxy: this.proxy, rejectUnauthorized: false} } ) : new EventSource(this.source)
or
const events = this.proxy ? new EventSource(this.source, {https: {proxy: this.proxy} } ) : new EventSource(this.source)
In both cases, we got a TIME OUT error.
Is there something we are missing? How should we set https connection behind proxy?
The solution is to use 'global-agent' or 'global-tunnel-ng' packages:
// Support working behind a corporate proxy
const MAJOR_NODEJS_VERSION = parseInt(process.version.slice(1).split('.')[0], 10);
if (MAJOR_NODEJS_VERSION >= 10) {
// `global-agent` works with Node.js v10 and above. Proxy env should be defined "export GLOBAL_AGENT_HTTP_PROXY=http://127.0.0.1:8080"
require('global-agent').bootstrap()
} else {
// `global-tunnel-ng` works only with Node.js v10 and below. Uses npm proxy settings
require('global-tunnel-ng').initialize()
}
const events = new EventSource(this.source)
Read here for more info: https://www.npmjs.com/package/global-agent
I need to listen PostgreSQL on changes in real-time with Node-RED.
How can I do this?
I created trigger on new record in the table and notify this to 'changes' channel.
CREATE FUNCTION notify_trigger() RETURNS trigger AS $$
DECLARE
BEGIN
PERFORM pg_notify('changes', TG_TABLE_NAME || ',id,' || NEW.id );
RETURN new;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER watched_table_trigger AFTER INSERT ON users
FOR EACH ROW EXECUTE PROCEDURE notify_trigger();
But I don't know how to listen it from Node-RED. Could you help me please?
Maybe I can do it differently?
Have a look at this previous SO question:
MQTT Client subscribe to PostgreSQL DB Changes
It looks like Postgress supports Python based triggers which could be used to send a MQTT message which Node-RED could easily subscriber to.
I found a good solution for yourself with WebSocket. Look at example below:
var pg = global.get('pg'),
WebSocket = global.get('ws'),
config = {
user: 'user',
password: 'user',
host: 'somehost',
port: 1234,
database: 'somedb'
},
client = new pg.Client(config);
client.connect(function(err) {
if (err) node.error(err);
client.on('notification', function(msg) {
node.send(msg);
});
var query = client.query("LISTEN changes");
});
delete msg._session;
return msg;
Post your solution, I really want to know more ways to solve this.