How to create Kafka topics for MongoDB updates using NodeJS? - node.js

Question

I am working on a large project where multiple technologies are used. We are processing large amount of data in every day.
We are using MSSQL and MongoDB to store data from the client. SQL server is used to store the data as RDBMS while MongoDB is used to store data like a view to speed up the fetching. ColdFusion is used as a backed and cfthreads are used to update the MongoDB. There is a huge memory leakage due to the cfthread. So we found a solution to use Kafka in a server, where MongoDB can be updated using the kafka.
Producer and consumer is created using kafka-node in the NodeJS. Kafka and ColdFusion are in different servers. After every sql updates, same changes should be added or updated in the MongoDB. Can we do this synchronization using Kafka?
How can I call the Kafka producer from a ColdFusion server?
Is prioritized topics can be created using a producer?
Do I need to create a topic for every database actions?
Code I have used to create a producer and consumer.
producer.js
var kafka = require('kafka-node');
var Producer = kafka.Producer,
client = new kafka.KafkaClient(),
producer = new Producer(client);
producer.on('ready', function () {
console.log('Producer is ready');
});
producer.on('error', function (err) {
console.log('Producer is in error state');
console.log(err);
})
const createProducer = async(req,res,next) => {
var sentMessage = JSON.stringify(req.body.messages);
payloads = [
{ topic: req.body.topic, messages: sentMessage, partition: 0}
];
producer.send(payloads, function( err, data) {
// res.json(data);
})
res.send({
message: 'Successfully send data from producer',
payloads: payloads
})
}
consumer.js
var kafka = require('kafka-node');
const createConsumer = async(req,res) =>{
var topic = req.params.topic;
Consumer = kafka.Consumer,
client = new kafka.KafkaClient(),
consumer = new Consumer(client,
[{ topic: topic, offset: 0}],
{
autoCommit: false
}
);
res.send({
message: "consumer created",
topic: topic
})
consumer.on('message', function (message) {
console.log('message : ',message);
});
consumer.on('error', function (err) {
console.log('Error:',err);
})
consumer.on('offsetOutOfRange', function (err) {
console.log('offsetOutOfRange:',err);
})
}
And I requesting to provide an example.