How to get header agent or device/browser details in mqtt message with mosquitto broker. My mqtt code sample:
var mqtt = require('mqtt');
var client = mqtt.connect('mqtt://127.0.0.1:1883',{
username: 'xxxx',
password: 'xxxx'
});
client.on('connect', function (err,done) {
if(err){
console.log(err)
}else{
console.log("Connected...")
client.subscribe('test');
}
})
client.on('message', function (topic, message) {
// want to get the header details here.
})
You can not get anything other than the message payload and topic from a MQTT message because no other information is included in the message format. This is by design, in Pub/Sub messaging the only thing that matters in the topic and the payload, not who sent it.
The on('message',function(){}) callback can take a 3rd parameter which is the raw mqtt-packet object. You can see full list of data available in the doc here. But the only extra information is about duplicate status, qos and if the message is retained.
client.on('message',function(topic, message, mqtt-packet) {
...
});
If you need more information you need to manually include it in the message payload published by the client yourself.
Related
I'm trying to setup a MQTT Microservice using NestJS according to the docs.
I've started a working Mosquitto Broker using Docker and verified it's operability using various MQTT clients. Now, when I start the NestJS service it seems to be connecting correctly (mqqt.fx shows new client), yet I am unable to receive any messages in my controllers.
This is my bootstrapping, just like in the docs:
main.ts
async function bootstrap() {
const app = await NestFactory.createMicroservice(AppModule, {
transport: Transport.MQTT,
options: {
host: 'localhost',
port: 1883,
protocol: 'tcp'
}
});
app.listen(() => console.log('Microservice is listening'));
}
bootstrap();
app.controller.ts
#Controller()
export class AppController {
#MessagePattern('mytopic') // tried {cmd:'mytopic'} or {topic:'mytopic'}
root(msg: Buffer) {
console.log('received: ', msg)
}
}
Am I using the message-pattern decorator wrongly or is my concept wrong of what a NestJS MQTT microservice even is supposed to do? I thought it might subscribe to the topic I pass to the decorator. My only other source of information being the corresponding unit tests
nest.js Pattern Handler
On nest.js side we have the following pattern handler:
#MessagePattern('sum')
sum(data: number[]): number {
return data.reduce((a, b) => a + b, 0);
}
As #Alexandre explained, this will actually listen to sum_ack.
Non-nest.js Client
A non-nest.js client could look like this (just save as client.js, run npm install mqtt and run the program with node client.js):
var mqtt = require('mqtt')
var client = mqtt.connect('mqtt://localhost:1883')
client.on('connect', function () {
client.subscribe('sum_res', function (err) {
if (!err) {
client.publish('sum_ack', '{"data": [2, 3]}');
}
})
})
client.on('message', function (topic, message) {
console.log(message.toString())
client.end()
})
It sends a message on the topic sum_ack and listens to messages on sum_res. When it receives a message on sum_res, it logs the message and ends the program. nest.js expects the message format to be {data: myData} and then call the param handler sum(myData).
// Log:
{"err":null,"response":5} // This is the response from sum()
{"isDisposed":true} // Internal "complete event" (according to unit test)
Of course, this is not very convenient...
nest.js Client
That is because this is meant to be used with another nest.js client rather than a normal mqtt client. The nest.js client abstracts all the internal logic away. See this answer, which describes the client for redis (only two lines need to be changed for mqtt).
async onModuleInit() {
await this.client.connect();
// no 'sum_ack' or {data: [0, 2, 3]} needed
this.client.send('sum', [0, 2, 3]).toPromise();
}
The documentation is not very clear, but it seem that for mqtt if you have #MessagePattern('mytopic') you can publish a command on the topic mytopic_ack and you will get response on mytopic_res. I am still trying to find out how to publish to the mqtt broker from a service.
See https://github.com/nestjs/nest/blob/e019afa472c432ffe9e7330dc786539221652412/packages/microservices/server/server-mqtt.ts#L99
public getAckQueueName(pattern: string): string {
return `${pattern}_ack`;
}
public getResQueueName(pattern: string): string {
return `${pattern}_res`;
}
#Tanas is right. Nestjs/Microservice now listens to your $[topic] and answer to $[topic]/reply. The postfix _ack and _res are deprecated.
For example:
#MessagePattern('helloWorld')
getHello(): string {
console.log("hello world")
return this.appService.getHello();
}
Listens now on Topic: helloWorld
Replies now on Topic helloWorld/reply
Regarding ID
You should also provide an id within the payload (See #Hakier) and Nestjs will reply with an answer, containing your id.
If you don't have any id, there still won't be any reply but the corresponding logic will still trigger.
For example (Using the snipped from above):
your msg:
{"data":"foo","id":"bar"}
Nestjs reply:
{"response":"Hello World!","isDisposed":true,"id":"bar"}
Without ID:
your message:
{"data":"foo"} or {}
No reply but Hello World in Terminal
I was fighting with MQTT today and this helped me a little, but I had more problems and below you can see my findings:
Wrong way of configuration broker URL
In my case when I used non-local MQTT server I started with this:
const app = await NestFactory.createMicroservice(AppModule, {
transport: Transport.MQTT,
options: {
host: 'test.mosquitto.org',
port: 1883,
protocol: 'tcp',
},
});
await app.listenAsync();
but like you can read in a constructor of ServerMqtt they use url option only (when not provided it fallbacks to 'mqtt://localhost:1883'. While I do not have local MQTT it will never resolve app.listenAsync() which is resolved only on connect and will also not run any handler.
It started to work when I adjusted code to use url option.
const app = await NestFactory.createMicroservice(AppModule, {
transport: Transport.MQTT,
options: {
url: 'mqtt://test.mosquitto.org:1883',
},
});
await app.listenAsync();
Messages require id property
Second very weird problem was that when I used Non-nest.js Client script from #KimKern I had to register two MessagePatterns: sum and sum_ack:
#MessagePattern('sum')
sum(data: number[]): number {
return data.reduce((a, b) => a + b, 0);
}
#MessagePattern('sum_ack')
sumAck(data: number[]): number {
return data.reduce((a, b) => a + b, 0);
}
When I used console.log I discovered that the latter is being run but only when the first one is present. You can push the same message to the broker using mqtt cli tool to check it:
mqtt pub -t 'sum_ack' -h 'test.mosquitto.org' -m '{"data":[1,2]}'
But the biggest problem was that it didn't reply (publish sum_res).
The solution was to provide also id while sending a message.
mqtt pub -t 'sum_ack' -h 'test.mosquitto.org' -m '{"data":[1,2], "id":"any-id"}'
Then we could remove 'sum_ack' MessagePattern and leave only this code:
#MessagePattern('sum')
sum(data: number[]): number {
return data.reduce((a, b) => a + b, 0);
}
The reason for this was hidden inside handleMessage method of ServerMqtt which will not publish response from a handler if a message didn't have id.
TL/DR
Specify url to message broker using url option only and always provide id for a message.
I hope that will save some time to others.
Happy hacking!
Quoting the book, RabbitMQ in Depth:
A Basic.Ack request is sent to a publisher when a message that it has
published has been directly consumed by consumer applications on all
queues it was routed to or that the message was enqueued and persisted
if requested.
Confused with Has been directly consumed, does it mean when consumer send ack to broker publisher will be informed that consumer process message successfully? or it means that publisher will be notified when consumer just receive message from the queue?
or that the message was enqueued and persisted if requested. Is this like conjuction or publisher will be informed when either of those happens? (In that case publisher would be notified twice)
Using node.js and amqplib wanted to check what is happening actually:
// consumer.js
amqp.connect(...)
.then(connection => connection.createChannel())
.then(() => { assert exchange here })
.then(() => { assert queue here })
.then(() => { bind queue and exchange here })
.then(() => {
channel.consume(QUEUE, (message) => {
console.log('Raw RabbitMQ message received', message)
// Simulate some job to do
setTimeout(() => {
channel.ack(message, false)
}, 5000})
}, { noAck: false })
})
// publisher.js
amqp.connect(...)
.then(connection => connection.createConfirmChannel())
.then(() => { assert exchange here })
.then(() => {
channel.publish(exchange, routingKey, new Buffer(...),{}, (err, ok) => {
if (err) {
console.log('Error from handling confirmation on publisher side', err)
} else {
console.log('From handling confirmation on publisher side', ok)
}
})
})
Running the example, i can see following logs:
From handling confirmation on publisher side undefined
Raw RabbitMQ message received
Time to ack the message
As far as i see, at least by this log, publisher will be notified only when message was enqueued? (So having consumer acking the message will not influence publisher in any way)
Quoting further:
If a message cannot be routed, the broker will send a Basic.Nack RPC
request indicating the failure. It is then up to the publisher to
decide what to do with the message.
Changing the above example, where i only changed the routing key of the message to something that should not be routed anywhere (there are no bindings that would match routing key), from logs i can see only following.
From handling confirmation on publisher side undefined
Now i'm more confused, about what publisher is notified exactly here? I would understand that it receive an error, like Can't route anywhere, that would be aligned with quote above. But as you can see err is not defined and as side question even if amqplib in their official docs are using (err, ok), in no single case i see those defined. So here output is same like in above example, how one can differ between above example and un-routable message.
So what im up to here, when exactly publisher will be notified about what is happening with the message? Any concrete example in which one would use PublisherConfirms? From logging above, i would conclude that is nice to have it in cases where you want to be 100% sure that message was enqueued.
After searching again and again i have found this
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
The basic rules are as follows:
An un-routable mandatory or immediate message is confirmed right after the basic.return
transient message is confirmed the moment it is enqueued
Persistent message is confirmed when it is persisted to disk or when it is consumed on every queue.
If more than one of these conditions are met, only the first causes a
confirm to be sent. Every published message will be confirmed sooner
or later and no message will be confirmed more than once.
by default publishers don't know anything about consumers.
PublisherConfirms is used to check if the message reached the broker, but not if the message has been enqueued.
you can use mandatory flag to be sure the message has been routed
see this https://www.rabbitmq.com/reliability.html
To ensure messages are routed to a single known queue, the producer
can just declare a destination queue and publish directly to it. If
messages may be routed in more complex ways but the producer still
needs to know if they reached at least one queue, it can set the
mandatory flag on a basic.publish, ensuring that a basic.return
(containing a reply code and some textual explanation) will be sent
back to the client if no queues were appropriately bound.
I'm not entirely sure about the notification on ack/nack question, but check out the BunnyBus Node library for a simpler api and RabbitMQ management :)
https://github.com/xogroup/bunnybus
const BunnyBus = require('bunnybus');
const bunnyBus = new BunnyBus({
user: 'your-user',
vhost: 'your-vhost', // cloudamqp defaults vhost to the username
password: 'your-password',
server: 'your.server.com'
});
const handler = {
'test.event': (message, ack) => {
// Do your work here.
// acknowledge the message off of the bus.
return ack();
}
};
// Create exchange and queue if they do not already exist and then auto connect.
return bunnyBus.subscribe('test', handler)
.then(() => {
return bunnyBus.publish({event: 'test.event', body: 'here\'s the thing.'});
})
.catch(console.log);
I have a webhook setup, and I am able to receive messages and reply to them. I would like to have the responses sent by my webhook to have messagingServiceSid attached to them.
I didn't find on documentation a way to configure that for responses from my webhook, only for new SMS using
client.sendMessage({
messagingServiceSid: 'MG9752274e9e519418a7406176694466fa',
to: '+16518675309',
body: 'Phantom Menace was clearly the best of the prequel trilogy.'
}, function(err, message) {
console.log(message);
});
Is there something similar for this code? Is it doable through the UI?
app.post('/foo/bar/sms', twilio.webhook({
host:'gassy-ocelot-129.herokuapp.com',
protocol:'https'
}), function(request, response) {
var twiml = new twilio.TwimlResponse();
twiml.message('This HTTP request came from Twilio!');
response.send(twiml);
});
Images:
No messagingService on reply messages sent using twiml response
Message Detail view from logs
Twilio developer evangelist here.
As far as I'm aware, there's no way to reply to message from a message service with TwiML.
However, rather than using TwiML, you could just send the SMS back to your user from the REST API and return an empty <Response> to the incoming webhook. Something a bit like this:
app.post('/foo/bar/sms', twilio.webhook({
host:'gassy-ocelot-129.herokuapp.com',
protocol:'https'
}), function(request, response) {
// send the message from the message service
client.sendMessage({
messagingServiceSid: 'MG9752274e9e519418a7406176694466fa',
to: request.body.From,
body: 'Your message'
}, function(err, message) {
console.log(message);
});
// send empty TwiML response
var twiml = new twilio.TwimlResponse();
response.send(twiml);
})
Let me know if that helps at all.
If you receive an incoming SMS on a phone number currently set up to that Messaging Service (via the web ui or phone number REST), then the incoming requests will have MessagingServiceSid in the query string.
Hello I am using mosca mqtt broker I want to read the message that a client publish to a topic.
Is there any way to do that?
In published event I log the packet.payload and it prints only clientid and topic.
server.on('published', function(packet, client) {
console.log('Published', packet.payload);
});
Thank you
You could use the callback "on published " to log the published message either in a file or maybe in your database for accessing whenever you need it, like:
server.on('published', function(packet, client) {
// Do what you want with your message here
var msg = packet.payload.toString('utf8');
});
Be careful of internal messages though. this callback does not filter for you so you have to do it yourself.
Refer to this answer here
server.on('published', function(packet, client) {
console.log('Published: ', packet.payload.toString('utf8'));
});
I'm trying to send a message to GCM server to forward to another phone. The request keeps saying 400. I've checked the code close to a hundred times, but I'll put it up just in case. I also have changed the registration token parameter to registration_id and various others, along with changing the data to arrays etc. Just to be clear that the API key and Registration ID work, I can get a push notification from this Web site:
http://demo.androidhive.info/gcm_chat/push_test.php
Here's the code:
var gcm = require('node-gcm');
var gcmSender = new gcm.Sender('A......1234568AAA'); //my api key
var GCMmessage = new gcm.Message();
GCMmessage.addData("message", message);
GCMmessage.addData("from", from);
GCMmessage.addNotification('title', 'Alert!!!');
GCMmessage.addNotification('body', 'Abnormal data access');
GCMmessage.addNotification('icon', 'ic_launcher');
var regtoken = response.GCM.RegID; //data returned from another function
//Gives the RegID for a specific user
gcmSender.send(GCMmessage, {"to" : regtoken}, function(err, gcmResponse) {
if(err){
console.log(err);
} else {
console.log(gcmResponse);
console.log("message sent");
}
});
Ok it seems that it is best to send your gcm message directly using an npm module like 'request' or something similar. When I sent a message using that I received a very detailed error message telling me that 'from' is a reserved word. I'm not sure if that was the whole issue, but having a detailed error message made the code modification pretty easy.