I am trying to send messages to our elasticsearch endpoint, if I'm not using datastreams it works fine but when I am it stops working.
I have added the datastream=true option to logger constructor.
I just get no messages
I am not getting any errors or any logs on the console, just silently fails.
The logger init code
basicTransports.push(
new ElasticsearchTransport({
dataStream: true,
indexPrefix: indexPrefix,
indexSuffixPattern: 'YYYY.MM.DD',
client: new Client({
node: node,
auth: {
username: username,
password: password
},
})
})
)
an example log message is:
Logger.getInstance().info('connected successfully');
EDIT: I used wireshark to follow requests hoping I can point an error that is not reported by the client, but I actually couldn't find any messages going out of there... To be fair there are a lot of elasticsearch communications because we have a few services also talking at the same time, but a search for the index yielded nothing...
Related
I'm trying to put my REST API built in Node and with PostgresSQL to Heroku. So, I created my application to Heroku, and created his own database. At this point, I tryied to commit, and the build worked corretly. This until I tryied to make some calls to the API. If the api calls I do has the uncorrect method, or doesn't exists, it gives me the correct error, but when the call is correct, there is a 503 error, with code H12, and description that is timeout error. Here is the code of one of the calls to the database that I'm testing:
router.get('/allpoints', async (req,res) =>{
try {
const points = await pool.query(
`SELECT nome, latitudine,longitudine
FROM luogo`);
res.json(points.rows);
}catch(err){
console.error(err.message);
}
});
Here there are the information about how I connect to the database.
const pool = new Pool({
connectionString: process.env.DATABASE_URL || 'postgresql://postgres:psw#localhost:5432/campione',
ssl: process.env.DATABASE_URL ? true : false
})
module.exports = pool;
The build on Heroku seems to work properly.
I read this question: Heroku h12 Timeout Error with PG / Node.js
It says that you have to put res.end() where there is not res.json(), but here there is the res.json(). So, I thought that the issue could be that there is an error that the route manage, and can't give back anything. So, I changed from console.log(err) to res.json(err), and the API response with `ssl self signed, as an error. At this point, in the second file, I put ssl as false by default, but it gaves me error because there is no SSL. I searched for a really long time for a solution, but I have not been able yet to fix the issue.
Someone thinks he knows what should I change? Thank you in advice
this option in databse config maybe useful
ssl: {
rejectUnauthorized : false,
}
I am trying to make quick test for the pub/sub mechanism to my registered device on TTN so I can build my complete solution app on the data coming to the TTN broker.
At the moment I am waiting for my loRa module to arrive, that is why I want to use a simple nodeJS script to publish dummy data, and the other to subscribe and build an app using the dummy data. I use the following code for this:
var mqtt = require('mqtt')
var options = {
port: 1883,
host: ‘mqtt://eu.thethings.network’,
username: ‘xxxx’, // here I wrote my app id
password: ‘xxxx’ // here I wrote the access key
};
var client = mqtt.connect(‘mqtt://eu.thethings.network’,options)
client.on(‘connect’, function () {
client.subscribe(‘appID/devices/MyDeviceName/down’, function (err) {
if (!err) {
client.publish(‘appID/devices/MyDeviceName/down’, ‘Hello mqtt’)
}
})
})
client.on(‘message’, function (topic, message) {
// message is Buffer
console.log(message.toString())
// client.end()
})
This is however not doing anything, I was watching the data on TTN, nothing coming in.
I also tried using mqtt explorer but it did not work.
Both methods worked fine when I played the broker on my machine, eclipse and mosquittoo on cloud.
Your help is greatly appreciated.
Thanks!
Ahmed
I have encountered a similar issue in the past. I believe the issue is with trying to use "mqtt" instead of "https". For me, it worked when I called
mqtt.connect('https://thethings.network:1883', {
"username": username,
"password": password
}
However, I wasn't using the community version of the website (The Things Stack V3), so there might be a slight difference. For example, instead of "My-App-Id" I had to use "My-App-Id#My-Company-Name".
Please, try the above and let me know if it works.
I am creating a project using node.js with mqtt.js and mosquitto broker.
In mosquitto config file, I have setup a pwfile, aclfile to control which topic can be accessed by which user.
Everything works fine, if the username, password, publish topic and subscribe topic are correct.
But if I change the publish topic to an inaccessible topic, it seems successfully publish the topic without any error in mqtt.js, but the message is never been publish.
Is there anyway to catch error when publish or subscribe to inaccessible topic?
https://github.com/mqttjs/MQTT.js/blob/master/README.md#publish
I try to catch the error in callback function with qos=1, but it seems the mosquitto broker acknowledge the client without any error.
Of cause, the message didn't publish to the topic, since that client don't have the access right to that topic.
client.publish('inaccessible_topic', 'hello world', {qos: 1}, (err) => {
if (err) console.log('error occur: ', err);
else console.log('message successfully publish');
});
I expect mosquitto will return some sort of error (error occur: <error message>) when publish to the inaccessible topic, but it didn't (message sucessfully publish).
I wonder if it is even possible to catch such error. Does mosquitto handle such error, if so, how do I catch such error using mqtt.js?
You don't.
The broker will not tell the client if it tried to publish to a topic it doesn't have access to, it will just silently just drop the message.
This would be a security vulnerability in that it would expose what topics are available.
With MQTT 5 it can be done. Mosquitto sends the needed response code in the PUBACK and PUBREC packets. All you need to do is to set your mqttjs-clients protocol to mqtt 5 and choose a qos of 1 or 2 (I tested it with qos=2).
const mqtt = require('mqtt');
const mqttClient = mqtt.connect('mqtt://insertyouradresshere.org', {
protocolId: 'MQTT',
protocolVersion: 5,
username: 'myusername',
password: 'mypassword',
});
If the publish fails then your callback will be called with an error.
I am running a scraper app on Heroku that sends emails every once in a while. Sometimes it'll be every 2 minutes, sometimes it won't send anything for 2 days-depends on what the scraper finds. I am running into continual errors with email authentication. I have tried configuring the email with nodemailer and emailjs. Both run into the same issue. I am wondering if it is because of the way Heroku servers are distributed?
I have taken the following steps:
1) unlocked the account at: https://accounts.google.com/DisplayUnlockCaptcha
2) allowed less secure apps here: https://myaccount.google.com/lesssecureapps
With both nodemailer and emailjs, I notice that it works fine for a couple hours but then if no emails are sent for a day, as soon as it tries to send another one I receive an authentication error. Here is the code using emailjs that works for a couple hours then fails with an authentication error:
var server = email.server.connect({
user: process.env.CRAWLER_MAIL,
password:process.env.CRAWLER_PWD,
host: "smtp.gmail.com",
ssl: true
});
var message = {
text: "Placeholder",
from: 'NAME <'+ process.env.CRAWLER_MAIL+'>',
to: eachEmail[0],
subject: emailSubject,
attachment:
[
{data:emailBody, alternative:true}
]
};
server.send(message, function(err, message) { console.log(err || message); });
And here is the error from the Heroku log:
Error: authorization.failed (bad response on command '[redacted]': -5.7.14 https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=[redacted]-)
I don't use 2-step verification for this account, so I don't think I need an app password. Any other ideas about how I can get this to consistently work?
I'm using node.js to create an app that gets a PUSH from Gmail each time an email is received, checks it against a third party database in a CRM and creates a new field in the CRM if the e-mail is contained there. I'm having trouble using Google's new Cloud Pub/Sub, which seems to be the only way to get push from Gmail without constant polling.
I've gone through the instructions here: https://cloud.google.com/pubsub/prereqs but I don't understand how exactly this is supposed to work from an app on my desktop. It seems that pub/sub can connect to a verified domain, but I can't get it to connect directly toto the .js script that I have on my computer. I've saved the api key in a json file and use the following:
var gcloud = require('gcloud');
var pubsub;
// From Google Compute Engine:
pubsub = gcloud.pubsub({
projectId: 'my-project',
});
// Or from elsewhere:
pubsub = gcloud.pubsub({
projectId: 'my-project',
keyFilename: '/path/to/keyfile.json'
});
// Create a new topic.
pubsub.createTopic('my-new-topic', function(err, topic) {});
// Reference an existing topic.
var topic = pubsub.topic('my-existing-topic');
// Publish a message to the topic.
topic.publish('New message!', function(err) {});
// Subscribe to the topic.
topic.subscribe('new-subscription', function(err, subscription) {
// Register listeners to start pulling for messages.
function onError(err) {}
function onMessage(message) {}
subscription.on('error', onError);
subscription.on('message', onMessage);
// Remove listeners to stop pulling for messages.
subscription.removeListener('message', onMessage);
subscription.removeListener('error', onError);
});
However, I get errors as if it isn't connecting to server and on the API list I see only errors, no actual successes. I'm clearly doing something wrong, any idea what it might be?
Thank you in advance!
TL;DR
Your cannot subscribe to push notifications from the client side.
Set up an HTTPS server to handle the messages. Messages will be sent
to the URL endpoint that you configure, representing that server's
location. Your server must be reachable via a DNS name and must
present a signed SSL certificate. (App Engine applications are
preconfigured with SSL certificates.)
Just subscribe to the push notifications on your server, and when you get the notification, you can figure out who it concerns. The data you will get from the notifications is what user that it concerns, and the relevant historyId, like so:
// This is all the data the notifications will give you.
{"emailAddress": "user#example.com", "historyId": "9876543210"}
Then you could e.g. emit an event through Socket.io to the relevant user if he is online, and have him do a sync with the supplied historyId on the client side.