I have a function (called rankCheck), which takes three parameters:
Guild object (aka a Discord server)
UserId
Callback Function
The function will fetch the last 500 messages from every text channel in the provided guild. It will then will then only keep any messages that start with "!rank" and were sent by the provided UserId. Finally, it will count the remaining messages and pass the integer to the callback function.
async function rankChecks(guild, userId = *REMOVED FOR PRIVACY*, callback){
sumOfRankChecks = 0;
guild.channels.cache.each(channel => { //for each text channel, get # of rank checks for userId in last 500 msgs.
if (channel.type === "text"){
fetchMessages(channel, 500).then(msgs => {
let filteredMsgs = msgs.filter(msg => msg.content.startsWith("!rank") && msg.member.user.id == userId);
sumOfRankChecks = sumOfRankChecks + filteredMsgs.length;
});
}
});
callback(sumOfRankChecks);
}
Since discord only allows fetching 100 messages at once, I use this function (fetchMessages) to bypass this limit, by sending multiple requests, and then combining the results into one.
async function fetchMessages(channel, limit) {
const sum_messages = [];
let last_id;
while (true) {
const options = { limit: 100 };
if (last_id) {
options.before = last_id;
}
const messages = await channel.messages.fetch(options);
sum_messages.push(...messages.array());
last_id = messages.last().id;
if (messages.size != 100 || sum_messages >= limit) {
break;
}
}
return sum_messages;
}
When I call the rankCheck function, the return value is always 0
rankChecks(msg.guild, *REMOVED FOR PRIVACY*, function(int){
console.log(int);
});
Output:
0
However when I add a console.log into my rankCheck function:
async function rankChecks(guild, userId = *REMOVED FOR PRIVACY*, callback){
sumOfRankChecks = 0;
guild.channels.cache.each(channel => { //for each text channel, get # of rank checks for userId in last 500 msgs.
if (channel.type === "text"){
fetchMessages(channel, 500).then(msgs => {
let filteredMsgs = msgs.filter(msg => msg.content.startsWith("!rank") && msg.member.user.id == userId);
sumOfRankChecks = sumOfRankChecks + filteredMsgs.length;
console.log(sumOfRankChecks) //NEW CONSOLE.LOG!!!!!!!!!!!!!!!
});
}
});
callback(sumOfRankChecks);
}
Output:
3
5
This is the output I was expecting. Since I have 2 text channels in my server, I got 2 logs. If you had 3 channels, you would get 3 logs, etc. 3 messages from channel #1, and 2 messages from channel #2, therefore in total, there are 5 messages.
5 should be the integer that is passed into the callback function, but 0 is passed instead. Why is this?
Your callback function is being called before you even change sumOfRankChecks. Collection#each (and Map#forEach() and the gang) cannot wait for Promises to resolve because of the way they're built. Your code also wouldn't wait anyway, because you're not using await.
Despite what one might think is happening, guild.channels.each() is called, and callback() is called immediately after. This is the source of your confusion.
For more about async vs sync, you can check out the explanation in my answer here. You must use a for loop and await, or refactor your code so that async/await syntax is not necessary.
NOTE: The Discord.js documentation hyperlinked is for recently released v12. If your Discord.js isn't up to date, switch to the correct version at the top of the page for accurate info.
Related
I want fetch all child nodes in a given path from Firebase Realtime Database and which I get using a similar method to this.
But my main issue is I want to merge then into a single string and I tried out something like this:
if(userInput === '/showUsers') {
const usersRef = `Data/users/`
let returnText = `Your users:`
admin.database().ref(usersRef).on("value", function (snapshot) {
snapshot.forEach(function (e) {
const xSnap = e.val()
const xName = xSnap.Name
const xId = xSnap.id
console.log(`${xName} - ${xId}`)
returnText.concat(`\n${xName}\n${xId}`)
console.log(returnText)
})
})
return res.status(200).send({
method: 'sendMessage',
chat_id,
reply_to_message_id: messageIdtoReply,
text: `${returnText}`,
parse_mode: 'HTML'
})
}
So all the child nodes are getting fetched and they get logged into the console but the returnText always remain to it's predefined value. I have to do this because I want to return the data into a Telegram bot so it must be a single string as I cannot return multiple values to telegram bot.
I don't want to use a for-loop because those nodes won't be named as 1,2,3 or so on that I can use a loop.
Also how do I fetch only first 10 users/records a time so that it won't overflow Telegram message limit?
Expected Output:
All I want is to get the data into a single message.
I am not very familiar with typescript to be honest but I will try to answer your question.
Getting values from firebase is asynchronous which means
1 const usersRef = `Data/users/`
2 let returnText = `Your users:`
3
4 admin.database().ref(usersRef).on("value", function (snapshot) {
5 snapshot.forEach(function (e) {
6 const xSnap = e.val()
7 const xName = xSnap.Name
8 const xId = xSnap.id
9
10 console.log(`${xName} - ${xId}`)
11 returnText.concat(`\n${xName}\n${xId}`)
12 console.log(returnText)
13 })
14 })
15 console.log(returnText)
If you access returnText variable on line 15 for example it might be executed before the above function.
I think you should put this in a function with a call back that gets you returnText when everything finishes executing.
For Example:
function concatResults(cb){
const usersRef = `Data/users/`
let returnText = `Your users:`
admin.database().ref(usersRef).on("value", function (snapshot) {
snapshot.forEach(function (e) {
const xSnap = e.val()
const xName = xSnap.Name
const xId = xSnap.id
console.log(`${xName} - ${xId}`)
returnText.concat(`\n${xName}\n${xId}`)
console.log(returnText)
})
cb(returnText)
})
}
Here we have function concatResults with a parameter cb callback which is called when the foreach finishes it's job.
How to use?
Simply:
concatResults(function(result){
console.log(result);
})
Here we have implemented our callback that will be called after concatResults function finishes it's job.
In JavaScript, the .concat() function does not alter the value of any string. It just returns the concatenated values (w3 reference). Thus, you should use it as follows:
returnText = returnText.concat(`\n${xName}\n${xId}`);
Also, due to the single threaded nature of JS, in order to return the concatenated text you should perform the return inside the admin.database().ref(usersRef).on("value", function (snapshot) { ... } block or better yet, use a Promise as explained here
I have the following function that gets a message from aws SQS, the problem is I get one at a time and I wish to get all of them, because I need to check the ID for each message:
function getSQSMessages() {
const params = {
QueueUrl: 'some url',
};
sqs.receiveMessage(params, (err, data) => {
if(err) {
console.log(err, err.stack)
return(err);
}
return data.Messages;
});
};
function sendMessagesBack() {
return new Promise((resolve, reject) => {
if(Array.isArray(getSQSMessages())) {
resolve(getSQSMessages());
} else {
reject(getSQSMessages());
};
});
};
The function sendMessagesBack() is used in another async/await function.
I am not sure how to get all of the messages, as I was looking on how to get them, people mention loops but I could not figure how to implement it in my case.
I assume I have to put sqs.receiveMessage() in a loop, but then I get confused on what do I need to check and when to stop the loop so I can get the ID of each message?
If anyone has any tips, please share.
Thank you.
I suggest you to use the Promise api, and it will give you the possibility to use async/await syntax right away.
const { Messages } = await sqs.receiveMessage(params).promise();
// Messages will contain all your needed info
await sqs.sendMessage(params).promise();
In this way, you will not need to wrap the callback API with Promises.
SQS doesn't return more than 10 messages in the response. To get all the available messages, you need to call the getSQSMessages function recursively.
If you return a promise from getSQSMessages, you can do something like this.
getSQSMessages()
.then(data => {
if(!data.Messages || data.Messages.length === 0){
// no messages are available. return
}
// continue processing for each message or push the messages into array and call
//getSQSMessages function again.
});
You can never be guaranteed to get all the messages in a queue, unless after you get some of them, you delete them from the queue - thus ensuring that the next requests returns a different selection of records.
Each request will return 'upto' 10 messages, if you don't delete them, then there is a good chance that the next request for 'upto' 10 messages will return a mix of messages you have already seen, and some new ones - so you will never really know when you have seen them all.
It maybe that a queue is not the right tool to use in your use case - but since I don't know your use case, its hard to say.
I know this is a bit of a necro but I landed here last night while trying to pull some all messages from a dead letter queue in SQS. While the accepted answer, "you cannot guarantee to get all messages" from the queue is absolutely correct I did want to drop an answer for anyone that may land here as well and needs to get around the 10 message limit per request from AWS.
Dependencies
In my case I have a few dependencies already in my project that I used to make life simpler.
lodash - This is something we use in our code for help making things functional. I don't think I used it below but I'm including it since it's in the file.
cli-progress - This gives you a nice little progress bar on your CLI.
Disclaimer
The below was thrown together during troubleshooting some production errors integrating with another system. Our DLQ messages contain some identifiers that I need in order to formulate cloud watch queries for troubleshooting. Given that these are two different GUIs in AWS switching back and forth is cumbersome given that our AWS session are via a form of federation and the session only lasts for one hour max.
The script
#!/usr/bin/env node
const _ = require('lodash');
const aswSdk = require('aws-sdk');
const cliProgress = require('cli-progress');
const queueUrl = 'https://[put-your-url-here]';
const queueRegion = 'us-west-1';
const getMessages = async (sqs) => {
const resp = await sqs.receiveMessage({
QueueUrl: queueUrl,
MaxNumberOfMessages: 10,
}).promise();
return resp.Messages;
};
const main = async () => {
const sqs = new aswSdk.SQS({ region: queueRegion });
// First thing we need to do is get the current number of messages in the DLQ.
const attributes = await sqs.getQueueAttributes({
QueueUrl: queueUrl,
AttributeNames: ['All'], // Probably could thin this down but its late
}).promise();
const numberOfMessage = Number(attributes.Attributes.ApproximateNumberOfMessages);
// Next we create a in-memory cache for the messages
const allMessages = {};
let running = true;
// Honesty here: The examples we have in existing code use the multi-bar. It was about 10PM and I had 28 DLQ messages I was looking into. I didn't feel it was worth converting the multi-bar to a single-bar. Look into the docs on the github page if this is really a sticking point for you.
const progress = new cliProgress.MultiBar({
format: ' {bar} | {name} | {value}/{total}',
hideCursor: true,
clearOnComplete: true,
stopOnComplete: true
}, cliProgress.Presets.shades_grey);
const progressBar = progress.create(numberOfMessage, 0, { name: 'Messages' });
// TODO: put in a time limit to avoid an infinite loop.
// NOTE: For 28 messages I managed to get them all with this approach in about 15 seconds. When/if I cleanup this script I plan to add the time based short-circuit at that point.
while (running) {
// Fetch all the messages we can from the queue. The number of messages is not guaranteed per the AWS documentation.
let messages = await getMessages(sqs);
for (let i = 0; i < messages.length; i++) {
// Loop though the existing messages and only copy messages we have not already cached.
let message = messages[i];
let data = allMessages[message.MessageId];
if (data === undefined) {
allMessages[message.MessageId] = message;
}
}
// Update our progress bar with the current progress
const discoveredMessageCount = Object.keys(allMessages).length;
progressBar.update(discoveredMessageCount);
// Give a quick pause just to make sure we don't get rate limited or something
await new Promise((resolve) => setTimeout(resolve, 1000));
running = discoveredMessageCount !== numberOfMessage;
}
// Now that we have all the messages I printed them to console so I could copy/paste the output into LibreCalc (excel-like tool). I split on the semicolon for rows out of habit since sometimes similar scripts deal with data that has commas in it.
const keys = Object.keys(allMessages);
console.log('Message ID;ID');
for (let i = 0; i < keys.length; i++) {
const message = allMessages[keys[i]];
const decodedBody = JSON.parse(message.Body);
console.log(`${message.MessageId};${decodedBody.id}`);
}
};
main();
I'm trying to retrieve all the child then when there's match display.
I print the value in the console and my code work well there after few second, but when I print it in the agent as a message it show not available before the response because it does not wait.
Here is my code:
function retrieveContact(agent) {
var query = admin.database().ref("/contacts").orderByKey();
query.once("value")
.then(function(snapshot) {
snapshot.forEach(function(childSnapshot) {
var key = childSnapshot.key;
var childName = childSnapshot.child('name').val();
if (agent.parameters.name == childName) {
console.log('find ' + childName);
agent.add('The email address for ' + childName + ' is ' + childSnapshot.child('email').val());
}
// console.log('testMode'+childName);
}); //// .then
}); //// .once }
SO, how can I wait my response then let the agent show the result?
How can I include the promise concept in my code ?
You don't show your entire Handler function, but if you're doing async operations (such as reading from the firebase db) you must return the Promise. This is how the Handler Dispatcher knows to wait for the Promise to complete before returning a response to the user.
In your case, it is probably as simple as
return query.once("value")
// etc
I'm trying to create a test, to verify that I've put an item in a dynamoDB table. In order to do so, right after I make a call that should put an Item (vehicle) in the database, I am trying to get the vehicle from the DB.
In my test I want to have a maximum number of retries (5). I want this while loop to be block the thread until my query has resolved to give a vehicle, or tried 5 times. Inside my test I have:
let count = 0
let car
while (!car || count < 5) {
setTimeout(async () => {
car = await findVehicle(greenCar.vehicleInfo)
}, 3000)
count++
}
And findVehicle is an asynchronous function that does a get from the dynamoDB table
If you want to wait on each iteration you can do this:
let count = 0;
let car;
while (!car || count < 5) {
await new Promise((resolve) =>
setTimeout(async () => {
car = await findVehicle(greenCar.vehicleInfo);
resolve();
}, 3000));
count++
}
So you are resolving the promise you are awaiting after you get your data. Also your function must be async in order to use await. Hope this helps.
I would like to filter sms per phone number and date the SMS was sent using REST API, however the output of the following code is not available outside of client.messages.each() block.
Please advise how I can use the latest sms code sent to the filtered number:
const filterOpts = {
to: '+13075550185',
dateSent: moment().utc().format('YYYY-MM-DD')
};
let pattern = /([0-9]{1,})$/;
let codeCollection = [];
client.messages.each(filterOpts, (record) => {
codeCollection.push(record.body.match(pattern)[0]);
console.log(record.body.match(pattern)[0], record.dateSent);
});
console.log(codeCollection,'I get an empty array here');//how to get
the latest sms and use it
doSomethingWithSMS(codeCollection[0]);
Twilio developer evangelist here.
The each function doesn't actually return a Promise. You can run a callback function after each has completed streaming results by passing it into the options as done like this:
const codeCollection = [];
const pattern = /([0-9]{1,})$/;
const filterOpts = {
to: '+13075550185',
dateSent: moment().utc().format('YYYY-MM-DD'),
done: (err) => {
if (err) { console.error(err); return; }
console.log(codeCollection);
doSomethingWithSMS(codeCollection[0]);
}
};
client.messages.each(filterOpts, (record) => {
codeCollection.push(record.body.match(pattern)[0]);
console.log(record.body.match(pattern)[0], record.dateSent);
});
Let me know if that helps at all.
Do you have access to the length of the array of messages? If so, you can do something like this
const filterOpts = {
to: '+13075550185',
dateSent: moment().utc().format('YYYY-MM-DD')
};
let pattern = /([0-9]{1,})$/;
let codeCollection = [];
var i = 0
client.messages.each(filterOpts, (record) => {
if (i < messages.length){
codeCollection.push(record.body.match(pattern)[0]);
console.log(record.body.match(pattern)[0], record.dateSent);
i++;
else {
nextFunction(codeCollection);
}
});
function nextFunction(codeCollection){
console.log(codeCollection,'I get an empty array here');
doSomethingWithSMS(codeCollection[0]);
}
messages.each() is running asynchronously, so your main thread moves on to the next call while the client.messages() stuff runs on a background thread. So, nothing has been pushed to codeCollection by the time you've tried to access it. You need to somehow wait for the each() to finish before moving on. Twilio client uses backbone style promises, so you can just add another .then() link to the chain, like below. You could also use a library like async which lets you use await to write asynchronous code in a more linear looking fashion.
const filterOpts = {
to: '+13075550185',
dateSent: moment().utc().format('YYYY-MM-DD')
};
let pattern = /([0-9]{1,})$/;
let codeCollection = [];
client.messages.each(filterOpts, (record) => {
codeCollection.push(record.body.match(pattern)[0]);
console.log(record.body.match(pattern)[0], record.dateSent);
}).then(
function() {
console.log(codeCollection,'I get an empty array here');
if( codeCollection.count > 0 ) doSomethingWithSMS(codeCollection[0]);
}
);