Get all message from aws sqs - node.js

Question

I have a ScheduledEvent on my lamda function for every 24 hours and then inside function, I am calling SQS to get my messages.
export class EmailNotificationProcessor {
public static async run(): Promise<void> {
console.log('event');
await this.getNotificationFromSqs();
}
private static async getNotificationFromSqs(): Promise<void> {
const messagesToDelete: DeleteMessageBatchRequestEntryList = [];
const messageRequest: ReceiveMessageRequest = {
QueueUrl: process.env.DID_NOTIFICATION_SQS_QUEUE,
MaxNumberOfMessages:10,
WaitTimeSeconds:20
}
const { Messages }: ReceiveMessageResult = await receiveMessage(messageRequest);
console.log('Messages', Messages);
console.log('Total Messages ', Messages.length);
if (Messages && Messages.length > 0) {
for (const message of Messages) {
console.log('body is ', message.Body);
messagesToDelete.push({
Id: message.MessageId,
ReceiptHandle: message.ReceiptHandle,
} as DeleteMessageBatchRequestEntry);
}
}
await deleteMessages(messagesToDelete);
}
}
I am expecting 1 to 30 messages inside my queue and want to process all messages before sending an email which consists of the content that I will parse from sqs body.
My function for receiving messages
export const receiveMessage = async (request: SQS.ReceiveMessageRequest): Promise<PromiseResult<SQS.ReceiveMessageResult, AWSError>> =>{
console.log('inside receive');
return sqs.receiveMessage(request).promise();
}
Now I am not able to receive all messages at once and only getting 3 messages or sometimes 1 message at a time.
I know limit for API call is 10 in one single request but is there any way to wait and get all your message.

Answer 1

First of all, There is no configuration to get more then 10 messages from queue.
ReceiveMessage: Retrieves one or more messages (up to 10), from the specified queue
For your other problems: I think you are using Short poll ReceiveMessage call.If the number of messages in the queue is extremely small, you might not receive any messages in a particular ReceiveMessage response.
Try Long Polling:
Long polling helps reduce the cost of using Amazon SQS by eliminating the number of empty responses (when there are no messages available for a ReceiveMessage request) and false empty responses (when messages are available but aren't included in a response).
Note* For getting more messages you need to wrap the call to SQS in a loop and keep requesting more messages until the queue is empty, which can lead you to get duplicate messages as well so try VisibilityTimeout in that problem.
Try VisibilityTimeout: The duration (in seconds) that the received messages are hidden from subsequent retrieve requests after being retrieved by a ReceiveMessage request.
Sample Wrap up SQS call code:
function getMessages(params, count = 0, callback) {
let allMessages = [];
sqs.receiveMessage(params, function (err, data) {
if (err || (data && !data.Messages || data.Messages.length <= 0)) {
if(++count >= config.SQSRetries ){
return callback(null, allMessages);
}
return setTimeout(() => {
return getMessages(params, count, callback);
}, 500);
} else if (++count !== config.SQSRetries ){
allMessages.push(data);
return setTimeout(() => {
return getMessages(params, count, callback);
}, 500);
} else {
allMessages.push(data);
callback(null, allMessages);
}
});
In config.SQSRetries, we have set the value according to our requirement but as your SQS have 1 to 30 messages, Then '7' will be good for you!
Links: RecieveMessage UserGuide