Is it possible to use RedisMqServer as simple task scheduler? For example to publish a message and execute it in feature (at specific time)?
RedisMQ doesn't provide any explicit support for delayed messages itself, but you could use simulate it with a Timer, e.g:
var sendInMs = 30 * 1000;
new System.Threading.Timer(msg => mqClient.Publish(msg),
msg,
sendInMs,
Timeout.Infinite);
Related
I am very new to NodeJS and trying to develop an application which acts as a scheduler that tries to fetch data from ELK and sends the processed data to another ELK. I am able to achieve the expected behaviour but after completing all the processes, scheduler job does not exists and wait for another scheduler job to come up.
Note: This scheduler runs every 3 minutes.
job.js
const self = module.exports = {
async schedule() {
if (process.env.SCHEDULER == "MinuteFrequency") {
var timenow = moment().seconds(0).milliseconds(0).valueOf();
var endtime = timenow - 60000;
var starttime = endtime - 60000 * 3;
//sendData is an async method
reports.sendData(starttime, endtime, "SCHEDULER");
}
}
}
I tried various solutions such Promise.allSettled(....., Promise.resolve(true), etc, but not able to fix this.
As per my requirement, I want the scheduler to complete and process and exit so that I can save some resources as I am planning to deploy the application using Kubernetes cronjobs.
When all your work is done, you can call process.exit() to cause your application to exit.
In this particular code, you may need to know when reports.sendData() is actually done before exiting. We would have to know what that code is and/or see the code to know how to know when it is done. Just because it's an async function doesn't mean it's written properly to return a promise that resolves when it's done. If you want further help, show us the code for sendData() and any code that it calls too.
I've been working with node for the first time in a while again and stumbled upon node-schedule, which for the most part has been a breeze, however, I've found resuming a scheduled task after canceling it via job.cancel() pretty difficult.
For the record, I'm using schedule to perform specific actions at a specific date (non-recurring) and under some circumstances cancel the task at a specific date but would later like to resume it.
I tried using job.cancel(true) after cancelling it via plain job.cancel() first as the documentation states that that would reschedule the task, but this has not worked for me. Using job.reschedule() after having cancelled job first yields the same result.
I could probably come up with an unelegant solution, but I thought I'd ask if anyone knows of an elegant one first.
It took me a while to understand node-schedule documentation ^^
To un-cancel a job, You have to give to reschedule some options.
If you don't pass anything to reschedule, this function returns false (Error occured)
For exemple, you can declare options, and pass this variable like this :
const schedule = require('node-schedule');
let options = {rule: '*/1 * * * * *'}; // Declare schedule rules
let job = schedule.scheduleJob(options, () => {
console.log('Job processing !');
});
job.cancel(); // Cancel Job
job.reschedule(options); // Reschedule Job
Hope it helps.
In my Angular application I need to make a http call in regular intervals of time (say every y mins) as long as the page is active and render the response on the screen.
Is it good to use NodeJS.Timer object to implement setTimeout()
or use Observable.timer().
Observable.timer() is a lot more useful for Angular
for example:
const tick3$ = Observable.timer(100, 60000);
If you just want to execute a regular function every 10 minutes, you can just use plain Javascript for that:
var timer = setInterval(function() {
// put your function code here
}, 10 * 60 * 60 * 1000);
Or, if the function is already separately defined:
var timer = setInterval(myFunction, 10 * 60 * 60 * 1000);
Is it good to use node js timer object to implement setTimeout()
It's really not clear what you mean by this question. A nodejs timer object would be running things on the nodejs server, not on the client. Since you said you want to run something in the browser web page, I presume you don't want a timer on nodejs to do that, but rather a timer in the web page as my example above shows.
Angular also has a wrapper around setInterval() called $interval() and it adds a few more features including a count and some angular specific functionality related to updating the view if the data changes. If you want those angular specific features, you can use $interval(). If not, then you can use either.
The Azure Service Bus supports a built-in retry mechanism which makes an abandoned message immediately visible for another read attempt. I'm trying to use this mechanism to handle some transient errors, but the message is made available immediately after being abandoned.
What I would like to do is make the message invisible for a period of time after it is abandoned, preferably based on an exponentially incrementing policy.
I've tried to set the ScheduledEnqueueTimeUtc property when abandoning the message, but it doesn't seem to have an effect:
var messagingFactory = MessagingFactory.CreateFromConnectionString(...);
var receiver = messagingFactory.CreateMessageReceiver("test-queue");
receiver.OnMessageAsync(async brokeredMessage =>
{
await brokeredMessage.AbandonAsync(
new Dictionary<string, object>
{
{ "ScheduledEnqueueTimeUtc", DateTime.UtcNow.AddSeconds(30) }
});
}
});
I've considered not abandoning the message at all and just letting the lock expire, but this would require having some way to influence how the MessageReceiver specifies the lock duration on a message, and I can't find anything in the API to let me change this value. In addition, it wouldn't be possible to read the delivery count of the message (and therefore make a decision for how long to wait for the next retry) until after the lock is already required.
Can the retry policy in the Message Bus be influenced in some way, or can a delay be artificially introduced in some other way?
Careful here because I think you are confusing the retry feature with the automatic Complete/Abandon mechanism for the OnMessage event-driven message handling. The built in retry mechanism comes into play when a call to the Service Bus fails. For example, if you call to set a message as complete and that fails, then the retry mechanism would kick in. If you are processing a message an exception occurs in your own code that will NOT trigger a retry through the retry feature. Your question doesn't get explicit on if the error is from your code or when attempting to contact the service bus.
If you are indeed after modifying the retry policy that occurs when an error occurs attempting to communicate with the service bus you can modify the RetryPolicy that is set on the MessageReciver itself. There is an RetryExponitial which is used by default, as well as an abstract RetryPolicy you can create your own from.
What I think you are after is more control over what happens when you get an exception doing your processing, and you want to push off working on that message. There are a few options:
When you create your message handler you can set up OnMessageOptions. One of the properties is "AutoComplete". By default this is set to true, which means as soon as processing for the message is completed the Complete method is called automatically. If an exception occurs then abandon is automatically called, which is what you are seeing. By setting the AutoComplete to false you required to call Complete on your own from within the message handler. Failing to do so will cause the message lock to eventually run out, which is one of the behaviors you are looking for.
So, you could write your handler so that if an exception occurs during your processing you simply do not call Complete. The message would then remain on the queue until it's lock runs out and then would become available again. The standard dead lettering mechanism applies and after x number of tries it will be put into the deadletter queue automatically.
A caution of handling this way is that any type of exception will be treated this way. You really need to think about what types of exceptions are doing this and if you really want to push off processing or not. For example, if you are calling a third party system during your processing and it gives you an exception you know is transient, great. If, however, it gives you an error that you know will be a big problem then you may decide to do something else in the system besides just bailing on the message.
You could also look at the "Defer" method. This method actually will then not allow that message to be processed off the queue unless it is specifically pulled by its sequence number. You're code would have to remember the sequence number value and pull it. This isn't quite what you described though.
Another option is you can move away from the OnMessage, Event-driven style of processing messages. While this is very helpful you don't get a lot of control over things. Instead hook up your own processing loop and handle the abandon/complete on your own. You'll also need to deal some of the threading/concurrent call management that the OnMessage pattern gives you. This can be more work but you have the ultimate in flexibility.
Finally, I believe the reason the call you made to AbandonAsync passing the properties you wanted to modify didn't work is that those properties are referring to Metadata properties on the method, not standard properties on BrokeredMessage.
I actually asked this same question last year (implementation aside) with the three approaches I could think of looking at the API. #ClemensVasters, who works on the SB team, responded that using Defer with some kind of re-receive is really the only way to control this precisely.
You can read my comment to his answer for a specific approach to doing it where I suggest using a secondary queue to store messages that indicate which primary messages have been deferred and need to be re-received from the main queue. Then you can control how long you wait by setting the ScheduledEnqueueTimeUtc on those secondary messages to control exactly how long you wait before you retry.
I ran into a similar issue where our order picking system is legacy and goes into maintenance mode each night.
Using the ideas in this article(https://markheath.net/post/defer-processing-azure-service-bus-message) I created a custom property to track how many times a message has been resubmitted and manually dead lettering the message after 10 tries. If the message is under 10 retries it clones the message increments the custom property and sets the en queue of the new message.
using Microsoft.Azure.ServiceBus;
public PickQueue()
{
queueClient = new QueueClient(QUEUE_CONN_STRING, QUEUE_NAME);
}
public async Task QueueMessageAsync(int OrderId)
{
string body = JsonConvert.SerializeObject(OrderId);
var message = new Message(Encoding.UTF8.GetBytes(body));
await queueClient.SendAsync(message);
}
public async Task ReQueueMessageAsync(Message message, DateTime utcEnqueueTime)
{
int resubmitCount = (int)(message.UserProperties["ResubmitCount"] ?? 0) + 1;
if (resubmitCount > 10)
{
await queueClient.DeadLetterAsync(message.SystemProperties.LockToken);
}
else
{
Message clone = message.Clone();
clone.UserProperties["ResubmitCount"] = ++resubmitCount;
await queueClient.ScheduleMessageAsync(message, utcEnqueueTime);
}
}
This question asks how to implement exponential backoff in Azure Functions. If you do not want to use the built-in RetryPolicy (only available when autoComplete = false), here's the solution I've been using:
public static async Task ExceptionHandler(IMessageSession MessageSession, string LockToken, int DeliveryCount)
{
if (DeliveryCount < Globals.MaxDeliveryCount)
{
var DelaySeconds = Math.Pow(Globals.ExponentialBackoff, DeliveryCount);
await Task.Delay(TimeSpan.FromSeconds(DelaySeconds));
await MessageSession.AbandonAsync(LockToken);
}
else
{
await MessageSession.DeadLetterAsync(LockToken);
}
}
Is there an implementation of setTimeout() and clearTimeout() in Haxe?
It's of course possible to use the Timer class, but for a one-shot execution it's not the best way, I guess.
For a one-shot execution I think that Timer.delay() is perfect. You can use the returned instance to stop the timer later:
var timer = haxe.Timer.delay(function() trace("Hello World!"), 250);
...
timer.stop();
You could also access the native setTimeout() with the js.html.Window extern:
var handle = js.Browser.window.setTimeout(function() trace("Hello World!"), 250);
...
js.Browser.window.clearTimeout(handle);
In case you're using the kha framework:
Kha modifies haxe.Timer to call kha.Scheduler, which in the end doesn't get the timestamps via setTimeout - it gets these via requestAnimationFrame().
This seems to not work while a tab is inactive, so it's not the same function while the tab is inactive.
I'm attempting a workaround, but at the moment, it doesn't give the same result as a native setTimeout()-JS does (although I found a workaround which I'll present for inclusion).