I'm trying to improve performance of processing my worker's incoming messages using a queue.
However, the sleep function takes anywhere between 16 to 30 milliseconds to complete instead of the 1 millisecond. Is there a better way to process the queue without this large delay, and without keeping my app at 100% cpu?
I am doing something like this:
var actions = new Queue();
parentPort.on('message', (msg) => actions.enqueue(msg));
loopy();
async function loopy() {
while (true) {
if (actions.size() > 0) {
let action = actions.dequeue();
//do work
continue;
}
await sleep(1);
}
}
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
Any help would be appreciated. Thanks!
while(true) is (usually) not a good idea.
You should call the dequeue function after you enqueue a message. The dequeue function should end when 1. there is already a dequeue function running 2. no more message in the queue.
var isProcessing = false;
var actions = new Queue();
parentPort.on('message', (msg) => {
actions.enqueue(msg)
tryDequeue();
});
async function tryDequeue() {
if(isProcessing || actions.size() == 0)
{
return;
}
isProcessing = true;
let action = actions.dequeue();
//do work
isProcessing = false;
tryDequeue();
}
Related
I have situation where i have to Use Task.Run In my ForEach loop
Requirement:
I'm going to be forced to manually kill thread
I have button where i can start and stop this Thread or Task.Run in For loop.
Problem
My problem is when i start the Task.Run method Its running but when i try to stop with using CancellationTokenSource or runningTaskThread.Abort(); it will not kill. its just stop when i start new Task.Run at that time it run with old thread so it become multiple thread every start process.
Code:
Below is my code for start Thread
var messages = rootObject.MultiQData.Messages.Where(m => m.TimeStamp > DateTime.Now).OrderBy(x => x.TimeStamp).ToList();
//Simulate MultiQ file in BackGroud
if (messages.Count > 0)
{
cancellationTokenSource = new CancellationTokenSource();
cancellationToken = cancellationTokenSource.Token;
Task.Factory.StartNew(
() =>
{
runningTaskThread = Thread.CurrentThread;
messages.ForEach(
m => SetUpTimer(m, rootObject.MultiQData.Connection.FleetNo));
}, cancellationToken);
}
For stop Task.Run
if (cancellationTokenSource != null)
{
if (cancellationToken.IsCancellationRequested)
return;
else
cancellationTokenSource.Cancel();
}
I have also use Thread with Thread.Abort but it is not working
Please Help to solve this issue
I got solution using timer.Stop(),timer.Dispose(). On creation of Thread i am calling SetUpTimer and this SetupTimer i have created multiple timer.
So on call of stop thread i have dispose timer and its work for me
For reference see below code
private void SetUpTimer(Message message, string fleetNo)
{
var ts = new MessageTimer();
var interval = (message.TimeStamp - DateTime.Now).TotalMilliseconds;
interval = interval <= 0 ? 100 : interval;
ts.MessageWrapper = new MessageWrapper(message, fleetNo);
ts.Interval = interval;
ts.Elapsed += ts_Elapsed;
ts.Start();
//Add timer in to the lost for disposing timer at time of stop Simulation
lsTimers.Add(ts);
}
private void StopTask()
{
try
{
// Attempt to cancel the task politely
if (cancellationTokenSource != null)
{
if (cancellationToken.IsCancellationRequested)
return;
else
cancellationTokenSource.Cancel();
}
//Stop All Timer
foreach (var timer in lsTimers)
{
timer.Stop();
timer.Dispose();
}
}
catch (Exception ex)
{
errorLogger.Error("Error while Stop simulation :", ex);
}
}
I am trying to create a hot observable that does not need to be subscribed too. This is for a library and I would like to give users the ability to call certain methods and avoid the need to call subscribe() to get the observables to fire. Initially I have this:
const q = new Queue();
q.add('foo bar baz').subscribe(); // <<< need to call subscribe
Queue.prototype.add = Queue.prototype.enqueue = function (lines) {
lines = _.flattenDeep([lines]);
var lockAcquired = false;
return this.init()
.flatMap(() => {
return acquireLock(this)
})
.flatMap(() => {
lockAcquired = true;
return appendFile(this, lines)
})
.flatMap(() => releaseLock(this))
.catch(err => {
if (lockAcquired) {
return releaseLock(this);
}
else {
return makeGenericObservable();
}
})
};
in order to make the observable hot, I thought I could do something like this:
const q = new Queue();
q.add('foo bar baz'); // <<< don't call subscribe
Queue.prototype.add = Queue.prototype.enqueue = function (lines) {
lines = _.flattenDeep([lines]);
var lockAcquired = false;
return this.init()
.flatMap(() => {
return acquireLock(this)
})
.flatMap(() => {
lockAcquired = true;
return appendFile(this, lines)
})
.flatMap(() => releaseLock(this))
.catch(err => {
if (lockAcquired) {
return releaseLock(this);
}
else {
return makeGenericObservable();
}
})
.publish()
.share() // this too?
};
however the problem is when I call publish(), nothing happens, and the add method never seems to get fully invoked (I assume the first observable in the sequence never fires at all, because effectively subscribe is not called). But I though publish() would automatically invoke the observable chain?
How can I make the observable returned from the add method, hot?
You misunderstood. Hot observable does require subscription like a cold one. Difference is that hot uses some external producer (like dome element) and start listen to it on subscription. On the other hand cold observable creates producer internally on subscription.
It results in that you can miss some events with hot observable because external producer knows nothing about subscription and emits independently. And with cold observable you can't miss anything because producer is created on subscription.
Long story short you can build any chain of observable on top of either hot or cold observable but until you subscribe to it nothing will happen.
PS. No need to use publish with share because latter is alias for .publish().refCount().
If you don't want to use subscribe, you could manually .connect() your stream or subscribe it inside your method:
const q = new Queue();
q.add('foo bar baz');
Queue.prototype.add = Queue.prototype.enqueue = function (lines) {
lines = _.flattenDeep([lines]);
var lockAcquired = false;
let add$ = this.init()
.flatMap(() => {
return acquireLock(this)
})
.flatMap(() => {
lockAcquired = true;
return appendFile(this, lines)
})
.flatMap(() => releaseLock(this))
.catch(err => {
if (lockAcquired) {
return releaseLock(this);
}
else {
return makeGenericObservable();
}
})
.publish();
add$.connect();
return add$; // optional, depends if you even need the stream outside of the add-method
};
or as an alternative to use an internal subscribe:
const q = new Queue();
q.add('foo bar baz'); // <<< don't call subscribe
Queue.prototype.add = Queue.prototype.enqueue = function (lines) {
lines = _.flattenDeep([lines]);
var lockAcquired = false;
let add$ = this.init()
...
.share();
add$.subscribe();
return add$;
};
I have this snippet of code:
const file = fs.createWriteStream('./test.txt');
let written = true;
// handler is added before even an attempt to write is made
file.on('drain', function () {
written = true;
console.log('drained');
});
const interval = setInterval(function () {
if (Date.now() - time > 10000) {
clearInterval(interval);
}
if (written) {
written = file.write(new Array(1000000).join('z'));
}
}, 100);
I'm wondering if that a standard practice to add handler even an attempt to write is made?
In case of using file.on('drain') listener you set up general listener to drain event of your stream.
Notice: This listener will be removed after closing of writable stream.
Generally that code will work proper, but most common practice in Node.js is to use stream.once('drain') handler for each case of internal buffer exceeding. That approach is covered in Node.js documentation for Event: 'drain':
function writeOneMillionTimes(writer, data, encoding, callback) {
var i = 1000000;
write();
function write() {
var ok = true;
do {
i -= 1;
if (i === 0) {
// last time!
writer.write(data, encoding, callback);
} else {
// see if we should continue, or wait
// don't pass the callback, because we're not done yet.
ok = writer.write(data, encoding);
}
} while (i > 0 && ok);
if (i > 0) {
// had to stop early!
// write some more once it drains
writer.once('drain', write);
}
}
}
I have a function that is recursive in framing tcp stream. But when I reach around 1000+ packets received per second I suddenly get Max call stack exceeded or something along those lines.
My code is:
var expectedRemaining = (this.expectedSz - this.receivedSz);
if (buff.length > expectedRemaining) {
var tmp = buff.slice(0, expectedRemaining);
buff = buff.slice(expectedRemaining);
recurse = true;
this.inPacket.push(tmp);
this.receivedSz = this.expectedSz;
} else {
this.inPacket.push(buff);
this.receivedSz += buff.length;
}
if (this.receivedSz === this.expectedSz) {
this.emit('data', Buffer.concat( this.inPacket, this.expectedSz));
this.reset();
}
if (recurse) this.handleData(buff);
Any suggestions?
Right now, your function looks something like this (psuedocode):
this.handleData = function handleData(buf) {
if ( someCondition ) {
// do stuff
recurse = true;
}
else {
// do other stuff
}
if (recurse) {
this.handleData(buf);
}
};
What I'm suggesting is that you implement the recursive behavior with a setImmediate call. This will allow the stack frame to be cleared and the data event to be emitted before entering your function again.
this.handleData = function handleData(buf) {
if ( someCondition ) {
// do stuff
recurse = true;
}
else {
// do other stuff
}
if (recurse) {
setImmediate(function() {
this.handleData(buf);
});
}
};
Just in case people don't read the comments, for the application described in the original question, nextTick ended up being a better fit. The main difference is that nextTick guarantees that the given function will execute before any queued events.
**HI Actually I want to throw gameobjects like Shooting Bullets But it does not work. It throws unlimited gameObjects without any wait. Here is My script on ARCamera.
//Script starts
var prefabBullet : Transform;
var speed : float;
var gameObjects : GameObject[];
function Update () { Invoke("Shoot", 2.0f); }
function Shoot () { yield WaitForSeconds(3);
var instanceBullet=Instantiate(prefabBullet,transform.position,Quaternion.identity);
instanceBullet.rigidbody.AddForce(transform.forward*speed) ;
SomeFunction();
}
function SomeFunction() {
gameObjects = GameObject.FindGameObjectsWithTag ("Player");
for(var i = 0 ; i < gameObjects.length ; i ++)
Destroy(gameObjects[i]);
}
========================================================================
I want to throw gameObjects between some time elapsing.**
function Update () { Invoke("Shoot", 2.0f); }
function Shoot () { yield WaitForSeconds(3);
All you are doing here is putting in a 5second delay on the call, but the call is still being made every Update frame, so you get a 5 second delay before the first call, but then they will be every frame.
set up a boolean to control when the shooting happens.
var fire : bool = true;
function Update()
{
if (fire)
{
Shoot();
fire = false;
}
}
function Shoot()
{
yield WaitForSeconds(3);
//do you projectile stuff
fire = true;
}