How to stop async code from running Node.JS - node.js

I'm creating a program where I constantly run and stop async code, but I need a good way to stop the code.
Currently, I have tried to methods:
Method 1:
When a method is running, and another method is called to stop the first method, I start an infinite loop to stop that code from running and then remove the method from the queue(array)
I'm 100% sure that this is the worst way to accomplish it, and it works very buggy.
Code:
class test{
async Start(){
const response = await request(options);
if(stopped){
while(true){
await timeout(10)
}
}
}
}
Code 2:
var tests = [];
Start(){
const test = new test();
tests.push(test)
tests.Start();
}
Stop(){
tests.forEach((t, i) => {t.stopped = true;};
tests = [];
}
Method 2:
I load the different methods into Workers, and when I need to stop the code, I just terminate the Worker.
It always takes a lot of time(1 sec) to create the Worker, and therefore not the best way, since I need the code to run without 1-2 sec pauses.
Code:
const Worker = require("tiny-worker");
const code = new Worker(path.resolve(__dirname, "./Code/Code.js"))
Stopping:
code.terminate()
Is there any other way that I can stop async code?
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code without one of the 2 methods.

Is there any other way that I can stop async code?
Keep in mind the basic of how Nodejs works. I think there is some misunderstanding here.
It execute the actual function in the actual context, if encounters an async operation the event loop will schedule it's execetution somewhere in the future. There is no way to remove that scheduled execution.
More info on event loop here.
In general for manage this kind of situations you shuold use flags or semaphores.
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code
If you need to hard "stop the code" you can do something like
func stop() {
process.exit()
}
But if i'm getting it right, you're launching requests every x time, at some point you need to stop sending the request without managing the response.
You can't de-schedule the response managemente portion, but you can add some logic in it to (when it will be runned) check if the "request loop" has been stopped.
let loop_is_stopped = false
let sending_loop = null
func sendRequest() {
const response = await request(options) // "wait here"
// following lines are scheduled after the request promise is resolved
if (loop_is_stopped) {
return
}
// do something with the response
}
func start() {
sending_loop = setInterval(sendRequest, 1000)
}
func stop() {
loop_is_stopped = true
clearInterval(sending_loop)
}
module.exports = { start, stop }

We can use Promise.all without killing whole app (process.exit()), here is my example (you can use another trigger for calling controller.abort()):
const controller = new AbortController();
class Workflow {
static async startTask() {
await new Promise((res) => setTimeout(() => {
res(console.log('RESOLVE'))
}, 3000))
}
}
class ScheduleTask {
static async start() {
return await Promise.all([
new Promise((_res, rej) => { if (controller.signal.aborted) return rej('YAY') }),
Workflow.startTask()
])
}
}
setTimeout(() => {
controller.abort()
console.log("ABORTED!!!");
}, 1500)
const run = async () => {
try {
await ScheduleTask.start()
console.log("DONE")
} catch (err) {
console.log("ERROR", err.name)
}
}
run()
// ABORTED!!!
// RESOLVE
"DONE" will never be showen.
res will be complited
Maybe would be better to run your code as script with it's own process.pid and when we need to interrupt this functionality we can kill this process by pid in another place of your code process.kill.

Related

Reporting during long running async function using set interval in node.js (backend service)

I need to report the status of a long running operation in node.js. The basic use case is outlined in the code below. awaiting the longProcess method I know will act synchronously to the caller, but I must await the method in my code. Should I handle this within the longProcess method? Not sure how to address this issue.
function sleep (ms: number) {
new Promise(resolve => setTimeout(resolve, ms));
}
let processedCount = 0;
async function longProcess() {
// really long operation
while (true) {
processedCount++;
await sleep(1000); // simulate long process
if (processedCount === 10) // just to end the test somehow
break;
}
}
async function report() {
console.log(processedCount);
}
async function main() {
const id = setInterval(report, 500);
await longProcess();
clearInterval(id);
}
main().then(() => console.log("Done"));
The sleep method is just for demonstration purposes to simulate a long running operation. 'longProcess' performs complex and time intensive processing. It calls a callback passed in to report back a processed count the caller. The class that contains the calling method (and the callback), also has a report method that I would like to call at regular intervals. And I need to be able to create a unit test for this
Your sleep function is not returning the promise you are creating. You are calling await on the value returned from the function, which in this case is undefined so it doesn't actually wait at all.
function sleep (ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}

Javascript recursion, memory leak?

I am trying to implement a class that will perform an action once every second, it works as expected but I am unsure about memory leaks. This code I'm writing will have an uptime of several months.
Will the code below lead to memory leaks since it's technically recursion which never ends?
class Algorithm{
constructor(){
//there will be many more things in this constructor
//which is why this is a class
const pidTimer = (r,e=0,y=Date.now()-r) => {
this.someFunction();
const now = Date.now();
const dy = now-y;
const err = e+r-dy
const u = err*0.2;
//console.log(dy)
setTimeout(()=>{pidTimer(r,err,now)},r+u);
}
pidTimer(1000);
}
someFunction = () => {}
}
It's not the kind of recursion that has any stack accumulation since the previous pidTimer() function call returns before the setTimeout() fires and calls pidTimer() again. I wouldn't even call this recursion (it's scheduled repeated calling), but that's more a semantic issue.
So, the only place I see there could be some memory leak or excess usage would be inside of this.someFunction(); and that's only because you don't show us the code there to evaluate it and see what it does. The code you show us for pidTimer() itself has no issues on its own.
modern async primitive
There's nothing "wrong" with the current function you have, however I think it could be improved significantly. JavaScript offers a modernized asynchrony atom, Promise and new syntax support async/await. These are preferred over stone-age setTimeout and setInterval as you can easily thread data through asynchronous control flow, stop thinking in terms of "callbacks", and avoid side effects -
class Algorithm {
constructor() {
...
this.runProcess(...)
}
async runProcess(...) { // async
while (true) { // loop instead of recursion
await sleep(...) // sleep some amount of time
this.someFunction() // do work
... // adjust timer variables
}
}
}
sleep is a simple function which resolves a promise after a specified milliseconds value, ms -
function sleep(ms) {
return new Promise(r => setTimeout(r, ms)) // promise
}
async iterables
But see how this.someFunction() doesn't return anything? It would be nice if we could capture the data from someFunction and make it available to our caller. By making runProcess an async generator and implementing Symbol.asyncIterator we can easily handle asynchrony and stop side effects -
class Algorithm {
constructor() {
...
this.data = this.runProcess(...) // assign this.data
}
async *runProcess(...) { // async generator
while (true) {
await sleep(...)
yield this.someFunction() // yield
...
}
}
[Symbol.asyncIterator]() { // iterator
return this.data
}
}
Now the caller has control over what happens as the data comes in from this.someFunction. Below we write to console.log but you could easily swap this for an API call or write to file system -
const foo = new Algorithm(...)
for await (const data of foo)
console.log("process data", data) // or API call, or write to file system, etc
additional control
You can easily add control of the process to by using additional data members. Below we swap out while(true) with a conditional and allow the caller to stop the process -
class Algorithm {
constructor() {
...
}
async *runProcess(...) {
this.running = true // start
while (this.running) { // conditional loop
...
}
}
haltProcess() {
this.running = false // stop
}
...
}
demo
Here's a functioning demo including the concepts above. Note we only implement halt here because run is an infinite generator. Manual halting is not necessary for finite generators. Verify the results in your own browser by running the snippet -
class Algorithm {
async *run() {
this.running = true
while(this.running) {
await sleep(1000)
yield this.someFunction()
}
}
halt() {
this.running = false
}
someFunction() {
return Math.random()
}
[Symbol.asyncIterator] = this.run
}
function sleep(ms) {
return new Promise(r => setTimeout(r, ms))
}
async function main() {
const foo = new Algorithm // init
setTimeout(_ => foo.halt(), 10000) // stop at some point, for demo
for await (const x of foo) // iterate
console.log("data", x) // log, api call, write fs, etc
return "done" // return something when done
}
main().then(console.log, console.error) // "done"
data 0.3953947360028206
data 0.18754462176783115
data 0.23690422070864803
data 0.11237466374294014
data 0.5123244720637253
data 0.39818889343799635
data 0.08627407687877853
data 0.3861902404922477
data 0.8358471443658225
data 0.2770336562516085
done

sync/async functions and frozen background loop

I'm trying to dev a lottery game on top of a blockchain.
In server.js, this loop is intended to run "in background" and gather data on the blockchain every second.
let test = 0;
setTimeout(async function run() {
test++;
console.log(test);
setTimeout(run, 1000);
}, 1000);
In an other module, I want to allow user to login with a keystore/password :
socket.on("login", data => {
helpers.checkCredentials(data).then(r => {
//console.log(jwt.verify(r, process.env.JWT_PASS));
//console.log(r);
io.emit("login", r);
});
});
that calls checkCredentials :
const int4 = require("int4.js");
module.exports = async function(cred) {
let account = null;
let RPC = int4.rpc(process.env.URL_RPC);
if (cred.keystore !== undefined && cred.password !== undefined) {
try {
account = int4.keystore.fromV3Keystore(cred.keystore, cred.password);
console.log(account);
} catch (e) {
return "bad password or keystore";
}
(...)
But the function int4.keystore.fromV3Keystore takes time to decrypt the keystore (approx. 5-7s) and during this time, my counter doesn't run, it's frozen.
I think I'm not familiar enough with practice of async functions, and despite a lot of readings I can't figure out why it freezes.
I tried to use util to util.promisify the int4.keystore.fromV3Keystore, hoping for some "async acting" but... nope.
Can you help me getting this "background" loop independant from the other functions ? Or making a clean code that would run every functions really asynchronously ?
Thanks a lot.
I use the beta lib https://github.com/intfoundation/int4.js for the INTChain testnet keystore decrypt
All of these async functions are running on the same thread. Promises and async/await is a way to get this one thread to run things asynchronously.
A result of this is that if any of these functions take too long before yielding execution (return or await on some other), the event loop gets stuck.
What you need is actual multithreading. You need to create a worker thread in which you run the long-running function and get the result back.
Here are some pointers to get started:
https://blog.logrocket.com/node-js-multithreading-what-are-worker-threads-and-why-do-they-matter-48ab102f8b10/
https://blog.logrocket.com/a-complete-guide-to-threads-in-node-js-4fa3898fe74f/

How to stop BrowserWindow from breaking after a single error in executeJavaScript?

I'm using the executeJavaScript method to control a page via Electron. However, I've noticed that the BrowserWindow instance essentially becomes unresponsive after one error occurs. Aside from putting every single line of code in a try/catch block, is it possible to disable this functionality?
Specifically, I'd prefer that the BrowserWindow continues to execute future code despite running into an error on a previous request. For example, in the code below, I want the console to successfully output I'm not being executed.
const {BrowserWindow, app} = require('electron')
async function main() {
var win = new BrowserWindow({webPreferences: {nodeIntegration: false} });
win.openDevTools();
win.loadURL('https://www.homedepot.com');
await sleep(10000); //letting page load
await win.webContents.executeJavaScript('console.log("Im being executed")')
await sleep(2000);//break
await win.webContents.executeJavaScript('undefinedVar.causeError()')
await sleep(2000);//break
await win.webContents.executeJavaScript('console.log("Im not being executed")')
}
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
app.on('ready', main)
First, I don't believe the BrowserWindow is unresponsive after executing the broken code. You can still move it around and maximize it, etc. It's just that the Promise returned by executeJavaScript is stuck in a pending state, and so you never finish awaiting it.
This is actually fixed in Electron 6 by this pull request. In Electron 6, you can do this:
var win;
async function createWindow2() {
win = new BrowserWindow();
win.openDevTools();
await win.loadURL('https://www.google.com');
await executeSafely('console.log("Im being executed")');
await executeSafely('undefinedVar.causeError()');
await executeSafely('console.log("Im not being executed")');
}
async function executeSafely(code) {
try {
await win.webContents.executeJavaScript(code);
}
catch (e) { } // assuming you want to ignore errors
}
If upgrading Electron isn't an option for you, then you can create a function that wraps your code in a Promise, or a try/catch depending on what you need.
If you just want to ignore all errors, then wrapping your executed-code in a try/catch is sufficient. Otherwise, you can wrap in a Promise, which should result in the Promise returned by executeJavaScript being rejected. Example:
async function main() {
var win = new BrowserWindow();
win.openDevTools();
win.loadURL('https://www.google.com');
await sleep(5000);
await win.webContents.executeJavaScript(makeSafe('console.log("Im being executed")'));
await win.webContents.executeJavaScript(makeSafe('undefinedVar.causeError()'));
await win.webContents.executeJavaScript(makeSafe('console.log("Im not being executed")'));
}
function makeSafe(code) {
return "try{" + code + "}catch{}";
}

Doing fs.watch until it succeeds

I'm watching multiple network folders using fs.watch(), each on a different IP. The problem is that sometimes these network drives go offline, and I need to retry watching them until they go online.
The code I wrote looks like this:
watchFolder() {
var self = this
let path = `\\\\${this.pathIP}\\folder1`
try {
fs.watch(path, (eventType, filename) => {
...
}
}
catch (err) {
//drive is offline
setTimeout ( () => {
this.watchFolder()
}, 1000);
}
}
I tried calling the same function from the catch every second, so that if it goes online it would continue normally, but apparently this takes too much memory because of the recursion. What can be an alternative way to do what I am trying to achieve?
One solution is to refactor your code into a loop, avoiding recursion, and to use the new sleep feature in JS to implement delay.
let keep_checking = true;
while (keep_checking) { // exit condition
let path = `\\\\${this.pathIP}\\folder1`
try {
fs.watch(path, (eventType, filename) => {
// Drive is online, do stuff
// ...
// Remember to set the exit condition
// eg. keep_checking = false;
}
}
catch (err) {
// Drive is offline
await sleep(1000);
}
}
This post gives a bit more details on the usage of sleep:
What is the JavaScript version of sleep()?

Resources