I have 3 different async methods say func1, func2, func3 on a struct object.
What I want to accomplish is something like:
loop {
obj.func1().await;
set_timeout(Duration::from_secs(5)).await;
}
loop {
obj.func2().await;
set_timeout(Duration::from_secs(5)).await;
}
loop {
obj.func3().await;
set_timeout(Duration::from_secs(5)).await;
}
I want all these 3 loops to run in parallel. Obviously, in this form it won't work because the 2nd and 3rd loops will be unreachable.
I've thought of the below solution:
loop {
thread::spawn(move || async move {
obj.func1().await;
obj.func2().await;
obj.func3().await;
set_timeout(Duration::from_secs(5)).await;
});
}
But it has 2 issues:
My struct does not implement the Copy trait (using some 3rd party crates, so cannot do much there).
As every function call will run as a seperate thread here, I doubt the timeout will work properly here!
How should I approach this?
You can use an async block to create a new future, so if you wrap each loop in an async block you'll get three futures. Then you can use the join! macro to await them all simultaneously:
let fut1 = async {
loop {
obj.func1().await;
set_timeout(Duration::from_secs(5)).await;
}
};
let fut2 = async {
loop {
obj.func2().await;
set_timeout(Duration::from_secs(5)).await;
}
};
let fut3 = async {
loop {
obj.func3().await;
set_timeout(Duration::from_secs(5)).await;
}
};
join!(fut1, fut2, fut3);
The join! macro will drive the futures.
Alternatively, your async runtime likely has a way to submit a future as a new independent task, such as tokio::spawn.
Related
I am trying to use async within find predicates.
We can use the find method to look up an element within an iterator as follows:
let element = some_elements.iter().find(|element| {
// some_async_method(element).await;
// true or false based on some logic
});
In my case, I need to call async methods as a part of my lookup logic. However, when I use await, I end up with the error that await cannot be called within non-async block, which is expected. When I try to make my function async as follows:
let element = some_elements.iter().find(|element| async move{
// some_async_method(element).await;
// true or false based on some logic
});
I get the error: expected bool, found opaque type
I tried to to use rayon, but could not find an example for what I am looking for.
You can use futures::stream::iter to turn your iterator into a Stream.
Since find is not implemented on StreamExt I replaced it with filter().next() in the following:
use futures::stream::StreamExt;
//…
let element = futures::stream::iter(&some_elements)
.filter(|e| Box::pin(async move { predicate(e).await }))
.next()
.await;
If you actually want to do your work asynchronously though you'd have to be a little more verbose and collect into a FuturesOrdered/FuturesUnordered depending on your actual requirements:
let element = some_elements
.iter()
.map(|&e| async move { (predicate(e).await, e) })
.collect::<FuturesUnordered<_>>()
.filter_map(|(p, e)| Box::pin(async move { p.then_some(e) }))
.next()
.await;
I am trying to implement a class that will perform an action once every second, it works as expected but I am unsure about memory leaks. This code I'm writing will have an uptime of several months.
Will the code below lead to memory leaks since it's technically recursion which never ends?
class Algorithm{
constructor(){
//there will be many more things in this constructor
//which is why this is a class
const pidTimer = (r,e=0,y=Date.now()-r) => {
this.someFunction();
const now = Date.now();
const dy = now-y;
const err = e+r-dy
const u = err*0.2;
//console.log(dy)
setTimeout(()=>{pidTimer(r,err,now)},r+u);
}
pidTimer(1000);
}
someFunction = () => {}
}
It's not the kind of recursion that has any stack accumulation since the previous pidTimer() function call returns before the setTimeout() fires and calls pidTimer() again. I wouldn't even call this recursion (it's scheduled repeated calling), but that's more a semantic issue.
So, the only place I see there could be some memory leak or excess usage would be inside of this.someFunction(); and that's only because you don't show us the code there to evaluate it and see what it does. The code you show us for pidTimer() itself has no issues on its own.
modern async primitive
There's nothing "wrong" with the current function you have, however I think it could be improved significantly. JavaScript offers a modernized asynchrony atom, Promise and new syntax support async/await. These are preferred over stone-age setTimeout and setInterval as you can easily thread data through asynchronous control flow, stop thinking in terms of "callbacks", and avoid side effects -
class Algorithm {
constructor() {
...
this.runProcess(...)
}
async runProcess(...) { // async
while (true) { // loop instead of recursion
await sleep(...) // sleep some amount of time
this.someFunction() // do work
... // adjust timer variables
}
}
}
sleep is a simple function which resolves a promise after a specified milliseconds value, ms -
function sleep(ms) {
return new Promise(r => setTimeout(r, ms)) // promise
}
async iterables
But see how this.someFunction() doesn't return anything? It would be nice if we could capture the data from someFunction and make it available to our caller. By making runProcess an async generator and implementing Symbol.asyncIterator we can easily handle asynchrony and stop side effects -
class Algorithm {
constructor() {
...
this.data = this.runProcess(...) // assign this.data
}
async *runProcess(...) { // async generator
while (true) {
await sleep(...)
yield this.someFunction() // yield
...
}
}
[Symbol.asyncIterator]() { // iterator
return this.data
}
}
Now the caller has control over what happens as the data comes in from this.someFunction. Below we write to console.log but you could easily swap this for an API call or write to file system -
const foo = new Algorithm(...)
for await (const data of foo)
console.log("process data", data) // or API call, or write to file system, etc
additional control
You can easily add control of the process to by using additional data members. Below we swap out while(true) with a conditional and allow the caller to stop the process -
class Algorithm {
constructor() {
...
}
async *runProcess(...) {
this.running = true // start
while (this.running) { // conditional loop
...
}
}
haltProcess() {
this.running = false // stop
}
...
}
demo
Here's a functioning demo including the concepts above. Note we only implement halt here because run is an infinite generator. Manual halting is not necessary for finite generators. Verify the results in your own browser by running the snippet -
class Algorithm {
async *run() {
this.running = true
while(this.running) {
await sleep(1000)
yield this.someFunction()
}
}
halt() {
this.running = false
}
someFunction() {
return Math.random()
}
[Symbol.asyncIterator] = this.run
}
function sleep(ms) {
return new Promise(r => setTimeout(r, ms))
}
async function main() {
const foo = new Algorithm // init
setTimeout(_ => foo.halt(), 10000) // stop at some point, for demo
for await (const x of foo) // iterate
console.log("data", x) // log, api call, write fs, etc
return "done" // return something when done
}
main().then(console.log, console.error) // "done"
data 0.3953947360028206
data 0.18754462176783115
data 0.23690422070864803
data 0.11237466374294014
data 0.5123244720637253
data 0.39818889343799635
data 0.08627407687877853
data 0.3861902404922477
data 0.8358471443658225
data 0.2770336562516085
done
So far what I have working is calling each future one after the other. All of which adds to the list.
However, sometimes one of the APIs may not work and therefore the rest that have not been loaded are bottlenecked. I'd like to start all the futures at the same time and have them all run in parallel to populate the list as results come on from any of them. How can I achieve this?
TLDR: Multi threading in dart/flutter?
If you have 3 futures to execute in parallel:
Future<int> async1() async {
await Future<String>.delayed(const Duration(seconds: 1));
return 10;
}
Future<int> async2() async {
await Future<String>.delayed(const Duration(seconds: 2));
return 20;
}
Future<int> async3() async {
await Future<String>.delayed(const Duration(seconds: 3));
return 30;
}
You can execute them in parallel and wait for all of them to finish by doing this:
Future.wait([async1(), async2(), async3()]).then((List<int> nums) {
// This code executes in 3 seconds (3 endpoints in parallel)
});
I'm creating a program where I constantly run and stop async code, but I need a good way to stop the code.
Currently, I have tried to methods:
Method 1:
When a method is running, and another method is called to stop the first method, I start an infinite loop to stop that code from running and then remove the method from the queue(array)
I'm 100% sure that this is the worst way to accomplish it, and it works very buggy.
Code:
class test{
async Start(){
const response = await request(options);
if(stopped){
while(true){
await timeout(10)
}
}
}
}
Code 2:
var tests = [];
Start(){
const test = new test();
tests.push(test)
tests.Start();
}
Stop(){
tests.forEach((t, i) => {t.stopped = true;};
tests = [];
}
Method 2:
I load the different methods into Workers, and when I need to stop the code, I just terminate the Worker.
It always takes a lot of time(1 sec) to create the Worker, and therefore not the best way, since I need the code to run without 1-2 sec pauses.
Code:
const Worker = require("tiny-worker");
const code = new Worker(path.resolve(__dirname, "./Code/Code.js"))
Stopping:
code.terminate()
Is there any other way that I can stop async code?
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code without one of the 2 methods.
Is there any other way that I can stop async code?
Keep in mind the basic of how Nodejs works. I think there is some misunderstanding here.
It execute the actual function in the actual context, if encounters an async operation the event loop will schedule it's execetution somewhere in the future. There is no way to remove that scheduled execution.
More info on event loop here.
In general for manage this kind of situations you shuold use flags or semaphores.
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code
If you need to hard "stop the code" you can do something like
func stop() {
process.exit()
}
But if i'm getting it right, you're launching requests every x time, at some point you need to stop sending the request without managing the response.
You can't de-schedule the response managemente portion, but you can add some logic in it to (when it will be runned) check if the "request loop" has been stopped.
let loop_is_stopped = false
let sending_loop = null
func sendRequest() {
const response = await request(options) // "wait here"
// following lines are scheduled after the request promise is resolved
if (loop_is_stopped) {
return
}
// do something with the response
}
func start() {
sending_loop = setInterval(sendRequest, 1000)
}
func stop() {
loop_is_stopped = true
clearInterval(sending_loop)
}
module.exports = { start, stop }
We can use Promise.all without killing whole app (process.exit()), here is my example (you can use another trigger for calling controller.abort()):
const controller = new AbortController();
class Workflow {
static async startTask() {
await new Promise((res) => setTimeout(() => {
res(console.log('RESOLVE'))
}, 3000))
}
}
class ScheduleTask {
static async start() {
return await Promise.all([
new Promise((_res, rej) => { if (controller.signal.aborted) return rej('YAY') }),
Workflow.startTask()
])
}
}
setTimeout(() => {
controller.abort()
console.log("ABORTED!!!");
}, 1500)
const run = async () => {
try {
await ScheduleTask.start()
console.log("DONE")
} catch (err) {
console.log("ERROR", err.name)
}
}
run()
// ABORTED!!!
// RESOLVE
"DONE" will never be showen.
res will be complited
Maybe would be better to run your code as script with it's own process.pid and when we need to interrupt this functionality we can kill this process by pid in another place of your code process.kill.
Suppose I have the following code:
foo();
function foo() {
func1("bla", function() {
console.log("done!");
});
}
function func1(value,callback) {
process.nextTick(callback);
}
Will the function above will be totally async ?
Or should I use this foo function? :
function foo() {
process.nextTick(function() {
func1("bla", function() {
console.log("done!");
});
}
Actually my question is if the parent blocks the child process from being Async ?
The first option is going to be "async" in the sense that node might do other things before calling the callback method.
There is no need to call the second method. As soon as your foo function finishes and any parent callers of foo finish node will start doing other work, which eventually will be the work registered by nextTick.