I'm using .Net Core 3.1 and I want to insert bulk data in the background, so I don't need my http request waiting for it "like fire and forget"
So I tried the following code
public object myFunction(){
Task.Factor.StartNew(() => {
_context.BulkInsertAsync(logs);
});
return data;
}
But nothing is happend, no data saved in database
is after my data returned my _context and logs will be null, so the process is filed?
or there is any another method to insert my data and don't wait for it
Note: the background task working if I replace insertion statment with sending mail or any other thing
Solved:
Thanks #Peter , I solved it using
Task.Run(async () => await _context.BulkInsertAsync(logs));
Task.Factory.StartNew or TaskFactory.StartNew cannot accept async delegates (Func<Task>), so you should use Task.Run instead which is indeed has an overload with Func < Task >. You would have to use await await or Unwrap against StartNew to get the same behaviour as with Run. Please read Stephen Toub's excellent blog post.
public object myFunction(){
Task.Run(async () => await _context.BulkInsertAsync(logs));
return data;
}
This is how I would implement it. This assumes the response is not important.
public NotImportantResult ProcessBulkData()
{
myFunctionAsync();
return new NotImportantResult()
}
private static async void myFunctionAsync()
{
await Task.Factory.StartNew(() => new MyBulkProccessor.BulkInsertAsync(logs));
}
Related
I am working with MySQL server, building a NodeJS server.
SomeWhere in the code I have async code that is not waiting, I believe its has something to do with the forEach function.
Is there anything wrong with my implementation of my next function:
async addResponsesToTasks(input, response, isFromCache) {
if (!isFromCache) {
this.saveResponseToCache(input, response);
}
console.log("===7===");
await response.forEach(async (pokemon) => {
console.log("===8===", pokemon);
await this.addTaskToFile(pokemon, false);
console.log("===13===");
});
return true;
}
And this is the output shows that it is not waiting for creation and save in the DB to finish, and jump to do some other things:
Prints of the output - can see 10,12,13,14,15 and only then 11
Hope you could maybe spot what I am missing.
Thanks in advance!
What's happening here is more of a JavaScript thing than a Sequelize thing.
The await is being applied to the lines of code that follow Item.create() inside of this function, but outer functions are continuing to execute.
console.log('9')
saveTaskToDb(task)
console.log('12')
In this code, '12' is going to get logged out before your '11', because this outer code is going to continue to execute, even if the code in saveTaskToDb is awaiting the resolution of the create() method.
To solve this, you'll either need to move all of your asynchronous code into the same code block, or you'll need to return your own promise in the saveTaskToDb function, and await saveTaskToDb.
So Solution was not using forEach, rather using for of loop like that:
async addResponsesToTasks(input, response, isFromCache) {
if (!isFromCache) {
this.saveResponseToCache(input, response);
}
for (const pokemon of response) {
await this.addTaskToFile(pokemon, false);
}
return true;
}
Based on this question: Using async/await with a forEach loop
I'm creating a program where I constantly run and stop async code, but I need a good way to stop the code.
Currently, I have tried to methods:
Method 1:
When a method is running, and another method is called to stop the first method, I start an infinite loop to stop that code from running and then remove the method from the queue(array)
I'm 100% sure that this is the worst way to accomplish it, and it works very buggy.
Code:
class test{
async Start(){
const response = await request(options);
if(stopped){
while(true){
await timeout(10)
}
}
}
}
Code 2:
var tests = [];
Start(){
const test = new test();
tests.push(test)
tests.Start();
}
Stop(){
tests.forEach((t, i) => {t.stopped = true;};
tests = [];
}
Method 2:
I load the different methods into Workers, and when I need to stop the code, I just terminate the Worker.
It always takes a lot of time(1 sec) to create the Worker, and therefore not the best way, since I need the code to run without 1-2 sec pauses.
Code:
const Worker = require("tiny-worker");
const code = new Worker(path.resolve(__dirname, "./Code/Code.js"))
Stopping:
code.terminate()
Is there any other way that I can stop async code?
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code without one of the 2 methods.
Is there any other way that I can stop async code?
Keep in mind the basic of how Nodejs works. I think there is some misunderstanding here.
It execute the actual function in the actual context, if encounters an async operation the event loop will schedule it's execetution somewhere in the future. There is no way to remove that scheduled execution.
More info on event loop here.
In general for manage this kind of situations you shuold use flags or semaphores.
The program contains Request using nodejs Request-promise module, so program is waiting for requests, it's hard to stop the code
If you need to hard "stop the code" you can do something like
func stop() {
process.exit()
}
But if i'm getting it right, you're launching requests every x time, at some point you need to stop sending the request without managing the response.
You can't de-schedule the response managemente portion, but you can add some logic in it to (when it will be runned) check if the "request loop" has been stopped.
let loop_is_stopped = false
let sending_loop = null
func sendRequest() {
const response = await request(options) // "wait here"
// following lines are scheduled after the request promise is resolved
if (loop_is_stopped) {
return
}
// do something with the response
}
func start() {
sending_loop = setInterval(sendRequest, 1000)
}
func stop() {
loop_is_stopped = true
clearInterval(sending_loop)
}
module.exports = { start, stop }
We can use Promise.all without killing whole app (process.exit()), here is my example (you can use another trigger for calling controller.abort()):
const controller = new AbortController();
class Workflow {
static async startTask() {
await new Promise((res) => setTimeout(() => {
res(console.log('RESOLVE'))
}, 3000))
}
}
class ScheduleTask {
static async start() {
return await Promise.all([
new Promise((_res, rej) => { if (controller.signal.aborted) return rej('YAY') }),
Workflow.startTask()
])
}
}
setTimeout(() => {
controller.abort()
console.log("ABORTED!!!");
}, 1500)
const run = async () => {
try {
await ScheduleTask.start()
console.log("DONE")
} catch (err) {
console.log("ERROR", err.name)
}
}
run()
// ABORTED!!!
// RESOLVE
"DONE" will never be showen.
res will be complited
Maybe would be better to run your code as script with it's own process.pid and when we need to interrupt this functionality we can kill this process by pid in another place of your code process.kill.
So no matter what I've read, even once I do it right, I can't seem to get the the hang of async and await. For example I have this in my startup.
startup.js
await CommandBus.GetInstance();
await Consumers.GetInstance();
Debugging jumps to the end of the get instance for CommandBus (starting up a channel for rabbitmq) and start Consumers.GetInstance() which fails since channel is null.
CommandBus.js
export default class CommandBus {
private static instance: CommandBus;
private channel: any;
private conn: Connection;
private constructor() {
this.init();
}
private async init() {
//Create connection to rabbitmq
console.log("Starting connection to rabbit.");
this.conn = await connect({
protocol: "amqp",
hostname: settings.RabbitIP,
port: settings.RabbitPort,
username: settings.RabbitUser,
password: settings.RabbitPwd,
vhost: "/"
});
console.log("connecting channel.");
this.channel = await this.conn.createChannel();
}
static async GetInstance(): Promise<CommandBus> {
if (!CommandBus.instance) {
CommandBus.instance = new CommandBus();
}
return CommandBus.instance;
}
public async AddConsumer(queue: Queues) {
await this.channel.assertQueue(queue);
this.channel.consume(queue, msg => {
this.Handle(msg, queue);
});
}
}
Consumers.js
export default class Consumers {
private cb: CommandBus;
private static instance: Consumers;
private constructor() {
this.init();
}
private async init() {
this.cb = await CommandBus.GetInstance();
await cb.AddConsumer(Queues.AuthResponseLogin);
}
static async GetInstance(): Promise<Consumers> {
if (!Consumers.instance) {
Consumers.instance = new Consumers();
}
return Consumers.instance;
}
}
Sorry I realize this is in Typescript, but I imagine that doesn't matter. The issue occurs specifically when calling cb.AddConsumer which can be found CommandBus.js. It tries to assert a queue against a channel that doesn't exist yet. What I don't understand, is looking at it. I feel like I've covered all the await areas, so that it should wait on channel creation. The CommandBus is always fetched as a singleton. I don't if this poses issues, but again it is one of those areas that I cover with awaits as well. Any help is great thanks everyone.
You can't really use asynchronous operations in a constructor. The problem is that the constructor needs to return your instance so it can't also return a promise that will tell the caller when it's done.
So, in your Consumers class, await new Consumers(); is not doing anything useful. new Consumers() returns a new instance of a Consumers object so when you await that it doesn't actually wait for anything. Remember that await does something useful with you await a promise. It doesn't have any special powers to await your constructor being done.
The usual way around this is to create a factory function (which can be a static in your design) that returns a promise that resolves to the new object.
Since you're also trying to make a singleton, you would cache the promise the first time you create it and always return the promise to the caller so the caller would always use .then() to get the finished instance. The first time they call it, they'd get a promise that was still pending, but later they'd get a promise that was already fulfilled. In either case, they just use .then() to get the instance.
I don't know TypeScript well enough to suggest to you the actual code for doing this, but hopefully you get the idea from the description. Turn GetInstance() into a factory function that returns a promise (that you cache) and have that promise resolve to your instance.
Something like this:
static async GetInstance(): Promise<Consumers> {
if (!Consumers.promise) {
let obj = new Consumers();
Consumers.promise = obj.init().then(() => obj);
}
return Consumers.promise;
}
Then, the caller would do:
Consumers.getInstance().then(consumer => {
// code here to use the singleton consumer object
}).catch(err => {
console.log("failed to get consumer object");
});
You will have to do the same thing in any class that has async operations involved in initializing the object (like CommandBus) too and each .init() call needs to call the base class super.init().then(...) so base class can do its thing to get properly initialized too and the promise your .init() is linked to the base class too. Or, if you're creating other objects that themselves have factory functions, then your .init() needs to call those factory functions and link their promises together so the .init() promise that is returned is linked to the other factory function promises too (so the promise your .init() returns will not resolve until all dependent objects are all done).
I need to refresh page while element is not presented
i'm trying something like this, but it doesn't help
When(/^"([^"]*)" task status changed$/, taskName => {
let needRefresh = true;
do {
client.url(`${client.globals.env.url}${client.globals.env.index}/messaging/messages`)
.pause(10000)
.getTagName(`//div[contains(#class, "task-checkbox")]//*[contains(text(), "${taskName}")]`, res => {
client.equal(res.value, 'div')
}).pause(20000);
} while (!needRefresh)
});
how to do it correctly?
To be able to use complex asynchronous operations I suggest to use the new async functions. If your NodeJs version does not support it natively I suggest to use Babel. There is an example for that in the nightwatch-cucumber example folder. To be able to refresh the page until some condition you can use the following example.
When(/^"([^"]*)" task status changed$/, async (taskName) => {
let needRefresh = true;
do {
await client.refresh();
await client.pause(10000);
needRefresh = await client.getTagName(...
} while (!needRefresh)
});
Specifically when doing MonoDroid uses of threads all the documentation I can find recommends calling RunOnUiThread() to call the callback. There is a similar function that can be used on MonoTouch however both of them require a GUI (Activity or whatever its counter part is on IOS). What I would like is to be able to start a thread, pass in a callback and call that callback on the thread that started the thread. For example
ThreadPool.QueueUserWorkItem(state =>
{
//Do Stuff
execute_callback_magically_on_main_thread(() => callback(response));
});
Any ideas? To be clear I would prefer this to not need a handle to the Activity etc.
What if you do something like this? (assuming they have the same signature) I haven't messed with RunOnUiThread, so I don't know it's signature.
public delegate void InvokeOnUIMethod(Action action);
public void CallingMethod()
{
//iOS
MyMethod(InvokeOnMainThread, () => { /* Your callback functionality */ });
//Android
MyMethod(RunOnUiThread, () => { /* Your callback functionality */ });
}
public void MyMethod(InvokeOnUIMethod execute_callback_magically_on_main_thread, Action callback)
{
System.Threading.ThreadPool.QueueUserWorkItem(state =>
{
//Do Stuff
execute_callback_magically_on_main_thread(() => callback(response));
});
}
I hope this helps.
Using the Alpha builds (Hopefully soon to be available as stable) you can use the new Async await idiom.
here is an overview on MSDN:
http://msdn.microsoft.com/en-gb/library/vstudio/hh191443.aspx
and here is a great video series on Channel9:
http://channel9.msdn.com/Series/Three-Essential-Tips-for-Async/Tip-1-Async-void-is-for-top-level-event-handlers-only
I found a solution that works and does not appear to be dependent on either platform.
Task<string> t = new Task<string>(() =>
{
//Do Stuff
return "my results";
});
t.ContinueWith(task =>{
if(callback != null)
callback(task.Result);
}, TaskScheduler.FromCurrentSynchronizationContext());
t.Start();
The important part is the TaskScheduler.FromCurrentSynchronizationContext() which tells the "ContinueWith" to execute on the original thread.