Multithreading for MultiMethods in Flutter - multithreading

may i ask how to multithreading some functions in dart
for example i have 16 Future functions
i want first to execute this function in the first thread
await GetRoot(); //Future<void>
then after the first thread end i want to execute theses functions in the second thread -togather at the same time-
await GetLocalRadio();//Future<void>
await GetMusic();//Future<void>
await GetTalk();//Future<void>
await GetSports();//Future<void>
await GetByLocation();//Future<void>
await GetByLanguage();//Future<void>
await GetPodcasts();//Future<void>
then after the second thread is finished , i want to execute these threads togather at the same time in the next thread
await GetPodcasts2();//Future<void>
await GetByLocation2();//Future<void>
then when the second thread end i want when the third functions is finished to execute these functions togather at the same time in a new thread
await FillPdcasts();//Future<void>
await FillLocalRadio();//Future<void>
await FillMusic();//Future<void>
await FillTalk();//Future<void>
await FillSports();//Future<void>
await FillByLocation();//Future<void>
here is a simple of the content of all methods that i have
Future<void> FillByLocation()
async {
for (var entry in xByLocation2) {
final responseFinalByLocation2 = await http.Client().get(entry.split('^')[1], headers: {'Content-Type': 'application/json; charset=utf-8'});
try{
final FinalByLocation2final = allInfosFromJson(utf8.decode(responseFinalByLocation2.bodyBytes));
for (int i = 0; i < FinalByLocation2final.body.length; i++) {
for (int j = 0; j < FinalByLocation2final.body[i].children.length; j++) {
if(FinalByLocation2final.body[i].children[j].text != null && FinalByLocation2final.body[i].children[j].image != null)
{
if(!xListFinalAllInfos.contains(FinalByLocation2final.body[i].children[j].text + "^" + FinalByLocation2final.body[i].children[j].subtext + "^" + FinalByLocation2final.body[i].children[j].nowPlayingId + "^" + FinalByLocation2final.body[i].children[j].currentTrack+ "^" + FinalByLocation2final.body[i].children[j].url+ "^" + FinalByLocation2final.body[i].children[j].image))
{
setState(() {
xListFinalAllInfos.add(FinalByLocation2final.body[i].children[j].text + "^" + FinalByLocation2final.body[i].children[j].subtext + "^" + FinalByLocation2final.body[i].children[j].nowPlayingId + "^" + FinalByLocation2final.body[i].children[j].currentTrack+ "^" + FinalByLocation2final.body[i].children[j].url+ "^" + FinalByLocation2final.body[i].children[j].image);
// ListFinalAllInfos.putIfAbsent(
// FinalByLocation2final.body[i].children[j].text, () =>
// FinalByLocation2final.body[i].children[j].image );
});
}
}
}
}
}catch(e){}
}
}
Because when I execute all the functions and await all of them, then this will take about 30 seconds.
This is a lot of time, so I need to make the time less than 30 seconds.

Multithreading is not an option here. Just stick with asynchrony.
All of your awaits are making your code much slower. You're forcing the previous function to finish before the next function can even start. Call them all at the same time and use Future.wait.
await GetRoot();
List<Future<void>> toWait = [];
toWait.add(GetLocalRadio());
toWait.add(GetMusic());
...
await Future.wait(toWait);
toWait.clear();
toWait.add(GetPodcasts2());
toWait.add(GetByLocation2());
await Future.wait(toWait);
toWait.clear();
toWait.add(FillPdcasts());
toWait.add(FillLocalRadio());
...
await Future.wait(toWait);
toWait.clear();
Additionally, the work you're doing within these function is also implemented inefficiently. You're waiting for each http call to finish before processing the data from the call and starting the next one. This can be made more efficient.
The following code uses .then to register callbacks for each of the network calls, stores the futures to a list, and uses Future.wait at the very end to ensure they're all completed before the wrapper function returns.
Future<void> FillByLocation()
async {
List<Future<void>> toWait = [];
for (var entry in xByLocation2) {
toWait.add(http.Client().get(entry.split('^')[1], headers: {'Content-Type': 'application/json; charset=utf-8'}).then((responseFinalByLocation2){
try{
final FinalByLocation2final = allInfosFromJson(utf8.decode(responseFinalByLocation2.bodyBytes));
for (int i = 0; i < FinalByLocation2final.body.length; i++) {
for (int j = 0; j < FinalByLocation2final.body[i].children.length; j++) {
if(FinalByLocation2final.body[i].children[j].text != null && FinalByLocation2final.body[i].children[j].image != null)
{
if(!xListFinalAllInfos.contains(FinalByLocation2final.body[i].children[j].text + "^" + FinalByLocation2final.body[i].children[j].subtext + "^" + FinalByLocation2final.body[i].children[j].nowPlayingId + "^" + FinalByLocation2final.body[i].children[j].currentTrack+ "^" + FinalByLocation2final.body[i].children[j].url+ "^" + FinalByLocation2final.body[i].children[j].image))
{
setState(() {
xListFinalAllInfos.add(FinalByLocation2final.body[i].children[j].text + "^" + FinalByLocation2final.body[i].children[j].subtext + "^" + FinalByLocation2final.body[i].children[j].nowPlayingId + "^" + FinalByLocation2final.body[i].children[j].currentTrack+ "^" + FinalByLocation2final.body[i].children[j].url+ "^" + FinalByLocation2final.body[i].children[j].image);
// ListFinalAllInfos.putIfAbsent(
// FinalByLocation2final.body[i].children[j].text, () =>
// FinalByLocation2final.body[i].children[j].image );
});
}
}
}
}
}catch(e){}
}));
}
await Future.wait(toWait);
}

Related

Run an event only once in NodeJS but use the data multiple times

So I have written this code :
manager.getInventoryContents(730, 2, true, (err,inventory,currencies) => {
if (err){
console.log(err)
} else {
console.log("Create order for : " + orderitemname.length + " items.")
var otn = orderitemname.length -1;
while (otn !== -1) {
var li= inventory.length - 1;
while (li !== -1){
if (inventory[li].market_name === orderitemname[otn]){
console.log("Add item to trade " + orderitemname[otn]);
li = li -1;
otn = otn -1;
} else {
console.log("ERR !!! ITEM NOT IN INVENTORY !!! " + orderitemname[otn]);
//Change Order Status To Failed !!
n = n-1;
otn = -1;
li = -1;
}
}
}
}
})
So what is happening is that the process is calling this event multiple times (since it is in a while loop), and the node module is ignoreing it saying :
Error: The request is a duplicate and the action has already occurred in the past, ignored this time
So is there a way I can just call the event once, save it to a variable or something and then use the data multiple time ?
You can wrap everything in a new context and initiate whatever variable you need there. Then, the variable will be accessible from within your callback function.
Something like this:
(() => {
let ctx_data = 0;
manager.getInventoryContents(730, 2, true, (err,inventory,currencies) => {
console.log(++ctx_data);
});
})();

FOR LOOP in Node js needing the result of the previous iteration

I need to do a loop in Node JS where I need the result of the previous iteration.
function jsonParser(txt_file, cb) {
var perc = 0;
lines = txt_file.split('\n');
insert_communication = 'INSERT INTO communication (account_id, contact_id, type_com_id, coin_id, com_datetime, ' +
'destination_key, destination_st, duration, cost, description, discount) VALUES ';
insert_internet = 'INSERT INTO internet_data (account_id, contact_id, type_com_id, coin_id, com_datetime, ' +
'duration, cost, description, discount) VALUES ';
for (let i = 2; i < lines.length; i++) {
readTextFile2(lines[i], function (cb) {
if (cb[0] == "communication")
insert_communication += cb[1] + ',';
if (cb[0] == "internet")
insert_internet += cb[1] + ',';
});
}
cb(insert_communication);
}
How I am supposed to achieve this?
Thanks
for (let i = 2; i < lines.length; i++) {
const previous = lines[i - 1 ];
const hasPrevious = !!previous;
if(hasPrevious) {
//ok you have a previously value, now you could use it
}
readTextFile2(lines[i], function (cb) {
if (cb[0] == "communication")
insert_communication += cb[1] + ',';
if (cb[0] == "internet")
insert_internet += cb[1] + ',';
});
}

Web Service in Nodejs

I want to print out all number from 1 to 1,000,000 with 1,000 numbers per line. and return the response without having a long delay.
Thanks in advance for any type of help!!!
Well, first you can create a function to print numbers from 1 to 1000.
function getThousand(index) {
var result = '';
for (var i = index; i < index + 1000; i++) {
result += i + ' ';
}
return result;
}
Then, you need a function to call this for 1 to 1000000.
function getAll() {
var result = '';
for (var i = 0; i < 1000; i++) {
result = getThousand((i * 1000) + 1) + " \n";
fs.appendFileSync('foo.txt', result);
}
}
Then call it all:
getAll();
This will save your lines into a file. At the end of getAll() you can print what you need.

Iterating with Q

I have this collection in MongoDB (I'm omitting _ids for brevity):
test> db.entries.find();
{
"val": 1
}
{
"val": 2
}
{
"val": 3
}
{
"val": 4
}
{
"val": 5
}
I need to perform some processing on each document which I cannot do with db.update(). So, in a nutshell, what I need to do is retrieving one document at a time, process it in Node and save it back to Mongo.
I'm using the Monk library, and Q for promises. Here's what I've done — I didn't include the processing/save bit for brevity:
var q = require('q');
var db = require('monk')('localhost/test');
var entries = db.get('entries');
var i = 1;
var total;
var f = function (entry) {
console.log('i = ' + i);
console.log(entry.val);
i++;
if (i <= total) {
var promise = entries.findOne({ val: i });
loop.then(function (p) {
return f(p);
});
return promise;
}
};
var loop = q.fcall(function () {
return entries.count({});
}).then(function (r) {
total = r;
return entries.findOne({ val: i });
}).then(f);
I would expect this code to print out:
i = 1
1
i = 2
2
i = 3
3
i = 4
4
i = 5
5
but it actually prints out:
i = 1
1
i = 2
2
i = 3
2
i = 4
2
i = 5
2
What am I doing wrong?
In your code, loop is one and only one promise. It is executed only once. It is not a function.
Inside f, loop.then(f) just trigger f with the result of the promise (it has already been executed so it is not executed again).
You actually want to create several promises.
What you are looking for is something that should looks like:
var q = require('q');
var db = require('monk')('localhost/test');
var entries = db.get('entries');
var i = 1;
var total;
var f = function (entry) {
console.log('i = ' + i);
console.log(entry.val);
i++;
if (i <= total) {
// I am not sure why you put entries.findOne here (looks like a mistake,
// its returned value isn't used) but if you really need it to be done
// before loop, then you must pipe it before loop
return entries.findOne({ val: i }).then(loop);
// do not pipe f again here, it is already appended at the end of loop
}
};
function loop(){
return q.fcall(function () {
return entries.count({});
}).then(function (r) {
total = r;
return entries.findOne({ val: i });
}).then(f);
}
loop();
If you are interested, here is a very nice article about promises.

Error when using fiber dependent wait.for npm module in Node.js

Im using the wait.for library to get rid of some asynchronous calls. Im doing this because I don't really care much about having my server run efficiently or anything, I'm using node.js for some data processing using the Natural NLP npm module.
Anyways, Im recursively building a tree and requesting an wikipedia article for each node. That wikipedia request is asynchronous and I am trying to make it synchronous via a call to wait.for(...). The request works fine if the recursive call ends after just one activation, but throws an error if the recursion carries on any further.
Visually, I mean
[ROOT]
/ | \
[] [] [] <--- This level is fine ... No Errors
|
[] <--- Error Here, on level 2
The error is:
/Users/me/Documents/Node Projects/Personal Website/node_modules/wait.for/waitfor.js:37
fiber.run(); //resume fiber after "yield"
^
TypeError: Cannot call method 'toString' of undefined
at TfIdf.tfidf (/Users/me/Documents/Node Projects/Personal Website/node_modules/natural/lib/natural/tfidf/tfidf.js:132:42)
at isIrrelevant (/Users/me/Documents/Node Projects/Personal Website/finalproject/wikitree.js:126:47)
Here is the code for the recursive call:
var WikiTree = function(node) {
this.name = node.name;
this.context = node.context;
this.text = node.article;
if (node.children) {
this.children = node.children;
}
function createArticleSummary(article) {
// TODO: Must Implement
return article;
}
function isIrrelevant(article, title, linkText) {
// Tweak Here
var articleTfIdf = new TfIdf();
articleTfIdf.addDocument(article);
var titleRelevanceToArticle = articleTfIdf.tfidf(title,0); // ERROR HERE: This is wikitree.js:126
var textRelevanceToArticle = articleTfIdf.tfidf(linkText,0);
var titleRelevanceToZen = zenTfidf.tfidf(title,0);
var textRelevanceToZen = zenTfidf.tfidf(linkText,0);
return ( titleRelevanceToZen / titleRelevanceToArticle >= 1 ) || ( textRelevanceToZen / textRelevanceToArticle >= 1 );
}
WikiTree.prototype.searchChild = function(child) {
console.log("Searching for Link Title \"" + child.title + "\" And Link Text \"" + child.linkText + "\"");
var childQuery = {
query : child.title,
format : "html",
summaryOnly : false
};
var childArticle = wait.for(wikipedia.searchArticle, childQuery);
console.log("Got Wikipedia Response for " + child.title);
var childWikiLinks = extractLinkedWikis(childArticle);
console.log("Found " + childWikiLinks.length + " Links for " + child.title);
for (var i = childWikiLinks.length - 1; i >= 0; i--) {
if ( LinkTrie.contains(childWikiLinks[i].title) || LinkTrie.contains(childWikiLinks[i].linkText) ) {
childWikiLinks.splice(i,1);
} else if ( isIrrelevant( childWikiLinks[i] ) ) {
childWikiLinks.splice(i,1);
} else {
LinkTrie.addStrings([ childWikiLinks[i].title, childWikiLinks[i].text ]);
}
};
console.log("After pruning, " + childWikiLinks.length + " Links left for " + child.title);
var newChildWT = new WikiTree({
name : child.title,
context : child.linkText,
article : childArticle,
children : createArticleSummary(childWikiLinks)
});
return newChildWT;
};
WikiTree.prototype.descend = function() {
// Descend to the next level ... Meaning find the children of the current children (If there are any)
if (!this.children) {
console.log("Leaf at " + this.name);
return;
}
for (var i = this.children.length - 1; i >= 0; i--) {
if ( this.children[i].title.match(/Category:/) ) {
console.log("Found 'Category:' at " + i + " in " + this.children[i].title);
this.children.splice(i,1);
} else
this.children[i] = this.searchChild(this.children[i]);
};
console.log("Done With Descend For " + this.name);
};
this.descend();
};
var result = ...; // A String
var zenLinks = ...; // An Array of objects
// Actually make the first recursive call here
var zenTree = new WikiTree({
name : 'Zen & The Art of Motorcycle Maintenance',
article : result,
children : zenLinks
});
Anyways, any help will be much appreciated. There wasn't much help from googling the issue so anyone that has had experience with either wait.for or node fibers could greatly help me out.
I think you might do this:
var childArticle = wait.forMethod(wikipedia, 'searchArticle', childQuery);

Resources