what the function does is deviding the numertor with the denomirator and updates the app's text view accordingly after every second, the problem is that it doesn't update the screen its just simply shows the original number of the numerator that is 60.
what do I change in order to make this work?
fun division() {
val numerator = 60
var denominator = 4
repeat(4) {
Thread.sleep(1_000)
findViewById<TextView>(R.id.division_textview).setText("${numerator / denominator}")
denominator--
}
}
Because you are setting (basically overwritting) the text everytime it loops through, you will only see the value of the last increment which would be 60/1 and that's why you are only seeing 60 value. Try like this:
fun division() {
val numerator = 60
var denominator = 4
repeat(4) {
Thread.sleep(1_000)
findViewById<TextView>(R.id.division_textview).append("${numerator / denominator}\n")
denominator--
}
}
setText() was overwriting the text with the new one but append() is gonna keep the previous text.
This is that dang Codelab again isn't it? I knew it looked familiar... I already answered a similar question here - but basically, when you run division on the main thread (which you must be since you're messing with UI components), you're freezing the app because you're blocking the thread with Thread.sleep
The display can't actually update until your code has finished running, i.e. after you exit the division function, because it's all running on the same thread, and the display update pass comes later. So this is what's actually happening:
freeze the app for 1 second
set the text as the result of 60 / 4 - it won't actually redraw until later, after your code has finished, so there's no visual change
freeze the app for 1 second
set the text as the result of 60 / 3 - again you won't see anything happen yet, but now it's going to show 60 / 3 instead of 60 / 4, because you just updated the state of that TextView
etc.
The last text you set is the result of 60 / 1, and then your code finishes, so the system can finally get around to updating the display. So the first thing you see after the app stops freezing is 60 - it's not just the numerator, it's the last calculation from your loop.
If you want something to update while the app is running, there are lots of solutions like coroutines, CountdownTimers, posting runnables that execute at a specific time, etc. The answer I linked shows how to create a separate thread to run basically the same code on, so you can block it as much as you like without affecting the running of the app. The one thing you don't do is block the main thread like that Codelab example does. It's a bad Codelab
You can use delay and then call from a coroutine:
private suspend fun division() {
val numerator = 60
var denominator = 4
repeat(4) {
delay(1000)
findViewById<TextView>(R.id.division_textview).text = "${numerator / denominator}"
denominator--
}
}
Then from your Activity/Fragment:
lifecycleScope.launch {
division()
}
Related
I have written an application to show GPU/CPU/RAM loads in electron.js! It was all ok until I started implementing gauges to graphically display temperatures. This section is shown below, I have also included a screenshot of task manager showing hundreds of command and powershell windows!
function getGPUtempHome(){
let stinker = si.graphics();
stinker.then(function(tempGPU){
GPUtempHome = tempGPU.controllers[0].temperatureGpu;
console.log(GPUtempHome);
document.getElementById("GPUtemps").innerHTML = GPUtempHome+'°';
document.documentElement.style.setProperty('--GPUTEMP', GPUtempHome*1.8 + 'deg');
});
setInterval( function() {getGPUtempHome();}, 5000 );
}
function getRAMhome(){
let stinker3 = si.mem();
stinker3.then(function(RAM){
RAMUSAGE = Math.round(((RAM.used/RAM.total)*100));
console.log(RAMUSAGE);
document.getElementById("RAMloads").innerHTML = RAMUSAGE+'%';
document.documentElement.style.setProperty('--RAMLOAD', RAMUSAGE*1.8 + 'deg');
});
setInterval( function() {getRAMhome();}, 5000 );
}
The code above fetches RAM load or GPU temperature every 5 seconds, it then updates a HTML/CSS gauge.
If writing a program like this is unviable in electron.js are there any other better options out there, I am open to playing around with new languages/libraries.
Remember, setInterval() sets a recurring timer, so your code is basically saying "Every five seconds create another timer for every five seconds." After ten seconds you will have 2 timers. After 20 you now have 8. After 60 seconds you have 4,096 of them. Every 5 seconds the number of timers doubles.
That sounds like a problem, doesn't it?
What you want to do is set one timer, once, per function:
function getRAMhome(){
let stinker3 = si.mem();
stinker3.then(function(RAM){
RAMUSAGE = Math.round(((RAM.used/RAM.total)*100));
console.log(RAMUSAGE);
document.getElementById("RAMloads").innerHTML = RAMUSAGE+'%';
document.documentElement.style.setProperty('--RAMLOAD', RAMUSAGE*1.8 + 'deg');
});
}
// Set timer outside of the function
setInterval(getRAMhome, 5000);
It's a common problem to confuse setInterval() which sets a repeating timer with setTimeout() which sets a one-shot timer that needs to be re-engaged. You're treating the former like the latter, with disastrous consequences.
The worldserver source code is full of Update methods that take as input a diff integer value:
How does it work?
And how is this linked to the "Update time diff" from the .server info command?
To fully understand how this works, it's necessary to have a look at the main World run process.
WorldRunnable::run()
File: src/server/worldserver/WorldThread/WorldRunnable.cpp
The method void WorldRunnable::run() is the "Main heartbeat for the World". This method runs the whole world process.
Inside it, there is a while loop that runs as long as the world is supposed to keep running:
void WorldRunnable::run()
{
uint32 realCurrTime = 0;
uint32 realPrevTime = getMSTime();
///- While we have not World::m_stopEvent, update the world
while (!World::IsStopped())
{
++World::m_worldLoopCounter;
realCurrTime = getMSTime();
uint32 diff = getMSTimeDiff(realPrevTime, realCurrTime);
sWorld->Update( diff );
realPrevTime = realCurrTime;
uint32 executionTimeDiff = getMSTimeDiff(realCurrTime, getMSTime());
devDiffTracker.Update(executionTimeDiff);
avgDiffTracker.Update(executionTimeDiff > WORLD_SLEEP_CONST ? executionTimeDiff : WORLD_SLEEP_CONST);
// ... some more code here
}
// at this point the world process is terminating
// ... some more code here
What this loop really does is basically:
1) calculate the elapsed time (in milliseconds) since the previous iteration, this will be the diff
2) call the sWorld->Update( diff ); function, that contains all the world process logic (see below) and passing the diff to it
3) calculate how much time it took to run sWorld->Update( diff ); and update the devDiffTracker and its average avgDiffTracker. These values will be displayed by the .server info command.
World::Update(uint32 diff)
File: src/server/game/World/World.cpp
The World::Update(uint32 diff) function gets constantly called by the main worldserver loop process and every time it takes in input the amount diff of elapsed time since the last call.
This function is responsible for constantly updating the world, this is where all the magic happens.
Timers
There are a set of timers (defined in World.h that are being updated within the World::Update function:
/// Timers for different object refresh rates
enum WorldTimers
{
WUPDATE_AUCTIONS,
WUPDATE_WEATHERS,
WUPDATE_UPTIME,
WUPDATE_CORPSES,
WUPDATE_EVENTS,
WUPDATE_CLEANDB,
WUPDATE_AUTOBROADCAST,
WUPDATE_MAILBOXQUEUE,
WUPDATE_PINGDB,
WUPDATE_5_SECS,
WUPDATE_COUNT
};
For example, WUPDATE_AUTOBROADCAST is responsible for the period global messages defined in the acore_auth.autobroadcast table.
Tasks
The World::Update function also takes care of many timed-tasks, for example:
/// Handle daily quests reset time
if (m_gameTime > m_NextDailyQuestReset)
ResetDailyQuests();
Calling Update(diff) functions of Managers
In AzerothCore there are singleton classes called Managers (Mgr) that handle specific parts of the game. For example BattlegroundMgr handles the Battlegrounds (BGs).
Those classes have their own Update(uint32 diff) functions and they are called by World::Update that passes down the diff to them, for example:
sBattlegroundMgr->Update(diff);
sOutdoorPvPMgr->Update(diff);
sBattlefieldMgr->Update(diff);
/// ... there are more!
OnWorldUpdate hook
Last but not least, it calls sScriptMgr->OnWorldUpdate(diff);.
This is part of the AzerothCore Module System, and defines a hook that can be used by third-part modules to attach custom logic to the World::Update function.
I need to create a new field sid on each document in a collection of about 500K documents. Each sid is unique and based on that record's existing roundedDate and stream fields.
I'm doing so with the following code:
var cursor = db.getCollection('snapshots').find();
var iterated = 0;
var updated = 0;
while (cursor.hasNext()) {
var doc = cursor.next();
if (doc.stream && doc.roundedDate && !doc.sid) {
db.getCollection('snapshots').update({ "_id": doc['_id'] }, {
$set: {
sid: doc.stream.valueOf() + '-' + doc.roundedDate,
}
});
updated++;
}
iterated++;
};
print('total ' + cursor.count() + ' iterated through ' + iterated + ' updated ' + updated);
It works well at first, but after a few hours and about 100K records it errors out with:
Error: getMore command failed: {
"ok" : 0,
"errmsg": "Cursor not found, cursor id: ###",
"code": 43,
}: ...
EDIT - Query performance:
As #NeilLunn pointed out in his comments, you should not be filtering the documents manually, but use .find(...) for that instead:
db.snapshots.find({
roundedDate: { $exists: true },
stream: { $exists: true },
sid: { $exists: false }
})
Also, using .bulkWrite(), available as from MongoDB 3.2, will be far way more performant than doing individual updates.
It is possible that, with that, you are able to execute your query within the 10 minutes lifetime of the cursor. If it still takes more than that, you cursor will expire and you will have the same problem anyway, which is explained below:
What is going on here:
Error: getMore command failed may be due to a cursor timeout, which is related with two cursor attributes:
Timeout limit, which is 10 minutes by default. From the docs:
By default, the server will automatically close the cursor after 10 minutes of inactivity, or if client has exhausted the cursor.
Batch size, which is 101 documents or 16 MB for the first batch, and 16 MB, regardless of the number of documents, for subsequent batches (as of MongoDB 3.4). From the docs:
find() and aggregate() operations have an initial batch size of 101 documents by default. Subsequent getMore operations issued against the resulting cursor have no default batch size, so they are limited only by the 16 megabyte message size.
Probably you are consuming those initial 101 documents and then getting a 16 MB batch, which is the maximum, with a lot more documents. As it is taking more than 10 minutes to process them, the cursor on the server times out and, by the time you are done processing the documents in the second batch and request a new one, the cursor is already closed:
As you iterate through the cursor and reach the end of the returned batch, if there are more results, cursor.next() will perform a getMore operation to retrieve the next batch.
Possible solutions:
I see 5 possible ways to solve this, 3 good ones, with their pros and cons, and 2 bad one:
👍 Reducing the batch size to keep the cursor alive.
👍 Remove the timeout from the cursor.
👍 Retry when the cursor expires.
👎 Query the results in batches manually.
👎 Get all the documents before the cursor expires.
Note they are not numbered following any specific criteria. Read through them and decide which one works best for your particular case.
1. 👍 Reducing the batch size to keep the cursor alive
One way to solve that is use cursor.bacthSize to set the batch size on the cursor returned by your find query to match those that you can process within those 10 minutes:
const cursor = db.collection.find()
.batchSize(NUMBER_OF_DOCUMENTS_IN_BATCH);
However, keep in mind that setting a very conservative (small) batch size will probably work, but will also be slower, as now you need to access the server more times.
On the other hand, setting it to a value too close to the number of documents you can process in 10 minutes means that it is possible that if some iterations take a bit longer to process for any reason (other processes may be consuming more resources), the cursor will expire anyway and you will get the same error again.
2. 👍 Remove the timeout from the cursor
Another option is to use cursor.noCursorTimeout to prevent the cursor from timing out:
const cursor = db.collection.find().noCursorTimeout();
This is considered a bad practice as you would need to close the cursor manually or exhaust all its results so that it is automatically closed:
After setting the noCursorTimeout option, you must either close the cursor manually with cursor.close() or by exhausting the cursor’s results.
As you want to process all the documents in the cursor, you wouldn't need to close it manually, but it is still possible that something else goes wrong in your code and an error is thrown before you are done, thus leaving the cursor opened.
If you still want to use this approach, use a try-catch to make sure you close the cursor if anything goes wrong before you consume all its documents.
Note I don't consider this a bad solution (therefore the 👍), as even thought it is considered a bad practice...:
It is a feature supported by the driver. If it was so bad, as there are alternatives ways to get around timeout issues, as explained in the other solutions, this won't be supported.
There are ways to use it safely, it's just a matter of being extra cautious with it.
I assume you are not running this kind of queries regularly, so the chances that you start leaving open cursors everywhere is low. If this is not the case, and you really need to deal with these situations all the time, then it does make sense not to use noCursorTimeout.
3. 👍 Retry when the cursor expires
Basically, you put your code in a try-catch and when you get the error, you get a new cursor skipping the documents that you have already processed:
let processed = 0;
let updated = 0;
while(true) {
const cursor = db.snapshots.find().sort({ _id: 1 }).skip(processed);
try {
while (cursor.hasNext()) {
const doc = cursor.next();
++processed;
if (doc.stream && doc.roundedDate && !doc.sid) {
db.snapshots.update({
_id: doc._id
}, { $set: {
sid: `${ doc.stream.valueOf() }-${ doc.roundedDate }`
}});
++updated;
}
}
break; // Done processing all, exit outer loop
} catch (err) {
if (err.code !== 43) {
// Something else than a timeout went wrong. Abort loop.
throw err;
}
}
}
Note you need to sort the results for this solution to work.
With this approach, you are minimizing the number of requests to the server by using the maximum possible batch size of 16 MB, without having to guess how many documents you will be able to process in 10 minutes beforehand. Therefore, it is also more robust than the previous approach.
4. 👎 Query the results in batches manually
Basically, you use skip(), limit() and sort() to do multiple queries with a number of documents you think you can process in 10 minutes.
I consider this a bad solution because the driver already has the option to set the batch size, so there's no reason to do this manually, just use solution 1 and don't reinvent the wheel.
Also, it is worth mentioning that it has the same drawbacks than solution 1,
5. 👎 Get all the documents before the cursor expires
Probably your code is taking some time to execute due to results processing, so you could retrieve all the documents first and then process them:
const results = new Array(db.snapshots.find());
This will retrieve all the batches one after another and close the cursor. Then, you can loop through all the documents inside results and do what you need to do.
However, if you are having timeout issues, chances are that your result set is quite large, thus pulling everything in memory may not be the most advisable thing to do.
Note about snapshot mode and duplicate documents
It is possible that some documents are returned multiple times if intervening write operations move them due to a growth in document size. To solve this, use cursor.snapshot(). From the docs:
Append the snapshot() method to a cursor to toggle the “snapshot” mode. This ensures that the query will not return a document multiple times, even if intervening write operations result in a move of the document due to the growth in document size.
However, keep in mind its limitations:
It doesn't work with sharded collections.
It doesn't work with sort() or hint(), so it will not work with solutions 3 and 4.
It doesn't guarantee isolation from insertion or deletions.
Note with solution 5 the time window to have a move of documents that may cause duplicate documents retrieval is narrower than with the other solutions, so you may not need snapshot().
In your particular case, as the collection is called snapshot, probably it is not likely to change, so you probably don't need snapshot(). Moreover, you are doing updates on documents based on their data and, once the update is done, that same document will not be updated again even though it is retrieved multiple times, as the if condition will skip it.
Note about open cursors
To see a count of open cursors use db.serverStatus().metrics.cursor.
It's a bug in mongodb server session management. Fix currently in progress, should be fixed in 4.0+
SERVER-34810: Session cache refresh can erroneously kill cursors that are still in use
(reproduced in MongoDB 3.6.5)
adding collection.find().batchSize(20) helped me with about a tiny reduced performance.
I also ran into this problem, but for me it was caused by a bug in the MongDB driver.
It happened in the version 3.0.x of the npm package mongodb which is e.g. used in Meteor 1.7.0.x, where I also recorded this issue. It's further described in this comment and the thread contains a sample project which confirms the bug: https://github.com/meteor/meteor/issues/9944#issuecomment-420542042
Updating the npm package to 3.1.x fixed it for me, because I already had taken into account the good advises, given here by #Danziger.
When using Java v3 driver, noCursorTimeout should be set in the FindOptions.
DBCollectionFindOptions options =
new DBCollectionFindOptions()
.maxTime(90, TimeUnit.MINUTES)
.noCursorTimeout(true)
.batchSize(batchSize)
.projection(projectionQuery);
cursor = collection.find(filterQuery, options);
in my case, It was a Load balancing issue, had the same issue running with Node.js service and Mongos as a pod on Kubernetes.
The client was using mongos service with default load balancing.
changing the kubernetes service to use sessionAffinity: ClientIP (stickiness) resolved the issue for me.
noCursorTimeout will NOT work
now is 2021 year, for
cursor id xxx not found, full error: {'ok': 0.0, 'errmsg': 'cursor id xxx not found', 'code': 43, 'codeName': 'CursorNotFound'}
official says
Consider an application that issues a db.collection.find() with cursor.noCursorTimeout(). The server returns a cursor along with a batch of documents defined by the cursor.batchSize() of the find(). The session refreshes each time the application requests a new batch of documents from the server. However, if the application takes longer than 30 minutes to process the current batch of documents, the session is marked as expired and closed. When the server closes the session, it also kills the cursor despite the cursor being configured with noCursorTimeout(). When the application requests the next batch of documents, the server returns an error.
that means: Even if you have set:
noCursorTimeout=True
smaller batchSize
will still cursor id not found after default 30 minutes
How to fix/avoid cursor id not found?
make sure two point
(explicitly) create new session, get db and collection from this session
refresh session periodically
code:
(official) js
var session = db.getMongo().startSession()
var sessionId = session.getSessionId().id
var cursor = session.getDatabase("examples").getCollection("data").find().noCursorTimeout()
var refreshTimestamp = new Date() // take note of time at operation start
while (cursor.hasNext()) {
// Check if more than 5 minutes have passed since the last refresh
if ( (new Date()-refreshTimestamp)/1000 > 300 ) {
print("refreshing session")
db.adminCommand({"refreshSessions" : [sessionId]})
refreshTimestamp = new Date()
}
// process cursor normally
}
(mine) python
import logging
from datetime import datetime
import pymongo
mongoClient = pymongo.MongoClient('mongodb://127.0.0.1:27017/your_db_name')
# every 10 minutes to update session once
# Note: should less than 30 minutes = Mongo session defaul timeout time
# https://docs.mongodb.com/v5.0/reference/method/cursor.noCursorTimeout/
# RefreshSessionPerSeconds = 10 * 60
RefreshSessionPerSeconds = 8 * 60
def mergeHistorResultToNewCollection():
mongoSession = mongoClient.start_session() # <pymongo.client_session.ClientSession object at 0x1081c5c70>
mongoSessionId = mongoSession.session_id # {'id': Binary(b'\xbf\xd8\xd...1\xbb', 4)}
mongoDb = mongoSession.client["your_db_name"] # Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True), 'your_db_name')
mongoCollectionOld = mongoDb["collecion_old"]
mongoCollectionNew = mongoDb['collecion_new']
# historyAllResultCursor = mongoCollectionOld.find(session=mongoSession)
historyAllResultCursor = mongoCollectionOld.find(no_cursor_timeout=True, session=mongoSession)
lastUpdateTime = datetime.now() # datetime.datetime(2021, 8, 30, 10, 57, 14, 579328)
for curIdx, oldHistoryResult in enumerate(historyAllResultCursor):
curTime = datetime.now() # datetime.datetime(2021, 8, 30, 10, 57, 25, 110374)
elapsedTime = curTime - lastUpdateTime # datetime.timedelta(seconds=10, microseconds=531046)
elapsedTimeSeconds = elapsedTime.total_seconds() # 2.65892
isShouldUpdateSession = elapsedTimeSeconds > RefreshSessionPerSeconds
# if (curIdx % RefreshSessionPerNum) == 0:
if isShouldUpdateSession:
lastUpdateTime = curTime
cmdResp = mongoDb.command("refreshSessions", [mongoSessionId], session=mongoSession)
logging.info("Called refreshSessions command, resp=%s", cmdResp)
# do what you want
existedNewResult = mongoCollectionNew.find_one({"shortLink": "http://xxx"}, session=mongoSession)
# mongoSession.close()
mongoSession.end_session()
Refer doc
MongoDB
ClientSession
refreshSessions
pymongo
find
command
I run my bot in a public channel with hundreds of users. Yesterday a person came in and just abused it.
I would like to let anyone use the bot, but if they spam commands consecutively and if they aren't a bot "owner" like me when I debug then I would like to add them to an ignored list which expires in an hour or so.
One way I'm thinking would be to save all commands by all users, in a dictionary such as:
({
'meder#freenode': [{command:'.weather 20851', timestamp: 209323023 }],
'jack#efnet': [{command:'.seen john' }]
})
I would setup a cron job to flush this out every 24 hours, but I would basically determine if a person has made X number of commands in a duration of say, 15 seconds and add them to an ignore list.
Actually, as I'm writing this answer I thought of a better idea.. maybe instead of storing each users commands, just store the the bot's commands in a list and keep on pushing until it reaches a limit of say, 15.
lastCommands = [], limit = 5;
function handleCommand( timeObj, action ) {
if ( lastCommands.length < limit ) {
action();
} else {
// enumerate through lastCommands and compare the timestamps of all 5 commands
// if the user is the same for all 5 commands, and...
// if the timestamps are all within the vicinity of 20 seconds
// add the user to the ignoreList
}
}
watch_for('command', function() {
handleCommand({timestamp: 2093293032, user: user}, function(){ message.say('hello there!') })
});
I would appreciate any advice on the matter.
Here's a simple algorithm:
Every time a user sends a command to the bot, increment a number that's tied to that user. If this is a new user, create the number for them and set it to 1.
When a user's number is incremented to a certain value (say 15), set it to 100.
Every <period> seconds, run through the list and decrement all the numbers by 1. Zero means the user's number can be freed.
Before executing a command and after incrementing the user's counter, check to see if it exceeds your magic max value (15 above). If it does, exit before executing the command.
This lets you rate limit actions and forgive excesses after a while. Divide your desired ban length by the decrement period to find the number to set when a user exceeds your threshold (100 above). You can also add to the number if a particular user keeps sending commands after they've been banned.
Well Nathon has already offered a solution, but it's possible to reduce the code that's needed.
var user = {};
user.lastCommandTime = new Date().getTime(); // time the user send his last command
user.commandCount = 0; // command limit counter
user.maxCommandsPerSecond = 1; // commands allowed per second
function handleCommand(obj, action) {
var user = obj.user, now = new Date().getTime();
var timeDifference = now - user.lastCommandTime;
user.commandCount = Math.max(user.commandCount - (timeDifference / 1000 * user.maxCommandsPerSecond), 0) + 1;
user.lastCommandTime = now;
if (user.commandCount <= user.maxCommandsPerSecond) {
console.log('command!');
} else {
console.log('flooding');
}
}
var obj = {user: user};
var e = 0;
function foo() {
handleCommand(obj, 'foo');
e += 250;
setTimeout(foo, 400 + e);
}
foo();
In this implementation, there's no need for a list or some global callback every X seconds, instead we just reduce the commandCount every time there's a new message, based on time difference to the last command, it's also possible to allow different command rates for specific users.
All we need are 3 new properties on the user object :)
Redis
I would use the insanely fast advanced key-value store redis to write something like this, because:
It is insanely fast.
There is no need for cronjob because you can set expire on keys.
It has atomic operations to increment key
You could use redis-cli for prototyping.
I myself really like node_redis as redis client. It is a really fast redis client, which can easily be installed using npm.
Algorithme
I think my algorithme would look something like this:
For each user create a unique key which counts the commands consecutively executed. Also set expire to the time when you don't flag a user as spammer anymore. Let's assume the spammer has nickname x and the expire 15.
Inside redis-cli
incr x
expire x 15
When you do a get x after 15 seconds then the key does not exist anymore.
If value of key is bigger then threshold then flag user as spammer.
get x
These answers seem to be going the wrong way about this.
IRC Servers will disconnect your client regardless of whether you're "debugging" or not if the client or bot is flooding a channel or the server in general.
Make a blanket flood control, using the method #nmichaels has detailed, but on the bot's network connection to the server itself.
I am trying to execute parallel functions on a list of objects using the new C# 4.0 Parallel.ForEach function. This is a very long maintenance process. I would like to make it execute in the order of the list so that I can stop and continue execution at the previous point. How do I do this?
Here is an example. I have a list of objects: a1 to a100. This is the current order:
a1, a51, a2, a52, a3, a53...
I want this order:
a1, a2, a3, a4...
I am OK with some objects being run out of order, but as long as I can find a point in the list where I can say that all objects before this point were run. I read the parallel programming csharp whitepaper and didn't see anything about it. There isn't a setting for this in the ParallelOptions class.
Do something like this:
int current = 0;
object lockCurrent = new object();
Parallel.For(0, list.Count,
new ParallelOptions { MaxDegreeOfParallelism = MaxThreads },
(ii, loopState) => {
// So the way Parallel.For works is that it chunks the task list up with each thread getting a chunk to work on...
// e.g. [1-1,000], [1,001- 2,000], [2,001-3,000] etc...
// We have prioritized our job queue such that more important tasks come first. So we don't want the task list to be
// broken up, we want the task list to be run in roughly the same order we started with. So we ignore tha past in
// loop variable and just increment our own counter.
int thisCurrent = 0;
lock (lockCurrent) {
thisCurrent = current;
current++;
}
dothework(list[thisCurrent]);
});
You can see how when you break out of the parallel for loop you will know the last list item to be executed, assuming you let all threads finish prior to breaking. I'm not a big fan of PLINQ or LINQ. I honestly don't see how writing LINQ/PLINQ leads to maintainable source code or readability.... Parallel.For is a much better solution.
If you use Parallel.Break to terminate the loop then you are guarenteed that all indices below the returned value will have been executed. This is about as close as you can get. The example here uses For but ForEach has similar overloads.
int n = ...
var result = new double[n];
var loopResult = Parallel.For(0, n, (i, loopState) =>
{
if (/* break condition is true */)
{
loopState.Break();
return;
}
result[i] = DoWork(i);
});
if (!loopResult.IsCompleted &&
loopResult.LowestBreakIteration.HasValue)
{
Console.WriteLine("Loop encountered a break at {0}",
loopResult.LowestBreakIteration.Value);
}
In a ForEach loop, an iteration index is generated internally for each element in each partition. Execution takes place out of order but after break you know that all the iterations lower than LowestBreakIteration will have been completed.
Taken from "Parallel Programming with Microsoft .NET" http://parallelpatterns.codeplex.com/
Available on MSDN. See http://msdn.microsoft.com/en-us/library/ff963552.aspx. The section "Breaking out of loops early" covers this scenario.
See also: http://msdn.microsoft.com/en-us/library/dd460721.aspx
For anyone else who comes across this question - if you're looping over an array or list (rather than an IEnumberable ), you can use the overload of Parallel.Foreach that gives the element index to maintain original order too.
string[] MyArray; // array of stuff to do parallel tasks on
string[] ProcessedArray = new string[MyArray.Length];
Parallel.ForEach(MyArray, (ArrayItem,loopstate,ArrayElementIndex) =>
{
string ProcessedArrayItem = TaskToDo(ArrayItem);
ProcessedArray[ArrayElementIndex] = ProcessedArrayItem;
});
As an alternate suggestion, you could record which object have been run and then filter the list when you resume exection to exclude the objects which have already run.
If this needs to be persistent across application restarts, you can store the ID's of the already executed objects (I assume here the objects have some unique identifier).
For anybody looking for a simple solution, I have posted 2 extension methods (one using PLINQ and one using Parallel.ForEach) as part of an answer to the following question:
Ordered PLINQ ForAll
Not sure if question was altered as my comment seems wrong.
Here improved, basically remind that parallel jobs run in out of your control order.
ea printing 10 numbers might result in 1,4,6,7,2,3,9,0.
If you like to stop your program and continue later.
Problems alike this usually endup in batching workloads.
And have some logging of what was done.
Say if you had to check 10.000 numbers for prime or so.
You could loop in batches of size 100, and have a prime log1, log2, log3
log1= 0..99
log2=100..199
Be sure to set some marker to know if a batch job was finished.
Its a general aprouch since the question isnt that exact either.