Does the following require some kind of synchronization?
Control
Worker 1
Worker 2
...
Worker n
Test if all a_n is true. If so, dispatch a new job. Otherwise, try again.
when done: set a_1 to true
when done: set a_2 to true
...
when done: set a_n to true
The test performed by the control thread is not atomic. However, a_n can never be set to false. Thus, it is guaranteed that all a_n will be true for some iteration in the control thread. Is synchronization still necessary?
This control flow is used to process jobs in a graph. All tasks is stored in a linked list. When all $a_{kn}$ are true, job $k$ is dispatched and removed from the list. If they are not, try $a_{k+1}$. When the end of the list is reached, start over from the beginning of the list. Continue until the list is empty.
Note: I assume that the graph is a DAG.
Related
Say I have a Process which has two optional View tasks to the finish:
--> Split --> optional_view_1 ---> Join ----> finish
| ^
| |
-----> optional_view_2 -------
Let's say both tasks are assigned, but a human logs in and completes optional_view_1. The Join itself and indeed the End node both end up with status DONE. However, the Process will get stuck and NOT exit cleanly due to optional_view_2.
The questions are:
how to finish the process cleanly (i.e. not cancel it) and without races
from where (e.g. from inside the Join or a Handler after each View to cancel the other one?)
I originally posted this as an issue, along with a possible coded solution. In summary the code:
Overrides the Activation "done" associated with the Join
The override checks if the Join has wait_all=False
And if so, tries to find any other tasks the Join is waiting for
And cancels them
The code for step 3 is the bit I am most worried about. It is a modified version of the Viewflow code handling the wait_all=True case and looks like this in summary:
previous = self.task.previous.exclude(status=STATUS.CANCELED).all()
join_prefixes = set(prev.token.get_common_split_prefix(self.task.token, prev.pk)
for prev in previous)
#
# Don't forget to exclude the previous tasks.
#
join_token_prefix = next(iter(join_prefixes))
active = self.flow_class.task_class._default_manager \
.filter(process=self.process, token__startswith=join_token_prefix) \
.exclude(status__in=[STATUS.DONE, STATUS.CANCELED], pk__in=[prev.pk for prev in previous])
for task in active:
cancel_task(task)
Any comments on whether this is the right approach or not would be welcome.
That BPMN process would not be complete, it's the right behavior. The join outgoing path would be executed twice.
To have an optional branch, you need to implement event-based gateway. That would subscribe to flow task end signal (override ready method) and cancel nonfinished tasks.
https://www.youtube.com/watch?v=pyJM_V8Ji2w
When adding Test Action with Stop/Stop Now action
You have Target options: All Threads/Current Thread
When choosing Current Thread in a multiple threads environment it stops and don't continue further this Sampler
The problem is that when choosing All Threads Some threads execute other sampler after than Test Action
It's very confusing, because I expect All Threads option to be more strict than just Current Thread
In code I saw that in Current Threads it also stop current threads context.getThread().stop(); and in All Threads option it doesn't.
Is it a bug or a feature (adding grace period of stopping)?
For example Test Plan:
Thread Group (5 Threads)
Sampler1
Test Action Target: All Threads, Action Stop/Stop Now
Sampler2
Sampler 2 is execute only when Target: All Threads and not when Target: Current Thread
Note: Also choosing Action Go to next loop iteration (Target field is disabled) prevent Sampler2 to be executed
Stop and Stop Now are different:
Stop is a clean shutdown, meaning the current running samples will complete. So it is ok if you see other samplers even after test action
Stop Now is a hard shutdown, meaning current running samples will be interrupted so again, it is ok if you see those other samplers after Test Action
Current thread will only stop the current thread not all thread, so it is ok that:
When choosing Current Thread in a multiple threads environment it stops and don't continue further this Sampler
All Threads will do action on all threads of test, in code we have:
if (action == STOP_NOW) {
log.info("Stopping all threads now from element {}", getName());
context.getEngine().stopTest();
} else {
log.info("Stopping all threads from element {}", getName());
context.getEngine().askThreadsToStop();
}
Regarding your particular case, here is what is happening:
When you select "Current Thread", JMeter immediately stops the current thread as this action is taken into action after the Test Action
When you select "All Threads", JMeter triggers asynchronously a call to all threads shutdown/stop, that's why Sampler2 is called
You may consider this a bug, but I think use case is different.
Still it is now fixed:
https://bz.apache.org/bugzilla/show_bug.cgi?id=61698
So I have been working with coroutines for a little while and I'm sort of having trouble trying to make what I want. I want a class that I can access objectivley creating objects as tasks or processes. I think showing you code would be irrelevant and its not what I want either. So I'm just going to show you how i want the functionality
local task1 = Tasker.newTask(function()
while true do
print("task 1")
end
end)
local task2 = Tasker.newTask(function()
while true do
print("task 2")
end
end)
task1:start()
task2:start()
This way I can run multiple tasks at once, I want to be able to add new tasks when ever during runtime. Also I would like a way to stop the tasks also for example:
task2:stop()
But I don't want the stop command to entirely delete the task instance, only stop the task itself so I can invoke
task2:start()
Then maybe I could use a command to delete it.
task2:delete()
This would be very helpful and thank you for help if you need more info please ask. Also i posted this on my phone so there may be typos and formatting issues
Lua doesn't natively support operating system threads, i.e. preemptive multitasking.
You can use coroutines to implement your own cooperative "threads", but each thread must relinquish control before another can do anything.
local task1 = Tasker.newTask(function()
while true do
print("task 1")
coroutine.yield()
end
end)
local task2 = Tasker.newTask(function()
while true do
print("task 2")
coroutine.yield()
end
end)
Your Tasker class must take the task function and wrap it in a coroutine, then take care of calling coroutine.resume on them. Operations like stop and start would set flags on the task that tell Tasker whether or not to resume that particular coroutine in the main loop.
You can do this via C. You might be able to use LuaLanes and Linda objects.
I am trying to model a system where there are multiple threads producing data, and a single thread consuming the data. The trick is that I don't want a dedicated thread to consume the data because all of the threads live in a pool. Instead, I want one of the producers to empty the queue when there is work, and yield if another producer is already clearing the queue.
The basic idea is that there is a queue of work, and a lock around the processing. Each producer pushes its payload onto the queue, and then attempts to enter the lock. The attempt is non-blocking and returns either true (the lock was acquired), or false (the lock is held by someone else).
If the lock is acquired, then that thread then processes all of the data in the queue until it is empty (including any new payloads introduced by other producers during processing). Once all of the work has been processed, the thread releases the lock and quits out.
The following is C++ code for the algorithm:
void Process(ITask *task) {
// queue is a thread safe implementation of a regular queue
queue.push(task);
// crit_sec is some handle to a critical section like object
// try_scoped_lock uses RAII to attempt to acquire the lock in the constructor
// if the lock was acquired, it will release the lock in the
// destructor
try_scoped_lock lock(crit_sec);
// See if this thread won the lottery. Prize is doing all of the dishes
if (!lock.Acquired())
return;
// This thread got the lock, so it needs to do the work
ITask *currTask;
while (queue.try_pop(currTask)) {
... execute task ...
}
}
In general this code works fine, and I have never actually witnessed the behavior I am about to describe below, but that implementation makes me feel uneasy. It stands to reason that a race condition is introduced between when the thread exits the while loop and when it releases the critical section.
The whole algorithm relies on the assumption that if the lock is being held, then a thread is servicing the queue.
I am essentially looking for enlightenment on 2 questions:
Am I correct that there is a race condition as described (bonus for other races)
Is there a standard pattern for implementing this mechanism that is performant and doesn't introduce race conditions?
Yes, there is a race condition.
Thread A adds a task, gets the lock, processes itself, then asks for a task from the queue. It is rejected.
Thread B at this point adds a task to the queue. It then attempts to get the lock, and fails, because thread A has the lock. Thread B exits.
Thread A then exits, with the queue non-empty, and nobody processing the task on it.
This will be difficult to find, because that window is relatively narrow. To make it more likely to find, after the while loop introduce a "sleep for 10 seconds". In the calling code, insert a task, wait 5 seconds, then insert a second task. After 10 more seconds, check that both insert tasks are finished, and there is still a task to be processed on the queue.
One way to fix this would be to change try_pop to try_pop_or_unlock, and pass in your lock to it. try_pop_or_unlock then atomically checks for an empty queue, and if so unlocks the lock and returns false.
Another approach is to improve the thread pool. Add a counting semaphore based "consume" task launcher to it.
semaphore_bool bTaskActive;
counting_semaphore counter;
when (counter || !bTaskActive)
if (bTaskActive)
return
bTaskActive = true
--counter
launch_task( process_one_off_queue, when_done( [&]{ bTaskActive=false ) );
When the counting semaphore is active, or when poked by the finished consume task, it launches a consume task if there is no consume task active.
But that is just off the top of my head.
What is the difference between a dead lock and a race around condition in programming terms?
Think of a race condition using the traditional example. Say you and a friend have an ATM cards for the same bank account. Now suppose the account has $100 in it. Consider what happens when you attempt to withdraw $10 and your friend attempts to withdraw $50 at exactly the same time.
Think about what has to happen. The ATM machine must take your input, read what is currently in your account, and then modify the amount. Note, that in programming terms, an assignment statement is a multi-step process.
So, label both of your transactions T1 (you withdraw $10), and T2 (your friend withdraws $50). Now, the numbers below, to the left, represent time steps.
T1 T2
---------------- ------------------------
1. Read Acct ($100)
2. Read Acct ($100)
3. Write New Amt ($90)
4. Write New Amt ($50)
5. End
6. End
After both transactions complete, using this timeline, which is possible if you don't use any sort of locking mechanism, the account has $50 in it. This is $10 more than it should (your transaction is lost forever, but you still have the money).
This is a called race condition. What you want is for the transaction to be serializable, that is in no matter how you interleave the individual instruction executions, the end result will be the exact same as some serial schedule (meaning you run them one after the other with no interleaving) of the same transactions. The solution, again, is to introduce locking; however incorrect locking can lead to dead lock.
Deadlock occurs when there is a conflict of a shared resource. It's sort of like a Catch-22.
T1 T2
------- --------
1. Lock(x)
2. Lock(y)
3. Write x=1
4. Write y=19
5. Lock(y)
6. Write y=x+1
7. Lock(x)
8. Write x=y+2
9. Unlock(x)
10. Unlock(x)
11. Unlock(y)
12. Unlock(y)
You can see that a deadlock occurs at time 7 because T2 tries to acquire a lock on x but T1 already holds the lock on x but it is waiting on a lock for y, which T2 holds.
This bad. You can turn this diagram into a dependency graph and you will see that there is a cycle. The problem here is that x and y are resources that may be modified together.
One way to prevent this sort of deadlock problem with multiple lock objects (resources) is to introduce an ordering. You see, in the previous example, T1 locked x and then y but T2 locked y and then x. If both transactions adhered here to some ordering rule that says "x shall always be locked before y" then this problem will not occur. (You can change the previous example with this rule in mind and see no deadlock occurs).
These are trivial examples and really I've just used the examples you may have already seen if you have taken any kind of undergrad course on this. In reality, solving deadlock problems can be much harder than this because you tend to have more than a couple resources and a couple transactions interacting.
As always, use Wikipedia as a starting point for CS concepts:
http://en.wikipedia.org/wiki/Deadlock
http://en.wikipedia.org/wiki/Race_condition
A deadlock is when two (or more) threads are blocking each other. Usually this has something to do with threads trying to acquire shared resources. For example if threads T1 and T2 need to acquire both resources A and B in order to do their work. If T1 acquires resource A, then T2 acquires resource B, T1 could then be waiting for resource B while T2 was waiting for resource A. In this case, both threads will wait indefinitely for the resource held by the other thread. These threads are said to be deadlocked.
Race conditions occur when two threads interact in a negatve (buggy) way depending on the exact order that their different instructions are executed. If one thread sets a global variable, for example, then a second thread reads and modifies that global variable, and the first thread reads the variable, the first thread may experience a bug because the variable has changed unexpectedly.
Deadlock :
This happens when 2 or more threads are waiting on each other to release the resource for infinite amount of time.
In this the threads are in blocked state and not executing.
Race/Race Condition:
This happens when 2 or more threads run in parallel but end up giving a result which is wrong and not equivalent if all the operations are done in sequential order.
Here all the threads run and execute there operations.
In Coding we need to avoid both race and deadlock condition.
I assume you mean "race conditions" and not "race around conditions" (I've heard that term...)
Basically, a dead lock is a condition where thread A is waiting for resource X while holding a lock on resource Y, and thread B is waiting for resource Y while holding a lock on resource X. The threads block waiting for each other to release their locks.
The solution to this problem is (usually) to ensure that you take locks on all resources in the same order in all threads. For example, if you always lock resource X before resource Y then my example can never result in a deadlock.
A race condition is something where you're relying on a particular sequence of events happening in a certain order, but that can be messed up if another thread is running at the same time. For example, to insert a new node into a linked list, you need to modify the list head, usually something like so:
newNode->next = listHead;
listHead = newNode;
But if two threads do that at the same time, then you might have a situation where they run like so:
Thread A Thread B
newNode1->next = listHead
newNode2->next = listHead
listHead = newNode2
listHead = newNode1
If this were to happen, then Thread B's modification of the list will be lost because Thread A would have overwritten it. It can be even worse, depending on the exact situation, but that's the basics of it.
The solution to this problem is usually to ensure that you include the proper locking mechanisms (for example, take out a lock any time you want to modify the linked list so that only one thread is modifying it at a time).
Withe rest to Programming language if you are not locking shared resources and are accessed by multiple threads then its called as "Race condition", 2nd case if you locked the resources and sequences of access to shared resources are not defined properly then threads may go long waiting for the resources to use then its a case of "deadlock"