As far as I know, programs using await async uses only 1 thread. That means we do not have to worry about any conflicting threads. When the compiler see "await" it will simply do other things on that same thread till what they're awaiting for is done.
I mean the stuff we're awaiting for may run in another thread. However, the program doesn't create another thread. It simply do something else in that same thread.
Hence, we shouldn't worry about conflicts
Yet, today I discover that something is running on at least 2 different thread
Public Sub LogEvents(ByVal whatToLog As String, Optional ByVal canRandom As Boolean = True)
Static logNumber As Integer
Dim timeStamp As String
timeStamp = CStr(Microsoft.VisualBasic.Now)
whatToLog = timeStamp & " " & " " & whatToLog & Microsoft.VisualBasic.vbNewLine
Try
Debug.Print(whatToLog)
System.IO.File.AppendAllText("log.txt", whatToLog, defaultEncoding)
...
Looking at the thread.
So one is worker thread
And another is main thread
Both threads stuck at the same place.
What confuses me is I thought everything should have been running on main thread. That's just how await async works. How can anything run on worker thread?
The task is created like this
For Each account In uniqueAccounts().Values
Dim newtask = account.getMarketInfoAsync().ContinueWith(Sub() account.LogFinishTask("Geting Market Info", starttime))
LogEvents("marketinfo account " + account.Exchange + " is being done by task " + newtask.Id.ToString + " " + newtask.ToString)
tasklist.Add(newtask)
'newtask.ContinueWith(Sub() LogEvents(account.ToString))
Next
This is the screen shot
That is followed by
LogEvents("Really Start Getting Market Detail of All")
Try
Await jsonHelper.whenAllWithTimeout(tasklist.ToArray, 500000)
Catch ex As Exception
Dim b = 1
End Try
That calls
Public Shared Async Function whenAllWithTimeout(taskar As Task(), timeout As Integer) As Task
Dim timeoutTask = Task.Delay(timeout)
Dim maintask = Task.WhenAll(taskar)
Await Task.WhenAny({timeoutTask, maintask})
If maintask.IsCompleted Then
Dim b = 1
For Each tsk In taskar
LogEvents("Not Time Out. Status of task " + tsk.Id.ToString + " is " + tsk.IsCompleted.ToString)
Next
End If
If timeoutTask.IsCompleted Then
Dim b = 1
For Each tsk In taskar
LogEvents("status of task " + tsk.Id.ToString + " is " + tsk.IsCompleted.ToString)
Next
End If
End Function
So I created a bunch of tasks and I use Task.Whenall and Task.Whenany
Is that why they run it on a different thread than the main thread?
How do I make it run on main thread only?
As far as I know, programs using await async uses only 1 thread.
This is incorrect.
When the compiler see "await" it will simply do other things on that same thread till what they're awaiting for is done.
Also incorrect.
I recommend reading my async intro.
await actually causes a return from the method. The thread may or may not be returned to the runtime.
How can anything run on worker thread?
When async methods resume executing after an await, by default they will resume executing on a context captured by that await. If there was no context (common in console applications), then they resume on a thread pool thread.
How do I make it run on main thread only?
Give them a single-threaded context. GUI main threads use a single-threaded context, so you could run this on a GUI main thread. Or if you are writing a console application, you can use AsyncContext from my AsyncEx library.
Related
i am running rails 3 thread to improve the performance of the rake task. sometimes i am getting uninitialized constant when i am trying to access outside class from my current class. as an example when i am trying to access NotificationService class from NotificationApp as shown below then getting error as uninitialized constant NotificationService. This error is not coming every time. sometimes rake task is running fine without the error and sometimes same rake is failing with uninitialized constant. what could be the reason for this and how can i fix this issue?
class NotificationApp < ActiveRecord::Base
def signal_event
NotificationService.notify
end
end
Edit
This is how i am creating threadpool
class ThreadPool
def initialize(size)
#size = size
#jobs = Queue.new
#pool = Array.new(#size) do |i|
Thread.new do
Thread.current[:id] = i
catch(:exit) do
loop do
job, args = #jobs.pop
job.call(*args)
end
end
end
end
end
# add a job to queue
def schedule(*args, &block)
#jobs << [block, args]
end
# run threads and perform jobs from queue
def run!
#size.times do
schedule { throw :exit }
end
#pool.map(&:join)
end
# get all threads created
def threads
#pool
end
def complete!
#pool.each{|t| t.kill}
end
end
and this is how i am using threads
def perform
ReadReplicaHelper.read_from_slave do
pool = ThreadPool.new(CONNECTION_POOL_COUNT - 1)
device_ids.in_groups_of(1000, false) do |devices|
devices.each do |device_id|
device = Device.find_by_id(device_id)
if device
ReadReplicaHelper.read_from_master do
Notification.where(device_id: device_id).each do |notification|
pool.schedule do
pool_method notification, last_status, before_last_status
end
end
end
end
end
end
# Run the Thread Pool
pool.run!
# Kill the threads created in the pool
pool.complete!
end
Make sure your NotificationService class is in a file named app/services/notification_service.rb and that you are loading this services directory.
Also as you are using threads, make sure to join all the threads after calling them
I am using QT 5.11.2 on both linux and windows. I am trying to optimize my UI to prevent it from freezing in large functions. I started using QThread and was able to do exactly what I want on windows. However, I tried to test the same functions on linux RHEL7 but threads never finished.
Here is what I tried:
void MainWidget::Configure_BERT_DSO(bool isOptimization, double lineRate, int Scaling)
{
QThread *bertThread = QThread::create([this, lineRate, Scaling]{ ConfigureBert(lineRate, Scaling);
QThread *dsoThread = QThread::create([this, lineRate]{ ConfigureDSO(lineRate); });
bertThread->setObjectName("My Bert Thread");
dsoThread->setObjectName("My DSO Thread");
bertThread->start();
dsoThread->start();
while(bertThread->isRunning() || dsoThread->isRunning()) // even tried isFinished()
{
qApp->processEvents();
}
bertThread->exit();
dsoThread->exit();
delete bertThread;
delete dsoThread;
}
In windows the while loop exits after a while and both threads execute correctly with no problem.
On linux, to make sure that both functions execute correctly, I added qDebug() at the start and end of each function and they are all reached at the excpected time. But the problem is that isRunning never becomes true again and the same goes for isFinished and my loop gets stuck.
Thread: QThread(0x1d368b0, name = "My Bert Thread") started.
Thread: QThread(0x1d368e0, name = "My DSO Thread") started.
Thread: QThread(0x1d368b0, name = "My Bert Thread") finished.
Thread: QThread(0x1d368e0, name = "My DSO Thread") finished.
Is it something platform dependent or is there something that I might be missing?
EDIT
I also tried to use bertThread->wait() and dsoThread->wait() to check if it exits after my functions finish but it never returns from the first one I called although both functions reach their end successfully
Your help is much appreciated
I'm working with an API that can only access its objects on the main thread, so I need to create a new thread to be used for my GUI and then swap back to the original thread for any lengthy calculations involving the API.
So far I have the following code:
[<EntryPoint; STAThread>]
let main _ =
Debug.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] - Inital thread")
let initCtx = SynchronizationContext.Current
let uiThread = new Thread(fun () ->
let guiCtx = SynchronizationContext.Current
Debug.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] - New UI thread")
async{
do! Async.SwitchToContext initCtx
Debug.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] - Back to initial thread")
// Lengthy API calculation here
do! Async.SwitchToContext guiCtx
Debug.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] - Back to UI thread")
} |> Async.RunSynchronously
)
uiThread.SetApartmentState(ApartmentState.STA)
uiThread.Start()
1
However when I run this I get the output:
[1] - Inital thread
[4] - New UI thread
[5] - Back to initial thread
[5] - Back to UI thread
So it doesn't seem to be switching contexts the way I would expect. How can I switch back to the original thread after creating a new thread this way?
I have tried calling
SynchronizationContext.SetSynchronizationContext(new DispatcherSynchronizationContext(Dispatcher.CurrentDispatcher)) first to ensure that the original thread has a valid SynchronizationContext but that causes the program to exit at the Async.SwitchToContext lines without throwing any exception.
I have also tried using Async.StartImmediate instead of RunSynchronously with the same result.
If I try both of these at the same time then the program just freezes up at the Async.SwitchToContext lines instead of exiting out.
I would like to use the library threads (or perhaps parallel) for loading/preprocessing data into a queue but I am not entirely sure how it works. In summary;
Load data (tensors), pre-process tensors (this takes time, hence why I am here) and put them in a queue. I would like to have as many threads as possible doing this so that the model is not waiting or not waiting for long.
For the tensor at the top of the queue, extract it and forward it through the model and remove it from the queue.
I don't really understand the example in https://github.com/torch/threads enough. A hint or example as to where I would load data into the queue and train would be great.
EDIT 14/03/2016
In this example "https://github.com/torch/threads/blob/master/test/test-low-level.lua" using a low level thread, does anyone know how I can extract data from these threads into the main thread?
Look at this multi-threaded data provider:
https://github.com/soumith/dcgan.torch/blob/master/data/data.lua
It runs this file in the thread:
https://github.com/soumith/dcgan.torch/blob/master/data/data.lua#L18
by calling it here:
https://github.com/soumith/dcgan.torch/blob/master/data/data.lua#L30-L43
And afterwards, if you want to queue a job into the thread, you provide two functions:
https://github.com/soumith/dcgan.torch/blob/master/data/data.lua#L84
The first one runs inside the thread, and the second one runs in the main thread after the first one completes.
Hopefully that makes it a bit more clear.
If Soumith's examples in the previous answer are not very easy to use, I suggest you build your own pipeline from scratch. I provide here an example of two synchronized threads : one for writing data and one for reading data:
local t = require 'threads'
t.Threads.serialization('threads.sharedserialize')
local tds = require 'tds'
local dict = tds.Hash() -- only local variables work here, and only tables or tds.Hash()
dict[1] = torch.zeros(4)
local m1 = t.Mutex()
local m2 = t.Mutex()
local m1id = m1:id()
local m2id = m2:id()
m1:lock()
local pool = t.Threads(
1,
function(threadIdx)
end
)
pool:addjob(
function()
local t = require 'threads'
local m1 = t.Mutex(m1id)
local m2 = t.Mutex(m2id)
while true do
m2:lock()
dict[1] = torch.randn(4)
m1:unlock()
print ('W ===> ')
print(dict[1])
collectgarbage()
collectgarbage()
end
return __threadid
end,
function(id)
end
)
-- Code executing on master:
local a = 1
while true do
m1:lock()
a = dict[1]
m2:unlock()
print('R --> ')
print(a)
end
I'm using this tool for talking to hbase:
org.hbase:asynchbase:1.7.0
Why does the following code take ~70 ms?
long start = System.currentTimeMillis();
Deferred<ArrayList<KeyValue>> meta = hbaseClient.get(new GetRequest(propsCtx.hbaseTable, rowIdMeta(tsnfdr.id)));
long end = System.currentTimeMillis();
log.info("supposed 'non-blocking' async hbase call took {} millis", end - start);
(The call to 'propsCtx.hbaseTable' and 'rowIdMeta(tsnfdr.id)' is not the problem).
The docs are very limited, but from the signature of the method, observations in visualvm on usage of netty threads under the hood, and a quick look at the source code tells me that I'm supposed to be using an async api.