Two threads, how to implement communication between them. (Pseudocode) - multithreading

I have two threads:
Main thread :
Main thread is listening for HTTP requests,
Main thread registers handler to HTTP request used for long polling
Second thread:
Getting data from different socket.
if it finds something in data from socket, updates thread local storage,
and IF Main thread has HTTP request pending it sends data to it somehow.
How do I make Main thread and second thread communicate to each other?
Main Thread http handler pseudocode:
function mg_handler(request){
var handle = parseHandle(request.data);
var storage_name = parseStorageName(request.data);
var response = WaitForResponse(handle,storage_name);
mg_printf(rasponse);
return;
}
Second thread pseudocode:
var storage
function t_run()
{
var buf
while(1){
recvfrom(socket,buf);
var result;
bool found_something = search(buf,result);
if(found_something){
update(storage,result);
//if WaitForResponse is waiting let it continue by sending storage to it somehow.
//???
}
}
cleanup:
return;
}

In your scenario there is no communication between any threads. Communication would mean for example one thread controls the execution of another thread. In your case you are simply sharing data (or the memory that holds the data to be more precise).
Note, that you have to provide a synchronization mechanism in order to synchronize the concurrent access to the shared memory. Otherwise you will experience all the concurrency issues that usually come with multithreading, especially when shared memory is involved.
Most languages that support multithreading also support synchronization on compiler level.
// Shared resources.
// Requires some synchronization mechanism for thread-safe access...
shared_pending_http_request_flag;
shared_data_variable;
function main()
{
// Main thread context
startBackgroundThread();
listenToHttpRequests();
}
function startBackgroundThread()
{
// Background thread context
result_data := readDataFromSocket();
IF
main thread has set shared variable shared_pending_http_request_flag
THEN
shared_data_variable := result_data ;
}
function listenToHttpRequests()
{
// Main thread context
WHILE
listening to HTTP requests
DO
IF
there is a pending request
THEN
set the shared_pending_http_request_flag;
IF
shared_pending_http_request_flag AND shared_data_variable are both set
THEN
handle value of shared_data_variable;
}
For an object oriented language or any language that supports callbacks or method references, you should implement the Observer pattern. This removes the need to poll for changes, as the observable can simply notify the observer about changes. Generally the observable invokes a callback that was registered by the observer.

Related

Executing GTK functions from other threads

This question is about GTK and threads.
You may find it useful if your application crashes, freezes or you want to have a multithreaded GTK application.
Main Loop
In order to understand GTK you must understand 2 concepts.
All contemporary GUIs are single-threaded. They have a thread which processes events from window system (like button, mouse events).
Such a thread is called main event loop or main loop.
GTK is also single threaded and not MT-safe. This means, that you must not call any GTK functions from other threads, as it will lead to undefined behaviour.
As Gtk documentation states,
Like all GUI toolkits, GTK+ uses an event-driven programming model. When the user is doing nothing, GTK+ sits in the “main loop” and waits for input. If the user performs some action - say, a mouse click - then the main loop “wakes up” and delivers an event to GTK+. GTK+ forwards the event to one or more widgets.
Gtk is event-based and asynchronous. It reacts to button clicks not in the exact moment of clicking, but a bit later.
It can be very roughly written like this (don't try this at home):
static list *pollable;
int main_loop (void)
{
while (run)
{
lock_mutex()
event_list = poll (pollable); // check whether there are some events to react to
unlock_mutex()
dispatch (event_list); // react to events.
}
}
void schedule (gpointer function)
{
lock_mutex()
add_to_list (pollable, something);
unlock_mutex()
}
I want a delayed action in my app
For example, hide a tooltip in several seconds or change button text.
Assuming your application is single-threaded, if you call sleep() it will be executed in main loop.
sleep() means, that this particular thread will be suspended for specified amount of seconds. No work will be done.
And if this thread is main thread, GTK will not be able to redraw or react to user interactions. The application freezes.
What you should do is schedule function call. It can be done with g_timeout_add or g_idle_add
In the first case our poll() from snippet above will return this event in several seconds. In the latter case it will be returned when there are no events of higher priority.
static int count;
gboolean change_label (gpointer data)
{
GtkButton *button = data;
gchar *text = g_strdup_printf ("%i seconds left", --count);
if (count == 0)
return G_SOURCE_REMOVE;
return G_SOURCE_CONTINUE;
}
void button_clicked (GtkButton *button)
{
gtk_button_set_label (button, "clicked");
count = 5;
g_timeout_add (1 * G_TIME_SPAN_SECOND, change_label, button);
}
Returning a value from function is very important. If you don't do it, the behaviour is undefined, your task may be called again or removed.
I have a long-running task
Long-running tasks aren't different from calling sleep. While one thread is busy with that task, it can't perform any other tasks, obviously. If that is a GUI thread, it can't redraw interface. That's why you should move all long-running tasks to other threads. There is an exception, though: non-blocking IO, but it's out of topic of my answer.
I have additional threads and my app crashes
As already mentioned, GTK is not MT-safe. You must not call Gtk functions from other threads.
You must schedule execution. g_timeout_add and g_idle_add are MT-safe, unlike other GTK functions.
That callbacks will be executed in main loop. If you have some shared resources between callback and thread you must read/write them atomically or use a mutex.
static int data;
static GMutex mutex;
gboolean change_label (gpointer data)
{
GtkButton *button = data;
int value;
gchar *text;
// retrieve data
g_mutex_lock (&mutex);
value = data;
g_mutex_unlock (&mutex);
// update widget
text = g_strdup_printf ("Current data value: %i", value);
return G_SOURCE_REMOVE;
}
gpointer thread_func (gpointer data)
{
GtkButton *button = data;
while (TRUE)
{
sleep (rand_time);
g_mutex_lock (&mutex);
++data;
g_mutex_unlock (&mutex);
g_idle_add (change_label, button);
}
}
Make sure mutexes are held as little as possible. Imagine you lock a mutex in another thread and do some IO. The main loop will be stuck until the mutex is released. There is g_mutex_try_lock() that returns immidiately, but it can bring additional syncronization problems because you can't guarantee that the mutex will be unlocked when mainloop tries to lock it.
Follow up: but python is single-threaded and GIL et cetera?
You can imagine that python is multi-threaded application run on a single-core machine.
You never know when the threads will be switched. You call a GTK function but you don't know in which state the main loop is. Maybe it free'd resources just a moment before. Always schedule.
What is not discussed and further reading
Detailed documentation on glib main loop can be found here
GSource as a more low-level primitive.
GTask

Boost ASIO as an event loop with boost lockfree queue for socket write

I am using boost ASIO for a TCP client. for the most part the ASIO is a glorified event loop for read and write. There is actually only one client managed by the ASIO.
The architecture is like this -
The TCP server streams continuous messages. The Client will read the messages, process it and ack back with proper code.
My code runs in client side. There is one thread running io_service. The io_service thread reads messages and distributes it to N number of worker threads using a boost lockfree SPSC queue. The workers after processing posts the replies to the io_service thread.
most important concern for me is the rate of read and write. So I am using synchronous reads and writes.
Read Code:
void read ()
{
if (_connected && !_readInProgress) {
_socket.async_read_some(boost::asio::null_buffers(),
make_boost_alloc_handler(_readAllocator,
[self = shared_from_this(), this] (ErrorType err, unsigned a)
{
connection()->handleRead(err);
_readInProgress = false;
if (err) disconnect();
else asyncRead();
});
_readInProgress = true;
}
}
Basically I use read_some with nullbuffer() and then directly use Unix system calls to read the messages. The read give N number of messages which are enqueued to threads in a loop.
I want use the boost SPSC queue in the reverse direction for writes to the socket from workers.
Write:
// Get the queue to post writes
auto getWriteQ ()
{
static thread_local auto q =
std::make_shared< LFQType >(_epoch);
return q;
}
So each thread gets a thread-local Q using getWriteQ. The writes to the queue looks like this:
void write (Buf& buf) override
{
auto q = getWriteQ();
while (!q->enqueue(buf) && _connected);
if (!_connected) return;
_ioService.post( [self = shared_from_this(), this, q]()
{
writeHelper(q); });
}
}
Now this is inefficent, as we do a ioservice post for each write. The write handler at a time actually writes upto 32 messages in a single system-call using sendmmsg()
So I am looking for help with 2 things:
Is the design any good?
Any fool proof way to minimize the no. of posts. I was thinking of keep an atomic enqueue count. The worker thread will do this -
the writing thread does this - (Pseudo code)
bool post = false;
if(enqueue_count == 0) post = true
// enqueue the message
++enqueue_count
if(post)
// post the queue event
The io-service thread does this -
enqueue_count -= num_processed;
if (enqueue_count)
// repost the queue for further processing
Would this work if the enqueue_count is atomic ?

Play Framework: What happens when requests exceeds the available threads

I have one thread in the thread-pool servicing blocking request.
def sync = Action {
import Contexts.blockingPool
Future {
Thread.sleep(100)
}
Ok("Done")
}
In Contexts.blockingPool is configured as:
custom-pool {
fork-join-executor {
parallelism-min = 1
parallelism-max = 1
}
}
In theory, if above request receives 100 simultaneous requests, the expected behaviour should be: 1 request should sleep(100) and rest of 99 requests should be rejected (or queued until timeout?). However I observed that extra worker threads are created to service rest of requests. I also observed that latency increases as (gets slower to service request) as number of threads in the pool gets smaller than the requests received.
What is expected behavior if a request larger than configured thread-pool size is received?
Your test is not correctly structured to test your hypothesis.
If you go over this section in the docs you will see that Play has a few thread pools/execution contexts. The one that is important with regards to your question is the default thread pool and how that relates to the HTTP requests served by your action.
As the doc describes, the default thread pool is where all application code is run by default. I.e. all action code, including all Future's (not explicitly defining their own execution context), will run in this execution context/thread pool. So using your example:
def sync = Action {
// *** import Contexts.blockingPool
// *** Future {
// *** Thread.sleep(100)
// ***}
Ok("Done")
}
All the code in your action not commented by // *** will run in the default thread pool.
I.e. When a request gets routed to your action:
the Future with the Thread.sleep will be dispatched to your custom execution context
then without waiting for that Future to complete (because it's running in it's own thread pool [Context.blockingPool] and therefore not blocking any threads on the default thread pool)
your Ok("Done") statement is evaluated and the client receives the response
Approx. 100 milliseconds after the response has been received, your Future completes
So to explain you observation, when you send 100 simultaneous requests, Play will gladly accept those requests, route to your controller action (executing on the default thread pool), dispatch to your Future and then respond to the client.
The default size of the default pool is
play {
akka {
...
actor {
default-dispatcher = {
fork-join-executor {
parallelism-factor = 1.0
parallelism-max = 24
}
}
}
}
}
to use 1 thread per core up to a max of 24.
Given that your action does very little (excl. the Future), you will be able to handle into the 1000's of requests/sec without a sweat. Your Future will however take much longer to work through the backlog because you are blocking the only thread in your custom pool (blockingPool).
If you use my slightly adjusted version of your action, you will see what confirms the above explanation in the log output:
object Threading {
def sync = Action {
val defaultThreadPool = Thread.currentThread().getName;
import Contexts.blockingPool
Future {
val blockingPool = Thread.currentThread().getName;
Logger.debug(s"""\t>>> Done on thread: $blockingPool""")
Thread.sleep(100)
}
Logger.debug(s"""Done on thread: $defaultThreadPool""")
Results.Ok
}
}
object Contexts {
implicit val blockingPool: ExecutionContext = Akka.system.dispatchers.lookup("blocking-pool-context")
}
All your requests are swiftly dealt with first and then your Future's complete one by one afterwards.
So in conclusion, if you really want to test how Play will handle many simultaneous requests with only one thread handling requests, then you can use the following config:
play {
akka {
akka.loggers = ["akka.event.Logging$DefaultLogger", "akka.event.slf4j.Slf4jLogger"]
loglevel = WARNING
actor {
default-dispatcher = {
fork-join-executor {
parallelism-min = 1
parallelism-max = 1
}
}
}
}
}
you might also want to add a Thread.sleep to your action like this (to slow the default thread pools lonesome thread down a bit)
...
Thread.sleep(100)
Logger.debug(s"""<<< Done on thread: $defaultThreadPool""")
Results.Ok
}
Now you will have 1 thread for requests and 1 thread for your Future's.
If you run this with high concurrent connections you will notice that the client blocks while Play handles the requests one by one. Which is what you expected to see...
Play uses AkkaForkJoinPool which extends scala.concurrent.forkjoin.ForkJoinPool.
It has internal queue of tasks.
You may also find this description interesting in respect to handling blocking code by fork-join-pool: Scala: the global ExecutionContext makes your life easier

CoGetInterfaceAndReleaseStream let my thread hangs

UINT __stdcall CExternal::WorkThread( void * pParam)
{
HRESULT hr;
CTaskBase* pTask;
CComPtr<IHTMLDocument3> spDoc3;
CExternal* pThis = reinterpret_cast<CExternal*>(pParam);
if (pThis == NULL)
return 0;
// Init the com
::CoInitializeEx(0,COINIT_APARTMENTTHREADED);
hr = ::CoGetInterfaceAndReleaseStream(
pThis->m_pStream_,
IID_IHTMLDocument3,
(void**)&spDoc3);
if(FAILED(hr))
return 0;
while (pThis->m_bShutdown_ == 0)
{
if(pThis->m_TaskList_.size())
{
pTask = pThis->m_TaskList_.front();
pThis->m_TaskList_.pop_front();
if(pTask)
{
pTask->doTask(spDoc3); //do my custom task
delete pTask;
}
}
else
{
Sleep(10);
}
}
OutputDebugString(L"start CoUninitialize\n");
::CoUninitialize(); //release com
OutputDebugString(L"end CoUninitialize\n");
return 0;
}
The above the code that let my thread hang, the only output is "start CoUninitialize".
m_hWorker_ = (HANDLE)_beginthreadex(NULL, 0, WorkThread, this, 0, 0);
This code starts my thread, but the thread can't exit safely, so it waits. What the problem with this code?
The problem is not in this code, although it violates core COM requirements. Which says that you should release interface pointers when you no longer use them, calling IUnknown::Release(), and that an apartment-threaded thread must pump a message loop. Especially the message loop is important, you'll get deadlock when the owner thread of a single-threaded object (like a browser) is not pumping.
CoUninitialize() is forced to clean up the interface pointer wrapped by spDoc3 since you didn't do this yourself. It is clear from the code that the owner of the interface pointer actually runs on another thread, something to generally keep in mind since that pretty much defeats the point of starting your own worker thread. Creating your own STA thread doesn't fix this, it is still the wrong thread.
So the proxy needs to context switch to the apartment that owns the browser object. With the hard requirement that this apartment pumps a message loop so that the call can be dispatched on the right thread in order to safely call the Release() function. With very high odds that this thread isn't pumping messages anymore when your program is shutting down. Something you should be able to see in the debugger, locate the owner thread in the Debug + Windows + Threads window and see what it is doing.
Deadlock is the common outcome. The only good way to fix it is to shut down threads in the right order, this one has to shut down before the thread that owns the browser object. Shutting down a multi-threaded program cleanly can be quite difficult when threads have an interdependency like this. The inspiration behind the C++11 std::quick_exit() addition.

Question about thread synchronisation

i have a question about thread situation.
Suppose i have 3 threads :producer,helper and consumer.
the producer thread is in running state(and other two are in waiting state)and when its done it calls invoke,but the problem it has to invoke only helper thread not consumer,then how it can make sure that after it releases resources are to be fetched by helper thread only and then by consumer thread.
thanks in advance
Or have you considered, sometimes having separate threads is more of a problem than a solution?
If you really want the operations in one thread to be strictly serialized with the operations in another thread, perhaps the simpler solution is to discard the second thread and structure the code so the first thread does the operations in the order desired.
This may not always be possible, but it's something to bear in mind.
You could have, for instance, two mutexes (or whatever you are using): one for producer and helper, and other for producer and consumer
Producer:
//lock helper
while true
{
//lock consumer
//do stuff
//release and invoke helper
//wait for helper to release
//lock helper again
//unlock consumer
//wait consumer
}
The others just lock and unlock normally.
Another possible approach (maybe better) is using a mutex for producer / helper, and other helper / consumer; or maybe distribute this helper thread tasks between the other two threads. Could you give more details?
The helper thread is really just a consumer/producer thread itself. Write some code for the helper like you would for any other consumer to take the result of the producer. Once that's complete write some code for the helper like you would for any other producer and hook it up to your consumer thread.
You might be able to use queues to help you with this with locks around them.
Producer works on something, produces it, and puts it on the helper queue.
Helper takes it, does something with it, and then puts it on the consumer queue.
Consumer take its, consumes it, and goes on.
Something like this:
Queue<MyDataType> helperQ, consumerQ;
object hqLock = new object();
object cqLock = new object();
// producer thread
private void ProducerThreadFunc()
{
while(true)
{
MyDataType data = ProduceNewData();
lock(hqLock)
{
helperQ.Enqueue(data);
}
}
}
// helper thread
private void HelperThreadFunc()
{
while(true)
{
MyDataType data;
lock(hqLock)
{
data = helperQ.Dequeue();
}
data = HelpData(data);
lock(cqLock)
{
consumerQ.Enqueue(data);
}
}
}
// consumer thread
private void ConsumerThreadFunc()
{
while(true)
{
MyDataType data;
lock(cqLock)
{
data = consumerQ.Dequeue();
}
Consume(data);
}
}
NOTE: You will need to add more logic to this example to make sure usable. Don't expect it to work as-is. Mainly, use signals for one thread to let the other know that data is available in its queue (or as a worst case poll the size of the queue to make sure it is greater than 0 , if it is 0, then sleep -- but the signals are cleaner and more efficient).
This approach would let you process data at different rates (which can lead to memory issues).

Resources