Consider the following two snippets of code where I am trying to launch 10000 threads:
Snippet 1
std::array<std::future<void>, 10000> furArr_;
try
{
size_t index = 0;
for (auto & fut : furArr_)
{
std::cout << "Created thread # " << index++ << std::endl;
fut = std::async(std::launch::async, fun);
}
}
catch (std::system_error & ex)
{
std::string str = ex.what();
std::cout << "Caught : " << str.c_str() << std::endl;
}
// I will call get afterwards, still 10000 threads should be active by now assuming "fun" is time consuming
Snippet 2
std::array<std::thread, 10000> threadArr;
try
{
size_t index = 0;
for (auto & thr : threadArr)
{
std::cout << "Created thread # " << index++ << std::endl;
thr = std::thread(fun);
}
}
catch (std::system_error & ex)
{
std::string str = ex.what();
std::cout << "Caught : " << str.c_str() << std::endl;
}
The first case always succeeds .i.e. I am able to create 10000 threads and then I have to wait for all of them to finish. In the second case, almost always I end up getting an exception("resource unavailable try again") after creating 1600+ threads.
With a launch policy of std::launch::async, I thought that the two snippets should behave the same way. How different std::async with a launch policy of async is from launching a thread explicitly using std::thread?
I am on Windows 10, VS2015, binary is built in x86 release mode.
Firstly, thanks to Igor Tandetnik for giving me the direction for this answer.
When we use std::async (with async launch policy), we are saying:
“I want to get this work done on a separate thread”.
When we use std::thread we are saying:
“I want to get this work done on a new thread”.
The subtle difference means that async (is usually) implemented using thread pools. Which means if we have invoked a method using async multiple times, often the thread id inside that method will repeat i.e. async allocates multiple jobs to the same set of threads from the pool. Whereas with std::thread, it never will.
This difference means that launching threads explicitly will be potentially more resource intensive (and thus the exception) than using async with async launch policy.
Related
I have some problem with st::async when is use this in other function other than Main function,
suppose, I have functions like flowing :
void printData()
{
for (size_t i = 0; i < 5; i++)
{
std::cout << "Test Function" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
void runningAsync()
{
auto r = std::async(std::launch::async, test);
}
int main()
{
runningAsync();
std::cout << "Main Function" << std::endl;
}
the output of this code is :
Test Function
Test Function
Test Function
Test Function
Test Function
Main Function
that is not good, Main thread wait for other thread that be end.
I want runningAsync() function run in other thread and at the same time "Main Function" in main thread print on screan, this is possible with std::thread.
is that way for this running this functions an same time (concurrency)?
The reason is that std::async returns a std::future which you store in an auto variable. As soon as your future runs out of scope (at the end of runningAsync()!), its destructor blocks until the task is finished. If you do not want that, you could for example store the future in a global container.
This QUESTION answered in :
main thread waits for std::async to complete
Can I use std::async without waiting for the future limitation?
Whoever, If you store the std::future object, its lifetime will be extended to the end of main and you get the behavior you want.
void printData()
{
for (size_t i = 0; i < 5; i++)
{
std::cout << "Test Function" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
std::future<void> runningAsync()
{
return std::async(std::launch::async, test);
}
int main()
{
auto a = runningAsync();
std::cout << "Main Function" << std::endl;
}
That's a problem because std::future's destructor may block and wait for the thread to finish. see this link for more details
I am using promises in C++, and have this really simple program. I create a promise, and pass it to a thread function by reference.
Inside the thread, I create a string and then set the promise's value to that string. Then in the main program, I retrieve the promise's value using a future. I'm confused about why the address of the string doesn't remain the same? Ideally it shouldn't vary between the threads since it belongs to shared memory right?
void tfunc(promise<string> & prms) {
string str("Hello from thread");
cout << (void *)str.data() << endl;
prms.set_value(str);
}
int main() {
promise<string> prms;
future<string> ftr = prms.get_future();
thread th(&tfunc, ref(prms));
string str = ftr.get();
// This address should be the same as that inside the thread, right?
cout << (void *)str.data() << endl;
th.join();
return 0;
}
This question has been asked before and if I am not wrong, the only way to read the result of a future is either to call get() and block until it is ready or using wait_for() with zero duration as mentioned in the answer - Get the status of a std::future
But, if I just want a worker thread to return me a result that I want it to compute and not wait or block myself for it to complete, can I not just pass it a callback that the worker thread can call when it has computed the result for me? Something like below -
#include <iostream>
#include <thread>
#include <functional>
void foo(std::function<void(int)> callback)
{
int result = 5;
callback(result);
}
int main()
{
int result = 0;
std::thread worker(foo, [](int result)
{
std::cout << "Result from worker is " << result << std::endl;
});
worker.join();
}
Here, the worker thread would just execute the callback when it has computed the result for me. I don't have to wait for it to finish or block or check in a loop to know when it's ready.
Please advice is this is a good approach to be used as currently there is no way to do this without blocking or checking for it in a loop?
You can certainly create your own thread with a callback, but as soon as you move away from a toy example you will notice that you have potentially created a synchronization problem. This is because your callback is being invoked from a separate thread. So you may want to have the worker thread instead post a message to a queue which you will read later, unless there is no shared state or a mutex is already in place.
In your specific example, let's add one line of code:
int main()
{
std::thread worker(foo, [](int result)
{
std::cout << "Result from worker is " << result << std::endl;
});
std::cout << "I am the main thread" << std::endl; // added
worker.join();
}
You might think that there are only two possible outputs:
I am the main thread
Result from worker is 5
and
Result from worker is 5
I am the main thread
But in fact there are other possible outputs, such as:
Result from worker is I am the main thread
5
So you have created a bug. You either need synchronization on your shared state (which includes I/O), or you need to orchestrate everything from the main thread (which is what blocking or checking for a future result gives you).
I've been trying to switch my code from using one io_service per network connection to using shared ones, and I'm seeing some very odd behaviour on server sockets (client ones seem to work OK).
In order to try to work out what's going on I've re-started building up a simple example that will allow me to check my assumptions about everything that ought to happen. The first problem I've hit is that io_service::run doesn't exit when there are no handlers left, and as far as I can tell the handlers aren't removed from the work queue.
I have one thread that does an async_accept followed by an async_read. There is a separate client thread (which has its own io_service). The client thread's io_service is never run, and the server's one is run in yet another thread.
I'm using a condition variable to wait in the server thread for the read to complete (which will never happen as the client never writes). This times out just fine and then I call socket.cancel(). I would expect this to remove the read handler and run to exit as the work queue is now empty.
I do see the read handler get called (with a cancel error), but run never exits. When I tie the socket lifetime to the handler lifetime (by lambda capturing a shared_ptr to the socket) the memory isn't freed either.
The server is set up like this:
std::mutex mutex;
std::unique_lock<std::mutex> lock(mutex);
std::condition_variable signal;
boost::asio::io_service server_service;
boost::asio::ip::tcp::acceptor listener(server_service);
std::mutex read_mutex;
std::unique_lock<std::mutex> read_lock(read_mutex);
std::condition_variable read_done;
std::thread server([&]() {
std::unique_lock<std::mutex> lock(mutex);
listener.open(boost::asio::ip::tcp::v4());
listener.set_option(boost::asio::socket_base::enable_connection_aborted(true));
listener.bind(boost::asio::ip::tcp::endpoint(boost::asio::ip::tcp::v4(), 4567));
listener.listen();
std::shared_ptr<connection> server_cnx(new connection(server_service));
listener.async_accept(server_cnx->socket,
[&, server_cnx](const boost::system::error_code& error) {
log_thread() << "Server got a connection " << error << std::endl;
boost::asio::async_read_until(server_cnx->socket, server_cnx->buffer, '\n',
[&, server_cnx](const boost::system::error_code& error, std::size_t bytes) {
log_thread() << "Got " << bytes << ", " << error << std::endl;
std::unique_lock<std::mutex> lock(read_mutex);
lock.unlock();
read_done.notify_one();
});
});
lock.unlock();
signal.notify_one();
if ( read_done.wait_for(read_lock, std::chrono::seconds(1)) == std::cv_status::timeout ) {
log_thread() << "Server read timed out -- cancelling socket jobs" << std::endl;
server_cnx->socket.cancel();
server_cnx->socket.close();
} else {
log_thread() << "Server data read" << std::endl;
}
log_thread() << "Exiting server thread" << std::endl;
});
signal.wait(lock);
log_thread() << "Server set up" << std::endl;
The io_service thread is set up like this:
std::thread server_io([&]() {
log_thread() << "About to service server IO requests" << std::endl;
try {
server_service.run();
} catch ( ... ) {
log_thread() << "Exception caught" << std::endl;
}
log_thread() << "**** Service jobs all run" << std::endl;
signal.notify_one();
});
The output is as follows:
10.0002 139992957945728 Server set up
10.0005 139992957945728 Client set up
10.0006 139992848398080 About to service server IO requests
10.0006 139992848398080 Server got a connection system:0
11.0003 139992934819584 Server read timed out -- cancelling socket jobs
11.0004 139992934819584 Exiting server thread
11.0004 139992848398080 Got 0, system:125
20.0006 139992957945728 IO thread timed out servicing requests -- stopping it
^^^ This should not happen because the server service should have run out of work
20.0006 139992957945728 Waiting for things to close....
22.0008 139992957945728 Wait over, exiting
(Columns are time + 10s, thread ID, log message)
At the 11 second mark you can see that the async_read_until is called. This is the last handler in the server's io_service and yet run doesn't exit.
Even after the time out waiting for run to exit fires and the waiting thread does io_service::stop(), still run doesn't exit (there's another 2 second wait there).
The full code is on github
The program is invoking undefined behavior when the server thread attempts to unlock the read_lock that it does not own.
int main()
{
...
std::mutex read_mutex;
std::unique_lock<std::mutex> read_lock(read_mutex); // Acquired by main.
std::condition_variable read_done;
std::thread server([&]() { // Capture lock reference.
std::unique_lock<std::mutex> lock(mutex);
...
// The next line invokes undefined behavior as this thread does did
// not acquire read_lock.mutex().
if (read_done.wait_for(read_lock, ...)
// ^^^^^^^^^ caller does not own.
{
...
}
});
signal.wait(lock);
...
}
In particular, when invoking condition_variable::wait_for(lock), the standard requires that lock.owns_lock() is true and lock.mutex() is locked by the calling thread.
Mixing synchronous and asynchronous flows often add complexity. In this particular case, where the synchronous calls are intertwined throughout each layer using lower-level constructs for event/signal notification without a persisted state, I think that it adds unnecessary complexity and overcomplicates the flow. Furthermore, the broad scope of variables can add complexity. If read_lock had never been captured by the lambdas, then a compiler error would have occurred.
Consider the separation in space when trying to observe two events:
// I will eventually be interested when the server starts
// accepting connections, so start setting up now.
std::mutex server_mutex;
std::unique_lock<std::mutex> server_lock(server_mutex);
std::condition_variable server_started;
std::thread server([&]()
{
// I will eventually be interested when the server reads
// data, so start setting up now.
std::mutex read_mutex;
std::unique_lock<std::mutex> read_lock(read_mutex);
std::condition_variable read_done;
listener.async_accept(...,
[&](...)
{
// Got connection.
async_read_until(...,
[&](...)
{
// Someone may be interested that data has been read,
// so use the correct mutex and condition_variable
// pair.
std::unique_lock<std::mutex> read_lock(read_mutex);
read_lock.unlock();
read_done.notify_one();
});
}); // async_accept
// Someone may be interested that I am accepting connections,
// so use the correct mutex and condition_variable pair.
std::unique_lock<std::mutex> server_lock(server_mutex);
server_lock.unlock();
server_done.notify_one();
// I am now interested in if data has been read.
read_done.wait_for(read_lock);
}); // server thread
// I am now interested in if the server has started.
server_started.wait(server_lock);
The caller has to prepare to handle an event, start an operation, then wait for the event, and the operation must know the event the caller is interested in. To worsen the situation, one must now consider lock ordering to prevent deadlocks. Note how in the above example, the server thread acquires the read_mutex and then the server_mutex. Another thread cannot acquire the mutexes in a difference order without introducing the chance of a deadlock. In terms of complexity, this approach scales poorly with the number of events.
It may be worth considering re-examining the program's flow and control structure. If it can be written to be primarily asynchronous, then callback chains, continuations, or a signal-and-slot system (Boost.Signals) may uncomplicate the solution. If one prefers to have asynchronous code read as if it was synchronous, then Boost.Asio's support for coroutines can provide a clean solution. Finally, if one needs to synchronously wait on an asynchronous operation's result or timeout, then consider using Boost.Asio's support for std::future or using them directly.
// Use an asynchronous operation so that it can be cancelled on timeout.
std::future<std::size_t> on_read = boost::asio::async_read_until(
socket, buffer, '\n',boost::asio::use_future);
// If timeout occurs, then cancel the operation.
if (on_read.wait_for(std::chrono::seconds(1)) == std::future_status::timeout)
{
socket.cancel();
}
// Otherwise, the operation completed (with success or error).
else
{
// If the operation failed, then on_read.get() will throw a
// boost::system::system_error.
auto bytes_transferred = on_read.get();
}
While I would strongly advocate re-examining the overall control structure and reducing variable scope, the following sample is roughly equivalent to the above example, but may be slightly easier to maintain with its use of std::future:
// I will eventually be interested when the server starts
// accepting connections, so start setting up now.
std::promise<void> server_started_promise;
auto server_started = server_started_promise.get_future();
std::thread server([&]()
{
// I will eventually be interested when the server reads
// data, so start setting up now.
std::promise<void> read_done_promise;
auto read_done = read_done_promise.get_future();
listener.async_accept(...,
[&](...)
{
// Got connection.
async_read_until(...,
[&](...)
{
// Someone may be interested that data has been read.
read_done_promise.set_value();
});
}); // async_accept
// Someone may be interested that I am accepting connections.
server_started_promise.set_value();
// I am now interested in if data has been read.
read_done.wait_for(...);
}); // server thread
// I am now interested in if the server has started.
server_started.wait();
Here is a complete example based on the original code that demonstrates using std::future to control flow and timeout asynchronous operations in a synchronous manner:
#include <future>
#include <iostream>
#include <thread>
#include <boost/asio.hpp>
#include <boost/asio/use_future.hpp>
#include <boost/optional.hpp>
#include <boost/utility/in_place_factory.hpp>
int main()
{
using boost::asio::ip::tcp;
// Setup server thread.
boost::asio::io_service server_io_service;
std::promise<tcp::endpoint> server_promise;
auto server_future = server_promise.get_future();
// Start server thread.
std::thread server_thread(
[&server_io_service, &server_promise]
{
tcp::acceptor acceptor(server_io_service);
acceptor.open(tcp::v4());
acceptor.set_option(
boost::asio::socket_base::enable_connection_aborted(true));
acceptor.bind(tcp::endpoint(tcp::v4(), 0));
acceptor.listen();
// Handlers will not chain work, so control the io_service with a work
// object.
boost::optional<boost::asio::io_service::work> work(
boost::in_place(std::ref(server_io_service)));
// Accept a connection.
tcp::socket server_socket(server_io_service);
auto on_accept = acceptor.async_accept(server_socket,
boost::asio::use_future);
// Server has started, so notify caller.
server_promise.set_value(acceptor.local_endpoint());
// Wait for connection or error.
boost::system::system_error error =
make_error_code(boost::system::errc::success);
try
{
on_accept.get();
}
catch (const boost::system::system_error& e)
{
error = e;
}
std::cout << "Server got a connection " << error.code() << std::endl;
// Read from connection.
boost::asio::streambuf buffer;
auto on_read = boost::asio::async_read_until(
server_socket, buffer, '\n', boost::asio::use_future);
// The async_read operation is work, so destroy the work object allowing
// run() to exit.
work = boost::none;
// Timeout the async read operation.
if (on_read.wait_for(std::chrono::seconds(1)) ==
std::future_status::timeout)
{
std::cout << "Server read timed out -- cancelling socket jobs"
<< std::endl;
server_socket.close();
}
else
{
error = make_error_code(boost::system::errc::success);
std::size_t bytes_transferred = 0;
try
{
bytes_transferred = on_read.get();
}
catch (const boost::system::system_error& e)
{
error = e;
}
std::cout << "Got " << bytes_transferred << ", "
<< error.code() << std::endl;
}
std::cout << "Exiting server thread" << std::endl;
});
// Wait for server to start accepting connections.
auto server_endpoint = server_future.get();
std::cout << "Server set up" << std::endl;
// Client thread.
std::promise<void> promise;
auto future = promise.get_future();
std::thread client_thread(
[&server_endpoint, &promise]
{
boost::asio::io_service io_service;
tcp::socket client_socket(io_service);
boost::system::error_code error;
client_socket.connect(server_endpoint, error);
std::cout << "Connected " << error << std::endl;
promise.set_value();
// Keep client socket alive, allowing server to timeout.
std::this_thread::sleep_for(std::chrono::seconds(2));
std::cout << "Exiting client thread" << std::endl;
});
// Wait for client to connect.
future.get();
std::cout << "Client set up" << std::endl;
// Reset generic promise and future.
promise = std::promise<void>();
future = promise.get_future();
// Run server's io_service.
std::thread server_io_thread(
[&server_io_service, &promise]
{
std::cout << "About to service server IO requests" << std::endl;
try
{
server_io_service.run();
}
catch (const std::exception& e)
{
std::cout << "Exception caught: " << e.what() << std::endl;
}
std::cout << "Service jobs all run" << std::endl;
promise.set_value();
});
if (future.wait_for(std::chrono::seconds(3)) ==
std::future_status::timeout)
{
std::cout << "IO thread timed out servicing requests -- stopping it"
<< std::endl;
server_io_service.stop();
}
// Join all threads.
server_io_thread.join();
server_thread.join();
client_thread.join();
}
So of course this multi-threading is a tricky business. Turns out that in this case the read lock is acquired in the wrong place so the handler is being blocked by the thread waiting for it to complete.
I guess the lesson here is to never handle thread locks without some sort of time out.
I am in the middle of developing a game and came across the problem of multithreading. I already used multithreading successfully when loading resources. I did that by creating some threads at some point, assigned them functions, and waited for them to finish, while drawing a loading screen, pretty straightforward.
Now I want to create some threads, that can wait idle till they receive a function, when they do, solve that, then stay idle again. They must operate in a game loop, which is roughly like this (I came up with these function names just for easy visualization):
std::thread t0,t1;
while(gamerunning)
{
UpdateGame();
t0.receiveFunc( RenderShadow );
t1.receiveFunc( RenderScene );
WaitForThreadstoFinishWork();
RenderEverything(); //Only draw everything if the threads finished (D3D11's Deferred Context rendering)
}
t0.Destroy();
t1.Destroy();
My rendering engine is working, and for the time being (for testing), I created threads in my game loop, which is a terrible way of even a quick test, because my rendering speed even slowed down. By the way, I am using C++11's library.
Long story short, I want to create threads before my game loop takes place, and use those in the game loop afterwards, hope someone can help me out. If it is an option, I would really want to stay away from the lower levels of threading, I just need the most straightforward way of doing this.
Following your most recent comments, here is an example implementation of a thread that wakes up on demand, runs its corresponding task and then goes back to sleep, along with the necessary functions to manage it (wait for task completion, ask for shutdown, wait for shutdown).
Since your set of functions is fixed, all you'll have left to do is to create as much threads as you need (ie. 7, probably in a vector), each with its own corresponding task.
Note that once you remove the debugging couts there's little code left, so I don't think there is a need to explain the code (it's pretty self-explanatory IMHO). However don't hesitate to ask if you need explanations on some details.
class TaskThread {
public:
TaskThread(std::function<void ()> task)
: m_task(std::move(task)),
m_wakeup(false),
m_stop(false),
m_thread(&TaskThread::taskFunc, this)
{}
~TaskThread() { stop(); join(); }
// wake up the thread and execute the task
void wakeup() {
auto lock = std::unique_lock<std::mutex>(m_wakemutex);
std::cout << "main: sending wakeup signal..." << std::endl;
m_wakeup = true;
m_wakecond.notify_one();
}
// wait for the task to complete
void wait() {
auto lock = std::unique_lock<std::mutex>(m_waitmutex);
std::cout << "main: waiting for task completion..." << std::endl;
while (m_wakeup)
m_waitcond.wait(lock);
std::cout << "main: task completed!" << std::endl;
}
// ask the thread to stop
void stop() {
auto lock = std::unique_lock<std::mutex>(m_wakemutex);
std::cout << "main: sending stop signal..." << std::endl;
m_stop = true;
m_wakecond.notify_one();
}
// wait for the thread to actually be stopped
void join() {
std::cout << "main: waiting for join..." << std::endl;
m_thread.join();
std::cout << "main: joined!" << std::endl;
}
private:
std::function<void ()> m_task;
// wake up the thread
std::atomic<bool> m_wakeup;
bool m_stop;
std::mutex m_wakemutex;
std::condition_variable m_wakecond;
// wait for the thread to finish its task
std::mutex m_waitmutex;
std::condition_variable m_waitcond;
std::thread m_thread;
void taskFunc() {
while (true) {
{
auto lock = std::unique_lock<std::mutex>(m_wakemutex);
std::cout << "thread: waiting for wakeup or stop signal..." << std::endl;
while (!m_wakeup && !m_stop)
m_wakecond.wait(lock);
if (m_stop) {
std::cout << "thread: got stop signal!" << std::endl;
return;
}
std::cout << "thread: got wakeup signal!" << std::endl;
}
std::cout << "thread: running the task..." << std::endl;
// you should probably do something cleaner than catch (...)
// just ensure that no exception propagates from m_task() to taskFunc()
try { m_task(); } catch (...) {}
std::cout << "thread: task completed!" << std::endl;
std::cout << "thread: sending task completed signal..." << std::endl;
// m_wakeup is atomic so there is no concurrency issue with wait()
m_wakeup = false;
m_waitcond.notify_all();
}
}
};
int main()
{
// example thread, you should really make a pool (eg. vector<TaskThread>)
TaskThread thread([]() { std::cout << "task: running!" << std::endl; });
for (int i = 0; i < 2; ++i) { // dummy example loop
thread.wakeup();
// wake up other threads in your thread pool
thread.wait();
// wait for other threads in your thread pool
}
}
Here's what I get (actual order varies from run to run depending on thread scheduling):
main: sending wakeup signal...
main: waiting for task completion...
thread: waiting for wakeup or stop signal...
thread: got wakeup signal!
thread: running the task...
task: running!
thread: task completed!
thread: sending task completed signal...
thread: waiting for wakeup or stop signal...
main: task completed!
main: sending wakeup signal...
main: waiting for task completion...
thread: got wakeup signal!
thread: running the task...
task: running!
thread: task completed!
thread: sending task completed signal...
thread: waiting for wakeup or stop signal...
main: task completed!
main: sending stop signal...
main: waiting for join...
thread: got stop signal!
main: joined!