I've been stuck for a whole night but can't figure out the problem.
I'm writing a server that accepts connections for a chatroom application. The idea is that the server gives the connected sockets to chat_manager, chat_manager puts them in a waitlist and starts a thread to constantly check if somebody is still in the waitlist. If somebody is in the waitlist, the chat_manager tries to read from the client to determine which room the client wants to join.
main.cpp looks like this :
boost::asio::io_service io_service;
chat_server server(io_service, 8091);
io_service.run();
Chat server code :
chat_server::chat_server(boost::asio::io_service& io_service, int port) : io_service_(io_service),
acceptor_(io_service, tcp::endpoint(tcp::v4(), port)) {
start_accept();
}
void chat_server::start_accept() {
chat_session_ptr new_session(new chat_session(io_service_));
acceptor_.async_accept(new_session->socket(), boost::bind(&chat_server::handle_accept, this, new_session,
boost::asio::placeholders::error));
}
void chat_server::handle_accept(chat_session_ptr s, const boost::system::error_code& error) {
if (!error) { // successfully connected
chat_manager::Instance().chat_session_join(s);
}
start_accept();
}
chat_manager is simply
chat_manager() { // constructor
waitlist_.resize(0);
manager_thread_ = boost::thread(&chat_manager::run, this);
}
int chat_session_join(chat_session_ptr chat_session) {
mutex_.lock();
waitlist_.push_back(chat_session);
mutex_.unlock();
return 1;
}
void run() {
while (1) {
mutex_.lock();
for (auto &s : waitlist_) {
std::string roomname;
if (s->wait_for_join(roomname)) {
...
}
break;
}
mutex_.unlock();
boost::this_thread::sleep( boost::posix_time::milliseconds(10) );
}
}
And session.cpp :
jsoncons::json chat_session::receive_request(boost::system::error_code& error) {
buffer_.consume(buffer_.size());
size_t bytes_transferred;
boost::system::error_code ec = boost::asio::error::would_block;
boost::asio::async_read_until(socket_, buffer_, "\n",
boost::bind(&chat_session::handle_read, shared_from_this(), _1, _2, &ec, &bytes_transferred ));
deadline_.expires_from_now(boost::posix_time::seconds(2));
deadline_.async_wait(boost::bind(&chat_session::check_deadline, shared_from_this()));
do pio_service_->run_one(); while (ec == boost::asio::error::would_block);
std::cout << "got it" << std::endl;
if ( ec || !socket_.is_open() ) {
std::cout << ec.message() << std::endl;
return jsoncons::json();
}
std::istream is(&buffer_);
return jsoncons::json::parse(is);
}
int chat_session::wait_for_join(std::string &roomname) {
boost::system::error_code error;
jsoncons::json request = receive_request(error);
...
}
void chat_session::handle_read(const boost::system::error_code& error, size_t bytes_transferred,
boost::system::error_code* out_error, size_t* out_bytes_transferred) {
deadline_.expires_at(boost::posix_time::pos_infin);
*out_error = error;
*out_bytes_transferred = bytes_transferred;
if (!error) {
} else {
std::cout << "handle_read error:" << error.message() << std::endl;
}
}
What happens is,
1. when the client connects and doesn't send anything, the connection is closed by the server after 2 seconds. which is correct. but
2. if the client sends one string before the connection expires, the server worked its way into handle_read but never gets past pio_service->run_one();
3. if the client send another string, wait_for_join will finish and the run() procedure starts to loop.
To me it seems that the use of io_services here is fine because check_deadline and handle_read worked, but if that really worked it would modify ec != would_block, and the do-while loop should return. What's happening here? I'm really confused.
Related
I am trying to create a multiplayer game where players can use voice chat to communicate.
I am using C++ and SFML-3.0 for everything.
Here is the GitHub page for the whole project: https://github.com/iwannabespace/test
First of all, here are some of the classes that I am using:
AudioCapturer.hpp
class AudioCapturer : public sf::SoundStream
{
public:
AudioCapturer(Client& client);
void receiveLoop(sf::Packet& packet);
private:
bool onGetData(sf::SoundStream::Chunk& data) override;
void onSeek(sf::Time timeOffset) override;
private:
std::recursive_mutex mutex;
std::vector<std::int16_t> samples;
std::vector<std::int16_t> tempbuffer;
size_t offset;
Client& client;
};
AudioCapturer.cpp
AudioCapturer::AudioCapturer(Client& client)
: offset(0), client(client)
{
initialize(1, 44100);
}
bool AudioCapturer::onGetData(sf::SoundStream::Chunk& data)
{
while (offset >= samples.size())
sf::sleep(sf::milliseconds(10));
{
std::scoped_lock lock(mutex);
tempbuffer.assign(samples.begin() + static_cast<std::vector<std::int64_t>::difference_type>(offset),
samples.end());
std::cout << "Tempbuffer address: " << &tempbuffer << std::endl;
}
data.samples = tempbuffer.data();
data.sampleCount = tempbuffer.size();
offset += tempbuffer.size();
return true;
}
void AudioCapturer::onSeek(sf::Time timeOffset)
{
offset = static_cast<std::size_t>(timeOffset.asMilliseconds()) * getSampleRate() * getChannelCount() / 1000;
}
void AudioCapturer::receiveLoop(sf::Packet& packet)
{
while (true)
{
if (client.receivedAudio)
{
sf::Packet copy = packet;
std::uint8_t command;
if (copy >> command)
{
std::size_t sampleCount = (copy.getDataSize() - 1) / sizeof(std::int16_t);
{
std::scoped_lock lock(mutex);
std::size_t oldSize = samples.size();
samples.resize(oldSize + sampleCount);
std::cout << "Samples address: " << &samples << std::endl;
std::cout << "Last item address: " << &samples[oldSize] << std::endl;
std::memcpy(&(samples[oldSize]),
static_cast<const char*>(copy.getData()) + 1,
sampleCount * sizeof(std::int16_t));
}
client.receivedAudio = false;
}
}
}
}
Client.hpp
class Client
{
public:
Client(const sf::IpAddress& ip, int port);
~Client();
bool connect();
void disconnect();
bool send(sf::Packet& packet);
bool receivePacket(sf::Packet& packet, std::unordered_map<std::uint32_t, Player>& players, std::uint32_t& thisPlayer);
public:
bool receivedAudio;
bool receivedPosition;
private:
int port;
sf::IpAddress ip;
sf::TcpSocket socket;
};
Client.cpp
Client::Client(const sf::IpAddress& ip, int port)
: ip(ip), port(port), receivedAudio(false), receivedPosition(false)
{
}
Client::~Client()
{
}
bool Client::connect()
{
if (socket.connect(ip, port) != sf::Socket::Done)
return false;
return true;
}
void Client::disconnect()
{
socket.disconnect();
}
bool Client::send(sf::Packet& packet)
{
if (socket.send(packet) != sf::Socket::Done)
return false;
return true;
}
bool Client::receivePacket(sf::Packet& packet, std::unordered_map<std::uint32_t, Player>& players, std::uint32_t& thisPlayer)
{
sf::Packet recv;
sf::Packet copy;
while (true)
{
if (socket.receive(recv) != sf::Socket::Done)
return false;
if (receivedAudio)
std::cout << "Data is not used!" << std::endl;
std::uint8_t command;
std::uint32_t id;
copy = recv;
if (copy >> command)
{
switch (command)
{
case ServerCommand::ADD_PLAYER:
copy >> id;
if (thisPlayer == 0)
{
thisPlayer = id;
std::cout << "My id is " << thisPlayer << std::endl;
}
players[id] = Player(5.f, id);
std::cout << "Player joined!" << std::endl;
break;
case ServerCommand::ADD_PLAYERS:
while (copy >> id)
players[id] = Player(5.f, id);
break;
case ServerCommand::RECEIVE_AUDIO:
receivedAudio = true;
packet = recv;
break;
case ServerCommand::RECEIVE_POSITION:
receivedPosition = true;
packet = recv;
break;
default:
break;
}
}
}
return true;
}
AudioRecorder.hpp
class AudioRecorder : public sf::SoundRecorder
{
public:
AudioRecorder(Client& client);
~AudioRecorder() override;
void activeness();
void setActive(bool active);
bool isActive() const;
private:
bool onStart() override;
bool onProcessSamples(const std::int16_t* samples, std::size_t sampleCount) override;
void onStop() override;
private:
Client& client;
bool active;
std::mutex mutex;
};
AudioRecorder.cpp
AudioRecorder::AudioRecorder(Client& client)
: client(client), active(false)
{
}
AudioRecorder::~AudioRecorder()
{
stop();
}
void AudioRecorder::activeness()
{
while (true)
{
if (sf::Keyboard::isKeyPressed(sf::Keyboard::Space) && !active)
{
std::scoped_lock lock(mutex);
active = true;
if (start()) {}
}
if (!sf::Keyboard::isKeyPressed(sf::Keyboard::Space) && active)
{
std::scoped_lock lock(mutex);
stop();
active = false;
}
}
}
void AudioRecorder::setActive(bool active)
{
this->active = active;
}
bool AudioRecorder::isActive() const
{
return active;
}
bool AudioRecorder::onStart()
{
std::cout << "Started!" << std::endl;
return true;
}
bool AudioRecorder::onProcessSamples(const std::int16_t* samples, std::size_t sampleCount)
{
sf::Packet packet;
packet << ServerCommand::RECEIVE_AUDIO;
packet.append(samples, sampleCount * sizeof(std::int16_t));
return client.send(packet);
}
void AudioRecorder::onStop()
{
std::cout << "Stopped!" << std::endl;
}
main.cpp
int main()
{
std::srand(std::time(0));
Client client("127.0.0.1", 4242);
if (client.connect())
{
std::cout << "Connected to server!" << std::endl;
sf::RenderWindow window(sf::VideoMode({ 1280, 720 }), "Multiplayer Game", sf::Style::Default);
window.setVerticalSyncEnabled(true);
sf::View view;
view.setCenter({ 640, 360 });
view.setSize(sf::Vector2f(window.getSize()));
sf::Packet packet;
std::unordered_map<std::uint32_t, Player> players;
std::uint32_t thisPlayer = 0;
StartScreen startscreen(window, sf::Color(172, 192, 146));
AudioRecorder recorder(client);
AudioCapturer capturer(client);
capturer.play();
std::thread receivePacket;
std::thread receiverloop;
std::thread activeness;
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
switch (event.type)
{
case sf::Event::Closed:
window.close();
break;
case sf::Event::Resized:
if (startscreen)
{
view.setCenter({
static_cast<float>(window.getSize().x) / 2,
static_cast<float>(window.getSize().y) / 2
});
view.setSize(sf::Vector2f(window.getSize()));
startscreen.updateOnResize();
}
break;
}
}
if (!receivePacket.joinable())
receivePacket = std::thread(&Client::receivePacket, std::ref(client), std::ref(packet), std::ref(players), std::ref(thisPlayer));
if (!receiverloop.joinable())
receiverloop = std::thread(&AudioCapturer::receiveLoop, std::ref(capturer), std::ref(packet));
if (!activeness.joinable())
activeness = std::thread(&AudioRecorder::activeness, std::ref(recorder));
if (client.receivedPosition)
{
sf::Packet copy = packet;
std::uint8_t command;
if (copy >> command)
{
float x, y;
std::uint32_t id;
copy >> id >> x >> y;
for (auto& player : players)
if (player.first == id)
player.second.setPosition({ x, y });
client.receivedPosition = false;
}
}
if (window.hasFocus())
players[thisPlayer].move(client);
window.clear(sf::Color::White);
window.setView(view);
for (const auto& player : players)
window.draw(player.second);
window.display();
}
client.disconnect();
std::terminate();
}
return 0;
}
I know it's a lot of code and I am sorry that I have to share too much.
I believe the problem is with the AudioCapturer class. For some reason when sending audio data, after a while client that receives it stops the execution and prompts this problem with different addresses:
a.out(60911,0x16b47b000) malloc: Incorrect checksum for freed object 0x12a80a400: probably modified after being freed.
Corrupt value: 0xe7026101a000bcff
By the way I am kind of using the SFML's VoIP example for the AudioCapturer class.
This is the Github page of it: https://github.com/SFML/SFML/blob/master/examples/voip/Server.cpp
I'd appreciate it If someone can help me with that problem.
We're experiencing a problem with asynchronous operation of standalone (non-Boost) Asio 1.10.6 on Linux, which is demonstrated using the following test app:
#define ASIO_STANDALONE
#define ASIO_HEADER_ONLY
#define ASIO_NO_EXCEPTIONS
#define ASIO_NO_TYPEID
#include "asio.hpp"
#include <chrono>
#include <iostream>
#include <list>
#include <map>
#include <thread>
static bool s_freeInboundSocket = false;
static bool s_freeOutboundSocket = false;
class Tester
{
public:
Tester(asio::io_service& i_ioService, unsigned i_n)
: m_inboundStrand(i_ioService)
, m_listener(i_ioService)
, m_outboundStrand(i_ioService)
, m_resolver(i_ioService)
, m_n(i_n)
, m_traceStart(std::chrono::high_resolution_clock::now())
{}
~Tester()
{}
void TraceIn(unsigned i_line)
{
m_inboundTrace.emplace_back(i_line, std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now() - m_traceStart));
}
void AbortIn(unsigned i_line)
{
TraceIn(i_line);
abort();
}
void TraceOut(unsigned i_line)
{
m_outboundTrace.emplace_back(i_line, std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::high_resolution_clock::now() - m_traceStart));
}
void AbortOut(unsigned i_line)
{
TraceOut(i_line);
abort();
}
void DumpTrace(std::map<unsigned, unsigned>& o_counts)
{
std::cout << "## " << m_n << " ##\n";
std::cout << "-- " << m_traceStart.time_since_epoch().count() << "\n";
std::cout << "- in - - out -\n";
auto in = m_inboundTrace.begin();
auto out = m_outboundTrace.begin();
while ((in != m_inboundTrace.end()) || (out != m_outboundTrace.end()))
{
if (in == m_inboundTrace.end())
{
++o_counts[out->first];
std::cout << " " << out->first << " : " << out->second.count() << "\n";
++out;
}
else if (out == m_outboundTrace.end())
{
++o_counts[in->first];
std::cout << in->first << " : " << in->second.count() << "\n";
++in;
}
else if (out->second < in->second)
{
++o_counts[out->first];
std::cout << " " << out->first << " : " << out->second.count() << "\n";
++out;
}
else
{
++o_counts[in->first];
std::cout << in->first << " : " << in->second.count() << "\n";
++in;
}
}
std::cout << std::endl;
}
//////////////
// Inbound
void Listen(uint16_t i_portBase)
{
m_inboundSocket.reset(new asio::ip::tcp::socket(m_inboundStrand.get_io_service()));
asio::error_code ec;
if (m_listener.open(asio::ip::tcp::v4(), ec)
|| m_listener.bind(asio::ip::tcp::endpoint(asio::ip::tcp::v4(), i_portBase+m_n), ec)
|| m_listener.listen(-1, ec))
{
AbortIn(__LINE__); return;
}
TraceIn(__LINE__);
m_listener.async_accept(*m_inboundSocket,
m_inboundStrand.wrap([this](const asio::error_code& i_error)
{
OnInboundAccepted(i_error);
}));
}
void OnInboundAccepted(const asio::error_code& i_error)
{
TraceIn(__LINE__);
if (i_error) { AbortIn(__LINE__); return; }
asio::async_read_until(*m_inboundSocket, m_inboundRxBuf, '\n',
m_inboundStrand.wrap([this](const asio::error_code& i_err, size_t i_nRd)
{
OnInboundReadCompleted(i_err, i_nRd);
}));
}
void OnInboundReadCompleted(const asio::error_code& i_error, size_t i_nRead)
{
TraceIn(__LINE__);
if (i_error.value() != 0) { AbortIn(__LINE__); return; }
if (bool(i_error)) { AbortIn(__LINE__); return; }
if (i_nRead != 4) { AbortIn(__LINE__); return; } // "msg\n"
std::istream is(&m_inboundRxBuf);
std::string s;
if (!std::getline(is, s)) { AbortIn(__LINE__); return; }
if (s != "msg") { AbortIn(__LINE__); return; }
if (m_inboundRxBuf.in_avail() != 0) { AbortIn(__LINE__); return; }
asio::async_read_until(*m_inboundSocket, m_inboundRxBuf, '\n',
m_inboundStrand.wrap([this](const asio::error_code& i_err, size_t i_nRd)
{
OnInboundWaitCompleted(i_err, i_nRd);
}));
}
void OnInboundWaitCompleted(const asio::error_code& i_error, size_t i_nRead)
{
TraceIn(__LINE__);
if (i_error != asio::error::eof) { AbortIn(__LINE__); return; }
if (i_nRead != 0) { AbortIn(__LINE__); return; }
if (s_freeInboundSocket)
{
m_inboundSocket.reset();
}
}
//////////////
// Outbound
void Connect(std::string i_host, uint16_t i_portBase)
{
asio::error_code ec;
auto endpoint = m_resolver.resolve(asio::ip::tcp::resolver::query(i_host, std::to_string(i_portBase+m_n)), ec);
if (ec) { AbortOut(__LINE__); return; }
m_outboundSocket.reset(new asio::ip::tcp::socket(m_outboundStrand.get_io_service()));
TraceOut(__LINE__);
asio::async_connect(*m_outboundSocket, endpoint,
m_outboundStrand.wrap([this](const std::error_code& i_error, const asio::ip::tcp::resolver::iterator& i_ep)
{
OnOutboundConnected(i_error, i_ep);
}));
}
void OnOutboundConnected(const asio::error_code& i_error, const asio::ip::tcp::resolver::iterator& i_endpoint)
{
TraceOut(__LINE__);
if (i_error) { AbortOut(__LINE__); return; }
std::ostream(&m_outboundTxBuf) << "msg" << '\n';
asio::async_write(*m_outboundSocket, m_outboundTxBuf.data(),
m_outboundStrand.wrap([this](const asio::error_code& i_error, size_t i_nWritten)
{
OnOutboundWriteCompleted(i_error, i_nWritten);
}));
}
void OnOutboundWriteCompleted(const asio::error_code& i_error, size_t i_nWritten)
{
TraceOut(__LINE__);
if (i_error) { AbortOut(__LINE__); return; }
if (i_nWritten != 4) { AbortOut(__LINE__); return; } // "msg\n"
TraceOut(__LINE__);
m_outboundSocket->shutdown(asio::socket_base::shutdown_both);
asio::async_read_until(*m_outboundSocket, m_outboundRxBuf, '\n',
m_outboundStrand.wrap([this](const asio::error_code& i_error, size_t i_nRead)
{
OnOutboundWaitCompleted(i_error, i_nRead);
}));
}
void OnOutboundWaitCompleted(const asio::error_code& i_error, size_t i_nRead)
{
TraceOut(__LINE__);
if (i_error != asio::error::eof) { AbortOut(__LINE__); return; }
if (i_nRead != 0) { AbortOut(__LINE__); return; }
if (s_freeOutboundSocket)
{
m_outboundSocket.reset();
}
}
private:
//////////////
// Inbound
asio::io_service::strand m_inboundStrand;
asio::ip::tcp::acceptor m_listener;
std::unique_ptr<asio::ip::tcp::socket> m_inboundSocket;
asio::streambuf m_inboundRxBuf;
asio::streambuf m_inboundTxBuf;
//////////////
// Outbound
asio::io_service::strand m_outboundStrand;
asio::ip::tcp::resolver m_resolver;
std::unique_ptr<asio::ip::tcp::socket> m_outboundSocket;
asio::streambuf m_outboundRxBuf;
asio::streambuf m_outboundTxBuf;
//////////////
// Common
unsigned m_n;
const std::chrono::high_resolution_clock::time_point m_traceStart;
std::vector<std::pair<unsigned, std::chrono::nanoseconds>> m_inboundTrace;
std::vector<std::pair<unsigned, std::chrono::nanoseconds>> m_outboundTrace;
};
static int Usage(int i_ret)
{
std::cout << "[" << i_ret << "]" << "Usage: example <nThreads> <nConnections> <inboundFree> <outboundFree>" << std::endl;
return i_ret;
}
int main(int argc, char* argv[])
{
if (argc < 5)
return Usage(__LINE__);
const unsigned nThreads = unsigned(std::stoul(argv[1]));
if (nThreads == 0)
return Usage(__LINE__);
const unsigned nConnections = unsigned(std::stoul(argv[2]));
if (nConnections == 0)
return Usage(__LINE__);
s_freeInboundSocket = (*argv[3] == 'y');
s_freeOutboundSocket = (*argv[4] == 'y');
const uint16_t listenPortBase = 25000;
const uint16_t connectPortBase = 25000;
const std::string connectHost = "127.0.0.1";
asio::io_service ioService;
std::cout << "Creating." << std::endl;
std::list<Tester> testers;
for (unsigned i = 0; i < nConnections; ++i)
{
testers.emplace_back(ioService, i);
testers.back().Listen(listenPortBase);
testers.back().Connect(connectHost, connectPortBase);
}
std::cout << "Starting." << std::endl;
std::vector<std::thread> threads;
for (unsigned i = 0; i < nThreads; ++i)
{
threads.emplace_back([&]()
{
ioService.run();
});
}
std::cout << "Waiting." << std::endl;
for (auto& thread : threads)
{
thread.join();
}
std::cout << "Stopped." << std::endl;
return 0;
}
void DumpAllTraces(std::list<Tester>& i_testers)
{
std::map<unsigned, unsigned> counts;
for (auto& tester : i_testers)
{
tester.DumpTrace(counts);
}
std::cout << "##############################\n";
for (const auto& count : counts)
{
std::cout << count.first << " : " << count.second << "\n";
}
std::cout << std::endl;
}
#if defined(ASIO_NO_EXCEPTIONS)
namespace asio
{
namespace detail
{
template <typename Exception>
void throw_exception(const Exception& e)
{
abort();
}
} // namespace detail
} // namespace asio
#endif
We compile as follows (the problem only occurs in optimised builds):
g++ -o example -m64 -g -O3 --no-exceptions --no-rtti --std=c++11 -I asio-1.10.6/include -lpthread example.cpp
We're running on Debian Jessie. uname -a reports (Linux <hostname> 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux.
The problem appears under both GCC (g++ (Debian 4.9.2-10) 4.9.2) and Clang (Debian clang version 3.5.0-10 (tags/RELEASE_350/final) (based on LLVM 3.5.0)).
[EDITED TO ADD: It also happens on Debian Stretch Linux <hostname> 4.6.0-1-amd64 #1 SMP Debian 4.6.1-1 (2016-06-06) x86_64 GNU/Linux with g++ (Debian 6.2.1-5) 6.2.1 20161124.]
In summary, the test app does the following:
We create N connections, each consisting of an inbound (listening)
end and an outbound (connecting) end. Each inbound listener is bound
to a unique port (starting at 25000), and each outbound connector
uses a system-selected originating port.
The inbound end performs an async_accept, and on
completion issues an async_read. When the read completes it issues
another async_read that we expect to return eof. When that
completes, we either free the socket immediately, or leave it as-is
(with no pending async operations) to be cleaned up by the relevant
destructors at program exit. (Note that the listener socket is
always left as-is, with no pending accept, until exit.)
The outbound end performs an async_connect, and on completion issues
an async_write. When the write completes it issues a shutdown
(specifically, shutdown(both)) followed by an async_read that we
expect to return eof. On completion, we once again either leave the
socket as-is, with no pending operations, or we free it immediately.
Any error or unexpected receive data results in an immediate abort()
call.
The test app lets us specify the number of worker threads for the
io_service, as well as the total number of connections to create, as
well as flags controlling whether inbound and outbound sockets
respectively are freed or left as-is.
We run the test app repeatedly, specifying 50 threads and 1000
connections.
i.e. while ./example 50 1000 n y >out.txt ; do echo -n . ; done
If we specify that all sockets are left as-is, the test loop runs indefinitely. To avoid muddying the waters with SO_REUSEADDR considerations, we take care that no sockets are in TIME_WAIT state from a previous test run before we start the test, otherwise the listens can fail. But with this caveat satisfied, the test app runs literally hundreds, even thousands of times with no error. Similarly, if we specify that inbound sockets (but NOT outbound sockets) should be explicitly freed, all runs fine.
However, if we specify that outbound sockets should be freed, the app stalls after a variable number of executions - sometimes ten or fewer, sometimes a hundred or more, usually somewhere in between.
Connecting to the stalled process with GDB, we see that the main thread is waiting to join the worker threads, all but one of the worker threads are idle (waiting on an Asio internal condition variable), and that one worker thread is waiting in Asio's call to epoll(). The internal trace instrumentation verifies that some of the sockets are waiting on async operations to complete - sometimes the initial (inbound) accept, sometimes the (inbound) data read, and sometimes the final inbound or outbound reads that normally complete with eof.
In all cases, the other end of the connection has successfully done its bit: if the inbound accept is still pending, we see that the corresponding outbound connect has successfully completed, along with the outbound write; likewise if the inbound data read is pending, the corresponding outbound connect and write have completed; if the inbound EOF read is pending, the outbound shutdown has been performed, and likewise if the outbound EOF read is pending, the inbound EOF read has completed due to the outbound shutdown.
Examining the process's /proc/N/fdinfo shows that the epoll file descriptor is indeed waiting on the file descriptors indicated by the instrumentation.
Most puzzlingly, netstat shows nonzero RecvQ sizes for the waiting sockets - that is, sockets for which there is a read operation pending are shown to have receive data or close events ready to read. This is consistent with our instrumentation, in that it shows that write data has been delivered to the inbound socket, but has not yet been read (or alternatively that the outbound shutdown has issued a FIN to the inbound side, but that the EOF has not yet been 'read').
This leads me to suspect that Asio's epoll bookkeeping - in particular its edge-triggered event management - is getting out of sync somewhere due to a race condition. Clearly this is more than likely due to incorrect operations on my part, but I can't see where the problem would be.
All insights, suggestions, known issues, and pointing-out-glaring-screwups would be greatly appreciated.
[EDITED TO ADD: Using strace to capture kernel calls interferes with execution such that the stall doesn't happen. Using sysdig doesn't have this effect, but it currently doesn't capture the parameters of the epoll_wait and epoll_ctl syscalls. Sigh.]
This appears to have been resolved by the maintainer of ASIO:
See https://github.com/chriskohlhoff/asio/issues/180
and https://github.com/chriskohlhoff/asio/commit/669e6b8b9de1309927b29d8b6be3630cc69c07ac
I am writing a server/client program using linux sockets. I want that both server and client can read and write at the same time with threads.
(The full code I followed can be downloaded at http://www.praxisbuch.net/embedded-linux/downloads.html).
When I start() my thread it throws a infinite loop of 'Bad file descriptor'.
My client.cpp is as follows:
class receiveThread : public Thread {
public:
receiveThread(RawMessage& rawMsg) : data_control(rawMsg) {}
virtual void run() {
cout << "starting recveive thread ..." << endl;
while(1) {
int recvBytes = bt_socket.recieveMessage(&temp_data_control);
try {
cout << "Received new Target Position: " << temp_data_control.data.value1 << endl;
data_control = temp_data_control;
}
catch (SocketException) {
cout << "Received exception " << endl;
sleep(1);
}
}
}
~receiveThread() {
bt_socket.disconnect();
}
private:
BtSocket bt_socket;
RawMessage temp_data_control;
RawMessage& data_control;
};
int main() {
BtSocket BtClient(PORTNUM);
// connect to the server.
BtClient.connectToServer();
RawMessage raw_mes;
receiveThread recv_thread(raw_mes);
recv_thread.start(); //the error occurs
while (1)
{
//do something
}
return 0;
}
thread.cpp is as follows:
void Thread::start() throw(Thread::Exception) {
if (pthread_create(&thread_id, 0, thread_starter, (void *) this) != 0) {
throw Exception("Thread::start: Cannot create thread");
}
}
void* Thread::thread_starter(void* thread_obj) {
Thread* self = reinterpret_cast<Thread*> (thread_obj);
self->setRunning(true);
self->run();
self->setRunning(false);
return 0;
}
In socket.cpp the exception is thrown:
int BtSocket::recieveMessage(AbstractDatagram *abstractDgm)
{
int numBytes;
if ((numBytes = recv(s, abstractDgm->getData(), abstractDgm->getSize(), 0)) < 0)
{
cout << strerror(errno) << endl;
}
return numBytes;
}
I really don't know what the problem is.
Any help would be extremely appreciated!
EDIT: (also from socket.cpp)
Socket::Socket(int pNumber)
{
portNumber = pNumber;
blocking = 1;
if ((s = socket(AF_BLUETOOTH, SOCK_STREAM, BTPROTO_RFCOMM)) == -1)
{
throw SocketException("socket failed (socket())", true);
}
locAddr.rc_family = AF_BLUETOOTH;
str2ba(dest, &locAddr.rc_bdaddr);
locAddr.rc_channel = portNumber;
}
void BtSocket::connectToServer()
{
struct sockaddr_rc serverAddr;
serverAddr.rc_family = AF_BLUETOOTH;
serverAddr.rc_channel = 1;
str2ba(dest, &serverAddr.rc_bdaddr);
if (connect(s, (struct sockaddr *)&serverAddr, sizeof(serverAddr)) == -1)
{
throw SocketException("Connect failed (connect())", true);
}
}
I think I opend the socket correctly!?
Could it have been closed somehow?
I tried yesterday to use std::thread correctly, but it's very dark for me.
My program implementation with pthread works well I don't have any problem with it. I would like to have the same solution with std::thread (if possible).
Solution with pthread:
void *MyShell(void *data) {
std::string str;
while(1) {
std::cin >> str;
std::cout << str << std::endl;
}
}
void mainloop() {
pthread_t thread;
pthread_create(&thread, NULL, aed::map::shell::Shell, this);
...
pthread_cancel(thread);
}
And now the solution which doesn't work everytime, with std::thread:
class ShellThreadInterrupFlag {
public:
void interrupt() {
throw std::string("Thread interruption test\n");
}
};
class ShellThread {
public:
template<typename FunctionType, typename ParamsType>
ShellThread(FunctionType f, ParamsType params) {
std::promise<ShellThreadInterrupFlag *> p[3];
_internal_thread = new std::thread(f, p, params);
_flag = p[0].get_future().get();
_internal_thread->detach();
p[1].set_value(_flag); // tell the thread that we detached it
p[2].get_future().get(); // wait until the thread validates the constructor could end (else p[3] is freed)
}
~ShellThread() {
delete _internal_thread;
}
void interrupt() {
_flag->interrupt();
}
private:
std::thread *_internal_thread;
ShellThreadInterrupFlag *_flag;
};
void Shell(std::promise<ShellThreadInterrupFlag *> promises[3],
aed::map::MapEditor *me)
{
ShellThreadInterrupFlag flag;
promises[0].set_value(&flag); // give the ShellThread instance the flag adress
promises[1].get_future().get(); // wait for detaching
promises[2].set_value(&flag); // tell ShellThread() it is able to finish
while(1) {
std::cin >> str;
std::cout << str << std::endl;
}
}
void mainloop()
{
ShellThread *shell_thread;
shell_thread = new ShellThread(Shell, this);
... // mainloop with opengl for drawing, events gestion etc...
shell_thread->interrupt();
}
Sometimes, when I launch the program, the std::cin >> str is called and the mainloop is blocked.
Does anyone know why the thread is blocking my mainloop ? And how could I avoid this problem ?
I'm accessing a video server with Live555 that is streaming to another program. I want to send rtsp PAUSE and PLAY commands to the video server to stop streaming to any other program. Is this possible? My code does not seem to do anything. I can connect to the server and the server verifies that I have received a full PAUSE command:
VideoServer.h
//must make this store session so we can access the session in the static
//callbacks
class MyRTSPClient: public RTSPClient{
protected:
MyRTSPClient(UsageEnvironment& env, char const* rtspURL,
int verbosityLevel, char const* applicationName, portNumBits tunnelOverHTTPPortNum):
RTSPClient(env, rtspURL, verbosityLevel, applicationName, tunnelOverHTTPPortNum)
{
}
public:
MediaSession* session_;
bool sessionStarted_;
static MyRTSPClient* createNew(UsageEnvironment& env, char const* rtspURL,
int verbosityLevel = 0,
char const* applicationName = NULL,
portNumBits tunnelOverHTTPPortNum = 0)
{
return new MyRTSPClient(env, rtspURL, verbosityLevel, applicationName, tunnelOverHTTPPortNum);
}
};
class VideoServer
{
public:
VideoServer();
private:
TaskScheduler* scheduler_;
UsageEnvironment* env_;
MyRTSPClient* rtspClient_;
char eventLoopWatchVariable;
//Asynchronously start the connection
void StartConnection();
static void callbackDESCRIBE(RTSPClient* rtspClient, int resultCode, char* resultString);
static void callbackPAUSEPLAY(RTSPClient* rtspClient, int resultCode, char* resultString);
};
VideoServer.cpp
VideoServer::VideoServer()
{
eventLoopWatchVariable = 0;
scheduler_ = BasicTaskScheduler::createNew();
env_ = BasicUsageEnvironment::createNew(*scheduler_);
//create rtsp client with default params and our url and environment
rtspClient_ = MyRTSPClient::createNew(*env_,
MINI_HARV_AXIS_RTSP_URL, 4, "jtalon5");
//call description to initialize the session
if (rtspClient_ == NULL) {
std::cout << "Failed to create a RTSP client for URL \"" <<
MINI_HARV_AXIS_RTSP_URL << std::endl;
return;
}
std::cout << "made the client!" << std::endl;
// Next, send a RTSP "DESCRIBE" command, to get a SDP description for the stream.
// Note that this command - like all RTSP commands - is sent asynchronously; we do not block, waiting for a response.
// Instead, the following function call returns immediately, and we handle the RTSP response later, from within the event loop:
rtspClient_->sendDescribeCommand(callbackDESCRIBE);
//start doEventLoop in separate thread so it is not blocking
boost::thread thr1(&MiniHarvAxisInterface::StartConnection, this);
}
void VideoServer::StartConnection()
{
env_->taskScheduler().doEventLoop(&eventLoopWatchVariable);
}
void MiniHarvAxisInterface::callbackDESCRIBE(RTSPClient* rtspClient, int resultCode, char* resultString)
{
std::cout << "describe" << resultString << std::endl;
UsageEnvironment& env = rtspClient->envir();
char* const sdpDescription = resultString;
((MyRTSPClient*)rtspClient)->session_ = MediaSession::createNew(env, sdpDescription);
((MyRTSPClient*)rtspClient)->sessionStarted_ = true;
if(((MyRTSPClient*)rtspClient)->session_ == NULL)
std::cout << "did not create session" << std::endl;
else
std::cout << "created session" << std::endl;
rtspClient->sendPauseCommand(*((MyRTSPClient*)rtspClient)->session_, &callbackPAUSEPLAY);
}
void MiniHarvAxisInterface::callbackPAUSEPLAY(RTSPClient* rtspClient, int resultCode, char* resultString)
{
//do nothing
}
It seems as if I can only pause and play a stream that I create in this process. Is this the case using Live555?
Yes this is possible in Live555.
By default Live555 does nothing unless you redefine the pauseStream function (see comment in pauseStream function in Live555 source below):
// default implementation: do nothing
You must create your own session class and you must redefine the pauseStream function as shown in the example below:
Your media session .h file should look something like:
#include <MediaSubsession.hh>
class YOURMediaSubsession: public MediaSubsession {
... //You can leave this empty if you like
private:
...
virtual void pauseStream(unsigned /*clientSessionId*/,void* /*streamToken*/);
};
Your media session .cpp file should look something like:
#include <YOURMediaSubsession.hh>
void YOURMediaSubsession::pauseStream(unsigned /*clientSessionId*/,
void* /*streamToken*/) {
// default implementation: do nothing
}
The you can add whatever you like in this function, whether it may be stopping/terminating all streams, or getting the encoder to keep encoding and the same frame to give the illusion of having the stream paused for a pre-set amount of time is all up to you.
Note that I can see in your code that you are using a default MediaSession class as shown in your code here:
((MyRTSPClient*)rtspClient)->session_ = MediaSession::createNew(env, sdpDescription);
You will need to construct your own YOURMediaSubsession class based on the MediaSession class to redefine the pauseStream function as shown above then YOU do the pausing, not live555. It should look more like:
((MyRTSPClient*)rtspClient)->session_ = YOURMediaSubsession::createNew(env, sdpDescription);