Let me give you a bigger picture of the problem... I am designing a ROS2-based system with multiple ROS2 nodes each containing a wrapper part (ROS2 layer) and driver/module part where my low-level logic is implemented. The wrapper part is using some ROS2-specific communication mechanisms (topics, services, actions...) to exchange the data/commands between the nodes.
Now, one of the nodes in the system should establish an MQTT connection with the Google Cloud Platform, keep the connectivity alive and allow data exchange between the Cloud and ROS2 system. For that purpose, I am using iot-device-sdk-embedded-c SDK from Google.
It has iotc_connect() blocking function for establishing and keeping connection with the Cloud so the challenge I am facing with is to simultaneously keep the ROS2 node spinning while keeping MQTT connectivity alive.
My idea was to launch a thread from ROS2 wrapper that will be used for establishing/keeping MQTT connectivity and use a callback function as an argument for the thread function that will enable me to forward the data received from the Cloud ithin the thread directly to ROS2 layer. Launching a separate thread for handling connectivity and data exchange would enable my ROS2 node to properly spin and rest synchronized with the rest of the ROS2 system.
ROS2_Wrapper.cpp
thread mqtt_thread(MqttConnHandler::ConnectToMqttServer, &MqttThreadCallback);
mqtt_thread.detach();
...
void MqttThreadCallback(void* data, size_t size){
}
MqttThreadCallback() should be called every time I receive the command/config data from the Cloud.
However, I am not sure how can I fire the callback function within the thread because I have two layers of nested callbacks within the thread:
my_thread.cpp
ConnectToMqttServer(void (*MqttThreadCallback)(void*, size_t)){
...
iotc_connect(...,&OnConnectionStateChanged);
...
}
OnConnectionStateChanged(...){
...
case IOTC_CONNECTION_STATE_OPENED:
iotc_subscribe(...,&iotc_mqttlogic_subscribe_callback,...);
...
}
iotc_mqttlogic_subscribe_callback(...){
//The place where data from the Cloud are received
}
iotc_connect() contains OnConnectionStateChanged() callback from where iotc_subscribe() function is called at the moment connection is established. iotc_subscribe() contains iotc_mqttlogic_subscribe_callback() where data from the Cloud are received.
I am not sure how can I mount the data from iotc_mqttlogic_subscribe_callback() up to the thread caller. Do you have any suggestions? Perhaps using the threads is not the best approach?
Usually C libraries provide an optional additional argument called user_data for this purpose:
extern iotc_state_t iotc_subscribe(iotc_context_handle_t iotc_h,
const char* topic, const iotc_mqtt_qos_t qos,
iotc_user_subscription_callback_t* callback,
void* user_data);
That way you can cast your callback function pointer to void when calling subscribe and catch it as argument in the iotc_mqttlogic_subscribe_callback function call. Where you should recast the data back to the function pointer type and use it.
In addition, you may find yourself in need to pass more data to the callback (mutex to protect the data, loggers from higher level code...). In that case, the best practice is to wrap all this info in a new class of your choice and pass a pointer to the instance in the callback.
Related
I know that you utilize a port to address a process and that you have to use sockets for handling multiple requests on web server, but how does it work? Is the process creating multiple socket threads for each connection? Is threading the answer?
Overview
This is a great question, and one that will take a bit to explain fully. I will step through different parts of this topic below. I personally learned multi-threading in Java, which has quite an extensive concurrency library. Although my examples will be in Java, the concepts will stand between languages.
Is threading valid?
In short, yes this is a perfect use case for multi-threading, although single-threaded is fine for simple scenarios as well. However, there does exist better designs that may yield better performance and safer code. The great thing is there are loads of examples on how to do this on the internet!
Multi-Threading
Lets investigate sample code from this article, seen below.
public class Server
{
public static void main(String[] args) throws IOException
{
// server is listening on port 5056
ServerSocket ss = new ServerSocket(5056);
// running infinite loop for getting
// client request
while (true)
{
Socket s = null;
try
{
// socket object to receive incoming client requests
s = ss.accept();
System.out.println("A new client is connected : " + s);
// obtaining input and out streams
DataInputStream dis = new DataInputStream(s.getInputStream());
DataOutputStream dos = new DataOutputStream(s.getOutputStream());
System.out.println("Assigning new thread for this client");
// create a new thread object
Thread t = new ClientHandler(s, dis, dos);
// Invoking the start() method
t.start();
}
catch (Exception e){
s.close();
e.printStackTrace();
}
}
}
}
The Server code is actually quite basic but still does the job well. Lets step through all the logic seen here:
The Server sets up on Socket 5056
The Server begins its infinite loop
The client blocks on ss.accept() until a client request is received on part 5056
The Server does relatively minimal operations (i.e. System.out logging, set up IO streams)
A Thread is created and assigned to this request
The Thread is started
The loop repeats
The mentality here is that the server acts as a dispatcher. Requests enter the server, and the server allocates workers (Threads) to complete the operations in parallel so that the server can wait for and assist the next, incoming request.
Pros
Simple, readable code
Operations in parallel allows for increased performance with proper synchronization
Cons
The dangers of multi-threading
The creation of threads is quite cumbersome and resource intensive, thus should not be a frequent operation
No re-use of threads
Must manually limit threads
Thread Pool
Lets investigate sample code from this article, seen below.
while(! isStopped()){
Socket clientSocket = null;
try {
clientSocket = this.serverSocket.accept();
} catch (IOException e) {
if(isStopped()) {
System.out.println("Server Stopped.") ;
break;
}
throw new RuntimeException("Error accepting client connection", e);
}
this.threadPool.execute(new WorkerRunnable(clientSocket,"Thread Pooled Server"));
}
Note, I excluded the setup because it is rather similar to the Multi-Threaded example. Lets step through the logic in this example.
The server waits for a request to arrive on its alloted port
The server sends the request to a handler that is given to the ThreadPool to run
The ThreadPool receives Runnable code, allocated a worker, and begin code execution in parallel
The loop repeats
The server again acts as a dispatcher; it listens for the request, receives one, and ships it to a ThreadPool. The ThreadPool abstracts the complex resource management from the developer and executes the code optimized fully. This is very similar to the multi-thread example, but all resource management is packaged into the ThreadPool. The code is reduced further from the above example, and it is much safer for non-multi-threading professionals. Note, the WorkerRunnable is only a Runnable, not a raw Thread, whilst the ClientHandler in the Multi-Thread example was a raw Thread.
Pros
Threads are managed and re-used by the pool
Further simplify code base
Inherits all benefits from the Multi-Threaded example
Cons
There is a learning curve to fully understanding pooling and different configurations of them
Notes
In Java, there exists another implementation called RMI, that attempts to abstract away the network, thus allowing the communication of Client-Server to happen as though it is on one JVM, even if it is on many. Although this as well can use multi-threading, it is another approach to the issue instead of sockets.
We have recently started working on Typescript language for one of the application where a queue'd communication is expected between a server and client/clients.
For achieving the queue'd communication, we are trying to use the ZeroMQ library version 4.6.0 as a npm package: npm install -g zeromq and npm install -g #types/zeromq.
The exact scenario :
The client is going to send thousands of messages to the server over ZeroMQ. The server in-turn will be responding with some acknowledgement message per incoming message from the client. Based on the acknowledgement message, the client will send next message.
ZeroMQ pattern used :
The ROUTER/DEALER pattern (we cannot use any other pattern).
Client side code :
import Zmq = require('zeromq');
let clientSocket : Zmq.Socket;
let messageQueue = [];
export class ZmqCommunicator
{
constructor(connString : string)
{
clientSocket = Zmq.socket('dealer');
clientSocket.connect(connString);
clientSocket.on('message', this.ReceiveMessage);
}
public ReceiveMessage = (msg) => {
var argl = arguments.length,
envelopes = Array.prototype.slice.call(arguments, 0, argl - 1),
payload = arguments[0];
var json = JSON.parse(msg.toString('utf8'));
if(json.type != "error" && json.type =='ack'){
if(messageQueue.length>0){
this.Dispatch(messageQueue.splice(0, 1)[0]);
}
}
public Dispatch(message) {
clientSocket.send(JSON.stringify(message));
}
public SendMessage(msg: Message, isHandshakeMessage : boolean){
// The if condition will be called only once for the first handshake message. For all other messages, the else condition will be called always.
if(isHandshakeMessage == true){
clientSocket.send(JSON.stringify(message));
}
else{
messageQueue.push(msg);
}
}
}
On the server side, we already have a ROUTER socket configured.
The above code is pretty straight forward. The SendMessage() function is essentially getting called for thousands of messages and the code works successfully but with load of memory consumption.
Problem :
Because the behavior of ZeroMQ is asynchronous, the client has to wait on the call back call ReceiveMessage() whenever it has to send a new message to ZeroMQ ROUTER (which is evident from the flow to the method Dispatch).
Based on our limited knowledge with TypeScript and usage of ZeroMQ with TypeScript, the problem is that because default thread running the typescript code (which creates the required 1000+ messages and sends to SendMessage()) continues its execution (creating and sending more messages) after sending the first message (handshake message essentially), unless all the 1000+ messages are created and sent to SendMessage() (which is not sending the data but queuing the data as we want to interpret the acknowledgement message sent by the router socket and only based on the acknowledgement we want to send the next message), the call does not come to the ReceiveMessage() call back method.
It is to say that the call comes to ReceiveMessage() only after the default thread creating and calling SendMessage() is done doing this for 1000+ message and now there is no other task for it to do any further.
Because ZeroMQ does not provide any synchronous mechanism of sending/receiving data using the ROUTER/DEALER, we had to utilize the queue as per the above code using a messageQueue object.
This mechanism will load a huge size messageQueue (with 1000+ messages) in memory and will dequeue only after the default thread gets to the ReceiveMessage() call at the end. The situation will only worsen if say we have 10000+ or even more messages to be sent.
Questions :
We have validated this behavior certainly. So we are sure of the understanding that we have explained above. Is there any gap in our understanding of either/or TypeScript or ZeroMQ usage?
Is there any concept like a blocking queue/limited size array in Typescript which would take limited entries on queue, and block any new additions to the queue until the existing ones are queues (which essentially applies that the default thread pauses its processing till the time the call back ReceiveMessage() is called which will de-queue entries from the queue)?
Is there any synchronous ZeroMQ methodology (We have used it in similar setup for C# where we pool on ZeroMQ and received the data synchronously)?.
Any leads on using multi-threading for such a scenario? Not sure if Typescript supports multi threading to a good extent.
Note : We have searched on many forums and have not got any leads any where. The above description may have multiple questions inside one question (against the rules of stackoverflow forum); but for us all of these questions are interlinked to using ZeroMQ effectively in Typescript.
Looking forward to getting some leads from the community.
Welcome to ZeroMQ
If this is your first read about ZeroMQ, feel free to first take a 5 seconds read - about the main conceptual differences in [ ZeroMQ hierarchy in less than a five seconds ] Section.
1 ) ... Is there any gap in our understanding of either/or TypeScript or ZeroMQ usage ?
Whereas I cannot serve for the TypeScript part, let me mention a few details, that may help you move forwards. While ZeroMQ is principally a broker-less, asynchronous signalling/messaging framework, it has many flavours of use and there are tools to enforce both a synchronous and asynchronous cooperation between the application code and the ZeroMQ Context()-instance, which is the cornerstone of all the services design.
The native API provides means to define, whether a respective call ought block, until a message processing across the Context()-instance's boundary was able to get completed, or, on the very contrary, if a call ought obey the ZMQ_DONTWAIT and asynchronously return the control back to the caller, irrespectively of the operation(s) (in-)completion.
As additional tricks, one may opt to configure ZMQ_SND_HWM + ZMQ_RCV_HWM and other related .setsockopt()-options, so as to meet a specific blocking / silent-dropping behaviours.
Because ZeroMQ does not provide any synchronous mechanism of sending/receiving data
Well, ZeroMQ API does provide means for a synchronous call to .send()/.recv() methods, where the caller is blocked until any feasible message could get delivered into / from a Context()-engine's domain of control.
Obviously, the TypeScript language binding/wrapper is responsible for exposing these native API services to your hands.
3 ) Is there any synchronous ZeroMQ methodology (We have used it in similar setup for C# where we pool on ZeroMQ and received the data synchronously) ?
Yes, there are several such :
- the native API, if not instructed by a ZMQ_DONTWAIT flag, blocks until a message can get served
- the native API provides a Poller()-object, that can .poll(), if given a -1 as a long duration specifier to wait for sought for events, blocking the caller until any such event comes and appears to the Poller()-instance.
Again, the TypeScript language binding/wrapper is responsible for exposing these native API services to your hands.
... Large memory consumption ...
Well, this may signal a poor resources management care. ZeroMQ messages, once got allocated, ought become also free-d, where appropriate. Check your TypeScript code and the TypeScript language binding/wrapper sources, if the resources systematically get disposed off and free-d from memory.
I have a .NET 4.5 WCF client app that uses the async/await pattern to make volumes of calls. My development machine is dual-proc with 8gb RAM (production will be 5 CPU with 8gb RAM at Amazon AWS) . The remote WCF service called by my code uses out and ref parameters on a web method that I need. My code instances a proxy client each time, writes any results to a public ConcurrentDictionary, and then returns null.
I ran Perfmon, watching the thread count on the system, and it goes between 28-30. It takes hours for my client to complete the volumes of calls that are made. Yes, hours. The remote service is backed by a big company, they have many servers to receive my WCF calls, so the more calls I can throw at them, the better.
I think that things are actually still happening synchronously, even though the method that makes the WCF call is decorated with "async" because the proxy method cannot have "await". Is that true?
My code looks like this:
async private void CallMe()
{
Console.WriteLine( DateTime.Now );
var workTasks = this.AnotherConcurrentDict.Select( oneB => GetData( etcetcetc ).Cast<Task>().ToList();
await Task.WhenAll( workTasks );
}
private async Task<WorkingBits> GetData(etcetcetc)
{
var commClient = new RemoteClient();
var cpResponse = new GetPackage();
var responseInfo = commClient.GetData( name, password , ref (cpResponse.aproperty), filterid , out cpResponse.Identifiers);
foreach (var onething in cpResponse.Identifiers)
{
// add to the ConcurrentDictionary
}
return null; // I already wrote to the ConcurrentDictionary so no need to return anything
responseInfo is not awaitable beacuse the WCF call has ref and out parameters.
I was thinking that way to speed this up is not to put async/await in this method, but instead create a wrapper method where I can make things await/async, but I am not that is the smartest/safest way to work it.
What is a smart way to get more outbound calls to the service (expand IO completion thread pool, trick calls into running in the background so Task.WhenAll can complete quicker)?
Thanks for all ideas/samples/pointers. I am hitting a bottleneck somewhere.
1) Make sure you're really calling it asynchronously, rather than just blocking on the calls. Code samples would help here.
2) You may need to do this:
ServicePointManager.DefaultConnectionLimit = 100;
By default it only allows 2 simultaneous connections to the same server.
3) Make sure you dispose the proxy object after the call is complete so you're not tying up resources.
If you're doing things asynchronously the threadpool size shouldn't be a bottleneck. To get a better idea of what kind of problem you're having, you can use Interlocked.Increment and Interlocked.Decrement to track the number of pending calls and see if it's being limited somewhere.
You could also substitute your real call with a call to a very simple method that you know will not have any bottlenecks, to see if the problem is in the client or server.
I am trying solve this problem. I have WCF service. Client can call web method from this service which only "fire" another method (this method only write data to database) in another thread.
Code is here:
//this method will write data to database
public void WriteToDb()
{
}
//this web method will call only mehod WriteToDb() in another thread
public void SomeWebMethod()
{
new Task(WriteToDb).Start();
}
Problem is that in same time can web method call 5 clients. This cause that method WriteToDb is called 5 times in 5 thread.
In all 5 cases method WriteToDb will use same data.
My aim is achieve this behavior. 5 clients called web method SomeWebMethod. Method WriteToDb will run in 5 thread.
But I would like execute first thread, then second thread ....etc and on the end 5th thread.
I don’t want run method WriteToDb in same time in 5 thread.
So maybe I can use lock.
{
private object locker = new object();
//this method will write data to database
public void WriteToDb()
{
lock(locker)
{
//write to DB
}
}
I am not sure because .net assembly is host on app domain a app domain is host on win process. I woud like to avoid deadlocks.
What happens if I have a machine with 6 CPU? Use mutex instead lock ?
Thank you for help...
I'm not particulary sure what you are writing to DB, but your question is loosely coupled with WCF to be frank, try to read CLR via C# on multithreading etc.
Also regarding WCF, you can setup how your service object is created upon requests, ie per call, per session or singleton, and for later use specify if it's methods will stuck in queue or will be called on object concurrently.
So depending on choosing architecture you can either relay on WCF ability to host single object which will have logic you described or you can go the way tried.
Links
http://msdn.microsoft.com/en-us/magazine/cc163590.aspx
http://msdn.microsoft.com/en-us/library/ms731193.aspx
A lock is fine here, but you should make your locker object static so the same object instance is used in the lock every time.
It does not matter how many cores you have - if you hold the lock on an object then any other threads that attempt to acquire the lock will wait until the lock is released.
A deadlock can only occur if you are acquiring multiple locks in different orders in different threads.
I suggest you read Joe Albahari's excellent free ebook
I am writing a web service which has to be able to reply to multiple http requests.
From what I understand, I will need to deal with HttpListener.
What is the best method to receive a http request(or better, multiple http requests), translate it and send the results back to the caller? How safe is to use HttpListeners on threads?
Thanks
You typically set up a main thread that accepts connections and passes the request to be handled by either a new thread or a free thread in a thread pool. I'd say you're on the right track though.
You're looking for something similar to:
while (boolProcessRequests)
{
HttpListenerContext context = null;
// this line blocks until a new request arrives
context = listener.GetContext();
Thread T = new Thread((new YourRequestProcessorClass(context)).ExecuteRequest);
T.Start();
}
Edit Detailed Description If you don't have access to a web-server and need to roll your own web-service, you would use the following structure:
One main thread that accepts connections/requests and as soon as they arrive, it passes the connection to a free threat to process. Sort of like the Hostess at a restaurant that passes you to a Waiter/Waitress who will process your request.
In this case, the Hostess (main thread) has a loop:
- Wait at the door for new arrivals
- Find a free table and seat the patrons there and call the waiter to process the request.
- Go back to the door and wait.
In the code above, the requests are packaged inside the HttpListernContext object. Once they arrive, the main thread creates a new thread and a new RequestProcessor class that is initialized with the request data (context). The RequsetProcessor then uses the Response object inside the context object to respond to the request. Obviously you need to create the YourRequestProcessorClass and function like ExecuteRequest to be run by the thread.
I'm not sure what platform you're on, but you can see a .Net example for threading here and for httplistener here.