Right now I'm using rabbitMQ to send data between two programs - 1 queue, 1 channel, 1 exchange. I'm going to extend it to be multithreaded and I want to declare another queue on the second thread.
I understand in this case I should use another channel, but what I would like to know is is it necessary to declare another exchange with a different name as well?
What exactly is the relationship between the two?
In what kind of situation would you need multiple exchanges?
As you figured out, the channel is the communication end point used to reach a rabbitMQ object.
There are so far 2 kinds of objects:
Queues, which are simply buffers for messages,
Exchanges, which are broadcasting devices.
As a simple analogy, queues are the pipes in which messages get accumulated, and exchanges are the T shaped, cross shaped and other types of connectors between pipes.
Perhaps it works better to compare it with a physical network, where queues would be like cables, and exchanges like switches or smart hubs, for which different distribution strategies can be chosen.
So basically, what you need to do is simply create a new queue from the new consumer thread, have it connect to the exchange object (for which you should have a name), and let the producer thread send its messages to the exchange object exclusively. Any new thread may follow the same protocol.
The last important point is to pick the correct distribution method for the exchange object, round-robin, fanout, etc. depending on how you wish your consumers to receive messages.
Take a look at our introduction to AMQP concepts
http://www.rabbitmq.com/tutorials/amqp-concepts.html
Related
I am trying to model an application which runs multiple concurrent flows.
In this situation multiple threads can create events and store them in a buffer which are then collected and displayed by another thread. The receiving thread is supposed to block and wait for incoming events.
I have currently modelled it like this:
This example uses object flows. However I am not sure if this is the correct way to model this type of inter thread communication.
The other option I was looking at is using signals but I'm not sure about that either.
Any help would be appreciated.
Every activity requires all tokens to be offered before it can start. You will have to use a buffer node as a queue.
Object flows capture inter thread communication well.
You could also use signals, if you want to be more specific and your system uses in fact messages.
There is one problem in your diagram though: The Display Event action consumes all offered control tokens and one object token on each invocation. I can‘t tell from your diagram, but probably there is only one control token. That means, the action will only run once. The solution is, to delete the control flow. The action then starts for each incoming object token.
Each output pin acts as a local buffer. If tokens are generated faster than the event can be displayed, tokens will potentially pile up in multiple pins. In this case it is undefined which pin will be the source of the next token. This is not necessarily a problem, but if tokens shall be processed in chronological order, you need to use a central buffer. The symbol is a rectangle with the keyword «central buffer»
I need to send and receive multiple values between 2 tasks. I am currently using tokio oneshot channel because I am only dealing with 2 tasks. But I can't seem to re-use the tx probably due to its one message limit. How is this situation handled usually?
Do I create a new oneshot channel everytime? or is there a way to reuse the channel?
Do I try packing all my interactions into 1 message and just do it once? -> seems very restrictive.
Do I use other channel types?
The tokio::sync::mpsc channel should be used in this situation.
I suspect you're worried by the fact that it supports multiple senders, but its perfectly fine to use it when you only have one sender.
The code should be written in C++. I'm mentioning this just in case someone will suggest a solution that won't work efficient when implementing in C++.
Objective:
Producer that runs on thread t1 inserts images to Consumer that runs on thread t2. The consumer has a list of clients that he should send the images to at various intervals. E.g. client1 requires images every 1sec, client2 requires images every 5sec and etc.
Suggested implementation:
There is one main queue imagesQ in Consumer to which Producer enqueues images to. In addition to the main queue, the Consumer manages a list of vector of queues clientImageQs of size as number of clients. The Consumer creates a sub-consumer, which runs on its own thread, for each client. Each such sub-consumer dequeues the images from a relevant queue from clientImageQs and sends images to its client at its interval.
Every time a new image arrives to imagesQ, the Consumer duplicates it and enqueus to each queue in clientImageQs. Thus, each sub-consumer will be able to send the images to its client at its own frequency.
Potential problem and solution:
If Producer enqueues images at much higher rate than one of the sub-consumers dequeues, the queue will explode. But, the Consumer can check the size of the queue in clientImageQs before enqueuing. And, if needed, Consumer will dequeue a few old images before enqueuing new ones.
Question
Is this a good design or there is a better one?
You describe the problem within a set of already determined solution limitations. Your description is complex, confusing, and I dare say, confused.
Why have a consumer that only distributes images out of a shared buffer? Why not allow each "client" as you call it read from the buffer as it needs to?
Why not implement the shared buffer as a single-image buffer. The producer writes at its rate. The clients perform non-destructive reads of the buffer at their own rate. Each client is ensured to read the most recent image in the buffer whenever the client reads the buffer. The producer simply over-writes the buffer with each write.
A multi-element queue offers no benefit in this application. In fact, as you have described, it greatly complicates the solution.
See http://sworthodoxy.blogspot.com/2015/05/shared-resource-design-patterns.html Look for the heading "unconditional buffer".
The examples in the posting listed above are all implemented using Ada, but the concepts related to concurrent design patterns are applicable to all programming languages supporting concurrency.
I need to delay each message I produce with a specific time.
As far as I know the rabbitmq-delayed-message-exchange plugin allows me to do exactly that, however I was warned that it doesn't scale properly which is a definite requirement. (Has there been any updates lately fixing scaling problems?)
So, the alternative was to use TTL and a DLQ. With this approach though, you set the time when creating the exchange instead of the actual message which means I wouldn't be able to set different times for different messages.
Did I miss something?
My use case: Basicly I will be receiving specific "appointments" from clients which I must store and send back to the client at a specific time supplied in the appointment object. I want to acheive this by specifying a delay on each message so that my consumers must not implement waiting logic.
Why don't you use a per-queue message TTL, and have different queues for each different TTL you want to set, originally publish the messages through direct exchange with key related to the specific TTL?
Then having configured the same dead letter exchange for all those queues, they'll end up in the "final" queue for your consumers with the desired delay.
Of course it wouldn't be great if the possible values for the delays were too numerous.
Scala Actor : Will messages passed from one actor to another within the same process always be processed in the original order as sent ?
Yep. For two actors this condition does hold.
the messages are guaranteed to be ordered for a given pair of sender
and receiver actors. If an actor A sends messages X and Y in that
order, the actor B will receive no messages, only the message X, only
the message Y, or the message X, followed by the message Y.
Learning concurrent programming in scala, page 270
Akka guarantees that messages will be received in the same order that they were sent assuming they are delivered at all, but does not guarantee message delivery. The akka documentation is quite clear on this as follows:
"for a given pair of actors, messages sent directly from the first to the second will not be received out-of-order"
Also, while messages sent via an intermediate actor are normally not guaranteed to be delivered in the same order to the final destination actor, this limitation is "eliminated under certain conditions" when running in the context of a single JVM. As you are asking specifically about in-process messaging this may be relevant to you although the documentation advises not to rely on this behaviour.
There are examples and further clarification in the Akka documentation here: http://doc.akka.io/docs/akka/snapshot/general/message-delivery-reliability.html#The_Rules_for_In-JVM__Local__Message_Sends