i am using the node.js as client to jms topic.There are two protocols available to make the connection on Topic.
Theses are Stomp and AMQP. I am read brief about them at http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol
and http://en.wikipedia.org/wiki/Streaming_Text_Oriented_Messaging_Protocol. Both seems to be wire level protocol i.e
data that is sent across the network as a stream of octets.I do not find any
concrete reason which one should be preferred. If someone can shed light on it, it would be helpful.
Another point is both the protocol takes pride in stating them as interoprable. Does the interoperable term means that if someone
want to take off specific message broker implementation say apache active MQ and instead want to plugin Websphere MQ , transition
will be smooth(provided both support AMQP/STOMP or any other wire levl protocol)?
You may see a difference in performance (refer to this benchmark based upon many factors including message size and persistence requirements for the queue entries.
As is often the case, there are other factors to consider as well, especially if your message size/count/etc. doesn't mean there's a clear winner in terms of performance and no one protocol meets your functional requirements in a way the other does not.
This article in particular hints there may be more fragmentation in the different STOMP broker implementations. Quoting from that article
STOMP...uses a SEND semantic with a “destination” string. The broker must map onto something that it understands internally such as a topic, queue, or exchange. Consumers then SUBSCRIBE to those destinations. Since those destinations are not mandated in the specification, different brokers may support different flavours of destination. So, it’s not always straightforward to port code between brokers.
At least with AMQP (which touts interoperability as one of its most important advantages) the only issues you should have with switching providers/languages are those inherent in setting up said new providers. For example, I've read ZeroMQ is likely to take more configuration work on your part than RabbitMQ, but that's not really due to any attributes specific to AMQP.
Related
I have been struggling to find any key pros and cons on using one over the other. When it comes to sharing data between two microservices. Especially when it comes to scale.
What my assumption and question is - if we use a CDC to queue & CDC (Queue) subscriber combination, we can more or less can get rid of the need to publish to the message queue from our application layer (which might be prone to more human errors).
I went into this thought process when evaluating Mongodb "changestreams" and have been curious ever since.
When using CDC in this way, you're basically turning your microservice's database into a message broker. That has the advantage of not requiring a separate message broker. It has the disadvantages of deeply coupling the consuming microservices to the producing microservice, especially since every new consuming microservice will effectively impose some extra load on the source microservice's database.
CDC can be a reliable way to feed a pubsub topic on a message broker, however, though it's probably best to recognize that the CDC still means a coupling between the source microservice's internal data model and the data model for interservice communication, which tends to mean changes to one require changes to all. Since one of the primary (and arguably the only always-valid-in-general) reasons to adopt microservices is to allow changes with minimal coordination, it might be advised to have the CDC feed a single service which is responsible for translating the CDC records into the wire model (e.g. domain events with an agreed upon schema).
Is it right way to use ZeroMQ as publisher and RabbitMQ as subscriber in software architecture?
Publisher uses ZeroMQ as message broker and my Node.js application needs to subscribe to a topic using RabbitMQ ( as RabbitMQ node package doesn't need prerequisites as ZeroMQ requires python & .NET framework to be installed ).
And also, Publisher uses different ZeroMQ library, other than the one required for Node.js.
Speaking about Architecture?Prologue:
A first note to be made is, ZeroMQ is a broker-less messaging/signalling framework. If your architecture needs a Broker, you either have to pick from other COTS shelf, or implement a custom-specific Broker-factory on your own.
The Tower of Babel:
Deployment pre-requisities are not a sufficient argument to stop thinking or to sacrifice an elementary logic of operations.
If one considers a proprietary protocol A sender to be a-priori interconnectable with a proprietary protocol B receiver to be a fair assumption, there is hard to help or explain the root cause of the trouble.
While ZeroMQ documentation includes a published set of ZMQ-Protocol-Proposals from their very origin in their respective RFC-states, there is no expressed warranty, AFAIK, the other "end-of-the-phone-line" understands and fully cooperates within the ZeroMQ specification with the "remote-caller", unless a mutually agreed compatibility is assured by the protocol design owners ( which would be well popularised, in case one has undertaken such efforts, which is hardly to become a reality due to lack of any business-motivated reasons to develop a product just to declare a compatibility with another protocol engine, which is for about a decade well adopted in FOSS domain ).
Sorry, this would never fly.
How to repair the flawed architecture?
The best one can do is to seek for a minimum common multiple - getting all heterogeneous systems capable to use the same messaging/signalling framework ( sharing the guaranteed cross-compatible protocol suite, different versions may and do appear in such designs due to external constraints ).
Candidates with light-weight resources requirements' footprints, almost linearly scaleable, ultimate speed / low latency and minimum pre-requisities are usually preferred:
ZeroMQ
nanomsg
would be the first I would test the architecture to have 'em system-wide deployed.
Good Hunt!
I want to use MQTT as a protocol communication with RabbitMQ Message Broker, but from rabbitmq website I found this paragraph:
These implementations are suitable for development but sometimes won't be for production needs. MQTT 3.1 specification does not define consistency or replication requirements for retained message stores, therefore RabbitMQ allows for custom ones to meet the consistency and availability needs of a particular environment. For example, stores based on Riak and Cassandra would be suitable for most production environments as those data stores provide tunable consistency.
https://www.rabbitmq.com/mqtt.html
So, from this paragraph, I should to use Cassandra as a database for RabbitMQ, but I didn't find anything about integration Cassandra as a database for rabbitmq.
can you help me by giving me something to make it possible.
NB:I'm newbie in RabbitMQ.
The paragraph refers specifically to "retained messages" part of MQTT spec, as in, the messages you want to keep for a long time. Like a "last know configuration", that you may want to apply to any MQTT subscriber, regardless whether or not it has been online and subscribed at the moment the message is published.
It's a very particular situation and unless you need that feature you don't have to worry about using RabbitMQ as MQTT broker. For regular messages built-in RabbitMQ replication options are perfectly suitable and production-ready.
Until now, RabbitMQ doesn't support this feature.
so, it's not possible to use another database instead of Mnseia database
Say you have microservice A,B, and C which all currently communicate through HTTP. Say service A sends a request to service B which results in a response. The data returned in that response must then be sent to service C for some processing before finally being returned to service A. Service A can now display the results on the web page.
I know that latency is an inherent issue with implementing a microservice architecture, and I was wondering what are some common ways of reducing this latency?
Also, I have been doing some reading on how Apache Thrift and RPC's can help with this. Can anyone elaborate on that as well?
Also, I have been doing some reading on how Apache Thrift and RPC's can help with this. Can anyone elaborate on that as well?
The goal of an RPC framework like Apache Thrift is
to significantly reduce the manual programming overhead
to provide efficient serialization and transport mechanisms
across all kinds of programming languages and platforms
In other words, this allows you to send your data as a very compactly written and compressed packet over the wire, while most of the efforts required to achieve this are provided by the framework.
Apache Thrift provides you with a pluggable transport/protocol stack that can quickly be adapted by plugging in different
transports (Sockets, HTTP, pipes, streams, ...)
protocols (binary, compact, JSON, ...)
layers (framed, multiplex, gzip, ...)
Additionally, depending on the target language, you get some infrastructure for the server-side end, such as TNonBlocking or ThreadPool servers, etc.
So coming back to your initial question, such a framework can help to make communication easier and more efficient. But it cannot magically remove latency from other parts of the OSI stack.
I have a multithreaded program that processes images and I want to make a queue in the program for inserting requests from an external program in this queue and the image processor's threads dequeue the requests from it. Now what is the best solution for when the queue is full and new requests arrive?Should I drop new or old or random requests?
That is totally up to you and how you want your program to behave. See what the outcome of each decision might lead and decide which is the lesser evil.
I recommend using a message broker for this approach.
You can send the images as messages to this broker which will handle the routing to the target program, the complete queuing system as well as the "full queue" problem.
I personally have made some good experience with RabbitMQ, although it might be a little overhead for your special purpose. You may have a look at ZeroMQ as well as it might be a little thinner and a better approach for you.
To do things right, determine what you really need and check out if those message broker are really useful in your current situation - from my point of view and in my opinion they are, but depends on your exact requirements and implementation.
If you're interested look at AMQP, the Advanced Messaging Queuing Protocol itself - it's the basic for all those message brokers and quite interesting.