Perhaps a silly question, but keep reading about SIs "lightweight messaging within Spring-based applications". I want to know how (if) SI uses messaging internally. When I run an SI (Boot) application (one that doesn't require AMPQ ... aka 'messaging' support), I don't have to run a Rabbit server. But, from what I gather, SI uses messaging internally. How is this accomplished? I can't seem to find any reference explaining this & what infrastructure is required to make this possible. Thanks!
The messages are simply Java objects (o.s.messaging.Message) passed between components. No external broker is needed, unless you need persistence.
I suggest you read Mark Fisher's book (Spring Integration in Action) and/or the reference manual.
The messaging inside spring integration are in-memory java objects passed from one service to another via channels/queue. It provides a mechanism to define the flow and order of processing, also allowing each service step to work in isolation. The spring integration queue is eventually an implementation of java.util.Queue interface.
It is different from commercial Messaging tools like IBM MQ or Active MQ as it doesnt offer persistence. Which means if you kill the jvm or the app process is stopped, all the messages in flight on the Spring queue/channel are lost. A lot if times this is acceptable if the process in idempotent, i.e When the application comes up, I can restart the process from beginning.
Related
I have developed a node sdk which has certain REST API.These API's are interacting with blockhchain framework for read and write operations.
There could be certain situations when many requests are coming on node sdk.
So for load balacing i have used NGNIX with having one more replica of sdk on another instance.This all works well.
It is being suggested to use rabbitMQ for load balancing as well. But in my API there are few straightforwards read and write operations by API & no heavy processing done.
I read rabbitMQ should be used for below purpose.
Integrating multiple microservices
Executing heavy task such as image processing,image uploading etc.
So how and when should i use rabbitMQ ?
I think your design is OK. Simply, your system had to manage more load and you added more replicas of your services, with a load balancer on the front that is able to distribute incoming load between the replicas. If your "sdk" is purely stateless (doesn't remeber client data collected from previous requests, but delegates all state to a DB/BC) your've done your job. A message queuing technology can help in other scenarios
when your application does things in a pure asynchronous fashion
when you have to manage big spikes of load
when some of your architecture component reacts to events (eg. receiving an alarm from a device, sending an email when your become the 1 million click etc)
when you're into event sourcing
when in some way there are stateful services that consume data from the same batch of requests (eg all data from user with id 1sw023)
various and possible
Adopting MQs has a big impact and needs some effort to integrate e manage things. Don't do it if you are not sure to leverage completely its benefits
RabbitMQ is a Message Queue. It's useful when your application is receiving more requests that what it can handle simultaneously.
The way it works is that the queue store the incoming messages until they are processed by worker nodes (for example your SDK). The worker nodes typically do some work (usually heavy processing), and when they are done with the work, they pull a new message from the queue, process it, do the work, and so on so forth.
In your case, you might need it if you see that your blockchain is rejecting a lot of messages (for example because there was too much request at once, and the blockchain couldn't reach a consensus quick enough).
The documentation of Service Fabric recommends service remoting, ICommunicationClient or WcfCommunicationClient to realize the communication between the micro services.
The ServiceBus, which I always used for inter-service communication, is not even mentioned. Why?
I think you misinterpreted the docs. It does not recommend any protocol or service (the word is not even present on the page). What it does do is list the built-in communication options and appropriate situations of when to use them.
There is nothing that prevent you from using service bus for inter service communications. In fact, if you google around you will find some projects like this one
The ability to plug in any desired service or protocol is one of the great things about SF, but they leave the implementation to you.
There are many approaches to do service to service communication, if they had to document all of then, they would spend more time writing the possible approaches than doing the actual communication.
They probably decided for the one with closest relation to the platform, but they could write about any possible, it is just a matter o preference.
I could name a few from many just to have an idea:
Http
Remoting
WCF
Service Bus
Event Hub
AMQP
MQTT
gRPC + protobuf
TCP
UDP
Pipes
And many more, Imagine if they had to document all of then.
The communication is flexible enough to let you implement using any communication mechanism.
Regarding the ones you mentioned,
I always opt for HTTP for being platform agnostic and widely implemented on most platforms, does not matter if is .Net, Java, NodeJs, Windows or Linux, they all talk the same language, the others are very tight to the .Net and Windows platform and force every other solution to be also tighten or adapted to then. And also there is the fact of some being synchronous and other asynchronous like Service bus.
Then, when performance is an issue, I evaluate the other options.
I have too many emails. I should write scheduler in order to send messages to them. Messages are different. I use spring framework 4.x.
I can write simple class, which connects to SMTP server. But in this case I should write my thread library too in order to send emails parallel.
Do spring have already written library which give me more flexible way to do this tasks? I do not want to use threads. It will be nice if spring already have this functionality.
Do I need Spring integration for this?
Best regards,
Yes, you definitely can do that with Spring Integration, because there is an ExecutorChannel implementation with can be supplied with an TaskExecutor from the Spring Core:
<channel id="sendEmailChannel">
<dispatcher task-executor="threadPoolTaskExecutor"/>
</channel>
<int-mail:outbound-channel-adapter channel="sendEmailChannel" mail-sender="mailSender"/>
But anyway you should keep in mind that all Spring Integration components are based on the Java and that ExecutorService is used on the background.
From other side if you need only the mail sending stuff from the Spring Integration, it would be an overhead and can simply use Core Spring Framework legacy like JavaMailSender as a bean and #Async for the sendMail method to achieve your parallel requirement.
UPDATE
could you tell me whether I need JMS for this situation?
I don't see any JMS-related stuff here. You don't have (or at least don't show) any real integration points in your solution. The same I can say even about Spring Integration just for email sending. However with the Spring Boot your SI config will be enough short. From other side if you'll study Spring Integration better eventually you'll get more gain to rely on the Integration components for your systems, as internally, as well as externally with other systems through JMS, AMQP, Kafka etc.
To be honest: a lot of years ago my first acquaintance with Spring Integration was due the requirement to get files from the FTP and have ability to pick up new files automatically. I found the solution only in the Spring Integration 1.0.0.M1. After that short XML config for the <int-ftp:inbound-channel-adapter> I loved Spring Integration and since that time it became as a part of my life. :-)
So, it's up to you to go ahead with Spring Integration in your simple app, or just follow with more formal solution with JavaMailSender direct usage.
You should use java executors framework. For example you can write something like the code below:
ExecutorService executor = Executors.newWorkStealingPool();
executor.execute(() -> mailSender.send(mail));
I'm new to RabbitMQ and MQ's in general. I'm using the rabbit.js Node.js module to interface with RabbitMQ, so all my application layer is going to mainly be in Node.js. What I'm wondering is, how do I manage RabbitMQ? How can I see everything that's going on with RabbitMQ, from what's messages are left in the queue to general configuration and administration?
I'm looking for something visual, but more importantly, easy to use and simple.
RabbitMQ has a web interface (part of the rabbitmq_management plugin which ships with RabbitMQ, but needs to be enabled) that allows you to see the servers, exchanges, queues, etc.
It's pretty easy to use. One thing I would recommend is to set the time-interval on the graphs to 10 minutes. I find if you set them to longer, say, an hour plus, the information (due to the way it's bucketed, I think) gets a bit wonky.
Check out this link for more info: https://www.rabbitmq.com/management.html
There is also a JSON API that can be used to programmatically determine, for example, how many items are in a particular queue.
There's also a cmdline tool, called rabbitmqadmin (https://www.rabbitmq.com/management-cli.html) which can come in really handy for things like setting up test RabbitMQ test environments via a bash script and things of that nature.
check JXM.io sources (open source messaging backend for node.js / JXcore) that uses RabbitMQ for multi server integration and there is a nice article showing how to cluster RabbitMQ http://jxm.io/multi-server-messaging-backend-installation/
we are building a distributed Java system (should be scalable ;-) ) that is connected only with JMS (ActiveMQ). I studied Spring Integration and I am not sure what the advantage would be if we use it. I think we are better off with using the JMS-Templeate from the Spring Core Project as we send only messages from a JavaService to another JavaService and so on.
Use Spring Integration (or any other framework) if you think the extra abstraction that it buys you is worth the cost. It should give you a more solid foundation on which to build your application. Software written by Spring is better than anything you or I would write from scratch.
All frameworks have a cost. There are additional dependencies. Sometimes greater abstraction can obscure too much.
You should prototype with and without Spring Integration to see if it's worth the cost.
If you're not a Spring user already, I'd recommend that you learn Spring before jumping into a big enterprise project.
SpringIntegration will give your Enterprise Integration Pattens ready to use.
Are you going to need splitters, routers, filters, gateways, aggregators, transformers, etc?
If the answer is no, go for plain Java+ActiveMQ.
If you need a really powerful system integration tier, then you should use Spring Integration - it's an additional level of abstraction which may help when your system will grow. With SI it's a matter of seconds to add new integration processing rule between two systems.
From the other hand, I've worked on some SpringIntegration+ActiveMQ project, and it was almost impossible to configure this broker to work with SI in the reliable way. So if you decide to use SI I'd recommend HornetQ as a JMS broker - this one works fine.