I'm not too familiar with ESB or Biztalk and am trying to get my head around what makes the most sense from an EAI perspective if you already own Biztalk. As I understand Biztalk is a message broker(hub and spoke) and the ESB pattern is an anti-broker where the conceptual "bus" is made of individual distributed components that talk to each other somehow.
Message brokers inherently represent a single point of failure as opposed to an ESB where one component failing does not bring down the "bus" as a whole. Also my understanding is that Biztalk is monolithic in the sense that messaging, orchestrations are tightly coupled and scaling is problematic.
If the scenario at hand is:
Biztalk is already being used primarily for running different orchestrations based on different files received from outside parties.
A bunch of inhouse custom appications which are currently tightly coupled to system such as CRM and payroll need to be refactored to abstract out these dependencies.
Would it make sense to use Biztalk directly or Biztalk ESB toolkit for achieving ESB functionality or would it make sense to use a proper ESB implementation such as NServiceBus or Service Bus for Windows which is based on Azure Service Bus.
What are the pros and cons of using Biztalk directly to achieve EAI vs. using a proper ESB.
Would each application take a hard dependency on Biztalk and whether this is desirable?
I will leave this as an open ended discussion since there is no right or wrong answer.
#StuartLC:
Thanks for your response. I've read several of the links you posted but still did not get a clear sense of whether Biztalk would make sense as an ESB solution vs. using something like NServiceBus. Both seem to implement the "ESB" pattern in one way or another. Question being which one has a cleaner implementation, better development experience and low ramp up time. My assessment so far(from pure research only) is that yes Biztalk can be used but it is painful and requires very specialized dev. skill sets. Latency and scaling being problematic and the fact that Biztalk will
eventually be assimilated into the (Azure?) Service Bus and the Biztalk SKU will cease to exist.
On the other hand, a framework like NService bus has relatively low ramp up time, can be easily picked up by a dev. having good .NET programming skills in general and can interface easily with Biztalk.
Given the above, would it still make sense to go the Biztalk route even though you currently have Biztalk in house or to future proof yourself go with a proper ESB such as NService Bus ?
I believe many components of your open-ended question have already been covered on SO:
Pros/Cons of using BizTalk instead of NServiceBus or MassTransit
What ESB systems work best for the .Net stack?
Microsoft BizTalk Server vs Azure AppFabric Service Bus...?
BizTalk 2009 ESB Confusion
Is BizTalk an ESB?
However, IMO it is just a flawed / short-sighted implementation which which would result in tight coupling between apps and endpoints. Loose coupling is easily achieved : (even without the ESB toolkit):
By using multipart messages and canonical internal schemas for decoupling dependencies.
Orchestrations and ports should be decoupled by publishing via the message box
Single point-of-failure is also avoidable:
Configuring Retries and Alternative / backup transports on communication adapters
Redundancy achieved e.g. via Server Groups and Clustering
And fall back using compensation for failed delivery
IMO the Achilles heel when using BizTalk as an ESB is the lack of guaranteed latency e.g. here which is exarcebated if BTS gets into a throttling state.
Update
IMO the choice boils down to a matter of whether you have control over all systems in your environment.
If you are integrating an internal enterprise consisting only of homogeneous, contemporary (and mostly SOA and EDA) applications over which you have direct control, MassTransit or NServiceBus would likely more than do the job give and you better productivity and ramp up time.
Related
I am trying to understand the difference of SOA and microservices archtitecture and some terms are so abstract to me, so it is hard for me to understand.
I read documents, generally mentioning about;
Soa is enterprise scope
Microservice is application scope
I can not understand this concepts, what is the difference of enterrise scope and application scope?
is that when dividing applications into multi services ? in both architectures applications are divided into multiple services and pieces are communicating by similar technology.
if difference is about how to divide application into multiple services and in soa it should be divided by busineess concepts, I dont think that there is rule in microservices architecture about not dividing by business concepts about the size of each service.
another abstract thing that I met:
This integration must occur by using service interfaces in soa- that's why it is Service-oriented.
what does that mean ? and how it is different from microservices integration?
Thanks for any ideas,answers
It's been a long since the question was asked so not sure if the insights below are relevant to you anymore.
Enterprise scope with respect to SOA usually refers to Service enablement of existing applications in an enterprise and unlocking their potential/investment while creating an overarching Business Process. SOA is very tightly bonded to the Business process and in simple terms can be assumed to be the actual implementation of an end-to-end process such as Sales Order Processing, Loan Application approval. Keep in mind that to create such a process, the SOA process/application would need to integrate with heterogeneous applications/systems/databases such as ERP, CRM, RDBMS, File systems, etc. In that sense, SOA architecture is not a pure app development architecture. Another aspect of SOA that you already mentioned is Service interfaces which can range from SOAP, JMS, and all other kinds of protocols
Microservices on the other hand are more aligned towards hardcore application development and breaking applications into granular and independent components (many times, even written in different programming languages). But most often the primary integration mechanism is lightweight REST API between the microservices.
Hope it helps.
The documentation of Service Fabric recommends service remoting, ICommunicationClient or WcfCommunicationClient to realize the communication between the micro services.
The ServiceBus, which I always used for inter-service communication, is not even mentioned. Why?
I think you misinterpreted the docs. It does not recommend any protocol or service (the word is not even present on the page). What it does do is list the built-in communication options and appropriate situations of when to use them.
There is nothing that prevent you from using service bus for inter service communications. In fact, if you google around you will find some projects like this one
The ability to plug in any desired service or protocol is one of the great things about SF, but they leave the implementation to you.
There are many approaches to do service to service communication, if they had to document all of then, they would spend more time writing the possible approaches than doing the actual communication.
They probably decided for the one with closest relation to the platform, but they could write about any possible, it is just a matter o preference.
I could name a few from many just to have an idea:
Http
Remoting
WCF
Service Bus
Event Hub
AMQP
MQTT
gRPC + protobuf
TCP
UDP
Pipes
And many more, Imagine if they had to document all of then.
The communication is flexible enough to let you implement using any communication mechanism.
Regarding the ones you mentioned,
I always opt for HTTP for being platform agnostic and widely implemented on most platforms, does not matter if is .Net, Java, NodeJs, Windows or Linux, they all talk the same language, the others are very tight to the .Net and Windows platform and force every other solution to be also tighten or adapted to then. And also there is the fact of some being synchronous and other asynchronous like Service bus.
Then, when performance is an issue, I evaluate the other options.
In "Implementing Domain-Driven Design", Vernon give detailed examples for integrating bounded context with a messaging or REST based solution, it also mention database integration, but I understand it is not a very clean solution to share database or at least db tables between BC.
But what if the 2 BCs I want to integrate are hosted locally on the same server, is it really a good idea to use a messaging/rest/rpc solution ? (which seems more suitable for a remotely hosted BC to me)
Otherwise, except with DB integration, what are the other alternatives ? Hosting both BC in the same process and calling it directly (still using adapters and translators for clean seperation) ?
Thanks
You could look into using something like 0MQ for inter-process communication on the same server. I've also in the past just hosted things in the same process as you suggest and just used interfaces / in-memory messaging to separate out contexts.
Everything is about trade-offs in the end, so you just need to decide what level of isolation you are willing to accept. The simplest solution would be to separate inside a solution via folders and interfaces, the other end of the spectrum being completely separate servers.
I don't think that location should come into play w.r.t. integration between BCs.
There really are other factors to consider such as guaranteed delivery to the recipient in order to ensure that the processing takes place. This should be required whether or not the two BCs are hosted on the same server.
Another reason to ignore location is that when you need to scale, your architecture should be able to handle it from the get-go.
As tomliversidge mentioned it is possible to use some deployment mechanisms such as non-durable messaging to speed up things but there will definitely be a trade-off and that has to be a conscious decision.
I have a few applications that I have built that leverage MSMQ on a Windows Server. I would like to port these to Azure-based Web Applications which helps me with some of the security and trust-barrier challenges I have faced in the past.
My question is, can I expect that the Message Queue aspects of Windows Azure storage will behave the same as MSMQ?
In general, not really. While both are queueing solutions, you will find that Windows Azure queues don't have the same guarantees that MSMQ will have (for good reason). For instance, WAQ supports a delivery model of 'at least once' versus 'exactly once'. It also uses a 2PC model where you must pop and then delete a message. It is also not involved in transactions like MSMQ can handle. However, for cloud queuing scenarios, some of these considerations matter less. There are other nuances as well (message TTL, invisibility, renewals, storage time, etc.) that differ.
It might be easier to answer more completely if you explain what features your queueing needs must have. Also, keep in mind that AppFabric has a queueing service that also might be appropriate depending on what you really need.
I am a BTS dev with four years experience implementing "classic" BTS solutions.
I have to do a surprise presentation tomorrow on the ESB toolkit 2. Don't ask why. I would like to be as prepared as possible. I am looking for resources which can be quickly assimilated. Ideally (in rough order of importance):
A technical overview of the main moving parts
A few of the implementation details, descriptions of itineraries, resolvers, on-ramps etc.
Installation and deployment.
Any swift help would be greatly appreciated.
If you're familiar with the picture in the Pro BizTalk 2009 book the ESB poster might help you translate some of that info.
Here are some resources:
(obvious) ESB 2.1 on MSDN
The Pro BizTalk 2009 book has a chapter in it on ESB with sections:
BizTalk and the ESB Concept
The Architecture
The Functional Components
Working with Itineraries
Using the Toolkit
Summary
Interesting thread where Richard Seroter posts some info. (author of SOA Patterns with BizTalk Server 2009)
The ESB Toolkit is large toolkit, it's somewhat unfair to ask you to a presentation on it with so little time to prepare.
In its most simple form, say you have a classic BizTalk implementation that does something simple like pickup a file a directory, drive it through some biz process in an orchestration, transform it, and drop it in another directory.
With the ESB toolkit, you would create an itinerary that gets attached to the message and the itinerary are basically instructions on how/what should happen to it. So your business process orchestration and maps are de-coupled from the solution. This also affords you the ability to make changes to the itinerary without re-building the solution. As the itinerary is being driven, services like the map that is needed to transform your messaged can be "looked up" in the UDDI registry. All the while the ESB exception management piece can give devs/admins/users a nice web interface into a central location where all your exceptions are sent. The exception management portal can also provide notifications and alerts for all kinds of exception criteria.
In addition to Derek's answer, I would also suggest readidng up on the ESB Portal and Exception Handling framework components in the Toolkit. I would say that most of my clients using the ESB components are only using those components, and not itineraries, UDDI, etc.