I've been reading about DDD and microservices. Started to prototype by taking a use case for CQRS part. The use case is a sports soccer app which has videos, news, scores, and homepage. In this, I've identified the domains and Bounded context which are
News
Videos
Scores
Homepage
First, 3 domains are totally independent of each other.
Now, homepage domain requirements.
1. Score section
2. Videos section
3. Content section
Content Section: It has its own database
Videos section: It will make HTTP call video service and get the data
Score section: It will make HTTP call Score service and get the data
My question is with homepage domain.
I find it is highly coupled with other services and it's not independent.
How can I design homepage domain?
I find it is highly coupled with other services and it's not independent.
Yes.
You can improve the independence by thinking carefully about the failure modes; how should the UI elements behave when the relevant data authority is not available. You can, for example, have the homepage simply display a "data not available" message if the remote authority doesn't respond in a timely fashion. Or you can have the homepage display a cached copy of some stale data. It may even make sense to always render the response using cached data, with updates to the cache happening in the background.
There are techniques that you can use to reduce the coupling between the services.
One approach is described by Udi Dahan, under the umbrella of UI Composition Techniques. His idea is that you use an element, like a widget, to encapsulate the logic of display and the failure mode.
The underlying mechanics aren't really all that different; the widget has all of the same problems we had before when the data isn't available. But what it does do is separate out two problems (what do you do when the widget isn't available, what do you do when the data isn't available) that belong to different teams: the homepage team gets to decide what to do when the widget isn't available (perhaps a cached copy is used), the service team gets to decide what to do when the widget can't reach the backing data (perhaps the widget has its own cache of stale data).
There's no magic; you've got a distributed system, and a consequence of this is that you need to consider in your design the fact that the remote processes won't always be available.
Related
Several sources claim that process managers do not contain any business logic. A Microsoft article for example says this:
You should not use a process manager to implement any business logic in your domain. Business logic belongs in the aggregate types.
Further up they also say this (emphasis mine):
It's important to note that the process manager does not perform any business logic. It only routes messages, and in some cases translates between message types.
However, I fail to see why translations between messages (e.g. from a domain event to a command) are not part of the business logic. You require a domain expert in order to know what the correct order of steps and the translations between them are. In some cases you also need to persist state in-between steps, and you maybe even select next steps based on some (business) condition. So not everything is a static list of given steps (although that alone I’d call business logic too).
In many ways a process manager (or saga for that matter) is just another aggregate type that persists state and may have some business invariants, in my opinion.
Assuming that we implement DDD with a hexagonal architecture, I‘d place the process manager in the application layer (not adapter!!) such that it can react to messages or be triggered by a timer. It would load a corresponding process manager aggregate via a repository and call methods on it that either set its (business) state or ask it for the next command to send (where the actual sending is done by the application layer of course). This aggregate lives in the domain layer because it does business logic.
I really don‘t understand why people make a distinction between business rules and workflow rules. If you delete everything except the domain layer, you should be able to reconstruct a working application without the need to consult a domain expert again.
I‘d be happy to get some further insight I might be missing from you guys.
A fair portion of the confusion here is a consequence of semantic diffusion.
The spelling "process manager" comes from Enterprise Integration Patterns (Hohpe and Woolf, 2003). There, it is a messaging pattern; more precisely, it is one possible specialization of a message router. The motivation for a message router is a decoupling of the sender and receiver.
If new message types are defined, new processing components are added, or routing rules change, we need to change only the Message Router logic, while all other components remain unaffected.
Process manager, in this context, refers to a specialization of message router that sits in the middle of a hub and spoke design, maintaining the state of the processing sequence and "determining the next processing step based on intermediate results".
The "process definition" is, of course, something that the business cares about -- we're passing these messages around to coordinate activities in different parts of the enterprise, after all.
And yes... this thing that maintains the "state of the processing sequence", sounds a lot like an example of a "domain entity", this is true.
BUT: it is an entity of the message routing domain; which is to say that it is bookkeeping to ensure that messages go to the right place rather than bookkeeping of business information (ie: the routing of shipping containers).
Expressed in the language of hexagonal architecture, what a process manager is doing is keeping track of messages sent to other hexagons (and, of course, the messages that they send back).
Domain logic is not only to be found in aggregates and domain services. Other places are:
Appropriately handling domain events. Domain events can be translated into one or multiple commands to aggregates; they can trigger those commands based on some condition/policy on the event itself and/or the state of other aggregates; they can inform an ongoing business process to proceed with its next step(s), and so forth. All of these things are part of domain logic.
A business process is a (potentially distributed) state machine that may involve various actors/users/systems. The allowed states and transitions between them are all a core part of the domain logic.
A saga is an eventually consistent transaction spanning multiple local or foreign aggregates that completes either successfully or compensates already executed steps in a best-effort manner. The steps that make up a saga can only be known to domain experts and thus are part of the domain logic.
The reasons why these three things are – in my opinion – mistaken as an application-layer-only concern are the following:
In order to handle a domain event, we must load and later save the affected aggregates. Because of this, the handler must also be part of the application layer. But crucially, not only. If we respect the idea behind a hexagonal architecture, then for each domain event handler residing in the application layer, there must be a corresponding one located in the domain layer. Even for the most trivial case where one domain event translates to exactly one command method invocation on some aggregate. This is probably omitted in many examples because it initially adds little value. But just imagine that later on, the translation will be based on some further business condition. Will we also just place that in the application layer handler? Remember: All of our domain logic should be in the domain layer.
A side note: Even if we respect this separation of concerns, we still have the choice of letting the domain event be handled by an aggregate itself or let it be translated into an aggregate command by a thin domain service. This choice however is based on an entirely different concern: Whether or not we want to couple aggregates more tightly. There is no right or wrong answer here. Some things just naturally are coupled more tightly while others might benefit from some extra indirection for increased flexibility.
In order to correctly implement a business process or saga, we have to take care of various application-specific concerns like message de-duplication, idempotency, retries, timeouts, logging etc. Although one might argue that the domain logic itself should be responsible for dealing with at least some of these aspects first-hand (see Vaughn Vernons excellent talk about modelling uncertainty). Remember, however, that the essence of the sequence of (allowed) steps/actions is entirely based on domain logic.
Finally, a word about coupling. In my view, there is a tendency in the community that coupling is a bad thing per-se and thus must be avoided/mitigated by all means. This might lead to solutions like placing event-command translations (remember: domain logic!) out in the adapter layer of a hexagonal/onion/clean architecture. This layers responsibility is to adapt something to something else with the same semantics/function, but with slightly different form (think power adapters). It is not the place to host any type of domain logic even if it is dead simple. Businesses have dependencies and coupling all over the place. The art is to embrace it where it actually is, and avoid it otherwise. There is a reason why we have partnership or customer/supplier relationships in DDD. And if we care about domain logic isolation, those dependencies are reflected right where they belong: In the domain layer.
A side note: An anti-corruption layer (DDD) is a valid example of an adapter. For example, it may take a bunch of remote domain events and transform/combine them in any way necessary to suit the local model. They still remain events that happened in the past, and don't just magically become commands. The transformation only changes form, not function. And it doesn't eliminate the inevitable coupling from a domain perspective. It just rephrases the same thing in a slightly different language.
I'm currently learning about Microservices because in the company where I work, we will split down our giant monolith into microservices.
We have a lot of business logic in our application, but this rules basically validate data from many different domains and act accordingly the status of this data.
Our domain has it's own data, of course, but I even dare to say we depend something like 60 ~ 70% of data from different domains, which makes our domain kind of an aggregator.
I created a little diagram to illustrate it:
So like I said before, my domain (Domain A) has a lot of business logic to validate data and status from all those different domains. And then after this validation, take the appropriate actions and save the result of it in the DB.
I feel like I hit a dead end, cause I have read a few articles how to break down a monolith, but I haven't got any good example where it explains this situation.
So I ask you guys, do you have any suggestions to tell me? :)
Thanks!
In DDD speak a bounded context approximates a microservice. The different domains in your diagram are probably going to be bounded contexts.
You most certainly do not want concepts from various BCs polluting each other as you are going to end up with quite a mess.
There is one place, however, where you may run into this and that is on integration/orchestration. Here you should approach this concern almost as a separate BC that relates to the orchestration or integration.
For instance, let's assume that you have an Assets domain and an Accounting domain. The two should know nothing about each other. However, when you decommission an asset (say some huge machine that grinds down rocks into stones) you perhaps need to have the accounting domain register some write-off value if the asset has not reached the end of its useful life. In this layer you would integrate the various bits of your process and manage the state using a process manager. Although the process manager, and related state, may belong to the Assets BC the AssetsOrchestration BC may make use of objects from both the Assets as well as the Accounting BCs. Typically you would attempt to limit that interaction to, say, messages using some messaging infrastructure but YMMV.
A good starting point may be Sam Newman's Microservice Decomposition Patterns.
About 18 minutes into the talk, Newman offers this pattern:
Within your monolith, identify the coarse modules of functionality.
Draw the dependency graph
OK, your dependency graph has cycles in it. That's no good, you want an acyclic graph. So iterate on the graph until you are able to eliminate the cycles. DO NOT PASS GO until you have this done
Identify candidate modules that have no inbound dependencies - you want the bits that nothing else depends on.
Pick one, and see if you can redesign your system to allow you to deploy that module independently of the monolith.
What I'm trying to achieve is to develop an application implementing the DDD approach.
The story might sound silly but it's an actual, real life problem. Believe me.
The business looks as follows:
Let's say a company specializes in manufacturing sweets which are distributed to its own shops for sale.
The craftsman makes different types of candy depending on what is - and what is not - currently at the display at one of the company's shop.
When a basket of one flavour 'disappears' the seller replaces this type of sweet with a different kind from the shops storage cabinet.
Duplicates of flavours at the display shouldn't exist and the display should be populated with as much as the capacity allows or how much the manafacturer can handle to produce.
The sweets are distributed from the manufacturer's lab's storage to the shop's one depending on the demand.
Let's assume each worker has public view access to the display and the storage cabinet. Each worker (user) decides on it's own what to provide. The shops display view will be publicly accessible through the application to a potential client as an information what is currently on sale.
So far I have split the business logic into three separate (sub?)domains which are:
Production
Distribution
Sale
And of course each entity like Sweets, Storage, Craftsman, it's Repository etc. are placed respectively in their domain.
The concerns I approach are:
Is it appropriate that an entity (Sweet) is being passed from one domain to another?
Should a Provider be able to reach the StorageCabinet of one domain and pass it's content to another?
Is my reasoning proper? Correct me if I'm wrong or violating any DDD rules.
Thanks in advance.
The story might sound silly but it's an actual, real life problem.
This is great, actually. In his recent retrospective, one of the things that Greg Young called out is that "shopping cart" models are a really lousy as a teaching tool. He points out briefly that the interesting questions are in the supply chain.
Is it appropriate that an entity (Sweet) is being passed from one domain to another?
No, but a message (DTO) describing an entity's state might be passed from one domain to another.
You want to keep the flexibility to define the entities differently in each domain; that's part of the point of identifying bounded contexts.
Should a Provider be able to reach the StorageCabinet of one domain and pass it's content to another?
Probably not: your domain model isn't the book of record for the storage cabinet. Listen very carefully to Greg's comments on one way commands.
I am to build a kind of website tracking system.
Think of a website where users click on various links – a unique user id and an identifier of the page tracks all page views.
Now, a single user might view 20 pages – some relevant some not. What I want to track is if a user follows a specific “path”. Example “Home Page” -> “Product A Page” -> “Get more info page” -> “Buy” -> “Paid”. There might be other page views in between each of these steps; the important thing is IF a user follows a given pattern.
In addition, I need to measure time between each step (each page view has a timestamp).
I have been playing around with Reactive Extensions, but I am not an expert in the area so I would like to hear if this would be a job for the Reactive Framework or if other technologies are more suitable?
I imagine a server getting a stream of website page views and then some fancy reactive LINQ queries that captures the events (this is where I need some help).
Next question that comes to my mind is how do you host this behind a load-balancer (on Windows Azure)? If you run two instances and the “Home Page”-page view goes to instance 1 and “Product A Page” goes to instance 2, how do they communicate about this or should some kind of sharding e.g. per userid be enforced?
Lastly, what about persistence? How to store? Should you store data in an Event Queue pattern and then load everything into memory when you “play back” from a restart of the server?
I know that were many questions, but I do love the philosophy behind Reactive Extensions; I just cannot get my head around how to “put into production in the cloud” :)
Thanks!
Casper
There are lots of solutions out there in this space already that you can integrate into your platform. Are you sure you're not reinventing the wheel? Google Analytics has functionality similar to this. If you need to go your own way, then SQL Server StreamInsight might be a better fit.
For behind-the-firewall solutions, Also look at http://piwik.org/ (free, open-source) and http://www.haveamint.com/.
This post is intended to start a deeper discussion on Single Page Applications for the web. There are questions that do not seem to have a clear answer in most resources on the subject.
They are in my mind
Authorization and authentication.
With entire web app being on the client, it may make calls to the server in any of its functions, even those that the user does not have rights to. The fact that the user cannot see a menu, does not preclude that person from invoking java script functions. This is easily handled in MVC app, for example, by using controllers that validate user rights to a specific function based on a cookie for example. However, some SPA apps just use single controller with Breeze or Web Api, which make authorization server side impossible.
Memory management on the client
For small sample apps this is not an issue, but imagine an app with 100's of screens or an app with a single screen that pulls thousands of records over the course of one day. With persistent caching one could imagine large memory issues, especially on under-powered devices with little RAM, like phones or tablets. How can a group of developers had SPA route without a clear vision of handling memory management?
Three Tier deployment
Some IT departments will never allow applications with a connection string to a database located on front end web servers. Every SPA demo I have seen is structured exactly like that, including Breeze or Web Api for that matter.
Unobtrusive validation.
It would require developers to use MVC partial views and controllers instead of just HTML files, which seems to fly in the face of SPA concepts, while it provides a very robust way to easily incorporate validation and UI to support it into web applications.
Exposing primary integer based keys in the url.
This is non-no in OWASP.
As a result, SPA applications "seem" to target areas with few security requirements and small feature sets. What do you think?
Thanks.
#Sergey - I think this is just too broad a question for StackOverflow. S.O. isn't a discussion forum; it's a place to go for specific answers. So while your questions are potentially valid, I don't think you should hold out much hope for deep substantive responses here.
May I add, in the friendliest possible way, that your sweeping, unsupported, and negative statements make you look like a troll. You're not a troll are you Sergey?
On the chance that you are in fact authentically concerned, I offer a few quick reactions, particularly as they pertain to Breeze.
Authorization. In Web API you can authorize at the method level. The ApiController base class has a User property that returns the IPrincipal. So whether you have one controller or many (and you can have many in Breeze if you want), the granularity is method level, not just class level.
Memory management. Desktop developers have coped with this concern for years. It may cause you some astonishment if you've always developed traditional web apps where process lifetimes are brief. But long-running processes are not news to those of us who built large apps in desktop technologies such as WinForms, WPF, and Silverlight. The issues and solutions are much the same in the land of HTML and JavaScript.
Layers on the backend. You've been looking at demos too long. Yes most demos dump everything into one project running on one server. We assume you know how to refactor the server to meet scaling, performance and security requirements for your environment. Our demos are concerned mostly with front-end SPA development. We do dabble at the service boundary to show how data flow through a service API, through an ORM, through to the database. We thought it sufficient to identify these distinct layers and leave as an exercise for the reader the comparatively trivial matter of moving these layers to different tiers. We may have to re-visit that assumption someday. But does anyone seriously believe that there are significant obstacles to distributing layers/responsibilities across server-side tiers? Really? Like what?
Unobtrusive validation. When most people start using the word "unobtrusive" in connection with HTML, they are usually making a point about keeping JavaScript out the HTML. Perhaps that's what you mean too, in which case SPA developers everywhere agree ... and that's why there are numerous "unobtrusive validation" libraries available. HTML 5 validation, jQuery validation and Knockout validation come to mind. All of them are in the SPA developer's toolkit and none of them "require developers to use MVC partial views and controllers". What gives you the impression that a SPA would need any server-side resources of any kind to implement validation with JavaScript-free HTML markup?
Ids as security risk. Really? This is bogus. The key value is no more a security risk than any other data value. Millions of applications - not just SPAs - communicate key values to the client, both in the URL and in the body. It's standard in REST APIs. It's standard in ODATA. And you want to dismiss them all by saying that they "target areas with few security requirements and small feature sets"? Good luck with that. I think you'll have to do better than rest your case on a link to a relatively obscure organization's entire web site.
I have built some SPA applications, ranging from small to large (over 100 scripts and views). Only a handful of them had every view accessible to the public. The rest went through a strict access structure. It was so simple to return a 401 unauthorized from the server and the client just handling the 401 to redirect it to the login screen. Mr. Ward and Mr. Papa put it right. Get out of the Demo mode and try to find solutions to the issues you come across. I have watched John Papa's SPA on pluralsight, gone through numerous articles and applications on Breeze and I have to tell you, none of my applications use Breeze to do queries from the client side, because YOU DON'T NEED TO!!
Moreover, I have only extended what I have learnt and come up with my own way of solving problems. This is not an answer to your queries, but I only can provide a short comment. No technique is perfect and there is no ONE way to do everything. My server side is locked down where it needs to be locked down, my routes on the client side are locked down (if using durandal take a look at guardRoute), my scripts are minified and my images are sprited (if there is a word like that). All in all, SPA is a great technique, you got to find solutions to the quirks!