I'm wondering if it's preferable to publish events rather than throw exceptions from aggregates. Say, I have a domain where there is a requirement that only students of a certain grade level can enroll for sports. If I call EnrollForSports on a student that does not satisfy the criteria, should the aggregate throw an exception or publish an event, particularly if other aggregates or process managers are interested in the result of the process?
If an event is published, doesn't it mean there would be a need for a corresponding internal event handler to handle the event when we're replaying, even though the event wouldn't change the state of the aggregate?
If an exception is thrown, how will the other parties get notified? Can the command handler catch the exception and then raise an event? Can events be raised from command handlers?
This question can't be generally answered. There is no "best practice", it really depends on many factors.
Nevertheless this topic has just recently been discussed at the DDD/CQRS mailing list. Read it and you will probably find some interesting and helpful points over there.
Oh, and here's a blog post on the topic: Business Errors are Just Ordinary Events
Principally speaking, a command should be either valid and executed, or invalid and not executed. The idea with spawning error events leaves you somewhere in the middle and the feedback to whomever sends the command is ambiguous and delayed. It is also an unnecessary complexity in your domain. If you throw an exception it will come as an immediate feedback to to the client code which sent the command.
Related
I am new to Event sourcing concept so there are a couple of moments I don't understand. One of them is how to handle following scenario:
I've got 2 instances of a service. Both of them listen to a event queue. There are two messages: CreateUser and UpdateUser. First instance picks up CreateUser and second instance picks up UpdateUser. For some reason second instance will handle its command quicker but there will be no User to update, since it was not created.
What am I getting wrong here?
What am I getting wrong here?
Review: Race Conditions Don't Exist
A microsecond difference in timing shouldn’t make a difference to core business behaviors.
In other words, what you want is logic such that the order of the messages doesn't change the final result, and a first writer wins policy (aka compare-and-swap), so that when you have two processes trying to update the same resource, the loser of the data race has to start over.
As a general rule, events should be understood to support multiple observers - all subscribers get to see all events. So a queue with competing consumers isn't the usual approach unless you are trying to distribute a specific subscriber across multiple processes.
You do not have a concurrency issue you can solve. This totally runs down to either using bad tools or not reading the documentation.
Both of them listen to a event queue.
And that queue should support that. Example are azure queues, where I Can listen AND TELL THE QUEUE not to show the event to anyone else for X seconds (which is enough for me to decide whether i handled it or not). If I do not answer -> event is reinserted after that time. If I kill it first, there is no concurrency.
So, you need a backend queue that can handle this.
I've been studying DDD for a while, and stumbled into design patterns like CQRS, and Event sourcing (ES). These patterns can be used to help achieving some concepts of DDD with less effort.
In the architecture exemplified below, the aggregates know how to handle the commands and events related to itself. In other words, the Event Handlers and Command Handlers are the Aggregates.
Then, I’ve started modeling one sample Domain just to understand how the implementation would follow the business logic. For this question here is my domain (It’s based on this):
I know this is a bad modeled example, but I’m using it just as an example.
So, using ES, at the end of the operation, we would save all the events (Green arrows) into the event store (if there were no Exceptions), each event into its given Event Stream (Aggregate Type + Aggregate Id):
Everything seems right until now. So If we want to Rebuild the internal state of an instance of any of this Aggregate, we only have to new it up (new()) and apply all the events saved in its respective Event Stream in the correct order.
My question is related to changes in the model. Because, software development is a process where we never stop learning about our domain, and we always come with new ideas. So, let’s analyze some change scenarios:
Change Scenario 1:
Let´s pretend that now, if the Reservation Aggregate check’s that the seat is not available, it should send an event (Seat not reserved) and this event should be handled by one new Aggregate that will store all people that got their seat not reserved:
In the hypothesis where the old system already handled the initial command (Place order) correctly, and saved all the events to its respective event streams:
When we want to Rebuild the internal state of an instance of any of this Aggregate, we only have to new it up (new()) and apply all the events saved in its respective Event Stream in the correct order. (Nothing changed). The only thing, is that the new Use case didn’t exist back in the old model.
Change Scenario 2:
Let’s pretend that now, when the payment is accepted we handle this event (Payment Accepted) in a new Aggregate (Finance Aggregate) and not in the Order Aggregate anymore. And It send a new Event (Payment Received) to the Order Aggregate. I know this scenario is not well structured, but something like this could happen.
In the hypothesis where the old system already handled the initial command (Place order) correctly, and saved all the events to its respective event streams:
When we want to Rebuild the internal state of an instance of any of this Aggregate, we have a problem when applying the events from the Aggregate Event Stream to itself:
Now, the order doesn’t know anymore how to handle Payment Accepted Event.
Problems
So as the examples showed, whenever a system change reflects in an event being handled by a different event handler (Aggregate), there are some major problems. Because, we cannot rebuild the internal state anymore.
So, this problem can have some solutions:
Possible Solution
When an event is not handled by the aggregate in which Event Stream it is stored, we can find the new handler and create a new instance and send the event to it. But to maintain the internal state correct, we need the last event (Payment Received) to be handled by the Order Aggregate. So, we let it dispatch the event (and possible commands):
This solution can have some problems. Let’s imagine that a new command (Place Order) arrives and it has to create this order instance and save the new state. Now we would have:
In gray are the events that were already saved in the last call when the system hadn’t already gone through model changes.
We can see that a new Event Stream is created for the new aggregate (Finance W). And we can see that Event Streams are append-only, so the Payment Accepted event in the Order Y Event Stream is still there.
The first Payment Accepted event in Finance W Event Stream is the one that was supposed to be handled by the Order but had to find a new handler.
The Yellow payment received event in Order’s Event Stream is the event that was generated by the new handler of the Payment Accepted when the Payment Accepted event from the Order’s Event Stream was handled by the Finance.
All the other Green Events are new events that were generated by handling the Place Order Command in the new model.
Problem With the Solution
The next time the aggregate needs to be rebuild, there will be a Payment Accepted event in the stream (because it is append-only), and it will again call the new handler, but this have already been done and the Payment Received event have already been saved to the stream. So, it is not necessary to go through this again, we could ignore this event and continue.
Question
So, my question is how can we handle with model changes that impact who handle each event? How can we rebuild the internal state of an Aggregate after a change like this?
Will we need to build some event Stream migration that changes the events from one stream to the new schema (one or more streams)? Just like we would need in a Relational database?
Will we never be allowed to remove one handler, so we can only add new handlers? This would lead to unmanageable system…
You got almost all right, except one thing: Aggregates should not handle events from other Aggregates. It's like a non-event-sourced Aggregate shares a table with another Aggregate: they should not.
In event-driven DDD, Aggregates are the system's building blocks that receive Commands (things that express the intent) and return Events (things that had happened). For every Command type must exist one and only one Aggregate type that handle it. Before executing a Command, the Aggregate is fed with all its own previously emitted Events, that is, every Event that was emitted in the past by this Aggregate instance is applied to this Aggregate instance, in the chronological order.
So, if you want to correctly model your system, you are not allowed to send events from one Aggregate as events to another Aggregate (a different type or instance).
If you need to model business processes that involve multiple Aggregates, the correct way of doing it is by using a Saga/Process manager. This is a different component. It is the opposite of an Aggregate.
It receive Events emitted by Aggregates and sends Commands to other Aggregates.
In simplest cases, a Saga manager simply takes properties from one Event and creates+populates a Command with those properties. Then it sends the Command to the destination Aggregate.
In more complicated cases, the Saga waits for multiple Events and when all are received only then it creates and sends a Command.
The Saga may also deduplicate or reorder events.
In your case, a Saga could be Sale, whose purpose would be to coordinate the entire sales process, from ordering to product dispatching.
In conclusion, you have that problem because you have not modeled correctly your system. If your Aggregates would have handled only their specific Commands (and not somebody else's Events) then even if you must create a new Saga when a new Business process emerges, it would send the same Command to the Same Aggregate.
Answering briefly
my question is how can we handle with model changes that impact who handle each event?
Handling events is generally an easy thing to change, because the handling part is ephemeral. Events have a single writer, but they can have many readers. You just need to arrange for the plumbing to notify each subscriber of the event.
So in scenario #1, its the PaymentAggregate that writes down the PaymentAccepted event (in its own stream), and then your plumbing notifies the OrderAggregate that the PaymentAccepted event happened, and it does the next thing in its own logic.
To change to scenario #2, we'd leave the Payment Aggregate unchanged, but we'd arrange the plumbing so that it tells the FinanceAggregate about PaymentAccepted, and that it tells the OrderAggregate about PaymentReceived.
Your pictures make it hard to see this; I think you aren't being careful to track that each change of state is stored in the stream of the aggregate that changed. Not your fault - the Microsoft picture is really awful.
In other words, your arrow #3 "Seats Reserved" isn't a SeatsReserved event, it's a Handle(SeatsReserved) command.
I am trying to implement another DDD bounded context with CQRS and ES.
I wonder, given there is CreateUserCommand that creates User in my domain model (not a word about saving). Then it fires UserCreatedEvent.
I have two event handlers for that event:
PersistUserEventHandler (updates state of app) and
SendWelcomeEmailEventHandler (sends welcome email to user)
Now, I know, that:
Order of processing event in Event Handlers should not matter
Saving state should be detail, because source of truth is in my event store.
But, what if I do not want to send welcome email until my read model is fully updated? Because, what if for example process is delayed or some error occurs and I am not able to persist that user into read model right now? Then I do not want send that welcome email now, because if user clicked to for example link to his profile in mail, he would see "user does not exists".
I saw people are persisting changes through repository directly in command handlers (which would solve this problem), but that does not make sense with Event Sourcing, because I want to be able to replay all events (with event handlers for persisting only to prevent all other side effects) and get actual state of application in persistence layer.
Or should I listen to UserCreatedEvent only with event handler that actually persists it into read model and then raise in this event handler another event CreatedUserSavedEvent and all emails etc. would have been sent by their handlers?
I suppose NO too, because it reminds me some event hell and also if I get EventBus into some event handler, I am getting into circular reference problem which is just effect of that I am violating rule that every depencency should point down to lower components of my system and not the other side.
So, how is this usualy solved or am I missing something?
PersistUserEventHandler (updates state of app)
You might be mistaking Read Models for a homogeneous whole that accurately represents the current state of an application, i.e. a second source of absolute truth besides the event log.
I tend to see them more as a bunch of partial, opinionated parcels of state that may not all be updated at the same time and may reflect different truths.
I don't recommend taking read models as a source of data in another context than the use case they were designed for. In your example, SendWelcomeEmail should probably not rely on the User read model but only on the data contained in the UserCreated event.
Now you can share code between read model projectors and other types of event handlers to avoid duplication, but sharing data seems risky.
If users have random UUID then it should not be a problem. If a user arrive at an url and the readmodel is not up to date then you could show a "loading in progress,please wait" message.
If you really want to know if the user really exists - for example you want to see the difference between "user does not exists" and "read model is not sunchronized yet" then you could send a special command that don't generate any events (or just test a command if your command dispatcher supports dry running of commands) and throw exception if user does not exist.
Example: Business rules states that the customer should get a confirmation message (email or similar) when an order has been placed.
Lets say that a NewOrderRegisteredEvent is dispatched from the domain and is picked up by an event listener that sends of the confirmation message. When that is done some other event handler throws an exception or something else goes wrong and the unit of work is rolled back. We've now sent the user a confirmation message for something that was rolled back.
What is the "cqrs" way of solving problems like this where you want to do something after a unit of work has been committed? Another complicating factor is replaying of events. I don't want old confirmation messages to be re-sent whenever I replay recorded events in order to build a new view / projection.
My best theory so far: I've just started to look into the fascinating world of cqrs and was wondering whether this is something that would be implemented as a saga? If a saga is like a state machine where each transition only can take place a single time then I guess that would solve this problem? I just have a hard time visualizing how this will fit together with the command bus and domain events..
An Event should only occur after the transaction has been completed. If anything goes wrong and there's a rollback, then the event didn't occur from an external point of view. Therefore it shouldn't be published at all. Though an OrderRegistrationFailed event could be published if necessary.
You wouldn't want the mail to be sent unless the command has sucessfully been executed.
First a few reasons why the command handler -- as proposed in another answer -- would be the wrong place: Under some circumstances the command handler wouldn't be able to tell if the command will eventually succeed or not. Having the command handler invoke the mail sending would also put process knowledge inside the command handler, which would break the SRM and too tightly couple business rules with the application layer.
The mail should be sent after the fact, i.e. from an event handler.
To prevent this handler from firing during replay, you can just not register it. This works similar to how you test your application. You only register the handlers that you actually need.
Production system -> register all event handlers
Tests -> register only the tested event handlers
Replay -> register only the projection/denormalization handlers
Another - even more loosely coupled, though a bit more complex - possibility would be to have a Saga handle the NewOrderRegisteredEvent and issue a SendMail command to the appropriate bounded context (thanks, Yves Reynhout, for pointing this out in the question's comments).
There are two likely solutions
1) The publishing of the event and the handling of the event (i.e. the email) are part of a single transaction. In this case, your transaction framework takes care of it for you. If the email fails, then the event is rolled back. You'll likely retry the command. This is conceptually clean and easy to think about. No event is finished publishing until everyone that has something to say about it has had their say. However practically speaking, this can be painful, as it typically involves distributed transactions. These are hard to come by. Can your email client enroll in the same transaction as the database which is holding your events?
2) The publishing of the event is transactional, but the event handlers each deal with transactions in their own way. The event handler which sends emails could keep track of which events it had seen. If it crashed, it would request old events and process them. You could make a business decision as to how big a deal it would be if people had missing or duplicate emails. (For money-related transactions, the answer is probably you shouldn't allow it.)
Solution (2) is typically what you see promoted in DDD/CQRS circles as it's the more loosely coupled solution. Solution (1) is quite practical in a small system where the event store and the projections are in a single database and the projections don't change often. Solution (2) allows a diversity of event handlers to work in their own way. Solution (1) can cause lots of non-overlapping concerns to become entagled. In this case your order business rules don't complete until the many bizarre things that happen in emailing are taken care of. For one thing, it may slow you down quite a bit.
If the sending of the email were more interesting than "saw the event, sent the email", then you're right, you might have a saga or workflow on your hands. Email in large operations is often a complex system in its own right which you're unlikely to have to implement much of. You just need to be sure you put your email into a request queue of some sort (using approach (2)), and the email system is likely to do retries/batching/spam avoidance/working overnight/etc.
I've been looking at CQRS but I find it restricting when it comes to showing the result of commands in lets say a Web Application.
It seems to me that using CQRS, one is forced to refresh the whole view or parts of it to see the changes (using a second request) because the original command request will only store an event which is to be processed in future.
In a Web Application, is it possible that a Command request could carry the result of the event it creates back to the browser?
The answer to the headline of this question is quite simple: nothing, void or from a webbrower/rest point of view 200 OK with an empty body.
Commands applied to the system (if the change is successfully committed) does not yield a result. And in the case that you wish to leave the business logic on the server side, yes you do need to refresh the data by executing yet another request (query) to the server.
However most often you can get rid of the 2nd roundtrip to the server. Take a table where you modify a row and press a save button. Do you really need to update the table? Or in the case a user submits a comment on a blog post just append the comment to the other comments in the dom without the round trip.
If you find yourself wanting the modified state returned from the server you need to think hard about what you are trying to achieve. Most scenarios can be changed so that a simple 200 OK is more than enough.
Update: Regarding your question about queuing incoming commands. It's not recommended that incoming commands are queued since this can return false positives (a command was successfully received and queued but when the command tries to modify the state of the system it fails). There is one exception to the rule and that is if you are having a system with an append only model as state. Then is safe to queue the mutation of the system state till later if the command is valid.
Udi Dahans article called Clarified CQRS is always a good read on this topic http://www.udidahan.com/2009/12/09/clarified-cqrs/
Async commands are a strange thing to do in CQRS considering that commands can be accepter or rejected.
I wrote about it, mentioning the debate between Udi Dahan's vision and Greg Young's vision on my blog: https://www.sunnyatticsoftware.com/blog/asynchronous-commands-are-dangerous
Answering your question, if you strive to design the domain objects (aggregates?) in a transactional way, where every command initiates a transaction that ends in zero, one or more events (independently on whether there are some process managers later on, picking one event and initiating another transaction), then I see no reason to have an empty command result. It's extremely useful for the external actor that initates the use case, to receive a command result indicating things like whether the command was accepted or not, which events did it produce, or which specific state has now the domain (e.g: aggregate version).
When you design a system in CQRS with asynchronous commands, it's a fallacy to expect that the command will succeed and that there will be a quick state change that you'll be notified about.
Sometimes the domain needs to communicate with external services (domain services?) in an asynchronous way depending on those services api. That does not mean that the domain cannot produce meaningful domain events informing of what's going on and which changes have occured in the domain in a synchronous way. For example, the following flow makes a lot of sense:
Actor sends a sync command PurchaseBasket
Domain uses an external service to MakePayment and knows that the payment is being processed
Domain produces the events BasketPurchaseAttempted and/or PaymentRequested or similar
Still, synchronously, the command returns the result 200 Ok with a payload indicating some information about what has happened. Even if the payment hasn't completed because the payment platform is asynchronous, at least the actor has a meaningful knowledge about the result of the transaction it initiated.
Compare this design with an asynchronous one
Actor sends an async command PurchaseBasket
The system returns a 202 Accepted with a transaction Id indicating "thanks for your interest, we'll call you, this is the ticket number")
In a separate process, the domain initiates a process manager or similar with the payment platform, and when the process completes (if it completes, assuming the command is accepted and there are no business rules that forbid the purchase basket), then the system can start the notifying process to the actor.
Think about how to test both scenarios. Think about how to design UX to accommodate this. What would you show in the second scenario in the UI? Would you assume the command was accepted? Would you display the transaction Id with a thank you message and "please wait"? Would you take a big faith leap and keep the user waiting with a loading screen waiting for the async process to finish and be notified with a web socket or polling strategy for XXX seconds?
Async commands in CQRS are a dangerous thing and make us lazy domain designers.
UPDATE: the accepted answer suggest not to return anything and I fully disagree. Checkout Eventuous library and you'll see that returning a result is extremely helpful.
Also, if an async command can't be rejected it's... because it's not really a command but a fact.
UPDATE: I am surprised my answer got negative votes. Especially because Greg Young, the creator of CQRS term, says literally in his book about CQRS
One important aspect of Commands is that they are always in the imperative tense; that is they are
telling the Application Server to do something. The linguistics with Commands are important. A situation
could for with a disconnected client where something has already happened such as a sale and could
want to send up a “SaleOccurred” Command object. When analyzing this, is the domain allowed to say
no that this thing did not happen? Placing Commands in the imperative tense linguistically shows that
the Application Server is allowed to reject the Command, if it were not allowed to, it would be an Event
for more information on this see “Events”.
While I understand certain authors are biased towards the solutions they sell, I'd go to the main source of info in CQRS, regardless of how many hundred of implementations are there returning void when they can return something to inform requester asap. It's just an implementation detail, but it'll help model better the solution to think that way.
Greg Young, again, the guy who coined the CQRS term, also says
CQRS and Event Sourcing describe something inside a single system or component.
The communication between different components/bounded contexts (which ideally should be event driven and asynchronous, although that's not a requirement either) is outside the scope of CQRS.
PS: ignoring an event is not the same as rejecting a command. Rejection implies a direct answer to the command sender. Something "difficult" if you return nothing to the sender (not even a correlation ID?)
Source:
https://gregfyoung.wordpress.com/tag/cqrs/
https://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf