Eric Evans talks a lot about evolving models in DDD so refactorings seem to be essential to DDD. When one has a relational persisted state of the world you can handle model changes by migrations that change the database schema.
How can I cope with model changes when using event sourcing? If there are incompatible changes to an aggregate that would prevent replay of events is there some sort of best practice? Or is it a just-don't?
If there are incompatible changes to an aggregate that would prevent replay of events
You have essentially two options in this scenario:
Patch the older events in such a way that they are made compatible and events can be replayed from the beginning. The benefit here is that you don't lose the history but the downside is that you have to expend some effort to patch the old events.
Take a snapshot/memento of the aggregate at the point of the schema change and "re-base" the event stream from this point onward. The benefit here is that you don't have to spend any effort (with event sourcing you most likely have a snapshot mechanism in place). The downside being that you lose the ability to replay events from before the snapshot.
As a general rule of thumb I'd say default to the second option unless you know for sure that you need to be able to go back and edit history before the schema change.
I have not much expierence myself. But I saw a concept called Upcasting
Originally a concept of object-oriented programming, where: "a subclass gets cast to it's superclass automatically when needed", the concept of upcasting can also be applied to event sourcing. To upcast an event means to transform it from its original structure to its new structure. Unlike OOP upcasting, event upcasting cannot be done in full automation because the structure of the new event is unknown to the old event. Manually written Upcasters have to be provided to specify how to upcast the old structure to the new structure.
You can refer to Axon's doc for more detail
Events are just DTOs. It doesn't matter how the model changes as long as you still have one object, if the event itself doesn't change. If you need to change the event, you can 'upgrade' it with the required properties. The Apply method will know what to do with it. I can't come up with something specific without knowing details.
If the model changes so much that basically now you have 2 Aggregate Roots(AR) instead of a previous one, this means you have new different aggregates which won't be using the old events. Basically you start from the old AR, create the new ones and generate the corresponding events which will be specific to those ARs. So you don't really have a compatibility problem in this case.
Working with events is not as straightforward as 'classic' OOP and RDBMS schema, but they are more flexible if you think in business terms and treat your objects as domain concepts. Changing the model means the business concept definition or usage has changed as well, so now you're dealing with a different (new as far as the persistence is concerned) concept.
Related
I have flicked through few popular Event Sourcing frameworks written in a variety of different common languages. I have got the impression all of them affect the domain models to a really high degree. As far as I understand ES is just an infrastructure concern - a way of persisting aggregate state. Of course, it facilitates message driven inter-context integration but in core domain's point of view is negligible. I consider commands and events to be part of the domain itself so it looks perfectly fine that aggregate creates events (but not publishes them) or handles commands.
The problem is that all of DDD building blocks tend to be polluted by ES framework. Events must inherit from some base class. Aggregates at least are supposed to implement foreign interfaces. I wonder if domain models should be even aware of using ES approach within an application. In my opinion, even necessity of providing apply() methods indicates that other layer shapes our domain.
How you approach this issue in your projects?
My answer applies only when CQRS is involved (write and read models are split and they communicate using domain events).
As far as I understand ES is just an infrastructure concern - a way of persisting aggregate state
Event sourcing is indeed an infrastructure concern, a kind of repository but event-based Aggregates are not. I consider them to be an architectural style, different from the classical style.
So, the fact that an Aggregate, in reaction to an command, generates zero or more domain events that are applied onto itself in order to build its internal (private) state used to decide what events to generate in the future is just a different mode of thinking and designing an Aggregate. This is a perfect valid style, along with classical style (the one not using events but only objects) or functional programming style.
Event sourcing just means that every time a command reaches an Aggregate, its entire internal state is rebuild instead of being loaded from a flat persistence. Of course there are other huge advantages (!) but they do not affect the design of an Aggregate.
... but not publishes them ...
I like the frameworks that permit us to just return (or better yield - Aggregate's command methods are just generators!) the events.
Events must inherit from some base class
It's sad that some frameworks require that but this is not necessarily. In general, a framework needs one mean of detecting an event class. However, they can be implemented to detect an event by other means instead of using marker interfaces. For example, the client (as in YOU) could provide a filter method that rejects non-event classes.
However, there is one thing that I couldn't avoid in my framework (yes, I know, I'm guilty, I have one): the Command interface with only one method: getAggregateId.
Aggregates at least are supposed to implement foreign interfaces.
Again, like with events, this is not a necessity. A framework could be given a custom client event-applier-on-aggregates function or a convention can be used (i.e. all event-applier methods have the form applyEventClassNameOrType.
I wonder if domain models should be even aware of using ES approach within an application
Of ES not, but event-based YES, so the apply method must still exists.
As far as I understand ES is just an infrastructure concern - a way of persisting aggregate state.
No, events are really core to the domain model.
Technically, you could store diffs in a domain agnostic way. For example, you could look at an aggregate and say "here is the representation before the change, here is the representation after, we'll compute the difference and store that.
The difference between patches and events is the fact that you switch from a domain agnostic spelling to a domain specific spelling. Doing that is normally going to require being intimate with the domain model itself.
The problem is that all of DDD building blocks tend to be polluted by ES framework.
Yup, there's a lot of crap framework in the examples you find in the wild. Sturgeon's Law at work.
Thinking about the domain model from a functional perspective can help a lot. At it's core, the most general form of the model is a function that accepts current state as an input, and returns a list of events as the output.
List<Event> change(State current)
From there, if you want to save current state, you just wrap this function in something that knows how to do the fold
State current = ...
List<Event> events = change(current)
State updated = State.fold(current, events)
Similarly, you can get current state by folding over the previous history
List<Event> savedHistory = ...
State current = State.reduce(savedHistory)
List<Event> events = change(current)
State updated = State.fold(current, events)
Another way of saying the same thing; the "events" are already there in your (not event sourced) domain model -- they are just implicit. If there is business value in tracking those events, then you should replace the implementation of your domain model with one that makes those events explicit. Then you can decide which persisted representation to use independent of the domain model.
Core of my problem is that domain Event inherits from framework Event and aggregate implements some foreign interface (from framework). How to avoid this?
There are a couple of possibilities.
1) Roll your own: take a close look at the framework -- what is it really buying you? If your answer is "not much", then maybe you can do without it.
From what I've seen, the "win" of these frameworks tends to be in taking a heterogeneous collection of events and managing the routing for you. That's not nothing -- but it's a bit magic, and you might be happier having that code explicit, rather than relying on implicit framework magic
2) Suck it up: if the framework is unobtrusive, then it may be more practical to accept the tradeoffs that it imposes and live with them. To some degree, event frameworks are like object relational mappers or databases; sure, in theory you should be able to change them out freely. In practice? how often do you derive benefit from the investment in that flexibility
3) Interfaces: if you squint a little bit, you can see that your domain behaviors don't usually depend on in memory representations, but instead on the algebra of the domain itself.
For example, in the domain model, we deposit Money into an Account updating its Balance. We don't typically care whether those are integers, or longs, or floats, or JSON documents. We can satisfy the model with any implementation that satisfies the constraints of the algebra.
So you can use the framework to provide the implementation (which also happens to have all the hooks the framework needs); the behavior just interacts with the interface it defined itself.
In a strongly typed implementation, this can get really twisty. In Java, for instance, if you want the strong type checks you need to be comfortable with the magic of generics and type erasure.
The real answer to this is that DDD is overrated. It is not true that you have to have one model to rule them all. You may have different views on the state of your world, depending on your current needs. One part of the application has one view, another part - completely different view.
To put it another way, your model is not "what is", but "what happened so far". The actual data model of your application is the event stream itself. Everything else you derive from there.
I am just starting out with ES/DDD and I have a question how one is supposed to do reporting in this architecture. Lets take a typical example, where you have a Customer Aggregate, Order Aggregate, and Product Aggregate all independent.
Now if i want to run a query across all 3 aggregates and/or services, but that data is each in a separate DB, maybe one is SQL, one is a MongoDB, and one something else. How is one supposed to design or be able to run a query that would require a join across these aggregates ?
You should design the Reporting as a simple read-model/projection, possible in its own bounded context (BC), that just listen to the relevant events from the other bounded contexts (Customer BC, Ordering BC and Inventory BC) and builds the needed reports with full data denormalization (i.e. at query time you won't need to query the original sources).
Because of events you won't need any joins as you could maintain a private local state attached to the Reporting read-model in which you can store temporary external models and query those temporary read-models as needed thus avoiding external additional queries to the other BCs.
An anti-corruption layer would not be necessary in this case as there would be no write-model involved in the Reporting BC.
Things are really as simple as that because you already have an event-driven architecture (you use Event sourcing).
UPDATE:
This particular solution is very handy in creating new reports that you haven't thought ahead of time. Every time you thing about a new report you just create a new Read-model (as in you write its source code) then you replay all the relevant events on it. Read-models are side-effect free, you can replay all the events (from the beggining of time) any time and as many time you want.
Read-model rebuilding is done in two situations:
you create a new Read-model
you modify an existing one by listening to a new event or the algorithm differs too much from the initial version
You can read more here:
DDD/CQRS specialized forum - Grey Young is there!
Event sourcing applied – the read model
Writing an Event-Sourced CQRS Read Model
A post in first group describing Read Model rebuilding
Or you can search about this using this text: event sourcing projection rebuilding
Domain-Driven Design is more concerned with the command side of things. You should not attempt to query your domain as that leads to pain and suffering.
Each bounded context may have its own data store and that data store may be a different technology as you have stated.
For reporting you would use a reporting store. How you get data into that store would either require each bounded context to publish events that the reporting BC would pick up and use to update the reporting store or you could make use of event sourcing where the reporting store would project the events into the relevant reporting structures.
There are known practices to solve this.
One might be having a reporting context, which, as Eben has pointed out, will listen to domain events from other contexts and update its store. This of course will lead to issues, since this reporting context will be coupled to all services it reports from. Some might say this is a necessary evil but this is not always the case.
Another technique is to aggregate on-demand. This is not very complex and can be done on different layers/levels. Consider aggregation on the web API level or even on the front-end level, if your reporting is on the screen (not sent by mail as PDF, for example).
This is well known as UI composition and Udi Dahan has wrote an article about this, which is worth reading: UI Composition Techniques for Correct Service Boundires. Also, Mauro Servienti has wrote a blog post about this recently: The secret of better UI composition.
Mauro mentions two types of composition, which I mentioned above. The API/server-side composition is called ViewModel Composition in his post, and front-end (JavaScript) composition is mentioned in the Client side composition process section. Server-side composition is illustrated by this picture:
DDD strategic modeling tools says:
Design two different models 1. Write Models (Handles Command Side) 2.Read Models (POCOs/POJOs) whatever u call them.
I came across that Event Sourcing assumes total encapsulation. Aggregates dosen`t allow to access their internal state. State is internaly kept only to impose valid transions. As far as I grasp this aggregates (in terms of outside world) just emits events. And I cant get my head around that actualy. I refine my models to reflect my bussiness needs which leads to objects that publish some API. For example, I have two aggregate roots: cart and order. I would like to build my order using ActiveItems from cart:
$order->addItems($cart->getActvieItems)
But this violates ES assumtion about total encapsulation of aggregate state. How order should be fulfilled with ActiveItmes according to ES good practices? Should I use read model? I think this leads to knowleadge leak out of the model (aggregate). Thank you in advance!
Alexey is right in that the Event Sourcing is just a persistence mechanism. I think the confusion comes when thinking about Aggregates. Encapsulation is an important concept when thinking about Aggregates. The implication here is that they are not used for query or the UI. Hence the reason CQRS fits in so well.
But most applications need to query the data or display things on the UI. And that's where Read Models come in handy. Assuming you are using CQRS and Event Sourcing (which you don't have to when using Aggregates) it's a fairly easy thing to do. The idea is to subscribe to the events and update the Read Model as you go. This doesn't 'leak' anything because the functionality is in the Aggregate domain objects.
Why is this a good thing?
Have no or extremely limited dependencies makes the aggregate's much simpler to work with.
Read models can be highly optimised for reading from and therefore very fast.
Read models don't require complex queries and joins.
There is a clear separation of concerns
This approach offers huge scaling potential
It's easy to test
I'm sure there are more. If it helps I have a blog post outlining a typical CQRS and ES architecture. You may find it helpful. You can find it here: CQRS + Event Sourcing – A Step by Step Overview
Event Sourcing does not assume anything in addition to the fact that you save the state of your object as series of events. There is even no requirements to have an "aggregate" when doing Event Sourcing.
If you are talking about the DDD terms Aggregate and Aggregate Root, again, Event Sourcing is just a way to save the object as a stream of events instead of the last actual state. There are no additionally imposed "requirements" like "total encapsulation" and inaccessibility of the internal state. Of course aggregates (and other objects) have state.
What could be confusing is that if you also use CQRS, you can have your aggregate state not being used since all its data is transient to the read model. But this is something else and does not need to be blindly applied.
Again, Event Sourcing is just a persistence method, nothing more, and nothing less.
$order->addItems($cart->getActvieItems)
In addition to the comprehensive coverage of CQRS in the answers, I'd like to point out that in the message driven systems commands(messages) should be self-contained and have all information encapsulated necessary to perform the action.
In the above example, Order aggregate receives command AddItems with the list of Items Ids as a payload. The fact that AddItems command handler needs to get additional information to handle the command points to a problem. AddItems command has no sufficient payload so Cart and Order are semantically coupled. You would want to avoid that.
Message passing reasoning is the key here. Here are some abstractions
class AddItems : Command
{
List ItemIds {get; set;}
}
class Order
{
void AddItems(AddItems command){}
}
Question: what is the best, efficient and future proof way to
rehydrate an aggregate from a repository? What are the pro's and con's of the provided ways and are my perceptions correct?
Let's say we have an Aggregate Root with private setters but public getters for accessing state
Behaviour is done through methods on the aggregate root.
A repository is instructed to load an aggregate.
At the moment I see a couple of possible ways to achieve this:
set the state through reflection (manual or automatic eg.
Automapper)
make constructors that accept properties so state is set
load the aggregate with a state object
1) Jimmy Bogard alludes that his tool Automapper isn't meant for two-way mapping. But some people argue that we have to be pragmatic, use tools in a way it helps you.
For me, I don't like a full rehydration through reflection. Maybe Automapper ceise to exist or aggregate roots are bent in such a way the mapping can be done (see some comments of Vaughn on his article).
2) creating constructors for rehydration, with a couple of parameters so the state of the aggregate is rehydrated in a correct way.
These couple of parameters can expand (= new constructors) or the definition can change. I like this approach, except the part of having a bunch of parameters.
3) the state is a property of the aggregate root. The state is encapsulated in a new object and this object is build by the repository and is then given to the aggregate root for proper init.
Some people argue that building this state object is more work (new class, exposure of state properties on entity and aggregate root to enforce business rules), but it provides a clean way to initiliaze the state.
Say that we need event sourcing, does the loading of a state resemble in loading events? And does the state object provide a way of handling events? Is it more future proof?
I would argue that trying to future-proof too much represents a trap that many people fall into that adds undue complexity to a codebase. There is a fine balancing act between sound architectural decisions and over-architecting a solution to problem that is not guaranteed to exist.
That being said, I fully agree with what Jimmy says, in regards to AutoMapper not being intended for two-way mapping. Your domain represents the "truth" in your application, and should not be directly mutable. I have worked on projects with two-way mappings, and while they do work, there is a tendency to start treating the domain objects as nothing more than DTOs. It becomes painful when you start having read-only properties, having to reflect to do your setting - tooling or not. From a DDD perspective, we should not be allowing for outside influences to simply say what a property value should be, because it will lead to an anemic domain model over time, most likely.
Internal states do work well, but they are at the cost of additional overhead and complexity. There is a legitimate trade-off, as you mention, in that you are adding a fair amount of work. However, you can use that opportunity to allow the aggregate to validate the state against the self-contained business rules within the aggregate, prior to allowing the state to be set. That addresses the largest concern that I have with two-way mapping. You can at least enforce that a state object contains valid data and then only construct the aggregate if it is valid. It is more testable, as well. The largest problem that I have seen with this approach is that the skill level of your team will have a direct bearing on the success of this being utilized correctly. It could be argued that the complexity does not add enough value to implement domain-wide, as you will likely have aggregates that have different levels of churn. A couple of projects that I have been involved in have used this approach, and I found little advantage over straight constructor usage.
Normally, I use constructors for rehydration in most cases. It walks the line between not being overly-complex, plus it leaves responsibility for the aggregate to allow or disallow the construction of the object - again, allowing for the domain to be in control of whether the hydration attempt would result in a valid object. A good compromise to constructor bloat is the use a mutable DTO as a parameter for the constructor, essentially acting as a data structure to maintain a consistent constructor signature over time. In that essence, it is also somewhat future-proof. It takes the most attractive perk of the state object approach, which is the clean signatures, but removes the additional layer of an internal abstraction.
You mention event sourcing as a possibility down the road. State loading is not very similar to what you would be doing, at all (in my opinion). With a state object, you are snapshotting the state of the aggregate at a given point in time. With event sourcing, you will be replaying events, each of which represents the data required to mutate the state, as opposed to the state, itself. As such, your constructor will likely be a collection of events, representing a chain of deltas to mutate the state repeatedly, until it reaches the current state. When you want to hydrate your aggregate, you will supply it with the events that are related to that aggregate, and it will replay them to get to the current state. This is one of the true strengths of event sourcing, as well. You are forcing the hydration of your domain objects to go through the business logic required to create them, each time. Given a list of events, the aggregate will enforce that each state change is valid by applying the event in a consistent fashion, whether the event is being applied in real-time, or replayed to get to the current state.
Back to the future-proof aspect, as it relates to event sourcing, there is a conscious effort required when events require change. Since you have to replay an event to get to the current state, you will very likely have to deprecate events and bring up new events to transition to as your business logic changes. You may (read as "likely will") find yourself versioning events. Not only does your aggregate need to understand current state change requirements, but it also needs to understand previous state change requirements. So, if you change an event handler, you will have to ensure that it will be valid for existing events, as well. When you are adding additional data to an event, it is usually not too involved. But when you start removing data from an event signature, you instantly make that event at risk for being incompatible with earlier structures. Likewise, even changing the names of the data structures inside of an event can cause backwards compatibility issues. If you start event sourcing, you do not need to worry as much about future-proofing as you do backwards compatibility. Event sourcing is great, but be prepared for additional complexity.
Suppose we have a situation when we need to implement some domain rules that requires examination of object history (event store). For example we have an Order object with CurrentStatus property, and we need to examine Order.CurrentStatus changes history.
Most likely you will answer that I need to move this knowledge to domain and introduce Order.StatusHistory property that contains a collection of status records, and that I should not query event store. And I will agree with you.
What I question is the need of Event Store.
We write in event store events that has business meaning (domain value), we do not record UserMovedMouse events (in most cases). And as with OrderStatusChanged event there is a high chance that most of events from EventStore will be needed at some point for domain logic, and we end up with a domain object that have a EventHistory property with the collection of events.
I can see a value in separate event store for patterns such as CQRS when you have a single write only event store and multiple read only query stores, which gives you some scalability. However the need to to introduce such thing in code is in question too for me. All decent databases support single write server, multiple read servers scalability (master-slave replication). Why should I introduce such thing at source code level? Why not to forget about Web Services, and Message buses and use write your own wrapers around Sockets.
I have a great respect to "old school" DDD as it was described be Eric Evans, and I see some fresh and good ideas in new wave DDD+SQRC+EventSourcing pattern aggregate. However the main idea of CQRS is under big question for me. Am I missing something?
In short: if event sourcing is not needed (for its added benefits or as workarounds for some quirks), then you definitely shouldn't bring it into your system just for the sake of it.
ES is just one of many ways to augment CQRS architectural style within a bounded context. It is not a requirement.