I am implementing micro-services using CQRS and Event-Sourcing.
I have seen different implementations of CQRS which are quite complex.
What I have understood and implemented is I have made two models for Read(Query) and Write(Command), the read model has a materialized view, and write model uses the Database, now whenever an update happens, the write model updates the database and generates an event, and logs the details to event store, which the read model has subscribed, and the read model updates its materialized view by reading from the event.
My question is whether this model is relying to the basis of CQRS and Event Sourcing?
the write model updates the database and generates an event, and logs the details to event store
That doesn't quite sound right. The write model doesn't update a database and an event store, it updates the database which is an event store.
The core idea of CQRS is that processing commands and processing queries can use different data models. If a bit of latency is acceptable, then we write the changes into one data model, then in the background we update the second data model to match the data in the first. Among other things, this allows us to choose data stores that are fit for purpose - if we need to support a bunch of graph queries, then we can use a graph database as part of the read model.
When we add event sourcing to the mix, the above pattern doesn't change. What does change is that the representation of state that we copy into the write store changes from a snapshot of current state to a snapshot of history. So our fit for purpose data store for the write model is an event store.
The event store replaces the snapshot database as the source of truth.
It sounds like you've implemented CQRS but not event sourcing.
CQRS means segregating the write-side (commands) from the read side (queries), which it sounds like you're doing.
However, it sounds like your events are merely a way of communicating from the write side to the read side, while in even sourcing events are the source of truth. A write-side implemented using event sourcing would only persists the events (and optionally snapshots) but wouldn't do so after doing some other updates to the database. The data model on the write side would be composed solely of the journal of events (and optionally snapshots).
Related
I am just starting out with ES/DDD and I have a question how one is supposed to do reporting in this architecture. Lets take a typical example, where you have a Customer Aggregate, Order Aggregate, and Product Aggregate all independent.
Now if i want to run a query across all 3 aggregates and/or services, but that data is each in a separate DB, maybe one is SQL, one is a MongoDB, and one something else. How is one supposed to design or be able to run a query that would require a join across these aggregates ?
You should design the Reporting as a simple read-model/projection, possible in its own bounded context (BC), that just listen to the relevant events from the other bounded contexts (Customer BC, Ordering BC and Inventory BC) and builds the needed reports with full data denormalization (i.e. at query time you won't need to query the original sources).
Because of events you won't need any joins as you could maintain a private local state attached to the Reporting read-model in which you can store temporary external models and query those temporary read-models as needed thus avoiding external additional queries to the other BCs.
An anti-corruption layer would not be necessary in this case as there would be no write-model involved in the Reporting BC.
Things are really as simple as that because you already have an event-driven architecture (you use Event sourcing).
UPDATE:
This particular solution is very handy in creating new reports that you haven't thought ahead of time. Every time you thing about a new report you just create a new Read-model (as in you write its source code) then you replay all the relevant events on it. Read-models are side-effect free, you can replay all the events (from the beggining of time) any time and as many time you want.
Read-model rebuilding is done in two situations:
you create a new Read-model
you modify an existing one by listening to a new event or the algorithm differs too much from the initial version
You can read more here:
DDD/CQRS specialized forum - Grey Young is there!
Event sourcing applied – the read model
Writing an Event-Sourced CQRS Read Model
A post in first group describing Read Model rebuilding
Or you can search about this using this text: event sourcing projection rebuilding
Domain-Driven Design is more concerned with the command side of things. You should not attempt to query your domain as that leads to pain and suffering.
Each bounded context may have its own data store and that data store may be a different technology as you have stated.
For reporting you would use a reporting store. How you get data into that store would either require each bounded context to publish events that the reporting BC would pick up and use to update the reporting store or you could make use of event sourcing where the reporting store would project the events into the relevant reporting structures.
There are known practices to solve this.
One might be having a reporting context, which, as Eben has pointed out, will listen to domain events from other contexts and update its store. This of course will lead to issues, since this reporting context will be coupled to all services it reports from. Some might say this is a necessary evil but this is not always the case.
Another technique is to aggregate on-demand. This is not very complex and can be done on different layers/levels. Consider aggregation on the web API level or even on the front-end level, if your reporting is on the screen (not sent by mail as PDF, for example).
This is well known as UI composition and Udi Dahan has wrote an article about this, which is worth reading: UI Composition Techniques for Correct Service Boundires. Also, Mauro Servienti has wrote a blog post about this recently: The secret of better UI composition.
Mauro mentions two types of composition, which I mentioned above. The API/server-side composition is called ViewModel Composition in his post, and front-end (JavaScript) composition is mentioned in the Client side composition process section. Server-side composition is illustrated by this picture:
DDD strategic modeling tools says:
Design two different models 1. Write Models (Handles Command Side) 2.Read Models (POCOs/POJOs) whatever u call them.
I came across that Event Sourcing assumes total encapsulation. Aggregates dosen`t allow to access their internal state. State is internaly kept only to impose valid transions. As far as I grasp this aggregates (in terms of outside world) just emits events. And I cant get my head around that actualy. I refine my models to reflect my bussiness needs which leads to objects that publish some API. For example, I have two aggregate roots: cart and order. I would like to build my order using ActiveItems from cart:
$order->addItems($cart->getActvieItems)
But this violates ES assumtion about total encapsulation of aggregate state. How order should be fulfilled with ActiveItmes according to ES good practices? Should I use read model? I think this leads to knowleadge leak out of the model (aggregate). Thank you in advance!
Alexey is right in that the Event Sourcing is just a persistence mechanism. I think the confusion comes when thinking about Aggregates. Encapsulation is an important concept when thinking about Aggregates. The implication here is that they are not used for query or the UI. Hence the reason CQRS fits in so well.
But most applications need to query the data or display things on the UI. And that's where Read Models come in handy. Assuming you are using CQRS and Event Sourcing (which you don't have to when using Aggregates) it's a fairly easy thing to do. The idea is to subscribe to the events and update the Read Model as you go. This doesn't 'leak' anything because the functionality is in the Aggregate domain objects.
Why is this a good thing?
Have no or extremely limited dependencies makes the aggregate's much simpler to work with.
Read models can be highly optimised for reading from and therefore very fast.
Read models don't require complex queries and joins.
There is a clear separation of concerns
This approach offers huge scaling potential
It's easy to test
I'm sure there are more. If it helps I have a blog post outlining a typical CQRS and ES architecture. You may find it helpful. You can find it here: CQRS + Event Sourcing – A Step by Step Overview
Event Sourcing does not assume anything in addition to the fact that you save the state of your object as series of events. There is even no requirements to have an "aggregate" when doing Event Sourcing.
If you are talking about the DDD terms Aggregate and Aggregate Root, again, Event Sourcing is just a way to save the object as a stream of events instead of the last actual state. There are no additionally imposed "requirements" like "total encapsulation" and inaccessibility of the internal state. Of course aggregates (and other objects) have state.
What could be confusing is that if you also use CQRS, you can have your aggregate state not being used since all its data is transient to the read model. But this is something else and does not need to be blindly applied.
Again, Event Sourcing is just a persistence method, nothing more, and nothing less.
$order->addItems($cart->getActvieItems)
In addition to the comprehensive coverage of CQRS in the answers, I'd like to point out that in the message driven systems commands(messages) should be self-contained and have all information encapsulated necessary to perform the action.
In the above example, Order aggregate receives command AddItems with the list of Items Ids as a payload. The fact that AddItems command handler needs to get additional information to handle the command points to a problem. AddItems command has no sufficient payload so Cart and Order are semantically coupled. You would want to avoid that.
Message passing reasoning is the key here. Here are some abstractions
class AddItems : Command
{
List ItemIds {get; set;}
}
class Order
{
void AddItems(AddItems command){}
}
I understand that basic stats in the read model can be computed on the fly as events from the domain model are generated. Does this really work for more complicated situations? What happens if new stats are required in the future - do you run past events against the new read model retrospectively? Any feedback related to reporting and cqrs would be very much appreciated.
I suppose you're talking about event sourcing, as CQRS does not necessarily mean you're storing events.
In the case of event sourcing, yes, you can just drop your entire read model and rehydrate it from past events. The idea in event sourcing is that your event stream is the log of everything what happened.
The read model is just a projection of those events. It's possible to change that projection when your insights change (or add projections for that matter). That's one of the great strengths of event sourcing.
Eric Evans talks a lot about evolving models in DDD so refactorings seem to be essential to DDD. When one has a relational persisted state of the world you can handle model changes by migrations that change the database schema.
How can I cope with model changes when using event sourcing? If there are incompatible changes to an aggregate that would prevent replay of events is there some sort of best practice? Or is it a just-don't?
If there are incompatible changes to an aggregate that would prevent replay of events
You have essentially two options in this scenario:
Patch the older events in such a way that they are made compatible and events can be replayed from the beginning. The benefit here is that you don't lose the history but the downside is that you have to expend some effort to patch the old events.
Take a snapshot/memento of the aggregate at the point of the schema change and "re-base" the event stream from this point onward. The benefit here is that you don't have to spend any effort (with event sourcing you most likely have a snapshot mechanism in place). The downside being that you lose the ability to replay events from before the snapshot.
As a general rule of thumb I'd say default to the second option unless you know for sure that you need to be able to go back and edit history before the schema change.
I have not much expierence myself. But I saw a concept called Upcasting
Originally a concept of object-oriented programming, where: "a subclass gets cast to it's superclass automatically when needed", the concept of upcasting can also be applied to event sourcing. To upcast an event means to transform it from its original structure to its new structure. Unlike OOP upcasting, event upcasting cannot be done in full automation because the structure of the new event is unknown to the old event. Manually written Upcasters have to be provided to specify how to upcast the old structure to the new structure.
You can refer to Axon's doc for more detail
Events are just DTOs. It doesn't matter how the model changes as long as you still have one object, if the event itself doesn't change. If you need to change the event, you can 'upgrade' it with the required properties. The Apply method will know what to do with it. I can't come up with something specific without knowing details.
If the model changes so much that basically now you have 2 Aggregate Roots(AR) instead of a previous one, this means you have new different aggregates which won't be using the old events. Basically you start from the old AR, create the new ones and generate the corresponding events which will be specific to those ARs. So you don't really have a compatibility problem in this case.
Working with events is not as straightforward as 'classic' OOP and RDBMS schema, but they are more flexible if you think in business terms and treat your objects as domain concepts. Changing the model means the business concept definition or usage has changed as well, so now you're dealing with a different (new as far as the persistence is concerned) concept.
Suppose we have a situation when we need to implement some domain rules that requires examination of object history (event store). For example we have an Order object with CurrentStatus property, and we need to examine Order.CurrentStatus changes history.
Most likely you will answer that I need to move this knowledge to domain and introduce Order.StatusHistory property that contains a collection of status records, and that I should not query event store. And I will agree with you.
What I question is the need of Event Store.
We write in event store events that has business meaning (domain value), we do not record UserMovedMouse events (in most cases). And as with OrderStatusChanged event there is a high chance that most of events from EventStore will be needed at some point for domain logic, and we end up with a domain object that have a EventHistory property with the collection of events.
I can see a value in separate event store for patterns such as CQRS when you have a single write only event store and multiple read only query stores, which gives you some scalability. However the need to to introduce such thing in code is in question too for me. All decent databases support single write server, multiple read servers scalability (master-slave replication). Why should I introduce such thing at source code level? Why not to forget about Web Services, and Message buses and use write your own wrapers around Sockets.
I have a great respect to "old school" DDD as it was described be Eric Evans, and I see some fresh and good ideas in new wave DDD+SQRC+EventSourcing pattern aggregate. However the main idea of CQRS is under big question for me. Am I missing something?
In short: if event sourcing is not needed (for its added benefits or as workarounds for some quirks), then you definitely shouldn't bring it into your system just for the sake of it.
ES is just one of many ways to augment CQRS architectural style within a bounded context. It is not a requirement.