I am developing an application that has employee time tracking module. When employee starts working (e.g. at some abstract machine), we need to save information about him working. Each day lots of employees work at lots of machines and they switch between them. When they start working, they notify the system that they have started working. When they finish working - they notify the system about it as well.
I have an aggregate Machine and an aggregate Employee. These two are aggregate roots with their own behavior. Now I need a way to build reports for any given Employee or any given Machine for any given period of time. For example, I want to see which machines did given employee used over period of time and for how long. Or I want to see which employees worked at this given machine for how long over period of time.
Ideally (I think) my aggregate Machine should have methods startWorking(Employee employee) and finishWorking(Employee employee).
I created another aggregate: EmployeeWorkTime that stores information about Machine, Employee and start,finish timestamps. Now I need a way to modify one aggregate and create another at the same time (or ideally some another approach since this way it's somewhat difficult).
Also, employees have a Shift that describes for how many hours a day they must work. The information from a Shift should be saved in EmployeeWorkTime aggregate in order to be consistent in a case when Shift has been changed for given Employee.
Rephrased question
I have a Machine, I have an Employee. HOW the heck can I save information:
This Employee worked at this Machine from 1.05.2017 15:00 to 1.05.1017 18:31.
I could do this simply using CRUD, saving multiple aggregates in one transaction, going database-first. But I want to use DDD methods to be able to manage complexity since the overall domain is pretty complex.
From what I understand about your domain you must model the process of an Employee working on a machine. You can implement this using a Process manager/Saga. Let's name it EmployeeWorkingOnAMachineSaga. It work like that (using CQRS, you can adapt to other architectures):
When an employee wants to start working on a machine the EmployeeAggregate receive the command StartWorkingOnAMachine.
The EmployeeAggregate checks that the employee is not working on another machine and if no it raises the EmployeeWantsToWorkOnAMachine and change the status of the employee as wantingToWorkOnAMachine.
This event is caught by the EmployeeWorkingOnAMachineSaga that loads the MachineAggregate from the repository and it sends the command TryToUseThisMachine; if the machine is not vacant then it rejects the command and the saga sends the RejectWorkingOnTheMachine command to the EmployeeAggregate which in turns change it's internal status (by raising an event of course)
if the machine is vacant, it changes its internal status as occupiedByAnEmployee (by raising an event)
and similar when the worker stops working on the machine.
Now I need a way to build reports for any given Employee or any given Machine for any given period of time. For example, I want to see which machines did given employee used over period of time and for how long. Or I want to see which employees worked at this given machine for how long over period of time.
This should be implemented by read-models that just listen to the relevant events and build the reports that you need.
Also, employees have a Shift that describes for how many hours a day they must work. The information from a Shift should be saved in EmployeeWorkTime aggregate in order to be consistent in a case when Shift has been changed for given Employee
Depending on how you want the system to behave you can implement it using a Saga (if you want the system to do something if the employee works more or less than it should) or as a read-model/report if you just want to see the employees that do not conform to their daily shift.
I am developing an application that has employee time tracking module. When employee starts working (e.g. at some abstract machine), we need to save information about him working. Each day lots of employees work at lots of machines and they switch between them. When they start working, they notify the system that they have started working. When they finish working - they notify the system about it as well.
A critical thing to notice here is that the activity you are tracking is happening in the real world. Your model is not the book of record; the world is.
Employee and Machine are real world things, so they probably aren't aggregates. TimeSheet and ServiceLog might be; these are the aggregates (documents) that you are building by observing the activity in the real world.
If event sourcing is applicable there, how can I store domain events efficiently to build reports faster? Should each important domain event be its own aggregate?
Fundamentally, yes -- your event stream is going to be the activity that you observe. Technically, you could call it an aggregate, but its a pretty anemic one; easier to just think of it as a database, or a log.
In this case, it's probably just full of events like
TaskStarted {badgeId, machineId, time}
TaskFinished {badgeId, machineId, time}
Having recorded these events, you forward them to the domain model. For instance, you would take all of the events with Bob's badgeId and dispatch them to his Timesheet, which starts trying to work out how long he was at each work station.
Given that Machine and Employee are aggregate roots (they have their own invariants and business logic in a complex net of interrelations, timeshift-feature is only one of the modules)
You are likely to get yourself into trouble if you assume that your digital model controls a real world entity. Digital shopping carts and real world shopping carts are not the same thing; the domain model running on my phone can't throw things out of my physical cart when I exceed my budget. It can only signal that, based on the information that it has, the contents are not in compliance with my budgeting policy. Truth, and the book of record are the real world.
Greg Young discusses this in his talk at DDDEU 2016.
You can also review the Cargo DDD Sample; in particular, pay careful attention to the distinction between Cargo and HandlingHistory.
Aggregates are information resources; they are documents with internal consistency rules.
Related
Question is related to CQRS - I have user that wants to order something from web and is presented with GUI showing his balance = 100$ and stock = 1 item. Let's say we have 2 services here AccountService and StockService with separate concerns. In order to generate GUI for client third service AggregatorService listens to domain events from AccountService and StockService, projects a view and creates GUI for clients.
When user decides to order this item, he clicks a button and Command for order is sent to AccountService. Here we load AccountAggregate in order to decrease balance for the price of the item that needs to be ordered. But before I can do this, I have to check if the item is still available (or somehow to reserve it). Only thing that comes up to my mind is:
Read current stock of the item from read model of StockService, but what can happen is that other services read model is updated just a second after I read it (e.g. somebody bought the item, so actual stock is =0. but read model still has =1).
Before decreasing a balance call some method on StockService to reserve the item for some time. If order is not successful (e.g. no enough funds on balance, I would have to un-reserve it somehow). This needs to be some sync-REST call and it is probably slower than some async solution (if any).
Are there any best practices for this kind of use-case?
You have 2 options, depending on whether you embrace eventual consistency or not.
Using immediate consistency I would have an OrderService that receives the order request and it makes async calls to AccountService.ReservePayment() and StockService.ReserveStock(). If either of those fail you call AccountService.UndoReservePayment() and StockService.UndoReserveStock(). If both succeed you call AccountService.CompleteReservePayment() and StockService.CompleteReserveStock(). Make sure each reservation should have a timestamp on it so a daemon process can occasionally run and Undo any reserves that are older than an hour or so.
This approach makes the user wait until both those services complete. If either the StockService or the AccountService are slow, the user experience is slow. I suggest a better way to do this is the eventual consistency approach which gives the user a very fast experience at the expense of receiving failure messages after the fact.
With eventual consistency you assume they have enough credit and you have enough inventory, and in response to their order request you immediately send back a success message. The user is happy and they go along to buy more stuff.
The OrderCreated event is then handled asynchronously to reserve stock and credit as described above. However, since there is no time pressure to reply to the waiting user you don’t have to scale up to handle as high a throughput. If the credit check and stock check take a minute or two, the user doesn’t care because he’s off doing other things.
The price you pay here is that the user may get a success message at the time of order submission, then a few minutes later get an email saying the sale didn’t go through after all because there’s no stock. This is what many large retailers do, including Amazon, Target, Walmart, etc. Eventual consistency can go a long way towards easing the load and cost of the back end.
Suppose there are two microservices: Order and Inventory. There is an API in order service that takes ProductId, Qty etc and place the order.
Ideally order should only be allowed to place if inventory exists in inventory service. People recommend to have Saga pattern or any other distributed transactions. That is fine and eventually consistency will be utilized.
But what if somebody wants to abuse the system. He can push orders with products (ProductIds) which are either invalid or out of inventory. System will be taking all these orders and place these orders in queue and Inventory service will be handling these invalid order.
Shouldn't this be handled upfront (in order service) rather than pushing these invalid orders to the next level (specially where productId is invalid)
What are the recommendations to handle these scenarios?
What are the recommendations to handle these scenarios?
Give your order service access to the data that it needs to filter out undesirable orders.
The basic plot would be that, while the Inventory service is the authority for the state of inventory, your Orders service can work with a cached copy of the inventory to determine which orders to accept.
Changes to the Inventory are eventually replicated into the cache of the Orders service -- that's your "eventual consistency". If Inventory drops off line for a time, Order's can continue providing business value based on the information in its cache.
You may want to be paying attention to the age in the data in the cache as well -- if too much time has passed since the cache was last updated, then you may want to change strategies.
Your "aggregates" won't usually know that they are dealing with a cache; you'll pass along with the order data a domain service that supports the queries that the aggregate needs to do its work; the implementation of the domain service accesses the cache to provide answers.
So long as you don't allow the abuser to provide his own instance of the domain service, or to directly manipulate the cache, then the integrity of the cached data is ensured.
(For example: when you are testing the aggregate, you will likely be providing cached data tuned to your specific test scenario; that sort of hijacking is not something you want the abuser to be able to achieve in your production environment).
You most definitely would want to ensure up-front that you can catch as many invalid business cases as possible. There are a couple ways to deal with this. It is the same situation as one would have when booking a seat on an airline. Although they do over-booking which we'll ignore for now :)
Option 1: You could reserve an inventory item as part of the order. This is more of a pessimistic approach but your item would be reserved while you wait for the to be confirmed.
Option 2: You could accept the order only if there is an inventory item available but not reserve it and hope it is available later.
You could also create a back-order if the inventory item isn't available and you want to support back-orders.
If you go with option 1 you could miss out on a customer if an item has been reserved for customer A and customer B comes along and cannot order. If customer A decides not to complete the order the inventory item becomes available again but customer B has now gone off somewhere else to try and source the item.
As part of the fulfillment of your order you have to inform the inventory bounded context that you are now taking the item. However, you may now find that both customer A and B have accepted their quote and created an order for the last item. One is going to lose out. At this point the one not able to be fulfilled will send a mail to the customer and inform them of the unfortunate situation and perhaps create a back-order; or ask the customer to try again in X-number of days.
Your domain experts should make the call as to how to handle the scenarios and it all depends on item popularity, etc.
I will not try to convince you to not do this checking before placing an order and to rely on Sagas as it is usually done; I will consider that this is a business requirement that you must implement.
This seems like a new sub-domain to me: bad-behavior-prevention (or how do you want to call it) that comes with a new responsibility: to prevent abusers. You could add this responsibility to the Order microservice but you would break the SRP. So, it should be done in another microservice.
This new microservice is called from your API Gateway (if you have one) or from the Orders microservice.
If you do not to add a new microservice (from different reasons) then you could implement this new functionality as a module inside of the Orders microservice but I strongly recommend to make it highly decoupled from its host (separate and private persistence/database/table).
I've a design problem that is turning around in my mind since a while and I'm not finding a good solution about it. It's about CQRS and domain boundaries.
Let's say I've a context, that's about taking bookings, and consequently events, for a system. The system allows to have a single booking linked to single event (that's already done, no problems), and weekly bookings linked to a collection of events. Weekly bookings are done defining a day of the week (extra data is not relevant); a weekly booking has always a starting and ending day (half year).
The system has also two types of days: normal days and not working days, where an event cannot be held.
As a business request the users want that for every weekly booking the system alone cancels that particular events that are hold on not working days.
Actually bookings and events are stored in two tables. An event is canceled when it's stored with a special flag. I've no link with the table of the days because I've never used it in my business context. As business boundary (with other small data, not relevant here) this was working great, up to now.
Here is my problem: to satisfy the users request (create an event for every deleted day), I need informations about all the days of half year (just the ones in the same weekday are enough). But, to obtain this information, how can I proceed?
My possible solutions:
Load all the days of half year in the root entity. This could be really heavy, and I've to extend my business boundary.
Preprocess the command, creating one with extra informations. It would be a command in command, something I've read being dangerous. That's enough for me.
Extend the command with the list of invalid days. How I check that a day is invalid? I've to access data outside my actual boundary, that's the same as 1.
Create a service that is used in the command handler to get the list of the not working days. The days context information would be moved in a common (or shared) context.
Create an event listener for weekly events. When a weekly event is created, it loads the list of not working days (for that weekday of that half year) and fires a sequence of commands to cancel that particular days. This would seal the boundaries, not adding extra data to a common context and reuse same code (cancel event) for extra purposes.
Which would be the best solution?
Litmus test : ask your stakeholders if it ever happens that a working day becomes a non-working day, and what is supposed to happen to weekly bookings on those days. Also, does it ever happen that a non-working day becomes a working day, and what is supposed to happen to bookings on those days.
Create an event listener for weekly events. When a weekly event is created, it loads the list of not working days (for that weekday of that half year) and fires a sequence of commands to cancel that particular days. This would seal the boundaries, not adding extra data to a common context and reuse same code (cancel event) for extra purposes.
Close, based on my understanding of what you have written.
To my mind, you really have two different aggregates; you have the definition of the weekly booking, and you have daily schedules which collect events from different bookings.
When you create a booking, your inputs are a start date, and end date, a day of the week, and probably a domain service that can return a list of days of the week in that range. Think schedule or itinerary -- we're defining the candidate days for this particular booking.
You event listener, upon seeing a new booking, fires a command to the schedule aggregate for that particular day, adding the event requested by the daily booking. Because the schedule knows whether or not it is a "non-working day", it can mark each of those events as scheduled or cancelled (if you want that information to be explicit; you could leave it implied by the state of the working day in some systems).
Empty schedules can be created in advance, or on demand using some generic recipe to determine whether or not they are working days, and can support changes to their own working status if that's part of your current domain.
The key ideas here are that non-working days are a part of your domain model, and since they span multiple booking objects, they clearly are an entity that sits outside of the booking aggregate.
We are using CQRS with EventSourcing.
In our application we can add resources(it is business term for a single item) from ui and we are sending command accordingly to add resources.
So we have x number of resources present in application which were added previously.
Now, we have one special type of resource(I am calling it as SpecialResource).
When we add this SpecialResource , id needs to be linked with all existing resources in application.
Linked means this SpecialResource should have List of ids(guids) (List)of existing resources.
The solution which we tried to get all resource ids in applcation before adding the special
resource(i.e before firing the AddSpecialResource command).
Assign these List to SpecialResource, Then send AddSpecialResource command.
But we are not suppose to do so , because as per cqrs command should not query.
I.e. command cant depend upon query as query can have stale records.
How can we achieve this business scenario without querying existing records in application?
But we are not suppose to do so , because as per cqrs command should not query. I.e. command cant depend upon query as query can have stale records.
This isn't quite right.
"Commands" run queries all the time. If you are using event sourcing, in most cases your commands are queries -- "if this command were permitted, what events would be generated?"
The difference between this, and the situation you described, is the aggregate boundary, which in an event sourced domain is a fancy name for the event stream. An aggregate is allowed to run a query against its own event stream (which is to say, its own state) when processing a command. It's the other aggregates (event streams) that are out of bounds.
In practical terms, this means that if SpecialResource really does need to be transactionally consistent with the other resource ids, then all of that data needs to be part of the same aggregate, and therefore part of the same event stream, and everything from that point is pretty straight forward.
So if you have been modeling the resources with separate streams up to this point, and now you need SpecialResource to work as you have described, then you have a fairly significant change to your domain model to do.
The good news: that's probably not your real requirement. Consider what you have described so far - if resourceId:99652 is created one millisecond before SpecialResource, then it should be included in the state of SpecialResource, but if it is created one millisecond after, then it shouldn't. So what's the cost to the business if the resource created one millisecond before the SpecialResource is missed?
Because, a priori, that doesn't sound like something that should be too expensive.
More commonly, the real requirement looks something more like "SpecialResource needs to include all of the resource ids created prior to close of business", but you don't actually need SpecialResource until 5 minutes after close of business. In other words, you've got an SLA here, and you can use that SLA to better inform your command.
How can we achieve this business scenario without querying existing records in application?
Turn it around; run the query, copy the results of the query (the resource ids) into the command that creates SpecialResource, then dispatch the command to be passed to your domain model. The CreateSpecialResource command includes within it the correct list of resource ids, so the aggregate doesn't need to worry about how to discover that information.
It is hard to tell what your database is capable of, but the most consistent way of adding a "snapshot" is at the database layer, because there is no other common place in pure CQRS for that. (There are some articles on doing CQRS+ES snapshots, if that is what you actually try to achieve with SpecialResource).
One way may be to materialize list of ids using some kind of stored procedure with the arrival of AddSpecialResource command (at the database).
Another way is to capture "all existing resources (up to the moment)" with some marker (timestamp), never delete old resources, and add "SpecialResource" condition in the queries, which will use the SpecialResource data.
Ok, one more option (depends on your case at hand) is to always have the list of ids handy with the same query, which served the UI. This way the definition of "all resources" changes to "all resources as seen by the user (at some moment)".
I do not think any computer system is ever going to be 100% consistent simply because life does not, and can not, work like this. Apparently we are all also living in the past since it takes time for your brain to process input.
The point is that you do the best you can with the information at hand but ensure that your system is able to smooth out any edges. So if you need to associate one or two resources with your SpecialResource then you should be able to do so.
So even if you could associate your SpecialResource with all existing entries in your data store what is to say that there isn't another resource that has not yet been entered into the system that also needs to be associated.
It all, as usual, will depend on your specific use-case. This is why process managers, along with their state, enable one to massage that state until the process can complete.
I hope I didn't misinterpret your question :)
You can do two things in order to solve that problem:
make a distinction between write and read model. You know what read model is, right? So "write model" of data in contrast is a combination of data structures and behaviors that is just enough to enforce all invariants and generate consistent event(s) as a result of every executed command.
don't take a rule which states "Event Store is a single source of truth" too literally. Consider the following interpretation: ES is a single source of ALL truth for your application, however, for each specific command you can create "write models" which will provide just enough "truth" in order to make this command consistent.
Problem:
Two employees (A & B) go off-line at the same time while editing customer #123, say version #20, and while off-line continue making changes...
Scenarios:
1 - The two employees edit customer #123 and make changes to one or more identical attributes.
2 - The two employees edit customer #123 but DO NOT make the same changes (they cross each other without touching).
... they then both come back on-line, first employee A appends, thereby changing the customer to version #21, then employee B, still on version #20
Questions:
Who's changes do we keep in scenario 1?
Can we do a merge in scenario 2, how?
Context:
1 - CQRS + Event Sourcing style system
2 - Use Event Sourcing Db as a Queue
3 - Eventual Consistency on Read Model
4 - RESTful APIs
EDIT-1: Clarifications based on the answers so far:
In order to perform fined grained merging, I'll need to have one command for each of field in a form for example?
Above, finely grained commands for ChangeName, ChangeSupplier, ChangeDescription, etc., each with their own timestamp would allow for auto-merging in the event A & B both updated ChangedName?
Edit-2: Follow up based on the the use of a particular event store:
It seems as though I'll make use of #GetEventStore for the persistence of my event streams.
They make use of Optimistic Concurrency as follows:
Each event in a stream increments stream version by 1
Writes can specify an expected version, making use of the ES-ExpectedVersion header on writers
-1 specifies stream should not already exist
0 and above specifies a stream version
Writes will fail if the stream is not at the version, you either retry with a new expected version number or you reprocessed the behavior and decided it's OK if you so choose.
If no ES-Expected Version specified, optimistic concurrency control is disabled
In this context, the Optimistic Concurrency is not only based on the Message ID, but also on the Event #
If I understand your design picture correctly, then the occasionally connected users enqueue commands, i.e., change requests, and when the user reconnects the queued commands are sent together; there is only one database authority (that the command handlers query to load the most recent versions of their aggretates); only the view model is synced to the clients.
In this setup, Scenario 2 is trivially auto-merged by your design, if you choose your commands wisely, read: make them fine-grained: For every possible change, choose one command. Then, on re-connection of the client, the commands are processed in any order, but since they only affect disjunct fields, there is no problem:
Customer is at v20.
A is offline, edits changes against stale model of v20.
B is offline, edits changes against stale model of v20.
A comes online, batch sends an queued ChangeName command, the Customer of v20 is loaded and persisted as v21.
B comes online, batch sends an queued ChangeAddress command, the Customer of v21 is loaded and persisted as v22.
The database contains the user with their correct name and address, as expected.
In Scenario 1, with this setup, both employees will overwrite the other employees' changes:
Customer is at v20.
A is offline, edits changes against stale model of v20.
B is offline, edits changes against stale model of v20.
A comes online, batch sends an queued ChangeName command to "John Doe", the Customer of v20 is loaded and persisted as v21 with name "John Doe"
B comes online, batch sends an queued ChangeName command to "Joan d'Arc", the Customer of v21 (named "John Doe") is loaded and persisted as v22 (with name "Joan d'Arc').
Database contains a user with name "Joan d'Arc".
If B comes online before A, then it's vice versa:
Customer is at v20.
A is offline, edits changes against stale model of v20.
B is offline, edits changes against stale model of v20.
B comes online, batch sends an queued ChangeName command to "Joan d'Arc", the Customer of v20 is loaded and persisted as v21 (with name "Joan d'Arc').
A comes online, batch sends an queued ChangeName command to "John Doe", the Customer of v21 is loaded and persisted as v22 with name "John Doe".
Database contains a user with name "John Doe".
There are two ways to enable conflict detection:
Check whether the command's creation date (i.e., the time of the employees modification) is after the last modification date of the Customer. This will disable the auto-merge feature of Scenario 2, but will give you full conflict detection against concurrent edits.
Check whether the command's creation date (i.e., the time of the employees modification) is after the last modification date of the individual field of the Customer it is going to change. This will leave the auto-merge of Scenario 2 intact, but will give you auto-conflict-detection in Scenario 1.
Both are easy to implement with event sourcing (since the timestamps of the individual events in the event stream are probably known).
As for your question "Who's changes do we keep in scenario 1?" -- this depends on your business domain and its requirements.
EDIT-1: To answer on the clarification question:
Yes, you'll need one command for each field (or group of fields, respectively) that can be changed individually.
Regarding your mockup: What you are showing is a typical "CRUD" UI, i.e., multiple form fields and, e.g., one "Save" button. CQRS is usually and naturally combined with a "task based" UI, where there would be, say, the Status field be displayed (read-only), and if a user wants to change the status, one clicks, say, a "Change Status" button, which opens a dialog/new window or other UI element, where one can change the status (in web based systems, in-place-editing is also common). If you are doing a "task based" UI, where each task only affects a small subset of all fields, then finely grained commands for ChangeName, ChangeSupplier etc are a natural fit.
Here's a generic overview of some solutions:
Scenario 1
Someone has to decide, preferably a human. You should ask the user or show that there is a conflict.
Dropbox solves this by picking the later file and keeping a file.conflict file in the same directory for the user to delete or use.
Scenario 2
Keep the original data around and see which fields actually changed. Then you can apply employee 1's changes and then employee 2's changes without stepping on any toes.
Scenario 3 (Only when the changes come online at different times)
Let the second user know that there were changes while they were offline. Attempt Scenario 2 and show the second user the new result (because this might change his inputs). Then ask him if he wants to save his changes, modify them first, or throw them out.
Aaron, where the events do actually conflict, i.e. in scenario 1 then I would expect a concurrency exception of some sort to be thrown.
The second scenario is much more interesting. Assuming your commands and events are reasonably well defined, i.e. not a wrapper for CRUD then you would be able to test if the events committed since your command was issued actually conflict. I use a concurrency conflict registry for this purpose. Essentially when I detect a potential conflict I grab the events that have been committed since the version I currently have and ask the registry to check if any of them actually conflict.
If you want to see a code example and and a bit more detail on this I put together a post outlining my approach. Take a look at it here: handling concurrency issues in cqrs es systems
Hope this helps!
In this case, maybe you can use the "aggregate root" concept, for the Item which powered by CEP Engine (Complex Event Process Engine) to perform these complex operations.