I am building a DDD/ES system where an admin can generate coupon codes to be shared and used publicly by consumers.
Each coupon has a maximum discount amount that is "shared" by all users of the coupon. An example is "Coupon 123 grants 5% off up for every use up to a total of $100,000." The coupon automatically expires once that maximum amount is reached. The coupon may also have rules around which items are eligible for purchase with that coupon.
So far, I have the Coupon and Purchases as separate ARs. Questions?
How do I update the coupon remaining amount when a purchase is made without updating multiple aggregates in a single transaction? Since this is about money, it feels like eventual consistency could lead to lost revenue. Should a domain service be responsible for that or a Saga?
How do i validate that the items being purchased with the coupon are eligible? Do ask the Coupon AR in the command handler before continuing with the purchase: couponAR.AreValidItems(request.PurchaseDetails)
I would like to track how much of the coupon is allocated to each purchase? Should this be another AR (CouponUsageAR) or just an event on the Coupon AR (CouponUsed)?
Thanks
In practice it is quite difficult to make a transaction (in the database sense) span multiple event streams. A more usual way to do this would be to use the saga pattern with compensation events in the event that the saga fails, and use correlation identifiers in the events to match the different entities they occurred to.
For example you might have a saga which updates the coupon remaining and if that succeeds then goes on to perform the purchase. Should the purchase fail the saga then needs to post a compensating event onto the coupon event stream to get it back to the state it was in at the start of the saga.
Have a look at the online video covering the saga pattern by Udi Dahan - or the documentation of that?
On the subject of strong vs. eventual consistency, ask the domain expert (this is domain-driven design, after all) the question:
which is worse: a coupon code delivering more discounts than its limit, or not being able to take an order?
If the answer is that they'd rather enforce the coupon limit and lose the sale, then strong consistency is called for. If they'd rather take the order, then eventual consistency is called for. It's worth noting that there's an easy way to prevent the coupon codes from ever breaching their limits: never have an order.
Related
I will try to keep this as generic as possible using the “order” and “product” example, to try and help others that come across this question.
The Structure:
In the application we have 3 different services, 2 services that follow the event sourcing pattern and one that is designed for read only having the separation between our read and write views:
- Order service (write)
- Product service (write)
- Order details service (Read)
The Background:
We are currently storing the relationship between the order and product in only one of the write services, for example within order we have a property called ‘productItems’ which contains a list of the aggregate Ids from Product for the products that have been added to the order. Each product added to an order is emitted onto Kafka where the read service will update the view and form the relationships between the data.
The Problem:
As we pull back by aggregate Id for the order and the product to update them, if a product was to be deleted, there is no way to disassociate the product from the order on the write side.
This in turn means we have inconsistency, that the order holds a reference to a product that no longer exists within the product service.
The Ideas:
Master the relationship on both sides, which means when the product is deleted, we can look at the associated orders and trigger an update to remove from each order (this would cause duplication of reference).
Create another view of the data that shows the relationships and use a saga to do a clean-up. When a delete is triggered, it will look up the view database, see the relationships within the data and then trigger an update for each of the orders that have the product associated.
Does it really matter having the inconsistencies if the Product details service shows the correct information? Because the view database will consume the product deleted event, it will be able to safely remove the relationship that means clients will be able to get the correct view of the data even if the write models appear inconsistent. Based on the order of the events, the state will always appear correct in the read view.
Another thought: as the aggregate Id is deleted, it should never be reused which means when we have checks on the aggregate such as: “is this product in the order already?” will never trigger as the aggregate Id will never be repurposed meaning the inconsistency should not cause an issue when running commands in the future.
Sorry for the long read, but these are all the ideas we have thought of so far, and I am keen to gain some insight from the community, to make sure we are on the right track or if there is another approach to consider.
Thank you in advance for your help.
Event sourcing suites very well human and specifically human-paced processes. It helps a lot to imagine that every event in an event-sourced system is delivered by some clerk printed on a sheet of paper. Than it will be much easier to figure out the suitable solution.
What's the purpose of an order? So that your back-office personnel would secure the necessary units at a warehouse, then customer would do a payment and you start shipping process.
So, I guess, after an order is placed, some back-office system can process it and confirm that it can be taken into work and invoicing. Or it can return the order with remarks that this and that line are no longer available, so that a customer could agree to the reduced order or pick other options.
Another option is, since the probability of a customer ordering a discontinued item is low, just not do this check. But if at the shipping it still occurs - then issue a refund and some coupon for inconvenience. Why is it low? Because the goods are added from an online catalogue, which reflects the current state. The availability check can be done on the 'Submit' button click. So, an inconsistency may occur if an item is discontinued the same minute (or second) the order has been submitted. And usually the actual decision to discontinue is made up well before the information was updated in the Product service due to some external reasons.
Hence, I suggest to use eventual consistency. Since an event-sourced entity should only be responsible for its own consistency and not try to fulfil someone else's responsibility.
I am trying to refactor a legacy order handling and stock system with into a cleaner service oriented event-driven architecture. However, I am having some difficulty deciding what service should be responsible for the reservation/allocation of stock.
A brief overview of the current system
Sales orders are placed with us via third party system but we do not necessarily have all order lines in stock.
If an order item is in stock then we allocate/reserve the stock for that order straight away.
However, if we do not have enough stock then we procure the stock from our suppliers via a purchasing system.
When the item arrives from the supplier, the system will search through all open sales orders for the item and reserve/allocate the available stock to them, prioritising by sales order date. ***
I have already identified two services that I think need to be developed
Sales - Responsible for receiving the sales order and inserting into the database. Has domain entities such as Order, OrderLine etc.
Inventory - Responsible for keeping track of how much stock is available in our warehouse. Has domain entities such as StockItem.
However, as the allocation/reservation of stock concerns both inventory and sales I am not sure where the behaviour in point 2 above should be put.
I welcome any help or thoughts on this.
I think you have 2 BCs (bounded contexts): Inventory and Sales. For the integration between them I would probably go for domain events approach.
When a new item arrives at the warehouse, the Inventory BC increments the stock for the item, and publish an event.
Sales BC subscribes to the event, and it updates the opened sales that are waiting for the stock item.
So, behaviour of "point 2" are shared by both BC:
Sales BC search for opened orders waiting for that item. And then it asks Inventory BC to get the number of items it needs (this request is synchronous) and close the order.
Inventory BC receives the request and decrements the stock for the item.
However, as the allocation/reservation of stock concerns both inventory and sales I am not sure where the behaviour in point 2 above should be put.
I've been thinking about this problem (purely academically), and my current conclusion is that reservation management belongs with the inventory system. That keeps the stock source (the loading of items procured from your suppliers) and the stock sink (fulfillment of orders) together.
So the inventory system caches its own copy of the data required to fill the order (allowing it to work autonomously). It should be able to make progress as soon as it is informed that the suppliers have provided new inventory, even if the sales system happens to be down for maintenance.
You mentioned SOA and NServiceBus, so my initial thought was that you've attended Udi Dahan his ADSD training? I'll assume you have. With that, I'll try to answer your question.
So far I don't have a lot of information. But with what we have, I figured we need these properties to store all that you mentioned.
ProductId, one for each available product
InventoryTotal, attached to a ProductId. This number goes up and down
OrderId, to create an order
OrderDate, to make sure we can find the order that should receive incoming stock first.
If you have an OrderId, you can attach one or more ProductId to create an actual order. Different ways of storing this technically. Maybe in a relational database with Order and OrderLine tables, or possibly in a DocumentDb where everything is stored in a single document. That's totally irrelevant at this point.
Assuming we need 4 attributes, I'm not sure why we would create more than 1 service to split this up? This might change when we have more information, but at this moment I don't see the need.
If you want to discuss this, contact us at support#particular.net, mention my name and we can continue the conversation.
You are talking about loosely coupled domain apps, managing your sales orders, managing your inventory and managing your purchase orders.
Inventory must always be up to date, in order to not sell what you can't deliver. So PO en SO app should both talk to inventory via synchronous (inventory) services. To keep everything consistent, events on purchasing side, like receiving less than you expected for a PO, will have an impact on any SO already assigning quantity of that PO, as persisted in inventory. So the same PO pcs for example, in which the event of receiving less as expected, is registered, should synchronously update inventory, to update the quantity available for SOs to assign from, and publish an event, to be picked up, asynchronously, in the So app, so that the user can be notified and talk relevant action. Etc.
This question is more of a general question about how to model simple one-to-many relations using collections: should a change in a list item be reflected in the version of the aggregate containing it?
The domain is about meeting scheduling (like in Outlook).
I have a Meeting entity, which can have multiple Participants.
A participant can accept/decline meeting requests.
Rescheduling a meeting nullifies all of the participants confirmations.
I thought of two ways to model this.
Option 1
The Meeting aggregate will contain a list of Participants where each Participant has a ParticipantId and a Status (accepted/denied).
The problem here is that every Accept or Deny command, for a specific participant, increments the Meeting's version, which means two participants will enter a race condition if trying to Accept the meeting request based on the same original version.
Although this could be solved by re-reading the Meeting's document and retrying the Accept command, it's quite annoying considering how often this could happen.
Another approach is to ignore the meeting's version when executing the Accept command, but this introduces a new problem: what happens if, after sending the meeting requests, the meeting has been rescheduled? In this case we can't afford to ignore the Meeting's version, because this time the version DOES represent a real version that should be considered.
BTW, is it at all a good practice to ignore the version in some of the commands and not in others?
Option 2
Extract a Participation aggregate out of Meeting.
Participation will have MeetingId, ParticipantId, and Status.
It will also have its own version.
This way, when participant X Accepts the meeting request, only the relevant Participation will be modified, and the rest will be left intact.
And, when rescheduling the meeting, a "Meeting Rescheduled" event will be published and an event handler will respond to it by resetting all of the Participations' statuses to "NotAccepted" regardless of their current version.
On the one hand this sounds logical in the sense that a meeting's version shouldn't be incremented just because someone accepted/denied its request.
On the other hand, modeling Participation as a standalone aggregate doesn't sound quite right to me, because it is has no meaning outside of the context of the meeting.
Anyway, would love to get feedback on this and see the various approaches to this problem.
Although this could be solved by re-reading the Meeting's document and retrying the Accept command, it's quite annoying considering how often this could happen.
This looks like a modeling error. You should keep in mind that the meeting aggregate is not the book of record for the participants availability - the real world is. So the message shouldn't be AcceptInvitation, but instead InvitationAccepted. There shouldn't be a conflict about this, because the domain model doesn't get to veto events outside of its authority boundary.
You might, depending on your implementation, end up with a concurrent modification exception in your plumbing, but that's something that you should be handling automatically (ie: expected version any, or a retry).
Another approach is to ignore the meeting's version when executing the Accept command, but this introduces a new problem: what happens if, after sending the meeting requests, the meeting has been rescheduled?
The solution here is to model more carefully. Yes, sometimes you will get a message that accepts or declines an invitation that has expired.
Put another way: race conditions don't exist.
A microsecond difference in timing shouldn’t make a difference to core business behaviors.
What happens to Alice, who replied instantly to the invitation, when the meeting is rescheduled? Why wouldn't the same thing happen to Bob, when his reply arrives just after the meeting is rescheduled?
Participation as a standalone aggregate doesn't sound quite right to me, because it is has no meaning outside of the context of the meeting.
I find that heuristic isn't particularly effective. It's much more important to understand whether entities can change state independently, or if their changes need to be coordinated.
Actually, the Meeting aggregate is used to track the participants availability. That's what it purpose is. Unless I didn't fully understand you...
It's a bit subtle, and I didn't spell it out very well.
Suppose the model says that I'm available, but an emergency in the real world calls me away. What happens? Am I blocked from going to the hospital because the model says I have to go to a meeting? Can somebody cancel my emergency by changing the invitation I've submitted?
Furthermore, if I'm away on an emergency, are you available for a meeting that is scheduled for the same time as the meeting you and I were going to have?
In this space, the real world is the authority for whether or not somebody is available. The model is just looking at a cached copy of a message describing whether or not somebody was available in the past.
The cached information being used by the model is not guaranteed to be complete. See Greg Young on warehouse systems and exception reports.
which makes me think that perhaps the Meeting aggregate should have two version fields: one will be a strong version which, when incremented, represents a breaking change, and another soft version for non-breaking changes. Does this make any sense?
Not really. Version is not, as far as I know, a term taken from the ubiquitous language of scheduling meetings. It's meta data, if it exists at all, and the business rules in your model should not depend upon meta data.
I agree, but a Meeting ID (or any ID for that matter) is also not part of the ubiquitous language, yet I might pass it back and forth between my domain world and external worlds.
I am working on DDD project and I am currently focused on two bouned contexts, Orders and Warehouse.
What confuses me is the following situation:
Order keep track of all the placed orders, and warehouse keeps track about all the available inventory. If user places one order for certain product item, that would mean one less item of that product in a warehouse. I am oversimplifying this process, so please bear with me.
Since two domain models are placed inside of a different BC, i am currently relying on eventual consistency ie. after one item has been sold, it would eventually be removed from the warehouse.
That situation unfortunately leads to the problem scenario where another user could simultaneously make another order of the same item, and it would appear as available until eventual consistency kicks is. That is something it is unacceptable by the domain expert.
So IMO I am stuck with two options
merge order and warehouse (at least the part regarding product
inventory, units available in warehouse) into one BC
have Order BC (or microservice if you wish) to be dependent of Warehouse BC (microservice) in order to pull a live product units
available
Which option does actually follows DDD patern? Is there another way out?
You could use a reservation system with a timeout.
Using a messaging analogy: With a broker-style queuing mechanism (such as RabbitMQ) you get a message from the queue and you have control over it until you either acknowledge that it can be removed from the queue or you release it back to the queue.
You could do the same thing in your ordering process. You reserve any items on your order. SO when you add them they have a status of, say, reserving and upon sending some message to reserve the items. If the response comes back you can decide how to proceed. Perhaps you could add any items that cannot be reserved onto a back order or try again later.
There are going to be different ways to approach this. Depending on your business case it may be acceptable to only check availability when someone really accepts the order.
If you domain expert reckons it is totally unacceptable that having this resolved at the end of the process then you could move it to the start. The issue is of course that user A could reserve and never buy thereby losing user B as a customer; whereas only applying the real "taking" of the item at the end of the process is closer to ensuring a purchase. But that is a business decision.
This issue is a really great example of where reality actually is eventually consistent. Is it really the best thing to decline an order if there is no inventory currently in the warehouse - even if there was a replenishment due in the next 20 minutes?
What if the item was actually on the shelf, but the operator hadn't yet keyed it into the system?
Sometimes designers and domain experts assume that people want 100% consistency, when really, users might be willing to accept a delay in confirmation of their order, if it increased the chance that their order would be accepted rather than rejected.
In the case above, why make it the user's job to retry their order N minutes later? In an eventually consistent system, you can accommodate such timing flexibility by including a timeout to retry the attempt to fulfill the order for a period of time before confirming to the client that it really wasn't possible.
There are technical solutions that will give you 100% consistency, but I think really this is not a technical challenge but a cultural/mindset one, changing people's minds about what is possible & acceptable to achieve an what is actually a better outcome.
IMO you can build a PlaceOrderSaga which will ask for inventory availability before placing the order.
So, not really sure if this is the right place for this but I have this current Context level data flow diagram for the bellow specification extract and I have never done one before so I was wondering if it was correct or if it needs fixing? any help appreciated
This is a link to a screen of my current one http://i.imgur.com/S4xvutc.png
SPECIFICATION
Currently the office staff operate the following processes:
Add/Amend/Delete Membership
This is run on-demand when a new membership application is received or when a member indicates that he/she wishes to make amendments to their details. It is also run in those rare instances when a membership is terminated at the discretion of the manager. A new member has an ID number allocated (simply incremented from the previous membership accepted). A membership balance is also maintained for accounting purposes.
Another process operates in a similar fashion on data associated with transfer partners.
Monthly Maintenance
This is run on the last day of each month to issue requests and reminders for subscriptions due, and to remove memberships where fees remain outstanding. Standard letters are also generated. Membership balances are updated as appropriate.
Payment Updates
This is run prior to the Monthly Maintenance, with membership balances being updated accordingly.
Payments to partners are also disbursed at this time.
New Member Search
This is run whenever a new member has been added to the database. The partners are partitioned in terms of vehicle category and location. Normally, there is a limited choice of partner in a particular location (if, indeed, there is any choice) but for some popular destinations, several partners are involved in providing the airport transfer. Thus, a search is then made through the appropriate section for potential matches in the following manner:
A search is then made on the grounds of sex (many female passengers in particular prefer a driver of their own sex, especially if travelling alone or in couples).
Matches are then selected according to factors such as cost (if available), availability of extra requested facilities (such as child seats, air-conditioning etc.)
Existing Member - Additional Searches
These are run on-demand in the same fashion as for a new member's search. Members may of course request any number of such searches, but a separate payment is due for each.
All financial transactions (payments) are also posted to the separate Accounts file, which also stores other financial details relating to running costs for the consideration of the firm's accountants at the end of the financial year.
Thanks for any help, regarding this level 0 Context only DFD
It needs some fixing.
The most obvious flaw is that you use verbs in your dataflows. In some cases this can be fixed easily by just discarding the verb. Return balance and status is not a datflow, but balance and status is.
In others cases it is not so easy. Check Balance, is it outstanding? sounds more like a Process than a dataflow. It looks like Accounting is responsible for doing that job. So will Accounting produce a list of outstanding balances? Or will it return a single balance and status, and if so, based on what input? Will your Airpot Transport System send a list of balances to check to Accounting?
Take for example Monthly Maintenance. What matters is that you want
requests and reminders for subscriptions due
Standard letters
These need to be visible in your DFD
The fact that you want to remove memberships where fees remain outstanding, probably has not place in the toplevel diagram, because that looks like an internal affair.
In general, focus on what the System produces. Maintaining internal state is secondary, is is a necessity to produce the desired output.