I have a question regarding my Firebase setup, with respect to security and practicality.
The goal is the enable a user to purchase credits, and spend those credits one by one. To enable this, I set up a "users" object where I store user-data (name, address etc.), a "transactions" object where I store all the purchases (amount, time etc.), and a "spendcredits" object where I store the data connected to the user spending a credit (time, on what, etc.).
Since the App must know how many credits the user can still spend, I created a variable in the user-object called validCredits, where the current available credits should be stored.
The user has read+write rights to his own user-object within "users" and read+write rights to his own object within "spendcredits". Only a different server has read+write rights to the "transactions" object.
So what happens is, the user purchases 5 credits. The server updates his validCredits variable with +5. The user spends a credit (-1), spends another (-1) and purchases 5 more credits (+5). His new validCredits amount is then 8.
I'm not sure whether this is a safe/optimal setup. I'm afraid that since a user has write rights to his own account, where the "validCredits" variable is stored, it might somehow be possible for him to add extra credits by increasing this value? Or can that be prevented by only allowing the user a "-1 credit" operator on this field?
I can also imagine that you might want to store everything in the "transactions" object, and just do a sum of all transactins when the App request the latest number of valid Credits? What is generally recommended for such a system with payments and credits?
The following rule can be used to enforce that a user can only decrease their validCredits by one at a time. I'm assuming that you have some way of distinguishing the server from a user, which is what auth.isServer represents.
{
"users": {
"$uid": {
validCredits": "auth.isServer || newData.val() === data.val() - 1"
}
}
}
Related
With the charges API, a token would eventually be handed to the server that would allow you to charge against the customer's credit card, and the last step to complete the payment is handled by my server. This would allow you to do something like:
// Prevent anyone else from purchasing widget while we sell this one
lock(database){
if (widgetsAvailableCount > 0) {
widgetsAvailableCount--;
var charge = chargeService.Create(optionsForWidgetCharge);
}
else {
throw new Exception("Item is out of stock");
// Don't create charge against token in this case
}
}
This allows you to prevent a race condition where you sell the last remaining item to two people.
But with the PaymentIntents API, it seems that the final step in charging the credit card no longer happens on my server, but will happen when the client calls:
https://stripe.com/docs/js/payment_intents/confirm_card_payment
And it's not clear how long that will actually take (say, if a 3D Secure prompt is shown).
I'm trying to solve the same problem as above, avoiding the situation of accepting payment when there is no inventory left (where inventory was available at the time the payment process started but not when it was completed). I could mark in the database the item in a "reserved" state, but I'm wondering if I'm thinking about this the right way, and how others have addressed this if so.
Question is related to CQRS - I have user that wants to order something from web and is presented with GUI showing his balance = 100$ and stock = 1 item. Let's say we have 2 services here AccountService and StockService with separate concerns. In order to generate GUI for client third service AggregatorService listens to domain events from AccountService and StockService, projects a view and creates GUI for clients.
When user decides to order this item, he clicks a button and Command for order is sent to AccountService. Here we load AccountAggregate in order to decrease balance for the price of the item that needs to be ordered. But before I can do this, I have to check if the item is still available (or somehow to reserve it). Only thing that comes up to my mind is:
Read current stock of the item from read model of StockService, but what can happen is that other services read model is updated just a second after I read it (e.g. somebody bought the item, so actual stock is =0. but read model still has =1).
Before decreasing a balance call some method on StockService to reserve the item for some time. If order is not successful (e.g. no enough funds on balance, I would have to un-reserve it somehow). This needs to be some sync-REST call and it is probably slower than some async solution (if any).
Are there any best practices for this kind of use-case?
You have 2 options, depending on whether you embrace eventual consistency or not.
Using immediate consistency I would have an OrderService that receives the order request and it makes async calls to AccountService.ReservePayment() and StockService.ReserveStock(). If either of those fail you call AccountService.UndoReservePayment() and StockService.UndoReserveStock(). If both succeed you call AccountService.CompleteReservePayment() and StockService.CompleteReserveStock(). Make sure each reservation should have a timestamp on it so a daemon process can occasionally run and Undo any reserves that are older than an hour or so.
This approach makes the user wait until both those services complete. If either the StockService or the AccountService are slow, the user experience is slow. I suggest a better way to do this is the eventual consistency approach which gives the user a very fast experience at the expense of receiving failure messages after the fact.
With eventual consistency you assume they have enough credit and you have enough inventory, and in response to their order request you immediately send back a success message. The user is happy and they go along to buy more stuff.
The OrderCreated event is then handled asynchronously to reserve stock and credit as described above. However, since there is no time pressure to reply to the waiting user you don’t have to scale up to handle as high a throughput. If the credit check and stock check take a minute or two, the user doesn’t care because he’s off doing other things.
The price you pay here is that the user may get a success message at the time of order submission, then a few minutes later get an email saying the sale didn’t go through after all because there’s no stock. This is what many large retailers do, including Amazon, Target, Walmart, etc. Eventual consistency can go a long way towards easing the load and cost of the back end.
I am trying to figure out how to handle consistency in the database.
In scenario:
User A has an accounting document in the database include a balance field representing the amount of his current money. (supposed initially he has 100$)
My system has many methods to charge his account.
Suppose 2 methods occur at the same time, each method charges him for 10$, these steps occur concurrently in below orders:
Method 1 READ his balance and store in memory (100$)
Method 2 READ his balance and store in memory (100$)
... some business logics
Method 1 UPDATE his balance by subtracting variable in memory by 10 (100$ - 10$) and then save it
Method 2 UPDATE his balance by subtracting variable in memory by 10 (100$ - 10$) and then save it
This means he has been charged only 10$ instead of 20$.
I searched this situation a while and can not get it clear (sorry for my stupidity).
Really appreciate yours helps to enlighten my featherbrained. :)
You just discovered why financial transactions are complicated :-)
Have you ever wondered why it takes time for you to have an updated balance in your bank account? Or why you actually have two balances, instead of one?
That's because your account can actually go negative and (up to a certain point) that will be fine.
So in a real life scenario what happens is that you have a balance of 100$, you pay 10$ and until that transaction is processed and confirmed by the receiver, you still have your 100$. If you do 20 transactions of 10$ each, you'll be able to complete them because the system will most likely not be able to notice.
And honestly, it shouldn't. Think of credit cards, you might not have enough money now, but maybe you know you'll have enough when the credit is due.
So, the race condition you describe only works if you actually read the value and then update it.
There are a few approaches:
Read the current balance, and update the row using the old balance as a field in the where statement. This way if it updates no rows you know that you need to re-read and update.
Don't update the balance and only do it time-based, say once per hour. Yes, you might still have to do some checks, but the system will overall be more responsive.
Lock the database row as your first step. This would work but there's a chance that it will make the app slower.
Race condition you describe is low level design concern. With backend engine like Node that will handle the incomming request in first come first serve fashion you don't need to think about this case. Race condition you describe is not possible if you respect the order in which database update callbacks are fired. They are fired in the same order they have been issued in. So you should call next update only when the previous has finished. Promisses are great way to do this.
Suppose there are two microservices: Order and Inventory. There is an API in order service that takes ProductId, Qty etc and place the order.
Ideally order should only be allowed to place if inventory exists in inventory service. People recommend to have Saga pattern or any other distributed transactions. That is fine and eventually consistency will be utilized.
But what if somebody wants to abuse the system. He can push orders with products (ProductIds) which are either invalid or out of inventory. System will be taking all these orders and place these orders in queue and Inventory service will be handling these invalid order.
Shouldn't this be handled upfront (in order service) rather than pushing these invalid orders to the next level (specially where productId is invalid)
What are the recommendations to handle these scenarios?
What are the recommendations to handle these scenarios?
Give your order service access to the data that it needs to filter out undesirable orders.
The basic plot would be that, while the Inventory service is the authority for the state of inventory, your Orders service can work with a cached copy of the inventory to determine which orders to accept.
Changes to the Inventory are eventually replicated into the cache of the Orders service -- that's your "eventual consistency". If Inventory drops off line for a time, Order's can continue providing business value based on the information in its cache.
You may want to be paying attention to the age in the data in the cache as well -- if too much time has passed since the cache was last updated, then you may want to change strategies.
Your "aggregates" won't usually know that they are dealing with a cache; you'll pass along with the order data a domain service that supports the queries that the aggregate needs to do its work; the implementation of the domain service accesses the cache to provide answers.
So long as you don't allow the abuser to provide his own instance of the domain service, or to directly manipulate the cache, then the integrity of the cached data is ensured.
(For example: when you are testing the aggregate, you will likely be providing cached data tuned to your specific test scenario; that sort of hijacking is not something you want the abuser to be able to achieve in your production environment).
You most definitely would want to ensure up-front that you can catch as many invalid business cases as possible. There are a couple ways to deal with this. It is the same situation as one would have when booking a seat on an airline. Although they do over-booking which we'll ignore for now :)
Option 1: You could reserve an inventory item as part of the order. This is more of a pessimistic approach but your item would be reserved while you wait for the to be confirmed.
Option 2: You could accept the order only if there is an inventory item available but not reserve it and hope it is available later.
You could also create a back-order if the inventory item isn't available and you want to support back-orders.
If you go with option 1 you could miss out on a customer if an item has been reserved for customer A and customer B comes along and cannot order. If customer A decides not to complete the order the inventory item becomes available again but customer B has now gone off somewhere else to try and source the item.
As part of the fulfillment of your order you have to inform the inventory bounded context that you are now taking the item. However, you may now find that both customer A and B have accepted their quote and created an order for the last item. One is going to lose out. At this point the one not able to be fulfilled will send a mail to the customer and inform them of the unfortunate situation and perhaps create a back-order; or ask the customer to try again in X-number of days.
Your domain experts should make the call as to how to handle the scenarios and it all depends on item popularity, etc.
I will not try to convince you to not do this checking before placing an order and to rely on Sagas as it is usually done; I will consider that this is a business requirement that you must implement.
This seems like a new sub-domain to me: bad-behavior-prevention (or how do you want to call it) that comes with a new responsibility: to prevent abusers. You could add this responsibility to the Order microservice but you would break the SRP. So, it should be done in another microservice.
This new microservice is called from your API Gateway (if you have one) or from the Orders microservice.
If you do not to add a new microservice (from different reasons) then you could implement this new functionality as a module inside of the Orders microservice but I strongly recommend to make it highly decoupled from its host (separate and private persistence/database/table).
I am developing an application that has employee time tracking module. When employee starts working (e.g. at some abstract machine), we need to save information about him working. Each day lots of employees work at lots of machines and they switch between them. When they start working, they notify the system that they have started working. When they finish working - they notify the system about it as well.
I have an aggregate Machine and an aggregate Employee. These two are aggregate roots with their own behavior. Now I need a way to build reports for any given Employee or any given Machine for any given period of time. For example, I want to see which machines did given employee used over period of time and for how long. Or I want to see which employees worked at this given machine for how long over period of time.
Ideally (I think) my aggregate Machine should have methods startWorking(Employee employee) and finishWorking(Employee employee).
I created another aggregate: EmployeeWorkTime that stores information about Machine, Employee and start,finish timestamps. Now I need a way to modify one aggregate and create another at the same time (or ideally some another approach since this way it's somewhat difficult).
Also, employees have a Shift that describes for how many hours a day they must work. The information from a Shift should be saved in EmployeeWorkTime aggregate in order to be consistent in a case when Shift has been changed for given Employee.
Rephrased question
I have a Machine, I have an Employee. HOW the heck can I save information:
This Employee worked at this Machine from 1.05.2017 15:00 to 1.05.1017 18:31.
I could do this simply using CRUD, saving multiple aggregates in one transaction, going database-first. But I want to use DDD methods to be able to manage complexity since the overall domain is pretty complex.
From what I understand about your domain you must model the process of an Employee working on a machine. You can implement this using a Process manager/Saga. Let's name it EmployeeWorkingOnAMachineSaga. It work like that (using CQRS, you can adapt to other architectures):
When an employee wants to start working on a machine the EmployeeAggregate receive the command StartWorkingOnAMachine.
The EmployeeAggregate checks that the employee is not working on another machine and if no it raises the EmployeeWantsToWorkOnAMachine and change the status of the employee as wantingToWorkOnAMachine.
This event is caught by the EmployeeWorkingOnAMachineSaga that loads the MachineAggregate from the repository and it sends the command TryToUseThisMachine; if the machine is not vacant then it rejects the command and the saga sends the RejectWorkingOnTheMachine command to the EmployeeAggregate which in turns change it's internal status (by raising an event of course)
if the machine is vacant, it changes its internal status as occupiedByAnEmployee (by raising an event)
and similar when the worker stops working on the machine.
Now I need a way to build reports for any given Employee or any given Machine for any given period of time. For example, I want to see which machines did given employee used over period of time and for how long. Or I want to see which employees worked at this given machine for how long over period of time.
This should be implemented by read-models that just listen to the relevant events and build the reports that you need.
Also, employees have a Shift that describes for how many hours a day they must work. The information from a Shift should be saved in EmployeeWorkTime aggregate in order to be consistent in a case when Shift has been changed for given Employee
Depending on how you want the system to behave you can implement it using a Saga (if you want the system to do something if the employee works more or less than it should) or as a read-model/report if you just want to see the employees that do not conform to their daily shift.
I am developing an application that has employee time tracking module. When employee starts working (e.g. at some abstract machine), we need to save information about him working. Each day lots of employees work at lots of machines and they switch between them. When they start working, they notify the system that they have started working. When they finish working - they notify the system about it as well.
A critical thing to notice here is that the activity you are tracking is happening in the real world. Your model is not the book of record; the world is.
Employee and Machine are real world things, so they probably aren't aggregates. TimeSheet and ServiceLog might be; these are the aggregates (documents) that you are building by observing the activity in the real world.
If event sourcing is applicable there, how can I store domain events efficiently to build reports faster? Should each important domain event be its own aggregate?
Fundamentally, yes -- your event stream is going to be the activity that you observe. Technically, you could call it an aggregate, but its a pretty anemic one; easier to just think of it as a database, or a log.
In this case, it's probably just full of events like
TaskStarted {badgeId, machineId, time}
TaskFinished {badgeId, machineId, time}
Having recorded these events, you forward them to the domain model. For instance, you would take all of the events with Bob's badgeId and dispatch them to his Timesheet, which starts trying to work out how long he was at each work station.
Given that Machine and Employee are aggregate roots (they have their own invariants and business logic in a complex net of interrelations, timeshift-feature is only one of the modules)
You are likely to get yourself into trouble if you assume that your digital model controls a real world entity. Digital shopping carts and real world shopping carts are not the same thing; the domain model running on my phone can't throw things out of my physical cart when I exceed my budget. It can only signal that, based on the information that it has, the contents are not in compliance with my budgeting policy. Truth, and the book of record are the real world.
Greg Young discusses this in his talk at DDDEU 2016.
You can also review the Cargo DDD Sample; in particular, pay careful attention to the distinction between Cargo and HandlingHistory.
Aggregates are information resources; they are documents with internal consistency rules.