According to Stripes documentation here if you add a Stripe object ID as the source when getting a list of balance transactions you should get all transactions related to that ID. (e.g. filtering by a charge ID will return all charge and refund transactions).
I have tried passing in a chargeId and do indeed get back the initial balance transaction that was created for that charge, however this charge also has 2 refunds associated with it that are not returned in the list. I have tried this with other charges and only ever seem to get back the initial balance transaction created, never any other transactions (particularly refunds which the docs say should be returned). I have my limit set to 100 items and I have tried using both the Stripe.Net API as well as PHP calls and get back the same results.
I've also attempted passing in customerId's to see if I could get back all balance transactions triggered by a Customer and in these cases I never get back any results at all. These are Customers that have triggered MANY transactions!
The arguments I am providing to the API are as follows:
Method: balance/list
Parameters: limit=100
source=[chargeId or customerId that has triggered multiple transactions]
My question is: Is this a bug in the API, is the documentation incorrect or is there an important parameter or aspect to this I am missing. I've also gone back to charges from 30+ days ago just to make sure it has nothing to do with the rolling transfer/pending/availability cycles.
Is the actual fact that you can only ever get back the initial transaction created by a chargeId, but nothing else. Anyone have any experience in this regard? Thanks in advance!
Looks like this is indeed either a bug or an error in the Stipe documentation. I've reported this to Stripe and just received the following message back from someone on the support team:
Hi there,
Thanks for reaching out and alerting us to this issue!
This does look like either a bug or an error in our documentation. I have shared this with my team and we are looking into this issue.
Thank you for using Stripe and please let me know if there's anything else we can do to help!
I will update this thread once I hear back from Stripe with either an update or a resolution.
My hunch is this isn't a bug: the Stripe behavior is technically correct. The BalanceTransaction API is returning all txns for that specific charge, which is only a single txn. The txns associated with the charge's refund are technically associated with those refund objects, not the charge. The same logic applied to disputes/chargebacks.
Here's some (untested!) ruby code demonstrating how to grab all of the txns for a charge:
def balance_transactions_for_charge(stripe_charge_id)
balance_transactions = []
charge = Stripe::Charge.retrieve(stripe_charge_id)
balance_transactions << charge.balance_transaction
balance_transactions += charge.refunds.data.map do |refund|
refund.balance_transaction
end
if charge.dispute
dispute = Stripe::Dispute.retrieve(charge.dispute)
# fairly certain that the dispute txn list includes the full object, not just IDs
balance_transactions += dispute.balance_transactions.data.map(&:id)
end
balance_transactions.map { |txn_id| Stripe::BalanceTransaction.retrieve(txn_id) }
end
Related
With the charges API, a token would eventually be handed to the server that would allow you to charge against the customer's credit card, and the last step to complete the payment is handled by my server. This would allow you to do something like:
// Prevent anyone else from purchasing widget while we sell this one
lock(database){
if (widgetsAvailableCount > 0) {
widgetsAvailableCount--;
var charge = chargeService.Create(optionsForWidgetCharge);
}
else {
throw new Exception("Item is out of stock");
// Don't create charge against token in this case
}
}
This allows you to prevent a race condition where you sell the last remaining item to two people.
But with the PaymentIntents API, it seems that the final step in charging the credit card no longer happens on my server, but will happen when the client calls:
https://stripe.com/docs/js/payment_intents/confirm_card_payment
And it's not clear how long that will actually take (say, if a 3D Secure prompt is shown).
I'm trying to solve the same problem as above, avoiding the situation of accepting payment when there is no inventory left (where inventory was available at the time the payment process started but not when it was completed). I could mark in the database the item in a "reserved" state, but I'm wondering if I'm thinking about this the right way, and how others have addressed this if so.
Our application creates/updates database entries based on an external service's webhooks. The webhook sends the external id of the object so that we can fetch more data for processing. The processing of a webhook with roundtrips to get more data is 400-1200ms.
Sometimes, multiple hooks for the same object ID are sent within microseconds of each other. Here are timestamps of the most recent occurrence:
2020-11-21 12:42:45.812317+00:00
2020-11-21 20:03:36.881120+00:00 <-
2020-11-21 20:03:36.881119+00:00 <-
There can also be other objects sent for processing around this time as well. The issue is that concurrent processing of the two hooks highlighted above will create two new database entries for the same single object.
Q: What would be the best way to prevent concurrent processing of the two highlighted entries?
What I've Tried:
Currently, at the start of an incoming hook, I create a database entry in a Changes table which stores the object ID. Right before processing, the Changes table is checked for entries that were created for this ID within the last 10 seconds; if one is found, it quits to let the other process do the work.
In the case above, there were two database entries created, and because they were SO close in time, they both hit the detection spot at the same time, found each other, and quit, resulting in nothing being done.
I've thought of adding some jitter'd timeout before the check (increases processing time), or locking the table (again, increases processing time), but it all feels like I'm fighting the wrong battle.
Any suggestions?
Our API is Django 3.1 with a Postgres db
Okay, this might not be a very satisfactory answer, but it sounds to me like the root of your problem isn't necessarily with your own app, but the webhooks service you are receiving from.
Due to inherent possibility for error in network communication, webhooks which guarantee delivery always use at-least-once semantics. A sender that encounters a failure that leaves receipt uncertain needs to try sending the webhook again, even if the webhook may have been received the first time, thus opening the possibility for a duplicate event.
By extension, all webhook sending services should offer some way of deduplicating an individual event. I help run our webhooks at Stripe, and if you're using those, every webhook sent will come with an event ID like evt_1CiPtv2eZvKYlo2CcUZsDcO6, which a receiver can use for deduplication.
So the right answer for your problem is to ask your sender for some kind of deduplication/idempotency key, because without one, their API is incomplete.
Once you have that, everything gets really easy: you'd create a unique index on that key in the database, and then use upsert to guarantee only a single entry. That would look something like:
CREATE UNIQUE INDEX index_my_table_idempotency_key ON my_table (idempotency_key);
INSERT INTO object_changes (idempotency_key, ...) VALUES ('received-key', ...)
ON CONFLICT (idempotency_key) DO NOTHING;
Second best
Absent an idempotency ID for deduping, all your solutions are going to be hacky, but you could still get something workable together. What you've already suggested of trying to round off the receipt time should mostly work, although it'll still have the possibility of losing two events that were different, but generated close together in time.
Alternatively, you could also try using the entire payload of a received webhook, or better yet, a hash of it, as an idempotency ID:
CREATE UNIQUE INDEX index_my_table_payload_hash ON my_table (payload_hash);
INSERT INTO object_changes (payload_hash, ...) VALUES ('<hash_of_webhook_payload>', ...)
ON CONFLICT (payload_hash) DO NOTHING;
This should keep the field relatively small in the database, while still maintaining accurate deduplication, even for unique events sent close together.
You could also do a combination of the two: a rounded timestamp plus a hashed payload, just in case you were to receive a webhook with an identical payload somewhere down the line. The only thing this wouldn't protect against is two different events sending identical payloads close together in time, which should be a very unlikely case.
If you look at the acquity webhook docs, they supply a field called action, which key to making your webhook idempotent. Here are the quotes I could salvage:
action either scheduled rescheduled canceled changed or order.completed depending on the action that initiated the webhook call
The different actions:
scheduled is called once when an appointment is initially booked
rescheduled is called when the appointment is rescheduled to a new time
canceled is called whenever an appointment is canceled
changed is called when the appointment is changed in any way. This includes when it is initially scheduled, rescheduled, or canceled, as well as when appointment details such as e-mail address or intake forms are updated.
order.completed is called when an order is completed
Based on the wording, I assume that scheduled, canceled, and order.completed are all unique per object_id, which means you can use a unique together constraint for those messages:
class AcquityAction(models.Model):
id = models.CharField(max_length=17, primary_key=True)
class AcquityTransaction(models.Model):
action = models.ForeignKey(AcquityAction, on_delete=models.PROTECT)
object_id = models.IntegerField()
class Meta:
unique_together = [['object_id', 'action_id']]
You can substitute the AcquityAction model for an Enumeration Field if you'd like, but I prefer having them in the DB.
I would ignore the change event entirely, since it appears to trigger on every event, according to their vague definition. For the rescheduled event, I would create a model that allows you to use a unique constraint on the new date, so something like this:
class Reschedule(models.Model):
schedule = models.ForeignKey(MyScheduleModel, on_delete=models.CASCADE)
schedule_date = models.DateTimeField()
class Meta:
unique_together = [['schedule', 'schedule_date']]
Alternatively, you could have a task specifically for updating your schedule model with a rescheduled date, that way it remains idempotent.
Now in your view, you will do something like this:
from django.db import IntegrityError
ACQUITY_ACTIONS = {'scheduled', 'canceled', 'order.completed'}
def webhook_view(request):
validate(request)
action = get_action(request)
if action in ACQUITY_ACTIONS:
try:
insert_transaction()
except IntegrityError:
return HttpResponse(200)
webhook_task.delay()
elif action == 'rescheduled':
other_webhook_task.delay()
...
I am trying to figure out how to handle consistency in the database.
In scenario:
User A has an accounting document in the database include a balance field representing the amount of his current money. (supposed initially he has 100$)
My system has many methods to charge his account.
Suppose 2 methods occur at the same time, each method charges him for 10$, these steps occur concurrently in below orders:
Method 1 READ his balance and store in memory (100$)
Method 2 READ his balance and store in memory (100$)
... some business logics
Method 1 UPDATE his balance by subtracting variable in memory by 10 (100$ - 10$) and then save it
Method 2 UPDATE his balance by subtracting variable in memory by 10 (100$ - 10$) and then save it
This means he has been charged only 10$ instead of 20$.
I searched this situation a while and can not get it clear (sorry for my stupidity).
Really appreciate yours helps to enlighten my featherbrained. :)
You just discovered why financial transactions are complicated :-)
Have you ever wondered why it takes time for you to have an updated balance in your bank account? Or why you actually have two balances, instead of one?
That's because your account can actually go negative and (up to a certain point) that will be fine.
So in a real life scenario what happens is that you have a balance of 100$, you pay 10$ and until that transaction is processed and confirmed by the receiver, you still have your 100$. If you do 20 transactions of 10$ each, you'll be able to complete them because the system will most likely not be able to notice.
And honestly, it shouldn't. Think of credit cards, you might not have enough money now, but maybe you know you'll have enough when the credit is due.
So, the race condition you describe only works if you actually read the value and then update it.
There are a few approaches:
Read the current balance, and update the row using the old balance as a field in the where statement. This way if it updates no rows you know that you need to re-read and update.
Don't update the balance and only do it time-based, say once per hour. Yes, you might still have to do some checks, but the system will overall be more responsive.
Lock the database row as your first step. This would work but there's a chance that it will make the app slower.
Race condition you describe is low level design concern. With backend engine like Node that will handle the incomming request in first come first serve fashion you don't need to think about this case. Race condition you describe is not possible if you respect the order in which database update callbacks are fired. They are fired in the same order they have been issued in. So you should call next update only when the previous has finished. Promisses are great way to do this.
My employer has challenged me to build him a custom Point of Sale system and stripe was my first thought for payment processing. I work in food-delivery, and as such our current (as well as every other) POS immediately charges the card (verifies it? whatever), and then at the end of the night when tips are counted (whether from the drivers or tipped receipts in the tip box) the charge changes to whatever the customer agreed to.
I see I can do a few similar things:
update metadata on a charge (not what I need)
capture a charge (not what I need - won't let you use a value higher than the initial)
do a second charge for the tip (would hope to avoid this)
save the card information with Stripe after creating a customer object and not do any charge until after I know the tip (would hope to avoid this)
But, can I verify the customer can pay the amount (that initial charge) and then increase it later once we know if/how much they tipped?
Simply put, I'd like my flow to be:
Customer orders
Charge is issued for 20$ of food (Stripe)
Food is delivered, 5$ tip secured on the receipt
Receipts are turned in, 5$ tip is entered into system
Charge is changed to reflect the added tip (now 25$) (Stripe)
Is this possible with Stripe? If so, how?
If not, do you know of any other payment system that could implement this flow?
Short answer: no, that's not possible.
You could do something like this:
Create an uncaptured charge for the base amount.
Once the customer confirms the tip, try to create a charge for the base amount + tip.
3a. If the charge succeeds, release the first uncaptured charge (by refunding it).
3b. If the charge fails, capture the first uncaptured charge (and maybe explain to your customer that they were only billed for the base amount).
How can we implement a Facebook like status message system in mongodb (using mongoose), where whenever any given user posts his status it gets broadcasted on all his friends timeline.
It doesn't have to be real-time, there will be a refresh button to get the latest statuses.
here is what I have come up with:
Plan A:
status(collection)
id, user_id(reference), status_msg
Benefit: faster write speed
Plan B:
status(collection)
id, user_id(reference), status_msg, friends_list[sub-document]
Benefit: faster read speed
With plan A, I'll have to loop through all the friends a user has in his friends list and then fetch all the status.
I'll have to do this every time (page refresh/ new login) for every single friend.
With Plan B, I'll only have to fetch the statuses which has the current user in the friends_list.
I would like to know your opinion and suggestion on this ?
Is there any better way of approaching this problem ?
I would also like to know how I can use rabbitMQ here to increase the efficiency and lower the unnecessary db i/o .
Assuming that each user will likely have several friends, and these friends refresh their timeline several times a day, you can assume that reading will happen much more frequently than writing. That means from a pure performance standpoint you would optimize for read-access, not for write-access, and store the receivers with the message.
However, keep the semantics in mind. What if the friend-list of the author changes after they posted a status message?
Do you want the message to disappear from the timelines of any ex-friends?
Do you want the message to appear in the timeline of any new friends they make?
When the answers to these questions are yes, you should rather determine the receivers on read than on write.
There is also a third option which might be worth considering: Do not handle messages by sender, handle them by receiver. When someone posts a message, create an individual copy of the message for each of their friends and save them as separate documents. You can then get all messages for a user by querying your messages collection for messages where they are the receiver. The friend/unfriend operation would then need to check for any messages which need to be added/removed. The major drawback of this approach would be that users with a very high number of friends would create a very high write-load when posting something.