I need to keep in sync with transactions on accounts for a set of items. To me that means:
1. Do an initial download of all historical transactions.
2. Get new transactions when available.
3. Make sure no transactions are dropped on the floor.
It's not clear from the documentation and the API how this can be accomplished reliably.
The create API has a webhook parameter, so it seems I should have a webhook set up immediately on getting transactions. If I don't have I missed out on all the transactions forever?
Can I pull all transactions via the API alone? I noticed the options have an offset. Is that for a cursor? Can I ask for transactions way back and the past to trigger a redownload of transactions?
What if a webhook drops a batch of transactions? How can I tell? How can I redownload the missing transactions?
And I remember reading somewhere in the doc the account IDs and transaction IDs are associated with an ACCESS_TOKEN. Does this mean that the account IDs and transaction IDs can't be used to identify data uniquely across tokens?
Plaid states that they can fetch transaction data up to past two years. However, the amount of historic transactions provided by banks vary from bank to bank. I’ve seen some banks provide data for the past three months, whereas some return data for last two years. I’ve also seen some banks not support returning any transaction data.
As for the webhook, please note that the amount of time it takes to retrieve historic data after connecting an account varies. That’s where a webhook is useful as you can be notified when data is available for fetching.
Plaid returns only 500 transactions per call (I think). So, you are responsible for pagination while retrieving historic data.
You can always retrieve historic data, but you will only be able to get the past two years maximum. Every day that passes, you will not be able to retrieve data for the first day two years ago. It’s a moving window. I’ve generally cached data on our side as you will not be able to access data older than two years.
If I recall correctly, each institution that is connected has a unique access token. You can use account id to uniquely identify transactions, but you might have to store the relations in your database as the returned data doesn’t have that.
Hope that helps.
Related
I wonder if you can help. I am writing an order system and currently have implemented an order microservice which takes care of placing an order. I am using DDD with event sourcing and CQRS.
The order service itself takes in commands that produce events, the actual order service listens to its own event to create a read model (The idea here is to use CQRS, so commands for writes and queries for reads)
After implementing the above, I ran into a problem and its probably just that I am not fully understanding the correct way of doing this.
An order actually has dependents, meaning an order needs a customer and a product/s. So i will have 2 additional microservices for customer and products.
To keep things simple, i would like to concentrate on the customer (although I have exactly the same issue with products but my thinking is that if I fix the customer issue then the other one is automatically fixed also).
So back to the problem at hand. To create an order the order needs a customer (and products), I currently have the customerId on the client, so sending down a command to the order service, I can pass in the customerId.
I would like to save the name and address of the customer with the order. How do I get the name and address of the customerId from the Customer Service in the Order Service ?
I suppose to summarize, when data from one service needs data from another service, how am I able to get this data.
Would it be the case of the order service creating an event for receiving a customer record ? This is going to introduce a lot of complexity (more events) in the system
The microservices are NOT coupled so the order service can't just call into the read model of the customer.
Anybody able to help me on this ?
If you are using DDD, first of all, please read about bounded context. Forget microservices, they are just implementation strategy.
Now back to your problem. Publish these events from Customer aggregate(in your case Customer microservice): CustomerRegistered, CustomerInfoUpdated, CustomerAccountRemoved, CustomerAddressChanged etc. Then subscribe your Order service(again in your case application service inside Order microservice) to listen all above events. Okay, not all, just what order needs.
Now, you may have a question, what if majority or some of my customers don't make orders? My order service will be full of unnecessary data. Is this a good approach?
Well, answer might vary. I would say, space in hard disk is cheaper than memory or a database query is faster than a network call in performance perspective. If your database host(or your server) is limited then you should not go with microservices. Moreover, I would make some business ideas with these unused customer data e.g. list all customers who never ordered anything, I will send them some offers to grow my business. Just kidding. Don't feel bothered with unused data in microservices.
My suggestion would be to gather the required data on the front-end and pass it along. The relevant customer details that you want to denormalize into the order would be a value object. The same goes for the product data (e.g. id, description) related to the order line.
It isn't impossible to have the systems interact to retrieve data but that does couple them on a lower level that seems necessary.
When data from one service needs data from another service, how am I able to get this data?
You copy it.
So somewhere in your design there needs to be a message that carries the data from where it is to where it needs to be.
That could mean that the order service is subscribing to events that are published by the customer service, and storing a copy of the information that it needs. Or it could be that the order service queries some API that has direct access to the data stored by the customer service.
Queries for the additional data that you need could be synchronous or asynchronous - maybe the work can be deferred until you have all of the data you need.
Another possibility is that you redesign your system so that the business capability you need is with the data, either moving the capability or moving the data. Why does ordering need customer data? Can the customer service do the work instead? Should ordering own the data?
There's a certain amount of complexity that is inherent in your decision to distribute the work across multiple services. The decision to distribute your system involves weighing various trade offs.
Our site currently asks customers to provide their CC data every time they put an order. Few days later we charge the card, very often with a different amount than shown during the checkout because of merging orders, changing shipment method and so on.
We'd like to migrate to Stripe, but we'd like not to change the current behaviour. Few questions arose while we read the Stripe docs:
1) Stripe.js allows to create a Source without a customer. This seems ideal for us, but the following text from Stripe docs caught our attention:
A card source must be used within a few minutes of its creation as CVC
information is only available for a short amount of time. Card sources
do not expire, but using them after a delay can result in a charge
request that is performed without CVC information. The consequences of
this can be higher decline rates and increased risk of fraud.
So we shouldn't charge the Source few days later? If so, isn't it also a problem for Sources attached to customers, because their CVCs are forgotten as well?
2) To keep things simple, we'd prefer to avoid creating customers and managing their sources in Stripe. Stripe has the concept of single-use sources, that can be charged without attaching them to customers, but the docs say that the amount is required when they are created. We don't know the exact amount at the checkout process, so could we put a bigger number as an upper limit of what we'll charge in the future and provide the exact amount during charging the source?
So you're correct, the CVC is lost anyway but this is not the point that Stripe is trying to make. When you attach the Source to a customer, Stripe will run a $0 or $1 authorization on the card with the bank to make sure it's valid. This lets them catch expired or invalid cards immediately. This also lets them send the CVC to the bank, since they have it at that point, to catch invalid CVC which is important to combat fraud.
If you were to create a Source and not charge it for a few days, that would technically work. The risk is that when you do charge it, the bank won't see the CVC and won't be able to tell you if it was correct or not, increasing the risk of fraud.
To avoid this, Stripe recommends that you save the Source on a Customer. This is just one extra API request which lets you catch invalid cards immediately. Then, days later, when you create the charge on that customer/source, the bank might still decline it (insufficient funds for example) but you would decrease the risk of charging a bad card and catch issues immediately on tokenization. It's better for you as a business but also better for the customer in case they mistyped their CVC or expiration date, etc.
We have a scenario where multiple clients would like to get updates from Document Db inserts, but they are not available online all the time.
Example: Suppose there are three clients registered with the system, but only one is online at present time. When the online client inserts/updates a document, we want the offline client(s) on wakes up to go look at change feed and update itself independently.
Now is there a way for each client to maintain it's own feed to the same partition (when they were last synced) and get the changes when the come online based on last sync?
When using change feed, you use continuation token per partition. Change feed continuation tokens do not expire, thus you can continue from any point. Each client can keep its own continuation token and read changes as needed/wakes up, this essentially means that each client can keep its own feed for each partition.
I am creating a payment processing page for use via stripe and I want to be able to have my customers manages saved cards or use a new one. I am worried about being too forthcoming with some customer information and the possibility of exposing too much card information to my servers.
If I retrieve sensitive info like last 4, exp date, etc. should I be putting some means of encryption on it or is it completely fine to display or store pieces of that information for relationships server side on my end?
EDIT: New dev here be gentle :)
The only PCI-sensitive information is the entire card number and the CVC.
The card's brand, last 4 digits and expiry date are not sensitive. If you use Stripe, you can retrieve this information the card object's attributes, e.g. exp_month and exp_year, etc.
From a PCI compliance point of view, you don't need to do anything special to handle this data since it is not considered sensitive. My recommendation would be to either simply query the information from Stripe as needed, and discard it afterwards, or store it as-is into your database (in that last case, you'll probably want to use webhooks to make sure your DB is synced with your Stripe account).
I've been playing with stripe recently and while i fully understand that the token hides the clients credit card details from the server. This tutorial suggests that the server should not rely on the data-amount since it can be changed by the client
Don’t Rely on the Form’s Price
A frequent mistake stems from using form data to contain the price of
the product being purchased, possibly via a hidden input. Because a
user can easily edit this input’s value, it’s unwise to depend on it.
Always fetch the price of the product from the server-side. Never rely
on the form to tell you. A simple database query is the preferred option.
Can someone explain to be why stripe does not include the data-amount value as a parameter in the token generation? Is there not a potential for a server side code to change the agreed price and overcharge the client.
The token is a placeholder of a pending charge, it does not know how much you are going to charge yet. Once you are ready to charge the card an api request will be sent to Stripe along with the token. The concern about the amount deals with relying on POST data from a form that can be manipulated by the customer.
Its up to you to set the charge amount. For example a hotel could authorize $100 to spend the night but then at check out discover that you used the minibar and then charge $150. Or the auto calculated shipping is off so when you actually purchase the shipping its $5 less and you decide to charge $5 less than your auth.
What you should be doing is calculating the amount to charge the customer, save it via a shopping cart like function in your DB (or serverside somehow) sending the checkout form to the customer then using the previously calculated amount run the auth then the charge.
Form data can easily be changed by the end user. Just open the page and right click (in chrome) and click inspect element. You can then arbitrarily change form data. So if you were using that, the user could set the price to $.01 for your $1,000.00 product.
The propose of tokenization in the PCI world is to keep sensitive data off your servers. Otherwise you would collect the PCI data yourself then send the amount off to the processor along with the PCI data. By not ever having the sensitive data touch your systems you save a ton of money and headache in PCI compliance. See this 115 page document: https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-1.pdf
Hope that helps, Please comment and I'll try to help further if it doesn't.