I am running a transaction to update an item that needs to be stored in two keys. To accomplish this, I have setup a nested transaction as follows, and it seems to run as expected:
firebaseOOO.child('relationships/main').child(accountID).child(friendAccountID).transaction(function(data) {
data.prop = 'newval';
firebaseOOO.child('relationships/main').child(friendAccountID).child(accountID).transaction(function(data) {
return r;
});
return r;
});
Are there any gotchas or possible unexpected implications to this? I am most worried about getting stuck in some sort of transaction loop under load, where each transaction cancels the other out forcing them both to restart, or similar.
Is there a better way of doing this?
I am using the NodeJS client.
You probably don't want to start another transaction from within the callback to the first one. There is no guarantee as to how many times the function for your first transaction will run, particularly if there is a lot of contention at the location you are trying to update.
A better solution, which I believe you hit on in your other question, is to start the second transaction from the completion callback, after checking that the first one committed.
Related
I have a Node.js web app with a route that marks some entity as deleted - flipping boolean field in a database. This route returns that entity. Right now I have code that looks like this:
UPDATE entity SET is_deleted=true WHERE entity.id = ?
SELECT * FROM entity WHERE entity.id = ?
For the moment I can't use RETURNING statement for other reasons.
So I got in the argument with colleague, I think that putting both UPDATE and SELECT inside transaction is unnecessary, because we are not doing anything significant with data, just returning it. As a user of the app I would expect that data that is returned is as fresh as possible, meaning that I would get same results on page refresh.
My question is, what is the best practice regarding reading data after write? Do you always wrap reading with writing inside transaction? Or it depends?
Well, for performance reasons you want to keep your transactions as small and quick as possible. This will minimize the chance to have potential locks and deadlocks that could bring your application to its knees. As such, unless there is a very good reason to do so, keep your select statements outside of the transaction. This is specially important if your need to execute a long running select statement. By putting the select inside the transaction, you keep the update locks much longer than needed.
The question title basically says it all, but to rephrase it:What handles the asynchronous function execution, if the main (and only) thread is occupied with running down the main code block?
So far I have only found that the async code gets executed elsewhere or outside the main thread, but what does this mean specifically?
EDIT: The proposed Node.js Event loop question's answers may also address this topic, but I was looking for a less complex, more specific answer, rather then an explanation of Node.js concept. Also, it does not show up in search for anything similar to "node asynchronous single-threaded".
EDIT, #Mr_Thorynque: Running a query to get data from database and log it to console. Nothing gets logged, because Node, being async, does not wait for query to finish and for data to populate. (this is just an example as requested, NOT a part of my question)
var = data;
mysql.query(`SELECT *some rows from database*`, function (err, rows, fields) {
rows.forEach(function(row){
data += *gather the requested data*
});
});
console.log(data);
What it really comes down to is that the node process (which is single threaded) hands off the work to "something else". This could be the OS's I/O process, or a networked resource or whatever. By handing it off, it frees its thread to keep working on the next in-memory thing. It uses file handles to keep track of any pending work and in the event loop marries the two back together and fire the callback when the work is done.
Note that this is also why you can block processing in your own code if poorly designed. If your code runs complex tasks and doesn't hand off the work, you'll block the single thread.
This is about as simple an answer as I can make, I think that the details are well explained in the links in your comments.
We are trying to create an algorithm/heuristic that will schedule a delivery at a certain time period, but there is definitely a race condition here, whereby two conflicting scheduled items could be written to the DB, because the write is not really atomic.
The only way to truly prevent race conditions is to create some atomic insert operation, TMK.
The server receives a request to schedule something for a certain time period, and the server has to check if that time period is still available before it writes the data to the DB. But in that time the server could get a similar request and end up writing conflicting data.
How to circumvent this? Is there some way to create some script in the DB itself that hooks into the write operation to make the whole thing atomic? By putting a locking mechanism on that script? What makes the whole thing non-atomic is the read and the wire time between the server and the DB.
Whenever I run into race condition I think of one immediate solution QUEUE.
Step 1) What you can do is that instead of adding data to a database directly you can add it to queue without checking anything.
Step 2) A separate reader will read from the queue check DB for any conflict and take necessary action.
This is one of the ways to solve this If you implement any better solution please do share it.
Hope that helps
I'm creating some kind of real-chat app and I have trouble. What I want to do is read previous 50 messages(documents) before specified _id. I'll explain more detail.
In first time user getting in the room, App automatically load recent 50 messages. After, if user scrolling up to the top, load more 50 previous messages.
The problem is I don't get it how to do. What I thought is find all documents and move the cursor, but every I tried were failed. If I log the "cursor" object in console, it saids:
Promise { <pending> }
so if I do this:
let cursor = db.find('room', { ... });
while(cursor.hasNext()) {
cursor.next();
}
it goes infinite loop, never stops. If will be very thanksful gimme a hand. :)
And if there is alternative way to not need to use cursor, that would be really nice.
one more final question: is using cursor causes performance low?
I'm not sure what library you use, it seems that cursor is an asynchronous object (that's what Promise suggests), so the while loop is incorrect anyway. It will always be pending cause you don't allow the other event (i.e. "i got response") to occure due to single-threaded nature of NodeJS. You probably have to use callbacks, not synchronous loops.
But that aside I do believe that your whole approach is incorrect.
If you know how to load the most recent 50 messages, then it means that you have to have some kind of logical ordering on the collection. Perhaps a timestamp (which might be a part of id_).
So what I propose instead is something similar to "pagination":
On the client side set timestamp_pointer = now()
Do a query: get me 50 most recent messages such that timestamp < timestamp_pointer
On the client side set timestamp_pointer = smallest timestamp of loaded messages
If a user scrolls up go back to point 2.
There are several advantages of this method, one of them is that you don't have to worry if a connection drops for a short moment since the state is tracked on the user side, not on the database side. And with a proper index it will be very fast.
And yes, using cursor like you do causes low performance because the database has to keep track of the query until it is fully iterated. Apart from pure memory and/or cpu usage it has some other nasty drawbacks, like Mongo has timeouts on cursors. What if a user scrolls up after 15 minutes? By default the timeout on cursor is 10 minutes. It would be very hard to implement your idea properly.
Use Postgres. #PostgresEvangelist
Is this a "proper" way to run Firebase transactions that depend on each other sequentially using the NodeJS client:
ref.child('relationships/main').child(accountID).transaction(function(data) {
return r;
}, function(error, committed, snapshot) {
if (error) {}
else if (!committed) {}
else {
runNextTransaction();
}
});
Originally I was going to put runNextTransaction() in the core function because transactions first run locally, but wouldn't that then hold open the original transaction until the last transaction in the chain is complete, possibly causing issues? (Also I need good data for the next step so I would have to handle collisions before moving on.)
Transactions run asynchronously, so kicking off the next transaction from within the first one would work, but it may not do what you want. Transactions functions can be run more than one time, and you likely don't want to initiate multiple secondary transactions in that case. What you have looks like the right way to do serial transactions. If you're interested in making things a little cleaner, especially if you're going to chain multiple transactions, consider looking into Promises.