Arango beginTransaction() not rolling back with trx.abort() - arangodb

I'm having some difficulty with arangodb.beginTransaction(). In my tests, creating a transaction, calling a function via trx.run() API and then aborting the trx does not roll back the data changes on the database. I'm unclear what is happening and the documentation is extremely sparse. There is this documentation which is quite ominous, but also very vague:
If the given function contains asynchronous logic, only the synchronous part of the function will be run in the transaction. E.g. when using async/await only the code up to the first await will run in the transaction.
What about nested async/await inside the async function being called? For example, if I have this:
async runQuery(query) {
try {
const cursor = await this.arangodb.query(query)
return cursor.all()
} catch (error) {
logger.error('ArangoDB query failed', { stack: error })
throw error
}
}
async touchDocument(id, revision) {
const type = await this.getObjectTypeFromId(id)
const collection = await this.getCollection(type, false, true)
const idCollection = await this.getCollection(ID_COLLECTION, false, true)
const touchAql = aql`
LET permanentDocId = DOCUMENT(${idCollection}, ${id}).docId
LET permanentDoc = MERGE( DOCUMENT(permanentDocId), { _rev : ${revision} })
UPDATE permanentDoc WITH permanentDoc in ${collection} OPTIONS { ignoreRevs: false }
RETURN NEW
`
return this.runQuery(touchAql)
}
trx.run(() => this.touchDocument(parentId, parentRevision))
Will this.arangodb.query() execute inside the transaction or not?

I'm the author of arangojs. For posterity I'd like to clarify that this answer is about arangojs 6, which is the current release of arangojs at the time of this writing and the version that first added support for streaming transactions. The API may change in future versions although there are currently no plans to do so.
Because of how transactions work (as of arangojs 6), you need to pay special attention to the caveat mentioned in the documentation for the run method:
If the given function contains asynchronous logic, only the synchronous part of the function will be run in the transaction. E.g. when using async/await only the code up to the first await will run in the transaction. Pay attention to the examples below.
In other words, async functions will likely not behave correctly if you just wrap them in trx.run. I'd recommend passing the transaction object to each function and wrapping the method calls in those functions in trx.run.
For example:
async runQuery(query, trx) {
try {
const cursor = await trx.run(() => this.arangodb.query(query))
return trx.run(() => cursor.all())
} catch (error) {
logger.error('ArangoDB query failed', { stack: error })
throw error
}
}
async touchDocument(id, revision, trx) {
const type = await this.getObjectTypeFromId(id, trx)
const collection = await this.getCollection(type, false, true, trx)
const idCollection = await this.getCollection(ID_COLLECTION, false, true, trx)
const touchAql = aql`
LET permanentDocId = DOCUMENT(${idCollection}, ${id}).docId
LET permanentDoc = MERGE( DOCUMENT(permanentDocId), { _rev : ${revision} })
UPDATE permanentDoc WITH permanentDoc in ${collection} OPTIONS { ignoreRevs: false }
RETURN NEW
`
return this.runQuery(touchAql, trx)
}
this.touchDocument(parentId, parentRevision, trx)
The reason behind this is that trx.run sets the entire driver into "transaction mode", executes the given function and then disables "transaction mode" after executing it so unrelated code doesn't accidentally run in the transaction.
The drawback of this approach is that if the function is async and contains multiple await statements, only the code leading up to and including the first await will be run in the transaction. In your code that means "transaction mode" is disabled after this.getObjectTypeFromId(id) returns. If that method itself contains multiple await expressions, again only the first one will be part of the transaction (and so forth).
I hope this clears up some of the confusion.

Related

Firebase function to add document with data to Firestore only works second and subsequent times of executing it, but not first

I'm getting a little problem which I'm not being capable to debug. I wrote a little Firebase Function to get data from a JSON object and to store it in a Firestore Document. Simple.
It works, except the first time I run it after deployed (or after a long time has passed since the last execution). I have to run it once (without working), and then the subsequent tries always work, and I can see the new document being created with all the data inside it.
In the first attempt, there are no logs: Function execution took 601 ms, finished with status code: 200. Despite that, no document is being created nor changes being made.
In the second and subsequent attempts, If I request the function execution with a HTTP POST to https://cloudfunctions/functionName?id=12345, then the document '12345' is created inside collection with all the data inside it.
The collection where the documents are stored (scenarios) already exist in the database before any function call is executed.
This is the code:
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const db = admin.firestore();
db.settings({ignoreUndefinedProperties: true});
const fetch = require("node-fetch");
let scenarioData;
const fetchScenarioJSON = async (scenarioId) => {
try {
const response = await fetch(`https://url/api/scenarios/single/${scenarioId}`);
const scenarioText = await response.text();
scenarioData = JSON.parse(scenarioText);
} catch (err) {
return ("not valid json");
}
return scenarioData;
};
/**
* Add data to Firestore.
* #param {JSON} scenario JSON array containing the scenario data.
*/
async function addDataToFirestore(scenario) {
const data = {
id: scenario.scenario._id,
name: scenario.scenario.name,
description: scenario.scenario.description,
language: scenario.scenario.language,
author: scenario.scenario.author,
draft: scenario.scenario.draft,
last_modified: scenario.scenario.last_modified,
__v: scenario.scenario.__v,
duration: scenario.scenario.duration,
grade: scenario.scenario.grade,
deleted: scenario.scenario.deleted,
view_count: scenario.scenario.view_count,
comments_count: scenario.scenario.comments_count,
favorites_count: scenario.scenario.favorites_count,
activities_duration: scenario.scenario.activities_duration,
activities: scenario.scenario.activities,
outcomes: scenario.scenario.outcomes,
tags: scenario.scenario.tags,
students: scenario.scenario.students,
created: scenario.scenario.created,
subjects: scenario.scenario.subjects,
};
const res = await db.collection("scenarios").doc(scenario.scenario._id).set(data);
}
exports.functionName =
functions.https.onRequest((request, response) => {
return fetchScenarioJSON(request.query.id).then((scenario) => {
if (typeof scenario === "string") {
if (scenario.includes("not valid json")) {
response.send("not valid json");
}
} else {
addDataToFirestore(scenario);
response.send(`Done! Added scenario with ID ${request.query.id} to the app database.`);
}
});
});
My question is if I am doing anything wrong with the code that makes the execution not work on the first call after it is deployed, but actually does work in subsequent calls.
It is most probably because you don't wait that the asynchronous addDataToFirestore() function is completed before sending back the response.
By doing
addDataToFirestore(scenario);
response.send()
you actually indicate (with response.send()) to the Cloud Function platform that it can terminate and clean up the Cloud Function (see the doc for more details). Since you don't wait for the asynchronous addDataToFirestore() function to complete, the doc is not written to Firestore.
The "erratic" behaviour (sometimes it works, sometimes not) can be explained as follows:
In some cases, your Cloud Function is terminated before the write to Firestore is fully executed, as explained above.
But, in some other cases, it may be possible that the Cloud Functions platform does not immediately terminate your CF, giving enough time for the write to Firestore to be fully executed. This is most probably what happens after the first call: the instance of the Cloud Function is still running and then the docs are written with the "subsequent calls".
The following modifications should do the trick (untested). I've refactored the Cloud Function with async/await, since you use it in the other functions.
// ....
async function addDataToFirestore(scenario) {
const data = {
id: scenario.scenario._id,
name: scenario.scenario.name,
description: scenario.scenario.description,
language: scenario.scenario.language,
author: scenario.scenario.author,
draft: scenario.scenario.draft,
last_modified: scenario.scenario.last_modified,
__v: scenario.scenario.__v,
duration: scenario.scenario.duration,
grade: scenario.scenario.grade,
deleted: scenario.scenario.deleted,
view_count: scenario.scenario.view_count,
comments_count: scenario.scenario.comments_count,
favorites_count: scenario.scenario.favorites_count,
activities_duration: scenario.scenario.activities_duration,
activities: scenario.scenario.activities,
outcomes: scenario.scenario.outcomes,
tags: scenario.scenario.tags,
students: scenario.scenario.students,
created: scenario.scenario.created,
subjects: scenario.scenario.subjects,
};
await db.collection("scenarios").doc(scenario.scenario._id).set(data);
}
exports.functionName =
functions.https.onRequest(async (request, response) => {
try {
const scenario = await fetchScenarioJSON(request.query.id);
if (typeof scenario === "string") {
if (scenario.includes("not valid json")) {
response.send("not valid json");
}
} else {
await addDataToFirestore(scenario); // See the await here
response.send(`Done! Added scenario with ID ${request.query.id} to the app database.`);
}
} catch (error) {
// ...
}
});

combining results from several async/await calls (again)

Note to would-be closers or markers-as-duplicate : this question is not answered in How do I convert an existing callback API to promises?, as all answers there treat of calls in isolation and do not explain how to deal with successive, dependent calls.
This question is basically the same as in combining results from several async/await calls, but because of my slightly different context I fail to see how the solution therein can be adapted/mimicked.
I have two successive calls to a database, using an old API which only knows about callbacks. The second call needs objects/values returned by the first.
I have a working callback-version of the code, as follows :
connection.query(sql1,function(error1,results1,fields1) {
if(error1) {
console.log("Error during first query",error1);
}
let sql2=computeUsingFirstResult(result1);
connection.query(sql2,function(error2,results2,fields2) {
if(error2) {
console.log("Error during second query",error2);
}
doSomething(connection,results1,results2);
})
});
Here is my unsuccessful attempt to do it in async/await-style :
const util = require('util');
async function firstQuery(connection) {
return util.promisify(connection.query).call(sql1).catch(
error1 => console.log("Error during first query : ", error1)
);;
}
async function secondQuery(connection, result1) {
let sql2 = computeUsingFirstResult(result1);
return util.promisify(connection.query).call(sql2).catch(
error2 => console.log("Error during second query : ", error2)
);
}
let finalResult = {};
async function main() {
const results1 = await firstQuery(connection);
const results2 = await secondQuery(connection, results1);
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}
main().catch(
err => console.log("Something went wrong towards the end", err)
);
My async/await version fails as all the intermediate results are undefined. Yet, as far as I can see it should be equivalent to the non-async version above.
What is the correct way to do this ?
There are 2 approaches I am hoping you can try and see if either of them help:
(1) Bind the util.promisify to connection for both firstQuery and secondQuery something like so:
async function firstQuery(connection) {
return util.promisify(connection.query).bind(connection,sql1)().catch(
error1 => console.log("Error during first query : ", error1)
);;
}
// bind makes sure that internal bindings of connection object are in tact
If the above approach doesn't yield any result, try the next approach:
(2) Try using Promises-Wrapper instead of util.promisify for both firstQuery and secondQuery like so:
function firstQuery(connection,sql1){
return new Promise((resolve,reject)=>{
connection.query(sql1,function(error1,results1,fields1){
return error1 ? reject(error1):resolve(results1);
})
});
}
And now call them as you were in your code using Async/Await in the main function.
Few Potential bugs/typos I noticed in the code:
(1) firstQuery doesn't seem to be passed sql1 as one of its arguments, maybe this is intentional if sql1 exists in global scope.
(2) If you attach a catch block to both your queries, they will have an impact on your 2 calls (see comments in the code below):
async function main() {
const results1 = await firstQuery(connection); //<-- if this call fails, results1 will be undefined and the call will go to next step
const results2 = await secondQuery(connection, results1); // same thing here
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}
A possible solution is to remove the catch blocks from individual functions and just return the respective promises.
Then you can deal with them via a single try/catch block like this:
async function main() {
try{
const results1 = await firstQuery(connection);
const results2 = await secondQuery(connection, results1);
doSomething(connection, results1, results2);
finalResult = [results1,results2];
console.log("Here is the finalResult : ", finalResult);
}catch(err){
console.log("Error",err);
}
}
This will ensure that as soon as the first call fails, function goes straight to catch block without executing any other lines.

Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call? Best practices for using Knex.Transaction

When working with a big application that has several tables and several DB operations it's very difficult to keep track of what transactions are occurring. To workaround this we started by passing around a trx object.
This has proven to be very messy.
For example:
async getOrderById(id: string, trx?: Knex.Transaction) { ... }
Depending on the function calling getOrderById it will either pass a trx object or not. The above function will use trx if it is not null.
This seems simple at first, but it leads to mistakes where if you're in the middle of a transaction in one function and call another function that does NOT use a transaction, knex will hang with famous Knex: Timeout acquiring a connection. The pool is probably full.
async getAllPurchasesForUser(userId: string) {
..
const trx = await knex.transaction();
try {
..
getPurchaseForUserId(userId); // Forgot to make this consume trx, hence Knex timesout acquiring connection.
..
}
Based on that, I'm assuming this is not a best practice, but I would love if someone from Knex developer team could comment.
To improve this we're considering to instead use knex.transactionProvider() that is accessed throughout the app wherever we perform DB operations.
The example on the website seems incomplete:
// Does not start a transaction yet
const trxProvider = knex.transactionProvider();
const books = [
{title: 'Canterbury Tales'},
{title: 'Moby Dick'},
{title: 'Hamlet'}
];
// Starts a transaction
const trx = await trxProvider();
const ids = await trx('catalogues')
.insert({name: 'Old Books'}, 'id')
books.forEach((book) => book.catalogue_id = ids[0]);
await trx('books').insert(books);
// Reuses same transaction
const sameTrx = await trxProvider();
const ids2 = await sameTrx('catalogues')
.insert({name: 'New Books'}, 'id')
books.forEach((book) => book.catalogue_id = ids2[0]);
await sameTrx('books').insert(books);
In practice here's how I'm thinking about using this:
SingletonDBClass.ts:
const trxProvider = knex.transactionProvider();
export default trxProvider;
Orders.ts
import trx from '../SingletonDBClass';
..
async getOrderById(id: string) {
const trxInst = await trx;
try {
const order = await trxInst<Order>('orders').where({id});
trxInst.commit();
return order;
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to fetch order, error: ${e}`);
}
}
..
Am I understanding this correctly?
Another example function where a transaction is actually needed:
async cancelOrder(id: string) {
const trxInst = await trx;
try {
trxInst('orders').update({ status: 'CANCELED' }).where({ id });
trxInst('active_orders').delete().where({ orderId: id });
trxInst.commit();
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to cancel order, error: ${e}`);
}
}
Can someone confirm if I'm understanding this correctly? And more importantly if this is a good way to do this. Or is there a best practice I'm missing?
Appreciate your help knex team!
No. You cannot have global singleton class returning the transaction for your all of your internal functions. Otherwise you are trying always to use the same transaction for all the concurrent users trying to do different things in the application.
Also when you once commit / rollback the transaction returned by provider, it will not work anymore for other queries. Transaction provider can give you only single transaction.
Transaction provider is useful in a case, where you have for example middleware, which provides transaction for request handlers, but it should not be started, since it might not be needed so you don't want yet allocate a connection for it from pool.
Good way to do your stuff is to pass transcation or some request context or user session around, so that each concurrent user can have their own separate transactions.
for example:
async cancelOrder(trxInst, id: string) {
try {
trxInst('orders').update({ status: 'CANCELED' }).where({ id });
trxInst('active_orders').delete().where({ orderId: id });
trxInst.commit();
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to cancel order, error: ${e}`);
}
}
Depending on the function calling getOrderById it will either pass a trx object or not. The above function will use trx if it is not null.
This seems simple at first, but it leads to mistakes where if you're in the middle of a transaction in one function and call another function that does NOT use a transaction, knex will hang with famous Knex: Timeout acquiring a connection. The pool is probably full.
We usually do it in a way that if trx is null, query throws an error, so that you need to explicitly pass either knex / trx to be able to execute the method and in some methods trx is actually required to be passed.
Anyhow if you really want to force everything to go through single transaction in a session by default you could create API modules in a way that for each user session you create an API instance which is initialized with transaction:
const dbForSession = new DbService(trxProvider);
const users = await dbForSession.allUsers();
and .allUsers() does something like return this.trx('users');

converting promiseAll to gradual promises resolve(every 3promises for example) does not work

I have a list of promises and currently I am using promiseAll to resolve them
Here is my code for now:
const pageFutures = myQuery.pages.map(async (pageNumber: number) => {
const urlObject: any = await this._service.getResultURL(searchRecord.details.id, authorization, pageNumber);
if (!urlObject.url) {
// throw error
}
const data = await rp.get({
gzip: true,
headers: {
"Accept-Encoding": "gzip,deflate",
},
json: true,
uri: `${urlObject.url}`,
})
const objects = data.objects.filter((object: any) => object.type === "observed-data" && object.created);
return new Promise((resolve, reject) => {
this._resultsDatastore.bulkInsert(
databaseName,
objects
).then(succ => {
resolve(succ)
}, err => {
reject(err)
})
})
})
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e)
})
So as you see here I use promise all and it works:
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e)
})
However I noticed it affects the database performance wise so I decided to resolve every 3 of them at a time.
for that I was thinking of different ways like cwait, async pool or wrting my own iterator
but I get confused on how to do that?
For example when I use cwait:
let promiseQueue = new TaskQueue(Promise,3);
const all=new Promise.map(pageFutures, promiseQueue.wrap(()=>{}));
I do not know what to pass inside the wrap so I pass ()=>{} for now plus I get
Property 'map' does not exist on type 'PromiseConstructor'.
So whatever way I can get it working(my own iterator or any library) I am ok with as far as I have a good understanding of it.
I appreciate if anyone can shed light on that and help me to get out of this confusion?
First some remarks:
Indeed, in your current setup, the database may have to process several bulk inserts concurrently. But that concurrency is not caused by using Promise.all. Even if you had left out Promise.all from your code, it would still have that behaviour. That is because the promises were already created, and so the database requests will be executed any way.
Not related to your issue, but don't use the promise constructor antipattern: there is no need to create a promise with new Promise when you already have a promise in your hands: bulkInsert() returns a promise, so return that one.
As your concern is about the database load, I would limit the work initiated by the pageFutures promises to the non-database aspects: they don't have to wait for eachother's resolution, so that code can stay like it was.
Let those promises resolve with what you currently store in objects: the data you want to have inserted. Then concatenate all those arrays together to one big array, and feed that to one database bulkInsert() call.
Here is how that could look:
const pageFutures = myQuery.pages.map(async (pageNumber: number) => {
const urlObject: any = await this._service.getResultURL(searchRecord.details.id,
authorization, pageNumber);
if (!urlObject.url) { // throw error }
const data = await rp.get({
gzip: true,
headers: { "Accept-Encoding": "gzip,deflate" },
json: true,
uri: `${urlObject.url}`,
});
// Return here, don't access the database yet...
return data.objects.filter((object: any) => object.type === "observed-data"
&& object.created);
});
const all: any = await Promise.all(pageFutures).catch(e => {
console.log(e);
return []; // in case of error, still return an array
}).flat(); // flatten it, so all data chunks are concatenated in one long array
// Don't create a new Promise with `new`, only to wrap an other promise.
// It is an antipattern. Use the promise returned by `bulkInsert`
return this._resultsDatastore.bulkInsert(databaseName, objects);
This uses .flat() which is rather new. In case you have no support for it, look at the alternatives provided on mdn.
First, you asked a question about a failing solution attempt. That is called X/Y problem.
So in fact, as I understand your question, you want to delay some DB request.
You don't want to delay the resolving of a Promise created by a DB request... Like No! Don't try that! The promise wil resolve when the DB will return a result. It's a bad idea to interfere with that process.
I banged my head a while with the library you tried... But I could not do anything to solve your issue with it. So I came with the idea of just looping the data and setting some timeouts.
I made a runnable demo here: Delaying DB request in small batch
Here is the code. Notice that I simulated some data and a DB request. You will have to adapt it. You also will have to adjust the timeout delay. A full second certainly is too long.
// That part is to simulate some data you would like to save.
// Let's make it a random amount for fun.
let howMuch = Math.ceil(Math.random()*20)
// A fake data array...
let someData = []
for(let i=0; i<howMuch; i++){
someData.push("Data #"+i)
}
console.log("Some feak data")
console.log(someData)
console.log("")
// So we have some data that look real. (lol)
// We want to save it by small group
// And that is to simulate your DB request.
let saveToDB = (data, dataIterator) => {
console.log("Requesting DB...")
return new Promise(function(resolve, reject) {
resolve("Request #"+dataIterator+" complete.");
})
}
// Ok, we have everything. Let's proceed!
let batchSize = 3 // The amount of request to do at once.
let delay = 1000 // The delay between each batch.
// Loop through all the data you have.
for(let i=0;i<someData.length;i++){
if(i%batchSize == 0){
console.log("Splitting in batch...")
// Process a batch on one timeout.
let timeout = setTimeout(() => {
// An empty line to clarify the console.
console.log("")
// Grouping the request by the "batchSize" or less if we're almost done.
for(let j=0;j<batchSize;j++){
// If there still is data to process.
if(i+j < someData.length){
// Your real database request goes here.
saveToDB(someData[i+j], i+j).then(result=>{
console.log(result)
// Do something with the result.
// ...
})
} // END if there is still data.
} // END sending requests for that batch.
},delay*i) // Timeout delay.
} // END splitting in batch.
} // END for each data.

How to prevent "Given transaction number 1 does not match any in-progress transactions" with Mongoose Transactions?

I am using Mongoose to access to my database. I need to use transactions to make an atomic insert-update.
95% of the time my transaction works fine, but 5% of the time an error is showing :
"Given transaction number 1 does not match any in-progress transactions"
It's very difficult to reproduce this error, so I really want to understand where it is coming from to get rid of it.
I could not find a very clear explanation about this type of behaviour.
I have tried to use async/await key words on various functions. I don't know if an operation is not done in time or too soon.
Here the code I am using:
export const createMany = async function (req, res, next) {
if (!isIterable(req.body)) {
res.status(400).send('Wrong format of body')
return
}
if (req.body.length === 0) {
res.status(400).send('The body is well formed (an array) but empty')
return
}
const session = await mongoose.startSession()
session.startTransaction()
try {
const packageBundle = await Package.create(req.body, { session })
const options = []
for (const key in packageBundle) {
if (Object.prototype.hasOwnProperty.call(packageBundle, key)) {
options.push({
updateOne: {
filter: { _id: packageBundle[key].id },
update: {
$set: {
custom_id_string: 'CAB' + packageBundle[key].custom_id.toLocaleString('en-US', {
minimumIntegerDigits: 14,
useGrouping: false
})
},
upsert: true
}
}
})
}
}
await Package.bulkWrite(
options,
{ session }
)
for (const key in packageBundle) {
if (Object.prototype.hasOwnProperty.call(packageBundle, key)) {
packageBundle[key].custom_id_string = 'CAB' + packageBundle[key].custom_id.toLocaleString('en-US', {
minimumIntegerDigits: 14,
useGrouping: false
})
}
}
res.status(201).json(packageBundle)
await session.commitTransaction()
} catch (error) {
res.status(500).end()
await session.abortTransaction()
throw error
} finally {
session.endSession()
}
}
I expect my code to add in the database and to update the entry packages in atomic way, that there is no instable database status.
This is working perfectly for the main part, but I need to be sure that this bug is not showing anymore.
You should use the session.withTransaction() helper function to perform the transaction, as pointed in mongoose documentation. This will take care of starting, committing and retrying the transaction in case it fails.
const session = await mongoose.startSession();
await session.withTransaction(async () => {
// Your transaction methods
});
Explanation:
The multi-document transactions in MongoDB are relatively new and might be a bit unstable in some cases, such as described here. And certainly, it has also been reported in Mongoose here. Your error most probably is a TransientTransactionError due to a write-conflict happening when the transaction is committed.
However, this is a known and expected issue from MongoDB and these comments explain their reasoning behind why they decided it to be like this. Moreover, they claim that the user should be handling the cases of write conflicts and retrying the transaction if that happens.
Therefore, looking at your code, the Package.create(...) method seems to be the reason why the error gets triggered, since this method is executing a save() for every document in the array (from mongoose docs).
A quick solution might be using Package.insertMany(...) instead of create(), since the Model.insertMany() "only sends one operation to the server, rather than one for each document" (from mongoose docs).
However, MongoDB provides a helper function session.withTransaction() that will take care of starting and committing the transaction and retry it in case of any error, since release v3.2.1. Hence, this should be your preferred way to work with transactions in a safer way; which is, of course, available in Mongoose through the Node.js API.
The accepted answer is great. In my case, I was running multiple transactions serially within a session. I was still facing this issue every now and then. I wrote a small helper to resolve this.
File 1:
// do some work here
await session.withTransaction(() => {});
// ensure the earlier transaction is completed
await ensureTransactionCompletion(session);
// do some more work here
await session.withTransaction(() => {});
Utils File:
async ensureTransactionCompletion(session: ClientSession, maxRetryCount: number = 50) {
// When we are trying to split our operations into multiple transactions
// Sometimes we are getting an error that the earlier transaction is still in progress
// To avoid that, we ensure the earlier transaction has finished
let count = 0;
while (session.inTransaction()) {
if (count >= maxRetryCount) {
break;
}
// Adding a delay so that the transaction get be committed
await new Promise(r => setTimeout(r, 100));
count++;
}
}

Resources