Combine file API with transactions for file saving or deleting - node.js

I am using node 8.11.1 with pg-promise 8.4.4 to handle queries and transactions in PostgreSQL. This is about node, but I guess is the same logic in other servers/tools too.
The scenario is common. I want to save an image file in a folder, then if this is successful, insert its details in the database, get the returned id and then do another insert in a secondary, many-to-many table.
Clearly, I need a transaction for the insert queries. But what about the actual file saving? My approach is
fs.rename(oldpath, newpath, (err) => {
if (err){throw new Error ;}
db.tx('my-transaction', t => {
return t.one('INSERT INTO images(whatever) VALUES($1) RETURNING id', ['whatever'])
.then(user => {
return t.batch([
t.none('INSERT INTO mtm(userId, name) VALUES($1, $2)', [user.id, 'created'])
]);
});
})
.then(data => {
// success
})
.catch(error => {
// error
});
}); //fs rename
Ok, if there is no error while saving the image file with fs.rename, then proceed with the transaction.
If there is an error while saving the image, nothing will execute, so all good.
But the problem is, what if the image is saved and there is an error in the transaction? I will end up with an image saved and nothing in the database. Sure, user will get an error and will have to re-upload, but I still have images in my server that are not related to anything. I would like to avoid this.
The solution would be to incorporate the image saving in the transaction, so if anything fails, nothing is completed. How can I do this? I dont know if the file API can be inside a query-related transaction. I dont even know if I am in the right mindset here.
Please advice or help me code this.
Thank you

Simply remove the file, if transaction fails (or you can rename it back, if you prefer):
const fs = require('fs-extra');
async function saveAll(oldpath, newpath) {
await fs.rename(oldpath, newpath);
try {
return await db.tx('my-transaction', async t => {
const imageId = await t.one('INSERT INTO images(whatever) VALUES($1) RETURNING id', ['whatever'], a => a.id);
await t.none('INSERT INTO mtm(userId, name) VALUES($1, $2)', [imageId, 'created']);
return imageId;
});
} catch (e) {
await fs.unlink(newpath); // deleting the file
throw e;
}
}
Function saveAll will return the new imageId, if successful, or throw an error, if anything fails:
async test() {
try {
const imageId = await saveAll('old-path', 'new-path');
// we are all good
} catch(e) {
// something failed, as per the error details
}
}

Related

Find document from mongoose collections with specific condition

Recently I start using MongoDB with Mongoose on Nodejs.
This code works as it should, and returns me all data i need :
const getAllPosts = async () => {
try {
return (await PostModel.find().populate('user')).reverse();
} catch (error) {
console.log(error);
throw Error('Error while getting all posts');
}
};
But now I only need individual posts, which in the tags (represented as an array in the PostModel) contain the data that I will pass in the request.
For example, I will make a GET request to /posts/tag111 and should get all posts that have "tag111" in the tags array.
Any ways to do this?
If you are using expressjs, your route should be something like:
whatever.get('/:tag', postController.getAllPostTagged)
The function you use in your route (called getAllPostTagged above) should be similar to this one, in which you get the path param tag from req:
const postController = {
getAllPostTagged = async(req, res) => {
try {
return (await PostModel.find({tags: req.params.tag}));
} catch (error) {
console.log(error);
throw Error('Error while getting all posts');
}
}
}
The key here is to know where the params are obtained (from req.params) .

node.js firestore delete with a where clause

I've tried to figure this out but don't know enough about node.js or firestore to make it happen. I want to delete a bunch of documents based on a where clause. I have tried a bunch of iterations of this but none of them delete, and using the Google provided code returns an async error:
db.collection("testcollection")
.where("office", "==", 12345).limit(3).get().then((snapshot)=> {
snapshot.docs.forEach(doc => {
//const res = await db.collection('testcollection').doc(id).delete();
const data = doc.data();
console.log(doc.id);
delete(doc.id);
})
});```
Well shoot. Right after posting this I figured it out by using an answer from here Cloud Firestore delete function
and putting doc.id into a variable:
snapshot.docs.forEach(doc => {
const tempid = doc.id;
db.collection("testcollection").doc(tempid).delete().then(function() {
console.log("Document successfully deleted!");
}).catch(function(error) {
console.error("Error removing document: ", error);
});
})
});

How can I force my function to wait promises before moving on?

I read another posts but they were not solving my particular problem.
I'm trying to move all dependent entries from one user to a default one before deleting the user. But even with Promise.all().then() I'm getting an error saying that I still have foreign key constraints preventing me to delete the user (But if i try to delete it manually after that it works fine, because I really moved all the dependencies to the default user).
async delete (req, res){
try {
const {id} = req.body;
// Here I call all the recipes that this user owns
const recipes = await Recipe.find({where: {chef_id: id}});
// Create the Promises for every UPDATE request in the Database
const changeChefPromises = recipes.map( recipe => {
Recipe.update(recipe.id,{chef_id : 1})
});
/* HERE is the problem, I'm trying to update every single recipe
before I delete the User, but my program is trying to delete
the User before finishing the Update process for all dependencies */
await Promise.all(changeChefPromises).then(() => {
Chef.delete(id);
});
return res.redirect(`/admin/chefs`);
} catch (err) {
console.error(err);
}
}
const changeChefPromises = recipes.map( recipe => {
Recipe.update(recipe.id,{chef_id : 1})
});
This is not creating an array of promises which is what you're expecting. You either need to remove the body of the arrow function or explicitly return the promise inside the body.
E.g.
const changeChefPromises = recipes.map(recipe => Recipe.update(recipe.id,{chef_id : 1}));
or
const changeChefPromises = recipes.map( recipe => {
return Recipe.update(recipe.id,{chef_id : 1})
});
Also, it's a bit weird to mix async/await and .then() so it might also be a good idea to change:
await Promise.all(changeChefPromises).then(() => {
Chef.delete(id);
});
to
await Promise.all(changeChefPromises);
await Chef.delete(id);

Unhandled Rejection (FirebaseError): No document to update

I'm still very new to coding so bear with me! I have followed a youtube course to build a note app and get a base to work from, but I'm now getting this error at random times when deleting the notes in firebase, hoping someone might be able to spot what's cooking here!
"Unhandled Rejection (FirebaseError): No document to update: projects/speakle-dc94b/databases/(default)/documents/notes/GdWPrQNxR3Z9TFMWmqOZ"
And it's referencing the node modules like so:
screenshot of the error in chrome
The code I have that interacts with firebase looks like this:
componentDidMount = () => {
firebase
.firestore()
.collection('notes')
.onSnapshot(serverUpdate => {
const notes = serverUpdate.docs.map(_doc => {
const data = _doc.data();
data['id'] = _doc.id;
return data;
});
console.log(notes);
this.setState({ notes: notes });
});
}
selectNote = (note, index) => this.setState({ selectedNoteIndex: index, selectedNote: note });
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
newNote = async (title) => {
const note = {
title: title,
body: ''
};
const newFromDB = await firebase
.firestore()
.collection('notes')
.add({
title: note.title,
body: note.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
const newID = newFromDB.id;
await this.setState({ notes: [...this.state.notes, note] });
const newNoteIndex = this.state.notes.indexOf(this.state.notes.filter(_note => _note.id === newID)[0]);
this.setState({ selectedNote: this.state.notes[newNoteIndex], selectedNoteIndex: newNoteIndex });
}
deleteNote = async (note) => {
const noteIndex = this.state.notes.indexOf(note);
await this.setState({ notes: this.state.notes.filter(_note => _note !== note) })
if(this.state.selectedNoteIndex === noteIndex) {
this.setState({ selectedNoteIndex: null, selectedNote: null});
} else {
this.state.notes.lenght > 1 ?
this.selectNote(this.state.notes[this.state.selectedNoteIndex - 1], this.state.selectedNoteIndex - 1) :
this.setState({ selectedNoteIndex: null, selectedNote: null });
}
firebase
.firestore()
.collection('notes')
.doc(note.id)
.delete()
.then(function() {
console.log("Document successfully deleted!");
}).catch(function(error) {
console.error("Error removing document: ", error);
});
}
}
It simply means that there is no document of that name to be uploaded.
you could either use set() or add() to add the document because it is not present.
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
replace the above code with this
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.add({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
or
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.set({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
I was working with something like this only especially in Cloud Functions and while writing an endpoint for doing some operation I came across the below-quoted error.
I was trying to read a document in a collection and if it existed I was trying to write a new doc into another collection. So it was kind of a nested code.
A piece of my code.
const firstDocRef = db.collection('myFirstCollection').doc('myDocName');
const existDoc = firstDocRef.get()
.then((resDoc)=>{
if(resDoc.exists)
{
db.collection('mySecondCollection').add({
.
.
.
.
.
orderCreatedAt:Firestore.FieldValue.serverTimestamp()
})
.then((new_doc)=>{
return res.status(200);
// return 200 ok what we were trying to achieve has completed.
})
.catch(()=>{
console.log("Log, as write failed somehow");
return res.status(500);
// return a 500 internal server error occurred
});
}
else
{
console.log("My first condition wasn't met, so logged it");
return res.end();
// properly terminate the processing of request
}
});
/*.catch((err)=>{
console.log("Our first doc presence check couldn't complete and hence I arrived here, log it");
res.writeHead(500);
return res.end();
// again give back 500 to client
});*/
UnhandledPromiseRejectionWarning: ReferenceError: Firestore is not defined
UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a catch block
Now I am also new to Firebase but I came across this and somehow solved it.
So I was not getting the above error if I was putting in a catch block in get() document.
Strange huh!
Removed the catch block by commenting it. Got this error.
Now, this is a haywire error, it says the catch is not there, but we did it on purpose.
So I began searching, came across this question here on stack overflow and saw it's still unanswered. Searched and read the documentation myself.
I would like to tell you that this isn't because of any Firestore Security Rules, or anything else. Because I came across some guesses around these notions too while searching for an answer.
The common thing we all are doing here is that we are trying to achieve the ServerTimeStamp at FireStore
I would like to bring your notice to my imports in my node cloud function code.
const functions = require('firebase-functions');
const express = require('express');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
So you see, I am using the new way of getting the permission to use Firestore because I am trying to establish a cloud function.
Now this is the documentation reference provided by Google: Click here!
The right syntax proposed by above API reference is
Firestore.FieldValue.serverTimestamp()
It is the culprit, it isn't providing me any timestamp and if there is not a catch block unhandled promise error is occuring and no error is being shown while debugging, it just doesn't work.
I did this, Solution part:
Even after those imports in my Node Program, I imported this:
const {Firestore} = require('#google-cloud/firestore');
Now all I did was that I used the statement in the timestamp field as
Firestore.FieldValue.serverTimestamp()
Exactly as mentioned and even used a catch block just in case any other problem occurs while at production. That is using db constant to do all the Database Transactional Stuff and just for serverTimeStamp I have to bring in new import.
And it worked, I guess require('#google-cloud/firestore') statement imported as {FireStore} brings in all the things that are required for the FieldValue thing to use as a reference.
I hope it helps any new person looking for it and saves a lot of time which I wasted searching for a solution.
I have verified it by running the cloud function on firebase emulator.
You could simply do it like this
You
Get it
If it exsits: you update it
If it doesn't exist, you set it.
const docRef = this.db.collection("users_holdings").doc(userId);
docRef.get().subscribe((doc) => {
if (doc.exists) {
docRef.update({
stockHoldings: stockHoldings,
});
} else {
docRef.set({
stockHoldings: stockHoldings,
});
}
});

Node - Mysql Transaction is not rolling back if one query fails

I am trying to perform node-mysql transaction which has three query in total. All three are 'INSERT' queries. I intensionally write third query wrong to test rollback but transaction is making entry to database without fail for first two queries.
I know similar question has already been asked several times and I tried almost all of them but no luck
exports.registerNewUserTransaction = async (
res,
userToBeAdded,
nameToBeAdded,
emailToBeAdded) => {
const conn = await db.getConnection();
await conn.beginTransaction();
try {
await this.insertOne('user', userToBeAdded);
await this.insertOne('name', nameToBeAdded);
await this.insertOne('email', emailToBeAdded);
await conn.commit();
res.status(200);
} catch(err) {
await conn.rollback();
res.status(400);
} finally {
await conn.release();
}
};
As you can see I am getting connection object from Pool, beginning the transaction and performing queries one by one. My third query has wrong column name, hence transaction should rollback but I see entry of first two queries. I would really appreciate the correct direction.
node version: 12.8.0
mysql (running in docker): 8.0.15
mysql (npm version): 2.17.1
After struggling a lot, finally figured it out. Here's the answer:
exports.registerNewUserTransaction = async (
res,
userToBeAdded,
nameToBeAdded,
emailToBeAdded) => {
const conn = await db.getConnection();
// My first mistake was not to promisify connection query
conn.query = util.promisify(conn.query);
await conn.beginTransaction();
try {
// My second mistake was not to use same connection
await conn.query('INSERT INTO ...', userToBeAdded);
await conn.query('INSERT INTO ...', nameToBeAdded);
await conn.query('INSERT INTO ...', emailToBeAdded);
await conn.commit();
return res.status(200);
} catch(err) {
await conn.rollback();
return res.status(400);
} finally {
await conn.release();
}
};
Hope this might help someone!
This is important part I missed.
conn.query = util.promisify(conn.query);

Resources