I'm retrieving a list of people from the database using getPeople(). As soon as I receive them as res, I want to prepare them to be stored in my local mongodb if they do not exist. Sometimes there're duplicate entries (for one id) within res. My issues is that it's not waiting for Person.create(pers) to finish, continues searching if this id is already in mongodb, can't find any since Person.create(pers) is still creating it and starts the second Person.create(pers)..
this.getPeople()
.then(res => {
return Promise.all(res.map(pers => {
pers.birthday = df(pers.birthday, 'dd.mm.yyyy')
pers.pickedUp = false
console.log(pers.id)
return Person
.find({ id: pers.id })
.exec()
.then(found => {
if (found === undefined || found.length == 0)
return pers
})
.then(pers => {
return Person
.create(pers)
.then(console.log('created'))
.catch(err => console.log(err))
})
}))
}).catch(err => console.log(err))
I expected the console output to be like this:
940191
created
940191
created
Instead, I'm getting this:
940191
940191
created
created
That's because Promise.all simply awaits all the promises you're mapping. It does not guarantee any order in which the promises are resolved.
If you want to sequentially process the elements of your res-array, you can simply use a for .. of loop in combination with async/await (note that this still needs some error handling, but it should give you something to start with):
async function getPeopleAndCreateIfNotExisting() {
const persons = [];
const res = await this.getPeople();
for (const pers of res) {
pers.birthday = df(pers.birthday, 'dd.mm.yyyy');
pers.pickedUp = false;
console.log(pers.id)
const found = await Person
.find({ id: pers.id }).exec();
if (found) {
persons.push(pers);
} else {
persons.push(await Person.create(pers));
}
}
return person;
}
Related
I am using Node.js (which I am very new at) and Google Cloud Firestore database to save documents according to:
Users
Tweets
One Users document, i.e. a User, has many Tweets in a subcollection 'Tweets'. I am interested in retrieving a User together with the last Tweet in the subcollection so I get a JSON-file like this. In other words, this is what I want to get:
users: {
{
name:'john',
twitter_username:'johnny',
tweets: {
text: "I am called johnny, this is my tweet",
created_at: "2021-06-29 12:00:00"
},
},
{
name:'anne',
twitter_username:'anne',
tweets: {
text: "I am called anne, this is another tweet",
created_at: "2019-06-28 12:00:00"
},
}
}
I have this function:
function getUserData() {
return db.collection('users').get()
.then((querySnapshot) => {
var docs = querySnapshot.docs.map(doc => [doc.data(), doc.id]);
//console.log(docs);
return docs
which, if I could fetch and replace doc.id (i.e. the User document ID) with the last tweet, would solve it I guess. But how can I do that?
Any other solution, possibly with for loops, would be fine as well. I have spent hours on this seemingly easy problem but can't get it to return both the User-data and the tweet-data.
Edit 2:
I realized I could do this:
function getUserData() {
return db.collection('users').get()
.then((querySnapshot) => {
var docs = querySnapshot.docs.map(doc => [doc.data(), doc.id, getStuff(doc.id)])
console.log(docs)
return docs
});
}
function getStuff(doc_id) {
return db.collection('users').doc(doc_id).collection('tweets').limit(1).get()
.then((querySnapshot) => {
var docs = querySnapshot.docs.map(doc => doc.data());
console.log("TWEETS", doc_id, docs[0]['text']);
return docs[0]['text']
});
}
which produces a log result as:
TWEETS DAU86mxIhmD6qQQpH4F God’s country!
TWEETS JQHTO0jUjAodMQMR6wI I’m almost there Danny!
from the getStuff-function.
The only issue now is that I can't get the map function to wait for getStuff so the return docs return a Promise { <pending> } for getStuff(doc.id).
I am not to familiar with Promises and await/async and I can't get that to work. Solving this Promise pending -> twitter text would then solve my problem. How do I do that?
If you want to get the data of a single user you could write the code like this:
const getUserData = async (userUid) => {
const userSnap = await db.collection("users").doc(userUid).get();
const tweetSnaps = await db
.collection("tweets")
.orderBy("created_at", "desc")
.limit(1)
.get();
let tweet = {};
tweetSnaps.forEach((doc) => {
tweet = doc.data();
});
return {
...userSnap.data(),
...tweet,
};
};
We first get the user and then query for the last tweet and get that. We sort the tweets collection by created_at and limit it for a single doc.
If you want to get the same data for all users at once we would need to change the code a little bit but the logic would be the same.
If the data is saved in separate collections you can't get them in a single database request.
UPDATE for Edit 2
Here your code how it should look like with correct async/await:
const getUserData = async () => {
const querySnapshot = await db.collection("users").get();
const docs = querySnapshot.docs;
for (let i = 0; i < docs.length; i++) {
const element = docs[i];
doc.data(),
doc.id,
await getStuff(doc.id),
}
console.log(docs);
return docs;
};
const getStuff = async (doc_id) => {
const querySnapshot = await db
.collection("users")
.doc(doc_id)
.collection("tweets")
.limit(1)
.get();
var docs = querySnapshot.docs.map((doc) => doc.data());
console.log("TWEETS", doc_id, docs[0]["text"]);
return docs[0]["text"];
};
Building a NodeJS REST API.
Trying to send load data from FireBase collection, then sending it to the user (as API response).
Looks like the problem is that it's not waits for the firebase fetch to resolve, but send back a response without the collection data. (tried to use ASYNC-AWAIT but its not working)
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {
snapshot.docs.forEach(msg => {
console.log(msg.data().messageContent)
return {
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}
})
})
}
try {
const chatData = await getChatData()
console.log(chatData)
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData
})
} catch (err) {
if (!err.statusCode) {
err.statusCode(500)
}
next(err)
}
}
As you can see, I've used 2 console.logs to realize what the problem, Terminal logs looks like:
[] (from console.logs(chatData))
All messages (from console.log(msg.data().messageContent))
Is there any way to block the code unti the firebase data realy fetched?
If I correctly understand, you want to send back an array of all the documents present in the messages subcollection. The following should do the trick.
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId;
const collectionRef = db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc');
try {
const chatsQuerySnapshot = await collectionRef.get();
const chatData = [];
chatsQuerySnapshot.forEach((msg) => {
console.log(msg.data().messageContent);
chatData.push({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
});
});
console.log(chatData);
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData,
});
} catch (err) {
if (!err.statusCode) {
err.statusCode(500);
}
next(err);
}
};
The asynchronous get() method returns a QuerySnapshot on which you can call forEach() for enumerating all of the documents in the QuerySnapshot.
You can only await a Promise. Currently, getChatData() does not return a Promise, so awaiting it is pointless. You are trying to await a fixed value, so it resolves immediately and jumps to the next line. console.log(chatData) happens. Then, later, your (snapshot) => callback happens, but too late.
const getChatData = () => new Promise(resolve => { // Return a Promise, so it can be awaited
db.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot(resolve) // Equivalent to .onSnapshot((snapshot) => resolve(snapshot))
})
const snapshot = await getChatData();
console.log(snapshot)
// Put your transform logic out of the function that calls the DB. A function should only do one thing if possible : call or transform, not both.
const chatData = snapshot.map(msg => ({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}));
res.status(200).json({
message: 'Chat Has Found',
chatData
})
Right now, getChatData is this (short version):
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {}) // some things inside
}
What that means is that the getChatData function calls some db query, and then returns void (nothing). I bet you'd want to return the db call (hopefully it's a Promise), so that your await does some work for you. Something along the lines of:
const getChatData = async () =>
db
.collection('chats')
// ...
Which is the same as const getChatData = async() => { return db... }
Update: Now that I've reviewed the docs once again, I see that you use onSnapshot, which is meant for updates and can fire multiple times. The first call actually makes a request, but then continues to listen on those updates. Since that seems like a regular request-response, and you want it to happen only once - use .get() docs instead of .onSnapshot(). Otherwise those listeners would stay there and cause troubles. .get() returns a Promise, so the sample fix that I've mentioned above would work perfectly and you don't need to change other pieces of the code.
I'm trying to reload a model in order to include association. This is my code:
const order = new Order({ total: 0 })
return order.save().then(savedOrder => {
const orderItems = req.body.items.map(i => {
return Eliquid.findByPk(i.eliquid).then(eliquid => {
const item = Item.build({
qty: i.qty,
n: i.n,
})
item.setEliquid(eliquid)
item.setOrder(savedOrder)
return item
.save().then(i => {
console.log('Saved, has id, has qty', i.id, i.qty)
return i.reload()
})
.then(i => {
console.log('Reloaded, qty is now NULL!', i.id, i.qty)
return i
})
})
})
Why after reloading my instance it gets whiped out? Don't know what I'm doing wrong
As far as I can tell, .reload() (unlike .save()) doesn't deliver the item back to the client; instead, it synchronises the original item's properties with those on the server.
I'm not sure you actually need to reload in this case because client & server representations of the item will automatically be in sync immediately after .save(), but here's an example of how you would exploit javascript's "closure" to maintain access to the item, after .reload().
const order = new Order({ 'total': 0 });
return order.save()
.then(savedOrder => {
const orderItems = req.body.items.map(i => {
return Eliquid.findByPk(i.eliquid)
.then(eliquid => {
const item = Item.build({ 'qty': i.qty, 'n': i.n });
item.setEliquid(eliquid);
item.setOrder(savedOrder);
return item.save();
})
.then(ii => {
console.log('Saved, has id, has qty', ii.id, ii.qty);
return ii.reload()
.then(() => { // .reload() does not repeat item back to client
console.log('same ii as before', ii.id, ii.qty); // here, access ii through closure.
return ii;
});
});
});
});
Notes:
for clarity ii is used in the second .then() callback to avoid reusing the .map() callback's symbol i.
the inner promise chain is flattened as far as possible. If you needed access to eliquid in the second inner .then(), you would need to revert to return item.save().then() and incur a futher level of nesting.
in addition to exploiting closure, other approaches exit - see the comprehensive answers here.
I currently have the following code
router.get('/uri', (request,response) => {
let final = [];
TP.find({userID: request.userID})
.then(tests =>{
tests.forEach(test => {
A.findById(test.assignmentID)
.then(assignment => {
final.push({
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
})
})
.catch(error => {
console.log(error)
})
})
return response.send(final);
})
.catch(err => {
console.log(err);
return response.sendStatus(500);
})
})
The code is supposed to query 2 MongoDB databases and construct an array of objects with specific information which will be sent to the client.
However, I always get an empty array when I call that endpoint.
I have tried making the functions async and make them wait for results of the nested functions but without success - still an empty array.
Any suggestions are appreciated!
forEach doesn't care about promises inside it. Either use for..of loop or change it to promise.all. The above code can be simplified as
router.get('/uri', async (request,response) => {
const tests = await TP.find({userID: request.userID});
const final = await Promise.all(tests.map(async test => {
const assignment = await A.findById(test.assignmentID);
return {
testID: test._id,
name: assignment.title,
successRate: `${test.passedTests}/${test.totalTests}`
};
}));
return response.send(final);
});
Hope this helps.
I'm trying to create new rows in the database using Sequalize ORM. I receive an array of collections from req.query.collections. For each of those collections I need to create a new userCollection. If none userCollections were created, I wanna respond with internal server error (line 41), otherwise return an array of objects with newly created userCollections.
The problem is, I keep getting an internal server error when I make test requests from Postman. When I check my database, I see that those userCollections were created, so no error occurred.
I know why this happens: because userCollection.build({ stuff }).save() returns a promise. So when I try to console.log userCollections from within .then() statement, I get an array with a newly created collections, just like I should. But by that time server has already responded with internal server error.
Here's my function code:
exports.addCollections = async (req, res, next) => {
const libraryId = req.params.libraryId;
const collections = req.query.collections;
if (!collections)
next(Boom.forbidden());
const userCollections = [];
collections.forEach(async (collectionId, index) => {
const collection = await Collection.findByPk(collectionId);
if (!collection)
return next(Boom.notFound());
userCollection.build({
user_id: req.user.id,
library_id: libraryId,
public_collection_id: collection.id,
title: collection.title,
description: collection.description
})
.save()
.then(newUserCollection => {
userCollections.push(newUserCollection.get({ plain: true }));
// should be printed first, but comes second
// prints out the array with newly created record
console.log(userCollections);
})
.catch(error => {
console.log(error);
});
});
// should be printed second, but comes first
// prints out empty array
console.log(userCollections);
if (userCollections.length === 0) {
next(Boom.internal());
}
res.json(userCollections);
}
Posting the solution
Thanks to Sebastien Chopin who created this tutorial:
https://codeburst.io/javascript-async-await-with-foreach-b6ba62bbf404
So I added this function:
const asyncForEach = async (array, callback) => {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array)
}
}
And instead of collections.forEach...(blah blah) (line 10 of the code posted in the question) I do:
try {
await asyncForEach(collections, async (collectionId, index) => {
const collection = await Collection.findByPk(collectionId);
if (!collection)
return next(Boom.notFound());
userCollection.build({
user_id: req.user.id,
library_id: libraryId,
public_collection_id: collection.id,
title: collection.title,
description: collection.description
})
.save()
.then(newUserCollection => {
userCollections.push(newUserCollection.get({ plain: true }));
console.log(userCollections);
})
.catch(error => {
console.log(error);
});
})
} catch (err) {
return next(Boom.internal(err.message));
}