Creating a multipart database object with Sequelize: when will the promise complete? - node.js

Imagine you are working with a database of national teams and athletes (like for the Olympic games). When testing your app, you create a new database every time. Here are two models:
const Team = sequelize.define('team', {
name: {type: Sequelize.TEXT, allowNull: false}
})
const Athlete = sequelize.define('athlete', {
name: {type: Sequelize.TEXT, allowNull: false}
})
Team.hasMany(Athlete)
Athlete.belongsTo(Team)
Now, you have a function that creates the teams for you:
function mock_database() {
let team0 = Team.create({
name: 'TeamGB'
}).then(team => {
generate_athletes(team)
// Or maybe I need to return the nested promises?
// return generate_athletes(team)
})
let team1 = Team.create({
name: 'TeamFr'
}).then(team => {
generate_athletes(team)
// Or maybe I need to return the nested promises?
// return generate_athletes(team)
})
return [team0, team1]
}
Now, the generate_athletes creates nested database objects (athletes in the team):
function generate_athletes(team) {
let athlete0 = Athlete.create({
name: 'John Smith'
}).then(athlete => {
team.setAthletes(athlete)
athlete.setTeam(team)
})
let athlete1 = Athlete.create({
name: 'Joanna Smith'
}).then(athlete => {
team.setAthletes(athlete)
athlete.setTeam(team)
})
return [athlete0, athlete1]
}
At this point, I would like to wait until the database has all the data. I can do that with Promise.all(mock_database()).then(() => {/*my application starts here*/}), which would be waiting on the two promises team0 and team1. But does that guarantee that I will be waiting on athlete0 and athlete1 implicitly too? Or do I need to return those promises back up, incorporate them all into a big array of [team0, team1, promises0, promises1] and wait on that?

You would need to return the nested promises. If you don't return the nested promise then your top-level promises will just resolve immediately after creating the teams (since creating the athletes is async and you aren't waiting for it to finish).
Essentially you would want something like this...
function mock_database() {
const team0 = Team.create({ // <-- team0::Promise
// ...
});
const team1 = Team.create({ // <-- team1::Promise
// ...
})
.then(team => {
return generate_athletes(team); // <-- Will only resolve team1 when this resolves
});
return Promise.all([team0, team1]); // Wait for both promises to resolve
}
function generate_athletes(team) {
const athlete1 = Athlete.create({ // athlete1::Promise
// ...
});
const athlete2 = Athlete.create({ // athlete2::Promise
// ...
});
return Promise.all([athlete1, athlete2]); // Wait for both promises to resolve
}
Unless you explicitly return the promises your program does not have access to the then and catch properties of the promise. So always remember, if you want to wait for a promise you will definitely need access to it! So always return it!
Good luck :)
EDIT: You can find a Promise.all explanation here. Essentially though what is happening is that each database action returns a promise. Those promises will resolve if and only if the database action is successful. Since promises are just regular JavaScript objects, they can be saved to a variable.
What I ended up doing was saving a reference to all of the promises I wanted to resolve and then used a builtin promise function to wait for the completion of all of the promises. Since the athletes depend on the team being created, for each team we also need to wait for the list of athletes to be generated.
I marked up the code with some more comments!

Related

Typeorm Unit Test FindOneOrFail

I am trying to add unit tests for my Nest app that uses typeorm. My user service has a mock repository set up like this:
const mockUsersRepository = {
create: jest.fn().mockImplementation((dto) => dto),
insert: jest.fn((user) => Promise.resolve({ id: 1, ...user })),
findOneOrFail: jest.fn(({ where: { id } }) =>
Promise.resolve({
id: id,
firstName: 'John',
lastName: 'Smith',
created: '2022-08-18T04:43:26.035Z',
}),
)}
This works fine as a simple test to see if my service can return a user based on an ID passed in to it. The problem is that I use the findOneOrFail method in other endpoints like save like this:
async save(createUserDto: CreateUserDto): Promise<User> {
const newUser = this.usersRepository.create(createUserDto);
const insertedRecord = await this.usersRepository.insert(newUser);
return this.findOneOrFail(insertedRecord.identifiers[0].id);
}
Since the save returns the findOneOrFail method, if I want to test this save method it will always return the value that I set in the mock repository above so I can't test if it will return a new save value or if it returns a new updated value. Am I supposed to make the mock findOneOrFail more generic to handle if it's used in multiple endpoints?
You are creating the mock in your code here:
const mockUsersRepository = {
create: jest.fn().mockImplementation((dto) => dto),
insert: jest.fn((user) => Promise.resolve({ id: 1, ...user })),
findOneOrFail: jest.fn(({ where: { id } }) =>
Promise.resolve({
id: id,
firstName: 'John',
lastName: 'Smith',
created: '2022-08-18T04:43:26.035Z',
}),
)}
The Promise.resolve is what is returning the data for you. I do this a little different, and use the #golevelup/ts-jest library for easy mocking. I highly recommend this for this type of testing.
import { createMock } from '#golevelup/ts-jest'; // Using this to very easily mock the Repository
The beforeEach to create your mock
beforeEach(async () => {
...
const mockUsersRepository = createMock<usersRepository>(); // Create the full mock of the TypeORM Repository
...
In the actual test then, the simple negative test for an exception being thrown:
it('should error out if ...', async () => {
jest.spyOn(mockUsersRepository, 'create').mockImplementation( () => {
throw new Error('Mock error'); // Just error out, contents are not important
});
expect.assertions(3); // Ensure there are 3 expect assertions to ensure the try/catch does not allow code to to bypass and be OK without an exception raised
try {
await service.save(...);
} catch (Exception) {
expect(Exception).toBeDefined();
expect(Exception instanceof HttpException).toBe(true);
expect(Exception.message).toBe('Mock error');
}
});
});
In another test, that requires two steps, you can use the mockResolvedValueOnce() multiple times. For instance, maybe the call to the actual database function needs to read twice, once that it does not exist, and then once that it does. This next test covers that the code being called will do 2 findAll calls.
mockResolvedValueOnce('A').mockResolvedValueOnce('B').mockResolvedValueOnce('C') will return the 'A' for the first call, 'B' for the second, and 'C' for the 3rd and any additional calls. You can also chain mockRejectValueOnce() and others.
In your save function, what I have found is you should simply test the save() function, and have the usersRepository mocked. This way you only need to test the return from this function, without all the work against a data source. Therefore, the mock is on the save() and you simply return the instance of the User that you want.
async save(createUserDto: CreateUserDto): Promise<User> {
const newUser = this.usersRepository.create(createUserDto);
const insertedRecord = await this.usersRepository.insert(newUser);
return this.findOneOrFail(insertedRecord.identifiers[0].id);
}
However, if you want to mock just the storage calls, then the usersRepository is what you need to mock. Then you need to create the mockResolvedValueOnce (or mockResolvedValue() for all calls), against the .create() method, something against the insert() method, and then something against the findOneOrFail() method. All of these are called, so you mock needs to handle all 3.
In these examples, the mockResolvedValueOnce() is simply declaring the return, without having to do the implementation you have done via jest.fn, and mockImplementation.

Firebase function to add document with data to Firestore only works second and subsequent times of executing it, but not first

I'm getting a little problem which I'm not being capable to debug. I wrote a little Firebase Function to get data from a JSON object and to store it in a Firestore Document. Simple.
It works, except the first time I run it after deployed (or after a long time has passed since the last execution). I have to run it once (without working), and then the subsequent tries always work, and I can see the new document being created with all the data inside it.
In the first attempt, there are no logs: Function execution took 601 ms, finished with status code: 200. Despite that, no document is being created nor changes being made.
In the second and subsequent attempts, If I request the function execution with a HTTP POST to https://cloudfunctions/functionName?id=12345, then the document '12345' is created inside collection with all the data inside it.
The collection where the documents are stored (scenarios) already exist in the database before any function call is executed.
This is the code:
const functions = require("firebase-functions");
const admin = require("firebase-admin");
admin.initializeApp();
const db = admin.firestore();
db.settings({ignoreUndefinedProperties: true});
const fetch = require("node-fetch");
let scenarioData;
const fetchScenarioJSON = async (scenarioId) => {
try {
const response = await fetch(`https://url/api/scenarios/single/${scenarioId}`);
const scenarioText = await response.text();
scenarioData = JSON.parse(scenarioText);
} catch (err) {
return ("not valid json");
}
return scenarioData;
};
/**
* Add data to Firestore.
* #param {JSON} scenario JSON array containing the scenario data.
*/
async function addDataToFirestore(scenario) {
const data = {
id: scenario.scenario._id,
name: scenario.scenario.name,
description: scenario.scenario.description,
language: scenario.scenario.language,
author: scenario.scenario.author,
draft: scenario.scenario.draft,
last_modified: scenario.scenario.last_modified,
__v: scenario.scenario.__v,
duration: scenario.scenario.duration,
grade: scenario.scenario.grade,
deleted: scenario.scenario.deleted,
view_count: scenario.scenario.view_count,
comments_count: scenario.scenario.comments_count,
favorites_count: scenario.scenario.favorites_count,
activities_duration: scenario.scenario.activities_duration,
activities: scenario.scenario.activities,
outcomes: scenario.scenario.outcomes,
tags: scenario.scenario.tags,
students: scenario.scenario.students,
created: scenario.scenario.created,
subjects: scenario.scenario.subjects,
};
const res = await db.collection("scenarios").doc(scenario.scenario._id).set(data);
}
exports.functionName =
functions.https.onRequest((request, response) => {
return fetchScenarioJSON(request.query.id).then((scenario) => {
if (typeof scenario === "string") {
if (scenario.includes("not valid json")) {
response.send("not valid json");
}
} else {
addDataToFirestore(scenario);
response.send(`Done! Added scenario with ID ${request.query.id} to the app database.`);
}
});
});
My question is if I am doing anything wrong with the code that makes the execution not work on the first call after it is deployed, but actually does work in subsequent calls.
It is most probably because you don't wait that the asynchronous addDataToFirestore() function is completed before sending back the response.
By doing
addDataToFirestore(scenario);
response.send()
you actually indicate (with response.send()) to the Cloud Function platform that it can terminate and clean up the Cloud Function (see the doc for more details). Since you don't wait for the asynchronous addDataToFirestore() function to complete, the doc is not written to Firestore.
The "erratic" behaviour (sometimes it works, sometimes not) can be explained as follows:
In some cases, your Cloud Function is terminated before the write to Firestore is fully executed, as explained above.
But, in some other cases, it may be possible that the Cloud Functions platform does not immediately terminate your CF, giving enough time for the write to Firestore to be fully executed. This is most probably what happens after the first call: the instance of the Cloud Function is still running and then the docs are written with the "subsequent calls".
The following modifications should do the trick (untested). I've refactored the Cloud Function with async/await, since you use it in the other functions.
// ....
async function addDataToFirestore(scenario) {
const data = {
id: scenario.scenario._id,
name: scenario.scenario.name,
description: scenario.scenario.description,
language: scenario.scenario.language,
author: scenario.scenario.author,
draft: scenario.scenario.draft,
last_modified: scenario.scenario.last_modified,
__v: scenario.scenario.__v,
duration: scenario.scenario.duration,
grade: scenario.scenario.grade,
deleted: scenario.scenario.deleted,
view_count: scenario.scenario.view_count,
comments_count: scenario.scenario.comments_count,
favorites_count: scenario.scenario.favorites_count,
activities_duration: scenario.scenario.activities_duration,
activities: scenario.scenario.activities,
outcomes: scenario.scenario.outcomes,
tags: scenario.scenario.tags,
students: scenario.scenario.students,
created: scenario.scenario.created,
subjects: scenario.scenario.subjects,
};
await db.collection("scenarios").doc(scenario.scenario._id).set(data);
}
exports.functionName =
functions.https.onRequest(async (request, response) => {
try {
const scenario = await fetchScenarioJSON(request.query.id);
if (typeof scenario === "string") {
if (scenario.includes("not valid json")) {
response.send("not valid json");
}
} else {
await addDataToFirestore(scenario); // See the await here
response.send(`Done! Added scenario with ID ${request.query.id} to the app database.`);
}
} catch (error) {
// ...
}
});

Returning the value in a promise.then

I have read some topics related to this but did not help, or I could not make them work. I have:
exports.findBookRating = (book) => {
var average;
Review.findAll({
where: {
bookId: book.id
},
attributes: [[sequelize.fn('avg', sequelize.col('rating')), 'avg_rated']],
raw: true
})
.then(number => {
average = number[0].avg_rated;
console.log(average); //this works
})
.catch(err => {
console.log(err);
});
console.log(average) // this does not work
}
I wanted to return number[0].avg_rated, and that is why I created that average variable but it did not work, can you please help me? Thank you!
"Promise.findA" is async, while the rest is sync. Your last line will execute before your Promise resolved and thus returning "undefined".
your Review.findAll is asynchronous which means right after it's called, it'll proceed to your console.log(average) // this does not work
And since it is asynchronous, by the time the console.log(average) // this does not work is called, average might not have yet been set.
Continue your code inside the .then part or use await if you need the value from an async call.
Example of using await:
const number = await Review.findAll()
const average = number[0].avg_rated

Multiple Callback Functions with the Node Twitter Library

I'm trying to make requests to Twitter with Twitter for Node and store responses in an array for handling later. I'm pushing returned tweets to an array with each push() happening in a callback, which seems to be working fine. My problem is that I'm having trouble accessing the full array with all pushed tweets.
The cause, of course, is that any attempt to work with that array is getting called before the results from the Twitter API have arrived, so I'm getting an empty array.
How can I (and should I) get my function working with the full array into another callback? I'm asking this from the perspective of someone still trying to get a firm grasp on asynchronous programming - especially multiple callbacks or functions that have to run asynchronously.
Again, current result is tweetHold = [], and I'd like tweetHold to contain all matched tweets for all users in the searchArray.
let searchArray = {
users: ['ByBuddha', 'thetweetofgod']
}
let tweetHold = [];
let T = new Twitter(config);
for (user of searchArray.users) {
let params = {
q: 'from:'+ user,
count: 1,
tweet_mode: 'extended',
result_type: 'recent',
lang: 'en'
}
T.get('search/tweets', params, returnedTweets);
}
function returnedTweets(err, tweets, response) {
tweetHold.push(tweets);
}
// obviously, doesn't work as the array is logged to console before T.get() is done
console.log(tweetHold);
T.get accepts a callback function that gets called once the asynchronous operation is done. But since you want to get multiple responses, and not just one, using just the callbacks by themselves would be a bit messy. For example, inside returnedTweets, you could increment a persistent counter variable, and call the next function once counter === searchArray.users.length, but it would be more elegant to use Promises instead.
Map each T.get call to a Promise that resolves with the tweets variable you're interested in, and then call Promise.all on an array of those Promises. Promise.all takes an array of Promises and returns a Promise that resolves once every Promise in the passed array has resolved.
Note that it looks like you're currently ignoring the err that might come back from T.get - that's probably not a good idea, it would be better to be able to check when errors occur, and then handle the error somehow (otherwise, the tweetHold array may sometimes contain broken data). Luckily, if you use Promises, implementing this is easy - just reject if there's an err, and catch after the Promise.all:
const T = new Twitter(config);
const searchObject = {
users: ['ByBuddha', 'thetweetofgod']
};
const searchPromises = searchArray.users.map((user) => {
return new Promise((resolve, reject) => {
const params = {
q: 'from:'+ user,
count: 1,
tweet_mode: 'extended',
result_type: 'recent',
lang: 'en'
};
T.get('search/tweets', params, (err, tweets) => {
if (err) reject(err);
else resolve(tweets);
});
});
});
Promise.all(searchPromises)
.then((tweetHold) => {
// tweetHold will be an array containing the `tweets` variable for each user searched
console.log(tweetHold);
})
.catch((err) => {
// there was an error, best to handle it somehow
// the `.then` above will not be entered
});

how to call function where parameter based on a promise

had following code which worked:
let sdk = new SDK({ name: "somevalue"});
module.exports= ()=> {
sdk.dothing();
}
I then needed to change the parameter to use data from an async function:
let asyncfunc = require('asyncfunc');
let sdk = new SDK({ name: (()=>{
return asyncfunc()
.then((data) => {
return data.value
})
});
module.exports= ()=> {
sdk.dothing();
}
Following the change, the call to new SDK is failing because the parameter passed is {} as the asyncFunc promise has not yet resolved.
I'm getting back into node after a year and new to promises. what is the proper way to do this?
As you've found, you can't pass in a promise to something that's expecting a string. You need to wait for the asynchronous operation to complete.
This means that your SDK won't be ready right away, so you have two options:
Change your module so it returns a promise for the needed value. Anyone who needs to use your module would need to use the returned promise.
Example:
let pSdk = asyncFunc()
.then(data => new SDK({ name: data.value }));
module.exports = () => pSdk.then(sdk => sdk.dothing());
Store an sdk value that's not populated immediately. Users of your module can obtain the SDK instance directly, but it might not be ready when they need it.
Example:
let sdk;
asyncFunc()
.then(data => sdk = new SDK({ name: data.value }));
module.exports = () => {
if(!sdk) { throw new Error("The SDK is not ready yet!"); }
return sdk.dothing();
};
if any bit of code, in node, is asynchronous then immediately next bit of code will be executed. it doesn't matter if the asynchronous code is wrapped in promise or not.( For codes wrapped in the promise the compiler will return a pending promise to be resoled or rejected and proceed to the next bit of code.) When you are creating an object using new SDK({ }) the name is having reference to a pending promise which is yet to be settled that's why your code is failing to fulfill your requirement. You can do it this way to resolve your problem.
asyncfunc()
.then((data) => {
return new SDK({ name: data.value });
}).then(function(sdk){
//do your work here using sdk
})
One important point to be noted here is you can't return from .then() to assign the value to any variable as you are doing. The value returned from .then() will be accessible from the next chained .then() not by outside global variable.Since you are exporting sdk.dothing() so you need to export it inside the last .then()

Categories

Resources