async await with promise all - node.js

I want to know if I am using promise.all correctly with async await.
Basically, I need to get the house data based on the ID, then I need to get all the reviews for that house as well as the count of reviews.
server.get("/api/houses/:id", async (req, res) => {
const { id } = req.params;
const house = await House.findByPk(id);
if (!house) {
return res.status(400).send("No house found");
}
const reviews = await Review.findAndCountAll({
where: {
houseId: house.id
}
});
house.dataValues.reviewsCount = reviews.count;
const results = await Promise.all([house.dataValues, reviews.rows]);
console.log(results);
res.send(results);
});
In the front end when I console.log the response after making the http request, I get back the below which seems okay since Promise.all gives you arrays. But I don't know if this is the best way to do this or if there is a better way.
[
{
id: 2329,
host: 2,
picture: '/img/houses/1.jpg',
type: 'Entire house',
town: 'Some town',
title: 'Some title',
price: 50,
description: 'Some description',
guests: 4,
bedrooms: 1,
beds: 2,
baths: 1,
wifi: true,
reviewsCount: 2
},
[
{
id: 1,
houseId: 2329,
userId: 1,
comment: 'An awesome review',
createdAt: '2019-01-11T22:00:00.000Z',
updatedAt: '2019-01-11T22:00:00.000Z'
},
{
id: 2,
houseId: 2329,
userId: 2,
comment: 'Another awesome review',
createdAt: '2019-01-11T22:00:00.000Z',
updatedAt: '2019-01-11T22:00:00.000Z'
}
]
]

You're not using Promise.all correctly. The code is working, because you're awaiting each promise individually.
Since Review.findAndCountAll depends on House.findByPk result, Promise.all won't do any good here.
You're using Promise.all with the already resolved values of the both promises, so you can just drop it.
server.get("/api/houses/:id", async (req, res) => {
const { id } = req.params;
const housePromise = await House.findByPk(id);
const reviews = await Review.findAndCountAll({
where: {
houseId: house.id
}
});
house.dataValues.reviewsCount = reviews.count;
res.send([house.dataValues, reviews.rows]);
});
Basically you're doing:
const res = await Promise.all([1, 5]); // [1, 5]
Which can be translated directly to:
const res = [1, 5];
Instead of sending it in an array, I think it's better to send an object:
{
house: house.dataValues,
reviews: reviews.rows
}

You can ignore async await and use Promise. You can try following code
server.get("/api/houses/:id", async (req, res) => {
const { id } = req.params;
return House.findByPk(id)
.then( house => {
// !house might be 'true' if house is 'undefined'
if( house === undefined || !house ) {
return res.status(400).send("No house found");
}
return Review.findAndCountAll({ where: { houseId: house.id } })
.then(reviews => {
house.dataValues.reviewsCount = reviews.count;
return {house, reviews};
})
.catch(error => res.send(error));
})
.then( result => {
return res.send(results);
})
.catch(error => {
return res.send(error);
});
})

Related

NodeJS Array not filling up

Im using mongodb as a database after getting the valuation data which are more then one , i loop inside them and get the offers according to the offres_id , I asure you that the database has data inside and to querys are working correctly in fact if i console log them in each iteration i get this result
{
offre: [
{
_id: new ObjectId("63320bf87123db5691c51392"),
user_id: '63304e44aa63c519d887dac1',
brand: 'HP',
model: 'AZGGH-89855A',
photo: '16642242480661659650294043-bs-5.png',
scan_method: 'manual',
__v: 0
}
],
valuation: {
_id: new ObjectId("63320d39a5677df3cebcbdae"),
user_id: '63304e44aa63c519d887dac1',
offre_id: '63320bf87123db5691c51392',
given_price: '1236',
comment: 'no comment',
__v: 0
}
}
{
offre: [
{
_id: new ObjectId("6334202a8c7e6d90b35ee999"),
user_id: '63304e44aa63c519d887dac1',
brand: 'DELL',
model: 'AZGGH-89855A',
photo: '1664360490280Capture.PNG',
scan_method: 'manual',
__v: 0
}
],
valuation: {
_id: new ObjectId("633420be8c7e6d90b35ee99e"),
user_id: '63304e44aa63c519d887dac1',
offre_id: '6334202a8c7e6d90b35ee999',
__v: 0
}
}
but when i try storing each offre & valuation at the same array cell and return it i get this as a result
[]
However this is the code
router.get('/get', async (req, res) => {
try {
Valuation.find({ user_id: req.session.userID })
.exec()
.then(valuation => {
let myData = [];
if (valuation) {
for (let i = 0; i < valuation.length; i++) {
Offre.find({_id : valuation[i].offre_id})
.exec()
.then(offre=>{
myData.push({offre : offre, valuation : valuation[i]})
})
}
res.status(200).json(myData)
} else {
res.status(404).json('no valuations found')
}
})
.catch(error => {
res.status(500).json(error.message)
})
} catch (error) {
res.status(500).json({ error: error.message })
}
})
Since you're already using async, it would be a shame to not use await to simplify your code:
router.get("/get", async (req, res) => {
try {
const valuation = await Valuation.find({ user_id: req.session.userID }).exec();
if (valuation) {
let myData = [];
for (let i = 0; i < valuation.length; i++) {
const offre = await Offre.find({ _id: valuation[i].offre_id }).exec();
myData.push({ offre: offre, valuation: valuation[i] });
}
res.status(200).json(myData);
} else {
res.status(404).json("no valuations found");
}
} catch (error) {
res.status(500).json({ error: error.message });
}
});
You can probably also speed up the Offre query by using an $in query to retrieve all offres with one query, but that's another thing.

API Only sends 1 chunk of metadata when called

I have a problem with my API that sends metadata when called from my smart contract of website. Its NFT tokens and my database is postgres and API is node.js
The problem is when I mint 1 NFT metadata works perfect, but if I mint 2 or more it will only ever send 1 chunk of data? So only 1 NFT will mint properly and the rest with no data?
Do I need to set a loop function or delay? Does anyone have any experience with this?
Any help would be much appreciated.
Below is the code from the "controller" folder labeled "nft.js"
const models = require("../../models/index");
const path = require("path");
const fs = require("fs");
module.exports = {
create_nft: async (req, res, next) => {
try {
const dir = path.resolve(__dirname + `../../../data/traitsfinal.json`);
const readCards = fs.readFileSync(dir, "utf8");
const parsed = JSON.parse(readCards);
console.log("ya data ha final ??", parsed);
parsed.forEach(async (item) => {
// return res.json(item)
let newNft = await models.NFT.create({
name: item.Name,
description: item.Description,
background: item.Background,
body: item.Body,
mouth: item.Mouth,
eyes: item.Eyes,
head_gear: item.Head_Gear,
tokenId: item.tokenId,
image: item.imagesIPFS,
});
});
return res.json({
data: "nft created",
error: null,
success: true,
});
} catch (error) {
console.log("server error", error.message);
next(error);
}
},
get_nft: async (req, res, next) => {
try {
const { id } = req.params;
// console.log("id ?????????",id)
// console.log("type of ",typeof(id))
// const n=Number(id)
// console.log("type of ",typeof(id))
const nft = await models.NFT.findByPk(id);
if (!nft) {
throw new Error("Token ID invalid");
}
if (!nft.isMinted) {
throw new Error("Token not minted");
}
console.log(nft);
// }
const resObj = {
name: nft.name,
description: nft.description,
image: `https://gateway.pinata.cloud/ipfs/${nft.image}`,
attributes: [
{ trait_type: "background", value: `${nft.background}` },
{ trait_type: "body", value: `${nft.body}` },
{ trait_type: "mouth", value: `${nft.mouth}` },
{ trait_type: "eyes", value: `${nft.eyes}` },
{ trait_type: "tokenId", value: `${nft.tokenId}` },
{
display_type: "number",
trait_type: "Serial No.",
value: id,
max_value: 1000,
},
],
};
return res.json(resObj);
} catch (error) {
console.log("server error", error.message);
next(error);
}
},
get_nft_all: async (req, res, next) => {
try {
// console.log("id ?????????",id)
// console.log("type of ",typeof(id))
// const n=Number(id)
// console.log("type of ",typeof(id))
const nft = await models.NFT.findAndCountAll({
limit: 10
});
// console.log(nft);
if (!nft) {
throw new Error("Token ID invalid");
}
// if (nft.isMinted) {
// throw new Error("Token not minted");
// }
// console.log(nft);
// }
var resObjarr = [];
for (var i = 0; i < nft.rows.length; i++) {
resObj = {
name: nft.rows[i].name,
description: nft.rows[i].description,
image: `https://gateway.pinata.cloud/ipfs/${nft.rows[i].image}`,
attributes: [
{ trait_type: "background", value: `${nft.rows[i].background}` },
{ trait_type: "body", value: `${nft.rows[i].body}` },
{ trait_type: "mouth", value: `${nft.rows[i].mouth}` },
{ trait_type: "eyes", value: `${nft.rows[i].eyes}` },
{ trait_type: "tokenId", value: `${nft.rows[i].tokenId}` },
{
display_type: "number",
trait_type: "Serial No.",
value: nft.rows[i].id,
max_value: 1000,
},
],
};
resObjarr.push(resObj);
}
console.log(JSON.stringify(resObjarr))
return res.json(resObjarr);
} catch (error) {
console.log("server error", error.message);
next(error);
}
},
mint: async (req, res, next) => {
try {
const { id } = req.params;
const updated = await models.NFT.findByPk(id);
if (!updated) {
throw new Error("NFT ID invalid");
}
if (updated.isMinted) {
throw new Error("NFT Already minted");
}
updated.isMinted = true;
updated.save();
return res.json({
data: "Token minted successfully",
error: null,
success: true,
});
} catch (error) {
console.log("server error", error.message);
next(error);
}
},
};
Below is from the routes folder.
const router = require("express").Router();
const auth=require("../middleware/auth")
const {
create_nft,
get_nft,
get_nft_all,
mint
} = require("../controller/nft");
router.post(
"/create",
create_nft
);
router.get(
"/metadata/:id",
get_nft
);
router.get(
"/metadata",
get_nft_all
);
router.put(
"/mint/:id",
mint
);
module.exports = router;
Looking your code,you may having some kind of asyncrhonous issue in this part:
parsed.forEach(async (item) => {
// return res.json(item)
let newNft = await models.NFT.create({
name: item.Name,
description: item.Description,
background: item.Background,
body: item.Body,
mouth: item.Mouth,
eyes: item.Eyes,
head_gear: item.Head_Gear,
tokenId: item.tokenId,
image: item.imagesIPFS,
});
});
Because .forEach is a function to be used in synchronous context and NFT.create returns a promise (that is async). So things happens out of order.
So one approach is to process the data first and then perform a batch operation using Promise.all.
const data = parsed.map(item => {
return models.NFT.create({
name: item.Name,
description: item.Description,
background: item.Background,
body: item.Body,
mouth: item.Mouth,
eyes: item.Eyes,
head_gear: item.Head_Gear,
tokenId: item.tokenId,
image: item.imagesIPFS,
})
})
const results = await Promise.all(data)
The main difference here is Promise.all resolves the N promises NFT.create in an async context in paralell. But if you are careful about the number of concurrent metadata that data may be too big to process in parallel, then you can use an async iteration provided by bluebird's Promise.map library.
const Promise = require('bluebird')
const data = await Promise.map(parsed, item => {
return models.NFT.create({
name: item.Name,
description: item.Description,
background: item.Background,
body: item.Body,
mouth: item.Mouth,
eyes: item.Eyes,
head_gear: item.Head_Gear,
tokenId: item.tokenId,
image: item.imagesIPFS,
})
})
return data

Array keep empty after push object from forEach

Need Help, I have problem Array Keep Empty After pushing object from forEach, do I miss something ? here's the code :
const allStatus = [];
result.forEach(async (element) => {
const count = await BpCandidate.count({
where: {
applicationStatusId: element.id
},
raw: true,
});
})
record = {
id: element.id,
name: element.name,
code: element.code,
description: element.description,
total: count
};
allStatus.push(record);
});
console.log(allStatus);
Thanks in advance
Using for...of is working for you, but can see an improvement here if you use Promise.all
const allStatus = await Promise.all(result.map(async (element) => {
const count = await BpCandidate.count({
where: {
applicationStatusId: element.id
},
raw: true,
});
return {
id: element.id,
name: element.name,
code: element.code,
description: element.description,
total: count
};
}))
for each doesn't perform async task and doesn't resolve promises.
Use for of or Promise.All the loop iterations.
const allStatus = [];
for(let element of result) {
const count = await BpCandidate.count({
where: {
applicationStatusId: element.id
},
raw: true,
});
})
record = {
id: element.id,
name: element.name,
code: element.code,
description: element.description,
total: count
};
allStatus.push(record);
});
console.log(allStatus);
first you need to define element variable in outer scope of foreach part to get the desired result, here is the helper code snippet:
const allStatus = [];
result.forEach(async (element) => {
const count = await BpCandidate.count({
where: {
applicationStatusId: element.id
},
raw: true,
});
record = {
id: element.id,
name: element.name,
code: element.code,
description: element.description,
total: count
};
allStatus.push(record);
})
});
console.log(allStatus);

Creating stub for sequeilze models with association

I am using mocha and chai for writing test for RESTful APIs
I have read some articles where people suggests to create stubs for queries, and you shouldn't be actually making a database query.
But How would I make sure if it works?
See below controller.
const Op = require('sequelize').Op
//Models
const {
Item,
Location,
Combo,
Service,
ComboItem,
ItemLocation
} = require('../models')
const _ = require('lodash')
//Services
const paginate = require('../services/PaginationService')
const getAllItems = async function(req, res) {
if(req.query.location_id){
let items
const item = await Location.findOne({
where: {
id: 1
},
include: {
model: Item,
through: {
model: ItemLocation,
attributes: []
},
as: 'itemsAtLocation',
include: [
{
model: Service,
as: 'service',
attributes: ["id"]
},
{
model: Combo,
as: 'combo',
attributes: ["start_date", "expiry_date"]
}
]
}
})
if(!item)
return res.status(200).send({
status: true,
message: "No item found at location!",
data: {}
})
items = item.itemsAtLocation
let data = {}
data.services = []
data.combos = []
_.forEach(items, item => {
let itemData = {
id: item.id,
name: item.name,
price: item.price,
discount_per: item.discount_per,
}
if(item.service)
data.services.push(itemData)
if(item.combo) {
itemData.start_date = item.combo.start_date
itemData.expiry_date = item.combo.expiry_date
data.combos.push(itemData)
}
})
return res.status(200).send({
status: true,
message: "Successfully fetch all items!",
data: data
})
} else {
const items = await Item.findAll({
include: [
{
model: Service,
as: 'service',
attributes: ["id"]
},
{
model: Combo,
as: 'combo',
attributes: ["start_date", "expiry_date"]
}
],
attributes: ["id", "name", "price", "discount_per", "description"],
...paginate(+req.query.page, +req.query.per_page)
})
let data = {}
data.services = []
data.combos = []
_.forEach(items, item => {
let itemData = {
id: item.id,
name: item.name,
price: item.price,
discount_per: item.discount_per,
}
if(item.service)
data.services.push(itemData)
if(item.combo) {
itemData.start_date = item.combo.start_date
itemData.expiry_date = item.combo.expiry_date
data.combos.push(itemData)
}
})
return res.status(200).send({
status: true,
message: "Successfully fetch all items!",
data: data
})
}
}
module.exports = {
getAllItems
}
As you can see from above code. I need queries to return data in a specific form. If it won't be in that form things won't work.
Can someone suggest how can I create stubs for such kind of functions so that structure also be preserved?
Below is the test that I have wrote, But it uses actual db calls.
describe('GET /api/v1/items', function () {
it('should fetch all items orgianized by their type', async () => {
const result = await request(app)
.get('/api/v1/items')
.set('Accept', 'application/json')
.expect('Content-Type', /json/)
.expect(200)
expect(result)
.to.be.a('Object')
expect(result.body.status)
.to.be.a('Boolean').true
expect(result.body.data, "data should be an Object and every key should an Array")
.to.satisfy(data => {
expect(data).to.be.a('Object')
.to.not.be.null
if(!_.isEmpty(data)) {
expect(data).to.have.any.keys('services', 'combos')
_.forOwn(data, (value, key) => {
expect(data[key]).to.be.a('Array')
})
return true
}
return true
})
})
})
One way you can do that is by stubbing the methods from your models, i.e. Location.findOne and Item.findAll. So your tests could look a bit like the code below:
const sinon = require('sinon');
const Location = require('../models/location'); // Get your location model
const Item = require('../models/item'); // Get your item model
describe('myTest', () => {
let findOneLocationStub;
let findAllItemsStub;
beforeEach(() => {
findOneLocationStub = sinon.stub(Location, 'findOne');
findAllItemsStub = sinon.stub(Item, 'findAll');
});
afterEach(() => {
findOneLocationStub.verifyAndRestore();
findAllItemsStub.verifyAndRestore();
});
it('returns 200 when location not found', () => {
findOneLocationStub.resolves(null);
expects...
});
});
I did not run the test, but something like that should work. But note that I had to split the models into their own file to do the stub. Probably there's a way to do the same using your current implementation.
Another thing I would suggest is having some kind of use case into your method that is responsible for database implementation. Something like:
const getAllItemsUseCase = (params, queryService) => {
if(params.locationId){
let items
const item = await queryService.findOneLocation({
};
So when you call this method from your controller, you can do call:
const getAllItems = async function(req, res) {
const params = {
locationId: req.query.location_id,
// and more parameters
};
const queryService = {
findOneLocation: Location.findOne,
};
const results = await getAllItemsUseCase(params, queryService);
}
This way you will detach your business logic from the controller and you will have a much easier time to mock your query: you just change the methods provided to queryService.
You can find some interesting read from this blog post: https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

Import large amounts of data, but do a .find() for each element

I have a collection of customers of 60.000 items. I need to import a list of people, of 50.000. But for each person, I need to find the ID of the customer and add that to the object that is being inserted.
How should this be done most efficently?
export default async ({ file, user, database }: Request, res: Response) => {
try {
const csv = file.buffer.toString("utf8");
let lines = await CSV({
delimiter: "auto" // delimiter used for separating columns.
}).fromString(csv);
let count = {
noCustomer: 0,
fails: 0
};
let data: any = [];
await Promise.all(
lines.map(async (item, index) => {
try {
// Find customer
const customer = await database
.model("Customer")
.findOne({
$or: [
{ "custom.pipeID": item["Organisasjon - ID"] },
{ name: item["Person - Organisasjon"] }
]
})
.select("_id")
.lean();
// If found a customer
if (!isNil(customer?._id)) {
return data.push({
name: item["Person - Navn"],
email: item["Person - Email"],
phone: item["Person - Telefon"],
customer: customer._id,
updatedAt: item["Person - Oppdater tid"],
createdAt: item["Person - Person opprettet"],
creator: users[item["Person - Eier"]] || "5e4bca71a31da7262c3707c5"
});
}
else {
return count.noCustomer++;
}
} catch (err) {
count.fails++;
return;
}
})
);
const people = await database.model("Person").insertMany(data)
res.send("Thanks!")
} catch (err) {
console.log(err)
throw err;
}
};
My code just never sends an response If I use this as a Express request.

Resources