In the code below, I want to check if the headers are valid and then stop reading the stream if they are and send an error back to the client (via my ErrorHandler function). I see the following in my console:
UnhandledPromiseRejectionWarning: Error: Please check your column headers.
Why is that?
uploadExercisesViaFile = async (
req: Request,
res: Response,
next: NextFunction
) => {
try {
let checker = (arr, target) => target.every((v) => arr.includes(v));
const columns = ["name", "link", "description"];
const results = [];
let errors: string[] = [];
const parseCSV = parseString(req.file.buffer.toString(), {
headers: true,
maxRows: 5,
})
.on("headers", (header) => {
if (!checker(header, columns)) {
errors.push("Please check your column headers.");
parseCSV.end();
}
})
.on("data", async (row) => {
console.log(row);
results.push(row);
})
.on("error", (error) => {
errors.push(error.message);
})
.on("end", async (rowCount) => {
// Check if errors
if (errors.length > 0) {
console.log(errors);
throw new ErrorHandler(400, errors[0]);
}
// Upload
const uploadedExercises = await Exercises.batchUploadExercise(
results.map((r) => {
return { ...r, access: req.body.access, coach: req.user.id };
})
);
res.status(200).json({ rowCount: rowCount });
return;
});
} catch (e) {
next(e);
}
};
Related
I am having trouble sending an error to the front end when a csv file is uploaded and the numbers already exist in the database. The backend is logging an error that the primary key value already exist, but the code I have written tells the front end that the file uploaded just fine.
Code snippet:
router.post('/:program/upload', upload.single('testUpload'), (req, res, next) => {
try {
CSVtoPSQL(req.params.program, req.file.filename)
return res.status(201).json({
message: 'File Uploaded Just fine :)'
});
} catch (error) {
return res.status(500).json({
message: error
})
}
});
const CSVtoPSQL = (program, filePath) => {
let stream = fs.createReadStream(path.resolve(__dirname, '../files', filePath));
let csvData = [];
let csvStream = csv
.parse({ headers: false })
.on('error', error => console.error(error))
.on('data', (data) => {
csvData.push(data.toString());
})
.on('end', () => {
csvData.forEach(item => {
queries.upload(program, item)
.then(() => {
console.log('QR Code Added: ' + item);
}).catch((err) => {
console.log(`oopsie: ${err}`);
});
})
});
stream.pipe(csvStream);
}
Pretty confident the issue is with my poor understanding of promises.
As expected, I wasn't handling my promises correctly. I've updated the code a bit and now it responds with 2 arrays of successful uploads and errored uploads.
router.post('/:program/upload', upload.single('testUpload'), async (req, res, next) => {
try {
const result = await CSVtoPSQL(req.params.program, req.file.filename)
return res.status(201).json(result);
}
catch (error) {
return res.status(500).json({
message: error,
})
}
});
const CSVtoPSQL = (program, filePath) => {
let stream = fs.createReadStream(path.resolve(__dirname, '../files', filePath));
let csvData = [];
return new Promise((resolve) => {
const results = {
seccess: [],
error: [],
}
let csvStream = csv
.parse({ headers: false })
.on('error', error => console.error(error))
.on('data', (data) => {
csvData.push(data.toString());
})
.on('end', async () => {
await Promise.all(
csvData.map(async (item) => {
try{
await queries.upload(program, item);
results.success.push(item);
console.log('QR Code Added: ' + item);
}
catch (error) {
console.log(`oopsie: ${error}`)
results.error.push(item);
}
})
)
resolve(results);
});
stream.pipe(csvStream);
})
}
In the following code, the csv2pg function is async
const postUsages = async (req: MulterRequest, res: Response, next: any) => {
try {
const result = await csv2pg(req, next);
res.status(200).json({
msg: 'File uploaded/import successfully!',
file: req.file,
});
} catch (err) {
res.status(400).json({
msg: 'File uploaded/import failed!',
file: req.file,
});
}
};
The problem is, the await doesn't seem to await the function being finished before returning the res.status(200)
Here is the content of the csv2pg function that is being called (the async bit in this code is the forEach and the pool.query)
const csv2pg = (req: MulterRequest, next: any): any => {
let provider_id = req.body!.provider_id;
const filePath = appRoot + '/reports/' + req.file.filename;
const stream = fs.createReadStream(filePath);
const csvData: any[] = [];
const csvStream = csv
.parse()
.on('data', (data: any) => {
csvData.push(data);
})
.on('error', (err: any) => {
throw err.message;
})
.on('end', () => {
csvData.shift();
csvData.forEach((row) => {
pool.query(
`INSERT INTO usage (date_and_time, consumption, reading_quality, provider_id)
VALUES ((TO_TIMESTAMP($1, 'DD/MM/YYYY HH24:MI') AT TIME ZONE 'Australia/Melbourne')::TIMESTAMP WITH TIME ZONE, $2, $3, ${provider_id})`,
row,
(err: any) => {
if (err) {
next(new Error(err));
}
}
);
});
fs.unlinkSync(filePath);
});
stream.pipe(csvStream);
};
Another problem with this is that when there's an issue, there's another error message on top saying that the headers were already set (of course, the server already returned to the client by that time)
Ideally, when an error is raised, the whole function should stop, return the error message, and wait for new incoming requests
In order to be able to wait for csv2pg to finish all async operations, it must return a promise. Since the async actions are in a loop, we have to use Promise.all. Try this:
const csv2pg = (req: MulterRequest, next: any): any => {
return new Promise((resolve, reject) => {
let provider_id = req.body.provider_id;
const filePath = appRoot + "/reports/" + req.file.filename;
const stream = fs.createReadStream(filePath);
const csvData: any[] = [];
const csvStream = csv
.parse()
.on("data", (data: any) => {
csvData.push(data);
})
.on("error", (err: any) => {
reject(err.message);
})
.on("end", () => {
csvData.shift();
const promisesArray = csvData.map((row) => {
return new Promise((resolve, reject) =>
pool.query(
`INSERT INTO usage (date_and_time, consumption, reading_quality, provider_id)
VALUES ((TO_TIMESTAMP($1, 'DD/MM/YYYY HH24:MI') AT TIME ZONE 'Australia/Melbourne')::TIMESTAMP WITH TIME ZONE, $2, $3, ${provider_id})`,
row,
(err: any) => {
if (err) {
reject(new Error(err));
} else {
resolve();
}
}
)
);
});
Promise.all(promisesArray).then(() => {
fs.unlinkSync(filePath);
resolve();
})
});
stream.pipe(csvStream);
});
};
Please note, I also wrapped your pool.query in a promise, since this function doesn't return a promise by default.
The reason could be that you are using this code inside the for-loop. Using Promise.all would fix the problem. I would recommend this post to you: https://www.freecodecamp.org/news/promise-all-in-javascript-with-example-6c8c5aea3e32/.
It explains everything in detail.
I am new to unit testing. I am trying to write test cases for controller.js files for nodejs microservices files. I am unable to understand where I am going wrong. Always throws an error "TypeError: Cannot read property 'empId' of undefined" for 2 of these properties.
This is the controller code:
const crmgDetails = db.crmgResource_details;
const employeeProposal = db.employee_Proposal;
const Op = db.Sequelize.Op;
const raDetails = db.crmgRaSheet_entity;
let results = [];
Sequelize = require('sequelize')
exports.findOne = (req, res) => {
console.log(req.body.empId);
crmgDetails.findAll({where: {
resEmployeeNumber: req.body.empId
}
})
.then(data => {
res.send(data);
})
.catch(err => {
res.status(500).send({
message:
err.message || "Some error occurred while retrieving tutorials."
});
});
};
exports.findMatchingDemandsForRmg = (req,res) => {
let proposedDemands = [];
employeeProposal.findAll({
where: {
emp_id: req.body.empId,
demandSbu : req.body.sbu
}
}).then(proposedEmployee => {
console.log('proposedEmployee',proposedEmployee);
if(proposedEmployee.length === 0){
crmgDetails.findAll({
where: {
resEmployeeNumber: req.body.empId,
demandSbu: req.body.sbu
}
}).then(matchingDemands => {
console.log('matchingDemands ',matchingDemands)
proposedDemands = matchingDemands;
})
}
else{
console.log("crmg Employee")
console.log(proposedEmployee)
for(let employee of proposedEmployee){
crmgDetails.findOne({
where: {
demandUid: employee.demandUid,
resEmployeeNumber: req.body.empId,
demandSbu: req.body.sbu
}
}).then( crmgProposed=> {
proposedDemands.push(crmgProposed);
})
}
}
setTimeout(() => {
console.log(proposedDemands)
res.send(proposedDemands);
}, 3000);
}).catch((err)=>{
res.status(500).send({
message:
err.message || "Some error occurred while retrieving tutorials."
});
})
}
exports.getResourceAllocationDetails = (req,res) => {
employeeProposal.findAll({
include: {
model: raDetails
},
where: Sequelize.and(
{activeFlag : true},
Sequelize.or({status:"Accepted By RMG"},
{status:"Rejected"}
))
}).then(employees => {
res.send(employees)
})
}
This is the test file I tried to write without my head:
const CrmgRaSheetModel = require('../controllers/crmgResource_Details.controller')
describe('Check for succcessful fetech API call', () => {
it('property getResourceAllocationDetails should be called', async () => {
CrmgRaSheetModel.getResourceAllocationDetails((res) => {
expect(res).to.be.an('object')
return res.json()
})
});
it('property findMatchingDemandsForRmg should be called', async () => {
CrmgRaSheetModel.findMatchingDemandsForRmg((res) => {
expect(res).to.be.an('object')
return res.json()
})
});
it('property findOne should be called', async () => {
CrmgRaSheetModel.findOne((res) => {
expect(res).to.be.an('object')
return res.json()
})
})
})
from test file you are calling controller method with only res, so no chance to send your input as your body.
So pass req,res both and pass your input value in req
so I have problem with optimizing code, accordingly to DRY rule.
I have 2 requests, 2 responses, 2 GETS, 2 POSTS,
I think I should use loop, which changing the number of iterator, but I have problem with syntax.
Here is my fetch from axios:
componentDidMount() {
fetch('http://localhost:3001/getcounterEbook1')
.then(response => {
console.log(response);
return response.json();
})
.then(count => {
console.log(count);
console.log(count.count);
this.setState({counterebook1: count.count});
}).catch(err => {
});
fetch('http://localhost:3001/getcounterEbook2')
.then(response => {
console.log(response);
return response.json();
})
.then(count => {
console.log(count);
console.log(count.count);
this.setState({counterebook2: count.count});
}).catch(err => {
});
}
Here is handleClick1,2 function:
handleClick1(e) {
console.log("Sending ...")
let data = {
clicked: true,
}
axios.post('http://localhost:3001/counterEbook1', data)
.then( res => {
console.log('ok')
})
.catch( () => {
console.log('Message not sent')
})
}
handleClick2(e) {
console.log("Sending ...")
let data = {
clicked: true,
}
axios.post('http://localhost:3001/counterEbook2', data)
.then( res => {
console.log('ok')
})
.catch( () => {
console.log('Message not sent')
})
}
How can I transform it into loop?
One possibility is to use an Object, call it myObject, and use map like so:
const myObject = {
'counterebook1' : 'http://localhost:3001/getcounterEbook1',
'counterebook2' : 'http://localhost:3001/getcounterEbook2',
};
class Foo extends Component {
....
for (let [key, url] of Object.entries(myObject)) {
fetch(url)
.then(response => {
console.log(response);
return response.json();
})
.then(count => {
console.log(count);
console.log(count.count);
this.setState({key: count.count});
}).catch(err => {
});
}
...
You can consolidate handleClicks by first retrieving the source that triggered the event, and then using that same object
handleClick = (e) => {
console.log("Sending ...")
const url = myObject[e.target.id];
let data = {
clicked: true,
}
axios.post(url, data)
.then( res => {
console.log('ok')
})
.catch( () => {
console.log('Message not sent')
})
}
Note that we're pulling the id prop frome.targetand referencing the key/value pair inmyObject. This means that the element rendered must have theid` prop set to some key in myObject:
For example:
<button onClick={this.handleClick} id={key}>
`Button for ${key}`
</button>
Hope this help!
Here I've created a data Object
const data = new Object();
Then I'm calling multiple API's parallelly
datamuse.request(`words?ml=${text}`)
.then((list) => {
data.ml = list;
})
.catch((error) => {
console.log(error);
});
datamuse.request(`words?sp=${text}`)
.then((list) => {
data.sp = list;
})
.catch((error) => {
console.log(error);
});
datamuse.request(`words?rel_trg=${text}`)
.then((list) => {
data.rel = list;
})
.catch((error) => {
console.log(error);
});
datamuse.request(`sug?s=${text}`)
.then((list) => {
data.sug = list;
})
.catch((error) => {
console.log(error);
});
datamuse.request(`words?sl=${text}`)
.then((list) => {
data.sl = list;
})
.catch((error) => {
console.log(error);
});
And finally returning the data:
return data;
And the data is returned as undefined.
I know I'm performing the asynchronous operations simultaneously.
But I don't want to use function generator in this case because it's too slow.
Can anyone help me out to get those values in the data and then return it?
Something like
Promise.all([
datamuse.request(`words?ml=${text}`),
datamuse.request(`words?sp=${text}`),
datamuse.request(`words?rel_trg=${text}`),
datamuse.request(`sug?s=${text}`),
datamuse.request(`words?sl=${text}`),
]).then(([
ml,
sp,
rel,
s,
sl
]) => {
const data = {
ml,
sp,
rel,
s,
sl,
};
}).catch((err) => {
// Deal with error
});
Or even better you gotta add something for sug and words difference, I let you figure it out :)
const elems = [
'ml',
'sp',
'rel_trg',
's',
'sl',
];
Promise.all(elems.map(x => datamuse.request(`words?${x}=${text}`))
.then((rets) => {
const data = elems.reduce((tmp, x, xi) => ({
...tmp,
[x]: rets[xi];
}), {});
}).catch((err) => {
// Deal with error
});
Ok here is one possible soluce for your words and sug problem
const elems = [{
acronym: 'ml',
req: 'words',
}, {
acronym: 'sp',
req: 'words',
}, {
acronym: 'rel_trg',
req: 'words',
}, {
acronym: 's',
req: 'sug',
}, {
acronym: 'sl',
req: 'words',
}];
Promise.all(elems.map(({
acronym,
req,
}) => datamuse.request(`${req}?${acronym}=${text}`))
.then((rets) => {
const data = elems.reduce((tmp, {
acronym,
}, xi) => ({
...tmp,
[acronym]: rets[xi];
}), {});
}).catch((err) => {
// Deal with error
});
Promise.resolve is the way to go here.
In addition, I would suggest separating the logic from the data.
function getResult(where) {
const result = {}
return Promise.all(
Object
.entries(where)
.map(([key, path]) => datamuse.request(path).then(list => result[key] = list))
)
.then(() => result)
}
function createMapForText(text) {
return {
ml: `words?ml=${text}`,
sp: `words?sp=${text}`,
rel: `words?rel_trg=${text}`,
sug: `sug?s=${text}`,
sl: `words?sl=${text}`
}
}
// ----- Testing -----
// Fake `datamuse` for testing purposes only
const datamuse = {
request: (path) => Promise.resolve('hello')
}
getResult(
createMapForText('hello')
).then(console.log.bind(console))