use koa2 ejs koa-router, ejs template how to use another middleware's ctx.state
localhost:3000/admin/usermsg
admin.get('/usermsg', async(ctx) => {
ctx.state.userMsg = {
page: Number(ctx.query.page),
limit: 4,
pages: 0,
count: count
}
var userMsg = ctx.state.userMsg;
ctx.state.users = await new Promise(function(resolve, reject){
userMsg.pages = Math.ceil(userMsg.count / userMsg.limit);
userMsg.page = userMsg.page > userMsg.pages ? userMsg.pages : userMsg.page;
userMsg.page = userMsg.page < 1 ? 1 : userMsg.page;
var skip = (userMsg.page - 1) * userMsg.limit;
User.find().limit(userMsg.limit).skip(skip).exec(function(err, doc){
if(doc){
resolve(doc);
}
if(err){
reject(err);
}
})
})
await ctx.render('admin/usermsg');
})
localhost:3000/damin/category
admin.get('/category', async(ctx) => {
await ctx.render('admin/category');
})
in the category templateļ¼can not get ctx.state.userMsg.
how should i get ctx.state.userMsg in category template?
Well, assuming userMsg is something you use a lot in your views, you could make a dedicated middleware just to obtain that value.
Middleware work in 'stacks': by calling next(), you can pass control to the next one in the stack (with access to the modified ctx.state). A trivial example:
const setUserMsg = async (ctx, next) => {
ctx.state.userMsg = await myFuncThatReturnsAPromise()
await next()
}
router.get('/someroute',
setUserMsg,
ctx => { ctx.body = ctx.state.userMsg })
router.get('/someotherroute',
setUserMsg,
ctx => { ctx.body = ctx.state.userMsg })
Here, setUserMsg's sole purpose is to extract a value (presumably from the database) and add it to the context.
Related
I'm fairly new to Node and Express and am struggling with creating a route that takes a user uploaded file and processes it into another file. The problem is that the second time the user sends a request, I am having to wait for the first process to complete to allow the user to upload a new file.
The basic structure of route that I have is below
THE PROBLEM is that the function convertFile below is a time taking process and it keeps the server busy from accepting new requests. How do I make it so that once the project is saved in mongo db at newProject.save() - the process to convertFile runs in the background while the server accepts new requests from the same route?
I'm sending a response back the user after newProject.save() and was hoping that would allow the user to send another request. And although this sends another request, the server doesn't accept it since its busy with the previous process to convertFile
router.post('/', upload.fields([{ name: 'zip' }]), async (req, res, next) => {
let data = {
title: req.body.title,
description: req.body.description,
}
if (req.files && req.files.zip) {
const newProject = new MongoProject(data);
newProject.save()
.then(async (project) => {
res.send(project);
console.log("Project created");
const uploadedFilePath = path.normalize(req.files.zip[0].path);
// below method - "convertFile" is a time taking method
const extractZipinfo = await convertFile(uploadedFilePath , data.masterFile).then((zipInfo) => {
console.log({ zipInfo })
data.zipInfo = {
sizes: zipInfo.sizes
}
})
})
.catch(err => {
console.log("Error creating project");
return res.send(err);
})
}
})
Below is the simplified version of code in convertFile function (code modified for brevity):
I know that this can be improvised a lot, but i'm struggling with getting it to function as expected first (allowing multiple routes)
async function convertFile(inputFilePath, outputInfo) {
const outputFilePath = "output.abc";
const jsonFilePath = "output.json";
const doc = new Document(); // this is a class to store all data of the output file that we will write at the end
const _FileAPI = new fileAPI();
const outputFinalData = await _FileAPI.Init() // this is an async method
.then(() => {
const dataClass = initiateClass(); // this is a class to store data in JSON format
const paragraphs = _FileAPI.GetallParagraphs(inputFilePath);
for (let i = 0, len = paragraphs.size(); i < len; i++) {
for (let j = 0, lenj = paragraphs.size(); j < lenj; j++) {
const para = paragraphs.get(j);
// read each para and Capitalize each word
dataClass.paragraphs.push(para);
}
}
fs.writeFileSync(jsonFilePath, JSON.stringify(dataClass, null, 2), 'utf-8');
console.log("then")
}).then(() => {
const io = new NodeIO(); // this class helps in writing the file in the desired output format
const outData = io.write(outputFilePath, doc).then(() => {
outputInfo.sizes.push(fs.statSync(outputFilePath).size);
return outputInfo;
});
return outData;
});
return outputFinalData;
}
I am implementing function monthlyRevenue.
Simply, it will return total monthly revenue,and it takes arguments of station array which will make revenues, month and year.
Problem
Inside of this function I have getStationPortion which will fetch the revenue portion of user's.
So I would like to make it return object like this.
stationsPortion = {station1 : 30, station2 : 20}
In the monthlyRevenue
const stationPortions = await getStationPortions(stations)
console.log("portion map", stationPortions //it will be shown very beginning with empty
getStationPortions
const getStationPortions = async (stations) => {
let stationPortions = {}
stations.map(async (value) => {
const doc = await fdb.collection('Stations').doc(value).get()
if (!doc.exists) {
console.log("NO DOC")
} else {
stationPortions[value] = doc.data().salesPortion
console.log(stationPortions) //it will be shown at the last.
}
})
return stationPortions
}
I thought that async function should wait for the result, but it does not.
I am kind of confusing if my understanding is wrong.
Thank you
(by the way, fdb is firebase admin(firestore)
Working code
const getStationPortions = async (stations) => {
let stationPortions = {}
await Promise.all(stations.map(async (value) => {
const doc = await fdb.collection('Stations').doc(value).get()
if (!doc.exists) {
console.log("NO DOC")
} else {
stationPortions[value] = doc.data().salesPortion
console.log(stationPortions)
}
}))
return stationPortions
}
module.exports = router;
I am building my first web application with node v12.18.1 and express v4.17.1. Since the start of development, I have the same error on all routes: when i quickly click a link multiple times the server crashes with this error: screenshot of the error. It can be fixed in the front-end by disabling events on user input after a click, but I prefer to know what is wrong with my code.
Route of the index page :
let express = require('express');
let router = express.Router();
let controller_index = require("../controller/controller_index.js")
router.get('/', controller_index.get_index);
router.get('/rubrique/:category', controller_index.get_category);
module.exports = router;
Controller of the index page :
const Query = require("../lib/dbs.js");
const ObjectId = require('mongodb').ObjectId;
const utils = require('../lib/utils');
exports.get_index = async (req, res, next) => {
try {
let user = await Query.findOne("users", "_id", ObjectId(req.user_id));
let notification;
if (user) {
notification = await Query.findSortToArray("notifications", "for_user", ObjectId(user._id));
notification.count = notification.filter((notif => !notif.hasOwnProperty("readedAt"))).length;
}
if (!req.query.q) {
let page = req.query.page > 1 ? (req.query.page * 10) - 10 : req.query.page <= 0 ? undefined : 0;
let current_page = req.query.page ? Number(req.query.page) : 1;
let [countDocuments, docs] = await Promise.all([Query.countAll("articles"), Query.findAll(page)]);
let nb_pages = Math.ceil(countDocuments / 10);
if (!docs.length && current_page !== 1) {
next();
}
else {
docs = await documents_processing(docs, user, req);
res.render("../views/index.pug", { docs: docs, user: user, notification: notification, nb_pages: nb_pages, current_page: current_page })
};
}
else if (req.query.q) {
let page = req.query.page > 1 ? (req.query.page * 10) - 10 : req.query.page <= 0 ? undefined : 0;
let current_page = req.query.page ? Number(req.query.page) : 1;
let [countDocuments, docs] = await Promise.all([Query.countAllBySearch("articles", req.query.q), Query.findAllBySearch(req.query.q, page)]);
let nb_pages = Math.ceil(countDocuments / 10);
if (!docs.length && current_page !== 1) {
next();
}
else {
docs = await documents_processing(docs, user, req);
res.render("../views/index.pug", { docs: docs, search: req.query.q, user: user, notification: notification, nb_pages: nb_pages, current_page: current_page })
};
};
}
catch (err) {
console.error(err);
return next(err);
};
};
Example of static function of the query object :
const connect = require("../index.js")
module.exports = class Query {
static async findOne(collection, field, item) {
const result = await connect.client.db("blog_db").collection(collection).findOne({ [`${field}`]: item })
return result;
};
static async findOneAndUpdateOrInsertOnUser(collection, field, itemToSearch, updateItem) {
const result = await connect.client.db("blog_db").collection(collection).findOneAndUpdate({ [`${field}`]: itemToSearch }, { $set: updateItem }, { upsert: true, returnOriginal: false });
return result;
};
static async findSortToArray(collection, field, item) {
const results = await connect.client.db("blog_db").collection(collection).find({ [`${field}`]: item }).sort({ date: -1 }).toArray()
return results;
};
};
I'm fairly new to programming so any advice is welcome, thank you in advance!
----- EDIT -----
Kind of solution :
I have found people who have talked about this error on node v12 and newer, with a downgrade to v10 the issue was resolved without any clear explanation yet.
The error in the screenshot you've shared shows that the error is "Callback called multiple times", but I don't see anywhere obvious in the code you've shared where this is happening. As you're saying this bug only happens when multiple requests are made rapidly one after the other, it suggests there might be a global variable which is being shared between requests, which is something that should be avoided.
To debug the error you're seeing I would recommend commenting out all of the code in the get_index function and gradually uncommenting it in small chunks until you see the error happen again. You will probably want to do the same with the code that is called by the get_index controller function e.g. documents_processing, as the issue might possibly lie there.
Express only support callback-style, and you're using async function to handle the logic.
Basically, with async function, you call a function without waiting for it to resolve the logics. Hence, it creates too many callbacks in the event loop when that route has a large amount of concurrent coming requests.
function asyncWrapper(fn) {
return (req, res, next) => {
return Promise.resolve(fn(req))
.then((result) => res.send(result))
.catch((err) => next(err))
}
};
router.get('/', asyncWrapper(controller_index.get_index));
I have an app using Node/Express/Mongo and I'm running into trouble when I want to edit a document. I can add documents no problem but when I built out the Edit form I get the error in the title. I can fetch the document as well and see the information I've put in. The problem is when I try to save whatever changes I've made.
Here's my edit function: The error occurs at the brief.brief_title = updated_brief_title in the FindByIdAndUpdate method.
exports.postEditBrief = (req, res, next) => {
const briefId = req.body.briefId;
const updated_brief_title = req.body.brief_title;
const updated_country = req.body.country;
const updated_psg = req.body.psg;
const updated_one_year_withholding = req.body.one_year_withholding;
const updated_withholding_only = req.body.withholding_only;
const updated_practice_advisory = req.body.practice_advisory;
const updated_courthouse = req.body.courthouse;
const updated_pages = req.body.pages;
const updated_additional_psg = req.body.additional_psg;
const updated_gangs = req.body.gangs;
const updated_gang_name = req.body.gang_name;
const updated_link = req.body.link;
Brief.findByIdAndUpdate(briefId)
.then(brief => {
brief.brief_title = updated_brief_title;
brief.country = updated_country;
brief.psg = updated_psg;
brief.one_year_withholding = updated_one_year_withholding;
brief.withholding_only = updated_withholding_only;
brief.practice_advisory = updated_practice_advisory;
brief.courthouse = updated_courthouse;
brief.pages = updated_pages;
brief.additional_psg = updated_additional_psg;
brief.gangs = updated_gangs;
brief.gang_name = updated_gang_name;
brief.link = updated_link;
return brief.save();
})
.then(result => {
console.log(Brief);
res.redirect('/');
})
.catch(err => console.log(err));
};
I've played around with the various "find and update" methods in Mongoose but the results are the same.
You need to create an object of the data you want to update and pass it as a second argument in the function, when you do .then() it gets you the result of the operation you are doing and you don't need to call .save() as it does that internally
Correct version should look something like this
exports.postEditBrief = (req, res, next) => {
const briefId = req.body.briefId;
let update = {
brief_title: req.body.brief_title,
country: req.body.country,
psg: req.body.psg,
one_year_withholding: req.body.one_year_withholding,
withholding_only: req.body.withholding_only,
practice_advisory: req.body.practice_advisory,
courthouse: req.body.courthouse,
pages: req.body.pages,
additional_psg: req.body.additional_psg,
gangs: req.body.gangs,
gang_name: req.body.gang_name,
link: req.body.link
}
Brief.findByIdAndUpdate(briefId, update)
.then(result => {
console.log(result);
res.redirect('/');
})
.catch(err => console.log(err));
};
You can simplify it even further using object destructuring.
Update:
You need to use $set with findOneandUpdate if you don't want to overwrite your document
exports.postEditBrief = (req, res, next) => {
const briefId = req.body.briefId;
let update = {
brief_title: req.body.brief_title,
country: req.body.country,
psg: req.body.psg,
one_year_withholding: req.body.one_year_withholding,
withholding_only: req.body.withholding_only,
practice_advisory: req.body.practice_advisory,
courthouse: req.body.courthouse,
pages: req.body.pages,
additional_psg: req.body.additional_psg,
gangs: req.body.gangs,
gang_name: req.body.gang_name,
link: req.body.link
}
Brief.findOneAndUpdate(briefId, {$set:update})
.then(result => {
console.log(result);
res.redirect('/');
})
.catch(err => console.log(err));
};
I am having a hard time interpretting the documentation from Neo4j about transactions. Their documentation seems to indicate preference to doing it this way rather than explicitly declaring tx.commit() and tx.rollback().
Does this look best practice with respect to multi-statement transactions and neo4j-driver?
const register = async (container, user) => {
const session = driver.session()
const timestamp = Date.now()
const saltRounds = 10
const pwd = await utils.bcrypt.hash(user.password, saltRounds)
try {
//Start registration transaction
const registerUser = session.writeTransaction(async (transaction) => {
const initialCommit = await transaction
.run(`
CREATE (p:Person {
email: '${user.email}',
tel: '${user.tel}',
pwd: '${pwd}',
created: '${timestamp}'
})
RETURN p AS Person
`)
const initialResult = initialCommit.records
.map((x) => {
return {
id: x.get('Person').identity.low,
created: x.get('Person').properties.created
}
})
.shift()
//Generate serial
const data = `${initialResult.id}${initialResult.created}`
const serial = crypto.sha256(data)
const finalCommit = await transaction
.run(`
MATCH (p:Person)
WHERE p.email = '${user.email}'
SET p.serialNumber = '${serial}'
RETURN p AS Person
`)
const finalResult = finalCommit.records
.map((x) => {
return {
serialNumber: x.get('Person').properties.serialNumber,
email: x.get('Person').properties.email,
tel: x.get('Person').properties.tel
}
})
.shift()
//Merge both results for complete person data
return Object.assign({}, initialResult, finalResult)
})
//Commit or rollback transaction
return registerUser
.then((commit) => {
session.close()
return commit
})
.catch((rollback) => {
console.log(`Transaction problem: ${JSON.stringify(rollback, null, 2)}`)
throw [`reg1`]
})
} catch (error) {
session.close()
throw error
}
}
Here is the reduced version of the logic:
const register = (user) => {
const session = driver.session()
const performTransaction = session.writeTransaction(async (tx) => {
const statementOne = await tx.run(queryOne)
const resultOne = statementOne.records.map((x) => x.get('node')).slice()
// Do some work that uses data from statementOne
const statementTwo = await tx.run(queryTwo)
const resultTwo = statementTwo.records.map((x) => x.get('node')).slice()
// Do final processing
return finalResult
})
return performTransaction.then((commit) => {
session.close()
return commit
}).catch((rollback) => {
throw rollback
})
}
Neo4j experts, is the above code the correct use of neo4j-driver ?
I would rather do this because its more linear and synchronous:
const register = (user) => {
const session = driver.session()
const tx = session.beginTransaction()
const statementOne = await tx.run(queryOne)
const resultOne = statementOne.records.map((x) => x.get('node')).slice()
// Do some work that uses data from statementOne
const statementTwo = await tx.run(queryTwo)
const resultTwo = statementTwo.records.map((x) => x.get('node')).slice()
// Do final processing
const finalResult = { obj1, ...obj2 }
let success = true
if (success) {
tx.commit()
session.close()
return finalResult
} else {
tx.rollback()
session.close()
return false
}
}
I'm sorry for the long post, but I cannot find any references anywhere, so the community needs this data.
After much more work, this is the syntax we have settled on for multi-statement transactions:
Start session
Start transaction
Use try/catch block after (to enable proper scope in catch block)
Perform queries in the try block
Rollback in the catch block
.
const someQuery = async () => {
const session = Neo4J.session()
const tx = session.beginTransaction()
try {
const props = {
one: 'Bob',
two: 'Alice'
}
const tx1 = await tx
.run(`
MATCH (n:Node)-[r:REL]-(o:Other)
WHERE n.one = $props.one
AND n.two = $props.two
RETURN n AS One, o AS Two
`, { props })
.then((result) => {
return {
data: '...'
}
})
.catch((err) => {
throw 'Problem in first query. ' + e
})
// Do some work using tx1
const updatedProps = {
_id: 3,
four: 'excellent'
}
const tx2 = await tx
.run(`
MATCH (n:Node)
WHERE id(n) = toInteger($updatedProps._id)
SET n.four = $updatedProps.four
RETURN n AS One, o AS Two
`, { updatedProps })
.then((result) => {
return {
data: '...'
}
})
.catch((err) => {
throw 'Problem in second query. ' + e
})
// Do some work using tx2
if (problem) throw 'Rollback ASAP.'
await tx.commit
session.close()
return Object.assign({}, tx1, { tx2 })
} catch (e) {
tx.rollback()
session.close()
throw 'someQuery# ' + e
}
}
I will just note that if you are passing numbers into Neo4j, you should wrap them inside the Cypher Query with toInteger() so that they are parsed correctly.
I included examples of query parameters also and how to use them. I found it cleans up the code a little.
Besides that, you basically can chain as many queries inside the transaction as you want, but keep in mind 2 things:
Neo4j write-locks all involved nodes during a transaction, so if you have several processes all performing operations on the same node, you will see that only one process can complete a transaction at a time. We made our own business logic to handle write issues and opted to not even use transactions. It is working very well so far, writing 100,000 nodes and creating 100,000 relationships in about 30 seconds spread over 10 processes. It took 10 times longer to do in a transaction. We experience no deadlocking or race conditions using UNWIND.
You have to await the tx.commit() or it won't commit before it nukes the session.
My opinion is that this type of transaction works great if you are using Polyglot (multiple databases) and need to create a node, and then write a document to MongoDB and then set the Mongo ID on the node.
It's very easy to reason about, and extend as needed.