I am using TSED - TypeScript Express Decorators (https://tsed.io) and it replaces express code like:
server.get('/api/tasks', passport.authenticate('oauth-bearer', { session: false }), listTasks);
With an annotated middleware class - https://tsed.io/docs/middlewares.html
So now the call to passport.authenticate() is in the use() method like:
#OverrideMiddleware(AuthenticatedMiddleware)
export class UserAuthMiddleware implements IMiddleware {
constructor(#Inject() private authService: AuthService) {
}
public use(
#EndpointInfo() endpoint: EndpointMetadata,
#Request() request: express.Request,
#Response() response: express.Response,
#Next() next: express.NextFunction
) {
const options = endpoint.get(AuthenticatedMiddleware) || {};
this.authService.authenticate(request, response, next); // <-- HERE
if (!request.isAuthenticated()) {
throw new Forbidden('Forbidden');
}
next();
}
}
And then my AuthService.authenticate() is
authenticate(request: express.Request, response: express.Response, next: express.NextFunction) {
console.log(`before passport authenticate time: ${Date.now()}`);
Passport.authenticate('oauth-bearer', {session: false})(request, response, next);
console.log(`after passport authenticate time : ${Date.now()}`);
}
My passport configuration is performed in this same AuthService class:
#Service()
export class AuthService implements BeforeRoutesInit, AfterRoutesInit {
users = [];
owner = '';
constructor(private serverSettings: ServerSettingsService,
#Inject(ExpressApplication) private expressApplication: ExpressApplication) {
}
$beforeRoutesInit() {
this.expressApplication.use(Passport.initialize());
}
$afterRoutesInit() {
this.setup();
}
setup() {
Passport.use('oauth-bearer', new BearerStrategy(jwtOptions, (token: ITokenPayload, done: VerifyCallback) => {
// TODO - reconsider the use of an array for Users
const findById = (id, fn) => {
for (let i = 0, len = this.users.length; i < len; i++) {
const user = this.users[i];
if (user.oid === id) {
logger.info('Found user: ', user);
return fn(null, user);
}
}
return fn(null, null);
};
console.log(token, 'was the token retrieved');
findById(token.oid, (err, user) => {
if (err) {
return done(err);
}
if (!user) {
// 'Auto-registration'
logger.info('User was added automatically as they were new. Their oid is: ', token.oid);
this.users.push(token);
this.owner = token.oid;
const val = done(null, token);
console.log(`after strategy done authenticate time: ${Date.now()}`)
return val;
}
this.owner = token.oid;
const val = done(null, user, token);
console.log(`after strategy done authenticate time: ${Date.now()}`);
return val;
});
}));
}
This all works - My Azure configuration and setup for this logs in and retrieves an access_token for my API, and this token successfully authenticates and a user object is placed on the request.
HOWEVER Passport.authenticate() seems to be asynchronous and doesn't complete until after the test for request.isAuthenticated(). I have put in timing comments as can be seen. The after passport authenticate time: xxx happens 2 milliseconds after the before one.
And the after strategy done authenticate time: xxx one happens a second after the after passport authenticate time: xxx one.
So it looks like Async behaviour to me.
Looking in node_modules/passport/lib/middleware/authenticate.js (https://github.com/jaredhanson/passport/blob/master/lib/middleware/authenticate.js) there are no promises or async mentioned. However in node_modules/passport-azure-ad/lib/bearerstrategy.js (https://github.com/AzureAD/passport-azure-ad/blob/dev/lib/bearerstrategy.js) is an async.waterfall:
/*
* We let the metadata loading happen in `authenticate` function, and use waterfall
* to make sure the authentication code runs after the metadata loading is finished.
*/
Strategy.prototype.authenticate = function authenticateStrategy(req, options) {
const self = this;
var params = {};
var optionsToValidate = {};
var tenantIdOrName = options && options.tenantIdOrName;
/* Some introduction to async.waterfall (from the following link):
* http://stackoverflow.com/questions/28908180/what-is-a-simple-implementation-of-async-waterfall
*
* Runs the tasks array of functions in series, each passing their results
* to the next in the array. However, if any of the tasks pass an error to
* their own callback, the next function is not executed, and the main callback
* is immediately called with the error.
*
* Example:
*
* async.waterfall([
* function(callback) {
* callback(null, 'one', 'two');
* },
* function(arg1, arg2, callback) {
* // arg1 now equals 'one' and arg2 now equals 'two'
* callback(null, 'three');
* },
* function(arg1, callback) {
* // arg1 now equals 'three'
* callback(null, 'done');
* }
* ], function (err, result) {
* // result now equals 'done'
* });
*/
async.waterfall([
// compute metadataUrl
(next) => {
params.metadataURL = aadutils.concatUrl(self._options.identityMetadata,
[
`${aadutils.getLibraryProductParameterName()}=${aadutils.getLibraryProduct()}`,
`${aadutils.getLibraryVersionParameterName()}=${aadutils.getLibraryVersion()}`
]
);
// if we are not using the common endpoint, but we have tenantIdOrName, just ignore it
if (!self._options._isCommonEndpoint && tenantIdOrName) {
...
...
return self.jwtVerify(req, token, params.metadata, optionsToValidate, verified);
}],
(waterfallError) => { // This function gets called after the three tasks have called their 'task callbacks'
if (waterfallError) {
return self.failWithLog(waterfallError);
}
return true;
}
);
};
Could that cause async code? Would it be a problem if run in 'normal express Middleware'? Can someone confirm what I've said or to deny what I've said and to provide a solution that works.
For the record I started asking for help on this Passport-Azure-Ad problem at my SO question - Azure AD open BearerStrategy "TypeError: self.success is not a function". The problems there seem to have been solved.
Edit - the title originally included 'in TSED framework' but I believe this problem described exists solely within passport-azure-ad.
This is a solution to work around what I believe is a problem with passport-azure-ad being async but with no way to control this. It is not the answer I'd like - to confirm what I've said or to deny what I've said and to provide a solution that works.
The following is a solution for the https://tsed.io framework. In https://github.com/TypedProject/ts-express-decorators/issues/559 they suggest not using #OverrideMiddleware(AuthenticatedMiddleware) but to use a #UseAuth middleware. It works so for illustration purposes that is not important here (I will work through the feedback shortly).
#OverrideMiddleware(AuthenticatedMiddleware)
export class UserAuthMiddleware implements IMiddleware {
constructor(#Inject() private authService: AuthService) {
}
// NO THIS VERSION DOES NOT WORK. I even removed checkForAuthentication() and
// inlined the setInterval() but it made no difference
// Before the 200 is sent WITH content, a 204 NO CONTENT is
// HAD TO CHANGE to the setTimeout() version
// async checkForAuthentication(request: express.Request): Promise<void> {
// return new Promise<void>(resolve => {
// let iterations = 30;
// const id = setInterval(() => {
// if (request.isAuthenticated() || iterations-- <= 0) {
// clearInterval(id);
// resolve();
// }
// }, 50);
// });
// }
// #async
public use(
#EndpointInfo() endpoint: EndpointMetadata,
#Request() request: express.Request,
#Response() response: express.Response,
#Next() next: express.NextFunction
) {
const options = endpoint.get(AuthenticatedMiddleware) || {};
this.authService.authenticate(request, response, next);
// AS DISCUSSED above this doesn't work
// await this.checkForAuthentication(request);
// TODO - check roles in options against AD scopes
// if (!request.isAuthenticated()) {
// throw new Forbidden('Forbidden');
// }
// next();
// HAD TO USE setTimeout()
setTimeout(() => {
if (!request.isAuthenticated()) {
console.log(`throw forbidden`);
throw new Forbidden('Forbidden');
}
next();
}, 1500);
}
Edit - I had a version that used setInterval() but I found it didn't work. I even tried inlining the code in to the one method so I could remove the async. It seemed to cause the #Post the UserAuthMiddleware is attached to, to complete immediately and return a 204 "No Content". The sequence would complete after this and a 200 with the desired content would be returned but it was too late. I don't understand why.
Related
I need to add a node.js express middleware for Emails duplicates controls, and a short and easy looking code.
This is my middleware.js file :
// ----------------------------------- MIDDLEWARE FUNCTIONS -------------------------------------------
module.exports = {
/*
* CHECKING FOR DUPLICATE EMAIL
* #params
* #return NEXT()
* #error Status 403s
*/
async duplicate_email(db, req, res, next) {
let availableEmail = await db.collection("users").findOne({ 'email': req.body.email })
if (!availableEmail) {
console.log(" FORBIDDEN ")
res.status(403).send({ errorCode: "403" })
return
} else {
next() // continue the process
}
}
}
And this is my Register web service, in another file called usersCrud.js , I'm struggling since 3 days to make it work but no way :
/*
* Register anonymous user
* #params JSON OBJECT : {}
* #return 200
* #error 400
*/
app.post("/registerUser", middleware.duplicate_email(), function(req, res) {
try {
var user = new User({
_id: null,
prenom: req.body.prenom,
nom: req.body.nom,
email: req.body.email,
password: bcrypt.hashSync(req.body.password, 10),
role: "user",
permissions: middleware.create_permissions("user"),
filenames: [],
groups: [],
last_update: new Date(),
img: "",
birthday: "",
age: "",
job: "",
mentra: "",
});
db.collection("users").insertOne(user);
console.log("Added one user");
res.sendStatus(200);
} catch (e) {
console.log(e);
res.sendStatus(400);
}
});
This is the error :
Error: Route.post() requires a callback function but got a [object Promise]
I have tried tons of things, like adding the db parameter, use a return function, removing all parameters, but I can' t make it work. The code would be really simplier if that thing could work like this, but I don't know if it is possible.
Maybe it is easy to solve for you .
The error message tells you what's wrong, since you invoke middleware.duplicate_email() in your app.post and thus returns a promise. You need to pass the function without invoking it or define it as a factory function that returns the middleware and takes the db object as a parameter. I'd go with the second approach, which would look something like:
module.exports = {
duplicate_email(db) {
return async (req, res, next) => {
let availableEmail = await db.collection("users").findOne({'email': req.body.email})
if (!availableEmail) {
console.log(" FORBIDDEN ")
res.status(403).send({errorCode: "403"})
return
} else {
next() // continue the process
}
}
}
}
You can then use it like this:
app.post("/registerUser", middleware.duplicate_email(db), function(req, res) { ... }
Have you tried to do something like this?
app.post("/registerUser", await middleware.duplicate_email(), function(req, res)
It has to be included in the async function of course to be able to use the await keyword. As the error says, the callback is expected, but you provided a not resolved Promise.
Another thing that may work is:
app.post("/registerUser", async () => {
const result = await middleware.duplicate_email();
return result;
}, function(req, res) {
// stuff
}
I'm creating an API using Express JS (Express 4) framework. I'm pretty new to NodeJS so I would like to know the best way to perform multiple async calls inside the same function.
For this example, I've got a function called Login in my Login Controller. Inside this function, let's say I gotta make quite a few async calls for authenticating a user, saving login info and similar functionalities like that.
But for some async calls, data to be processed should be fetched from the previous async call (they are dependent functions) and some functions are not like that.
Right now, this is how I'm doing the calls.
exports.login = function (req, res) {
var user = {
email: 'fakeemail#id.com',
password: 'password'
};
//Async call 1
Login.authUser(user, function (err1, rows1) {
var user_id = rows1[0].id;
//Async call 2 (depends on async call 1 for user_id)
Login.fetchUserDetails(user_id, function (err2, rows2) {
//Async call 3
Login.updateLoginInfo(param3, function (err3, rows3) {
//Some functionality occurs here then async call 4 happens
//Async call 4
Login.someOtherFunctionality(param4, function (err4, rows4) {
//return response to user
res.json({
success: true
});
});
});
});
});
};
Now all these async calls are nested. Is there any other way I can do this?
P.S: I didnt add error handling in this example
you can use promise as well. It will make you syntax more pretty. Your code will look like
Login.authUser(user).
then(fetchUser).
then(updateLoginInfo).
then(someOtherFunctionality).
catch(function(error){
//log your error or whatever
});
You can use Promise as suggested by Shahzeb.
Promises are objects that get resolved in future (asynchronous). Once a Promise is completed it either resolves or rejects. All promises resolved are then ed and those get rejected are catch ed
pseudo code
let myPromise = function() {
return new Promise(function(resolve, reject) {
resolve('foo');
});
};
myPromise().then( (v) => {
console.log(v);
})
Use Promise Chaining
new Promise(function(resolve, reject) {
setTimeout(() => resolve(1), 1000); // (*)
}).then(function(result) { // (**)
alert(result); // 1
return result * 2;
}).then(function(result) { // (***)
alert(result); // 2
return result * 2;
}).then(function(result) {
alert(result); // 4
return result * 2;
});
More details refer Promise Chaining and chain promises in javascript
Using async/await (Requires Node.js v7.6)
This strategy also uses promises. In my opinion it improves readability since you are refactoring out each individual promise call into separate methods.
Codepen
// mock async calls
const authUser = user => Promise.resolve([{ id: 1 }]); // returns users id
const fetchUserDetails = user_id => Promise.resolve({ name: 'Fred', age: '10000' }); // returns users details
const updateLoginInfo = param3 => Promise.resolve({ status: 'success' }); // returns success?
const someOtherFunctionality = param3 => Promise.resolve({ field: 'value' }); // returns something
// all async functions return a promise
const login = async (/*req, res*/) => {
// User object
const user = {
email: 'fakeemail#id.com',
password: 'password'
};
// Async call 1
console.log(`Authorizing user...`);
const rows1 = await authUser(user);
const user_id = rows1[0].id;
console.log(`User ${user_id} authorized.`);
// Async call 2 (depends on async call 1 for user_id)
console.log(`Fetching user detail...`);
const rows2 = await fetchUserDetails(user_id);
console.log(`User Detail was fetched: ${JSON.stringify(rows2)}`);
// Async call 3
console.log(`Updating login info...`);
const param3 = `something`;
const rows3 = await updateLoginInfo(param3);
console.log(`Login info was successful: ${JSON.stringify(rows3)}`);
// Some functionality occurs here then async call 4 happens
console.log(`\nDoing stuff after async call 3, but before async call 4....\n`);
// Async call 4
console.log(`Async call 4...`);
const param4 = `something`;
const rows4 = await someOtherFunctionality(param4);
console.log(`END OF LOGIN FUNCTION`);
return 'returned value';
}
// run the async function
login()
.then(result => {
// respond
// res.json({ success: true });
console.log(`Promise value: ${result}`);
console.log(`Response: { success: true }`);
})
.catch(err => {
console.log(err);
})
Being new to Node, I am still having some troubles with callbacks.
In the mapBpiIfindex function I am trying to loop through all of the VLANs found by the vlans function. Once it has looped through all VLANs, creating the map, I want to output the map to the browser. But, the only output I am getting is {}. How can I send the mapping to the browser? I am not even sure if I am using my callbacks correctly.
var express = require('express');
var router = express.Router();
var snmp = require('snmp-native');
// Create a Session with explicit default host, port, and community.
let session = new snmp.Session({ host: 'AASW0120', port: 161, community: 'community' })
let Mibs = {
hostname: [1,3,6,1,2,1,1,5,0],
vlans: [1,3,6,1,4,1,9,9,46,1,3,1,1,2],
dot1dBasePortIfIndex: [1,3,6,1,2,1,17,1,4,1,2]
}
/* Get all VLANs on switch */
function vlans(snmpSession, cb) {
let vlans = []
session.getSubtree({ oid: Mibs.vlans }, function (error, varbinds) {
if (error) {
console.log('Fail :(');
} else {
varbinds.forEach(function (varbind) {
vlans.push(varbind.oid[varbind.oid.length -1])
})
}
cb(vlans)
})
}
/* Map BPIs to Ifindices */
function mapBpiIfindex(session, cb) {
let map = {}
vlans(session, function (vlans) {
vlans.forEach(function (vlan) {
session.getSubtree({oid: Mibs.dot1dBasePortIfIndex, community: 'community#' + vlan}, function (error, varbinds) {
if (error) {
console.log('Fail :(')
} else {
varbinds.forEach(function (varbind) {
map[varbind.oid[varbind.oid.length -1]] = {ifindex: varbind.value, vlan: vlan}
})
}
})
})
cb(map)
})
}
router.get('/vlans', function (req, res, next) {
vlans(session, function (vlans) {
res.send(vlans)
})
})
router.get('/bpi-ifindex', function (req, res, next) {
mapBpiIfindex(session, function (mapping) {
res.send(mapping)
})
})
The answer is no, youre not using it correctly ;)
A few things here:
You should be clear that only the code within the callback is executed after the operation has finished, so cb(map)does not wait until all youre looped callbacks have finished. Thats why nothing is returned (because when cb is called, the async functions have not finished yet and map values are undefined. Have a look at this How do I return the response from an asynchronous call?, its the same principle.
Have a look at async module. Specifically, the do* or whilst methods. It'll help you process loops with async function calls.
Apart from that, you should not use forEach if you mind about performance.
I want to handle 10,000,000 rows
This is my code:
stmt.each('select * from very_big_table',function(err,rows,next){
console.log('This can take 2 seconds'
})
The question is: It will eat all my RAM, or it will read from hard-drive row after row?
I've just recently created this helper class to handle streaming:
import { ReadableTyped } from '#naturalcycles/nodejs-lib'
import { Database, Statement } from 'sqlite'
import { Readable } from 'stream'
/**
* Based on: https://gist.github.com/rmela/a3bed669ad6194fb2d9670789541b0c7
*/
export class SqliteReadable<T = any> extends Readable implements ReadableTyped<T> {
constructor(private stmt: Statement) {
super( { objectMode: true } );
// might be unnecessary
// this.on( 'end', () => {
// console.log(`SQLiteStream end`)
// void this.stmt.finalize()
// })
}
static async create<T = any>(db: Database, sql: string): Promise<SqliteReadable<T>> {
const stmt = await db.prepare(sql)
return new SqliteReadable<T>(stmt)
}
/**
* Necessary to call it, otherwise this error might occur on `db.close()`:
* SQLITE_BUSY: unable to close due to unfinalized statements or unfinished backups
*/
async close(): Promise<void> {
await this.stmt.finalize()
}
// count = 0 // use for debugging
override async _read(): Promise<void> {
// console.log(`read ${++this.count}`) // debugging
try {
const r = await this.stmt.get<T>()
this.push(r || null)
} catch(err) {
console.log(err) // todo: check if it's necessary
this.emit('error', err)
}
}
}
Both. statement.each will not use any extra memory if the function is fast and synchronous, however with asynchronous functions it will start all asynchronous processing as fast as it can load the data from disk, and this will eat up all your RAM.
As is also said in the issue you posted here https://github.com/mapbox/node-sqlite3/issues/686 it is possible to get your desired behavior using Statement.get(). Statement.get() without any parameters will fetch the next row. So you can implement your own async version like this:
function asyncEach(db, sql, parameters, eachCb, doneCb) {
let stmt;
let cleanupAndDone = err => {
stmt.finalize(doneCb.bind(null, err));
};
stmt = db.prepare(sql, parameters, err => {
if (err) {
return cleanupAndDone(err);
}
let next = err => {
if (err) {
return cleanupAndDone(err);
}
return stmt.get(recursiveGet);
};
// Setup recursion
let recursiveGet = (err, row) => {
if (err) {
return cleanupAndDone(err);
}
if (!row) {
return cleanupAndDone(null);
}
// Call the each callback which must invoke the next callback
return eachCb(row, next);
}
// Start recursion
stmt.get(recursiveGet);
});
}
Note, slightly different syntax than the built in Statement.each, errors are only sent to the last callback and no support for optional parameters.
Also note that this is slower than the normal Statement.each function, which can be improved by issuing multiple get calls so the next row is waiting when next is invoked, but it is much harder to follow that code.
Example for using the snippet:
let rowCount = 0;
asyncEach(db, 'SELECT * from testtable', [], (row, next) => {
assert.isObject(row);
rowCount++;
return next();
}, err => {
if (err) {
return done(err);
}
assert.equal(rowCount, TEST_ROW_COUNT);
done();
});
for my isomorphic application, i'm usign react router (2.0.1) and i need to handle authentication, so i used the onEnter hook , according to the documentation api. I need to handle authentication but getting
TypeError: transition.redirect is not a function
routes.jsx file
**/**
*
* #param next
* #param transition
* #param callback
*/
function requireAuth (next, transition, callback) {
if (!window.user){
transition.abort();
transition.redirect ({
pathname: '/login',
state: { nextPathname: next.location.pathname }
}, transition);
}
else {
callback(null, transition);
}
}**
<Router history={history} onUpdate={onUpdate} onEnter={requireAuth}>
<Route path="/rt/" component={RealTime}/>
<Route path="/" component={Blank}/>
{devRoutes}
</Router>
server.js file
app.get('*', function(req, res, next) {
Router.run(routes(), location, function(e, i, t) {
var str = React.renderToString(
React.createElement(Router, i));
res.send (str)
}
}
I think your requireAuth should look like this
function requireAuth (next, transition, callback) {
if (!window.user){
transition({
pathname: '/login',
state: { nextPathname: next.location.pathname }
});
}
else {
callback();
}
}
you can see here that transition doesn't call methods and if 'if' statement fails you just pass empty callback to continue to the page
There is no transition.redirect function, i need to modify the transition redirectInfo object in the onEnter transition hook using transition.to and then verfiy in the main server handler function.
/**
*
* #param next
* #param transition
* #param callback
*/
function requireAuth (next, transition, callback) {
var loginPath = '/account/login/' ;
if (!window.user && !(next.location.pathname === loginPath)){
transition.abort();
transition.to(loginPath, null, {
//data: req.body,
//errors: res.body
})
}
callback()
}
The redirection is not handled on onEnter hook but in the main server get handler.
app.get('*', function(req, res, next) {
Router.run(routes(), location, function(e, i, transition) {
// **********************************************
// handling redirection info
if (transition.isCancelled) {
console.log ('transition cancelled');
return res.redirect(302, transition.redirectInfo.pathname);
}
// ********************************************************
var str = React.renderToString(
React.createElement(Router, i));
res.send (str)
}
}