In the docs for Application there's a note on using context to expose widely used properties in ctx:
For example, to add a reference to your database from ctx:
app.context.db = db();
app.use(async ctx => {
console.log(ctx.db);
});
However, it also notes that relying more on ctx "could be considered an anti-pattern". Fair enough.
Then we have koa-knex-realworld-example which suggests something like this:
module.exports = function (app) {
// ...
const db = require('knex')(config.db)
app.db = db
// ...
return async function (ctx, next) {
if (ctx.app.migration && promise) {
await promise
}
return next()
}
}
And passes the app instance to the middleware:
app.use(db(app))
That feels strange to me. Having to pass app to its own middleware seems a little backwards, but maybe it's a known Koa pattern?
Finally, we have koa-knex which exposes knex as this.knex in middleware further down the stack:
app.use(_.get('/:userid', function *(userid) {
this.body = {
profile: yield this.knex('users').where('id', userid);
};
});
That one's a bit dated (generators, which are fine but not the default API for Koa anymore) and chucks Knex in global:
global.__knex || (global.__knex = Knex.initialize({
// ...
}));
this.knex = global.__knex;
yield next;
What's the alternative?
In summary, I think I'm pretty comfortable putting a reference to Knex in the context, and I don't think I'll be tempted to pollute it with too much else. However, if I wanted an alternative, is there a middleware approach to making Knex available in Koa2 that is preferable to the above options?
Related
I want to force Node to wait for the promise to complete (either by failure or success). So far, I'm not seeing what I wanted. I'm trying to wait for two web services to complete before merging, and especially before the function ends.
I've tried these two approaches below. In both cases, Node refuses to wait.
Approach 1:
function getStrategy() {
// this is a web service that takes a few ms to run,
// but so far I haven't seen any evidence that it bothers.
}
function getConfig() {
// Both strategy and jwt are set to Promises
const strategy = getStrategy();
const jwt = getJwt();
const lodash = require('lodash');
var config = {};
try {
Promise.all([strategy,jwt])
.then(data => { config = lodash.merge(config,data)})
} catch(error) {
console.log(' Print error message');
}
return config;
}
Approach 2:
function getStrategy() {
// this is a web service that takes a few ms to run,
// but so far I haven't seen any evidence that it bothers.
}
async function getConfig() {
// Both strategy and jwt are set to Promises
const strategy = getStrategy();
const jwt = getJwt();
const lodash = require('lodash');
var config = {};
try {
var promiseResult = await Promise.all([strategy, jwt]);
const lodash = require('lodash');
config = lodash.merge(config, strategy[0]);
config = lodash.merge(config, jwt);
} catch (reason) {
console.error('------------------------------------------------------------------------------------');
console.error(reason);
console.error("in getConfig(): Could not fetch strategy or jwt");
console.error('------------------------------------------------------------------------------------');
}
return config;
}
I wish approach 2 worked, but it does not. It will not print any console.log statements after the call to Promise.all. So within that function it does wait. Except that it because I told it to "await", I have to make the function async. That allows Node to say, "oh, it's an async function, I can just go off and do something else and completely ignore the await keyword." It does this by returning to the function calling getConfig().
In the first approach, neither the "then" handler, nor any exceptions are thrown. It just impatiently leaves the function and goes back to the caller.
How do I get the thread that calls the getConfig() function to wait for the result. I mean really wait, not like, partially "await". Or, throw an exception and let me handle that. I'm finding that in Node, as soon as something because asynchronous, I have no idea how to get the await, or the then handler to work.
Updated attempt:
I separated the two service calls to individually control each service. I now have
async function getSynchronousStrategy(isVaultAvailable) {
const secretsVaultReader = require('./src/configuration/secretsVaultReader');
const secretsConfigReader = require('./src/configuration/secretsConfigReader');
const strategy = isVaultAvailable
? secretsVaultReader.getStrategy()
: secretsConfigReader.getStrategy();
await strategy;
console.log('## strategy after await=' + strategy);
return strategy; ''
}
...
const strategy = await getSynchronousStrategy(isVaultAvailable);
console.log('### strategy =' + JSON.stringify(strategy));
...
In the above case, Node sees the await strategy, but then prints
"## strategy after await=[object Promise]"
However, the second await seems to work, and it prints the desired strategy. I'm guessing the promise was eventually settled and it was able to print the result. I don't mind the time, I just want it to wait.
Obviously, it did not wait. It
I would suggest simply using await with both the function calls directly, in this manner it will wait for both the functions to return responses before proceeding further.
i tried to add a trigger function on my cloud functions.
When a document changed i want to perform some work. I have already the code for the work to do in another file (i separate my work into files, so it's get easier for me), so i have a function that performs work and when it is finished the callback is called.
This is what my index.js look like:
// requires...
// Express, i also use express for simulating an API
const app = express()
// So there is my Listener
const listenerForChange = functions.firestore
.document('myPath/documentName')
.onWrite((change, context) => {
const document = change.after.exists ? change.after.data() : null;
if (document != null) {
const oldDocument = change.before.data();
const newDocument = change.after.data()
// Here i call my function that is in the worker-listeners.js
listenerWorker.performStuff(newDocument, function() {
return 0
})
}
else {
console.error("Listener for classic was triggered but doc does not exist")
return 0
}
});
const api = functions.https.onRequest(app)
module.exports = {
api,
listenerForChange
}
...
There is my listenerWorker.js :
module.exports = {
performStuff: function(data, complete) {
performStuff(data, complete)
}
};
function performStuff(data, complete) {
// do stuff here ...
complete()
}
Problem is that i always an error in the Firebase console saying : Function returned undefined, expected Promise or value
Even if i do not do anything inside my worker and calling the callback as soon as i can i get the error.
So i understand the functions needs a response, promise or value. But it's like i'm not in the scope anymore but i do not want to return before the work is finished !
Any help ? thank you guys
All asynchronous work that you perform should be represented by a promise. If you have a utility object that does async work, it should return a promise so that the caller can decide what to do next with it. Right now you're just using callbacks, and that's going to make things much more difficult than necessary (because you'll have to "promisify" those callbacks).
Just use promises everywhere and it'll be much more clear how your function should work.
In my express app, I declare request handlers like this(simplified here):
export const CreateProduct = async (req, res, next) => {
try {
// ProductModel is a sequelize defined model
const product = await ProductModel.create(req.body)
res.send(product)
} catch (err) {
// I have raygun setup as the error handler for express
// in this example, it will finally return 500
next(err)
}
}
And then use it like so:
import { CreateProduct } from './create_product'
export const ProductRouter = express.Router()
ProductRouter.post('/', CreateProduct)
However, when running my test, nyc/istanbul will complain that line 9 is an Uncovered Line (in my example, it is the next(err) function), how do I go about simulating the error in my example?
The easier to go is to create a validation mechanism for your ProductModel, and when you create the product with invalid data throw some validation errors.
And in your mocha you will send an invalid product body and it will catch your next.
Two of the easiest possible ways of doing this are as follows:
1) Admit that controllers can not be unit tested, and because they can not be unit tested, they will be a part of your integration tests, and because that is true, you are going to move as much unit testable code out of them as physically possible, and limit them to doing little more than composing pieces of code which are able to be tested. This would be fine; an Express controller is sort of like a main method into your application, from the view of an individual request, anyway, so having it mix together the hard concepts to decide what gets built is okay, as long as the main isn't doing the heavy lifting, but rather instantiating and running things which are doing the lifting.
2) Compose your functions so that they all receive all of the props that they require, to do their jobs, and moreso, they nearly always receive them from other files (not the file they're defined in), unless the assignment is just as a default value.
In the case of a controller, if you wanted the controller to be unit-testable, you would preconfigure it to accept a ProductModel to use (or a function to use) rather than assuming you are going to pull in the real model.
export const CreateProduct = (ProductModel) => async (req, res, next) => {
const productData = req.body
try {
const product = await ProductModel.create(productData)
res.send(product)
} catch (err) {
next(err)
}
}
import { CreateProduct } from './create_product'
import { ConfigureProductModel } from './somewhere_else'
export const ProductRouter = express.Router()
ProductRouter.post('/', CreateProduct(ConfigureProductModel(database)))
Now to test that, you can easily create a CreateProduct where you pass in a fake ProductModel. And if you use the example, there of ConfigureProductModel as a factory, you can test it by passing it a fake db instance (if you do, indeed have that level of control).
And personally, like I said, as a controller, I want to remove as much control as possible, so I'd probably take most of the imperative code away.
const CreateProduct = ProductModel => (req, res, next) =>
ProductModel.create(req.body)
.then(product => res.send(product))
.catch(next);
I'm running Express on Node.js and am wondering how I can effectively pass a single database connection context object between distinct Node modules (think of them sort of like application models).
I'd like to do this to be able to start a database transaction in one model and preserve it across calls to other affected models, for the duration of a single HTTP request.
I've seen people attempt to solve this using per-request database connections exposed as middleware before my route is run (taking from a connection pool, then running another piece of middleware after the routes to return the connection to the pool). That unfortunately means explicitly passing around a context object to all the affected functions, which is inelegant and clunky.
I've also seen people talking about the continuation-local-storage and AsyncWrap modules, but I'm unclear how they can solve my particular problem. I tried working with continuation-local-storage briefly but because I primarily use promises and generators in my code, it wasn't able to pass state back from the run method (it simply returns the context object passed into its callback).
Here's an example of what I'm trying to do:
// player-routes.js
router.post('/items/upgrade', wrap(function *(req, res) {
const result = yield playerItem.upgrade(req.body.itemId);
res.json();
});
// player-item.js
const playerItem = {
upgrade: Promise.coroutine(function *(user, itemId) {
return db.withTransaction(function *(conn) {
yield db.queryAsync('UPDATE player_items SET level = level + 1 WHERE id = ?', [itemId]);
yield player.update(user);
return true;
});
})
};
module.exports = playerItem;
// player.js
const player = {
update(user) {
return db.queryAsync('UPDATE players SET last_updated_at = NOW() WHERE id = ?', [user.id]);
})
};
module.exports = player;
// db.js
db.withTransaction = function(genFn) {
return Promise.using(getTransactionConnection(), conn => {
return conn.beginTransactionAsync().then(() => {
const cr = Promise.coroutine(genFn);
return Promise
.try(() => cr(conn))
.then(res => {
return conn.commitAsync().thenReturn(res);
}, err => {
return conn.rollbackAsync()
.then(() => logger.info('Transaction successfully rolled back'))
.catch(e => logger.error(e))
.throw(err);
});
});
});
};
A couple of notes here:
The wrap function is just a little piece of wrapper middleware that allows me to use generators/yield in my routes.
The db module is also just a small wrapper around the popular mysql module, that has been promisified.
What I'd like to do, probably in db.queryAsync, is check if there's a conn object set on the current context (which I'd set around the return Promise... call in db.withTransaction). If so, use that connection to do all subsequent database calls until the context goes out of scope.
Unfortunately, wrapping the return Promise... call in the CLS namespace code didn't allow me to actually return the promise -- it just returned the context object, which is incorrect in my case. It looks like most usages of CLS rely on not actually returning anything from inside the run callback. I also looked at cls-bluebird, but that didn't seem to do what I need it to do, either.
Any ideas? I feel like I'm close, but it's just not all hooking up exactly how I need it to.
I love generators in nodejs. They help nodejs look more like server-side code. I'm trying to use generators with my Sails app. This is my controller and it works when I visit 'get /hi':
/**
* FooController
*
* #description :: Server-side logic for managing foos
* #help :: See http://sailsjs.org/#!/documentation/concepts/Controllers
*/
module.exports = {
hi: function (req, res){
return res.send("Hi there!");
}
};
However when I change that hi action to a generator function...
module.exports = {
hi: function* (req, res){
return res.send("Hi there!");
}
};
this action never gets to return the response. Like it's waiting for something. How does one utilize ES6 generators within SailsJS controllers and in-fact all of Sails?
You can use it, in fact it'll be great if we all use this style, it adds a lot of readability and flexibility in your code (no more callback-hell), but that is not the way to use it.
As #Cohars stated, sails expect a function as controller actions, you can't pass a generator like in Koa, but that does not prevent you from using them, the thing is that a generator by itself is very useless, you need a function that calls it and iterates it and i believe koa does this for you at framework level, but you have some options available to you at library level, like co or yortus/asyncawait/ which is what i use because node-fibers implemented as functions are way more flexible than es6 generators (though they are almost the same) and i'll show you why in a sec.
So here is how you use it:
First npm install co and require it in your controller, or add it as a global in config/bootstrap.js since you will be using it a lot. Once you are done with it, you can use it in your controllers.
module.exports = {
hi: function(req, res){
co(function* (){
// And you can use it with your models calls like this
let user = yield User.findOne(req.param('id'))
return res.send("Hi there" + user.name + "!");
})
}
};
That's it
Now if you rather use async/await, its pretty similar, it goes like this:
module.exports = {
hi: function(req, res){
async(function(){
// Since await is a function, you can access properties
// of the returned values like this which is nice to
// access object properties or array indices
let userName = await(User.findOne(req.param('id'))).name
return res.send("Hi there" + userName + "!");
})();
}
};
There is a caveat though, if you call other methods of you controller via this, remember that they will refer to the generator fn or the passed regular fn if you use async/await, so be sure to save its context, or since you are already using es6 syntax, you can use fat arrows with async/await, unfortunately not with co, since there is not fat arrow version of generators (..yet?)
So it'll be like this:
module.exports = {
hi: function(req, res){
async(() => {
let params = this._parseParams(req);
let userName = await(User.findOne(params.id)).name
return res.send("Hi there" + userName + "!");
})();
},
_parseParams: function(req){
let params = req.allParams()
// Do some parsing here...
return params
}
};
I've been using the second method for months now, and works perfectly, i've tried with co too and works as well, i just liked more the async/await module (and is supposed to be a little bit faster) and its a perfect match if you are using coffeescript since your sintax will be like do async => await User.find()
UPDATE:
I've created a wrapper that you can use if you use yortus async/await or check the source code and modify it to work with co or something else if you wish.
Here is the link https://www.npmjs.com/package/async-handler, is in alpha but i'm using on my own apps and works as expected, if not submit an issue.
With that the las example would look like this:
module.exports = {
hi: asyncHandler((req, res)->{
let params = this._parseParams(req);
let userName = await(User.findOne(params.id)).name
return res.send("Hi there" + userName + "!");
}),
_parseParams: function(req){
let params = req.allParams()
// Do some parsing here...
return params
}
};
With this handler you get the extra benefit of having async/promise errors to propagate correctly on sails if they are not caught by a try/catch
Sails expects a regular function here, not a generator. Maybe you could take a look at co, not sure it would really help with Sails though. If you really want to use generators, you should probably try Koa, which has several frameworks based on it
The way I am doing it is like this:
const Promise = require('bluebird');
module.exports = {
hi: Promise.coroutine(function* (req, res) {
let id = req.params('id'),
userName;
try {
userName = yield User.findOne(id).name;
}
catch (e) {
sails.log.error(e);
return res.serverError(e.message);
}
return res.ok(`Hi there ${userName}!`);
})
}
Works great. You just need to ensure any functions you call from your controllers return promises.
Put this code in your bootstrap.js, and every thing work like a charm!
var _ = require('lodash');
var coExpress = require('co-express');
sails.modules.loadControllers(function (err, modules) {
if (err) {
return callback(err);
}
sails.controllers = _.merge(sails.controllers, modules);
// hacking every action of all controllers
_.each(sails.controllers, function(controller, controllerId) {
_.each(controller, function(action, actionId) {
actionId = actionId.toLowerCase();
console.log('hacking route:', controllerId, actionId);
// co.wrap，generator => callback
action = coExpress(action);
sails.hooks.controllers.middleware[controllerId][actionId] = action;
});
});
// reload routes
sails.router.load(function () {
// reload blueprints
sails.hooks.blueprints.initialize(function () {
sails.hooks.blueprints.extendControllerMiddleware();
sails.hooks.blueprints.bindShadowRoutes();
callback();
});
});
});