What's the best practice for sharing DB query code between multiple Node.js Express controller methods? I’ve searched but the samples I’ve found don’t really get into this.
For example, I have this getUser method (using Knex for MySQL) that makes a call to get user info. I want to use it in other methods but I don't need all the surrounding stuff like the response object.
export let getUser = (req: Request, res: Response, next: NextFunction) => {
try {
knex.select().where('email', req.params.email)
.table('users')
.then( (dbResults) => {
const results: IUser = dbResults[0];
res
.status(200)
.set({ 'Content-Type': 'application/json', 'Connection': 'close' })
.send(results);
});
} catch (err) {
res.send({ error: "Error getting person" + req.params.email });
return next(err);
}
};
It seems wrong to repeat the query code somewhere else where I need to get the user. Should I turn my DB query code into async functions like this example and then call them from within the controller methods that use the query? Is there a simpler way?
/**
* #param {string} email
*/
async function getUserId(email: string) {
try {
return await knex.select('id')
.where('email', email)
.table('users');
} catch (err) {
return err;
}
}
You can for example create "service" modules, which contain helpers for certain type of queries. Or you could use ORM and implement special queries in each model that is called "fat model" design. Pretty much anything goes as long as you remember to not create new knex instance in every helper module, but you just pass knex (containing its connection pool) for the helper methods so that all queries will share the same connection pool.
ORM's like objection.js also provides a way to extend query builder API so you can inherit custom query builder with any special query helper that you need.
Related
i've just started using Next js with mongodb and i have a question about how should i organize the API route files.
I have a simple application that add, update and delete documents of a mongodb collection. For each operation i created a .ts file inside the api folder. Like this
And for example my new_task.ts file looks like this
export default async function AddTask (req:NextApiRequest, res:NextApiResponse) {
const task:Task = req.body
const client = await clientPromise;
const db = client.db("diary");
const myCollection: Collection = db.collection('tasks');
try {
await myCollection.insertOne(task)
res.send('Success')
} catch (error) {
res.status(400).json({error})
console.log(error)
}
}
Everything is working ok but i think it's kinda messy the file organization. Is there a way to put every operation inside just one file? Or to do so i would have to build a custom server with express?
Thanks
In one route function, you can check the request object req to see if the HTTP request method is GET POST PUT PATCH or DELETE. Depending on which method, you can call a different function.
Here is an example from the NextJS docs.
import type { NextApiRequest, NextApiResponse } from 'next'
export default function userHandler(req: NextApiRequest, res: NextApiResponse) {
const {
query: { id, name },
method,
} = req
switch (method) {
case 'GET':
// Get data from your database
res.status(200).json({ id, name: `User ${id}` })
break
case 'PUT':
// Update or create data in your database
res.status(200).json({ id, name: name || `User ${id}` })
break
default:
res.setHeader('Allow', ['GET', 'PUT'])
res.status(405).end(`Method ${method} Not Allowed`)
}
}
Another thing you can do to make your code more re-useable and easier to maintain is to write reusable function definitions in a lib folder and then import them into your api route files when you want to use them.
Have you tried creating a file in the lib folder and writing function definitions there for MongoDB and then importing those function definitions into your api route file?
Then call the appropriate function depending upon the request method.
In ./lib/mongodb, write a function definition and import any Mongo-related imports you need.
export async function updateUserInfo(parameters) {
// . . . your code needs to return something, probably an array or object from MongoDB
}
In your api route file, import that function definition.
import { updateUserInfo } from "../../lib/mongodb"
Inside your route function, call updateUserInfo and pass whatever arguments you need based on the parameters you put in the definition. Handle its return value using await.
import type { NextApiRequest, NextApiResponse } from 'next'
import { updateUserInfo } from "../../lib/mongodb"
export default function userHandler(req: NextApiRequest, res: NextApiResponse) {
const {
query: { id, name },
method,
} = req
switch (method) {
case 'GET':
// Get data from your database
res.status(200).json({ id, name: `User ${id}` })
break
case 'PUT':
// Update or create data in your database
const updateResult = await updateUserInfo( . . .)
// FIX THE OBJECT IN .JSON BELOW TO SUIT YOUR CODE
res.status(200).json({ id, name: name || `User ${id}` })
break
default:
res.setHeader('Allow', ['GET', 'PUT'])
res.status(405).end(`Method ${method} Not Allowed`)
}
}
You can reuse updateUserInfo anywhere you have arguments for the required parameters.
Also, consider when you are calling the API route. At build time or after. At build, you call from static functions and after you call from client-side.
So by using the lib file for function definitions, you can reuse them in server functions and static functions.
The structure of the files inside the api folder is your api architecture. So it's organization depends upon your application's needs. You can use static and dynamic routes, as you maybe already know.
Consider API best practices when designing your architecture.
I would like to build a node application using REST, need to read data from readymade api and store it in class for temp, and then save it to MySQL. Can anyone have any idea about this?
This is a very simple job.
Let's consider Typescript, but you can achieve the same result with JavaScript. I'll be using node-fetch as an example of the rest API library. Do note that the code might not be syntactically correct.
First: Create interfaces/classes that reflect the data you will receive from the REST API
interface Food {
id: number,
name: string,
...
}
Second:
Create a Repository
Create a class Repository which you will use to communicate with the rest API
class Repository {
async function getFoods(...args): List<Food> {
let foods = await fetch({url: "url"});
return foods;
}
async function addFood(food: Food): Response {
let response = await fetch({
url: "url-to-add-food",
method: "post",
data: JSON.stringify(food)
});
}
}
Third:
Use the repository to fetch the data and use conventional methods to save it to a MySQL database
let foods = await repository.getFoods();
foods.forEach(food => {
connection.query('INSERT INTO foods SET ?', food,
function (err, resp) {
if (err) throw err;
}
);
});
I'm trying to implement a service for my backend that allows the session data to be called from anywhere in the code, which means I want to create a service file that exports the values from the functions that get the session data. Otherwise I can only get the session data from inside functions that have both req: Request and res: Response parameters. So I'm basically trying to lock the values to a variable that can be used and called from anywhere in my project. My code now looks like this but if I use the exports anywhere else in the file, it just prints the actual code (function) snippet instead of the return value specified inside the code. I'm pretty new to typescript and node in general which means I might be doing some really silly errors here.
Thanks for all the help in advance!
/Victor
import { Request, Response, NextFunction } from "express";
function getUserSessionData(req: Request, res: Response) {
const userData = res.status(200).send(req.session.userData);
return userData;
}
function getUserSessionLang(req: Request, res: Response) {
const userLang = res.status(200).send(req.session.language);
return userLang;
}
function getUserSessionAll(req: Request, res: Response) {
const userAll = res.status(200).send(req.session);
return userAll;
}
module.exports = {
userData: getUserSessionData,
userLang: getUserSessionLang,
userAll: getUserSessionAll
};
How I would like it to work:
const sessionService = require("./services/sessionService");
function getStuff(req: Request, res: Response, next: NextFunction) {
let redisKey;
if (!req.session.language) {
redisKey = "getStuffEN";
}
else {
redisKey = "getStuff" + sessionService.userLang;
}
console.log(redisKey);
getRedis(redisKey, function success(reply: any) {
res.status(200).json(
JSON.parse(reply)
);
}, function error(err: Error) {
console.log("Something went wrong");
});
}
This is how it is right now (and working)
function getStuff(req: Request, res: Response, next: NextFunction) {
let redisKey;
if (!req.session.language) {
redisKey = "getStuffEN";
}
else {
redisKey = "getStuff" + req.session.language;
}
console.log(redisKey);
getRedis(redisKey, function success(reply: any) {
res.status(200).json(
JSON.parse(reply)
);
}, function error(err: Error) {
console.log("Something went wrong");
});
}
I want it to work like the first example, since there are some instances in my code where I want to access the data without having to pass the req, res parameters, if possible.
First, a short explanation on sessions:
When a user logs in (and you verify his credentials, etz) you start a new session for that user.
The middleware you're using will assign this session a unique ID and you can assign some data to it.
The ID is transferred to the users Browser in form of a cookie
The Data is stored on the Server (in Redis for your case)
The middleware will check if a session-cookie with a valid ID is included in a request and do the following:
Fetch Session-Data for the given ID from Redis
Populate the req.session-object with the Data from Redis
Call the next Route
With this out of the way, a word of advice: Don't store the session-data in your applications memory. Why? The data should only be relevant in the context of a request from a user. You'll need the session data to handle the request, but you don't need it otherwise.
Instead of storing it globally, build your handlers and functions to accept the specific Session-Data as parameters, like this:
// handler, after middleware:
const getUserBio = (req, res) => {
const userId = req.session.userId;
const bioData = fetchBio(userId);
res.render("userBio", bioData);
}
// somewhere else in your code
function fetchBio(userId) {
const fullBio = database.fetchBio(userId);
return {
full: fullBio,
excerpt: fullBio.substring(0, 24);
}
}
This has two important advantages:
You don't have to keep the session-Data in your memory synchronized with the one in Redis
These (almost) pure functions make your code easier to understand
If you write functions that work entirely on their input parameters and don't use any global state, things like "order in which to call functions" or "check if data is available" become irrelevant. The caller is responsible for getting the data, the callee is responsible for working with it.
Extending on that, express routes should never use any global in-memory data, if you ever want to scale your application horizontally (by using multiple instances). It can't be guaranteed that the same client will always connect to the same server-instance, so the globally stored data might be available for one request but not for the next. In this case, you'll have to find a way to share the global data between all your instances, which is what Redis already does in your case.
tl;dr: Only access the session-data in a request-handler, then pass it on as parameters to any functions that need to work on it. Don't keep global in-memory state in your server if you ever want to scale it horizontally.
You can write data to a db or to a file as it say madvic, but we can use also global variables. But global variables is a bad practise, as I know, so It's better to write someone your data.
I would like to prevent a registration with an email address which already exists. Is it possible to use express-validator's new syntax for this? For example:
router.post('/register', [
check('email').custom((value, {req}) => {
return new Promise((resolve, reject) => {
Users.findOne({email:req.body.email}, function(err, user){
if(err) {
reject(new Error('Server Error'))
}
if(Boolean(user)) {
reject(new Error('E-mail already in use'))
}
resolve(true)
});
});
})
]
....
How would i pass Users?
express-validator is only aware of the request object itself, what keeps its complexity pretty low for the end-user.
More importantly, it only truly knows about the request's input locations -- body, cookies, headers, query and params.
Your custom validator is completely correct. That being said, it might not be testable, as you seem to be depending on global context.
In order to make it testable, the 2 options that I see are:
1. Inject req.Users:
This one would involve using some middleware that sets your store objects onto req:
// Validator definition
const emailValidator = (value, { req }) => {
return req.Users.findOne({ email: value }).then(...);
}
// In production code
// Sets req.Users, req.ToDo, req.YourOtherBusinessNeed
app.use(myObjectsStore.middleware);
app.post('/users', check('email').custom(emailValidator));
// In tests
req = { Users: MockedUsersObject };
expect(emailValidator('foo#bar.com', { req })).rejects.toThrow('email exists');
2. Write a factory function that returns an instance of your validator:
This is my preferred solution, as it doesn't involve using the request object anymore.
// Validator definition
const createEmailValidator = Users => value => {
return Users.findOne({ email: value }).then(...);
};
// In production code
app.post('/users', [
check('email').custom(createEmailValidator(myObjectsStore.Users)),
]);
// Or in tests
expect(createEmailValidator(MockedUsersObject)('foo#bar.com')).rejects.toThrow('email exists');
Hope this helps!
Converting my comments into a final, conclusive answer here :
A validator is simply supposed to validate the fields of request entities against the given criteria of data type / length / pattern.
You would need to write the method yourself, to determine if the user pre-exists or not. An express-validator ( or rather any validator ) would not do the task of cherry picking if the item exists in your list of items ( or your data-source), neither should it interact with the data-source concerned.
I have JSON API built with koa which I am trying to cover with integration tests.
A simple test would look like this:
describe("GET: /users", function() {
it ("should respond", function (done) {
request(server)
.get('/api/users')
.expect(200, done);
});
});
Now the issue comes when the actions behind a controller - lets say saveUser at POST /users - use external resources. For instance I need to validate the users phone number.
My controller looks like this:
save: async function(ctx, next) {
const userFromRequest = await parse(ctx);
try {
// validate data
await ctx.repo.validate(userFromRequest);
// validate mobile code
await ctx.repo.validateSMSCode(
userFromRequest.mobile_number_verification_token,
userFromRequest.mobile_number.prefix + userFromRequest.mobile_number.number
);
const user = await ctx.repo.create(userFromRequest);
return ctx.data(201, { user });
} catch (e) {
return ctx.error(422, e.message, e.meta);
}
}
I was hoping to be able to mock the ctx.repo on the request object but I can't seem to able to get a hold on it from test, which means that my tests are actually hitting the phone number verification service.
Are there any ways I could go around hitting that verification service ?
Have you considered using a mockup library like https://github.com/mfncooper/mockery?
Typically, when writing tests requiring external services, I mock the service client library module. For example, using mocha:
mockery = require('mockery');
repo = require('your-repo-module');
before(function() {
mockery.enable();
repo.validateSMSCode = function() {...};
mockery.registerMock('your-repo-module', repo);
}
This way, every time you require your-repo-module, the mocked module will be loaded rather than the original one. Until you disable the mock, obviously...
app.context is the prototype from which ctx is created from. You may
add additional properties to ctx by editing app.context. This is
useful for adding properties or methods to ctx to be used across your
entire app, which may be more performant (no middleware) and/or easier
(fewer require()s) at the expense of relying more on ctx, which could
be considered an anti-pattern.
app.context.someProp = "Some Value";
app.use(async (ctx) => {
console.log(ctx.someProp);
});
For your sample your re-define app.context.repo.validateSMSCode like this, assuming that you have following setup lines in your test:
import app from '../app'
import supertest from 'supertest'
app.context.repo.validateSMSCode = async function(ctx, next) {
// Your logic here.
};
const request = supertest.agent(app.listen())
After re-defining app.context.repo.validateSMSCode method that your will define in your test, will work, instead of original method.
https://github.com/koajs/koa/blob/v2.x/docs/api/index.md#appcontext
https://github.com/koajs/koa/issues/652