I have an existing Node.js/Express app which connects to 2 separate databases, it has a MySQL DB for all the relational and a MongoDB store for the non-relational vertical data.
It uses Sequelize and Mongoose and works absolutely swimmingly.
I've been looking at Next.js today and I'm pretty impressed, one of my pet peeves with React is actually how much bootstrapping there is and how much code it takes to achieve something simple. Next.js seems to solve some of those issues for me, so I'm willing to embrace it.
First issue - Is it possible to connect Next.js to existing DB's and read their objects directly in the view?
e.g. ./server.js:
const mongoDb = mongoose.connect(configDB.url); // MongoDB connection
const models = require('./models'); // Sequelize connection
app.prepare().then(() => {
server.use((req, res, next) => {
req.mongodb = mongoDb
req.mysqldb = models
// Logging req.mysqldb/req.mongodb at this point gives the correct result.
next()
});
server.get('*', (req, res) => {
return handle(req, res)
})
})
./pages/index.js:
Index.getInitialProps = async function(req) {
console.log(req.mongodb);
console.log(req.mysqldb)
// Example of what I want: req.mysqldb.users.findAll()....... to populate collection for this view
}
When the console statements are executed in the index.js page, they are logged as undefined.
Ideally I want to use the objects/ORM layer directly in the next.js templates, I do not want to have to call my own API internally, it seems like a huge waste of resources!
Any help, greatly appreciated.
Just for future reference. getInitialProps gets passed in an object with one of the keys being req. So you're meant to do something like the following instead
// add the curly braces around req
Index.getInitialProps = async function({ req }) {
// code
}
This is known as Function Parameter Destructuring and was introduced in ES6. What this accomplishes is similar to the following code
Index.getInitialProps = async function(_ref) {
var req = _ref.req;
}
Meaning, it takes the value of req of the object that gets passed and uses that value.
Well apparently by the time the request gets to the template it has changed a bit! Namely, it is nested within another request object.
req.req.mongodb and req.req.mysqldb both work fine :).
Related
so may be this is very basic question so please bear with me. Let me explain what I am doing and what I really need.
EXPLANATION
I have created a graphql server by using ApolloGraphql (apollo-server-express npm module).
Here is the code snippet to give you an idea.
api.js
import express from 'express'
import rootSchema from './root-schema'
.... // some extra code
app = express.router()
app.use(jwtaAuthenticator) // --> this code authenticates Authorization header
.... // some more middleware's added
const graphQLServer = new ApolloServer({
schema: rootSchema, // --> this is root schema object
context: context => context,
introspection: true,
})
graphQLServer.applyMiddleware({ app, path: '/graphql' })
server.js
import http from 'http'
import express from 'express'
import apiRouter from './api' // --> the above file
const app = express()
app.use([some middlewares])
app.use('/', apiRouter)
....
....
export async function init () {
try {
const httpServer = http.createServer(app)
httpServer
.listen(PORT)
.on('error', (err) => { setTimeout(() => process.exit(1), 5000) })
} catch (err) {
setTimeout(() => process.exit(1), 5000)
}
console.log('Server started --- ', PORT)
}
export default app
index.js
require('babel-core')
require('babel-polyfill')
require = require('esm')(module/* , options */)
const server = require('./server.js') // --> the above file
server.init()
PROBLEM STATEMENT
I am using node index.js to start the app. So, the app is expecting Authorization header (JWT token) to be present all the times, even for the introspection query. But this is not what I want, I want that introspection query will be resolvable even without the token. So that anyone can see the documentation.
Please shed some light and please guide what is the best approach to do so. Happy coding :)
.startsWith('query Introspection') is insecure because any query can be named Introspection.
The better approach is to check the whole query.
First import graphql and prepare introspection query string:
const { parse, print, getIntrospectionQuery } = require('graphql');
// format introspection query same way as apollo tooling do
const introspectionQuery = print(parse(getIntrospectionQuery()));
Then in Apollo Server configuration check query:
context: ({ req }) => {
// allow introspection query
if (req.body.query === introspectionQuery) {
return {};
}
// continue
}
There's a ton of different ways to handle authorization in GraphQL, as illustrated in the docs:
Adding middleware for express (or some other framework like hapi or koa)
Checking for authorization inside individual resolvers
Checking for authorization inside your data models
Utilizing custom directives
Adding express middleware is great for preventing unauthorized access to your entire schema. If you want to allow unauthenticated access to some fields but not others, it's generally recommended you move your authorization logic from the framework layer to the GraphQL or data model layer using one of the methods above.
So finally I found the solution and here is what I did.
Let me first tell you that there were 2 middle-wares added on base path. Like this:
app //--> this is express.Router()
.use(jwtMw) // ---> these are middlewares
.use(otherMw)
The jwtMw is the one that checks the authentication of the user, and since even introspection query comes under this MW, it used to authenticate that as well. So, after some research I found this solution:
jwtMw.js
function addJWTMeta (req, res, next) {
// we can check for null OR undefined and all, then check for query Introspection, with better condition like with ignore case
if (req.body.query.trim().startsWith('query Introspection')) {
req.isIntrospection = true
return next()
}
...
...
// ---> extra code to do authentication of the USER based on the Authorization header
}
export default addJWTMeta
otherMw.js
function otherMw (req, res, next) {
if (req.isIntrospection) return next()
...
...
// ---> extra code to do some other context creation
}
export default otherMw
So here in jwtMw.js we are checking that if the query is Introspection just add a variable in req object and move forward, and in next middleware after the jwtMw.js whosoever wants to check for introspection query just check for that variable (isIntrospection, in this case) and if it is present and is true, please move on. We can add this code and scale to every middleware that if req.isIntrospection is there just carry on or do the actual processing otherwise.
Happy coding :)
I'm creating a website using React and Express GraphQL, with a MariaDB database. I've come across a problem with making frequent requests/subscriptions to the GraphQL API, however.
After the page in question loads in React, I call this:
componentDidMount() {
this.setState({
waitingTimer: setInterval(this.queryWaiting.bind(this), 1000)
});
}
where queryWaiting is a function that carries out a fetch request to my GraphQL server at localhost:3000/graphql. That URL itself is a proxy defined in my setupProxy.js file which proxies the URL from localhost:4000/graphql, so that I don't need to use CORS.
It's worth noting that I also clearInterval this waitingTimer in componentWillUnmount.
In my GraphQL server file, which I'm simply running with node, I set up my server like this:
var app = express();
app.use("/graphql", graphqlHTTP({
schema: schema,
rootValue: root,
graphiql: true,
}));
app.listen(4000);
Where schema is a valid schema, and root is my root resolver.
The API query that I'm repeatably calling is called getWaitingCount, and it acts like this:
getWaitingCount: () => {
return new Promise((resolve, reject) => {
var currentTime = Date.now() / 1000;
if (cachedWaitingTime + cachedWaitingInterval > currentTime) {
return cachedWaiting;
}
connection.query("SELECT COUNT(*) FROM profiles WHERE isWaiting=1;", function (error, results, fields) {
if (error)
reject(error);
else {
// Update cache values
cachedWaiting = results[0]["COUNT(*)"];
cachedWaitingTime = Date.now() / 1000;
resolve(cachedWaiting);
}
});
});
}
I've implemented some caching to reduce server load. The caching variables are defined elsewhere.
The problem I experience is that, after between 2-10 seconds, the POST requests for this specific query from my React app stop getting resolved, and I have to reload the page for them to start getting accepted again. Weirdly, other requests go through fine.
I've tried various solutions, such as switching to using the proxy method I'm using currently, and I've also considered the possibility of a built-in anti-DDOS mechanism in the Express code, but I can't find any evidence for that.
Any help with this issue is very much appreciated.
Turns out the best solution is to type out and explain your code :)
This code:
if (cachedWaitingTime + cachedWaitingInterval > currentTime) {
return cachedWaiting;
}
is inside a promise, so really, it should do:
resolve(cachedWaiting);
instead of returning it.
That is all. I am currently kicking myself.
In my express application I have a module called helpers thats is required in almost all my routes and modules. This module has a logger method that logs to fluentd (but that's unimportant). While building the data to log I'd like to add a unique identifier of the request, so that all the logs written for the same request have the same unique ID. Using a global var in the app entry point app.use doesn't work because this var would be overwritten every time a new request hits, so the global uuid will change would obviously change in case of high load or long running tasks. The res.locals is not available outside routing, so I can't use it for this matter. Is there a way to create a var that would be unique per request and available in every module or maybe a way to access the res.locals data outside routing? Thank you
EDIT
Maybe an example will help understand better the question.
Suppose I have a module called helpers.js like this:
let helpers = {};
helpers.log = (logData, logName) => {
fluentLogger.emit('', {
name: logName,
//uuid: the needed uuid,
message: logData
});
}
module.exports = helpers;
Now obviously I can do this in my app.js entry point:
app.use(function (req, res, next) {
res.locals.uuid = uuid.v4();
next();
});
and then in every loaded middleware module that requires helpers(adding a new param to the helpers.log method):
const helpers = require('helpers');
router.post('/', (req, res, next) => {
helpers.log('my log message', 'myLogName', res.locals.uuid);
next();
});
and this will normally work. But suppose a big or middle size project where there are hundreds of custom modules and models (not middlewares) and a module may require other modules that require other modules that require finally the helpers module. In this case I should pass the res.locals.uuid as a parameter to every method of every method so that I have it available in the logger method. Not a very good idea. Suppose I have a new module called dbmodel.js that is required in a middleware function:
const helpers = require('helpers');
let dbmodel = {};
dbmodel.getSomeData = (someParam) => {
//some logic
helpers.log('my log message', 'myLogName');
}
module.exports = dbmodel;
The dbmodel has no idea about the res.locals data if I don't pass it from the middleware, so the helpers.log method will also have no idea about this.
In PHP one would normally write a GLOBAL var in the application's entry point so a hypothetical logger function would have access to this global on every method request from whichever class of the application.
Hope this explanation will help :) Thank you
EDIT 2
The solution for this kind of problems is CLS. Thanks to #robertklep for the hint. A good slideshare explaining exactly the same problem (logger with unique ID) and explaining the CLS solutions can be found here: https://www.slideshare.net/isharabash/cls-and-asynclistener
I answered a very similar question here which will solve this problem.
I used to solve the problem the libraries node-uuid and continuation-local-storage. Take a look to the answer of this question and see if it helps:
NodeJS Express - Global Unique Request Id
And you want a bigger explanation, take a look here:
Express.js: Logging info with global unique request ID – Node.js
Yes you can do so by one method .
Every request comes to his routes pass that request inside the middleware.
Suppose you have
app.get('/', function(req, res) {
res.sendFile(path.join(public + "index.html"));
});
a request.
Place Middleware in it .and edit req field coming , in this way you will get the unique variable values for each request
check out this .
https://expressjs.com/en/guide/writing-middleware.html
Like this
var requestTime = function (req, res, next) {
req.requestTime = Date.now()
next()
}
app.use(requestTime)
app.get('/', function (req, res) {
var responseText = 'Hello World!<br>'
responseText += '<small>Requested at: ' + req.requestTime + '</small>'
res.send(responseText)
})
Here req.requestTime is unique for each request.
I created a set of REST services based on Express.js to find some results stored in a Mongo Database. A very minimal version of the code for one of the services could be something like:
var express = require('express');
var app = express();
var bodyParser = require('body-parser');
var mongoClient = require('mongodb').MongoClient;
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.get('/results/:name', function(req, res){
var name = req.params.name;
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.find({ name: name }).toArray( function (err, docs) {
res.json({results: docs});
});
});
});
app.listen(3000);
I'm coming from Java and I've been beaten by SQL injections in the past. So I'm not at all comfortable with using the user's input directly in the find request. With my very thin knowledge of the subject, I tried several special characters ( } ) " ' ; and so on) but I've not been able to produce any weird result.
What could go wrong here? What kind of validations or checks should I implement to make sure that it is not possible to inject code or to make the program fail?
Now, let's do something stupid and let's trust the user to input a correct record:
app.post('/results/', function(req, res){
var record = req.body;
if( record.name ) {
mongoClient.connect('mongodb://localhost/test', function (err, db) {
var collection = db.collection('results');
collection.insert( record, function(err, doc){});
res.json({message: 'ok'});
});
}
});
How can I validate the schema of the input? And apart from filling the DB with thousand of gigantic inputs, is it possible to exploit this code to inject some code? If yes, how to prevent that?
Thanks a lot!
I would have put this as a comment but since I'm not yet allowed to do that i'll just put it as a responce.
I go into the details since I'm not the expert here but here is an article I've found to be really interresting about vulnerability when using mongo and node.js.
For validating the model, I use mongoose as a client to my mongoDB, it helps a lot as it has its own validators and you can as well make your own.
I hope it helps you into your search.
Mongodb access is api-based, as oposed to SQL that is language-based. SQL is a language, and if you let inputs from users to be inserted in the language, then it is easy to make code injection and do almost everything to your database.
MongoDB has a different approach, when you are doing a search, you call an API function to do the search, and this API function can only do searchs.
If you let the user choose the fields and the values, then he can make searchs that you don't expect, but that's all.
The same applies for inserts, updates and deletes, be careful of not letting the user to choose the fields and the values, because he can choose ones that you do not expects nor wants.
Here's the basic setup. I'm trying to create a simple middleware component that would allow me to easily pass data from my route directly to my javascript in the client side. (Very similiar to the Gon gem in ruby). The way I'm doing it is by having a module that looks like this:
module.exports = function(){
return function(req,res,next){
var app = req.app;
if(typeof(app) == 'undefined'){
var err = new Error("The JShare module requires express");
next(err);
return;
}
res.jshare = {};
app.dynamicHelpers({
includeJShare: function(req,res){
if(typeof(res.jshare) === 'undefined'){
return "";
}
return function(){
return '<script type="text/javascript">window.jshare=' + JSON.stringify(res.jshare) + '</script>';
}
}
});
next();
};
}
Then, in my route I can do this:
exports.index = function(req, res){
res.jshare.person = {firstName : "Alex"};
res.render('index', { title: 'Express' })
};
Finally in the layout.jade:
!{includeJShare()}
What that does is in outputs a line of javascript on the client that creates the exact JSON object that was created server side.
Here's the question; it all works as expected, but being new to Express and Node.js in general, I was just curious if attaching properties onto the response object is OK, or is there something wrong with doing it that I'm simply overlooking? For some reason it doesn't pass my "smell test" but I'm not sure why.....
I know this is an old thread, but there is something else to add to this topic.
Express has a response.locals object which is meant for this purpose - extending the response from middleware to make it available to views.
You could add a property directly to the response object, and as #hasanyasin indicated, is how JavaScript is designed. But Express, more specifically, has a particular way they prefer we do it.
This may be new in express 3.x, not sure. Perhaps it didn't exist when this question was asked.
For details, see
http://expressjs.com/en/api.html#res.locals
There is also an app.locals for objects which don't vary from request to request (or response to response I suppose).
http://expressjs.com/en/api.html#app.locals
See also: req.locals vs. res.locals vs. res.data vs. req.data vs. app.locals in Express middleware
It is perfectly OK. It is how JavaScript is designed. Only thing you should be careful is to not accidentally overriding already existing properties or being overridden by others. To be safer, instead of adding everything directly to req/res objects, you might consider going a level deeper:
res.mydata={}
res.mydata.person= ...
Like that.
Use res.locals for including custom variables in your response object.