Hi I'm new to nodeJs and currently developing a Rest API using node.I'm planning to develop it with a good folder structure, so I can scale it up easily. There I'm using several route files according to the business logic.
ex :- authRoutes,profileRoutes,orderRoutes ......
Currently in every single route file I had to include following codes
var express = require('express');
var router = express.Router();
var jwt = require('jsonwebtoken');
var passport = require('passport');
My problem is , Is it totally fine to use above code segments in all the route files( I'm concerning about the code optimisation/coding standards and execution speed ) or is there any better way to do this.
It's better if you can explain the functionality of require() function.
Thanks
In my experience, this is very standard to do. I suggest reading this question and answer to learn more about the way it affects the speed of your programs.
TL;DR of require()
When your lines of code are ran that include a variable that exists because of a require(), for instance
var https = require('https');
https.get(url, function(response) {...});
The compiler reads it and goes into the https module folder, and looks for the .get function.
However, if you are trying to require() a certain JavaScript file, such as analysis.js, you must navigate to that file from the file you are currently in. For instance, if the file you want is on the same level as the file you are in, you can access it like this:
var analysis = require('./analysis.js');
//Let analysis have a function called analyzeWeather
analysis.analyzeWeather(weather_data);
These lines of code are a little different from above. In this require() statement, we are saying grab the .js file with this name, analysis. Once you require it, you can access any public function inside of that analysis.js file.
Edits
Added require() example for .js file.
Related
I´ve setup a little NodeJS API with 'expressjs'. In my code I have some similar code blocks looking like this:
app.get('/api/my/path', (req, res) => {
doSomethingUseful();
});
All thoses requests do something different, so the function being called is always different too.
I´m asking myself if this is good practice for writing code in NodeJS or not.
If there is a better / cleaner way of writing all those paths I would appreciate if you could tell me how or at least giving me a place to look for it.
EDIT 1: To be clear: I ask if it´s a good idea to write a lot of 'app.get(...)' into one source code file or if there is a better way?
Yes there is a better way than writing all routes in one file. For example, let us say you have routes for users and questions.
For users, you want get/set/put/delete for profile, and similarly for questions.So you create the following folder structure: /api/users and /api/questions
In /api/users,
const express=require('express')
const router=express.Router()
//this handles route: /users
router.get('/',(req,res)=>{})
//this handles route: /users/profile
router.get('/profile',(req,res){})
//this is to retrieve profile of specific user by user id :/users/profile/:userID
router.get('/profile/:userId',(req,res))
router.post('/profile',(req,res))
.
.
Then, in your index.js or entry point of your project,
const users=require('./api/users')
const questions=require('./api/questions')
app=require('express')
app.use('/users',users)
app.use('/questions',questions)
So in effect, you say for any /users route, refer to the users.js file, for any /questions routes, refer questions.js file and so on
Use a simple module, don't be invoking routes until you form a heap.
Try Route Magic
You want to do just 2 lines of code and have the module read your directory and file structure to handle all the app.use routing invocations, like this:
const magic = require('express-routemagic')
magic.use(app, __dirname, '[your route directory]')
For those you want to handle manually, just don't use pass the directory to Magic.
I'm working on a MEAN (Mongo Express.js Angular Node.js) CRUD application. I have it working but everything is in one .js file. The single source code file is quite large. I want to refactor the code so CRUD functionality is in different source code files. Reading through other posts, I've got a working model but am not sure it is the right way in Node using Mongo to get it done.
Here's the code so far:
<pre>
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
var path = require('path');
var db;
var connect = 'mongodb://<<mddbconnect string>>;
const MongoClient = require('mongodb').MongoClient;
var ObjectID = require("mongodb").ObjectID;
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static(__dirname + '/'));
// viewed at http://localhost:<<port referecnes in app.listen>>
app.get('/', (req, res) => {
res.sendFile(path.join(__dirname + '/index.html'));
});
MongoClient.connect(connect, (err, database) => {
if (err) return console.log(err)
db = database
app.listen(3000, () => {
console.log('listening on 3000' + Date() );
// Here's the require for the search function in another source code file.
var searchroute = require('./serverSearch')(app, db);
})
})
//Handlers
The rest of the CRUD application functions with app.post, app.get. These are other functions I want to move into different source code files, like serverSearch.js.
</pre>
The code I separated right now is the search functionality which is inside of the MongoClient.connection function. This function has to successfully execute to make sure the variable 'db' is valid before passing both variables 'app' and 'db' to the the search function built out in the source code file serverSearch.js.
I could now build out my other CRUD functions in separate files in put them in the same area as 'var searchroute = require('./serverSearch)(app,db);
Is this the best way to separate code in a MEAN application where the main app and db vars need to be instantiated then passed to functions in other source code files?
What you are basically describing is modular coding heading towards "services" perhaps even micro-services. There are a few factors to keep in mind for your system. (I have no doubt that there are many other approaches to this btw). Basically in most NodeJS systems I have worked on (not all) I try to apply the following architecture in development and then bring over as much as possible I to production.
Create a directory under the main one. I usually use some type of name that points to the term functions. In this directory I maintain function and /or class files divided into categories. Function wrappers for DB would be held in DB functions. This file would only contain functions for the DB. Security functions in another file. Helper functions in another. Time manipulation in another. I am sure you get the idea. These are all wrapped in module exports
Now in any file in my project where say I would need DB and helpers I will start it by:
let nhelpers = require("helpfuncs");
let ndb = require("dbfuncs");
Obviously names are different.
And btw I divide all the NPM packages in the same way under an environment directory.
Maintaining that kind of structure allows you to maintain sane order over the code, logical chaining in any decent IDE, and having relevant methods show up in your IDE without having to remember every function name and all the methods within.
It also allows you to write an orderly system of micro-services making sure each part dies exactly what you want and allows for sane debugging.
It took me awhile to settle on this method and refine it.
It paid off for me. Hope this helps.
Edit to clarify for the OP:
When it comes to the process.env variables I became a great fan of dotenv https://www.npmjs.com/package/dotenv
This little package has saved me an incredible amount of headaches. Of course you will have to decide if you include it in production or not. I have seen arguments for both, but i think in a well set up AWS, Google, Azure environment (or in Docker of course) I am of the opinion it can safely be used.
A couple of caveats.
Do not leave your dotenv file in the root. Move it somewhere else in your directory structure. It is simple and I actually put it in the same directory as all my environment files and helper files.
Remember it is simply a text file. So an IDE will not pick up your specific env variables in chaining. (Unless someone knows of a trick which I would love to hear about)
If you put env variables like access info to your DB system or other sensitive stuff, HASH THEM FIRST, put the hash in your env and have a function in your code which specifically just does the hash to the string. Do not under any conditions leave sensitive information in your environment file without hashing it first.
The final Gatcha. These are not PHP magic globals which cannot be overwritten. If you lose track and overwrite one of those process.env variables in your code it will take the new value until you restart your node app and it reads from the dotenv file again. (But then again that is the rule with all environment variables not only user defined ones)
Any typos above excuse me. Done from my cell in the train.
I'm new to Node but I figured I'd jump right in and start converting a PHP app into Node/Express. It's a bilingual app that uses gettext with PO/MO files. I found a Node module called node-gettext. I'd rather not convert the PO files into another format right now, so it seems this library is my only option.
So my concern is that right now, before every page render, I'm doing something like this:
exports.home_index = function(req, res)
{
var gettext = require('node-gettext'),
gt = new gettext();
var fs = require('fs');
gt.textdomain('de');
var fileContents = fs.readFileSync('./locale/de.mo');
gt.addTextdomain('de', fileContents);
res.render(
'home/index.ejs',
{ gt: gt }
);
};
I'll also be using the translations in classes, so with how it's set up now I'd have to load the entire translation file again every time I want to translate something in another place.
The translation file is about 50k and I really don't like having to do file operations like this on every page load. In Node/Express, what would be the most efficient way to handle this (aside from a database)? Usually a user won't even be changing their language after the first time (if they're changing it from English).
EDIT:
Ok, I have no idea if this is a good approach, but it at least lets me reuse the translation file in other parts of the app without reloading it everywhere I need to get translated text.
In app.js:
var express = require('express'),
app = express(),
...
gettext = require('node-gettext'),
gt = new gettext();
Then, also in app.js, I create the variable app.locals.gt to contain the gettext/translation object, and I include my middleware function:
app.locals.gt = gt;
app.use(locale());
In my middleware file I have this:
mod
module.exports = function locale()
{
return function(req, res, next)
{
// do stuff here to populate lang variable
var fs = require('fs');
req.app.locals.gt.textdomain(lang);
var fileContents = fs.readFileSync('./locales/' + lang + '.mo');
req.app.locals.gt.addTextdomain(lang, fileContents);
next();
};
};
It doesn't seem like a good idea to assign the loaded translation file to app, since depending on the current request that file will be one of two languages. If I assigned the loaded translation file to app instead of a request variable, can that mix up users' languages?
Anyway, I know there's got to be a better way of doing this.
The simplest option would be to do the following:
Add this in app.js:
var languageDomains = {};
Then modify your Middleware:
module.exports = function locale()
{
return function(req, res, next)
{
// do stuff here to populate lang variable
if ( !req.app.locals.languageDomains[lang] ) {
var fs = require('fs');
var fileContents = fs.readFileSync('./locales/' + lang + '.mo');
req.app.locals.languageDomains[lang] = true;
req.app.locals.gt.addTextdomain(lang, fileContents);
}
req.textdomain = req.app.locals.gt.textdomain(lang);
next();
};
};
By checking if the file has already been loaded you are preventing the action from happening multiple times, and the domain data will stay resident in the server's memory. The downside to the simplicity of this solution is that if you ever change the contents of your .mo files whilst the server is running, the changes wont be taken into account. However, this code could be extended to keep an eye on the mtime of the files, and reload accordingly, or make use of fs.watchFile — if required:
if ( !req.app.locals.languageDomains[lang] ) {
var fs = require('fs'), filename = './locales/' + lang + '.mo';
var fileContents = fs.readFileSync(filename);
fs.watchFile(filename, function (curr, prev) {
req.app.locals.gt.addTextdomain(lang, fs.readFileSync(filename));
});
req.app.locals.languageDomains[lang] = true;
req.app.locals.gt.addTextdomain(lang, fileContents);
}
Warning: It should also be noted that using sync versions of functions outside of server initialisation is not a good idea because it can freeze the thread. You'd be better off changing your sync loading to the async equivalent.
After the above changes, rather than passing gt to your template, you should be able to use req.textdomain instead. It seems that the gettext library supports a number of requests directly on each domain object, which means you hopefully don't need to refer to the global gt object on a per request basis (which will be changing it's default domain on each request):
Each domain supports:
getTranslation
getComment
setComment
setTranslation
deleteTranslation
compilePO
compileMO
Taken from here:
https://github.com/andris9/node-gettext/blob/e193c67fdee439ab9710441ffd9dd96d027317b9/lib/domain.js
update
A little bit of further clarity.
Once the server has loaded the file into memory the first time, it should remain there for all subsequent connections it receives (for any visitor/request) because it is stored globally and wont be garbage collected — unless you remove all references to the data, which would mean gettext would need to have some kind of unload/forget domain method.
Node is different to PHP in that its environment is shared and wraps its own HTTP server (if you are using something like Express), which means it is very easy to remember data globally as it has a constant environment that all the code is executed within. PHP is always executed after the HTTP server has received and dealt with the request (e.g. Apache). Each PHP response is then executed in its own separate run-time, which means you have to rely on databases, sessions and cache stores to share even simple information and most resources.
further optimisations
Obviously with the above you are constantly running translations on each page load. Which means the gettext library will still be using the translation data resident in memory, which will take up processing time. To get around this, it would be best to make sure your URLs have something that makes them unique for each different language i.e. my-page/en/ or my.page.fr or even jp.domain.co.uk/my-page and then enable some kind of full page caching using something like memcached or express-view-cache. However, once you start caching pages you need to make certain there aren't any regions that are user specific, if so, you need to start implement more complicated systems that are sensitive to these areas.
Remember: The golden rule of optimisation, don't do so before you need to... basically meaning I wouldn't worry about page caching until you know it's going to be an issue, but it is always worth bearing in mind what your options are, as it should shape your code design.
update 2
Just to illustrate a bit further on the behaviour of a server running in JavaScript, and how the global behaviour is not just a property of req.app, but in fact any object that is further up the scope chain.
So, as an example, instead of adding var languageDomains = {}; to your app.js, you could instantiate it further up the scope of wherever your middleware is placed. It's best to keep your global entities in one place however, so app.js is the better place, but this is just for illustration.
var languageDomains = {};
module.exports = function locale()
{
/// you can still access languageDomains here, and it will behave
/// globally for the entire server.
languageDomains[lang]
}
So basically, where-as with PHP, the entire code-base is re-executed on each request — so the languageDomains would be instantiated a-new each time — in Node the only part of the code to be re-executed is the code within locale() (because it is triggered as part of a new request). This function will still have a reference to the already existing and defined languageDomains via the scope chain. Because languageDomains is never reset (on a per request basis) it will behave globally.
Concurrent users
Node.js is single threaded. This means that in order for it to be concurrent i.e. handle multiple requests at the "same" time, you have to code your app in such a way that each little part can be executed very quickly and then slip into a waiting state, whilst another part of another request is dealt with.
This is the reason for the asynchronous and callback nature of Node, and the reason to avoid Sync calls whilst your app is running. Any one Sync request could halt or freeze execution of the thread and delay handling for all other requests. The reason why I state this is to give you a better idea of how multiple users might interact with your code (and global objects).
Basically once a request is being dealt with by your server, it is it's only focus, until that particular execution cycle ends i.e. your request handler stops calling other code that needs to run synchronously. Once that happens the next queued item is dealt with (a callback or something), this could be part of another request, or it could be the next part in the current request.
I'm designing an app with node.js and Express, and I was wondering if it was possible to move certain routing logic out of the app.js file. For exapmle, my app.js currently contains:
app.get('/groups',routes.groups);
app.get('/',routes.index);
Is there a way to move this logic out of the app.js file, and only have something like:
app.get('/:url',routes.get);
app.post('/:url",routes.post);
such that all GET requests would be processed by routes.get and all POST requests processed with routes.post?
You could pass a regular expression as the route definition:
app.get(/.+/, someFunction);
This would match anything. However, if you simply want to move your route definitions outside of your main app.js file, it is much clearer to do something like this:
app.js
var app = require('express').createServer();
...
require('routes').addRoutes(app);
routes.js
exports.addRoutes = function(app) {
app.get('/groups', function(req, res) {
...
});
};
This way, you're still using Express' built-in routing, rather than re-rolling your own (as you'd have to do in your example).
FULL DISCLOSURE: I am the developer of the node module mentioned below.
There is a node module that does kind of what you're asking for (and will, eventually, do more). It offers automatic routing based on convention over configuration for express. The module name is honey-express, but is currently in alpha development and not yet available on NPM (but you can get it from the source at https://github.com/jaylach/honey-express.
A short example of how it works: (Please note that this coffeescript)
# Inside your testController.coffee file. Should live inside /app/controllers
honey = require 'honey-express'
TestController = new honey.Controller
index: ->
# #view() is a helper method to automatically render the view for the action you're executing.
# As long as a view (with an extension that matches your setup view engine) lives at /app/views/controller/actionName (without method, so index and not getIndex), it will be rendered.
#view()
postTest: (data) ->
# Do something with data
Now, inside your app.js file you just have to setup some simple configuration:
# in your app.configure callback...
honey.config 'app root', __dirname + '/app'
app.use honey.router()
Now anytime a request comes in, honey will automatically look for a controller with the specified route, and then look for a matching action.. for example -
/test will automatically route to the index/getIndex() method of
testController
/ will automatically route to the index/getIndex() method of the homeController (the default controller is home), if it exists
/test/test will automatically route to the postTest() method of testController if the http method is POST.
As I mentioned, the module is currently in it's alpha state but the automatic routing works wonderfully and has been tested on two different projects now :) But since it's in alpha development, the documentation is missing. Should you decide to go this route, you can look at the sample I have up on the github, look through the code, or reach out to me and I'd be happy to help :)
EDIT: I should also note that honey-express does require the latest (BETA) version of express as it uses features that are not present in 2.x of express.
I'm trying to create a little application to store snippets of code using nodejs and mongodb
I'm using Coffeescript to write the app.
The problem is, i want to separate the code in modules
so i create this folder structure
/app
/lib
/models
/routes
core.coffee
The core.coffe is the "server" app using expressjs
so in this file i have
mongoose = module.exports.mongoose = require 'mongoose'
app = module.exports.app = express.createServer()
Snippet = module.exports.Snippet = require __dirname+'/lib/models/Snippet'
#App configurations
routes = require(__dirname+'/lib/routes/general')
In lib/models/Snippet
mongoose = module.parent.exports.mongoose
Snippet = new mongoose.Schema
title:
type: String
default:'Title'
mongoose.model 'Snippet',Snippet
exports.Snippet = mongoose.model 'Snippet'
In /lib/routes/general.coffee
app = module.parent.exports.app
mongoose = module.parent.exports.mongoose
Snippet = module.parent.exports.Snippet
app.get '/test', (req,res)->
snip = new Snippet()
res.send snip
But this don't work i get the following error message
TypeError: object is not a function
at Object.CALL_NON_FUNCTION_AS_CONSTRUCTOR (native)
How can I accomplish that?
I see a noteworthy typo:
Snippet = module.exports.Snippt = require __dirname+'/lib/models/Snippet'
Change module.exports.Snippt to module.exports.Snippet.
Let's start by looking at how you're using require. It looks like you're trying to load all the project's requirements in core.coffee, and then re-export them elsewhere. That's an odd way of doing it, most people just require those libraries in each module that needs them (for now at least, see the end of my answer).
For example, you need mongoose in lib/models/Snippet, so just require it there:
lib/models/Snippet:
mongoose = require 'mongoose'
Next, there's no need to use __dirname to require a relative path, require copes fine with a path starting with ./:
require './lib/models/Snippet'
I still wasn't able to get the code to work cleanly (I'm guessing we're not seeing the full code), but it might be enough to set you on the right path.
Finally, if you want to go down the route of exporting everything on the main module can I suggest taking a look at dave-elkan's layers project. The plain version doesn't support coffeescript, but I've created a fork that does.
It's very lightweight, and makes almost no assumptions about your project structure. The basic idea is that you give layers() your express app object and a directory. Layers will scan that directory and set up any subdirectories as layers on your app object.
In your case you'd pass in a rootPath: __dirname + '/lib' and your app object would get app.models.Snippet and app.routes.general added onto it. That's still not quite how I'd structure it, but you might be able to come up with something that matches your style from there.