In an attempt to modularize a large existing Node+Express+Mongoose
application into multiple mountable apps, each developed as a
separate NPM package, we're wondering whether sharing a single
Mongoose instance between them is a good idea?
Let's say we have a suite of NPM packages each containing client-side assets,
Mongoose models, and a REST-API implemented with Express. They do share
a few common traits but are essentially to be considered separate
reusable artefacts. A host application, also Express-based, mounts these
under various root URIs:
var discussions = require('discussions'),
tickets = require('tickets'),
events = require('events'),
express = require('express'),
app = express();
var environment = { ...see below... };
...
app.use('/events-api', events(environment));
app.use('/tickets-api', tickets(environment));
app.use('/discussions-api', discussions(environment));
Now, since the events, tickets and discussions apps (separate NPM packages
pulled in via the host package.json) use Mongoose, as do the
host application itself, we figured we would pass in the host Mongoose instance
through some kind of environment object that also include other
stuff that the host wants to share with the mounted apps.
Do you see any obvious flaws with this approach? The mounted apps
in this case would not specify Mongoose as a dependency in their
respective package.json, and they would not require('mongoose')
as normally done but instead get the Mongoose instance from the host
which is responsible for connecting it to MongoDB.
If this is a bad idea and you suggest each sub-app declare a dependency
towards Mongoose on their own, each NPM package would get their own
copy of Mongoose and would each have to connect to MongoDB, right?
Some background info:
We really do want to include the apps in a host application, running
in a single process, rather that having multiple Node instances. The
host contains middleware for authentication and other things.
We do want to have the apps as separately developed NPM packages included
as versioned dependencies of the various host applications that we build,
rather than just copying their source to a host application.
We realize that reusing the same Mongoose instance between multiple
mounted apps will have them share the same model namespace.
Edit: To clarify the package structure after all has been npm installed:
host/
assets/
models/
routes/
node_modules/
express/ ...
mongoose/ ...
events/
assets/ ...
models/ ...
routes/ ...
tickets/
assets/ ...
models/ ...
routes/ ...
discussions/
assets/ ...
models/ ...
routes/ ...
That is, the events, tickets, and discussions apps do not include
Mongoose (or Express) of their own but are designed to rely on an always-present
host application that suppliesd those dependencies.
We're assuming here that an NPM package like tickets cannot simply
require stuff from the parent, right?
If you want to reuse your Mongoose package between other NPM packages, the best way to do it is to install the shared package at the top level app and then use it to initialize the other NPM packages.
In the top level:
var db = require('myMongooseDb'),
events = require('events')(db),
...
Then your events package just needs to export a function that takes the db as a parameter.
I suggest you have a look at https://github.com/jaredhanson/node-parent-require, a recently published package which solved this issue for me.
The node-parent-require Readme file on the Github project page provides a detailed walkthrough using mongoose.
Basically, you need to dig in your submodule and replace this:
mongoose = require("mongoose");
... with this:
try {
var mongoose = require('mongoose');
} catch (_) {
// workaround when `npm link`'ed for development
var prequire = require('parent-require')
, mongoose = prequire('mongoose');
}
Don't forget to add mongoose as a peerDependency in your submodule's package.json. For example:
"peerDependencies": {
"mongoose": "3.x"
}
You may also want to read http://blog.nodejs.org/2013/02/07/peer-dependencies/ first.
const mongoose = require('mongoose-global')();
You can just require one global mongoose instance and use it anywhere.
https://www.npmjs.com/package/mongoose-global
Related
I'm using Lerna to handle multiple packages in my repo.
Basically, I have a package #pkg/server where my Node/Express App is and an other package #pkg/model where my mongoose Schemas reside.
My REST API is using some of the models, with an import ModelA from '#pkg/model'.
The problem is that both of these packages have mongoose as a dependency (the #pkg/server uses mongoose to establish a connection to the mongoDB server, and #pkg/model uses mongoose to define schemas and models).
In my server, whenever I try to call ModelA.find(...), it just hangs forever. From the mongoose docs, it looks like I have to register a model to a mongoose connection and not to the mongoose object itself.
My first guess is to pass the connection object created by the server to the #pkg/model and then register the models to this connection object.
My question is, is this the right way to do it? Or is there a way to do something like (in the #pkg/server) mongoose.loadSchemas(#pkg/model) ?
What I have ended up doing, is that I'm passing the connection to the package requiring mongoose thus sharing the connection between packages.
Some package can register models, an other one can make queries, etc...
All the Node.js tutorials that I have followed have put everything in one file. It includes importing of libraries, routing, database connecting and starting of the server, by say, express.js:
var app = require('express');
app.get('/somePath', blah blah);
app.listen(...);
Now, I have 4 node servers behind an Nginx load balancer. It then becomes very difficult to have the source code updated on all the four servers.
Is there a way to keep the source code out of the server creation code in such a way that I can deploy the source code on the servers as one package? The server creation code should not know anything about routing or database connections. It should only be listening to changes in a folder and the moment a new module meta file appears, it starts hosting that web application.
Much like how we deploy a Java code packaged as war by Maven and deployed to the webapp of Tomcat, because Tomcat instantiation is not part of the source code. In node.js it seems server is also part of the source code.
For now, the packaging is not my concern. My concern is how to separate the logic and how do I point all my servers to one source code base?
Node.js or JavaScript for that matter doesn't have a concept like WAR. But what it does have is something similar. To achieve something WAR like, you would essentially bundle the code into one source file using something like webpack. However, this will probably not work with Node.js modules like http (Express uses `http since it likely calls or relies on native V8/C++ functions/libraries.
You could also use Docker and think of the Docker containers as WARs.
Here is what I figured out as a work around:
Keep the servers under a folder say, "server_clusters" and put different node servers there, namely: node1.js, node2.js, node3.js, node4.js, etc (I know, in the real world, the clusters would be different VMs or CPUs altogether but for now, I simply want to separate server creation logic from source code). These files would have this code snippet:
var constants = require('./prop');
var appBasePath = constants.APP_BASE_DIR;
var appFilePath = appBasePath + "/main";
var app = require(appFilePath);
//each server would have just different port number while everything else would remain constant
app.listen(8080, function (req, res) {
console.log("server started up");
});
Create a properties file that would have the path to the source code and export the object. That simple. This is what is used on line#1 in the above code
Create the source directory project wherever you want on the machine and just update its home directory in the constant file above. The source code directory can export one landing file that will provide the express app to the servers to start:
var express = require('express');
var app = express();
module.exports = app;
With this, there are multiple servers that are pointing to the same source code.
Hope this helps to those who are facing the same problem.
Other approaches are welcome.
I'm trying to create a library that uses the ORM Bookshelf.js (so basically a wrapper on top of Bookshelf.js if you will). Now Bookshelf.js requires you to run a driver installation to be able to connect to the correct database (such as: npm install mysql) in the root of your project.
My own library also requires this, since Bookshelf.js requires it, but it keeps saying Error: Cannot find module 'mysql' which is normal since the Bookshelf instance does not have the mysql dependency in its node_modules folder.
If I type out the structure it would look like this:
* New Project
** mysql
** LibThatUsesBookshelf
**** Bookshelf
Now Bookshelf.js probably requires this structure:
* New Project
** LibThatUsesBookshelf
**** mysql
**** Bookshelf
Is there a way that I am able to still get bookshelf working with the first structure shown?
P.S. I know the explanation might be confusing, but it is a hard problem and I got no other way of explaining it.
Let the client of your module pass in an instance to bookshelf:
// mysuperfancymodule.js
module.exports = function(bookshelf) {
// do fancy things with the bookshelf instance
}
Then in your client application:
// some app code:
var mySuperFancyModule = require('mysuperfancymodule'); // assuming it is published to npm
var knex = require('knex')({
// init knex...
});
var bookshelf = require('bookshelf')(knex);
mySuperFancyModule(bookshelf); // init my super fancy module
What my team has been doing: server.js requires server-config.js (Express routing, API requests, etc), which requires query.js (where my Mongoose MongoDB schema is defined) which in turn requires db-config.js (where I actually connect to Mongo). Then in my db-config I have to require the query.js Schema anyways, which creates a cyclic dependency.
Wouldn't it be better just to require db-config instead of query.js inside of server-config? That way, the database connection is started and then the cyclic dependency is broken.
First off, I am a totally new developer with regards to Node.js.
I have started creating a sample express application, and I wanted to use an additional module to learn more about it. I have installed via npm the "mysql" module, all fine.
I have added it at the beginning of app.'s, like this:
var mysql = require('mysql');
now, as you already know, express created an index.js file inside the directory routes: i would like to be able to access the mysql variable to connect to the db from this index.js page but, using the command
var connection = mysql.createConnection({
host : 'localhost',
user : 'root',
password : 'root',
});
obviously doesn't work, giving the "500 ReferenceError: mysql is not defined".
Of course, I am sure I need to pass this variable, but I really have no clue, can any good soul enlighten me? I know this is a very small and basic thing, but I tried this already, and doesn't seem to work:
... app.get('/', routes.index, mysql); ...
and on index.js:
exports.index = function(req, res, mysql){ ...
In Node.js, you should require modules in the files you need to use them in; so, if you want to use the mysql package in your routes/index.js file, at the top of that file, do
var mysql = require('mysql');
You may end up requiring a module in more than one file; this is normal and, in many ways, a good thing (no namespacing issues, etc.)
If you want to learn more about modules and packages, you may be interested in these two screencasts:
Modules: http://nodecasts.net/episodes/3-modules
Packages & npm: http://nodecasts.net/episodes/4-packages-and-npm
Passing the mysql object through app.get() would be a normal reaction, but I'm pretty sure you're over-thinking it.
It's as simple as including var mysql = require('mysql'); at the top of routes/index.js. In fact, you may find that you don't even need to require mysql in app.js if all database interactions are done in your routes.