How to separate express server code from Express business logic code? - node.js

All the Node.js tutorials that I have followed have put everything in one file. It includes importing of libraries, routing, database connecting and starting of the server, by say, express.js:
var app = require('express');
app.get('/somePath', blah blah);
app.listen(...);
Now, I have 4 node servers behind an Nginx load balancer. It then becomes very difficult to have the source code updated on all the four servers.
Is there a way to keep the source code out of the server creation code in such a way that I can deploy the source code on the servers as one package? The server creation code should not know anything about routing or database connections. It should only be listening to changes in a folder and the moment a new module meta file appears, it starts hosting that web application.
Much like how we deploy a Java code packaged as war by Maven and deployed to the webapp of Tomcat, because Tomcat instantiation is not part of the source code. In node.js it seems server is also part of the source code.
For now, the packaging is not my concern. My concern is how to separate the logic and how do I point all my servers to one source code base?

Node.js or JavaScript for that matter doesn't have a concept like WAR. But what it does have is something similar. To achieve something WAR like, you would essentially bundle the code into one source file using something like webpack. However, this will probably not work with Node.js modules like http (Express uses `http since it likely calls or relies on native V8/C++ functions/libraries.
You could also use Docker and think of the Docker containers as WARs.

Here is what I figured out as a work around:
Keep the servers under a folder say, "server_clusters" and put different node servers there, namely: node1.js, node2.js, node3.js, node4.js, etc (I know, in the real world, the clusters would be different VMs or CPUs altogether but for now, I simply want to separate server creation logic from source code). These files would have this code snippet:
var constants = require('./prop');
var appBasePath = constants.APP_BASE_DIR;
var appFilePath = appBasePath + "/main";
var app = require(appFilePath);
//each server would have just different port number while everything else would remain constant
app.listen(8080, function (req, res) {
console.log("server started up");
});
Create a properties file that would have the path to the source code and export the object. That simple. This is what is used on line#1 in the above code
Create the source directory project wherever you want on the machine and just update its home directory in the constant file above. The source code directory can export one landing file that will provide the express app to the servers to start:
var express = require('express');
var app = express();
module.exports = app;
With this, there are multiple servers that are pointing to the same source code.
Hope this helps to those who are facing the same problem.
Other approaches are welcome.

Related

Make a logger for Node Js

I have a project in Node Js, which executes the project on port 3000 and I access from ngrok with my browser to said localhost port, and it executes a server on port 3001 to make requests to a Maria database db. The project is done in react and the server with express.
I want to save the application logs (errors, warnings, etc.) in a log file so that I can see them whenever I want.
My intention was to use winston, and while I have no problem on the server side (3001), when I try to adapt it to the main project, I get an error that it cannot save files (the reason that appears is that it runs from the browser, and you can't create such a file because you don't have access to the project folders)
Can anyone give me some advice? Am I wrong to use winston, and should I use another?
Greetings and thanks
I've never used winston before and I couldn't find anything online about your error. In the past I've always just used node's fs module to create a log of errors and restarts.
const fs = require('fs')
Node's File System Documentation: https://nodejs.dev/learn/the-nodejs-fs-module
Short YouTube Tutorial: https://www.youtube.com/watch?v=U57kU311-nE

expressJS exporting other things in addition to app

In my web application (I'm using expressJS), there are many services (such as mongoDB connection, MQTT connection, etc.) that need to be executed once the whole application is executed (using npm start command). Therefore, I can make use of these services in my entire application. For example, I want to use my MQTT connection in different files.
My idea is to export the MQTT connection, MongoDB connection, etc. in addition to the app this way:
//app.js
module.exports = {
app: app,
mqttConnection: myMQTTConnection,
db: myMongoDB
};
However, we know that this approach doesn't work (I tested it and got an error saying: TypeError: app.set is not a function).
How can I export other things in addition to app from app.js file?
If my approach is not possible, what other approaches can I use? (considering the fact that many services (such as connecting to a server, etc.) are asynchronous)

Set up a server from within an electron app

I haven't had any success looking for this because I mostly find misleading questions, about people wanting to use data from a server inside of their electron app. That's not my case.
I have a regular app, which uses a server on the internet, just like any other, but we want to make it available for schools without internet (without any or without reliable internet), so what I'm trying to do is to create a version of my server which runs from an electron exe and serves files for the students conected to the wifi (but no the internet) to access. After the process is done "offline", I will sync the data from the electron app itself.
I tried to run a server from express but I didn't have any progress so far. What I tried was to put the exact same code from my node server in my main.js file and had no luck.
I know that's not what electron is supposed to do, if you're positively sure there is no way to do that, please tell me so I can search for another alternative.
A simple approach is to create a cluster where the master process is the Electron Main and the worker process is the server.
Example:
Change the main on package.json to start.js
On start.js write:
const cluster = require('cluster');
if (cluster.isMaster) {
require('./main.js'); // your electron main file
cluster.fork();
} else {
require('./server.js'); // your server code
}

Node.js app on AWS for beginner

I started learning node.js a couple weeks ago and just finished my first small project, a basic live chat website using socket.io and express. The structure for my project looks like this:
ChatApp
|
|____backend.js // node server side code
|
|____ static
| |
| |_____ libs
| |
| |___ app.js // front end logic
| |
| |___ jquery.min.js
|____ views
|
|_____ index.html // Client website
My goal right now is to learn how to use AWS to make my application available so people on different machines can talk to one another, not just me on my local server. I tried following this guide which uses Elastic Beanstalk to deploy a sample repository, but I'm having a hard time seeing how to translate it to my folder structure, since they don't even have an HTML for instance.
My server code looks like this:
//*****************//
// Sets up backend //
//*****************//
var app = require('express')();
var server = require('http').Server(app);
var io = require('socket.io')(server);
var express = require('express');
server.listen(8080);
app.use(express.static(__dirname + '/views'));
app.use(express.static(__dirname + '/static'));
var users = [];
//*****************//
// Sends out html //
//*****************//
app.get('/', function(req, res){ // Main page
res.render('index.html');
});
//*************************//
// Handles socket requests //
//*************************//
io.on("connection", handleIO); // Called when user connects
function handleIO(socket){
console.log('Client connected...');
// Bunch of socket.io code I didn't think was necessary to add
}
Anyways, I was wondering if any of you enlightened folks could help a noob out with deploying his first website. If you could either give me a general outline or point me to one I'd really appreciate it as AWS can be pretty intimidating when first starting out. Thanks.
I would say jumping straight into Amazon Web Services would be a mistake as AWS is just an abstraction layer on millions of tasks that you can perform as a Cloud Administrator.
If you do not have the basic concepts of server administration or have not worked in a similar capacity, it can prove to be counter-productive.
Still if you are willing to make the jump, here are the steps I would recommend:
Learn to create an EC2 instance
Setup/install required software on your EC2 instance
Transfer your code to the EC2 instance
Configure/run your application.
If it helps, EC2 is just a VPC with shell access and you can use it through the command line as you normally would a desktop linux.
Boxfuse lets you deploy your Node.js app to AWS effortlessly in literally two steps from within your project directory:
npm-bundle: creates a tgz including your app and all the required node modules (install using: npm install -g npm-bundle)
boxfuse run -env=prod:
Creates a minimal image containing the contents of the tgz bundle as well as the Node.js runtime, the Linux kernel and a bootloader
Pushes that image in the secure Boxfuse Vault
Converts it into an AMI
Creates a new domain name
Provisions an elastic IP or an ELB (depending on the app type you configured)
Creates a security group with the correct ports mapped
Launches a new EC2 instance and ensures it is healthy
Assign the elastic IP to the instance
Boxfuse is based on 3 principles: Immutable Infrastructure, Minimal Images and Blue/Green deployments with zero downtime.
Boxfuse also comes with out-of-the-box support for auto-scaling, relational databases and fast local development and testing on VirtualBox.
We have a tutorial you can follow to get started in 5 minutes: https://boxfuse.com/getstarted/nodejs
Disclaimer: I am the founder and CEO of Boxfuse
start with the AWS docs and get a free account on AWS, make your hand dirty.Make things break things and again fix them, this is how you will learn AWS.
Regarding study material
1. AWS DOCS
2. REinvent videos on youtube
3. Buy a Pluralsite account they have very good course over these.
Start with Ec2 and VPC

Can I define Express routes in a child process?

So I run a bunch of a little chatbots written in node, nothing too exciting. However, I recently decided to give them their own little web page to display information in a graphical manner. To do this, I figured I'd just run express.
However, I'm running my bots with a wrapper file that starts each chatbot as a child process. Which makes using express a little tricky. Currently I'm starting the express server in the wrapper.js file like so:
var express = require("express");
var web = express();
web.listen(3001);
And then in the child processes, I'm doing this:
var express = require("express");
var web = express();
web.get("/urlforbot",function (req,res) {
res.send("Working!");
});
However, when I navigate to :3001/urlforbot, I get Cannot GET /urlforbot.
Any idea what I'm doing wrong and how to fix this?
Edit: This is my complete wrapper file: http://snippi.com/s/3vn56m2
Edit 2: This is what I'm doing now. I'm hosting each bot on it's own port, and storing that information in the configs. This is the code I'm using, and it appears to be working:
web.get("/"+cfg.route, function (req,res) { // forward the data
res.redirect('http://url.com:'+cfg.port+"/"+cfg.route);
});
Since your bots run as separate processes (any particular reason?), you have to treat each one as having to implement their own HTTP server with Express:
var express = require("express");
var web = express();
web.get("/urlforbot",function (req,res) {
res.send("Working!");
});
web.listen(UNIQUE_PORT_NUMBER);
Each bot process needs to listen on a unique port number, it can't be shared.
Next, you need to map requests coming in on port 3001 in the 'master' process to the correct child process' Express server.
node-http-proxy has a useful option called a ProxyTable with which to create such a mapping, but it requires the master process to know what the endpoint (/urlforbot in your terms) for each bot is. It also requires that the master knows on which port the bots are listening.
EDIT: alternatively, you can use child_process.fork to fork a new process for each of your bots, and communicate between them and the master process (port numbers and such, or even all the data required to generate the /urlforbot pages) using the comm channel that Node provides, but that still sounds like an overly complex setup.
Wouldn't it be possible to create a Bot class instead? You'd instantiate the class for each bot you want to run, and that instance loads its specific configuration and adds its routes to the Express server. All from the same process.

Resources