I am trying to develop an node js application that acts as api-gateway / facade layer for Services(developed in Spring Boot).
Is it a good practice or not?
If yes, which nodejs framework should I use?(Async / co / Promise / Async-Await ) etc. I mean what is currently used mostly on production enviornemnt?
"Is it a good practice or not?"
What is your question related to? Using an API gateway/facade? Using spring boot? Using async/await...? What exactly is your problem?
I guess you want to develop a spring boot based microservice architecture with a nodeJS based api orchestrator as a frontcontroller and single entry point?
Don't confuse the technical side of naive routing (load balancing with nginx, round robin, reverse proxy, etc.) to increase capacity, speed, availability etc. with the semantic business integration of services through url path mapping.
An API Orchestrator addresses the semantic abstraction and integration of an underlying service landscape. API Gateway vs. API Orchestrator!
To my personal view, using an API Orchestrator is a acceptable solution in conjunction with microservices. It is the easiest and modest way to
integrate and componse an underlying service layer.
Just to state a few positive and negative aspects:
Single entry point for standard business cases such as
authentification, security issues, session mangament, logging etc.
Can also be started and managend as a microservice. Feel free to use
a 3-tier layered architecture for the API orchestrator microservice
Abstracts the complexity of an underlying microservice layer
Might become a god thing.
In context of microservices, the API Orchestrator performs to much
of business cases
High coupling, complexity...
Design trial of a nodeJS based API Orchestrator with HTTP Communication ...
Evaluate a (web) server (express.js, hapi.js, your-own-node-server)
Evaluate a http-request API (axios, node-fetch, r2, your-own-http-api). HTTP-API should resolve to a promise object!
Example of an express.js based API Orchestrator:
const express = require('express');
const http = require('http');
const path = require('path');
const app = express();
const port = 3000;
// define middleware plugins in express.js for your API gateway like session management ...
app.use(express.static(path.join(__dirname, 'public')));
// define relevant business/use case relevant semantic routes or commands e.g. /getAllUsers or REST-URL or /whatever
app.get('/whatever', (request, response) => {
//consumes whatever service
const getWhatEverToGet = () => {
return new Promise((resolve, reject) => {
//connection data should be read from a service registry or by configuration management (process level, file level, environemnt level)
http.get({
hostname: 'localhost',
port: 3001,
path: `/whatever_service_url`
}, (res) => {
// built-in HTTP-API http.get() uses streams, hence "onData"-event should be buffered, not done here!
res.on('data', (data) => {
resolve(data.toString());
});
});
});
}
// Here you can consume more services with the same code, when they are connected to each other use async/await to share data synchronized...
//consumes whatever2 service returns promise
//consumes whatever3 service returns promise
const respondWhatEverData = async () => {
let whatEver = await getWhatEverToGet();
response.send(whatEver)
}
// trigger service complete
respondWhatEverData();
})
app.listen(port, (err) => {
if (err) {
return console.log('Shit happens...', err)
}
console.log(`server listens on ${port}`)
})
TL;DR If your NodeJS application is only expected to forward the request to Spring Boot application, then NodeJS setup would probably not be worth it. You should look at Nginx revere proxy which can do all that efficiently.
Async / co / Promise / Async-Await are not frameworks. Promise / async-await are programming constructs in NodeJS; Async / co are convenience libraries to make manage asynchronous code manageable before Promises and async-await were introduced. That said there are multiple rest frameworks, that you could use to receive and pipe requests to your SpringBoot servers. Take a look at Express.JS, Restify, Sails.js all of them can add REST capabilities to NodeJS. You will also need a Rest Client library (like axios or request both of them then support Promises) to be able to forward your requests to target server.
Related
I am required to save logs into a MySQL database of each request and response made to the backend. The issue is that we are migrating to microservices architecture. The backend was made with NodeJS and Express, and it has a middleware that does this task. Currently, it has this middleware attached to each microservice.
I would like to isolate this middleware as its own microservice. The issue is that I don't know how to redirect the traffic this way. This is how I would like to manage it:
I would like to do it this way, because we can make changes or add features to the middleware without having to implement it in each microservice. This is the middleware's code:
const connection = require("../database/db");
const viewLog = (req, res, next) => {
const oldWrite = res.write,
oldEnd = res.end,
chunks = [],
now = new Date();
res.write = function (chunk) {
chunks.push(chunk);
oldWrite.apply(res, arguments);
};
res.end = function (chunk, error) {
if (chunk) chunks.push(chunk);
const bodyRes = Buffer.concat(chunks).toString("utf8");
connection.query("CALL HospitalGatifu.insertLog(?,?,?,?,?)", [
`[${req.method}] - ${req.url}`,
`${JSON.stringify(req.body) || "{}"}`,
bodyRes,
res.statusCode === 400 ? 1 : 0,
now,
]);
oldEnd.apply(res, arguments);
};
next();
};
module.exports = viewLog;
I think there might be a way to manage this with Nginx which is the reverse proxy that we are using. I would like to get an approach of how to change the logs middleware.
Perhaps you might want to take a look at the sidecar pattern which is used in microservice architectures for common tasks (like logging).
In short, a sidecar runs in a container besides your microservice container. One task of the sidecar could be intercepting network traffic and logging requests and responses (and a lot of other possible tasks). The major advantage of this pattern is that you don't need to change any code in your microservices and you don't have to manage traffic redirection yourself. The latter will be handled by the sidecar itself.
The disadvantage is that you are required to run your microservices containerized and use some kind of container orchestration solution. I assume this being the case since you are moving towards a microservices based application.
One question about the log service in between of the webapp and the NGNIX server. What if the logging services goes down for some reason, is it acceptable for the entire application to go down?
Let me give you not exactly what you requested but something to think about.
I can think on 3 solutions for the issue of logging in microservices, each 1 have its own advantages and disadvantages:
Create a shared library that handles the logs, I think its the best choice in must cases. An article I wrote about shared libraries
You can create API gateway, it is great solution for shared logic to all the requests. So it will probably be more work but then can be used for other shared logic. Further read (not written by me :) )
A third option (which I personally don't like) is create a log microservice that listens to LogEvent or something like that. Then from your MSs publish this event whenever needed.
My question could be flagged as "opinion based" but I am wondering which approach is the best for my application as I am able to do it in both ways.
I am building chat application in which users and conversations are saved in MongoDB. I will have my react application consuming API/APIs. The question is - is it better to have REST API and Socket.io applications running separate? For example:
Have REST API running on port 3005
Have Socket.io running on port 3006
React Application consuming these 2 separately and basically they will not know about each other. My endpoints in REST API endpoints and socket.io will be invoked only in front-end.
On the other hand, I can have my socket.io application and REST API working together in 1 big application. I think it is possible to make it working without problems.
To sum up, at first glance I would take the first approach - more cleaner and easy to maintain. But I would like to hear other opinions or if somebody had a similar project. Usually how the things are made in this kind of projects when you have socket.io and REST API?
I would check the pros and cons for both scenario. For example code and resource reusability is better if you have a single application and you don't have to care about which versions are compatible with each other. On the other hand one error can kill both applications, so from security perspective it is better to have separate applications. I think the decision depends on what pros and cons are important to you.
you can make a separate file for socket.io logic like this:
// socket.mjs file
import { Server } from "socket.io"
let io = new Server()
const socketApi = {
io: io
}
io.on('connection',(socket)=>{
console.log('client connected:', socket.id)
socket.join('modbus-room')
socket.on('app-server', data=>{
console.log('**************')
console.log(data)
io.to('modbus-room').emit('modbus-client', data)
})
socket.on('disconnect',(reason)=>{
console.log(reason)
})
})
export default socketApi
and add it to your project like this:
// index.js or main file
//...
import socketApi from "../socket.mjs";
//...
//
/**
* Create HTTP server.
*/
const server = http.createServer(app);
socketApi.io.attach(server);
//
I am new to node.js and docker as well as the microservices architecture.
I am trying to understand what microservices architecture actually is and theoretically I do understand what microservices arch is.Please see the following implementation
This is the index.js file:
var express = require("express");
var app = express();
var service1 = require("./service1");
var service2 = require("./service2");
app.use("/serviceonerequest",service1);
app.use("/servicetwo",service2);
app.listen(3000,function(){
console.log("listening on port 3000");
});
The file service1:
var express = require("express");
var router = express.Router();
router.use(express.json());
router.get("/",(req,res)=>{
//perform some service here
res.send("in the get method of service 1");
res.end();
});
router.post("/letsPost",(req,res)=>{
res.send(req.body);
res.end("in post method here");
})
module.exports = router;
The file service2:
var express = require("express");
var router = express.Router();
router.use(express.json());
router.get("/",(req,res)=>{
//perform some service here
res.end("in the GET method for service 2");
});
router.post("/postservice2",(req,res)=>{
res.send(req.body);
});
module.exports = router;
Does the above qualifies as 'micro service architecture'?Since there are two services and they can be accessed through the 'api-gateway' index.js?
I have read the basic tutorial of Docker.Is it possible to have the above three "modules" in separate containers?
If the above does not qualify as a microservice what should be done to convert the above sample into microservices?
This does not really qualify as a microservice architecture.
The whole code you provided is small enough to be considered one single microservice (containing two routes), but this is not an example of a microservice architecture.
According to this definition;
"Microservices are small, autonomous services that work together"
Building Microservices <-- tip: you should read this book
Both service1 and service2 to be considered a microservice should be autonomous, what is not happening when you place them together in the same express app. For example; you cant restart one without not-affecting the other. You cant upgrade version of service1 without also having to deploy service2. They are not distributed in the sense that they can leave in separate machines.
Actually I think you are missing the concept of microservice architecture. Your services must be independent and if they need to communicate with each other they must use a service discovery mechanism that will return a healthy instance of that service. Another pattern of microservices architecture is that every single service must have an endpoint (/health) that returns the health status of the service, having this your service discovery can check if that instance is healthy and return it as a healthy instance..
Microservices is not about technology it's about the concept and implementing the right patterns. Otherwise you will have a chaos architecture :D
If you want to understande the concepts I really recommend this book: http://shop.oreilly.com/product/0636920033158.do
I need a real-time data to be streamed by a client app. Does StrongLoop (or any StrongLoop component) supports websockets-based CRUD. Consider this image:
Please advise.
I'm not sure if I understand correctly, but in my opinion it's totally doable. In your image, there is an intermediate layer between your client app and your API. Assuming such layer exists, it should call your API's endpoints whenever a given event is emitted in your client app.
I would suggest using http://socket.io/ and plain old http://expressjs.com/ with http://visionmedia.github.io/superagent/ for you intermediate layer.
Something like this:
var app = require('http').createServer(handler)
var io = require('socket.io')(app);
var request = require('superagent');
app.listen(80);
io.on('connection', function (socket) {
socket.on('eventOne', function (data) {
request
.get('/yourapiurl/someresource')
.end(function(err, res){
socket.emit('get-someresource', res.body);
});
});
});
I wouldn't suggest using websockets in the same Strongloop project because, I don't know how complex your API is. This may increase your API complexity and lower your API maintainability.
BTW. You didn't mention what type of data you are trying to send via websockets.
I'm writing an api using nodejs and express and my app is hosted by openshift free plan.
I want to protect my routes from brute force. For example if an IP sends more than 5 requests /sec then block it for 5 minutes. :)
There's nothing stopping you from implementing this in Node.js/express directly, but this sort of thing is typically (and almost certainly more easily) handled by using something like nginx or Apache httpd to handle traffic to your app.
This has the added benefit of allowing you to run the app entirely as an unprivileged user because nginx (or whatever) will be binding to ports 80 and 443 (which requires administrative/superuser/whatever privileges) rather than your app. Plus you can easily get a bunch of other desirable features, like caching for static contents.
nginx has a module specifically for this:
The ngx_http_limit_req_module module (0.7.21) is used to limit the request processing rate per a defined key, in particular, the processing rate of requests coming from a single IP address.
There are several packages on NPM that are dedicated to this, if you are using the Express framework:
express-rate-limiter
express-limiter
express-brute
These can be used for limiting by ip, but also by other information (e.g. by username for failed login attempts).
It is better to limit rates on reverse-proxy, load balancer or any other entry point to your node.js app.
However, it doesn't fit requirements sometimes.
rate-limiter-flexible package has block option you need
const { RateLimiterMemory } = require('rate-limiter-flexible');
const opts = {
points: 5, // 5 points
duration: 1, // Per second
blockDuration: 300, // block for 5 minutes if more than points consumed
};
const rateLimiter = new RateLimiterMemory(opts);
const rateLimiterMiddleware = (req, res, next) => {
// Consume 1 point for each request
rateLimiter.consume(req.connection.remoteAddress)
.then(() => {
next();
})
.catch((rejRes) => {
res.status(429).send('Too Many Requests');
});
};
app.use(rateLimiterMiddleware);
You can configure rate-limiter-flexible for any exact route. See official express docs about using middlwares
There are also options for Cluster or distributed apps and many others useful