I am building two sets of services on a website (all written in NodeJS on the server), both are using a RESTful approach. For the sake of modularity I decided to make both services separate entities. The first service deals with the products of the site and the second specifically deals with user related functions. So the first might have functions like getProducts, deleteProduct etc... The second would have functions like isLoggedIn, register, hasAccessTo etc... The product module will make several calls to the user module to make sure that the person making the calls has the privilege to do so.
Now the reason I separated them like this, was because in the near future I foresee a separate product range opening up, but will need to use the same user system as the first (even sharing the same database). The user system will use a database that spans the entire site and all subsequent products
My question is about communication between these projects and the users project. What is the most effective way of keeping the users module separate without suffering any significant speed hits. If the product API made a call to the user API on the same server (localhost), is there a signifcant cost to this, versus building the user API into each of the subsequent projects? Is there a better way to do this through interprocess communication maybe? Is simply having the users API run as its own service an effective solution?
If you have two nodes on same server (machine) then you have not bad performance in terms of network latency because both are on localhost.
Then, nodes will be communicating using a rest api, so on the underground, you will use node js sockets. You could use unix sockets instead of http sockets because are faster BUT are worst to debug, so I recommend you don't to that (but it's ok know alternatives).
And finally, your system looks like an "actor design pattern". At first glance this design patter is a little difficult to understand but you could have a look at this if you want more info about actor model pattern:
Actor model for NodeJS https://github.com/benlau/nactor
Actor model explanation http://en.wikipedia.org/wiki/Actor_model
Related
I am developing an application where there is a dashboard for data insights.
The backend is a set of microservices written in NodeJS express framework, with MySQL backend. The pattern used is the Database-Per-Service pattern, with a message broker in between.
The problem I am facing is, that I have this dashboard that derives data from multiple backend services(Different databases altogether, some are sql, some are nosql and some from graphDB)
I want to avoid multiple queries between front end and backend for this screen. However, I want to avoid a single point of failure as well. I have come up with the following solutions.
Use an API gateway aggregator/composition that makes multiple calls to backend services on behalf of a single frontend request, and then compose all the responses together and send it to the client. However, scaling even one server would require scaling of the gateway itself. Also, it makes the gateway a single point of contact.
Create a facade service, maybe called dashboard service, that issues calls to multiple services in the backend and then composes the responses together and sends a single payload back to the server. However, this creates a synchronous dependency.
I favor approach 2. However, I have a question there as well. Since the services are written in nodeJs, is there a way to enforce time-bound SLAs for each service, and if the service doesn't respond to the facade aggregator, the client shall be returned partial, or cached data? Is there any mechanism for the same?
GraphQL has been designed for this.
You start by defining a global GraphQL schema that covers all the schemas of your microservices. Then you implement the fetchers, that will "populate" the response by querying the appropriate microservices. You can start several instances to do not have a single point of failure. You can return partial responses if you have a timeout (your answer will incluse resolver errors). GraphQL knows how to manage cache.
Honestly, it is a bit confusing at first, but once you got it, it is really simple to extend the schema and include new microservices into it.
I can’t answer on node’s technical implementation but indeed the second approach allows to model the query calls to remote services in a way that the answer is supposed to be received within some time boundary.
It depends on the way you interconnect between the services. The easiest approach is to spawn an http request from the aggregator service to the service that actually bring the data.
This http request can be set in a way that it won’t wait longer than X seconds for response. So you spawn multiple http requests to different services simultaneously and wait for response. I come from the java world, where these settings can be set at the level of http client making those connections, I’m sure node ecosystem has something similar…
If you prefer an asynchronous style of communication between the services, the situation is somewhat more complicated. In this case you can design some kind of ‘transactionId’ in the message protocol. So the requests from the aggregator service might include such a ‘transactionId’ (UUID might work) and “demand” that the answer will include just the same transactionId. Now the sends when sent the messages should wait for the response for the certain amount of time and then “quit waiting” after X seconds/milliseconds. All the responses that might come after that time will be discarded because no one is expected to handle them at the aggregator side.
BTW this “aggregator” approach also good / simple from the front end approach because it doesn’t have to deal with many requests to the backend as in the gateway approach, but only with one request. So I completely agree that the aggregator approach is better here.
In a microservice architecture, I have 2 separate services with different mongoDb databases. Teacher-Service and Student-service.
Now I'm trying to create a login function, once a user submits an email I'd love to query both databases to determine if the user is either a teacher or a student.
How do I implement this in Node.js using the express framework.
I guess you can make HTTP requests to both services (synchronous communication).
For e.g. GET http://teachers.service.com/api/users?email=<input-email> and GET http://students.service.com/api/users?email=<input-email> with some auth header with mind of course :).
Or another way to communicate through services is using asynchronous communication transporters like RabbitMQ, Kafka, NATS and so on. You can search for their documentations to get ground up.
I want to show the user exactly to the second when he can have access to a given page, othervise it will be blocked. Lets say that I receive specific date and time from the server.
I guess I could use setTimeout function but I'm sure its a bad idea.
I can use a scheduler like node cron in backend but I'd need to send a message to frontend somehow after given time has passed.
Are webSockets an option? Or is there easier way?
I want to show the user exactly to the second when he can have access to a given page
For such accuracy, indeed the WebSocket communication is the way to go. This protocol is widely used on the web for push notification in email/social services like Gmail, Facebook etc.
Regarding the backend, I would suggest you to use a more scalable approach. You could use Bull to create a scheduling service. Bull uses Redis as a store and can operate with multiple processors(Node Processes), ensuring that each task is processed only by one processor. With one word it abstracts away the complexities which arise in distributed systems.
We have a application landscape with many micro services and use a backend for front-end for the UI to aggregate data.
Would you put the aggregation logic (combine data from multiple micro services) in the domain or application layer of the Front End For Backend Application?
It feels like business logic, however, there is no persistance only data retrieval, so I am in doubt where to put it?
Backend-for-front-end implementations shouldn't need to contain any real business logic as they only serve as gateways which are focused on specific front-end clients (mobile, desktop, web, etc.).
So of course there is logic in your BFFs but it will rather be the aggregation logic. Taking care of processing the front-end client requests and managing the workflow to communicate with the involved microservices - be it via synchronous or asynchronous communication - as well as aggregating the data and serving it back to the client.
What you usually end up with are some kind of controllers handling the API requests (e.g. via REST), some services containing the aggregation logic, some kind of anemic models (i.e. without logic) representing the client requests and responses and stuff like that.
I don't know what technology stack you are on but you can have a look at some sample BFF implementations in a microservices architecture at the Microsoft powered eshopOnContainers project.
Note: if you are really sure there is business logic required in your BFF I guess you might have to consider if this should rather be part of an already existing microservice or if you even might not have discovered a new bounded context yet this logic rather fits in.
Ideally Backend for frontend should not have any business logic. It should do authentication/authorization & routing logic.
If you are willing to do lots of aggregation and lots of frontends (need different set of data) then you may use GraphQL.
I have designed a Node.JS server that allows users to log in, join a room and exchange data with other users in this room using websockets. However I am now looking for a way to make this setup scalable.
I spend all afternoon researching various loadbalancers such as nginx and haproxy, but I still can't figure out how to organise my setup.
Initally users can login and view the active rooms. No biggie to makes this part scalable. However then they can join a specific room and at that point they need to be connected to the same node.js instance as the others in the room. It's this part I have trouble figuring out.
For now my solution exists out of creating two different types of node.js instances. One generic type to handle the login and room overview request methods and one roomtype that handle a number of rooms. The generic type then keeps track of which specific instance is responsible for which room and can deliver the correct address to the user's application.
However I am not satisfied with this solution, so I am open for suggestions. I understand that this is a rather vague question, but I am not looking for a exact solution, rather hints as to how to organize everything.
udidu touched on a possible solution, but to expand, you should look at a scalable pub/sub solution; Redis, a popular data store, has pub/sub built in and I use it often to great effect.
Using Redis (or some other system) to help make sure every instance of your Node.js app receives information about who's chatting in which room removes the dependency that all users in a room are connected to the same Node.js instance.