I'm developing a chat application that consists of several micro-services or nodes. And to handle database I use Sequelize ORM.
However, I've two separate nodes one for handling socket messages and another one is a general API server. Both of them have to use Sequelize.
How can I use Sequelize in both? I don't want the same code to copy paste in two different services.
What we have done in our project, is to connect socket server and API server. Thus API server is a special socket client for the socket server. Thus if we need to do some db query we pass data to-fro between them.
Related
In a microservice architecture, I have 2 separate services with different mongoDb databases. Teacher-Service and Student-service.
Now I'm trying to create a login function, once a user submits an email I'd love to query both databases to determine if the user is either a teacher or a student.
How do I implement this in Node.js using the express framework.
I guess you can make HTTP requests to both services (synchronous communication).
For e.g. GET http://teachers.service.com/api/users?email=<input-email> and GET http://students.service.com/api/users?email=<input-email> with some auth header with mind of course :).
Or another way to communicate through services is using asynchronous communication transporters like RabbitMQ, Kafka, NATS and so on. You can search for their documentations to get ground up.
I have several clients. Each client has multiple data acquisition stations. Each station has multiple sensors. I need to store this data on a MongoDB server (using mongoose and Node.js). So, I thought about the organization that follows.
Each client has its own database inside the MongoDB server.
Within this database, each station has its own collection.
Data is sent to the MongoDB server via an MQTT broker (Node.js). So, depending on which client the broker receives the message from, I need to create a new connection to the MongoDB server (using mongoose.createConnection).
I'm not sure if this is the best alternative. I don't know if creating multiple different connections will slow down the system.
What do you think?
I currently have a Node server running that works with MongoDB. It handles some HTTP requests, but it largely used WebSockets. Basically, the server connects multiple users to rooms with WebSockets.
My server currently has around 12k WebSockets open and it's almost crippling my single threaded server, and now I'm not sure how to convert it over.
The server holds HashMap variables for the connected users and rooms. When a user does an action, the server often references those HashMap variables. So, I'm not sure how to use clusters in this. I thought maybe creating a thread for every WebSocket message, but I'm not sure if this is the right approach, and it would not be able to access the HashMaps for the other users
Does anyone have any ideas on what to do?
Thank you.
You can look at the socket.io-redis adapter for architectural ideas or you can just decide to use socket.io and the Redis adapter.
They move the equivalent of your hashmap to a separate process redis in-memory database so all clustered processes can get access to it.
The socket.io-redis adapter also supports higher-level functions so that you can emit to every socket in a room with one call and the adapter finds where everyone in the room is connected, contacts that specific cluster server, and has it send the message to them.
I thought maybe creating a thread for every WebSocket message, but I'm not sure if this is the right approach, and it would not be able to access the HashMaps for the other users
Threads in node.js are not lightweight things (each has its own V8 instance) so you will not want a nodejs thread for every WebSocket connection. You could group a certain number of WebSocket connections on a web worker, but at that point, it is likely easier to use clustering because nodejs will handle the distribution across the clusters for you automatically whereas you'll have to do that yourself for your own web worker pool.
I am planning to design the chat(web socket) application for my eCommerce django REST application.
What is the best design
Integrate the chat inside application inside the same django REST server
Deploy chat as a separate end point for chat only.
If put different server and db for chat. how we communicate with rest sever to validate.? and whats the best design. ?
As per the discussion in Chat.
Here is the architecture:
Your Chat app will run separately and will have no impact on your main app.
I suggest using a different Database for Chat (MongoDB recommended but you can create separate MySQL DB too) to not put Chat burden on main DB.
Keep the authentications on REST server and not allow Chat server to access main DB. Create an endpoint when you need it.
You can move Chat to a different server too in future, if you need to.
For these types of real-time application, you should be using websocket.
Since, you are using django, I would recommend you to try django-channels
I'm building a web application that will allow team collaboration. That is, a user within a team will be able to edit shared data, and their edits should be pushed to other connected team members.
Are Socket.io rooms a reasonable way of achieving this?
i.e. (roughly speaking):
All connected team members will join the same room (dynamically created upon first team member connecting).
Any edits received by the
server will be broadcast to the room (in addition to being persisted,
etc).
On the client-side, any edits received will be used to update
the shared data displayed in the browser accordingly.
Obviously it will need to somehow handle simultaneous updates to the same data.
Does this seem like a reasonable approach?
Might I need to consider something more robust, such as having a Redis database to hold the shared data during an editing session (with it being 'flushed' to the persistant DB at regular intervals)?
All you need is Socket.IO (with RedisStore) and Express.js. With Socket.IO you can setup rooms and also limit the access per room to only users who are auth.
Using Redis you can make your app scale outside a process.
Useful links for you to read:
Handling Socket.IO, Express and sessions
Scaling Socket.IO
How to reuse redis connection in socket.io?
socket.io chat with private rooms
How to handle user and socket pairs with node.js + redis
Node.js, multi-threading and Socket.io