Now here's a really weird question that I couldn't find the answer to on the internet. Here's how I'm planning to build a project:
Controller App --> Node.js Server (probably Express) --> Some IoT Device Running Node.js Who Knows Where
So essentially, the Controller App wants to control an IoT device, but it could be anywhere. So, it communicates to a server which sits on a static IP which will keep track of where this IoT device is (could be on any network/IP/port). So the controller app will send a request to the server, and the server will tell this IoT device wherever it is to do something.
The problem is, how will this Node.js Server know where the device is?
Proposed Solution A: One way I thought of was to have a server, and share a secret string between the server and the IoT device. The server will have some 'endpoint(?)' that the IoT device can 'subscribe' to.
Proposed Solution B: The IoT device forms a WebSocket or a Sockets.io connection. Whilst this might be a better and easier solution, when you add many devices, will the server take up much more resources when it's communicating to multiple devices in real time?
So yeah, a really weird question, because here, it's really a push notification from Node.js -> Node.js, rather than what every other search result is about, for Node.js -> Some Notification Service like iOS or Google or Web Service Workers.
Thanks!
The "push" options are generally as follows:
Client polls an endpoint every once in a while to check if there's something new. Not really push, but very simple to implement. Feasability for using this implementation depends upon how "real-time" you need the push to be.
Client creates and maintains a constant connection with the server and the server can then send data over that connection at any time. This would be the webSocket or socket.io option or, in some cases SSE (server sent events) which is a version of continuous http. The client will need the ability to detect when the connection has dropped and re-establish the connection as needed. Obviously, the server needs the ability to handle a simultaneous connection (but mostly idle connection) from every device you're supporting. If the traffic is low, custom server configurations can support hundreds of thousands of connections. Typical shared hosting solutions are much more limited in this regard as they don't give you access to the whole server's resources.
Server uses some existing "push service" that is built into the client. This would work for an iOS or Android device that has a push service as part of the platform. Not available to a custom IoT device.
Third party push services or libraries. Google has Firebase Cloud Messaging which purports to be usable with IoT devices, but I'm mostly just finding examples of the IoT device initiating the event and having that event then pushed to more classic devices (phones, browsers, etc...), not from node.js server to IoT device.
Related
I have a websocket server which handles connection to some devices (from third parties so I don't control their implementation).
My system was working fine with .NET framework on an app service until I figured out that app services have a max outbound IPs of 8000 connections in my case.
I need to move towards a more scalable server which brings me some questions.
My constraints : I need to keep a constant websocket open with the devices and be able to reach them at any time to send them messages (one by one).
I started looking into Azure app service Signal R (or the Web Pub Sub which seems very similar). The code and the pricing seems to fit my needs. Using the upstream feature I could also send custom messages to my devices.
However I don't understand the scaling part, it says each unit can contain 1000 devices.
Following this question : What is a Unit in terms of Azure Signal R Service?
All my devices are going to connect to myapp.com, 5000 of them so divided in 5 units. They are going to send me messages sent to Azure functions for analysis.
But if I decide to send a message to device n° 4300 do I need to know on which Unit it is? Can I reach it if I have several units?
I couldn't find the answer on azure's docs or signalr.
I'm new to working with sockets and have a small system design question:
I have 2 separate node processes for a web app, 1 is a simulator that is constantly running and the 2nd is an api server. Both share the same MongoDB database and we have a React app running for the client, served by the api server.
I'm looking to implement socket.io for real-time notifications and so I've set up a simple connection between the api and client.
My problem is that while the simulator runs, there are some events that I also want to trigger push notifications for so my question is how to hook that into everything?
The file hierarchy is like:
app/
simulator/
api/
client/
I saw this article for communication between node processes and I currently have 3 solutions in mind:
Leave hierarchy as it is and install socket.io package inside simulator as well. I'm not sure if sockets work this way but can both simulator and api connect to the same socket?
Move simulator file into api file to fork as a child process so that the 2 processes can communicate via child/parent messaging. simulator will message api which will then emit updates through the socket to client
Leave hierarchy as is and communicate via node-ipc. Same situation as above with simulator messaging api first before api emits that to client
If 1 is possible, that seems like the best solution in my impression. It seems like extra work to add an additional layer of messaging for 2 and 3.
Leave hierarchy as it is and install socket.io package inside simulator as well. I'm not sure if sockets work this way but can both simulator and api connect to the same socket?
The client would have to create a separate socket.io connection to the simulator process. Then, the client can receive data from the API server over one connection and from the simulator over another connection. You would need two separate, independent socket.io connections from the client, one to the API server and one to the simulator. Simulator and API server cannot share the same socket unless they are in the same process.
Move simulator file into api file to fork as a child process so that the 2 processes can communicate via child/parent messaging. simulator will message api which will then emit updates through the socket to client
This is really part of a broader option that the simulator communicates with the API server and sends it data that the API server can then send to the client over the single socket.io connection that the client made to the API server.
There are lots of different ways for the simulator process to communicate with the API server.
Since it's already an API server, you can just make an API for this (probably non-public). The simulator calls an API to send data to the client. The API server receives that data and sends it to the client.
As you suggest, if the simulator is run from the API server as a child process, then you can use parent/child communication messaging built into node.js. Note, you don't have to move the simulator files into the API file at all. You can just use child_process to launch the simulator as another nodejs app from another project. You just have to know the path to that other project.
You can use any another communication mechanism you want between the simulator process and the API server process. There could be a socket.io connection between them. You could use several forms of IPC, etc...
If 1 is possible, that seems like the best solution in my impression.
Your #1 option is not possible as separate processes can't use the same socket.io connection.
It seems like extra work to add an additional layer of messaging for 2 and 3.
My options #1 and #2 are not much code in each server. You're doing interprocess communication. You should expect to use some code to enable that. But, it's not hard at all.
If the lifetime of the simulator server and the API server are always together (they have no independent uses), then I'd probably do the child process thing where the API server launches the simulator and then use parent/child messaging to communicate between them. You do NOT have to combine sources to do this.
The child_process module can run the simulator process by just knowing what directory it is located in.
Otherwise, I'd probably make a small web server on a non-public port in the API server and have the simulator just send data to that other web server. I often refer to this as a control port. It's a way of "controlling or diagnosing" the API server internals and can only be accessed from within the private network and/or with credentials. The reason I'd use a separate web server (in the same nodejs app as the API server) is to make it easy to secure so it can't be accessed from the outside world like the regular public APIs can. You just put the internal web server on a port that is not exposed to the outside world.
You should check Socket.IO docs about adapters and Emitters. This allows to connect to sockets from different node processes and scalability.
I'm coding a project which needs cloud control device operation, and want to keep information in sync.
The cloud needs to know the state of device, such as when the network is interrupted and when the network is restored.
When the network is restored, the modified information on the cloud is synchronized to device.
anyone got an idea of how my approach should be like? any tips?
I intend to add resident programs in the background at both ends to determine, but in fact, it is impossible for the cloud in the project to connect only one device, and multiple apps may run in one device, which is very tedious to do. Is there any simple component to realize this function?
I wish control information and data information to be synchronized on the cloud and device
Based on your tag, I'm assuming that you are using MQTT as a messaging protocol for your system. If so, to address your need for tracking the device-cloud connection state, MQTT specifies a feature called "Last Will and Testament".
From the MQTT 3.1.1 Standard Section 3.1.2.5:
If the Will Flag is set to 1 this indicates that, if the Connect request is accepted, a Will Message MUST be stored on the Server and associated with the Network Connection. The Will Message MUST be published when the Network Connection is subsequently closed unless the Will Message has been deleted by the Server on receipt of a DISCONNECT Packet [MQTT-3.1.2-8].
This can be leveraged to let the remote MQTT client on the cloud know when the device is connected and when it disconnects by publishing an online payload to a topic (for example) device/conn_status after a successful connection, and registering a Last Will offline message to the same topic. Now, whenever the device client goes offline, the broker will publish the offline payload on his behalf to the cloud client that can now act accordingly.
I am referring to the diagram
NodeJS is used as run time in this case and AWS Lambda is used as event notifier (updates comes from other lambda or DB).
My challenge is, the "user browser" can also be a mobile client. The "API" should acts as a service which allows client (mobile or web) to subscribe, unsubscribe, or publish data, nothing else.
Can lambda works as API that has capabilities of "pushing events notifications" to directly clients?
Is there any solution and also sample work/source code can be used as POC?
Next question is, how can I scale such architecture since it becomes stateful (requires memory to remember states of clients connections)?
Or else, how possible is it persist client connections on DB (using frameworks like websocket or socket.io)?
AWS has the SNS service to send notifications, which you can use from Lambda.
You can also directly use the relevant platform's notification system e.g for iOS, Node has an "apn" module that is used to communicate with Apple's APNS service - it's straightforward to use and can be implemented in a Lambda function.
In brief:
Your iOS app registers for APNS which responds with an APNS device token. Your app should then send this to your API / server for storage.
Your API can then send notifications to APNS, referencing any device tokens, along with the private key file you create from the Apple Developer page.
APNS will send the notifications to the registered devices.
Here is a good tutorial.
Your other queries should perhaps be separate questions.
Can lambda works as API that has capabilities of "pushing events notifications" to directly clients?
Yes! As #AndyOS mentioned, SNS is a great service that is quite literally intended to send notifications. I won't go into details here to avoid duplication of response.
Is there any solution and also sample work/source code can be used as POC?
Or else, how possible is it persist client connections on DB (using frameworks like websocket or socket.io)?
If you are looking to use websockets, I'd encourage you to take a look at IoT (https://aws.amazon.com/iot). IoT supports the MQTT protocol (http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html). This page also contains sample client-side code which might help you bootstrap your solution.
Next question is, how can I scale such architecture since it becomes stateful (requires memory to remember states of clients connections)?
You can view the service limits of IoT at http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html#limits_iot. You would need to decide if your app fits within these bounds, depending on the various metrics your app has (number of requests per second, number of concurrent connections, etc.).
I'm developing a web and mobile app using spring mvc and I'm having a problem understanding the different protocols and tools to make instant messaging (like facebook messaging in the website or whatsapp instant notifications). one functionality besides chatting in the project is for a user to send a request and waiting for the other user to respond with notifying them instantly).
However I'm lost cause some say to use GCM or FCM for the mobile and others say there other protocols like STOMP and AQMP and some others. I don't know what to see and use in my rest api so that it works for both browsers and mobiles while taking performance and other issues into consideration and how to consume these messages from client (I mean does the consumption method varies based on the chosen protocol?). should I use multiple protocols and tools based on the source of the request (i.e. if mobile and Android --> GCM or FCM, if browser STOMP for example, if iOS --> don't know what to use).
I know it looks like a general question but I really got lost specially that I don't know what are the right things to choose these days to start with.
Messaging stack consists of multiple components. One of them is message transport - used to pas messages between the server and the client. FCM/GCM/APNS in this context is the transport protocol.
GCM is deprecated in favor of FCM. FCM is going to work for Android, the latest versions of iOS, and even some browsers. However if the user disables notifications for your app, the messaging will stop working. There is also some unpredictable latency with push notifications, particularly if you would like to send high-volume messages like typing notifications.
Message format is another component of the stack. For instance, STOMP is the message format protocol. It's defined for any serial transport, i.e. can be used over FCM or TCP or websocket.
Given the questions you ask it looks completely impractical for you to write your own messaging stack. Just pick something off the shelf, like one of the million XMPP servers or a more modern one like Tinode. Google it.