How to keep information synchronized between the cloud and the device - python-3.x

I'm coding a project which needs cloud control device operation, and want to keep information in sync.
The cloud needs to know the state of device, such as when the network is interrupted and when the network is restored.
When the network is restored, the modified information on the cloud is synchronized to device.
anyone got an idea of how my approach should be like? any tips?
I intend to add resident programs in the background at both ends to determine, but in fact, it is impossible for the cloud in the project to connect only one device, and multiple apps may run in one device, which is very tedious to do. Is there any simple component to realize this function?
I wish control information and data information to be synchronized on the cloud and device

Based on your tag, I'm assuming that you are using MQTT as a messaging protocol for your system. If so, to address your need for tracking the device-cloud connection state, MQTT specifies a feature called "Last Will and Testament".
From the MQTT 3.1.1 Standard Section 3.1.2.5:
If the Will Flag is set to 1 this indicates that, if the Connect request is accepted, a Will Message MUST be stored on the Server and associated with the Network Connection. The Will Message MUST be published when the Network Connection is subsequently closed unless the Will Message has been deleted by the Server on receipt of a DISCONNECT Packet [MQTT-3.1.2-8].
This can be leveraged to let the remote MQTT client on the cloud know when the device is connected and when it disconnects by publishing an online payload to a topic (for example) device/conn_status after a successful connection, and registering a Last Will offline message to the same topic. Now, whenever the device client goes offline, the broker will publish the offline payload on his behalf to the cloud client that can now act accordingly.

Related

How would you push a 'message' to a device using Node.js?

Now here's a really weird question that I couldn't find the answer to on the internet. Here's how I'm planning to build a project:
Controller App --> Node.js Server (probably Express) --> Some IoT Device Running Node.js Who Knows Where
So essentially, the Controller App wants to control an IoT device, but it could be anywhere. So, it communicates to a server which sits on a static IP which will keep track of where this IoT device is (could be on any network/IP/port). So the controller app will send a request to the server, and the server will tell this IoT device wherever it is to do something.
The problem is, how will this Node.js Server know where the device is?
Proposed Solution A: One way I thought of was to have a server, and share a secret string between the server and the IoT device. The server will have some 'endpoint(?)' that the IoT device can 'subscribe' to.
Proposed Solution B: The IoT device forms a WebSocket or a Sockets.io connection. Whilst this might be a better and easier solution, when you add many devices, will the server take up much more resources when it's communicating to multiple devices in real time?
So yeah, a really weird question, because here, it's really a push notification from Node.js -> Node.js, rather than what every other search result is about, for Node.js -> Some Notification Service like iOS or Google or Web Service Workers.
Thanks!
The "push" options are generally as follows:
Client polls an endpoint every once in a while to check if there's something new. Not really push, but very simple to implement. Feasability for using this implementation depends upon how "real-time" you need the push to be.
Client creates and maintains a constant connection with the server and the server can then send data over that connection at any time. This would be the webSocket or socket.io option or, in some cases SSE (server sent events) which is a version of continuous http. The client will need the ability to detect when the connection has dropped and re-establish the connection as needed. Obviously, the server needs the ability to handle a simultaneous connection (but mostly idle connection) from every device you're supporting. If the traffic is low, custom server configurations can support hundreds of thousands of connections. Typical shared hosting solutions are much more limited in this regard as they don't give you access to the whole server's resources.
Server uses some existing "push service" that is built into the client. This would work for an iOS or Android device that has a push service as part of the platform. Not available to a custom IoT device.
Third party push services or libraries. Google has Firebase Cloud Messaging which purports to be usable with IoT devices, but I'm mostly just finding examples of the IoT device initiating the event and having that event then pushed to more classic devices (phones, browsers, etc...), not from node.js server to IoT device.

How to push information to all users from many horizontal EC2 servers?

One of the requirements of my app is that when one user makes any insert/update/delete, all users viewing a page with a list of that record type get pushed an update containing the change. The user should not be expected to repeat an API call to refresh the dozens of records that did not change, because the push should contain a short summary of the change that occurred.
I accomplished this in my small dev server using SocketIO. I can't scale this across more than one server. My target infrastructure is AWS, and I know AWS has a push notification service, but I believe it's mobile-only and not what I'm looking for. The huge number of data streams being subscribed to is the reason I haven't consider a server-less infrastructure.
I'm new to AWS and have never attempted horizontal scaling either, so please forgive me if my entire question is ignorant.
Have you taken a look at using AWS IoT MQTT messaging protocol? Each browser is a 'device' and you have javascript listening in the browser for messages published via a socket protocol. Each service pushes a message to MQTT when it has an update. There's some good POCs out there (i.e. medium.com/#jparreira/…)

How to configure MassTransit in an unreliable network environment?

I'm trying to get my head around MassTransit in combination with RabbitMQ.
The basic concepts are working in a test project, but what I need is the following:
My system will have one or more servers that react to real life events (telephony). These events wil, by means of MassTransit and RabbitMQ, translate into messages that will be picked up by one or more receivers via a separate server, set up as RabbitMQ host. So far so good.
However, I cannot assume that I always have a connection between the publisher and the host machines. Just assume that the publishing server will continue to consume the real life events, but now cannot publish it's messages.
So, the question is: Does MassTransit have some kind of mechanism to store messages locally some way until the connection is re-established?
Or should I install RabbitMQ on every publishing server as well, in order to create a local exchange? Then I have to make the exchanges synchronize themselves after a reconnect.
Probably you have to implement a store and forward policy. Instead of publishing directly your message through MassTransit and RabbitMQ, you can store the message in a persistence repository (a local database) and delegate to some other process the notification through Masstransit of the messages stored before. This approach is often referred as "Client High Availability". This does not substitute the standard HA (High Availability) on server like the one implemented by RabbitMQ. But it's a good approach to use in a distributed system (like the one you described) because it could help you a lot in scenarios of server failure (e.g. an issue on RabbitMQ server that causes some loss of messages that you still have inside the store of some client and therefore you can make it process again).

How come it is always the GATT server that exposes services?

Looking at various GATT-based profiles, it seems that services are always exposed in the GATT server rather than the GATT client. For instance, the Time Profile (TIP) has the server exposing the Current Time Service (CTS). So, if a phone is to update a heart rate monitor with the current time using TIP, the phone will be the server whereas the monitor will be the client. But, being a heart rate monitor, the Heart Rate Profile expects the monitor to be a GATT server.
So, for a monitor that takes the current time from a phone, should it be a GATT client or server? Should it be set as a client whilst time syncing with the phone and set as a server otherwise? Should a custom profile be implemented such that the CTS is exposed in the client instead?
Thanks
Generic Attribute Profile (GATT) defines how server and client communicate with
each other using Attribute Protocol for the purpose of transporting data. Client
and server roles are determined when a procedure is initiated and released when the procedure is ended. Hence, a device can act in both roles at the same time.
I would suggest you to read Bluetooth Spec. In Part G 2.2 it explains the roles and configurations.
Client—This is the device that initiates commands and requests towards the
server and can receive responses, indications and notifications sent by the
server.
Server—This is the device that accepts incoming commands and requests
from the client and sends responses, indications and notifications to a client.
Back to your question:
The Time profile enables the device to get the date, time, time zone,
and DST information and control the functions related the time.
In your case, the monitor will be the GATT client when it takes the time from a phone. However, it can be a server at the same time for another procedure (operation, request etc.) with the phone.
In short, client and server roles are not fixed to the devices. When your phone exposes the current time, it will be server. Similarly, when it gets the current time from the monitor, it will be client. no need to customize the profile. If you want your phone to get the current time from a device and expose it to another device, just implement the same profile for client and server roles to your phone.
EDIT:
According to TIP profile spec, to get the current time information, the GATT Read Characteristic Value sub-procedure shall be used with the handle of the Current Time Characteristic. Monitor as a client will read the Current Time Characteristic from the GATT Table of the server (in this case it is the phone). As soon as the monitor retrieves the value from phone, it can update its Current Time Characteristic Value, and expose it to its environment in three ways:
Notifying it to its subscribed clients (BLE notifications). If you do it in this way, you will customize the Bluetooth TIP profile since this procedure is not defined there (I had a quick look to the document and didn't see it).
Broadcasting it in the advertisement packet (Doesn't require BLE connection)
Another BLE device connects to the monitor and reads the Current Time Characteristic value. This is the recommended way if you want to use Bluetooth SIG defined TIP profile as a server.

Controlling multiple Chromecast Receivers from one Sender?

Is it possible to write a Chrome extension (or Android app) that creates multiple Senders, each connecting to a different Receiver?
In other words, I need to build an interface from which an operator can control the streams on multiple different Chromecasts in the vicinity - each will be playing a different video stream.
I understand from other posts that the chrome.cast API does not allow for this - that the Chrome extension may acts as a single Sender only? This restriction seems arbitrary - I read somewhere that someone was able to control two devices by running two different versions of Chrome, so if this restriction exists in the Chrome API, it's not due to any limitation of the underlying protocol, correct? (what then, politics?)
Is there a lower-level API (perhaps on Android?) that would permit you to create multiple Senders and connect them to different Receivers?
I've seen some apps (such as Videostream) which appear to continue to run on the Receiver after you've closed the Sender. Might it be possible to, for example, launch a Receiver app on multiple devices, one at a time, have them identify themselves and connect to a local webserver, e.g. via WebSockets, and then have my webserver send messages to those Receiver apps to ask them to change videostreams?
As a last resort, is there an open specification of the underlying protocol?
There is nothing to stop you from writing a sender app that connects to a chromecast, launches an app and then disconnects from that device while letting the chromecast continue running the app; you would need to make sure that you do not stop the receiver when it detects that there are no connected devices. Then, on the sender side, you can repeat the same process but this time connect to a second device and so on. The important thing to keep in mind is that your sender device cannot hold multiple concurrent connections to multiple devices (MediaRouter is a global instance); this means you cannot receive messages (status updates, etc) from different Cast devices except the one you are directly connected to at that time. Also, there is nothing to stop a different user to connect to one of these devices and launch a different app.
To answer your other question, the underlying protocol is not open.

Resources