Thingsboard; Data about same sensor, but from a different (multiple) gateway - sensors

I have just started using Thingsboard and made some good progress in understanding how some of the basic stuff works ( mainly sending sensor data using mqtt ). But I have come to a complete halt at one point. Here is an explanation of the problem. Thingsboard version is 1.3.1
My setup:
4x RPi (Raspberry Pi) used as gateways to gather data from bluetooth
beacons
A set of bluetooth beacons
Here's what the system is supposed to do:
Send periodic data from the gateways ( RPi ) to inform thingsboard that the gateway is alive
The above part is working fine. I have set up the gateways to connect to thingsboard using access token, and post the data ( both the attributes and telemetry ). For sending attributes I use "v1/devices/me/attributes" and for telemetry data I use "v1/devices/me/telemetry", as mentioned in the documentation.
A typical string from the gateway indicating that it is alive is
{"gwA.macid": "00:00:00:00:00:00", "gwA.timestamp": "2018-02-16T19:20+01:00"}
The above part is working well. This is just for an indication that the gateway is well and able to communicate with the Thingsboard server. All the gateways connect to thingsboard using it's own respective access token and posts the above data.
Now the problem part ( or where I got halted )
The gateways gather the data about the nearby bluetooth beacons and post it too.
The data format in which the gateways posts the beacon data is
{"bcn000001.mac": "00:00:00:00:00:00", "bcn000001.timeepoch": 1518939044}
The gateway ensures that each beacon data will have the correct prefix. For example
{"bcn000001.mac": "50:80:25:AA:BB:CC", "bcn000001.timeepoch": 1518939044}
{"bcn000002.mac": "50:80:25:RR:AA:DD", "bcn000002.timeepoch": 1518939039}
{"bcn000003.mac": "50:80:25:GG:33:EE", "bcn000003.timeepoch": 1518939020}
But the data about the same sensor can also come from another gateway. How do I show it on a single widget irrespective of which gateway the data comes from. If I choose "entity list" then either it does not work, or shows up all the 4 RPi gateways on the widget. This is not what I want. I have attached an image. I am not sure if I am able to explain this well but pls do let me know if there is any other information that I can furnish. I am adding an image to point to some things that I just mentioned. I have even separated the data out by using a prefix ( as mentioned in one of the earlier SO posts ). I have spent close to couple of days on this. This was not supposed to be difficult. At least that's what I thought when I got started.
Note: As I explore, I just learnt (realized) one more thing. I think this part was a little confusing so I thought I should inform others. When one adds a device in Thingsboard, there is an option to specify if the device "Is a Gateway". What it is trying to tell(ask?) is that whether thingsboard gateway service is installed on that device. In my case I made the mistake(?) of thinking that a device that gathers data from sensors and posts it to the server is a gateway device. I guess I'll try to install thingsboard gateway service on the raspberry pi, then see how it goes. I'll post an update once I am done but in the meantime any useful comments or suggestions - please keep em coming.
I have seen others ask very similar question on SO. Something fundamental like this shouldn't be so difficult. No point in have a hundred pages of documentation without a clear explanation of most basic steps towards setting up an IoT backend/dashboard. All I wanted to do is to display the same sensor data ( say Sensor A ) coming from "any one of the devices", to show on the same widget. If I choose single entity, the widget won't update if the data comes from other device. If I choose entity list, and turn on "Resolve as multiple entities", then I end up with the widget showing all the devices

you are using wrong API. The API you are looking for is gateway API which allows to push telemetry and other messages on behalf of other devices. Please use this API https://thingsboard.io/docs/reference/gateway-mqtt-api/ on the Raspberry Pi gateways. You can also use our Gateway project and modify it for your needs.

Related

HM-10 BLE Module - connect to other Devices

first of all: What i am trying to do is only for private interest.
I'd like to connect a AT-09/HM-10 BLE-Module with Firmware 6.01 to another device which provides also a BLE Module, which it is not based on the CC254X-Chip,
I am able to communicate with this Device using my Laptop with integrated Bluetooth, Linux and the bluepy-helper. I am also able to make a connection using the HM10 through a USB-RS232-Module and "Hterm", but after that quite Stuck in my progress.
By "reverse-engineering" the Android-Application for controlling this particular device i found a set of Commands, stored as Strings in Hex-Format. The Java-Application itself sends out the particular Command combined with a CRC16-Modbus-Value in addition with a Request (whatever it is), to a particular Service and Characteristic UUID.
I also have a Wireshark-Protocol pulled from my Android-Phone while the application was connected to the particular device, but i am unable to find the commands extracted from the .apk in this protocol.
This is where i get stuck. After making a connection and sending out the Command+CRC16-Value i get no response at all, so i am thinking that my intentions are wrong. I am also not quite sure how the HM-10-Firmware handles / maps the Service and Char-UUIDs from the destination device.
Are there probably any special AT-Commands which would fit my need?
I am absolutely not into the technical depths of Bluetooth and its communication layer at all. The only thing i know is that the HM-10 connects to a selected BLE-Device and after that it provides a Serial I/O and data flows between the endpoints.
I have no clue how and if it can handle Data flow to certain Service/Char UUIDs from the destination endpoint, althrough it seems to have built-in the GATT , l2cap-Services and so on. Surely it handles all the neccessary communication by itself, but i don´t know where i get access to the "front-end" at all.
Best regards !

Can I reach a device from any unit using Azure Service SignalR?

I have a websocket server which handles connection to some devices (from third parties so I don't control their implementation).
My system was working fine with .NET framework on an app service until I figured out that app services have a max outbound IPs of 8000 connections in my case.
I need to move towards a more scalable server which brings me some questions.
My constraints : I need to keep a constant websocket open with the devices and be able to reach them at any time to send them messages (one by one).
I started looking into Azure app service Signal R (or the Web Pub Sub which seems very similar). The code and the pricing seems to fit my needs. Using the upstream feature I could also send custom messages to my devices.
However I don't understand the scaling part, it says each unit can contain 1000 devices.
Following this question : What is a Unit in terms of Azure Signal R Service?
All my devices are going to connect to myapp.com, 5000 of them so divided in 5 units. They are going to send me messages sent to Azure functions for analysis.
But if I decide to send a message to device n° 4300 do I need to know on which Unit it is? Can I reach it if I have several units?
I couldn't find the answer on azure's docs or signalr.

How to push information to all users from many horizontal EC2 servers?

One of the requirements of my app is that when one user makes any insert/update/delete, all users viewing a page with a list of that record type get pushed an update containing the change. The user should not be expected to repeat an API call to refresh the dozens of records that did not change, because the push should contain a short summary of the change that occurred.
I accomplished this in my small dev server using SocketIO. I can't scale this across more than one server. My target infrastructure is AWS, and I know AWS has a push notification service, but I believe it's mobile-only and not what I'm looking for. The huge number of data streams being subscribed to is the reason I haven't consider a server-less infrastructure.
I'm new to AWS and have never attempted horizontal scaling either, so please forgive me if my entire question is ignorant.
Have you taken a look at using AWS IoT MQTT messaging protocol? Each browser is a 'device' and you have javascript listening in the browser for messages published via a socket protocol. Each service pushes a message to MQTT when it has an update. There's some good POCs out there (i.e. medium.com/#jparreira/…)

Developing a web app to log messages from GPS device

this is my first question here and I realize this question might be open ended, but I'm looking for specific solutions, and any solution would be accepted.
I have GPS devices which send data packets to an IP on a port, both of which I can configure. I wish to use one of Google's, Amazon's or Microsoft's offering of cloud services. I am using python. Here is an implementation I found online :-
https://github.com/rdkls/gps-tracker-server
The data is coming as packets which are not over HTTP protocol. I have considered building a network listener over a socket on Google Compute Engine, but I'm not sure if it will be able to handle simultaneous requests from 1000 devices if such a situation ever arises. The Google Cloud IoT core offering seems to fit my need perfectly, but it is in private beta right now, which means I can't use it. I think I'll need a message queue service. But most of the offerings from these three companies requires messages over HTTP. Keep in mind that I can't change how the messages are sent from the GPS devices.
The messages sent are in this format -
https://drive.google.com/file/d/0B2EklrIn3KugS2NJYWZGWlVWeGdMbjM4WHQ2TUZmYWhIRmt3/view?usp=drive_web
Format:
data is sent in (byte sized) packets directly to the IP:Port over GPRS connections, one heartbeat packet every minute and GPS details every minute from each device. It also requires teh server to eply to the messagee for acknowledgement since it's not over TCP/IP.
So basically, which service and which architecture should I use keeping scalability, reliability and cost in mind?
I think for a 1000 devices, that would send such messages every minute, total would be 43M messages. I'm not sure but I'm looking for something that'll cost me about 1000$ that is 1$ per device per month.

BLE Pairing from microcontroller

As part of my requirement, we need to have paring connection between microcontroller (Cortox M3) and Mobile or Pc. We are using Alpwise stack for our purpose. My problem is with the controller configurations, not with the mobile, because without paring i am able to work with it.
Where should i the API "BLESMP_InitiatePairing" api. Presently i am calling the api inside GAP_callback function, inside BLEGAP_EVENT_CONNECTED.
Is this a correct location to call at this API, because if i call over here, control goes to BLEEVENT_PAIRING_COMPLETE (returning SMERROR_UNKNOWREASON) then it goes into BLEEVENT_PAIRING_REQUEST then once again it goes into
BLEEVENT_PAIRING_COMPLETE (returning SMERROR_UNKNOWREASON). Whereas when i call "BLESMP_InitiatePairing" api, it returns me with success message
(You listed the core-bluetooth tag, so I'm assuming you want to use iOS.)
I don't know the specific stack you are using. However, if you are developing an LE peripheral and want to connect with an iOS device as central, it is sufficient to reply to a read / write request with an InsufficientAuthentication error code. iOS will then initiate pairing from its side.
More details on this process can be found in the Bluetooth Accessory Design Guidelines for Apple Products Section 3.9.

Resources