We implemented PubNub and we set UUID. Yesterday I tested it. I used only 2 iPhones, but here are 15 Daily Active Devices and around 16 Peak UUIDs. We hardcoded UUID for our 2 iPhones "iPhone4S" and "iPhone5C", but still Daily Active Devices and Peak UUIDs were showing 10+.
Can someone help us with this. We even compared with PubNub's demo app and it's the same. I don't know why is this happening, why we have Peak UUIDs showing 10+ when we hardcoded 2 values.
PubNub UUID - Create, Set, Store, Get & Reuse
For accurate active device counting and tracking, PubNub uses unique IP addresses by default. However, a more accurate way to do this is with UUIDs.
With the exception of the JavaScript SDK & iOS SDK (as of this writing), PubNub's SDKs generate a new UUID with every initialization/instantiate of the PubNub. So you can end up with a client device or server process reporting several different UUIDs each day.
Many developers will generate, locally store and reuse a UUID for the client applications but forget to do this on the server side processes that use PubNub, as well.
For more details about UUID generate/store/reuse with PubNub, please read PubNub KB article How Do I Set the UUID.
Related
https://firebase.google.com/docs/firestore/manage-data/enable-offline
How does Firestore work with offline data?
How are writes merged by many clients editing the same data offline, that then come online at the same time?
How long is offline data persisted? If my user uses my app for 5 years offline, then comes back online, will this be an issue? Do offline changes persist after device restarts?
Does query performance of offline data degrade as the data set gets larger?
Im specifically interested in the web Firestore client.
Do all the language clients implement the above in the same manner?
Thanks.
How are writes merged by many clients editing the same data offline, that then come online at the same time?
The write operations that will take place on Firebase servers, will be in the order in which that series of operations happened. The last operation (the most recent one) will be the one that will be available in the database by the time the synchronization occurs.
How long is offline data persisted? If my user uses my app for 5 years offline, then comes back online, will this be an issue?
The problem is not about how long is about how many operations do you make while the device is offline. While offline, Firestore will keep in queue all the write operations. As this queue grows, local operations and app startup will slow down. Nothing major, but over time these may add up. The major problem in this case is that the end result of this will be that the data on the server stays unmodified. Then what is the purpose of a realtime database? Firestore is really designed as an online database that came work for short to intermediate periods of being disconnected, not to stay offline for 5 years. Beside that, in 5 years it might be a problem of compatibility and not of the number of writes.
Do offline changes persist after device restarts?
The offline persistence is also called disk persistence. This type of persistence is enabled by default in Cloud Firestore and it means that recently listened data (as well as any pending writes from the app to the database) are persisted to disk. The data in this cache survives app restarts and device reboots.
Does query performance of offline data degrade as the data set gets larger?
Yes it does, like explained above.
Do all the language clients implement the above in the same manner?
No. For iOS and Android, the offline feature works fine while for web, this feature is still experimental.
We tested the fluid framework with minimal requirements.
Everything works fine in the demo application. https://github.com/microsoft/FluidExamples
questions:
How does the Fluid framework work offline?
Our backend is written in asp.net core. The server side of the fluid framework is written in node.js.
How to integrate the asp.net core backend with the fluid service to get the latest document state?
Need to write a driver for a Fluid service?
Offline.
Offline is an interesting topic for CRDTs generally. The Fluid Framework handles intermittent (short) offline scenarios well as long as all the connected clients have the metadata necessary to merge the change in. When the user makes changes, she does so with respect to a minimum sequence number (MinSeq.) If the her offline changes get added to the total order broadcast such that they are above her change's MinSeq, they will be merged with no additional handling.
MinSeq deserves a whole post, but it's the sequence number beneath which all connected clients can garbage collect the metadata necessary to merge changes. Therefore, as long as every client has that metadata, even if you were offline, the merge is easy.
For longer offline scenarios, the reconnecting client could request to lower the MinSeq (probably to the last MinSeq of the offline client.) All clients would then fetch the ops since the requested MinSeq and recreate the metadata. At this point, new changes could easily go in as we have mimicked the scenario where the new changes are above the minimum sequence number.
This could cause temporary perf & memory issues, but hasn't been a problem in reasonable experiments. I don't believe this feature is currently implemented in the Fluid Framework code base, but has been designed as above. (A PR on this topic would be very interesting!)
For even longer offline scenarios, you would probably need a three-way merge. For example: Two users open a document. User A goes on an airplane (loses internet) and writes Macbeth and user B writes Pride & Prejudice, what is the expected behavior when user A rejoins the session? These are entirely incompatible document states, so we'd need to present the users with a dialogue of some sort.
This is not implemented, but we have discussed some of the framework ergonomics of handling the three-way merge (i.e. what APIs would the app developer need.)
Running Fluid in a non NodeJS backend
Document state is managed by the clients and is opaque to the server. There's a similar discussion in this issue. The easiest way to get Fluid Framework document state in an .NET backend would be to run a JS engine and open the Fluid document. You could also run a NodeJS micro service that hosts the Fluid document. This service would have a simple API surface that lets you fetch the data you need.
My client has 1000 taxi, he want to track every taxi location and see his display. My question is how to track all taxi information using driver mobile device. I am using mongoDb for database.
I plan to solve this problem using develop a api and mobile device send their location after 10 seconds. but the problem is server is very busy that time and can not working properly using api.
I saw firebase store client information realtime . I need to know its possible for me to work as like as firebase database using mongodb.
I am using nodejs for backend development . If anyone know any way how to store real time data please help
You cannot (generally speaking) track taxi in real time, the problem is your Internet connection may be poor due to low GPS signal, have really low latency sometimes, or even be down. Instead design two independent applications:
One, which will store current GPS location inside a FIFO queue locally
Second, which will flush the queue to a remote sever
This approach will ensure you will, eventually, receive all the positions without having to worry about dropped packets and other issues that may and will occur.
Instead of the TCP connection you can consider using UDP (or better DTLS) instead which is faster, but less reliable. If reliability is a must (doubt it if it is just a taxi), then go for TCP (or better TLS). How will you send or receive the data is just a detail.
Also make sure you authenticate the device before you store any data, especially if the connection between devices is not secure.
You can use Firebase as a real time database. Have a look at this link: https://github.com/firebase/geofire
But in case you want to go with MongoDB you can use MongoDB Geospatial Queries and use socket.io to enable real time behavior.
For more details: https://docs.mongodb.com/manual/geospatial-queries/
You can use https://socket.io/ for the real-time tracking system.
It is a JavaScript library for real-time web applications.
You just need to configure MongoDB.
There are many blogs which can explain to you how to set up socket.io with MongoDB.
Some of them are...
http://blog.slatepeak.com/creating-a-real-time-chat-api-with-node-express-socket-io-and-mongodb/
https://blog.feathersjs.com/building-a-rest-and-real-time-api-with-express-feathers-and-mongodb-12071e5417e1
I think you are going to implement the tracking in frontend
But that is not good way and not secure, because drivers can send the fake request real time
You can use websocket method to implement the checking real time of taxies
Please check this link
I guess this way is similar with your idea
I hope this way is good for you
Thanks
I reviewed documentation about receive updates of getstream but only way is using javascript client. https://getstream.io/docs/#realtime
Is there any way for implementing real-time update on android?
There are a lot of Android libraries for websockets, but several considerations to keep in mind if you choose to develop this in your app:
keeping a data connection open to Stream will use a lot more data on mobile plans
open data connections will keep wifi/cell radios engaged, draining more battery
each websocket connection can only watch a single feed -- if you have multiple feeds to watch then you'll use up more connections per mobile client
you have a limited number of websocket connections for real-time notifications (free plans only get 250 connections, for example)
We typically recommend having a back-end system that watches all of these feeds periodically, and doing a push notification to the mobile client instead of using the websocket connections. This gives you a lot more flexibility on how often to watch for changes and minimize your users' data and battery consumption.
Our SDKs, which would be part of your back-end application anyway, would generate the token for the client. You can check our SDKs' README files for examples, such as https://github.com/GetStream/stream-python (search for read-only token on the page).
There must be a solution to this already but i'm having an issue finding it.
We have data stored in table storage and we are syncing it with an offline capable client web app over a restful api (Web API).
We are using a high watermark(currently a date time) to make sure we only download the data which has changed/added.
e.g. clients/get?watermark=2013-12-16 10:00
The problem we are facing with this approach is what happens in the edge case where multiple servers are inserting data whilst a get happens. There is a possibility that data could be inserted with a timestamp lower than the client's timestamp.
Should we worry about this or can someone recommend a better way of doing this?
I believe our main issue is inserting the data into the store. At this point there is no way to guarantee the timestamp used or the Azure box has the correct time against the other azure boxes.
Are you able to insert data into queues when inserting data into table storage? If you are able to do so, you can build off a sync that monitors the queue and inserts data based upon what's in the queue. This will allow you to not worry about timestamps and date-sync issues.
Will also make your table storage scanning faster, as you'll be able to go direct to table storage by Partition/Row keys that would presumably be in the queue messages
Edited to provide further information:
I re-read your question and realized you're looking to sync with many client applications and not necessary with a single premise-sync system which I assumed originally.
In this case, I'm slightly tweaking my suggestion:
Consider using Service Bus and publishing messages to a Service Bus Topic, everytime you change/insert Azure Table Story (ATS) entity. This message could contain an individual PartitionKey/RowKey or perhaps some other meta information as to which ATS entities have been changed.
Your individual disconnectable clients would subscribe to the Service Bus Topic through an individual Service Bus Topic Subscription and be able to pull and handle individual service bus messages and sync whatever ATS entities described in those messages.
This way you'll not really care about last-modified timestamps of your entities and only care about handling pulling messages from the service bus topic. If your client pulls all of the messages from a topic and synchronizes all of the entities that those messages describe, it has synchronized itself, regardless of the number of workers that are inserting data into ATS and timestamps with which they insert those entities.
When you're working in a disconnected/distributed environment is hard to keep things in sync based on actual time (for this to work correctly the time needs to be in sync between all actors).
Instead you should try looking at logical clocks (like a vector clock). You'll find plenty of Java examples but if you're planning to do this in .NET the examples are pretty limited.
On the other hand you might want to take a look at how the Sync Framework handles synchronization.