I am looking for suggestions how to best implement a semi-realtime leaderboard. I have implemented an app in React Native and for the backend I am using NestJS with MongoDB. The app uses Mongo to store several performance metrics generated by the app. In the app I want to show a top-x ranking of the performance metrics from all clients. What is the best way to notify the app that the ranking has been updated by a different client. I don’t need an instant response like a chat app, a refresh interval of lets say 10sec is sufficient. For this phase of the project there will be less than 10k clients. I prefer I can keep using the nestJS-Mongo backend.
I am fairly new to the world of app programming. I tried to find suitable solution online but want to know how this is normally done.
I am kind of confused about when an API is needed. I have recently created a mobile app with flutter and cloud firestore as the database where i simply queried and wrote to the database when needed. Now i am learning full stack web development and I recently watched a tutorial where he built like an Express API with GET, POST, and DELETE functionality for a simple item in the database.
Coming from a background where i just directly accessed the database i am not sure why an API in this case is necessary, is it so I wouldnt have to rewrite the queries every time? This is a very simple project so he's definitely not making a 3rd party api for other developers to use. Am i misunderstanding what an API does exactly?
It was really simple, there was one collection in a MongoDB database and he was using postman to read and write to and from the database to check if it works.
API is a standard way with which your front-end (web/mobile) stores/gets information for your application. Your front-end can/should not directly access database ever. Understand the purpose of front-end which is to just display the interface and should do minimal processing. All the application logic should be at your backend (API server) which is exposed to your frontend via API (GET, POST etc) calls. So to store an item in your database, you will write data storing logic in your backend, and expose an API end-point which when triggered will perform the storing operation. That API call should be used by your front-end to trigger the storing process. In this way your logic of storing/database or any other thing is not exposed, only the API URL is. The purpose of front-end is to be exposed whereas backend/database should never be exposed and used from front-end
May be for you, an API is not necessary. But, the use-cases of an API is a lot.
For example:
You don't have to write business logic for every platform. (iOS, Android, Web, Whatever)
Your app will be lightweight since some computation would be offloaded to server.
Your app can be reverse engineered to get secret informations. (or, Your secret algorithm may be?)
What if you need to store something in filesystem that you want share with others?
Also a good read: Why we should use REST?
In your case, you are using a pre-written SDK which knows how to connect to Firestore, does caching and updates application data when needed, and provides a standard method of reading, writing and deleting data in Firestore (with associated documentation and example data from google).
Therefore, using an API (as described for the mongoDB) is not required and is undesirable.
There are some cases where you might want to have no read or write access to a firestore collection or document, and in this case, you could write a cloud function which your app calls with parameters, that receives the data that you want to write and does some sort of checking or manipulation beyond the capabilities of cloud firestore rules (although these can get pretty sophisticated). See https://firebase.google.com/docs/firestore/security/get-started
Todd (in the video contained in this link) does a few good videos on this subject.
However, this is not really working in the same was as the API you mentioned in your question.
So in the case of using Firestore, you should use the SDK and not re-invent the wheel by creating your own API.
If you want to share photos for example, you can also store them in firebase storage and then provide a URL for other devices to access them without your app being installed.
If you want to write something to firestore which is then sent to all other users then you can use listeners on each app, and the data will be sent to the apps after it arrives at Firestore.
https://firebase.google.com/docs/firestore/query-data/listen gives an overview of this.
One thing to always look at with firebase is the cost of doing anything. Cloud functions cost more than doing a read of a firestore document.
This gives an overview of pricing for different capabilities within the firebase set of capabilities.
https://firebase.google.com/pricing
Another most important factor is coupling. To add to #Dijkstra API provides a way to decouple the logic from each other, thus allowing for more application reliability, maintainability, fault-tolerance and if required scalability.
Thus there is no right or wrong here, or the comparison of API vs DB call is in itself not justified for the fact that fetching the data from Database is the ultimate aim. Even if you use a REST API or Query a database.
The means to achieve the same can differ based on specific requirements. For example, fetching water from the well.
You can always climb down the well and fetch a bucket of water if you need 1 bucket per day and you are the only user.
But if there are many users you would want to install a pull and wheel where people use it to pour fetched water into their bucket, yet again this will depend if there are 100 users per day using or more than that. As this will not work in the case of more than 100 users.
IF the case is that an entire community of say 1000 user are going to need the water you would go with a more complex solution of installing a motorized water pump to pump out the water and supply it to the user's home via a pipeline. This solution has many benefits like fast supply, easy to use, filtered water, scheduled, etc. But the cost and effort to achieve the solution is higher as well.
All in all, It comes down to the cost-vs-benefit ratio which you and only you can chart out, for different solutions vs the particular problem, as you are the best judge of scale and future user flow.
While doing that you can ask the following question about the solution to help decide :
Is the solution satisfying the primary requirement of the problem?
How much time is it going to take to build it?
For the time we spend to build a solution, is it going to working at more than 75% or more of its capacity?
If not is there a simpler solution that I can use to satisfy the problem and scale it as the requirement increases?
HTH.
I have an Angular 7 project that is currently and has lots of components that talk to an API and updates data from it.
The is constantly refreshed using setTimeout so it's getting very busy with all the components refreshing data from the API.
I am therefore thinking of adding ngrx/Store to the project.
Is ngrx/Store a solution for this kind of issue or should I look for other solutions?
To make it's easy for you. I would say yes because ngrx provide API to break the application into smaller peice.
If you have multiple feature you will have 1 main store and for every feature you will have feature store connect to main store. So you can easy manage everything into feature seperation concern.
Another thing is ngrx provide effect middleware allow user to working with side effect like API call so you will have completely seprate code when working with API
I would recommend you read there small demo here for every feature they create effects, actions, reducers, selector, services.
I want to write a series of small apps for myself as micro services. This is for practicality and self learning. I want these apps to be able to work independently, but build a separate frontend client that has a search bar that can find data across all of the services. I wanted to implement a live autocomplete with search results as the user is typing that will search across multiple databases.
My current approach was to split each app into two apps, backend API, and a frontend client. Have a common auth service that all of the apps utilize for authorization.
I think this approach would work fine except for speed and performance, which I am not sure about. It is a personal requirement of mine to be able to implement this search bar with autocomplete search results. This means it will have to make API requests to each service to get those results, it just feels like it might be too slow.
Also, in case anyone is wondering, I was planning on using node, express, and mongodb for backend. Probably go with node, express, vue or something for the front end.
Q. Does anyone have personal experience with the performance aspect of working with multiple APIs?
Q. If this approach is too slow, is there a better approach that still allows for a separation of the applications?
The performance will be depending on so many other factors before/with the number of rest calls. It is really hard to say anything beforehand.
But according to my previous experience if you need to provide such a search functionality for a autocomplete feature you may most likely need elastic search for that.
That means instead of sending a request to each and every service for each autocomplete request you should have an index to search in (partial or maybe all data you need to show in your frontend) and search that index first to find the corresponding items then ask for the remaining/full data to other services if necessary. Details are totally depends on your requirements.
Of course if you go that way you should also implement a data population/syncronization mechanism to elastic search.
Just trying to spec out a new system. We want to build a mobile app with offline storage using react-native.
We were originally going to use node and create a web api layer that worked with a MongoDB. However after looking into offline storage we found PouchDB. But pouch has to use CouchDB. So we are considering using CouchDb and having a flat db layer with all our business and data logic modelled in the apps.
However there may be some server side things that we need to do (for example pulling other records from somewhere else in the organisation).
So can we deploy a web api using node alongside CouchDb? How would this work? Totally separate or in the same node process?
So can we deploy a web api using node alongside CouchDb?
Yes, of course.
How would this work?
I'm not sure about what solution suites your needs, but one solution is to develop a web app with this tech-stack: NodeJS/Express, CouchDB/PouchDB and ReactJS/MobX. One important point about CouchDB is that, you shouldn't think about it like any other database . Basically, you don't need any server-side code to talk to CouchDB, it can all be done on the client-side with PouchDB. The live sync of PouchDB is extremely convenient at handling offline/online storage. If you're considering ReactJS, take a look at MobX.