I'm working on a simple game using SignalR 2 and MVC 5. I need to track number of "deaths" and "kills" for each user. These stats need to be read/write from multiple game instances (user can have multiple concurrent sessions) across multiple servers.
Is adding fields to ApplicationUser : IdentityUser a reasonable solution? I'm planning on using the built-in authentication system because I like how easy it is to support Facebook and other OAuth providers.
How should I update these stats in an optimized manor that reduces multi-user/threads/server issues and is highly scalable? The stats themselves are simple, and probably only update once every few seconds per user, but I'd like a design that can support millions of users across multiple servers.
For example, I know I could add this code inside an MVC controller to update the stats:
var um = HttpContext.GetOwinContext().GetUserManager<ApplicationUserManager>();
var user = um.FindById(User.Identity.GetUserId());
user.Deaths++;
um.Update(user);
However, that doesn't seem very safe/transactional. If another process/connection/server is updating that user at the same time, bad things are likely.
In a pure SQL design I'd probably have a stored procedure that runs in a SQL transaction to get current counter, and increment it. Not sure how to translate that to a good SignalR design that takes advantage of all that the various API layers have to offer (OWIN, MVC, ASP.NET, etc). Ideally something I can easily add Redis to down the road, if direct SQL access becomes an issue.
Related
I would like to share my thoughts with you and try to get some advice. I would like to define my application with the best architecture as possible. Any comment would be highly appreciated. Here we go...
My technologies: NestJs(Node), neo4j/arangodb(graph DB), Nginx for proxy(Micro-services Approach).
My business case: SaaS application. Many customers with many users, one database per customer and the same code (just one instance) of our codebase.
we have a set of data models which will be same for all customer but a relation between them will differ. As per my research GraphDB is the best match for such operations. so I'm planning to create separate Instance/Database for each customer otherwise too many relations will make harder to scale.
Problem: From my point of view the problem can be seen with two different approach.
I need to allow multiple users to connect to different databases at the same time with the same code (just one installation). In Nestjs App how can I change the database configuration on each API request. Shall I save DB URI in a table, based on user/customer type it will fetch DB URI? then other concerns like does it affect on latency time, if any request failed then is there any possibility that request can fetch data from wrong DB?
How can we create sub-graphs in neo4j/arangodb so we can fetch sub-graph based on the customer.
On the other hand, I found a couple of interesting links:
https://neo4j.com/developer/multi-tenancy-worked-example/
https://www.arangodb.com/enterprise-server/oneshard/
https://dzone.com/articles/multitenant-graph-applications
Someone could provide me aditional info?
Thanks for your time
Best regards
With ArangoDB, a solution that works is:
Use a single database for all customers
Use Foxx microservices in that database to provide access to all data
Enforce a tenantId value on every call to Foxx
Use dedicated collections for each tenant in that database
Set up a web server (e.g. Node.js) in front of ArangoDB that serves data to all tenants
Only allow connections to Foxx from that front end web server
Each tenant will need a few collections, depending on your solution, try to keep that number as low as possible.
This model works pretty well, and you're able to migrate customers between instances / regions as their data is portable, because it's in collections.
I am building a platform, where there is a master DB which holds all the user data, this db is used when user tries to login and after successful login application should fetch data from respective user DB. We are using MongoDB, mongoose driver and epxpress framework. Please guide
To answer your question:
You can use createConnection() to create multiple connections to different databases.
However, unless you'll only have a handful of users & databases (maybe 10, maybe more), this will very likely not scale very well, since mongoose is not made for this kind of usage. I'd rather advise to either combine the databases into a single one or if you'll have a lot of data or security concerns, create a multi-tenant system hosting an app-instance connected to each database.
I am developing the FreeCodeCamp full stack voting application and would like to apply a microservices architecture. The user stories of the voting application are as follows:
As an authenticated user, I can keep my polls and come back later to access them.
As an authenticated user, I can share my polls with my friends.
As an authenticated user, I can see the aggregate results of my polls.
As an authenticated user, I can delete polls that I decide I don't want anymore.
As an authenticated user, I can create a poll with any number of possible items.
As an unauthenticated or authenticated user, I can see and vote on everyone's polls.
I am conceptualizing an architecture and come up with this:
The application is composed of 6 microservices:
1. UI
2. Aggregator
3. Authorization (login, logout)
4. Social Media (sharing)
5. Polls (with db)
6. Users (with db)
Curious how a developer having built microservices would break down these user stories into microservices. Thank you!
It looks like you have already done a good job spliting the application in microservices. Every one of the has its own persistence and they should communicate with a technology agnostic protocol or by asynchronous events.
I would do it pretty much like you did. Maybe Auth should be splitted into Authentication (i.e. using a stateless jwt) and Authorization (+its own database).
The Authentication would ensure that the User is who he says it is.
The Authorization would verify that a User may modify only his own polls.
Polls Module is as of now seems to be responsible for creating a polls, conducting polls, managing poll session (given that you might have more than one question to answer in your poll) and keeping track of results.
Alright, you might have just one db to keep all of these things but they have separate concerns and (if you scale this up) different biz teams managing it. So I would suggest you can split polls alongs those lines. (They might be helpful when you start scaling it up)
we are developing a big node app with express and MongoDB. We are trying to get the best performance, because we will have multiple clients (maybe 100+) running on the same server.
We were thinking in a one-to-n APP, one instance, one database and multiple clients accessing their domains.
I want to know what is the best settings for this scenario (one server, multiple clients) to performance and development
One instance, one database (clients data would be identified by a company ObjectId on the entry and clients would access a domain or subroute)
One instance, multiple tables (or databases, what is the best?)
Multiple instances, multiple tables
Any other ideas?
On the first setting, the developers will always worry about the current company and this can bring limitations to the app
On the second setting, the concern will continue but the company will not interfere on the database entries (more clean model)
On the third setting (maybe the best for development) only one company will be treated and brings a lot of possibilities, but may bring performance issues (all instances will run on a single server)
Other settings I have not thought of can be better.
Notes:
We are using the mongoose library
I have some experience with WordPress and i like the way themes and plugins are created for it. We are trying to achieve a level of performance similar to Wordpress with PHP (several Wordpress running on a server efficiently)
sorry about bad english
You don't need to manage multiple instance as you can create a company collection and in that collection you can store every single company and then you just need to create a reference of all these values in users.Please make sure that you have made unique index on company collection.It is really easy handle such scenarios in RDBMS(mysql).
And one more thing you can also run multiple mongod client on same instance by just changing the port and if you are looking for that sort of solution then you can do that as well.
Please note following things before using mongo:-
Please use mongo only if you have over TB's of data because that doesn't make any sense to use mongodb for some mb's or gb's of data.
Use of indexes is must in mongo if you want maximum performance.
Mongo stores all the indexes in main memory and if the indexes size is more then memory that it start swapping of indexes which is really costly and hence please make sure that you have different servers for your application and your db.
I still says it would be better to use RDBMS if you don't have TB's of data to deal with.
Why this approach:-
Let me give you a scenario.
You have 100 companies and with in 100 companies you have 1000 users for each of the company. i.e. you have 1L records in your user collection.Now i want to delete a single user or i want to update a user or i want to fetch a user from a single company then i don't need to traverse my complete database as i can make a index on my user collection using user-id and company id(compound index) or even i can make a simple filter query on company id.
For index please read this
https://docs.mongodb.com/manual/core/index-compound/
And btw we are not saving company id as an object instead i am saving only the value of _id from company collection.
I am developing an application utilizing the microservices development approach with the mean stack. I am running into a situation where data needs to be shared between multiple microservices. For example, let's say I have user, video, message(sending/receiving,inbox, etc.) services. Now the video and message records belong to an account record. As users create video and send /receive message there is a foreign key(userId) that has to be associated with the video and message records they create. I have scenarios where I need to display the first, middle and last name associated with each video for example. Let's now say on the front end a user is scrolling through a list of videos uploaded to the system, 50 at a time. In the worst case scenario, I could see a situation where a pull of 50 occurs where each video is tied to a unique user.
There seems to be two approaches to this issue:
One, I make an api call to the user service and get each user tied to each video in the list. This seems inefficient as it could get really chatty if I am making one call per video. In the second of the api call scenario, I would get the list of video and send a distinct list of user foreign keys to query to get each user tied to each video. This seems more efficient but still seems like I am losing performance putting everything back together to send out for display or however it needs to be manipulated.
Two, whenever a new user is created, the account service sends a message with the user information each other service needs to a fanout queue and then it is the responsibility of the individual services to add the new user to a table in it's own database thus maintaining loose coupling. The extreme downside here would be the data duplication and having to have the fanout queue to handle when updates needs to be made to ensure eventual consistency. Though, in the long run, this approach seems like it would be the most efficient from a performance perspective.
I am torn between these two approaches, as they both have their share of tradeoffs. Which approach makes the most sense to implement and why?
I'm also interested in this question.
First of all, scenario that you described is very common. Users, videos and messages definitely three different microservices. There is no issue in how you broke down system into pieces.
Secondly, there are multiple options, how to solve data sharing problem. Take a look at great article from auth0: https://auth0.com/blog/introduction-to-microservices-part-4-dependencies/
Don't restrict your design decision to those 2 options you've outlined. The hardest thing about microservices is to get your head around what a service is and how to cut your application into chunks/services that make sense to be implemented as a 'microservice'.
Just because you have those 3 entities (user, video & message) doesn't mean you have to implement 3 services. If your actual use case shows that these services (or entities) depend heavily on each other to fulfil a simple request from the front-end than that's a clear signal that your cutting was not right.
From what I see from your example I'd design 1 microservice that fulfills the request. Remember that one of the design fundamentals of a microservice is to be as independent as possible.
There's no need to over complicate services, it's not SOA.
https://martinfowler.com/articles/microservices.html -> great read!
Regards,
Lars