I have a User database that houses all of the user information plus permissions to applications, etc. If I have a general database as described and then other databases for each Web Application, can I link up databases to make Relationships between the two databases using Fluent API or Code First? There are not so elegant ways to do this, but I wanted to ask the question first before getting involved with a custom solution.
For example: 1 DbContext, DbSets for each table in the 2 databases. Ability to relate entities between databases with Fluent API.
Thanks in advance.
The answer is no. The context is related to a single database. There is even no easy way to hack this because the context still can create queries only for a single database so if you want to have access to multiple databases you need either a context for each database (no cross context queries or relations exists) or you need to expose all tables from other databases as views or aliases in the database used by the context.
Related
I made a CRM app using NestJs with Nodejs. I designed it in a way that each team has its own database because every teams data is difference and has no relation with other teams and also it made the process of back up much easier.
However, Now that I want to deploy my service I noticed that for each team I must create a separate nodejs Instance which makes ram usage very high. Imagine just for 10 teams I may need around ~500MB ram which will hurt me economically even in short run.
Solutions
I used TypeORM in NestJs so the first thought I had was to find a way to have multiple databases (not multiple connections) having them sharing same schema but dynamicly use one of them based on request's scope and details. Which seems the best solution so I can avoid creating another NodeJs instance and in same time I now have seperate database for each team.
I read nestJs and TypeORM documents but didn't found any way to accomplish that. So my other solution was to just use one database for everone and add something like team_id column to each table to make a filter data for each team.
Is it a good way?
Is there any other solutions to use one nestJs instance but with same schema for multiple databases?
I recommend to use one database.
The database can have a table saving all of the teams and other tables will have a new team_id column as you think.
One database for each team has disadvantages.
Multiple DB Connections
Since you need to use same Entities for all of the databases for the teams, you cannot use Single Database Connection. According to every incoming API request, the server will have to switch db connections.
DB Configuration in TypeORM
For multiple databases, the configuration will be looking like below:
imports: [
...,
TypeOrmModule.forRoot({
name
type
host
port
username
password
...
}),
TypeOrmModule.forRoot({
name
type
host
port
username
password
}),
...
]
If you need to add a new team, you have to update your code base for adding a new db for the team and have to redeploy your application. (maybe you will create a new database and perform migration too?)
Backup
I agree with you that it's better to backup a single team with multiple databases. But how about when you want to backup all teams? In most of cases, I believe it will need to backup all teams, not just a specific team.
Teams Management
Where do you save a team's information? How to know what team has what db?
Maybe you saved teams somewhere(in a separated db?). To know which database connection should be used in each request, it needs to make a new query?
Cost
If there are 100 teams, you are gonna make 100 databases? Also each application has development and production environment. In some cases, there can be more environments like staging. 2 envs will double the number of dbs.
Conclusion
Of course there will be a way to automate some of the items in the above list and it's still possible to use multipe databases in NestJS + TypeORM for your project but it looks not a good way and not a worth effort for your project.
I have seen some big multi-tenant applications (like grafana) and they weren't using multiple databases strategy.
I don't know how you are storing users, but since you are speaking about teams I suppose you have a place where users are stored and assigned to a team, could it be a table in a login common database?
A solution could be to bind each team to it's own database; once a user login (accessing data from common login database) you read the team which it belongs and the database for its data, then you can access CRM data from the database bound to the team the user belongs.
I ran into the microservices architecture for e-commerce application where each table has it's own micro service basically with CRUD operations (something like rest client for each table).
Now I am thinking about combine and model them around business domains, before that I wanted to know does anyone encountered such situation and is it right architecture or not.
Any suggestions will be very helpful.
Thanks.
Each microservice should have its own set of SQL tables that no other microservice can access. But having one microservice per SQL table, and having each microservice just support CRUD operations is generally an anti-pattern: it turns a powerful DBMS and query language into a simple record manager: no cross-table transactions, joins, filtering, sorting, pagination, etc.
You're mixing up different, unrelated things.
(micro)services are logical entities that do some specific task. they communicate with other services to perform a larger-scope task.
Tables/CRUD/SQL/NO-SQL come from an entirety different level. its where data is saved and how its accessed.
Its true that services use SQL and have tables. Its also probably a good idea to have separate tables for each service. I would even go as far as saying that if 2 services directly use the same table you're probably looking at a design problem.
but you can't equate services with tables, conceptually, they belong in different worlds.
Microservices are logical block for any application , combining them at sql level dosen't make any sense.
For eg: let's consider you create an order service , which allow customer to place order.
Now a order contain order items as well and may have a reference of customer object , for all these you might end up creating multiple tables. So don't just think sql table and microservices together
If you still have doubts post a more exact question , will help :)
I am new to Netsuite, and am wondering if there is ever the situation where either the user, or a 3rd party developer/consultant requires direct access to the backend database (where the application data is stored)?
Or is the database, hosted by NetSuite, hidden away from the user, and only accessible via a set of services in front on it?
I come from a background (of other ERP solutions) where consultants and users are quite used to working with the solution at database level (be it reporting off it, adding custom views, tables, stored procedures etc.), so I would like to understand how this works with NetSuite.
Questions are:
Does a user/consultant have direct read/write access to the
database?
Can a user/consultant make changes to the database in
terms of the schema?
Many thanks
For #1, the answer is No, you do not have direct access to DB. You can use the SuiteScript APIs/ SuiteTalk APIs to read/write from the Database.
For #2, You can extend Standard NetSuite Record Types (DB Schema Objects) by defining your own custom fields (Schema Object Attributes), you can also define an entire new custom record with its own custom fields. This can be done through the NetSuite's User Interface.
For #1:
You can sort of get DB access if you pony up for their ODBC access option. I think they call it SuiteAnalytics Connections now. I didn't find it particularly useful overall as it left me with even more questions than I started with.
For #1, the answer is No, you do not have direct access to DB.
You can use the SuiteScript APIs/ SuiteTalk APIs to read/write from the Database but it is very very inefficient,slow and very hard.
There are word limit for your code, it is really very in-effective
I'm creating a social site using mean stack and I need some suggestions regarding mongoDB and mongoose.
I'm part of a startup and we decided to use these amazing technologies to fulfil our task.
Basically, I need some suggestions.
Currently, I have finished creation of simple CRUD and implemented local passport JS. I have currently one single collection in my mongoDB called users.
Our social site will have a blog, marketplace and many other pages (features) that will be related to a single user.
Since I never worked with mongoDB before, I'm curious if mongoDB should use one collection per user or have multiple collections for each feature.
To clarify it, let's say I use User model for user registration, blog model for blogs etc etc.
This would really mean a lot to me if you would shortly explain me how to structure my mongoose models, if all data should be inside one collection or if one user should have separate collections for different features. And if you recommend multiple collections, how do I then link these collections together and make sure that all data is saved for one user etc.
Thanks a lot in advance!
I will explain partitioning/dividing into two level.
Of course, you're going to create different collections for different models. Such as Users, Blogs, Messages etc.
Now comes the 2nd part, if we are talking about millions of data. How you partition them for faster data lookup.
For example, you have 1M users, which you are going to put in one big collection of 'Users'. But if you look for a user whose first name is 'Imdad' and age is 28, Now your query looks through these 1M items in your single Users collection, which will take a good amount of time.
To solve this problem, users collection can be divided into multiple collections through horizontal partition (Users1 (age between 10-20), Users2 (age between 20-30), Users3 (age between 30-40)). Now based on your query predicate monggoDB is to look up into a different collection/s. This is the idea that MongoDB has applied like other SQL DB. You don't have to explicitly execute your query to the chunk collection but the mongoDB itself take care of that.
Shard key generation
Mongoose shard key
If you are using mongoDB as a backend for a REST interface, the best practice is to create on collection per resource. For example, if you intend on having a /api/users endpoint, you should have users collection and it should contain any and everything you intend to return on that endpoint.
If you are using node to compile server-side templates, structure can be more flexible. In this case, the above still applies (as you will probably eventually want to expose a REST service), but there is more flexibility. In fact, if a many-to-many style relationship is appropriate, it is easier to separate these collections and load them together in the same page.
As an aside, you mention having users and a marketplace. The bigger issue than the separation of data into collections is the use of transactions. Any time you intend on performing a transaction of data, it should be performed within a SQL transaction. There are no notions of transactions in MongoDB. This is by design, as MongoDB is designed to be a fast, scaleable data store. It is not unreasonable to amalgamate SQL and noSQL data, in this case.
Currently I'm in the process of evaluating CouchDB for a new project.
Key constraint for this project is strong privacy. There need to be resources that are readable by exactly two users.
One usecase may be something similar to Direct Messages (DMs) on Twitter. Another usecase would be User / SuperUser access level.
I currently don't have any ideas about how to solve these kind of problems with CouchDB other than creating one Database that is accessable only by these 2 users. I wonder how I would then build views aggregating data from several databases?
Do you have any hints / suggestions for me?
I've asked this question several times on couchdb mailing lists, and never got an answer.
There are a number of things that couchdb is missing.
One of them is the document level security which would :
allow only certain users to view a doc
filter the documents indexed in a view on a user level permission base
I don't think that there is a solution to the permission considerations with the current couchdb implementation.
One solution would be to use an external indexing tool like lucene, and tag your documents with user rights, then issue a lucene query with user right definition in order to get the docs. It also implies extra load on your server(s) (lucene requires a JVM) and an extra delay for the data to be available (lucene indexing time ... )
As for the several databases solution, there are language framework implementations that simply don't allow to use more then one databases ( for instance couch_potato for Ruby ).
Having several databases also means that you'll have several replication processes if your databases are replicated.
Also, this means that the views will be updated for each of the database. In some cases this is better then have huge views indexed in a single database, but it also means that distinct users might not be up to date for a single source of information ( i.e some will have their views updated, other won't). So you cannot guarantee that the data is consistent for all users.
So unless something is implemented in the couch core in order to manage document level authorizations, CouchDB does not seem appropriate for managing data with privacy constraints.
There are a bunch of details missing about what you are trying to accomplish, what the data looks like, so it's hard to make a specific recommendation. You may be able to create a database per user and copy items into each users database (for the DM use case you described). Each user would only be able to access their own database, and then you could have an admin user that could access all databases. If you need to later update those records copying them to multiple databases might not be a good idea, and then you might consider whether you want to control permissions at a different level from storage.
For views that aggregate data from several databases, I recommend looking at lounge and bigcouch, which take different approaches.
http://tilgovi.github.com/couchdb-lounge/
http://support.cloudant.com/faqs/views/chained-mapreduce-views