I'm trying to make a service architecture which includes two Node.js apps which shares the same database. The overall service architecture looks like below (simplified version)
I'm planning to use Sequelize as an ORM to access the database. As far as I know, if a service uses Sequelize, it needs model to get the structure of data tables. In my case, api and service will access the same database, which means they should share the same Sequelize model.
So here is the question: where should I locate the common Sequelize relevant files? It seems I have two choices:
put them on the upper common location (assuming the project structure is monorepo) so that each apps can use the single same files
maintain copies of files in each apps' project folders. In this case, each apps will be independent(Let's say I want to dockerize each apps) but in case the Sequelize files modified, the same action should be done for the other.
I'm not sure how I understood is correct. Is my question valid? If so, what is the better choice and practice? I appreciate for your answers in advance.
There is no correct answer, it depends on the specific situation, but sharing a database between multiple microservices is a bad design.
Sharing a database means tight coupling at the data level. The direct consequence is that when a service modifies the database table structure, such as deleting the name field of the user table, it may break the APIs of other services and all use the sequelize user model. All services need to update the model definition and modify the implementation code of the API.
If all of your services are maintained by a team, I suggest you choose the first solution, which costs less and is easier to maintain. If your services are maintained by different teams, the two solutions are actually similar, because as long as the table structure is modified, the application layer model needs to be modified or verified whether it still works well.
Therefore, I recommend following the best practices of microservice architecture, first splitting the database vertically according to the business model, and building application APIs on top of it.
Core principles of microservices:
loose coupling
high cohesion
Related
I read the BFF pattern and I have a doubt, if one microservice is only for iOS and other microservice is only for Android, how must be created the entities if that two services use the same database and the same tables?
I'm trying to use the JDL-Studio and importing the model with import-idl command but I don't know if the command must run in every micro service's workspace
Edit:
For more context, I want to build a full stack application that could have a lot of concurrency from a web page, iOS and Android applications with REST calls and I don't know if correct to repeat the entities in every microservices (to have separated the API for every plataform) or add just one microservices as database layer.
Edit 2:
I found this blog talking about create jhipster applications with microservices and this guy show how the gateway have they own entities and the microservices have they own too..
now, I have more clear the real base of the microservices architecture but what if I want a microservice with the all entities and the gateway with only the UI entities? the blog show how could be this but with just one entity and I have a full model.jhl with the all entities
I wouldn't use import-idl for any of them apart from the original master back-end API application. You don't want a full back-end stack for each BFF, otherwise you'll have to maintain several applications much of what do the same thing and plus you'll need to synchronize your data between these data sources into some sort of "master". If you repoint everything to a single database and share all entities between BFF components, then it doesn't fit the microservice model.
The BFF pattern is supposed to be a thin facade in front of an existing service API that filters and perhaps calls multiple service APIs when necessary to aggregate stuff to suit each client type. I see this pattern more of a convenience band-aid when you have no control over the existing API, or a (temporary) step in incremental service decomposition. Ideally microservices should not have such synchronous dependencies, and I'm not a huge fan of horizontal decomposition.
In my opinion there are better ways of implementing "BFF" functionality if developing from scratch without the complicated architecture and added latency of adding yet another layer of indirection. Microservice architecture is often compared to UNIX commands. The same UNIX command is capable of supplying more detailed information when desired to suit different needs. Compare the output of ls with ls -l for example. Such a strategy can be applied to single microservice endpoints as well.
Background
We are looking at porting a 'monolithic' 3 tier Web app to a microservices architecture. The web app displays listings to a consumer (think Craiglist).
The backend consists of a REST API that calls into a SQL DB and returns JSON for a SPA app to build a UI (there's also a mobile app). Data is written to the SQL DB via background services (ftp + worker roles). There's also some pages that allow writes by the user.
Information required:
I'm trying to figure out how (if at all), Azure Service Fabric would be a good fit for a microservices architecture in my scenario. I know the pros/cons of microservices vs monolith, but i'm trying to figure out the application of various microservice programming models to our current architecture.
Questions
Is Azure Service Fabric a good fit for this? If not, other recommendations? Currently i'm leaning towards a bunch of OWIN-based .NET web sites, split up by area/service, each hosted on their own machine and tied together by an API gateway.
Which Service Fabric programming model would i go for? Stateless services with their own backing DB? I can't see how Stateful or Actor model would help here.
If i went with Stateful services/Actor, how would i go about updating data as part of a maintenance/ad-hoc admin request? Traditionally we would simply login to the DB and update the data, and the API would return the new data - but if it's persisted in-memory/across nodes in a cluster, how would we update it? Would i have to expose this all via methods on the service? Similarly, how would I import my existing SQL data into a stateful service?
For Stateful services/actor model, how can I 'see' the data visually, with an object Explorer/UI. Our data is our Gold, and I'm concerned of the lack of control/visibility of it in the reliable services models
Basically, is there some documentation on the decision path towards which programming model to go for? I could model a "listing" as an Actor, and have millions of those - sure, but i could also have a Stateful service that stores the listing locally, and i could also have a Stateless service that fetches it from the DB. How does one decide as to which is the best approach, for a given use case?
Thanks.
What is it about your current setup that isn't meeting your requirements? What do you hope to gain from a more complex architecture?
Microservices aren't a magic bullet. You mainly get four benefits:
You can scale and distribute pieces of your overall system independently. Service Fabric has very sophisticated tools and advanced capabilities for this.
You can deploy and upgrade pieces of your overall system independently. Service Fabric again has advanced capabilities for this.
You can have a polyglot system - each service can be written in a different language/platform.
You can use conflicting dependencies - each service can have its own set of dependencies, like different framework versions.
All of this comes at a cost and introduces complexity and new ways your system can fail. For example: your fast, compile-time checked in-proc method calls now become slow (by comparison to an in-proc function call) failure-prone network calls. And these are not specific to Service Fabric, btw, this is just what happens you go from in-proc method calls to cross-machine I/O - doesn't matter what platform you use. The decision path here is a pro/con list specific to your application and your requirements.
To answer your Service Fabric questions specifically:
Which programming model do you go for? Start with stateless services with ASP.NET Core. It's going to be the simplest translation of your current architecture that doesn't require mucking around with your data layer.
Stateful has a lot of great uses, but it's not necessarily a replacement for your RDBMS. A good place to start is hot data that can be stored in simple key-value pairs, is accessed frequently and needs to be low-latency (you get local reads!), and doesn't need to be datamined. Some examples include user session state, cache data, a "snapshot" of the most recent items in a data stream (like the most recent stock quote in a stream of stock quotes).
Currently the only way to see or query your data is programmatically directly against the Reliable Collection APIs. There is no viewer or "management studio" tool. You have to write (and secure) an API in each service that can display and query data.
Finally, the actor model is a very niche model. It serves specific purposes but if you just treat it as a data store it will not work for you. Like in your example, a listing per actor probably wouldn't work because you can't query across that list, or even have multiple users reading the same listing simultaneously.
I ran into the microservices architecture for e-commerce application where each table has it's own micro service basically with CRUD operations (something like rest client for each table).
Now I am thinking about combine and model them around business domains, before that I wanted to know does anyone encountered such situation and is it right architecture or not.
Any suggestions will be very helpful.
Thanks.
Each microservice should have its own set of SQL tables that no other microservice can access. But having one microservice per SQL table, and having each microservice just support CRUD operations is generally an anti-pattern: it turns a powerful DBMS and query language into a simple record manager: no cross-table transactions, joins, filtering, sorting, pagination, etc.
You're mixing up different, unrelated things.
(micro)services are logical entities that do some specific task. they communicate with other services to perform a larger-scope task.
Tables/CRUD/SQL/NO-SQL come from an entirety different level. its where data is saved and how its accessed.
Its true that services use SQL and have tables. Its also probably a good idea to have separate tables for each service. I would even go as far as saying that if 2 services directly use the same table you're probably looking at a design problem.
but you can't equate services with tables, conceptually, they belong in different worlds.
Microservices are logical block for any application , combining them at sql level dosen't make any sense.
For eg: let's consider you create an order service , which allow customer to place order.
Now a order contain order items as well and may have a reference of customer object , for all these you might end up creating multiple tables. So don't just think sql table and microservices together
If you still have doubts post a more exact question , will help :)
If I am developing an application using DDD, where do the infrastrucure and behavior components go? For example, user management, user specific configuration, permissions, application menuing, etc.
These components really have nothing to do with the business requirements being fullfilled by my domain, but they are still required elements of my application. Many of them also require persistance.
It's pretty normal to have non-domain components along with the domain in your project - after all not everything is business domain oriented. Where they belong actually depends on how you structure your solution. In most cases I tend to follow Onion Architecture, so all of my logic is provided by Application Services, regardless if it's domain or non-domain oriented.
Well if you find that your usecases rarely demands information from your core domain joined with application specific, you can probably split that into a separate database. Access this information through Application Service layer, since this layer is suppose to serve your application needs. If that includes user profile persistence etc, that's fine.
But you remember that if you got infrastructural failure and you want to do a rollback with some transaction logs or database backups, you'd probably want all persisted data be roll-backed. So then it's easier to have these domains share a database. Pros and cons - always compromise...
If I know that this application would have minor interaction with it's environment, I would put this in one database and let the application service layer interact with clients.
If I know that there will be several applications/clients I may consider to split database so that Webb application user specifics are stored in separate database. Very hard to say, since I have no overview of all the requirements.
/Magnus
I curently have an application writen in php using the symfony framework. Rather than have seperate installs for customer on a hosted server, I would like to move to as SaaS model with one install for all customers posible running of google code or another cloud based service. I am not tied to PHP though i would like to have the benifits of a good framework.
So the chalenge: If all customers are using the same application we then have fin a way isolating each customers data. Customers do for eample have admin access and can manager their own users and privlages. At a simplistic leve you could just have a organisation identifier in each table take and add that to all database operations. However most application framewors use and ORM of some kind, and I have not been able to find one that will easly / seemlesly facinatate this at a leve the has minimum impact on the application code.
Has anyone looked at this, are there any good aproaches to this problem?
As Itay says, a multi-tenant system is a common requirement. A while back I was doing some research on this problem and came across a pretty good presentation on the different ways to handle this issue, and the pros and cons of each: http://aac2009.confreaks.com/06-feb-2009-14-30-writing-multi-tenant-applications-in-rails-guy-naor.html
This particular presentation is targeted to a Rails audience, but the principles are the same as with any language.
The approach you described is common, and PHP (One of the strengths) will allow you to comparatively easily go into the ORM code and modify it to your needs.
Second approach is to create a separate DB for each organization and a joint DB for shared resources.
A bit of a design challenge (but just a bit).
if you are really big, then you will even need to consider a separate DB server for each organization (I would say this is a serious overkill in 99.99999% of the cases).
This MSDN article gives you a very good overview of Data Architecture in Multi-tenancy: http://msdn.microsoft.com/en-us/library/aa479086.aspx