Currently I am working toward splitting monolith REST api into microservices. I would like to introduce Domain Driven Design (currently learning). My biggest concern at this moment is implementing Repository.
Domain UBIQUITOUS LANGUAGE defines Profile entity (Social media profile e.g Twitter profile). I am thinking to extract Profile in to separate microservice. To query profiles i would introduce ProfileRepository.
Other microservices including API gateway have their own Profile search patterns. How should I design Repository to meet all those search patterns. Should I create find method for every query? Should I introduce some sort of dynamic query language?
In a monolith architecture I could create multiple repositories. One for each use case. In microservices architecture I would need to change microservice responsible for Profile every time other microservices needs new query.
Adding a dynamic query language will introduce another level of complexity my personal opinion is that you should avoid this unless you really, really need it - i.e. when other people are going to integrate your system a lot. I strongly agree with #plalx 's comment to your post - adding complexity always has its price and that goes both ways.
On the ubiquitous language & terminology mix - you should really avoid duplicate terms in your domain. The "Profile" concerning a social media account could be named "SocialProfile". So when a "Profile" means two different things in two different contexts: try to find a better term for one of those. Sure you can know that a 'Profile' in the context of 'API gateway' is something and at the same time it is something else in that other context however this will not do you good in the long term - try adding new people to the project and explaining how many different things you call 'a Profile'.
On your last question:
In a monolith architecture I could create multiple repositories. One for each use case. In microservices architecture I would need to change microservice responsible for Profile every time other microservices needs new query.
It really depends on your architecture - do your micro-services use the same code-base & Repository classes? Are they even in the same name-space? You could have one Repository for each micro-service to use and that is the logical approach since your different micro-services will do different things.
If you take my advice and find concrete terms for each Entity in your Domain you will not have to wonder around this. :)
And a side-note on REST APIs from my practice:
If your API is truly REST you can just use Repositories behind each endpoint. However if you find your API endpoints are not just reading & returning data (or entities) but also 'doing something' then there are 2 things to note:
- your API is most-probably not REST but plain RPC
- the objects you use behind endpoints are most-likely Services and not Repositories
Related
I am new in microservices. I am coming from monolithic background in current environment i have different kinds services for different purposes like search, file, email, notification. I have taken so many courses but in that the instructor separate each entity and make it's own database also create API for that(like separate shopping cart entity, product entity) it makes no sense, I am not getting what is real world use of microservices or how to make separate component to build it's own microservice.
Can anyone give Real Project example?
Thanks in advance
Read this and this. Also look here and here. I don't think that anyone will give a link to the real working project, so you can try this.
I am not getting what is real world use of microservices
mostly as you heard in all of those tutorials the microservices architecture leverage advantages of:
the smaller services are easy to maintain and develop
easily can scale specific services rather than the whole project(monolith). for example you scale service-1 to 4 instances that request traffic split into these 4 instance and service-2 to 2 instances and go on (load balance). and these services may distributed in to different servers and locations.
if one service failed to work it does not terminate the whole system since they are independent.
services can be reusable for other scenarios or features.
small team can works for each services and its easy to manage both project and development flow.
and also it suffer from disadvantages of
services are simple and small but all as a whole system is complex so designing part are very critical.
poor performance and it requires do some extras to improve the performance (different types of caching on different levels).
transactions are complex and its developments are time costly. imagine simple update should be projected to other services if its required and you have to consider failure and rollback strategy ( SAGA ).
how to make separate component to build it's own microservice
this is the most challenging part of microservices. you need deep study on Domain driven design DDD.
Decompose by subdomain
Decompose by Business Capabilities
Can anyone give Real Project example?
there are many projects the develop microservices with different patterns. I think you have to start your own and make your hands dirty.
I am working on a monolith system. All of it's code is in one repository (Web API and background workers). System is written in Nodejs and MongoDB (Mongoose) is used as a data store. My goal is to set a new path how project should evolve. At first I was wondering if I could move towards microservices based architecture.
Monolith architecture creates some problems:
If my background workers needs to scale. I have to deploy all the project to the server despite only using a small fraction of it.
All system must be redeployed when code changes. What if payment processor calls webhook while system is being redeployed?
Using microsevices advantages are quite obvious:
Smaller code base for individual microservice. Easier to reason about it.
Ability to select programming tools best for particular use case.
Easier to scale.
Looking at the current code I noticed that Mongoose ODM (Object Document Mapper) models are used across all the project to create, query and update models in database. As a principle of a good programming all such interactions with database should be abstracted. Business logic should not leak into other system layers. I could do that by introducing REPOSITORY pattern (Domain Driven Design). While code is still being shared across web api and it's background workers it is not a hard task to do.
If i decide to extract repositories into standalone microservices than all bunch of problems arise:
Some sort of query language must be introduced to accommodate complex search queries.
Interface must provide a way to iterate over search results (cursor based navigation) without returning all database documents over network.
Since project is in it's early stage and I am the only developer, going to microservices based architecture seems like an overkill. Maybe there are other approaches I should consider?
Extracting business logic and interaction with database into separate repository and sharing among services to avoid complex communication protocols between services?
Based on my experience with working in Microservices for last few years, it seems like an overkill in current scenario but pays off in long-term.
Based on the information stated above, my thoughts are:
Code Structure - Microservices Architecture (MSA) applying in above context means not separating DAO, Business Logic etc. rather is more on the designing system as per business functions. For example, if it is an eCommerce application, then you can shipping, cart, search as separate services, which can further be divided into smaller services. Read it more about domain-driven design here.
Deployment Unit - Keeping microservices apps as an independent deployment unit is a key principle. Hence, keep a vertical slice of the application and package them as Docker Image with Application Code, App Server (if any), Database and OS (Linux etc.)
Communication - With MSA, communication between services become a key and hence general practice is to remain with the message-oriented approach for communication (read about the reactive system and reactive programming for more insight).
PaaS Solution - There are multiple PaaS solutions available, which you can apply so that you don't need to worry about all the other aspects like container management, container orchestration, auto-scaling, configuration management, log management and monitoring etc. See following PaaS solutions:
https://www.nanoscale.io/ by TIBCO
https://fabric8.io/ - by RedHat
https://openshift.io - by RedHat
Cloud Vendor Platforms - AWS, Azure & Google Cloud all of them have specific support for Microservices App from the deployment perspective, which we can use as an alternative solution if you don't want to deploy PaaS solution in your organization.
Hope these pointers will have in understanding the overall landscape so that you can structure your architecture for future need.
I am working on a monolith system... My goal is to set a new path how project should evolve. At first I was wondering if I could move towards microservices based architecture.
In what ways do you need to evolve the project? Will it be mostly bugfixes, adding features, improving performance and/or scalability? Do you anticipate other developers collaborating in the future? Are you currently having maintenance issues? The answers to these questions (and many more) should be considered in guiding your choices.
You seem to be doing your homework around the pros and cons of a microservice architecture, so if you haven't asked yourself why you're even doing this in the first place, now would be good time to do so.
Maybe there are other approaches I should consider?
There's always the good old don't-break-what's-going ;)
I read the BFF pattern and I have a doubt, if one microservice is only for iOS and other microservice is only for Android, how must be created the entities if that two services use the same database and the same tables?
I'm trying to use the JDL-Studio and importing the model with import-idl command but I don't know if the command must run in every micro service's workspace
Edit:
For more context, I want to build a full stack application that could have a lot of concurrency from a web page, iOS and Android applications with REST calls and I don't know if correct to repeat the entities in every microservices (to have separated the API for every plataform) or add just one microservices as database layer.
Edit 2:
I found this blog talking about create jhipster applications with microservices and this guy show how the gateway have they own entities and the microservices have they own too..
now, I have more clear the real base of the microservices architecture but what if I want a microservice with the all entities and the gateway with only the UI entities? the blog show how could be this but with just one entity and I have a full model.jhl with the all entities
I wouldn't use import-idl for any of them apart from the original master back-end API application. You don't want a full back-end stack for each BFF, otherwise you'll have to maintain several applications much of what do the same thing and plus you'll need to synchronize your data between these data sources into some sort of "master". If you repoint everything to a single database and share all entities between BFF components, then it doesn't fit the microservice model.
The BFF pattern is supposed to be a thin facade in front of an existing service API that filters and perhaps calls multiple service APIs when necessary to aggregate stuff to suit each client type. I see this pattern more of a convenience band-aid when you have no control over the existing API, or a (temporary) step in incremental service decomposition. Ideally microservices should not have such synchronous dependencies, and I'm not a huge fan of horizontal decomposition.
In my opinion there are better ways of implementing "BFF" functionality if developing from scratch without the complicated architecture and added latency of adding yet another layer of indirection. Microservice architecture is often compared to UNIX commands. The same UNIX command is capable of supplying more detailed information when desired to suit different needs. Compare the output of ls with ls -l for example. Such a strategy can be applied to single microservice endpoints as well.
I would like to use DDD\CQRS\ES in my project. I just started using it so I don’t have a lot of experience. I am aware of existing core domain, supporting domain and CRUD domain. My core domain have references to CRUD data. For example There are a lot of business rules in Order, but it contain basic information about Delivering Company, receiving point etc.
These informations are managed by admin. In my opinion I should use crud approach but what happen when I need to rollback events?
System is going to store mix of data – restored from event source and actual from crud part of app. I will end up with inconsistent data. For example, The order may be maintained by not existing company ( That company was deleted by admin when their have delivered package, but after rollback order is still active)
In every project is part of CRUD data so how do you deal with this problem?
Should I store company events?
Additionally, When I’m adding new order I should send company name and id via event because when Im rebuilding my ReadStore , there may not be company in database so I can’t get companyName from repository
PS. Do you know any CRUD framework to handle with simples CRUD operations?
PS.2 Do you know any example opensource project which containt CRUD part of app in ES\CQRS approach?
Ok. Maybe I described it too complicated. I just want to know:
how to implement the simplest part of application ( poor business
logic - mainly CRUD operations) when I store my core domain in
event store and I would like to be able to revert previous state of
my core domain
Should I store crud operation in event store too or not?
How provide data consistent after rollback event store?
Which CRUD framework do you recommend for Java applications?
I think this is a common problem when dealing with DDD and especially when dealing with ES. This might sound simple but what you need to look for are the bounded contexts in your domain. Turns out that this bounded contexts map very well onto the services from SOA. Now when you get to SOA you realise that not all the services ( service as the S in SOA not as in web/windows service ) need to be implemented the same.
In my experience you are always going to have some services dealing mostly with CRUD operations and very little business logic, usually consumed by apps that are used by admins/special users. This services can and should be implemented with the least effort possible, without CQRS and ES and without over complicating them. Just make sure they publish relevant messages when something happens.
I strongly recommend Eric Evans - what i've learned about DDD since the book video.
Also Udi Dahan has some very very good videos about SOA and what SOA means: Avoid a failed SOA and Udi on SOA
I realise this is not actually answering all the details of your question, but i hope it will point you in the right direction.
Why not simply treat the CRUD part as append only (maybe allow update if you must) or copy enough data into the event such that important data isn't lost?
I don't what exactly your trying to ask, but if it concerns DDD and CRUD operations, the Cocktail framework might help you although i haven't tried it my self.
I curently have an application writen in php using the symfony framework. Rather than have seperate installs for customer on a hosted server, I would like to move to as SaaS model with one install for all customers posible running of google code or another cloud based service. I am not tied to PHP though i would like to have the benifits of a good framework.
So the chalenge: If all customers are using the same application we then have fin a way isolating each customers data. Customers do for eample have admin access and can manager their own users and privlages. At a simplistic leve you could just have a organisation identifier in each table take and add that to all database operations. However most application framewors use and ORM of some kind, and I have not been able to find one that will easly / seemlesly facinatate this at a leve the has minimum impact on the application code.
Has anyone looked at this, are there any good aproaches to this problem?
As Itay says, a multi-tenant system is a common requirement. A while back I was doing some research on this problem and came across a pretty good presentation on the different ways to handle this issue, and the pros and cons of each: http://aac2009.confreaks.com/06-feb-2009-14-30-writing-multi-tenant-applications-in-rails-guy-naor.html
This particular presentation is targeted to a Rails audience, but the principles are the same as with any language.
The approach you described is common, and PHP (One of the strengths) will allow you to comparatively easily go into the ORM code and modify it to your needs.
Second approach is to create a separate DB for each organization and a joint DB for shared resources.
A bit of a design challenge (but just a bit).
if you are really big, then you will even need to consider a separate DB server for each organization (I would say this is a serious overkill in 99.99999% of the cases).
This MSDN article gives you a very good overview of Data Architecture in Multi-tenancy: http://msdn.microsoft.com/en-us/library/aa479086.aspx