The setup
For the sake of this example, let's assume that I have two classes; php and apache.
php can exist alone, because some machines (like CI/CD build slaves) only run php in CLI mode.
One of php's tasks is to manage php.ini files for various Linux distributions and use cases.
apache can exist alone, because not every web deployment runs on php.
One of apache's tasks is to manage the apache2 Service for various Linux distributions.
The issue:
In use cases where both php and apache are provisioned on a node, a changed php.ini should result in a restart of the apache2 Service, because mod_php would not pick up the change in the file otherwise.
So, following the documentation on Relationships and Ordering, I would have two choices in implementing this:
notify the apache2 Service from a php.iniFile
subscribe to a php.iniFile from the apache2 Service
This works fine when both Classes are included in a node manifest, but in situations where only one of them is included, it would yield an invalid relationship, like:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Invalid relationship: Service[apache] { subscribe => File[php.ini] }, because File[php.ini] doesn't seem to be in the catalog
The question
What kind of patterns or solutions exist to implement this type of 'flexible' relationships in Puppet? Something where you could notify or subscribe to objects that are not guaranteed to exist in the catalog (Like how include works for defining a Class multiple times)?
So far I've come up with two solutions that could work, but are both unfit (in my opinion):
By using if tagged('php') { } else { } (or if defined(Class['php']) { } else { } for that matter) I could include an apache Service object twice that subscribes to php.ini when it is included in the catalog, and doesn't when it's not. This does however create a lot of duplicate code and it contradicts the loose coupling / high cohesion principle.
By using parameterized classes I could pass knowledge about one to the other Class, changing it's behaviour based on the option provided. This does remove include flexibility/resilience however and you'd still need an if {} else {} in your class to implement the option, resulting in the same cons as the option above.
So, is there anything 'less hacky' around that I don't know about?
What kind of patterns or solutions exist to implement this type of 'flexible' relationships in Puppet? Something where you could notify or subscribe to objects that are not guaranteed to exist in the catalog (Like how include works for defining a Class multiple times)?
It's probably best to use the Roles & Profiles pattern, and have a separate profile for Apache alone and Apache + PHP, with the latter (only) setting the relationship you want. Then you know why the relationship exists for a given node (or should), and where it's needed you ensure that both components are declared. For nodes that don't want Apache, you assign PHP via an unrelated profile.
Related
I am using the Azure API for FHIR service, and I would like to generate a resource with an Id I create on my side. Looking at the docs, it looks like you are supposed to be able to do a PUT request to /, but it doesn't seem to be working. If I do a POST to / and specify my Id, it gets ignored.
In the future, I want it to be able to generate the Ids, but for the "Importing Legacy Data" phase, I want to be able to specify my own Ids to make the linking side of things easier.
Any ideas?
Update
The problem I am trying to avoid is having to push all the data to the FHIR Endpoint and then go over everything again to create the links.
If the server allows an upsert, you would do the PUT to '/[ResourceType]/[id]', not to just '/'.
For example, if you have a Patient with technical id '123', you could do:
PUT [base]/Patient/123
Make sure the Patient resource has its 'id' field set to '123' inside the resource as well. It has to match the id you put on the url.
Another option would be to construct a transaction Bundle, where you can make entries for each resource you want to upsert. You would use the PUT verb and '[ResourceType]/[id]' again for the entry.request details. After constructing the Bundle, you can send it with a POST to the base, so the server knows to process this as a transaction.
Transaction Bundles are also very useful if you have a bunch of resources that are related and reference each other, since the server is required to update all references for new resources inside the Bundle. See http://hl7.org/fhir/http.html#trules for more information about that. Note that not all servers support transactions, just like not all servers allow upserts as Lloyd mentions in his answer.
Servers are not required to allow external systems to assign ids. Many servers won't support 'upsert' as it risks id collisions. Those that do will typically only enable it in certain circumstances. E.g. all ids are required to be UUIDs, so collisions aren't possible; there's only one source and it's synchronizing with the target system, so it 'owns' the id and there can't be collisions.
Some servers may have a configuration ability to turn on 'upsert' capability if it's been determined to be safe to do.
I am building a REST service over CQRS using EventSourcing to distribute changes to my domain across services. I have the REST service up and running, with a POST endpoint for creating the initial model and then a series of PATCH endpoints to change the model. Each end-point has a command associated with it that the client sends as a Content-Type parameter. For example, Content-Type=application/json;domain-command=create-project. I have the following end-points for creating a Project record on my task/project management service.
api.foo.com/project
Verb: POST
Command: create-project
What it does: Inserts a new model in the event store with some default values set
api.foo.com/project/{projectId}
Verb: PATCH
Command: rename-project
What it does: Inserts a project-renamed event into the event store with the new project name.
api.foo.com/project/{projectId}
Verb: PATCH
Command: reschedule-project
What it does: Inserts a project-rescheduled event into the event store with the new project due date.
api.foo.com/project/{projectId}
Verb: PATCH
Command: set-project-status
What it does: Inserts a project-status-changed event into the event store with the new project status (Active, Planning, Archived etc).
api.foo.com/project/{projectId}
Verb: DELETE
Command: delete-project
What it does: Inserts a project-deleted event into the event store
Traditionally in a REST service you would offer a PUT endpoint so the record could be replaced. I'm not sure how that works in the event-sourcing + CQRS pattern. Would I only ever use POST and PATCH verbs?
I was concerned I was to granular and that every field didn't need a command associated with it. A PUT endpoint could be used to replace pieces. My concern though was that the event store would get out of sync so I just stuck with PATCH endpoints. Is this level of granularity typical? For a model with 6 properties on it I have 5 commands to adjust the properties of the model.
This is a common question that we get a lot of the time when helping developers getting started with CQRS/ES. We need to acknowledge that applying REST in a pure way is a really bad match for DDD/CQRS since the intention of the commands are not explicitly expressed in the verbs GET/POST/PUT/PATCH/DELETE (even though you can use content-type like you did). Also the C/R-side of the system are definitely different resources in a CQRS-system which does not match up with REST.
However, to use HTTP to provide an API for a CQRS/ES system is very practical.
We usually only use POST for sending commands, to either a /commands endpoint or to endpoints with the name of the command, i.e /commands/create-project. It's all about how strict you want to be. In this case we embed the command type in the payload or as a content-type.
However, it is all a matter of what matches the tech stack better and what you choose here usually does not make or break the solution. The more important part is usually to create a good domain model and get the whole team onboard with this way of thinking.
Good luck!
One question that comes to mind is, is REST the right paradigm for CQRS at all?
One completely different way to structure this is to not have action-focused endpoints, but instead structure your REST API as a series of events that you add new events to (with POST).
Events should be immutable and append-only, so maybe a DELETE method doesn't make that much sense for mutations.
If you're going all in with CQRS (good luck, I've heard the war stories) I would be inclined to build an API that reflects that model well.
Would I only ever use POST and PATCH verbs?
Most of the time, you would use POST.
PUT, and PATCH are defined with remote authoring semantics - they are methods used to copy new representations of a resource from the client to the server. For example, the client GETs a representation of /project/12345, makes local edits, and then uses PUT to request that the server accept the client's new representation of the resource as its own.
PATCH, semantically, is a similar exchange of messages - the difference being that instead of sending the full representation of the resource, the client returns a "patch-document" that the server can apply to its copy to make the changes.
Now, technically, the PATCH documentation does put any restrictions on what a "patch-document" is. In order for PATCH to be more useful that POST, however, we need patch document formats that are general purpose and widely recognized (for instance, application/merge-patch+json or application/json-patch+json).
And that's not really the use case you have here, where you are defining command messages that are specific to your domain.
Furthermore, remote authoring semantics don't align very well with "domain modeling" (which is part of the heritage of CQRS). When we're modeling a domain, we normally give the domain model the authority to decide how to integrate new information with what the server already knows. PUT and PATCH semantics are more like what you would use to write information into an anemic data store.
On the other hand, it is okay to use POST
POST serves many useful purposes in HTTP, including the general purpose of “this action isn’t worth standardizing.” -- Fielding, 2009
It may help to recall that REST is the architectural style of the world wide web, and the only unsafe method supported by html is POST.
So replacing your PATCH commands with POST, and you're on the right path.
Fielding, 2008
I should also note that the above is not yet fully RESTful, at least how I use the term. All I have done is described the service interfaces, which is no more than any RPC. In order to make it RESTful, I would need to add hypertext to introduce and define the service, describe how to perform the mapping using forms and/or link templates, and provide code to combine the visualizations in useful ways. I could even go further and define these relationships as a standard, much like Atom has standardized a normal set of HTTP relationships with expected semantics
The same holds here - we aren't yet at "REST", but we have improved things by choosing standardized methods that are better aligned with our intended semantics.
One final note -- you should probably replace your use of DELETE with POST as well. DELETE is potentially a problem for two reasons -- the semantics aren't what you want, and the standard delete payload has no defined semantics
Expressed another way: DELETE is from the transferring documents over a network domain, not from your domain. A DELETE message sent to your resources should be understood to mean the same thing as a DELETE message sent to any other resource is understood. That's the uniform interface constraint at work: we all agree that the HTTP method tokens mean the same thing everywhere.
Relatively few resources allow the DELETE method -- its primary use is for remote authoring environments, where the user has some direction regarding its effect -- RFC 7231
As before: remote authoring semantics are not obviously a good fit for sending messages to a domain model.
This Google Cloud article API design: Understanding gRPC, OpenAPI and REST and when to use them clarifies the REST vs RPC debate. REST is more relevant for entity-centric API whereas RPC is more relevant for action-centric API (and CQRS). The most mature REST level 3 with hypermedia controls works well only for entities with simple state models.
Understand and evaluate first the benefits of REST for your case. Many APIs are REST-ish and not RESTful. OpenAPI is actually RPC mapped over and HTTP endpoints but it doesn't prevent it to be widely adopted.
Recently I've been trying to make my web application use separated layers.
If I understand the concept correctly I've managed to extract:
Domain layer
This is where my core domain entities, aggregate roots, value objects reside in. I'm forcing myself to have pure domain model, meaning i do not have any service definitions here. The only thing i define here is the repositories, which is actually hidden because axon framework implements that for me automatically.
Infrastructure layer
This is where the axon implements the repository definitions for my aggregates in the domain layer
Projection layer
This is where the event handlers are implemented to project the data for the read model using MongoDB to persist it. It does not know anything other than event model (plain data classes in kotlin)
Application layer
This is where the confusion starts.
Controller layer
This is where I'm implementing the GraphQL/REST controllers, this controller layer is using the command and query model, meaning it has knowledge about the Domain Layer commands as well as the Projection Layer query model.
As I've mentioned the confusion starts with the application layer, let me explain it a bit with simplified example.
Considering I want a domain model to implement Pokemon fighting logic. I need to use PokemonAPI that would provide me data of the Pokemon names stats etc, this would be an external API i would use to get some data.
Let's say that i would have domain implemented like this:
(Keep in mind that I've stretched this implementation so it forces some issues that i have in my own domain)
Pokemon {
id: ID
}
PokemonFight {
id: ID
pokemon_1: ID
pokemon_2: ID
handle(cmd: Create) {
publish(PokemonFightCreated)
}
handle(cmd: ProvidePokemonStats) {
//providing the stats for the pokemons
publish(PokemonStatsProvided)
}
handle(cmd: Start) {
//fights only when the both pokemon stats were provided
publish(PokemonsFought)
}
The flow of data between layers would be like this.
User -> [HTTP] -> Controller -> [CommandGateway] -> (Application | Domain) -> [EventGateway] -> (Application | Domain)
Let's assume that two of pokemons are created and the use case of pokemon fight is basically that when it gets created the stats are provided and then when the stats are provided the fight automatically starts.
This use case logic can be solved by using event processor or even saga.
However as you see in the PokemonFight aggregate, there is [ProvidePokemonStats] command, which basically provides their stats, however my domain do not know how to get such data, this data is provided with the PokemonAPI.
This confuses me a bit because the use case would need to be implemented on both layers, the application (so it provides the stats using the external api) and also in the domain? the domain use case would just use purely domain concepts. But shouldn't i have one place for the use cases?
If i think about it, the only purpose saga/event processor that lives in the application layer is to provide proper data to my domain, so it can continue with it's use cases. So when external API fails, i send command to the domain and then it can decide what to do.
For example i could just put every saga / event processor in the application, so when i decide to change some automation flow i exactly know what module i need to edit and where to find it.
The other confusion is where i have multiple domains, and i want to create use case that uses many of them and connects the data between them, it immediately rings in my brain that this should be application layer that would use domain APIs to control the use case, because I don't think that i should add dependency of different domain in the core one.
TL;DR
What layer should be responsible of implementing the automated process between aggregates (can be single but you know what i mean) if the process requires some external API data.
What layer should be responsible of implementing the automated process between aggregates that live in different domains / micro services.
Thank you in advance, and I'm also sorry if what I've wrote sounds confusing or it's too much of text, however any answers about layering the DDD applications and proper locations of the components i would highly appreciate.
I will try to put it clear. If you use CQRS:
In the Write Side (commands): The application services are the command handlers. A cmd handler accesses the domain (repositories, aggreagates, etc) in order to implement a use case.
If the use case needs to access data from another bounded context (microservice), it uses an infraestructure service (via dependency injection). You define the infraestructure service interface in the application service layer, and the implementation in the infra layer. The infra then access the remote microservice via http rest for example. Or integration through events.
In the Read Side (queries): The application service is the query method (I think you call it projection), which access the database directly. There's no domain here.
Hope it helps.
I do agree your wording might be a bit vague, but a couple of things do pop up in my mind which might steer you in the right direction.
Mind you, the wording makes it so that I am not 100% sure whether this is what you're looking for. If it isn't, please comment and correct my on the answer I'll provide, so I can update it accordingly.
Now, before your actual question, I'd firstly like to point out the following.
What I am guessing you're mixing is the notion of the Messages and your Domain Model belonging to the same layer. To me personally, the Messages (aka your Commands, Events and Queries) are your public API. They are the language your application speaks, so should be freely sharable with any component and/or service within your Bounded Context.
As such, any component in your 'application layer' contained in the same Bounded Context should be allowed to be aware of this public API. The one in charge of the API will be your Domain Model, that's true, but these concepts have to be shared to be able to communicate with one another.
That said, the component which will provide the states to your aggregate can be viewed from two directions I think.
It's a component that handles a specific 'Start Pokemon Match' Command. This component has the smarts to know to firstly retrieve the states prior to being able to dispatch a Create and ProvidePokemonStats command, thus ensuring it'll consistently create a working match with the stats in it by not dispatching any of both of the external stats-retrieval API fails.
Your angle in the question is to have an Event Handling Component that reacts on the creation of a Match. From here, I'd state a short-lived saga would be in place, as you'd need to deal with the fault scenario of not being able to retrieve the stats. A regular Event Handler is likely to lean to deal with this correctly.
Regardless of the two options you select, this service will deal with messages, a.k.a. your public API. As such it's within your application and not a component others will deal with directly, ever.
When it comes to your second question, I feel the some notion still holds. Two distinct applications/microservices only more so suggests your talking about two different Bounded Contexts. Certainly then a Saga would be in place to coordinate the operations between both contexts. Note that between Bounded Contexts, you want to share consciously when it comes to the public API, as you'd ideally not expose everything to the outside world.
Hope this helps you out and if not, like I said, please comment and provide me guidance how to answer your question properly.
I need to create a folder on my local drive when "something" changes in my domain. So in DDD fashion I need to raise an event and not let my domain create the folder.
My question is what if my event fails (i.e. the creation of the folder fails)?
Do I have to re-raise another command to undo the first change which I think is called a compensating command?
Also what if the compensating command fails? Now I have a domain change but the folder does not exist.
The way you describe your proposed solution isn't really DDD; it's more CQRS (i.e. events & compensating commands) which I think is possibly over complicating the situation.
Do you really need to take a CQRS approach for this scenario which is intended for asynchronous operations? As in, what advantages are there for the folder being created in a separate transaction to the business logic being invoked and persisted? There is good reason for this approach when raising events that a query service handles, as the query service is likely to be on a separate physical machine, therefore requiring a RPC. Also, the event may require many de-normalised tables to be updated. So for performance it makes sense for this process to use the async eventing model. But for creating a local folder I'm not sure it does?
A possible approach
public class ApplicationService : IApplicationService
{
private readonly IMyAggregateRepository _myAggregateRepository;
private readonly IFolderCreationService _folderCreationService;
public ApplicationService(IMyAggregateRepository myAggregateRepository, IFolderCreationService folderCreationService)
{
_myAggregateRepository = myAggregateRepository;
_folderCreationService = folderCreationService;
}
public void SomeApplicationServiceMethod(Guid id)
{
using (IUnitOfWork unitOfWork = UnitOfWorkFactory.Create())
{
MyAggregate aggregate = _myAggregateRepository.GetById(id);
aggregate.SomeMethod();
_myAggregateRepository.Save(aggregate);
_folderCreationService.CreateFolder();
}
}
}
Here, the changes are only committed to the database once all of the code within the unit of work's using statement completes without error.
Note that it isn't a Domain Service, or Domain Entity that invokes the folder service... it's the Application Service. I prefer keeping domain services focused on pure domain logic. It's the Application Service's job to orchestrate client calls to the domain, database and any other infrastructure services such as this folder service.
If you decide that there is good reason for this to use the event model, then what you said is correct. If the folder creation failed in the event handler you would have to issue a compensating command. You would need to ensure this handler cannot fail by design (by this I mean the entity in question is always in a state where this compensating command can be executed; and the state reverted). Having good unit tests that cover all scenarios will help. If there is a flaw in the design that allows this compensating command to fail, I guess you'd have to resort to manual intervention by sending an email/notification on failure.
P.S. Although not the point of the question, I'd really recommend not creating physical folders unless there really is a good reason. In my experience it just causes a head ache for deployments/machine upgrades and things. Obviously I don't know your reasons for doing so, but I'd really recommend using a document store/database for whatever you need to store instead.
If a file system folder is vital to your domain, I would implement the creation of folders as a domain service. That way, you would let the entity handle all business rules and logic and if the creation of a folder fails, your domain is not left in an invalid state.
Pass the service as a parameter to your method handling the logic (the double dispatch pattern).
Folder service example.
IFolderService
{
CreateFolder(string path);
}
Entity example.
class MyEntity
{
public void DoWork(IFolderService folderService, ...)
{
folderService.CreateFolder(...);
// do work.
}
}
After the work is done, you could raise domain events to notify sub systems.
Is there an open source filter that I can use on Tomcat to prevent access to certain path (say /a/b/*) when the request comes from a given interface?
The use case: I have REST services deployed on Tomcat that should only be accessed by the application itself, but which for security reasons should not be accessible from the outside world. I could do this in Apache, if Apache was used as a front-end, but in this case it is not.
Not sure if there's an open source one that you can use, but writing one that checks where does a request come from is quite easy.
See getPathTranslated(), getRemoteAddr, etc.
yc