I am looking at developing a solution with Dialogflow, where the User Interface events (e.g. Typed user input) are going to arrive as http REST events.
The problem with this is that the Dialogflow Client API (Note: This is NOT THE SAME THING as the fulfilment API), looks like it is stateful: https://github.com/googleapis/nodejs-dialogflow
(... But I could be wrong on that...)
And stateful is hard in most (all?) modern serverless paradigms that any sane person is probably using. Mainly - There is no gaurantee that subsequent events will arrive at the same run-time instance.
It's a bit infuriating that although many examples are supplied at the repo above, none of them look like realistic (but small) applications, handling multi-session, multi-turn (even 2-3 turns) user input.
Is the idea that any run-time instance could/would call new dialogflow.SessionsClient(); just once PER RUNTIME initialisation (NOT per session) to initialise the API, and then everything else is done via like await sessionClient.detectIntent(request); ???
Now that I've typed this, I am pretty confident that is right. But if anyone could confirm it would be great!
The underlying Dialogflow Session API itself is stateless - any statefull decisions that are made must be included in your call to detectIntent. The JavaScript library reflects this stateless scenario as well.
The SessionsClient constructor sets up the connection and authentication information, but does not maintain any other conversational state. So you can use the same object to manage multiple conversations.
Note that this still means you need to maintain conversational state (the session id, the values and lifespans returned in contexts, etc) in order to pass it to detectIntent as part of the request.
Related
Let's assume a monolithic web service. The architectural design is based on the DDD and divides the domain into sub-domains. These are structured according to the CQRS pattern. If a request arrives via the presentation layer, in this case a RESTful interface, the controller generates a corresponding command and executes it via the command bus.
So far so good. The problem now is that certain commands may only be executed by certain users, so access control must take place. In a three-layer architecture, this can be solved relatively easily using ABAC. However, to be able to use ABAC, the domain resource must be loaded and known. The authentication is taken over by the respective technology of the presentation layer and the authenticated user is stored in the request. For example, JWT Bearer Tokens using Passport.js for the RESTful interface.
Approach #1: Access Control as part of the command handler
Since the command handler has access to the repository and the aggregate has to be loaded from the database anyway in order to execute the command, my first approach was to transfer access control to the command handler based on ABAC. In case of a successful execution it returns nothing as usual, if access is denied an exception is thrown.
// change-last-name.handler.ts
async execute(command: ChangeUserLastNameCommand): Promise<void> {
const user = await this._userRepository.findByUsername(command.username);
if (!user) {
throw new DataNotFoundException('Resource not found');
}
const authenticatedUser = null; // Get authenticated user from somewhere
// Handle ABAC
if(authenticatedUser.username !== user.username) {
throw new Error();
}
user.changeLastName(command.lastName);
await this._userRepository.save(user);
user.commit();
}
However, this approach seems very unclean to me. Problem: In my opinion, access control shouldn't be the responsibility of a single command handler, should it? Especially since it is difficult to get the authenticated user or the request, containing the authenticated user, at this level. The command handler should work for all possible technologies of the presentation layer (e.g. gRPC, RESTful, WebSockets, etc.).
Approach #2: Access control as part of the presentation layer or using AOP
A clean approach for me is to take the access control from the handler and carry it out before executing the command. Either by calling it up manually in the presentation layer, for example by using an AccessControlService which implements the business security rules, or by non-invasive programming using AOP, whereby the aspect could also use the AccessControlService.
The problem here is that the presentation layer does not have any attributes of the aggregate. For ABAC, the aggregate would first have to be loaded using the query bus. An ABAC could then be carried out in the presentation layer, for example in the RESTful controller.
That's basically a good approach for me. The access control is the responsibility of the presentation layer, or if necessary an aspect (AOP), and the business logic (domain + CQRS) are decoupled. Problem: The main problem here is that redundancies can arise from the database point of view. For the ABAC, the aggregate must be preloaded via query in order to be able to decide whether the command may be executed. If the command is allowed to be executed, it can happen that it loads exactly the same aggregate from the database again, this time simply to make the change, even though the aggregate has already been loaded shortly before.
Question: Any suggestions or suggestions for improvement? I tried to find what I was looking for in the literature, which was not very informative. I came across the following Security in Domain-Driven Design by Michiel Uithol, which gives a good overview but did not answer my problems. How do I address security concerns in a CQRS architecture? Or are the redundant database access negligible and I actually already have the solution?
I would handle authentication and the overall authorization in the infrastructure, before it reaches the command handlers, because it is a separate concern.
It is also important to handle authentication separately from authorization, because there are separate concerns. It can become pretty messy if you handle authentication and authorization at the same time.
Then you I would do final authorization in the handler (if needed), for example if you have a command AddProductToCart, then I would make sure that the user who initially created the cart-aggreagate is the same as the one making the AddProductToCart command.
I am building a REST service over CQRS using EventSourcing to distribute changes to my domain across services. I have the REST service up and running, with a POST endpoint for creating the initial model and then a series of PATCH endpoints to change the model. Each end-point has a command associated with it that the client sends as a Content-Type parameter. For example, Content-Type=application/json;domain-command=create-project. I have the following end-points for creating a Project record on my task/project management service.
api.foo.com/project
Verb: POST
Command: create-project
What it does: Inserts a new model in the event store with some default values set
api.foo.com/project/{projectId}
Verb: PATCH
Command: rename-project
What it does: Inserts a project-renamed event into the event store with the new project name.
api.foo.com/project/{projectId}
Verb: PATCH
Command: reschedule-project
What it does: Inserts a project-rescheduled event into the event store with the new project due date.
api.foo.com/project/{projectId}
Verb: PATCH
Command: set-project-status
What it does: Inserts a project-status-changed event into the event store with the new project status (Active, Planning, Archived etc).
api.foo.com/project/{projectId}
Verb: DELETE
Command: delete-project
What it does: Inserts a project-deleted event into the event store
Traditionally in a REST service you would offer a PUT endpoint so the record could be replaced. I'm not sure how that works in the event-sourcing + CQRS pattern. Would I only ever use POST and PATCH verbs?
I was concerned I was to granular and that every field didn't need a command associated with it. A PUT endpoint could be used to replace pieces. My concern though was that the event store would get out of sync so I just stuck with PATCH endpoints. Is this level of granularity typical? For a model with 6 properties on it I have 5 commands to adjust the properties of the model.
This is a common question that we get a lot of the time when helping developers getting started with CQRS/ES. We need to acknowledge that applying REST in a pure way is a really bad match for DDD/CQRS since the intention of the commands are not explicitly expressed in the verbs GET/POST/PUT/PATCH/DELETE (even though you can use content-type like you did). Also the C/R-side of the system are definitely different resources in a CQRS-system which does not match up with REST.
However, to use HTTP to provide an API for a CQRS/ES system is very practical.
We usually only use POST for sending commands, to either a /commands endpoint or to endpoints with the name of the command, i.e /commands/create-project. It's all about how strict you want to be. In this case we embed the command type in the payload or as a content-type.
However, it is all a matter of what matches the tech stack better and what you choose here usually does not make or break the solution. The more important part is usually to create a good domain model and get the whole team onboard with this way of thinking.
Good luck!
One question that comes to mind is, is REST the right paradigm for CQRS at all?
One completely different way to structure this is to not have action-focused endpoints, but instead structure your REST API as a series of events that you add new events to (with POST).
Events should be immutable and append-only, so maybe a DELETE method doesn't make that much sense for mutations.
If you're going all in with CQRS (good luck, I've heard the war stories) I would be inclined to build an API that reflects that model well.
Would I only ever use POST and PATCH verbs?
Most of the time, you would use POST.
PUT, and PATCH are defined with remote authoring semantics - they are methods used to copy new representations of a resource from the client to the server. For example, the client GETs a representation of /project/12345, makes local edits, and then uses PUT to request that the server accept the client's new representation of the resource as its own.
PATCH, semantically, is a similar exchange of messages - the difference being that instead of sending the full representation of the resource, the client returns a "patch-document" that the server can apply to its copy to make the changes.
Now, technically, the PATCH documentation does put any restrictions on what a "patch-document" is. In order for PATCH to be more useful that POST, however, we need patch document formats that are general purpose and widely recognized (for instance, application/merge-patch+json or application/json-patch+json).
And that's not really the use case you have here, where you are defining command messages that are specific to your domain.
Furthermore, remote authoring semantics don't align very well with "domain modeling" (which is part of the heritage of CQRS). When we're modeling a domain, we normally give the domain model the authority to decide how to integrate new information with what the server already knows. PUT and PATCH semantics are more like what you would use to write information into an anemic data store.
On the other hand, it is okay to use POST
POST serves many useful purposes in HTTP, including the general purpose of “this action isn’t worth standardizing.” -- Fielding, 2009
It may help to recall that REST is the architectural style of the world wide web, and the only unsafe method supported by html is POST.
So replacing your PATCH commands with POST, and you're on the right path.
Fielding, 2008
I should also note that the above is not yet fully RESTful, at least how I use the term. All I have done is described the service interfaces, which is no more than any RPC. In order to make it RESTful, I would need to add hypertext to introduce and define the service, describe how to perform the mapping using forms and/or link templates, and provide code to combine the visualizations in useful ways. I could even go further and define these relationships as a standard, much like Atom has standardized a normal set of HTTP relationships with expected semantics
The same holds here - we aren't yet at "REST", but we have improved things by choosing standardized methods that are better aligned with our intended semantics.
One final note -- you should probably replace your use of DELETE with POST as well. DELETE is potentially a problem for two reasons -- the semantics aren't what you want, and the standard delete payload has no defined semantics
Expressed another way: DELETE is from the transferring documents over a network domain, not from your domain. A DELETE message sent to your resources should be understood to mean the same thing as a DELETE message sent to any other resource is understood. That's the uniform interface constraint at work: we all agree that the HTTP method tokens mean the same thing everywhere.
Relatively few resources allow the DELETE method -- its primary use is for remote authoring environments, where the user has some direction regarding its effect -- RFC 7231
As before: remote authoring semantics are not obviously a good fit for sending messages to a domain model.
This Google Cloud article API design: Understanding gRPC, OpenAPI and REST and when to use them clarifies the REST vs RPC debate. REST is more relevant for entity-centric API whereas RPC is more relevant for action-centric API (and CQRS). The most mature REST level 3 with hypermedia controls works well only for entities with simple state models.
Understand and evaluate first the benefits of REST for your case. Many APIs are REST-ish and not RESTful. OpenAPI is actually RPC mapped over and HTTP endpoints but it doesn't prevent it to be widely adopted.
I was recently interviewed by a MNC technical panel and they asked me different questions related to RestAPI , i was able to answer all but below 2 questions though i answered but not sure if those are correct answers. Can somebody answer my queries with real time examples
1) How can i secure my Rest API when somebody send request from Postman.The user provides all the correct information in the header like session id, Token etc.
My answer was: The users token sent in the header of the request should be associated with the successfully authenticated user info then only the user will be granted access if the Request either comes from Postman or application calls these API.(The panel said no to my answer)
2) How can i handle concurrency in Rest API Means if multiple users are trying to access the API at the same given time (For e.g multiple post request are coming to update data in a table) how will you make sure one request is served at one time and accordingly the values are updated as requested by different user request.
2) My answer was: In Entitiy framework we have a class called DbUpdateConcurrencyException, This class takes of handling concurrency and serves one request is served at a time.
I am not sure about my both the above answers and i did not find any specific answer on Googling also.
Your expert help is appreciated.
Thanks
1) It is not possible, requests from Postman or any other client or proxy (Burp, ZAP, etc) are indistinguishable from browser requests, if the user has appropriate credentials (like for example can observe and copy normal requests). It is not possible to authenticate the client application, only the client user.
2) It would be really bad if a web application could only serve one client at a time. Think of large traffic like Facebook. :) In many (maybe most?) stacks, each request gets its own thread (or similar) to run, and that finishes when the request-response ends. These threads are not supposed to directly communicate with each other while running. Data consistency is a requirement of the persistence technology, ie. if you are using a database for example, it must guarantee that database queries are run one after the other. Note that if an application runs multiple queries, database transactions or locks need to be used on the database level to maintain consistency. But this is not at all about client requests, it's about how you use your persistence technology to achieve consistent data. With traditional RDBMS it's mostly easy, with other persistence technologies (like for example using plaintext files for storage) it's much harder, because file operations typically don't support a facility similar to transactions (but they do support locks, which you have to manage manually).
So we were starting a new project from scratch and one of the developers suggested why have any GET API requests as POST API's are better in every which way. (At least when using a mobile client)
On further looking into this it does seem POST can do everything GET can do and it can do it better -
slightly more secure as parameters are not in URL
larger limit than GET request
So is there even a single reason to have a GET API ? (This will only be used from a mobile client so browser specific cacheing doesn't affect us)
Is there ever a need to have GET request API as POST is better in every way?
In general, yes. In your specific circumstances -- maybe no.
GET and POST are method tokens.
The request method token is the primary source of request semantics
They are a form of meta data included in the http request so that general purpose components can be aware of the request semantics and contribute constructively.
POST is, in a sense, the wildcard method - it can mean anything. But one of the consequences of this is - because the method has unconstrained semantics, general purpose components can't do anything useful other than pass the request along.
GET, however, has safe semantics (which includes idempotent semantics). Because the request is idempotent, general purpose components know that they can resend a GET request when the server returns no response (ie messages being lost on unreliable transport); general purpose components can know that representations of the resource can be pre-fetched, reducing perceived latency.
You dismissed caching as a concern earlier, but you may want to rethink that - the cache constraint is an important element that helped the web take over the world.
Reducing everything to POST reduces HTTP from an application for transferring documents over a network to dumb transport.
Using HTTP for transport isn't necessarily wrong: Simple Object Access Protocol (SOAP) works that way, as does gRPC. You still get authorization, and conditional requests; features of HTTP that you might otherwise need to roll your own.
You aren't doing REST at that point, but that's OK; not everybody has to.
That doesn’t mean that I think everyone should design their own systems according to the REST architectural style. REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them. (Fielding, 2008)
I have a SOA which makes heavy use of nonces (i.e, one-time one-use security tokens).
My app takes a nonce from a client, verifies it, then sends a new nonce back to said client as part of every reply. Also included in each reply are the results of business logic operations that executed right after the nonce was authenticated.
The nonce verification and generation are operationally coupled with the business logic, since both occur in response to every client request. However I don't want the two to be coupled in code. What's the right way to partition them in accordance with SOA principles? Is it too much to break the security and business logic into two separate services, with one calling the other as part of each reply to each client request?
Yes it makes sense to separate them. But I don't think they should have awareness of each other at all (Call each other directly).
I'll dive into a specific example and technology of how something similar is implemented.
In the web frame work Struts2 all incoming requests pass through a stack of operations(called interceptors) before arriving at a user defined object (called an action). The action then will access the business tier.
When submitting a web form there is the issue of double submission. So one way to protect against this is with a token that is sent along with the form submission. So we need to create a unique token place it as a hidden field, and then when we receive the request only process it if the token is good. This prevent users from doing something like accidentally buying something more than once.
In Struts2 there is a special server side token tag which creates the hidden field for us. So there is something that needs to be done for each form. The token interceptor if active will enforce that this value always exists and is good when receiving the form and will redirect responses that do not somewhere else.
The idea of implementing a nonces interceptor/filter that checks that the incoming nonce value is good and for responses adds the correct nonces value to the response should be completely independent of the business logic.
The example here is with html forms but adding an interceptor(or whatever you call "that which handles cross cutting concerns at the request/response level" for your appropriate technology) which adds such a value to json or xml messages should be pretty easy and likely produce the most elegant result.
The following is a link to struts2 interceptor reference (it might clarify the idea better):
http://struts.apache.org/2.2.1.1/docs/interceptors.html
The following two links are both interceptors which manage tokens:
http://struts.apache.org/2.2.1.1/docs/token-interceptor.html
http://struts.apache.org/2.2.1.1/docs/token-session-interceptor.html
I expect only the first few paragraphs of each link will be useful but something like it for your technology should be nice.
I think what you outlined above would be in keeping with SOA principles. You're keeping two distinct sets of operations separated - once service has the business logic, the other has the security logic.
This would be especially true if you have (or the potential of having) other services that would rely on nonces.