I am having a hard time understanding RPC in terms of the implementation. Several articles that I have read on RPC, I have seen following examples related to RPC:
Example: A RPC api
GET /readStudent?studentid=123
Example: A RPC call
POST /student HTTP/1.1
HOST: api.demo.com
Content-Type: application/json
{"name": "John Doe"}
As far as I have read and understood, RPC allows a client application to directly call methods on a server application on a different machine as if it was a local object.
So, what is the above examples all about ? Why are we making api calls instead of invoking methods ?
I am assuming that in these RPC examples above, the URLs might be pointing to public methods and the method arguments are passed here in the query string or body.
And if that is the case, why can't I simply use REST then ? Why make an effort of exposing public methods(whose actual implementation must be elsewhere according to RPC principles) through HTTP api ?
I am also confused about what is the actual RPC way and which way should be preferred.
Your examples could indicate how one RPC implementation transports requests to a distinct process. But there should be a translation layer that lets a client simply call methods/functions/procedures, like readStudent(123) and createStudent("John Doe"). Often there is also a corresponding server-side layer that lets the application code implement only those methods/functions/procedures (and not the details of the JSON/HTTP or other transport). These translation (or "marshalling") layers are often machine-generated from an application-specific interface specification, to avoid tedious manual coding of the translation boilerplate. Such interface specification is written in an Interface Definition Language (IDL).
REST imposes some conventional semantics that method calls may not honor. And it does not necessarily offer a translation layer to give the illusion of application-specific method calls.
Related
We are fairly new to hex arch in my team and trying to figure out best implementation for various scenarios.
Most examples online illustrate the typical use case having command/query -> use case -> db/msg/api
One scenario that has caused a lot of debate is async response handling (e.g. response topic, http callback, polling remote endpoint)
Assumptions:
Driven ports should only be called by Application Layer / use cases
Use cases should only be called by Driving layer
In most modern frameworks, you implement a method that acts a server for consuming messages or accepting HTTP responses. e.g. #kafkaListener, #RestController, #SqsListener.
Having response handling in driving layer aligns very well with these abstractions for messaging and api.
Handlers consequently needs to call some sort of implementation. (e.g. use case)
On the other hand, Alistair Cockburn described secondary actors/driven layer as handling a conversation with another party. The argument here is that async response is part of the same conversation.
Keeping response handling in driven layer makes it easy to switch to Sync later. However this adds overhead and complexity. e.g. by having app layer as orchestrator to poll for responses
What are interpretations of this out there?
Response handling in driving side:
Response handling in driven side:
I am developing an application where there is a dashboard for data insights.
The backend is a set of microservices written in NodeJS express framework, with MySQL backend. The pattern used is the Database-Per-Service pattern, with a message broker in between.
The problem I am facing is, that I have this dashboard that derives data from multiple backend services(Different databases altogether, some are sql, some are nosql and some from graphDB)
I want to avoid multiple queries between front end and backend for this screen. However, I want to avoid a single point of failure as well. I have come up with the following solutions.
Use an API gateway aggregator/composition that makes multiple calls to backend services on behalf of a single frontend request, and then compose all the responses together and send it to the client. However, scaling even one server would require scaling of the gateway itself. Also, it makes the gateway a single point of contact.
Create a facade service, maybe called dashboard service, that issues calls to multiple services in the backend and then composes the responses together and sends a single payload back to the server. However, this creates a synchronous dependency.
I favor approach 2. However, I have a question there as well. Since the services are written in nodeJs, is there a way to enforce time-bound SLAs for each service, and if the service doesn't respond to the facade aggregator, the client shall be returned partial, or cached data? Is there any mechanism for the same?
GraphQL has been designed for this.
You start by defining a global GraphQL schema that covers all the schemas of your microservices. Then you implement the fetchers, that will "populate" the response by querying the appropriate microservices. You can start several instances to do not have a single point of failure. You can return partial responses if you have a timeout (your answer will incluse resolver errors). GraphQL knows how to manage cache.
Honestly, it is a bit confusing at first, but once you got it, it is really simple to extend the schema and include new microservices into it.
I can’t answer on node’s technical implementation but indeed the second approach allows to model the query calls to remote services in a way that the answer is supposed to be received within some time boundary.
It depends on the way you interconnect between the services. The easiest approach is to spawn an http request from the aggregator service to the service that actually bring the data.
This http request can be set in a way that it won’t wait longer than X seconds for response. So you spawn multiple http requests to different services simultaneously and wait for response. I come from the java world, where these settings can be set at the level of http client making those connections, I’m sure node ecosystem has something similar…
If you prefer an asynchronous style of communication between the services, the situation is somewhat more complicated. In this case you can design some kind of ‘transactionId’ in the message protocol. So the requests from the aggregator service might include such a ‘transactionId’ (UUID might work) and “demand” that the answer will include just the same transactionId. Now the sends when sent the messages should wait for the response for the certain amount of time and then “quit waiting” after X seconds/milliseconds. All the responses that might come after that time will be discarded because no one is expected to handle them at the aggregator side.
BTW this “aggregator” approach also good / simple from the front end approach because it doesn’t have to deal with many requests to the backend as in the gateway approach, but only with one request. So I completely agree that the aggregator approach is better here.
So I have been reading about flow based programming (FBP) in the last few days and I have also been reading J. Paul Morrison's book about it. However I feel I still can't really wrap my head around it. The general concept is that you see programming as some sort of assembly line where you have components that take some packet as input and produce some packets as output. You can connect these components and packets travel through the network. While I totally see how this can work for ETL type applications or batch processing, I have no good idea how you could handle things like synchronous request/response patterns or database transactions with it.
For example let's say I have a web server implemented as FPB. This webserver has a GET /user/{id} which should return a JSON with some information about a user. It also has a POST /user/{id} where you can update the user by sending some JSON back to the server. So here is how I would imagine this flow to be looking:
I tried to have many re-usable components instead of putting the whole logic of handling a request into a single component. So there is a HTTP server component which sends out requests to a dispatcher component which then dispatches the requests into subsequent flows. In each flow the request is parsed by a generic "Request parser" component which outputs various parts of the request into the rest of the flow.
The upper part is quite straightforward, I read the entity of the user with the given ID from DB, serialize the object to JSON and then send it back. However at this point we don't really have a reference to the HTTP request anymore, so how would I know where to send this request to?
On the lower part we have some additional complexity because I would like to write to the database in a transactional way. So first a transaction is started (in parallel the request body is parsed into some object), then the user object is retrieved from the database and merged with the inputs from the request. At the end it is written back to the database and the transaction is committed. Finally some "OK" status is responded to the caller. Here I have the additional problem that when committing the transaction I really don't know which transaction to commit. And of course when sending the response I don't know which request to send it to.
So both problems seem to have something in common - a kind of "Context" that spans over many components. On one example it is a HTTP request/response context in the other a transactional context. In regular programming, these contexts are usually handled at the thread level. Since a request runs in a single thread, the transaction and request contexts are bound to a thread-local so they can be accessed everywhere as long as everything is running in the same thread.
In flow based programming, every component runs independently and ideally on separate threads. This is actually a key thing because it allows for parallelization and effective use of multiple processors. However when that thread-local context is no longer there, how can you handle these problems in flow based programming? This would get even more complicated with proper error handling (which I left out in my example).
I figure that when you do reactive style programming where most of the processing is asynchronous and multithreaded as well you will have the same issues, so I wonder if there are patterns to handle this. Do you have real life experience with either reactive style programming or flow based programming and have some hints on how I could solve this problem?
I wrote a quick answer on Twitter - thought I would post it here as well... Apologies for double-posting!
I like substreams for this/these problem(s), where the first Information Packet in the substream provides the "context" you were talking about. This may help: https://github.com/jpaulm/javafbp-websockets... HTH!
PS This loop-style network topology is also the basis of Facebook's new "Flux" technology - see Jing Chen's presentation, in which she compares this approach with MVC: https://www.youtube.com/watch?v=nYkdrAPrdcw
Hopefully this may nudge you in the right direction. I had a similar issue where I needed to perform a synchronous operation in an asynchronous microservice architecture.
How I solved it was using the Observer pattern. I have 3 components; a http server, a callback server and a timer wheel. The http server similar to yours receives the incoming request, the callback server receives the overal result after asynchronous processing and the timer wheel that queues the original http context and reconciles the response to the http request.
When an incoming request is received, the http server creates a correlation id ,appends it to the request metadata, appends the callback server url to the request metadata and finally adds the request and the original http context together into the timer wheel. Then the http server would pass the request to the dispatcher like in your case and send messages to the relevant components for asynchronous processing.
Depending on the outcome of execution of the current processing component, it will retrieve the callback url from the metadata and send the response to the callback server.In your case there's the json serialization or the database write that would do this. The callback server will then extract the correlation Id that was appended and get the corresponding http context and write the response.
NB each timer object in the timer wheel has a timeout that's configurable, that way if the asynchronous processing delays it will timeout and return a configurable message to the http client of the corresponding http context.
I'm new in AUTOSAR, I'm working on a project and my only concern is modeling (Software Components layer), without Basic Software implementation. I'm looking for a way to specify crypto information in the model (a way to specify that a specific communication has to be treated by the Crypto Service Manager). Does someone know a way to do so? Any tips or advice would be accepted.
The principle is same as with other services, model a SwcServiceDependency that aggregates a CryptoServiceNeeds. Create RoleBasedPortAssignments to indicated which PortPrototypes shall be used to interact with the Csm.
The SWC defines a way to specify the Crypto Service Needs of an SWC. This is defined in the standard/AUTOSAR_TPS_SoftwareComponentTemplate.pdf
But the actual sighing and authentication is done in the BSW by first routing incoming SecuredIPdus by the PduR to the SecOC, which will forward authentication to the CryptoStack (Csm, Cry, CryIf, CAL / CrySHE). They'll return an (authenticated) IPdu back to the PduR, which routes it up to Com, which provides you the ISignalGroups and ISignals. To transmission is just the opposite way, where the SecOC gets an IPdu and delivers back a SecuredIPdu, which is routed by PduR down to the If to the -Driver to transmit.
On the receiving side, failed authentication will, the same as other failures usually cause the IPdu to be discared to higher layers, which looks like a message was never received.
This BasicSW parts are defined in the SystemDescription, which is defined in the standard/AUTOSAR_TPS_SystemTemplate.pdf
If to call own api for building the website is a good practice.
Which is the best way to call own api on the same server in a nodejs application?
simply calling the api-methods
using socket.io with emit() and listen it with .on('event', function(){})
install jquery on the server and use the ajax call
or not use at all the own api and rewrite the methods
i'm just confusing. Hope someone can clarify me on this.
If you need to call own API from another process it would be good to use some messaging protocol. ZeroMQ sounds like perfect fit here. It allows to create different patterns of communication between different services in internal networks, and communicate in different ways. Simplest example is Request > Response pattern that is similar to HTTP requests. And it might be a good start point.
Remember that if you using routing system within express, then ZeroMQ solution will not utilize that, it would be able directly communicate, not through HTTP interface. It is much more efficient as well, as HTTP has unnecessary overhead especially for internal communication.
If you still want to use express routing then your option would be to use http.request, which behaves very similar to curl or $.ajax. This function makes HTTP requests, so you can reuse your express routing system.