I have a simple scenario where in response to a user action:
JSP ===> Controller ===> Service ===> DAO
DAO essentially running a number of disparate queries. Is there a way for the service to call multiple DAOs or DAO methods concurrently, have each perform its one complex query and the have the service aggregate the result from each DAO?
Can use JMS, but how to return the results back to the originating service?
Same goes for spring-batch.
It appears these mechanisms are good to perform work which does not need to be 'returned' to a higher layer.
Any pointers will be greatly appreciated.
check out the #Async annotation combined with a returntype of Future.
http://static.springsource.org/spring/docs/3.0.x/reference/scheduling.html
you will still need to wait and collect all the result before returning it to the higher layer.
if this is not good enough, you might wanna look at websockets push or longpolling in the web-tier, but that makes the application more complex
Related
I'll be starting on a greenfield project in a few months.
The project will contain lot's of business logic, spread over several subdomains. Yes, we'll be using principles of Domain Driven Design.
Tech will consist of Spring, Spring Boot & Hibernate stack.
I was looking after some Java libs to cover infrastructural things like:
domain event publication
event store
event deduplication
resequencers on consumer side
projections
reliable publishing
reliable delivery & redelivery
...
I came across the Axon Framework. I already heard about it, didn't know it in details. So I read some blogposts, little bit of documentation and watched some broadcasts on Youtube.
It seems very promising, I'm considering to use it because I don't want to reinvent to wheel over and over again on the infrastructural side.
So I'm looking for someone to answer and clarify my questions:
Command handling
Axon use CommandHandlers with void methods. Is it possible to make them return a value (for instance a generated business id) or objects for notification purposes concerning the business operation? It's not a issue to me that the method will be I/O blocking by this.
Local vs remote domain events publication
I want to have a clear separation of local vs remote domain events.
Local domain events should only be visible and consumed to the local subdomain. Is it possible to configure event consumption sync and/or async?
My Local domain events can be 'fat'. They are allowed to carry more data because it won't cross the domain boundaries.
Remote domain events will be 'thin', so only the minimum data necessary for remote domains. This type op events need always to be handles async.
Is it possible to convert a local (fat) domain event to a remote (thin) domain event at the edge of a domain? By 'edge', I mean the infrastructural side.
By this, the domain model doesn't need to know distinction between local & remote domain events.
CQRS synchronously
My application will consist of 1 (maybe 2) core domains and several subdomains. Some domains contain lot's of business logic and will require CQRS.
Other domain will be more 'crudy' style.
Is it possible to do CQRS synchronously? I want to start this way before adding technical complexities like async handling. It this plossible with Axon?
This also means that domain events will be stored in a events store without using event sourcing. Can Axon's event store be used without event sourcing?
Same for projection stuff, I just want to projection domain events to build my read model.
Modular monolith
We'll use a modular monolith.
Not very trendy these days with all the microservices stuff. Although, I'm convinced of having a monolith where each domain is completely separated (application code & DB-schema), where operations will be handled with eventual consistency and domain events contain the necessary data.
Later on, and if necessary, it will be easier to migrate to a microservices architecture.
Is Axon a framework that fits in a modular monolith kind of architecture? Is there anything to take into account?
Fully separated domain model (persistence agnostic)
The domain model will be completely separated from the data model.
We need to have a repository that reads a data model (using Hibernate) and uses a data mapper to create an aggregate when it needs to be loaded.
The other way is also needed, an aggregate needs to be converted and saved into the data model (using data mapper).
Additionally, the aggregates's domain events need to be stored into an event store and published to local or remote event handlers.
This has some consequences:
we need to have full control of repository implementation that communicates with one or more DAO's (Spring data repositories) to take the necessary data out of Hibernate entities and construct an aggregate with it. An aggregate might be modeled in 2 or even 3 relational tables after all.
we don't need any Hibernate annotation in the domain model
Is this approach possible with Axon? I only see examples using direct JPA (domain model maps 1 to 1 to entities) or event sourcing.
This approach is really a deal breaker for us, a separated domain model gives so much more possibilities than mapping it directly to data entities.
Below an example of what I want to achieve:
Aggregate (without JPA) in some domain model package:
public class ScoringResultAggregate {
// members, constructor, operation omitted for brevity
}
Hibernate Entity in some infrastructure package:
#Entity
#Table(name ="SOME_TABLE_NAME)
public class ScoringResultEntity {
// member and getters & setters; no domain logic
}
Repository interface that belongs to the domain model:
public interface ScoringResultRepository {
void save(ScoringResultAggregate scoringResultAggregate);
ScoringResultAggregate findByApplicationNumber(ApplicationNumber applicationNumber);
}
Adapter that implements repository interface; responsible for mapping aggregate from/to data (JPA) model:
class ScoringResultAdapterRepository implements ScoringResultRepository {
private ScoringResultJpaRepository scoringResultJpaRepository;
ScoringResultJPARepository(ScoringResultJpaRepository scoringResultJpaRepository) {
this.scoringResultJpaRepository= scoringResultJpaRepository;
public void save(ScoringResultAggregate scoringResultAggregate) {
// converts aggregate to ScoringResultEntity and saves the state into DB
}
public ScoringResultAggregate findByApplicationNumber(ApplicationNumber applicationNumber) {
// loads an ScoringResultEntity from DB and converts it into an aggregate
}
}
Axon Server
Axon server looks very promising. Although, is it only useful for event sourcing?
Can it be used together with a Sql DB where aggregates are stored (state persistence) and domain events get persisted in Axon Server?
Lot of questions. Hopefully, someone with Axon experience can help me out :-)
I feel Jasper is saying the right things, but I also think I can emphasize them a little more:
Command handling - Yes you can have return values on command handlers. Just be mindful that you do no abuse this to return state of the to the user, as that would be mixing the Command Model (your Aggregate handling the command) with your Query Model.
Local vs remote domain events publication - Jasper states this clearly and he's right. Your hitting the desire to form bounded context's, for which Axon Server (Enterprise) has support. If you'd not use Axon, you'll have to build this infrastructure yourself.
CQRS synchronously - Axon provides handles for asynchronous and synchronous messaging just fine. The main difference is that you'll block on the result of sending your messages. The CommandGateway for example has a send and sendAndWait method, thus providing you with sync and async command dispatching. Lastly, it's perfectly fine to use Axon Server as the event store without doing Event Sourcing. Event Sourcing is an choice when using Axon, not a requirement.
Modular monolith - AxonIQ as a company actively encourages this approach to building software. So yes, you can do this, and no, I cannot think of anything you should think of prior to doing this.
Fully separated domain model (persistence agnostic) - From your Query Model you have full control over how you'd want to map your data model to and from the actual model you'd use. The Aggregate in Axon terms should be regarded as your Command Model, for which you can choose the Event Sourced storage approach or the State Stored storage approach. The state-stored implementation given by Axon Framework works based on JPA, which would thus require you to set some annotations along side the axon annotations in your Command Model. If you need to segregate this, I could imagine you'd create your own variant of the Repository and AggregateFactory. Then again, the Event Sourcing approach would make your domain model clear of persistence annotations altogether, so I'd go for that route to be honest.
Axon Server - Yes you can use Axon Server even if you go the state-stored approach for Aggregates. Know that Axon Server next to being an event store is a unified routing solution for commands, events and queries. If you would move from a modular monolith to a (micro) services set up, having Axon Server in place to perform all the message routing will make your life very, very easy.
I hope I can answer some of them, but I'm also not really experienced in using Axon:
Return values from command handler - Yes, thats possible. We had an example where we return the generated aggregate id (I'm not 100% sure about this answer)
Local vs remote domain events publication - Yes, Axon Server ENTERPRISE (!) supports multi-context thats build for this purpose. https://axoniq.io/product-overview/axon-server-enterprise
CQRS synchronously - The question is not totally clear but it's not necessary to model your complete system with CQRS. You can use CQRS for some domains and other architecture for subdomains.
Use Saga's for any kind of "transaction" like stuff. Rollbacks should be written by the developer. The system can't do this for you.
Modular monolith - Shouldn't be a technical problem.
Fully separated domain model (persistence agnostic) - The question is not totally clear but store only events in Axon Server. Aggregates are build up by a sequence of aggregates. Don't use any other data for it. The aggregate are used to do the command handling with state checks and apply new events.
I a system gets a command message, Axon Framework will look at the aggregate id and re-creates the aggregate by replay all the existing events for that aggregate. Then the method for #CommandHandler and command message type is called on the aggregate with the state of the system. Don't do this by yourself.
On the other hand. Create own custom projections (view models) by listening to the events (#EventHandler) and store the data in your own format to any kind of data models/repository. You can for example build a REST api on top of this to use the data.
Axon Server - Use it where it's built for. Use it as event store and not for other purposes.
See for more info and why: https://www.youtube.com/watch?v=zUSWsJteRfw
I was wondering what is the reasoning behind making messages immutable in Spring Integration.
Is it only because of thread-safety in multi threaded evnironments?
Performance? Don't you get a performance penalization when you have to create a new message each time you want to add something to an existing message?
Avoiding a range of bugs when passing by reference?
Just guessing here.
The simplest way to explain this comes from the original Java Immutable Objects idea:
Immutable objects are particularly useful in concurrent applications. Since they cannot change state, they cannot be corrupted by thread interference or observed in an inconsistent state.
Since we talk here about Messaging we should always keep in mind the Loose coupling principle where the producer (caller) and consumer (executor) know nothing about each other and they communicate only via messages (events, commands, packages etc.). At the same time the same message may have several consumers to perform absolutely not related business logics. So, supporting immutable state for the active object we don't impact one process from another. That's might be also as a part of the security between processes when we execute a message in isolation.
The Spring Integration is really pure Java, so any concurrency and security restrictions just simply applied here as well and you would be surprised distributing a message to different independent processes and see modifications from one process in the other.
There is some information in the Reference Manual:
Therefore, when a Message instance is sent to multiple consumers (e.g. through a Publish Subscribe Channel), if one of those consumers needs to send a reply with a different payload type, it will need to create a new Message. As a result, the other consumers are not affected by those changes.
As you see it is applied for Message object per se and its MessageHeaders. The payload is fully your responsibility and I really had in past some problems adding and removing elements to the ArrayList payload in multi-threaded business logic.
Anyway the Framework suggest a compromise: MutableMessage, MutableMessageHeaders and MutableMessageBuilder. You also can globally override the MessageBuilder used in the Framework internally to the MutableMessageBuilderFactory. For this purpose you just need to register such a bean with the bean name IntegrationUtils.INTEGRATION_MESSAGE_BUILDER_FACTORY_BEAN_NAME:
#Bean(name = IntegrationUtils.INTEGRATION_MESSAGE_BUILDER_FACTORY_BEAN_NAME)
public static MessageBuilderFactory mutableMessageBuilderFactory() {
return new MutableMessageBuilderFactory();
}
And all messages in your integration flows will be mutable and supply the same id and timestamp headers.
I read through the official documentation and the official whitepaper, but I couldn't find a satisfying answer to how Thrift handles failures in the following scenario:
Say you have a client sending a method call to a server to insert an entry in some data structure residing in that server (it doesn't really matter what it is). Suppose the server has processed the call and inserted the entry but the client couldn't receive a response due to a network failure. In such a case, how should the client handle this? A simple retry of sending the call would possibly result in a duplicate entry being inserted. Does the Thrift library persist the response somewhere so that it can resend to the client when it is back online? Or is it the application's responsibility to do so?
Would appreciate it if someone could point out the details of how it works, besides directing to its source code.
The question is an interesting one, but it is by no means limited to Thrift. A better name would be
Handling failures in asynchronous or remote calls in general
because that's in essence, what it is. Altough in the specific case of an RPC-style API like, for example, a Thrift service, the client blocks and it seems to be an synchronous call, it really isn't that way.
The whole problem can be rephrased to the more general question about
Designing robust distributed systems
So what is the main problem, that we have to deal with? We have to assume that every call we do may fail. In particular, it can fail in three ways:
request died
request sent, server processing successful, response died
request sent, server processing failed, response died
In some cases, this is not a big deal, regardless of the exact case we have. If the client just wants to retrieve some values, he can simply re-query and will get some results eventually if he tries often enough.
In other cases, especially when the client modifies data on the server, it may become more problematic. The general recommendation in such cases is to make the service calls idempotent, meaning: regardless, how often I do the same call, the end result is always the same. This could be achieved by various means and more or less depends on the use case.
For example, one method is it to send some logical "ticket" values along with each request to filter out doubled or outdated requests on the server. The server keeps track and/or checks these tickets, before the processing starts eventually. But again, if that method suits your needs depends on your use case.
The Command and Query Responsibility Segregation (CQRS) pattern is another approach to deal with the complexity. It basically breaks the API into setters and getters. I'd recommend to look into that topic, but it is not useful for every scenario. I'd also recommend to look at the Data Consistency Primer article. Last not least the CAP theorem is always a good read.
Good Service/API design is not simple, and the fact, that we have to deal with a distributed parallel system does not make it easier, quite the opposite.
Let me try to give a straight answer.
... is it the application's responsibility to do so?
Yes.
There're 4 types of Exceptions involved in Thrift RPC, including TTransportException, TProtocolException, TApplicationException, and User-defined exceptions.
Based on the book Programmer's Guide to Apache Thrift, the former 2 are local exceptions, while the latter 2 are not.
As the names imply, TTransportException includes exceptions like NOT_OPEN, TIMED_OUT, and TProtocolException includes INVALID_DATA, BAD_VERSION, etc. These exceptions are not propagated from the server the the client and act much like normal language exceptions.
TApplicationExceptions involve problems such as calling a method that isn’t implemented or failing to provide the necessary arguments to a method.
User-defined Exceptions are defined in IDL files and raised by the user code.
For all of these exceptions, no retry operations are done by Thrift RPC framework itself. Instead, they should be handled properly by the application code.
In our ColdFusion application we have stateless model objects.
All the data I want I can get with one method call (it calls other internally without saving the state).
Methods usually ask the database for the data. All methods are read only, so I don't have to worry about thread safety (please correct me if I'm wrong).
So there is no need to instantiate objects at all. I could call them statically, but ColdFusion doesn't have static methods - calling the method would mean instantiating the object first.
To improve performance I have created singletons for every Model object.
So far it works great - each object is created once and then accessed as needed.
Now my worry is that all requests for data would go through only 1 model object.
Should I? I mean if on my object I have a method getOfferData() and it's time-consuming.
What if a couple of clients want to access it?
Will second one wait for the first request to finish or is it executed in a separate thread?
It's the same object after all.
Should I implement some kind of object pool for this?
The singleton pattern you are using won't cause the problem you are describing. If getOfferData() is still running when another call to that function gets called on a different request then this will not cause it to queue unless you do one of the following:-
Use cflock to grant an exclusive lock
Get queueing connecting to your database because of locking / transactions
You have too many things running and you use all the available concurrent threads available to ColdFusion
So the way you are going about it is fine.
Hope that helps.
I am working on an application in which I need a connection to a server. I also need to access this connection from different activities.
To achieve this I was going to override the Application class and create the connection there. This would allow for easy interaction from every Activity as I could simply call getApplicationContext().getConnection() to get access to my own connection class.
The problem with this approach is that the Application class does not have any onDestroy() method or similar in which I can release the connection and any related resources. I do not think that leaving it idle until onLowMemory() is called is the best approach here.
I cannot add a custom release() method, as I don't know when to call it (there are two Activities that can be the last one to be active, and depending on the users actions they do not know if the other is to be started when the active one is shut down).
Is there a good solution to this, should I just ignore releasing resources (before onLowMemory()) or is there a better way to achieve what I want (possibly a Service, but as there will be several calls to an underlying class it might get overly problematic with the Service?)
Just use Singleton Design Pattern. Making your Connection class Singleton gives you approach to access connection from different activities, and don`t forget to handle multithreading.