Akka.net Passivation for DDD Aggregate Coordinator (repository) - domain-driven-design

I am using Akka.net and looking to implement a reactive equivalent of a 'DDD repository', from what I have seen from here http://qnalist.com/questions/5585484/ddd-eventsourcing-with-akka-persistence and https://gitter.im/petabridge/akka-bootcamp/archives/2015/06/25
I understand the idea of having a coordinator that keeps a number of actors in memory according to some live in-memory count or some amount of elapsed time.
As a summary (based on the links above) I am trying to:
Create an Aggregate coordinator (for each actor type) that returns aggregates on request.
Each aggregate uses Context.SetReceiveTimeout method to identify if it's not used for some period of time. If so, it will receive ReceiveTimeout message.
On receipt of timeout message, the Child will send a Passivate message back to coordinator (which in turn will then cause the coordinator to shut the child down).
Whilst the child is being shutdown, all messages to child are intercepted by the coordinator and buffered.
Once shutdown of child has been confirmed (in the coordinator), if there are buffered messages for that child it is recreated and all messages flushed through to the recreated child.
How would one intercept the messages that are being attempted to be sent to the child (step 4) and instead route them to the parent? Or in other words I want the child to say at the point of sending the Passivate message to also say "hey don't send me anymore messages, send them to my parent instead".
This would save me routing everything through the coordinator (or am i going about it in the wrong way and message intercept impossible to do, and should instead proxy everything through the parent)?
I have my message contracts:
public class GetActor
{
public readonly string Identity;
public GetActor(string identity)
{
Identity = identity;
}
}
public class GetActorReply
{
public readonly IActorRef ActorRef;
public GetActorReply(IActorRef actorRef)
{
ActorRef = actorRef;
}
}
public class Passivate // sent from child aggregate to parent coordinator
{
}
Coordinator class, which for every aggregate type there is a unique instance:
public class ActorLifetimeCoordinator r<T> : ReceiveActor where T : ActorBase
{
protected Dictionary<Identity,IActorRef> Actors = new Dictionary<Identity, IActorRef>();
protected Dictionary<Identity, List<object>> BufferedMsgs = new Dictionary<Identity, List<object>>();
public ActorLifetimeCoordinator()
{
Receive<GetActor>(message =>
{
var actor = GetActor(message.Identity);
Sender.Tell(new GetActorReply(actor), Self); // reply with the retrieved actor
});
Receive<Passivate>(message =>
{
var actorToUnload = Context.Sender;
var task = actorToUnload.GracefulStop(TimeSpan.FromSeconds(10));
// the time between the above and below lines, we need to intercept messages to the child that is being
// removed from memory - how to do this?
task.Wait(); // dont block thread, use pipeto instead?
});
}
protected IActorRef GetActor(string identity)
{
IActorRef value;
return Actors.TryGetValue(identity, out value)
? value : Context.System.ActorOf(Props.Create<T>(identity));
}
}
Aggregate base class from which all aggregates derive:
public abstract class AggregateRoot : ReceivePersistentActor
{
private readonly DispatchByReflectionStrategy _dispatchStrategy
= new DispatchByReflectionStrategy("When");
protected AggregateRoot(Identity identity)
{
PersistenceId = Context.Parent.Path.Name + "/" + Self.Path.Name + "/" + identity;
Recover((Action<IDomainEvent>)Dispatch);
Command<ReceiveTimeout>(message =>
{
Context.Parent.Tell(new Passivate());
});
Context.SetReceiveTimeout(TimeSpan.FromMinutes(5));
}
public override string PersistenceId { get; }
private void Dispatch(IDomainEvent domainEvent)
{
_dispatchStrategy.Dispatch(this, domainEvent);
}
protected void Emit(IDomainEvent domainEvent)
{
Persist(domainEvent, success =>
{
Dispatch(domainEvent);
});
}
}

Easiest (but not simplest) option here is to use Akka.Cluster.Sharding module, which covers areas of coordinator pattern with support for actors distribution and balancing across the cluster.
If you will choose that you don't need it, unfortunately you'll need to pass messages through coordinator - messages themselves need to provide identifier used to determine recipient. Otherwise you may end up sending messages to dead actor.

Related

ArchUnit: Verify method only calls one outside method

In a Controller-Service-Datalayer architecture, I'm searching for a way to verify that my controller methods perform exactly one call to the service layer like this:
#DeleteMapping(value = "/{id}")
public ResponseEntity<String> deleteBlubber(#PathVariable("id") long blubberId) {
service.deleteBlubber(blubberId);
return new ResponseEntity<>("ok", HttpStatus.OK);
}
This should not be allowed:
#DeleteMapping(value = "/{id}")
public ResponseEntity<String> deleteBlubber(#PathVariable("id") long blubberId) {
service.deleteOtherStuffFirst(); // Opens first transaction
service.deleteBlubber(blubberId); // Opens second transaction - DANGER!
return new ResponseEntity<>("ok", HttpStatus.OK);
}
As you can see from the comments, the reason for this is to make sure that each request is handled in one transaction (that is started in the service layer), not multiple transactions.
It seems that ArchUnit can only check meta-data from classes and methods and not what's actually going on in a method. I would have to be able to count the request to the service classes, which seems to not be possible in ArchUnit.
Any idea if this might be possible? Thanks!
With JavaMethod.getMethodCallsFromSelf() you have access to all methods calls of a given method. This could be used inside a custom ArchCondition like this:
methods()
.that().areDeclaredInClassesThat().areAnnotatedWith(Controller.class)
.should(new ArchCondition<JavaMethod>("call exactly one service method") {
#Override
public void check(JavaMethod item, ConditionEvents events) {
List<JavaMethodCall> serviceCalls = item.getMethodCallsFromSelf().stream()
.filter(call -> call.getTargetOwner().isAnnotatedWith(Service.class))
.toList();
if (serviceCalls.size() != 1) {
String message = serviceCalls.stream().map(JavaMethodCall::getDescription).collect(joining(" and "));
events.add(SimpleConditionEvent.violated(item, message));
}
}
})

How to multi-thread parsing of JMS messages

In my Spring Boot project, I have two JMS listeners listening to one queue. All messages received from the queue have to be processed in the same way and persisted / updated in the database (Oracle). Currently, I have a synchronized method in a class that is doing the parsing of the messages. As expected, all thread read messages simultaneously, but parsing is done one by one as the method (parseMessage()) is synchronized. What I want is to parse the messages simultaneously and do database operations as well.
How can I solve this?
I don't want to create two different classes with the same code and use #Qualifier to call different classes in each listener, as the code for parsing the message is the same.
The ideal solution, I think, is to do database operations using a new synchronized method in a new class, but parsing the message in a multi-threaded way. So, at a time only one thread can say persist / update. When a thread is not waiting to persist / update, it continues the parsing on its own thread.
Please correct me if I am wrong or if you find the optimal solution. Let me know if any other info is needed.
JMS Controller Class
#RestController
#EnableJms
public class JMSController {
#Autowired
private IParseMapXml iParseMapXml;
#JmsListener(destination = "${app.jms_destinaltion}")
public void receiveMessage1(String recvMsg) {
try {
InputSource is = new InputSource(new StringReader(recvMsg.replaceAll("&", "&amp")));
Document doc = new SAXReader().read(is);
iParseMapXml.parseMessage(doc);
} catch (Exception e) {
}
}
#JmsListener(destination = "${app.jms_destinaltion}")
public void receiveMessage2(String recvMsg) {
try {
InputSource is = new InputSource(new StringReader(recvMsg.replaceAll("&", "&amp")));
Document doc = new SAXReader().read(is);
iParseMapXml.parseMessage(doc);
} catch (Exception e) {
}
}
}
Parse XML Interface
public interface IParseMapXml {
public void parseMessage(Document doc);
}
Parsing Implementation
public class ParsingMessageClass implements IParseMapXml{
#Override
#Transactional
synchronized public void parseMessage(Document doc) {
// TODO Auto-generated method stub
....
PROCESS DATA/MESSAGE
....
DO DB OPERATIONS
}
}

DDD entity with complex creation process

How entities with complex creation process should be created in DDD? Example:
Entity
- Property 1
- Property 2: value depends on what was provided in Property 1
- Property 3: value depends on what was provided in Property 1
- Property 4: value depends on what was provided in Property 1, 2 and 3
I have two ideas but both looks terrible:
Create entity with invalid state
Move creation process to service
We are using REST API so in first scenario we will have to persist entity with invalid state and in second scenario we move logic outside of the entity.
You can use the Builder Pattern to solve this problem.
You can make a Builder that has the logic for the dependencies between properties and raise Exceptions, return errors or has a mechanism to tell the client which are the next valid steps.
If you are using an object orienterd language, the builder can also return different concrete classes based on the combination of these properties.
Here's a very simplified example. We will store a configuration for EventNotifications that can either listen on some Endpoint (IP, port) or poll.
enum Mode { None, Poll, ListenOnEndpoint }
public class EventListenerNotification {
public Mode Mode { get; set; }
public Interval PollInterval { get; set; }
public Endpoint Endpoint { get; set; }
}
public class Builder {
private Mode mMode = Mode.None;
private Interenal mInterval;
private Endpoint mEndpoint;
public Builder WithMode(Mode mode) {
this.mMode = mode;
return this;
}
public Builder WithInterval(Interval interval) {
VerifyModeIsSet();
verifyModeIsPoll();
this.mInterval = interval;
return this;
}
public Builder WithEndpoint(Endpoint endpoint) {
VerifyModeIsSet();
verifyModeIsListenOnEndpoint();
this.mInterval = interval;
return this;
}
public EventListenerNotification Build() {
VerifyState();
var entity = new EventListenerNotification();
entity.Mode = this.mMode;
entity.Interval = this.mInterval;
entity.Endpoint = this.mEndpoint;
return entity;
}
private void VerifyModeIsSet() {
if(this.mMode == Mode.None) {
throw new InvalidModeException("Set mode first");
}
}
private void VerifyModeIsPoll() {
if(this.mMode != Mode.Poll) {
throw new InvalidModeException("Mode should be poll");
}
}
private void VerifyModeIsListenForEvents() {
if(this.mMode != Mode.ListenForEvents) {
throw new InvalidModeException("Mode should be ListenForEvents");
}
}
private void VerifyState() {
// validate properties based on Mode
if(this.mMode == Mode.Poll) {
// validate interval
}
if(this.mMode == Mode.ListenForEvents) {
// validate Endpoint
}
}
}
enum BuildStatus { NotStarted, InProgress, Errored, Finished }
public class BuilderWithStatus {
private readonly List<Error> mErrors = new List<Error>();
public BuildStatus Status { get; private set; }
public IReadOnlyList<Error> Errors { get { return mErrors; } }
public BuilderWithStatus WithInterval(Interval inerval) {
if(this.mMode != Mode.Poll) {
this.mErrors.add(new Error("Mode should be poll");
this.Status = BuildStatus.Errored;
}
else {
this.mInterval = interval;
}
return this;
}
// rest is same as above, but instead of throwing errors you can record the error
// and set a status
}
Here are some resources with more information and other machisms that you can use:
https://martinfowler.com/articles/replaceThrowWithNotification.html
https://martinfowler.com/eaaDev/Notification.html
https://martinfowler.com/bliki/ContextualValidation.html
Take a look at chapter 6 of the Evans book, which specifically talks about the life cycle of entities in the domain model
Creation is usually handled with a factory, which is to say a function that accepts data as input and returns a reference to an entity.
in second scenario we move logic outside of the entity.
The simplest answer is for the "factory" to be some method associate with the entity's class - ie, the constructor, or some other static method that is still part of the definition of the entity in the domain model.
But problem is that creation of the entity requires several steps.
OK, so what you have is a protocol, which is to say a state machine, where you collect information from the outside world, and eventually emit a new entity.
The instance of the state machine, with the data that it has collected, is also an entity.
For example, creating an actionable order might require a list of items, and shipping addresses, and billing information. But we don't necessarily need to collect all of that information at the same time - we can get a little bit now, and remember it, and then later when we have all of the information, we emit the submitted order.
It may take some care with the domain language to distinguish the tracking entity from the finished entity (which itself is probably an input to another state machine....)

EntryProcessor without locking entries

In my application, I'm trying to process data in IMap, the scenario is as follows:
application recieves request (REST for example) with set of keys to be processed
application processes entries with given key and returns result - map where key is original key of the entry and result is calculated
for this scenario IMap.executeOnKeys is almost perfect, with one problem - the entry is locked while being processed - and really it hurts thruput. The IMap is populated on startup and never modified.
Is it possible to process entries without locking them? If possible without sending entries to another node and without causing network overhead (sending 1000 tasks to single node in for-loop)
Here is reference implementation to demonstrate what I'm trying to achieve:
public class Main {
public static void main(String[] args) throws Exception {
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
IMap<String, String> map = instance.getMap("the-map");
// populated once on startup, never modified
for (int i = 1; i <= 10; i++) {
map.put("key-" + i, "value-" + i);
}
Set<String> keys = new HashSet<>();
keys.add("key-1"); // every requst may have different key set, they may overlap
System.out.println(" ---- processing ----");
ForkJoinPool pool = new ForkJoinPool();
// to simulate parallel requests on the same entry
pool.execute(() -> map.executeOnKeys(keys, new MyEntryProcessor("first")));
pool.execute(() -> map.executeOnKeys(keys, new MyEntryProcessor("second")));
System.out.println(" ---- pool is waiting ----");
pool.shutdown();
pool.awaitTermination(5, TimeUnit.MINUTES);
System.out.println(" ------ DONE -------");
}
static class MyEntryProcessor implements EntryProcessor<String, String> {
private String name;
MyEntryProcessor(String name) {
this.name = name;
}
#Override
public Object process(Map.Entry<String, String> entry) {
System.out.println(name + " is processing " + entry);
return calculate(entry); // may take some time, doesn't modify entry
}
#Override
public EntryBackupProcessor<String, String> getBackupProcessor() {
return null;
}
}
}
Thanks in advance
In executeOnKeys the entries are not locked. Maybe you mean that the processing happens on partitionThreads, so that there may be no other processing for the particular key? Anyhow, here's the solution:
Your EntryProcessor should implement:
Offloadable interface -> this means that the partition-thread will be used only for reading the value. The calculation will be done in the offloading thread-pool.
ReadOnly interface -> in this case the EP won't hop on the partition-thread again to save the modification you might have done in the entry. Since your EP does not modify entries, this will increase the performance.

Azure Service Bus SessionHandler issue with partitioned queue

I got into an issue with IMessageSessionAsyncHandlerFactory where new instances of IMessageSessionAsyncHandler are not created when the volume of writing goes to 0 and then up to a normal level.
To be more precise, I'm using SessionHandlerOptions with a value of 500 for MaxConcurrentSessions. This allows reading at a speed of more than 1k msg/s.
The queue I'm reading from is a partitioned queue.
The volume of messages in the queue is rather constant, but from time to time it gets down to 0. When the volume gets back to the normal level, the SessionFactory is not spawning any handlers so I'm not able to read messages anymore. It's like the sessions were not correctly recycled or are held into a sort of continuous waiting.
Here is the code for the factory registering:
private void RegisterHandler()
{
var sessionHandlerOptions = new SessionHandlerOptions
{
AutoRenewTimeout = TimeSpan.FromMinutes(1),
MessageWaitTimeout = TimeSpan.FromSeconds(1),
MaxConcurrentSessions = 500
};
_queueClient.RegisterSessionHandlerFactoryAsync(new SessionHandlerFactory(_callback), sessionHandlerOptions);
}
The factory class:
public class SessionHandlerFactory : IMessageSessionAsyncHandlerFactory
{
private readonly Action<BrokeredMessage> _callback;
public SessionHandlerFactory(Action<BrokeredMessage> callback)
{
_callback = callback;
}
public IMessageSessionAsyncHandler CreateInstance(MessageSession session, BrokeredMessage message)
{
return new SessionHandler(session.SessionId, _callback);
}
public void DisposeInstance(IMessageSessionAsyncHandler handler)
{
var disposable = handler as IDisposable;
disposable?.Dispose();
}
}
And the handler:
public class SessionHandler : MessageSessionAsyncHandler
{
private readonly Action<BrokeredMessage> _callback;
public SessionHandler(string sessionId, Action<BrokeredMessage> callback)
{
SessionId = sessionId;
_callback = callback;
}
public string SessionId { get; }
protected override async Task OnMessageAsync(MessageSession session, BrokeredMessage message)
{
try
{
_callback(message);
}
catch (Exception ex)
{
Logger.Error(...);
}
}
I can see that the session handlers are closed and that the factories are disposed when the writing/reading is at a normal level. However, once the queue empties, there's no way new session handlers are created. Is there a policy for allocating session IDs that forbids reallocating the same sessions after a period of inactivity?
Edit 1:
I'm adding two pictures to illustrate the behavior:
When the writer is stopped and restarted, the running reader is not able to read as much as before.
The number of sessions created after that moment is also much lower than before:
The volume of messages in the queue is rather constant, but from time to time it gets down to 0. When the volume gets back to the normal level, the SessionFactory is not spawning any handlers so I'm not able to read messages anymore. It's like the sessions were not correctly recycled or are held into a sort of continuous waiting.
When using IMessageSessionHandlerFactory to control how the IMessageSessionAsyncHandler instances are created, you could try to log the creation and destruction for all of your IMessageSessionAsyncHandler instances.
Based on your code, I created a console application to this issue on my side. Here is my code snippet for initializing queue client and handling messages:
InitializeReceiver
static void InitializeReceiver(string connectionString, string queuePath)
{
_queueClient = QueueClient.CreateFromConnectionString(connectionString, queuePath, ReceiveMode.PeekLock);
var sessionHandlerOptions = new SessionHandlerOptions
{
AutoRenewTimeout = TimeSpan.FromMinutes(1),
MessageWaitTimeout = TimeSpan.FromSeconds(5),
MaxConcurrentSessions = 500
};
_queueClient.RegisterSessionHandlerFactoryAsync(new SessionHandlerFactory(OnMessageHandler), sessionHandlerOptions);
}
OnMessageHandler
static void OnMessageHandler(BrokeredMessage message)
{
var body = message.GetBody<Stream>();
dynamic recipeStep = JsonConvert.DeserializeObject(new StreamReader(body, true).ReadToEnd());
lock (Console.Out)
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.WriteLine(
"Message received: \n\tSessionId = {0}, \n\tMessageId = {1}, \n\tSequenceNumber = {2}," +
"\n\tContent: [ title = {3} ]",
message.SessionId,
message.MessageId,
message.SequenceNumber,
recipeStep.title);
Console.ResetColor();
}
Task.Delay(TimeSpan.FromSeconds(3)).Wait();
message.Complete();
}
Per my test, the SessionHandler could work as expected when the volume of messages in the queue from normal to zero and from zero to normal for some time as follows:
I also tried to leverage QueueClient.RegisterSessionHandlerAsync to test this issue and it works as well. Additionally, I found this git sample about Service Bus Sessions, you could refer to it.

Resources