I did create an aggregate service as below
#EnableBinding(Processor.class)
class Configuration {
#Autowired
Processor processor;
#ServiceActivator(inputChannel = Processor.INPUT)
#Bean
public MessageHandler aggregator() {
AggregatingMessageHandler aggregatingMessageHandler =
new AggregatingMessageHandler(new DefaultAggregatingMessageGroupProcessor(),
new SimpleMessageStore(10));
//AggregatorFactoryBean aggregatorFactoryBean = new AggregatorFactoryBean();
//aggregatorFactoryBean.setMessageStore();
aggregatingMessageHandler.setOutputChannel(processor.output());
//aggregatorFactoryBean.setDiscardChannel(processor.output());
aggregatingMessageHandler.setSendPartialResultOnExpiry(true);
aggregatingMessageHandler.setSendTimeout(1000L);
aggregatingMessageHandler.setCorrelationStrategy(new ExpressionEvaluatingCorrelationStrategy("requestType"));
aggregatingMessageHandler.setReleaseStrategy(new MessageCountReleaseStrategy(3)); //ExpressionEvaluatingReleaseStrategy("size() == 5")
aggregatingMessageHandler.setExpireGroupsUponCompletion(true);
aggregatingMessageHandler.setGroupTimeoutExpression(new ValueExpression<>(3000L)); //size() ge 2 ? 5000 : -1
aggregatingMessageHandler.setExpireGroupsUponTimeout(true);
return aggregatingMessageHandler;
}
}
Now i want to release the group as soon as a new group is created, so i only have one group at a time.
To be more specific i do receive two types of requests 'PUT' and 'DEL' . i want to keep aggregating per the above rules but as soon as i receive a request type other than what i am aggregating i want to release the current group and start aggregating the new Type.
The reason i want to do this is because these requests are sent to another party that don't support having PUT and DEL requests at the same time and i can't delay any DEL request as sequence between PUT and DEL is important.
I understand that i need to create a custom release Pojo but will i be able to check the current groups ?
For Example
If i receive 6 messages like below
PUT PUT PUT DEL DEL PUT
they should be aggregated as below
3PUT
2 DEL
1 PUT
OK. Thank you for sharing more info.
Yes, you custom ReleaseStrategy can check that message type and return true to lead to the group completion function.
As long as you have only static correlationKey, so only one group is there in the store. When your message is stepping to the ReleaseStrategy, there won't be much magic just to check the current group for completion signal. Since there are no any other groups in the store, there is no need any complex release logic.
You should add expireGroupsUponCompletion = true to let the group to be removed after completion and the next message will form a new group for the same correlationKey.
UPDATE
Thank you for further info!
So, yes, your original PoC is good. And even static correlationKey is fine, since you are just going to collect incoming messages to batches.
Your custom ReleaseStrategy should analyze MessageGroup for a message with different key and return true in that case.
The custom MessageGroupProcessor should filter a message with different key from the output List and send that message to the aggregator back to let to form a new group for a sequence for its key.
i ended up implementing the below ReleaseStrategy as i found it simpler than removing message and queuing it again.
class MessageCountAndOnlyOneGroupReleaseStrategy implements org.springframework.integration.aggregator.ReleaseStrategy {
private final int threshold;
private final MessageGroupProcessor messageGroupProcessor;
public MessageCountAndOnlyOneGroupReleaseStrategy(int threshold,MessageGroupProcessor messageGroupProcessor) {
super();
this.threshold = threshold;
this.messageGroupProcessor = messageGroupProcessor;
}
private MessageGroup currentGroup;
#Override
public boolean canRelease(MessageGroup group) {
if(currentGroup == null)
currentGroup = group;
if(!group.getGroupId().equals(currentGroup.getGroupId())) {
messageGroupProcessor.processMessageGroup(currentGroup);
currentGroup = group;
return false;
}
return group.size() >= this.threshold;
}
}
Note that i did used new HeaderAttributeCorrelationStrategy("request_type") instead of just FOO for CollorationStrategy
Related
Let's say I have two types of Hazelcast nodes running on cluster:
"Leader" nodes – these are able to load and populate Hazelcast map M. Leaders will also update values in M from time to time (based on external resource).
"Follower" nodes – these will need to read from M
My intent is for Follower nodes to trigger loading missing elements into M (loading thus needs to be done on Leader side) .
Roughly, the steps made to get an element from map could look like this:
IMap m = hazelcastInstance.getMap("M");
if (!m.containsKey(k)) {
if (iAmLeader()) {
Object fresh = loadByKey(k); // loading from external resource
return m.put(k, fresh);
} else {
makeSomeLeaderPopulateValueForKey(k);
}
}
return m.get(k);
What approach could you suggest?
Notes
I want Followers to act as nodes, not just clients, because there are going to be far more Follower instances than Leaders and I would like them to participate in load distribution.
I could just build another level of service, that would run only on Leader nodes and provide interface to populate map with requested keys. But that would mean adding extra layer of communication and configuration, and I was hoping that the kind of requirements stated above could be solved within single Hazelcast cluster.
I think I may have found an answer in the form of MapLoader (EDIT since originally posting, I have confirmed this is indeed the way to do this).
final Config config = new Config();
config.getMapConfig("MY_MAP_NAME").setMapStoreConfig(
new MapStoreConfig().setImplementation(new MapLoader<KeyType, ValueType>(){
#Override
public ValueType load(final KeyType key) {
//when a client asks for data for corresponding key of type
//KeyType that isn't already loaded
//this function will be invoked and give you a chance
//to load it and return it
ValueType rv = ...;
return rv;
}
#Override
public Map<KeyType, ValueType> loadAll(
final Collection<KeyType> keys) {
//Similar to MapLoader#load(KeyType), except this is
//a batched version of it for performance gains.
//this gets called on first access to the cache,
//where MapLoader#loadAllKeys() is called to get
//the keys parameter for this funcion
Map<KeyType, ValueType> rv = new HashMap<>();
keys.foreach((key)->{
rv.put(key, /*figure out what key means*/);
});
return rv;
}
#Override
public Set<KeyType> loadAllKeys() {
//Prepopulate all the keys. My understanding is that
//this is an initialization step, to give you a chance
//to load data on startup so an initial set of datas
//will be available to anyone using the cache. Any keys
//returned here are sent to MapLoader#loadAll(Collection)
Set<KeyType> rv = new HashSet<>();
//figure out what keys need to be in the return value
//to load a key into cache at first access to this map,
//named "MY_MAP_NAME" in this example
return rv;
}
}));
config.getGroupConfig().setName("MY_INSTANCE_NAME").setPassword("my_password");
final HazelcastInstance hazelcast = Hazelcast
.getOrCreateHazelcastInstance(config);
I have basic workflow with replicator activity inside it. Replicator contains my custom sequence activity with standard create task --> ontaskchanged --> complete task sequence.
Now: tasks are created and can be completed without problem. The thing is I cannot find a way to get a value of completed task. Was it approved or rejected ?
Please provide couple lines of code of replicator's ChildCompleted event to get anything out of Sequence activity instance (or any other way).
thanks
UPDATE: It seems in order to exchange values between instances of workflow you need to use DependencyProperty. So solution here is:
1) add DependencyProperty to parent workflow and add property which you will use to store value like this:
public static DependencyProperty childStatusProperty =
System.Workflow.ComponentModel.DependencyProperty.Register("childStatus",
typeof(string), typeof(parentWorkflowTypeName));
public string childStatus
{
get
{
return (string)base.GetValue(childStatusProperty);
}
set
{
base.SetValue(childStatusProperty, value);
}
}
2) in custom sequence activity access parent's instance and use defined DependencyProperty to set property to value like this:
private void completeTask1_MethodInvoking(object sender, EventArgs e)
{
var replicator = this.Parent;
var workflowParent = (parentWorkflowTypeName)replicator.Parent;
workflowParent.childStatus = "my custom status value";
}
3) read this value using normal property:
//from parent workflow
string status = childStatus;
The issue is that you have to record somewhere the list of all tasks created. I guess you are creating the tasks in parallel (not sequential).
I had the same issue, it took me a while to fix this.
Please check this link as a good starting point: http://rmanimaran.wordpress.com/2010/12/02/sharepoint-workflow-replicator-parallel-approval-problem-solution/
I have got a question related to CQRS in data centric processes. Let me explain it better.
Consider we have a SOAP/JSON/whatever service, which transfers some data to our system during an integration process. It is said that in CQRS every state change must be achieved by the means of commands (or events if Event Sourcing is used).
When it comes to our integrating process we have got a great deal of structured DATA instead of a set of commands/events and I am wondering how to actually process those data.
// Some Façade service
class SomeService
{
$_someService;
public function __construct(SomeService $someService)
{
$this->_someService = $someService;
}
// Magic function to make it all good and
public function process($dto)
{
// if I get it correctly here I need somehow
// convert incoming dto (xml/json/array/etc)
// to a set of commands, i. e
$this->someService->doSomeStuff($dto->someStuffData);
// SomeStuffChangedEvent raised here
$this->someService->doSomeMoreStuff($dtom->someMoreStuffData);
// SomeMoreStuffChangedEvent raised here
}
}
My question is whether my suggestion is suitable in the given case or there may be some better methods to do what I need. Thank you in advance.
Agreed, a service may have a different interface. If you create a rest-api to update employees, you may want to provide an UpdateEmployeeMessage which contains everything that can change. In a CRUD-kind of service, this message would probably mirror the database.
Inside of the service, you can split the message into commands:
public void Update(UpdateEmployeeMessage message)
{
bus.Send(new UpdateName
{
EmployeeId = message.EmployeeId,
First = message.FirstName,
Last = message.LastName,
});
bus.Send(new UpdateAddress
{
EmployeeId = message.EmployeeId,
Street = message.Street,
ZipCode = message.ZipCode,
City = message.City
});
bus.Send(new UpdateContactInfo
{
EmployeeId = message.EmployeeId,
Phone = message.Phone,
Email = message.Email
});
}
Or you could call the aggregate directly:
public void Update(UpdateEmployeeMessage message)
{
var employee = repository.Get<Employee>(message.EmployeeId);
employee.UpdateName(message.FirstName, message.LastName);
employee.UpdateAddress(message.Street, message.ZipCode, message.City);
employee.UpdatePhone(message.Phone);
employee.UpdateEmail(message.Email);
repository.Save(employee);
}
About Domain Driven Design, Order and OrderLines are always seen as an aggregate, where Order is the root. Normally, once an order is created, one cannot change it. In my case however, that is possible. Instead each order has a state determining whether the order can be changed or not.
In this case, are both Order and OrderLines their own “aggregate root”? I need to be able to update order lines, so I figure that they should have their own repository. But I do not want to retrieve order lines, and persist them without the order. So this indicates that there’s still an aggregate where Order is the root with a factory method to create order lines (Order.CreateOrderLine(quantity, text, …).
Another approach could be to update the Order when the order lines collection has been modified, and then call UpdateOrder(Order). I would need some way of detecting that only the collection should be updated, and no the Order itself (using Entity Framework).
What do you think?
Order lines shouldn't be an aggregate of it's own, and doesn't need it's own repository. Your aggregate should be setup something like this...
public class Order
{
private List<OrderLine> _orderLines;
private OrderState _orderState;
public IEnumerable<OrderLine> OrderLines
{
get { return _orderLines.AsReadOnly();}
}
public OrderState Status
{
get { return _orderState; }
}
public void DeleteOrderLine(Guid orderLineID)
{
if (Status.IsProcessed)
throw new InvalidOperationException("You cannot delete items from a processed order");
OrderLine lineToRemove = _orderLines.Find(ol => ol.Id == orderLineID);
_orderLines.Remove(lineToRemove);
}
public void AddOrderLine(Product product, int quantity)
{
if (Status.IsProcessed)
throw new InvalidOperationException("You cannot add items to a processed order");
OrderLine line = new OrderLine(product.ProductID, (product.Price * quantity), quantity);
_orderLines.Add(line);
}
}
Entity framework has some built in features to detect changes to your object. This is explained here (conveniently with an order/order lines example): http://msdn.microsoft.com/en-us/library/dd456854.aspx
Im using subsonic 2.2
I tried asking this question another way but didnt get the answer i was looking for.
Basically i ususally include validation at page level or in my code behind for my user controls or aspx pages. However i haev seen some small bits of info advising this can be done within partial classes generated from subsonic.
So my question is, where do i put these, are there particular events i add my validation / business logic into such as inserting, or updating. - If so, and validation isnt met, how do i stop the insert or update. And if anyone has a code example of how this looks it would be great to start me off.
Any info greatly appreciated.
First you should create a partial class for you DAL object you want to use.
In my project I have a folder Generated where the generated classes live in and I have another folder Extended.
Let's say you have a Subsonic generated class Product. Create a new file Product.cs in your Extended (or whatever) folder an create a partial class Product and ensure that the namespace matches the subsonic generated classes namespace.
namespace Your.Namespace.DAL
{
public partial class Product
{
}
}
Now you have the ability to extend the product class. The interesting part ist that subsonic offers some methods to override.
namespace Your.Namespace.DAL
{
public partial class Product
{
public override bool Validate()
{
ValidateColumnSettings();
if (string.IsNullOrEmpty(this.ProductName))
this.Errors.Add("ProductName cannot be empty");
return Errors.Count == 0;
}
// another way
protected override void BeforeValidate()
{
if (string.IsNullOrEmpty(this.ProductName))
throw new Exception("ProductName cannot be empty");
}
protected override void BeforeInsert()
{
this.ProductUUID = Guid.NewGuid().ToString();
}
protected override void BeforeUpdate()
{
this.Total = this.Net + this.Tax;
}
protected override void AfterCommit()
{
DB.Update<ProductSales>()
.Set(ProductSales.ProductName).EqualTo(this.ProductName)
.Where(ProductSales.ProductId).IsEqualTo(this.ProductId)
.Execute();
}
}
}
In response to Dan's question:
First, have a look here: http://github.com/subsonic/SubSonic-2.0/blob/master/SubSonic/ActiveRecord/ActiveRecord.cs
In this file lives the whole logic I showed in my other post.
Validate: Is called during Save(), if Validate() returns false an exception is thrown.
Get's only called if the Property ValidateWhenSaving (which is a constant so you have to recompile SubSonic to change it) is true (default)
BeforeValidate: Is called during Save() when ValidateWhenSaving is true. Does nothing by default
BeforeInsert: Is called during Save() if the record is new. Does nothing by default.
BeforeUpdate: Is called during Save() if the record is new. Does nothing by default.
AfterCommit: Is called after sucessfully inserting/updating a record. Does nothing by default.
In my Validate() example, I first let the default ValidatColumnSettings() method run, which will add errors like "Maximum String lenght exceeded for column ProductName" if product name is longer than the value defined in the database. Then I add another errorstring if ProductName is empty and return false if the overall error count is bigger than zero.
This will throw an exception during Save() so you can't store the record in the DB.
I would suggest you call Validate() yourself and if it returns false you display the elements of this.Errors at the bottom of the page (the easy way) or (more elegant) you create a Dictionary<string, string> where the key is the columnname and the value is the reason.
private Dictionary<string, string> CustomErrors = new Dictionary<string, string>
protected override bool Validate()
{
this.CustomErrors.Clear();
ValidateColumnSettings();
if (string.IsNullOrEmpty(this.ProductName))
this.CustomErrors.Add(this.Columns.ProductName, "cannot be empty");
if (this.UnitPrice < 0)
this.CustomErrors.Add(this.Columns.UnitPrice, "has to be 0 or bigger");
return this.CustomErrors.Count == 0 && Errors.Count == 0;
}
Then if Validate() returns false you can add the reason directly besides/below the right field in your webpage.
If Validate() returns true you can safely call Save() but keep in mind that Save() could throw other errors during persistance like "Dublicate Key ...";
Thanks for the response, but can you confirm this for me as im alittle confused, if your validating the column (ProductName) value within validate() or the beforevalidate() is string empty or NULL, doesnt this mean that the insert / update has already been actioned, as otherwise it wouldnt know that youve tried to insert or update a null value from the UI / aspx fields within the page to the column??
Also, within asp.net insert or updating events we use e.cancel = true to stop the insert update, if beforevalidate failes does it automatically stop the action to insert or update?
If this is the case, isnt it eaiser to add page level validation to stop the insert or update being fired in the first place.
I guess im alittle confused at the lifecyle for these methods and when they come into play