Ok completely new to this essentially we want to be able to time out a session after 10mins. that's pretty easy.
We also want to wait for external user input -- essentially data from a multi step form. also pretty easy.
We want to be able to Task.WaitAny (waitforexternalevent("updatedata"), timeout)
But this is causing issues in the orchestration.
Individually these concepts work, however we see the Task.WaitAny to unblock and reuse the first "updatedata" event.. other "updatedata" events never reach the orchestration.
Is this expected behavior, are we mixing concepts in an invalid way, or is this a bug?
We might need to see some more of your code, but with what you've described here I think the behavior you're seeing is what should be expected.
Your orchestration is "waiting" on the timeout or the external event. Once that external event is triggered, the orchestration is going to move forward and, even if something triggers that event again, the orchestration is not expecting/waiting on it.
Again, this is based on the sliver of code you've included in your question thus far. If you need to handle the event being broadcast into the orchestration multiple times you would need to have a loop of some kind.
Related
I'll clarify my question by first describing my use case:
I am writing an IMAP server, which receives commands from clients. If it is in the right state to do so, it reads a token from the network socket, and interprets that token as a command's name. As of right now, my server reads the command's name from the token and uses a switch statement to execute the corresponding code.
Here's a glimpse of it:
case ("CAPABILITY"): this.executeCapability(tag); break;
case ("NOOP"): this.executeNoop(tag); break;
case ("LOGOUT"): this.executeLogout(tag); break;
case ("LOGIN"): this.executeLogin(tag, args); break;
My question is: is there an objectively preferable reason to use events to dispatch commands instead? Would performance be better? Is there a security advantage?
My proposed alternative might look something like this:
server.on("CAPABILITY", executeCapability);
server.on("NOOP", executeNoop);
server.on("LOGOUT", executeLogout);
server.on("LOGIN", executeLogin);
Events are more easily extensible and usable by other code. They live in a system where the infrastructure already exists for anyone else to listen for any of those events. If you're calling a function, passing an argument and then using a switch statement to dispatch, then that's a custom, non-standard interface and if someone else (even another part of your own code) wants to listen for one of those events and act on it themselves, then they have to build some of their own custom code to somehow do that.
If you use events, then that infrastructure is already there. They can just listen for the event with their own listener.
Here's a little example. Imagine that for performance reasons you are caching some data from the database for each user. When the user logs out, you'd like to clear the cache. Your cache system is its own module because it can be reused in other apps. If you use events, the cache can just set its own event listener for the logout event and can do its own housekeeping when a given user logs out. In the switch design, you'd probably have to insert code in the executeLogout() function to call some method in the cache. That would work, but the code wouldn't be as encapsulated as it could be. Now, you have cache logic in the logout() method where you wouldn't need that with the event system because the cache can just watch for the events its interested in all on its own.
I'm not saying this is a killer example that means you have to do it one way or the other. Just showing some architectural advantages of events where it allows sub-systems easier access to the events so they can manage themselves (more encapsulated code) more easily.
I don't think there would be a meaningful performance difference. If you were in a tight performance sensitive loop, the event subsystem probably introduces a few more function calls to dispatch vs. the switch statement, but for regular code this wouldn't be consequential.
What needs to be done to make InMemoryTransientMessageService run in a background thread? I publish things inside a service using
base.MessageProducer.Publish(new RequestDto());
and they are exececuted immediately inside the service-request.
The project is self-hosted.
Here is a quick unit test showing the blocking of the current request instead of deferring it to the background:
https://gist.github.com/lmcnearney/5407097
There is nothing out of the box. You would have to build your own. Take a look at ServiceStack.Redis.Messaging.RedisMqHost - most of what you need is there, and it is probably simpler (one thread does everything) to get you going when compared to ServiceStack.Redis.Messaging.RedisMqServer (one thread for queue listening, one for each worker). I suggest you take that class and adapt it to your needs.
A few pointers:
ServiceStack.Message.InMemoryMessageQueueClient does not implement WaitForNotifyOnAny() so you will need an alternative way of getting the background thread to wait to incoming messages.
Closely related, the ServiceStack.Redis implementation uses topic subscriptions, which in this class is used to transfer the WorkerStatus.StopCommand, which means you have to find an alternative way of getting the background thread to stop.
Finally, you may want to adapt ServiceStack.Redis.Messaging.RedisMessageProducer as its Publish() method pushes the message requested to the queue and pushes the channel / queue name to the TopicIn queue. After reading the code you can see how the three points tie together.
Hope this helps...
Example: Business rules states that the customer should get a confirmation message (email or similar) when an order has been placed.
Lets say that a NewOrderRegisteredEvent is dispatched from the domain and is picked up by an event listener that sends of the confirmation message. When that is done some other event handler throws an exception or something else goes wrong and the unit of work is rolled back. We've now sent the user a confirmation message for something that was rolled back.
What is the "cqrs" way of solving problems like this where you want to do something after a unit of work has been committed? Another complicating factor is replaying of events. I don't want old confirmation messages to be re-sent whenever I replay recorded events in order to build a new view / projection.
My best theory so far: I've just started to look into the fascinating world of cqrs and was wondering whether this is something that would be implemented as a saga? If a saga is like a state machine where each transition only can take place a single time then I guess that would solve this problem? I just have a hard time visualizing how this will fit together with the command bus and domain events..
An Event should only occur after the transaction has been completed. If anything goes wrong and there's a rollback, then the event didn't occur from an external point of view. Therefore it shouldn't be published at all. Though an OrderRegistrationFailed event could be published if necessary.
You wouldn't want the mail to be sent unless the command has sucessfully been executed.
First a few reasons why the command handler -- as proposed in another answer -- would be the wrong place: Under some circumstances the command handler wouldn't be able to tell if the command will eventually succeed or not. Having the command handler invoke the mail sending would also put process knowledge inside the command handler, which would break the SRM and too tightly couple business rules with the application layer.
The mail should be sent after the fact, i.e. from an event handler.
To prevent this handler from firing during replay, you can just not register it. This works similar to how you test your application. You only register the handlers that you actually need.
Production system -> register all event handlers
Tests -> register only the tested event handlers
Replay -> register only the projection/denormalization handlers
Another - even more loosely coupled, though a bit more complex - possibility would be to have a Saga handle the NewOrderRegisteredEvent and issue a SendMail command to the appropriate bounded context (thanks, Yves Reynhout, for pointing this out in the question's comments).
There are two likely solutions
1) The publishing of the event and the handling of the event (i.e. the email) are part of a single transaction. In this case, your transaction framework takes care of it for you. If the email fails, then the event is rolled back. You'll likely retry the command. This is conceptually clean and easy to think about. No event is finished publishing until everyone that has something to say about it has had their say. However practically speaking, this can be painful, as it typically involves distributed transactions. These are hard to come by. Can your email client enroll in the same transaction as the database which is holding your events?
2) The publishing of the event is transactional, but the event handlers each deal with transactions in their own way. The event handler which sends emails could keep track of which events it had seen. If it crashed, it would request old events and process them. You could make a business decision as to how big a deal it would be if people had missing or duplicate emails. (For money-related transactions, the answer is probably you shouldn't allow it.)
Solution (2) is typically what you see promoted in DDD/CQRS circles as it's the more loosely coupled solution. Solution (1) is quite practical in a small system where the event store and the projections are in a single database and the projections don't change often. Solution (2) allows a diversity of event handlers to work in their own way. Solution (1) can cause lots of non-overlapping concerns to become entagled. In this case your order business rules don't complete until the many bizarre things that happen in emailing are taken care of. For one thing, it may slow you down quite a bit.
If the sending of the email were more interesting than "saw the event, sent the email", then you're right, you might have a saga or workflow on your hands. Email in large operations is often a complex system in its own right which you're unlikely to have to implement much of. You just need to be sure you put your email into a request queue of some sort (using approach (2)), and the email system is likely to do retries/batching/spam avoidance/working overnight/etc.
anybody know patterns to design idempotent operations to azure manipulation, specially the table storage? The more common approach is generate a id operation and cache it to verify new executions, but, if I have dozen of workers processing operations this approach will be more complicated. :-))
Thank's
Ok, so you haven't provided an example, as requested by knightpfhor and codingoutloud. That said, here's one very common way to deal with idempotent operations: Push your needed actions to a Windows Azure queue. Then, regardless of the number of worker role instances you have, only one instance may work on a specific queue item at a time. When a queue message is read from the queue, it becomes invisible for the amount of time you specify.
Now: a few things can happen during processing of that message:
You complete processing after your timeout period. When you go to delete the message, you get an exception.
You realize you're running out of time, so you increase the queue message timeout (today, you must call the REST API to do this; one day it'll be included in the SDK).
Something goes wrong, causing an exception in your code before you ever get to delete the message. Eventually, the message becomes visible in the queue again (after specified invisibility timeout period).
You complete processing before the timeout and successfully delete the message.
That deals with concurrency. For idempotency, that's up to you to ensure you can repeat an operation without side-effects. For example, you calculate someone's weekly pay, queue up a print job, and store the weekly pay in a Table row. For some reason, a failure occurs and you either don't ever delete the message or your code aborts before getting an opportunity to delete the message.
Fast-forward in time, and another worker instance (or maybe even the same one) re-reads this message. At this point, you should theoretically be able to simply re-perform the needed actions. If this isn't really possible in your case, you don't have an idempotent operation. However, there are a few mechanisms at your disposal to help you work around this:
Each queue message has a DequeueCount. You can use this to determine if the queue message has been processed before and, if so, take appropriate action (maybe examine the Table row for that employee, for example).
Maybe there are stages of your processing pipeline that can't be repeated. In that case: you now have the ability to modify the queue message contents while the queue message is still invisible to others and being processed by you. So, imagine appending something like |SalaryServiceCalled . Then a bit later, appending |PrintJobQueued and so on. Now, if you have a failure in your pipeline, you can figure out where you left off, the next time you read your message.
Hope that helps. Kinda shooting in the dark here, not knowing more about what you're trying to achieve.
EDIT: I guess I should mention that I don't see the connection between idempotency and Table Storage. I think that's more of a concurrency issue, as idempotency would need to be dealt with whether using Table Storage, SQL Azure, or any other storage container.
I believe you can use Reply log storage way to solve this problem
I've been looking at CQRS but I find it restricting when it comes to showing the result of commands in lets say a Web Application.
It seems to me that using CQRS, one is forced to refresh the whole view or parts of it to see the changes (using a second request) because the original command request will only store an event which is to be processed in future.
In a Web Application, is it possible that a Command request could carry the result of the event it creates back to the browser?
The answer to the headline of this question is quite simple: nothing, void or from a webbrower/rest point of view 200 OK with an empty body.
Commands applied to the system (if the change is successfully committed) does not yield a result. And in the case that you wish to leave the business logic on the server side, yes you do need to refresh the data by executing yet another request (query) to the server.
However most often you can get rid of the 2nd roundtrip to the server. Take a table where you modify a row and press a save button. Do you really need to update the table? Or in the case a user submits a comment on a blog post just append the comment to the other comments in the dom without the round trip.
If you find yourself wanting the modified state returned from the server you need to think hard about what you are trying to achieve. Most scenarios can be changed so that a simple 200 OK is more than enough.
Update: Regarding your question about queuing incoming commands. It's not recommended that incoming commands are queued since this can return false positives (a command was successfully received and queued but when the command tries to modify the state of the system it fails). There is one exception to the rule and that is if you are having a system with an append only model as state. Then is safe to queue the mutation of the system state till later if the command is valid.
Udi Dahans article called Clarified CQRS is always a good read on this topic http://www.udidahan.com/2009/12/09/clarified-cqrs/
Async commands are a strange thing to do in CQRS considering that commands can be accepter or rejected.
I wrote about it, mentioning the debate between Udi Dahan's vision and Greg Young's vision on my blog: https://www.sunnyatticsoftware.com/blog/asynchronous-commands-are-dangerous
Answering your question, if you strive to design the domain objects (aggregates?) in a transactional way, where every command initiates a transaction that ends in zero, one or more events (independently on whether there are some process managers later on, picking one event and initiating another transaction), then I see no reason to have an empty command result. It's extremely useful for the external actor that initates the use case, to receive a command result indicating things like whether the command was accepted or not, which events did it produce, or which specific state has now the domain (e.g: aggregate version).
When you design a system in CQRS with asynchronous commands, it's a fallacy to expect that the command will succeed and that there will be a quick state change that you'll be notified about.
Sometimes the domain needs to communicate with external services (domain services?) in an asynchronous way depending on those services api. That does not mean that the domain cannot produce meaningful domain events informing of what's going on and which changes have occured in the domain in a synchronous way. For example, the following flow makes a lot of sense:
Actor sends a sync command PurchaseBasket
Domain uses an external service to MakePayment and knows that the payment is being processed
Domain produces the events BasketPurchaseAttempted and/or PaymentRequested or similar
Still, synchronously, the command returns the result 200 Ok with a payload indicating some information about what has happened. Even if the payment hasn't completed because the payment platform is asynchronous, at least the actor has a meaningful knowledge about the result of the transaction it initiated.
Compare this design with an asynchronous one
Actor sends an async command PurchaseBasket
The system returns a 202 Accepted with a transaction Id indicating "thanks for your interest, we'll call you, this is the ticket number")
In a separate process, the domain initiates a process manager or similar with the payment platform, and when the process completes (if it completes, assuming the command is accepted and there are no business rules that forbid the purchase basket), then the system can start the notifying process to the actor.
Think about how to test both scenarios. Think about how to design UX to accommodate this. What would you show in the second scenario in the UI? Would you assume the command was accepted? Would you display the transaction Id with a thank you message and "please wait"? Would you take a big faith leap and keep the user waiting with a loading screen waiting for the async process to finish and be notified with a web socket or polling strategy for XXX seconds?
Async commands in CQRS are a dangerous thing and make us lazy domain designers.
UPDATE: the accepted answer suggest not to return anything and I fully disagree. Checkout Eventuous library and you'll see that returning a result is extremely helpful.
Also, if an async command can't be rejected it's... because it's not really a command but a fact.
UPDATE: I am surprised my answer got negative votes. Especially because Greg Young, the creator of CQRS term, says literally in his book about CQRS
One important aspect of Commands is that they are always in the imperative tense; that is they are
telling the Application Server to do something. The linguistics with Commands are important. A situation
could for with a disconnected client where something has already happened such as a sale and could
want to send up a “SaleOccurred” Command object. When analyzing this, is the domain allowed to say
no that this thing did not happen? Placing Commands in the imperative tense linguistically shows that
the Application Server is allowed to reject the Command, if it were not allowed to, it would be an Event
for more information on this see “Events”.
While I understand certain authors are biased towards the solutions they sell, I'd go to the main source of info in CQRS, regardless of how many hundred of implementations are there returning void when they can return something to inform requester asap. It's just an implementation detail, but it'll help model better the solution to think that way.
Greg Young, again, the guy who coined the CQRS term, also says
CQRS and Event Sourcing describe something inside a single system or component.
The communication between different components/bounded contexts (which ideally should be event driven and asynchronous, although that's not a requirement either) is outside the scope of CQRS.
PS: ignoring an event is not the same as rejecting a command. Rejection implies a direct answer to the command sender. Something "difficult" if you return nothing to the sender (not even a correlation ID?)
Source:
https://gregfyoung.wordpress.com/tag/cqrs/
https://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf