Azure Service Bus Queue: How the ordering of the message work? - azure

public static async Task DoMessage()
{
const int numberOfMessages = 10;
queueClient = new QueueClient(ConnectionString, QueueName);
await SendMessageAsync(numberOfMessages);
await queueClient.CloseAsync();
}
private static async Task SendMessageAsync(int numOfMessages)
{
try
{
for (var i = 0; i < numOfMessages; i++)
{
var messageBody = $"Message {i}";
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
message.SessionId = i.ToString();
await queueClient.SendAsync(message);
}
}
catch (Exception e)
{
}
}
This is my sample code to send message to the service bus queue with session id.
My question is if I call DoMessage function 2 times: Let's name it as MessageSet1 and MessageSet2, respectively. Will the MessageSet2 be received and processed by the received azure function who dealing with the receiving ends of the message.
I want to handle in order like MessageSet1 then the MessageSet2 and never handle with MessageSet2 unless MessageSet1 finished.

There are a couple of issues with what you're doing.
First, Azure Functions do not currently support sessions. There's an issue for that you can track.
Second, the sessions you're creating are off. A session should be applied on a set of messages using the same SessionId. Meaning your for loop should be assigning the same SessionId to all the messages in the set. Something like this:
private static async Task SendMessageAsync(int numOfMessages, string sessionID)
{
try
{
var tasks = new List<Task>();
for (var i = 0; i < numOfMessages; i++)
{
var messageBody = $"Message {i}";
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
message.SessionId = sessionId;
tasks.Add(queueClient.SendAsync(message));
}
await Task.WhenAll(tasks).ConfigureAwait(false);
}
catch (Exception e)
{
// handle exception
}
}
For ordered messages using Sessions, see documentation here.

Related

How to handle Sql Server rollbacks when Azure service bus message fails to save on the queue and they depend on each other?

I'm saving a row to my db (class with teacher/students, time, date, etc), once I have the id I create a message on my Azure service bus where the unique id of the row from my db is used as the message body of the service bus message. I'm creating scheduled messages so I can notify the students before the class and after the class is over so they can rate/review their teacher.
QUESTION - I'd like to know how to roll back or an easy way to remove the db row by not allowing it to fully save if the message to the Azure service bus fails to save?
Currently I'm using a generic repository with UnitOfWork to save to my db and I'm catching the exception from my service bus service if it fails, then deleting the row that was just saved, but it's sloppy looking and I can see it will lead to problems.
Here is what I'm doing now in the controller.
[HttpPost("create")]
public async Task<IActionResult> OnCreating(OnCreatingEventDto onCreatingDto)
{
var userFromRepo = await _userManager.FindByEmailFromClaimsPrinciple(HttpContext.User);
if (userFromRepo == null)
return Unauthorized(new ApiResponse(401));
var newEvent = _mapper.Map<ClassEvent>(onCreatingDto);
_unitOfWork.Repository<ClassEvent>().Add(newEvent);
var success = await _unitOfWork.Complete();
if (success > 0) {
try {
var sequenceNUmber = await _serviceBusProducer.SendMessage(newEvent.Id.ToString(), newEvent.eventTime.addDays(1), queueName);
newEvent.ServiceBusSequenceNumber = sequenceNUmber;
_unitOfWork.Repository<ClassEvent>().Update(newEvent);
var secondSuccess = await _unitOfWork.Complete();
if (secondSuccess > 0) {
return Ok();
}
} catch(Exception ex) {
_logger.LogError("error saving to service bus");
_unitOfWork.Repository<ClassEvent>().Delete(newEvent);
var deleteSuccess = await _unitOfWork.Complete();
if (deleteSuccess > 0) {
}
return BadRequest(new ApiResponse(400, "Problem Creating Event"));
}
}
return BadRequest(new ApiResponse(400, "Problem Creating Event"));
}
Here is the method from my service that creates the message on the queue
public async Task<long> SendMessage(string messageBody, DateTimeOffset scheduledEnqueueTime, string queueName)
{
await using (ServiceBusClient client = new ServiceBusClient(_config["ServiceBus:Connection"]))
{
ServiceBusSender sender = client.CreateSender(_config["ServiceBus:" + queueName]);
ServiceBusMessage message = new ServiceBusMessage(messageBody);
var sequenceNumber =
await sender.ScheduleMessageAsync(message, scheduledEnqueueTime);
return sequenceNumber;
}
}

Service Fabric Actor notification fails to trigger event handler after first invocation due to possible blocking

I've got 2 Reliable actors called GameActor and PlayerActor. The ClientApp send a message to the PlayerActor when the player makes a move. Then the PlayerActor sends a message to the GameActor to indicate a movement was made. Upon being invoked, the method in the GameActor fires a notification. This notification gets handled by the ClientApp GameEventsHandler. The ClientApp then calls a method on the GameActor to retrieve the latest player positions.
ClientApp -> PlayerActor.MoveTo() -> GameActor.NotifyPlayerMoved() ->
Fire ScoreBoardUpdated event
GameEventsHandler triggered by that event ->
GameActor.GetLatestPlayerInfo()
The problem I'm having is this. The very first time I run it, the GameEventsHandler gets triggered and it tries to call the GameActor as expected. The GameActor receives the message and returns the response expected. But the client doesn't seem to receive the message. It looks like it's blocked as it doesn't throw and error or any output. Any subsequent notifications don't get handled by the event handler at all.
GameActor
public async Task<IList<PlayerInfo>> GetLatestPlayerInfoAsync(CancellationToken cancellationToken)
{
var allPlayers = await StateManager.GetStateAsync<List<string>>("players", cancellationToken);
var tasks = allPlayers.Select(actorName =>
{
var playerActor = ActorProxy.Create<IPlayerActor>(new ActorId(actorName), new Uri(PlayerActorUri));
return playerActor.GetLatestInfoAsync(cancellationToken);
}).ToList();
await Task.WhenAll(tasks);
return tasks
.Select(t => t.Result)
.ToList();
}
public async Task NotifyPlayerMovedAsync(PlayerInfo lastMovement, CancellationToken cancellationToken)
{
var ev = GetEvent<IGameEvents>();
ev.ScoreboardUpdated(lastMovement);
}
PlayerActor
public async Task MoveToAsync(int x, int y, CancellationToken cancellationToken)
{
var playerName = await StateManager.GetStateAsync<string>("playerName", cancellationToken);
var playerInfo = new PlayerInfo()
{
LastUpdate = DateTimeOffset.Now,
PlayerName = playerName,
XCoordinate = x,
YCoordinate = y
};
await StateManager.AddOrUpdateStateAsync("positions", new List<PlayerInfo>() { playerInfo }, (key, value) =>
{
value.Add(playerInfo);
return value;
}, cancellationToken);
var gameName = await StateManager.GetStateAsync<string>("gameName", cancellationToken);
var gameActor = ActorProxy.Create<IGameActor>(new ActorId(gameName), new Uri(GameActorUri));
await gameActor.NotifyPlayerMovedAsync(playerInfo, cancellationToken);
}
public async Task<PlayerInfo> GetLatestInfoAsync(CancellationToken cancellationToken)
{
var positions = await StateManager.GetStateAsync<List<PlayerInfo>>("positions", cancellationToken);
return positions.Last();
}
Client
private static async Task RunDemo(string gameName)
{
var rand = new Random();
Console.WriteLine("Hit return when the service is up...");
Console.ReadLine();
Console.WriteLine("Enter your name:");
var playerName = Console.ReadLine();
Console.WriteLine("This might take a few seconds...");
var gameActor = ActorProxy.Create<IGameActor>(new ActorId(gameName), new Uri(GameActorUri));
await gameActor.SubscribeAsync<IGameEvents>(new GameEventsHandler(gameActor));
var playerActorId = await gameActor.JoinGameAsync(playerName, CancellationToken.None);
var playerActor = ActorProxy.Create<IPlayerActor>(new ActorId(playerActorId), new Uri(PlayerActorUri));
while (true)
{
Console.WriteLine("Press return to move to new location...");
Console.ReadLine();
await playerActor.MoveToAsync(rand.Next(100), rand.Next(100), CancellationToken.None);
}
}
GameEventHandler
public void ScoreboardUpdated(PlayerInfo lastInfo)
{
Console.WriteLine($"Scoreboard updated. (Last move by: {lastInfo.PlayerName})");
var positions = _gameActor.GetLatestPlayerInfoAsync(CancellationToken.None).ConfigureAwait(false).GetAwaiter().GetResult();
//this hangs
foreach (var playerInfo in positions) // this line never gits hit
{
Console.WriteLine(
$"Position of {playerInfo.PlayerName} is ({playerInfo.XCoordinate},{playerInfo.YCoordinate})." +
$"\nUpdated at {playerInfo.LastUpdate}\n");
}
}
But if I wrap the event handler logic inside a Task.Run() it seems to work.
Task.Run(async () =>
{
var positions = await _gameActor.GetLatestPlayerInfoAsync(CancellationToken.None);
foreach (var playerInfo in positions)
{
Console.WriteLine(
$"Position of {playerInfo.PlayerName} is ({playerInfo.XCoordinate},{playerInfo.YCoordinate})." +
$"\nUpdated at {playerInfo.LastUpdate}\n");
}
}
);
Full source code for the demo here https://github.com/dasiths/Service-Fabric-Reliable-Actors-Demo
AFAIK notifications aren't blocking and are not reliable. So I don't understand why my initial implementation doesn't work. The reentrant pattern doesn't apply here as per my understanding either. Can someone explain to me what's going on here? Is it expected behaviour or a bug?

How to get all the session id of a queue in azure service bus .Net Core

i'm making a web application which is connected to Azure by using .Net Core 2.1.1. I encountered some problem related with service bus queue when i'm trying to get the sessionid of the related queue.
I found some code, but it isn't supported by the .Net Core. Here below is the code:
var queueClient = QueueClient.CreateFromConnectionString(AppSettings.ServiceBusConnection, queueName);
var sessions = await queueClient.GetMessageSessionsAsync();
return sessions;
I also already tried this function,
var connString = Configuration.GetConnectionString("servicebus");
sessionClient = new SessionClient(connString, queue,ReceiveMode.PeekLock);
List<IMessageSession> sessions=new List<IMessageSession>();
while (true)
{
var session = await sessionClient.AcceptMessageSessionAsync();
if (session == null)
break;
sessions.Add(session);
}
return sessions;
}
But it keep giving me Timeout Exception. Can anyone help me ?
This is something which i tried and it worked for me, Please check the screenshot below
Here is the code i have tried
using System;
namespace Core.SBConsole
{
using Microsoft.Azure.ServiceBus;
using Microsoft.Azure.ServiceBus.Core;
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
class Program
{
// Connection String for the namespace can be obtained from the Azure portal under the
// 'Shared Access policies' section.
const string ServiceBusConnectionString = "{Connection String}";
const string QueueName = "mvq";
static IMessageSender messageSender;
static ISessionClient sessionClient;
const string SessionPrefix = "session-prefix";
static void Main(string[] args)
{
MainAsync().GetAwaiter().GetResult();
}
static async Task MainAsync()
{
const int numberOfSessions = 5;
const int numberOfMessagesPerSession = 3;
messageSender = new MessageSender(ServiceBusConnectionString, QueueName);
sessionClient = new SessionClient(ServiceBusConnectionString, QueueName);
// Send messages with sessionId set
await SendSessionMessagesAsync(numberOfSessions, numberOfMessagesPerSession);
// Receive all Session based messages using SessionClient
await ReceiveSessionMessagesAsync(numberOfSessions, numberOfMessagesPerSession);
Console.WriteLine("=========================================================");
Console.WriteLine("Completed Receiving all messages... Press any key to exit");
Console.WriteLine("=========================================================");
Console.ReadKey();
await messageSender.CloseAsync();
await sessionClient.CloseAsync();
}
static async Task ReceiveSessionMessagesAsync(int numberOfSessions, int messagesPerSession)
{
Console.WriteLine("===================================================================");
Console.WriteLine("Accepting sessions in the reverse order of sends for demo purposes");
Console.WriteLine("===================================================================");
for (int i = 0; i < numberOfSessions; i++)
{
int messagesReceivedPerSession = 0;
// AcceptMessageSessionAsync(i.ToString()) as below with session id as parameter will try to get a session with that sessionId.
// AcceptMessageSessionAsync() without any messages will try to get any available session with messages associated with that session.
IMessageSession session = await sessionClient.AcceptMessageSessionAsync();// (SessionPrefix + i.ToString());
if (session != null)
{
// Messages within a session will always arrive in order.
Console.WriteLine("=====================================");
Console.WriteLine($"Received Session: {session.SessionId}");
while (messagesReceivedPerSession++ < messagesPerSession)
{
Message message = await session.ReceiveAsync();
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
// Complete the message so that it is not received again.
// This can be done only if the queueClient is created in ReceiveMode.PeekLock mode (which is default).
await session.CompleteAsync(message.SystemProperties.LockToken);
}
Console.WriteLine($"Received all messages for Session: {session.SessionId}");
Console.WriteLine("=====================================");
// Close the Session after receiving all messages from the session
await session.CloseAsync();
}
}
}
static async Task SendSessionMessagesAsync(int numberOfSessions, int messagesPerSession)
{
if (numberOfSessions == 0 || messagesPerSession == 0)
{
await Task.FromResult(false);
}
for (int i = numberOfSessions - 1; i >= 0; i--)
{
var messagesToSend = new List<Message>();
string sessionId = SessionPrefix + i;
for (int j = 0; j < messagesPerSession; j++)
{
// Create a new message to send to the queue
string messageBody = "test" + j;
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
// Assign a SessionId for the message
message.SessionId = sessionId;
messagesToSend.Add(message);
// Write the sessionId, body of the message to the console
Console.WriteLine($"Sending SessionId: {message.SessionId}, message: {messageBody}");
}
// Send a batch of messages corresponding to this sessionId to the queue
await messageSender.SendAsync(messagesToSend);
}
Console.WriteLine("=====================================");
Console.WriteLine($"Sent {messagesPerSession} messages each for {numberOfSessions} sessions.");
Console.WriteLine("=====================================");
}
}
}
Things to consider before creating queue
1) Make sure service bus is not in the free or basic tier, if yes then scale it to Standadrd
2) Make sure to enable session while creating queue.
I am using Microsoft.Azure.ServiceBus nuget pakcage 3.4 which is latest now.If you are using some other package try to upgrade/downgrade it.
Hope it helps.

Servicestack RabbitMQ: Infinite loop fills up dead-letter-queue when RabbitMqProducer cannot redeclare temporary queue in RPC-pattern

When I declare a temporary reply queue to be exclusive (e.g. anonymous queue (exclusive=true, autodelete=true) in rpc-pattern), the response message cannot be posted to the specified reply queue (e.g. message.replyTo="amq.gen-Jg_tv8QYxtEQhq0tF30vAA") because RabbitMqProducer.PublishMessage() tries to redeclare the queue with different parameters (exclusive=false), which understandably results in an error.
Unfortunately, the erroneous call to channel.RegisterQueue(queueName) in RabbitMqProducer.PublishMessage() seems to nack the request message in the incoming queue so that, when ServiceStack.Messaging.MessageHandler.DefaultInExceptionHandler tries to acknowlege the request message (to remove it from the incoming queue), the message just stays on top of the incoming queue and gets processed all over again. This procedure repeats indefinitely and results in one dlq-message per iteration which slowly fills up the dlq.
I am wondering,
if ServiceStack handles the case, when ServiceStack.RabbitMq.RabbitMqProducer cannot declare the response queue, correctly
if ServiceStack.RabbitMq.RabbitMqProducer muss always declare the response queue before publishing the response
if it wouldn't be best to have some configuration flag to omit all exchange and queue declaration calls (outside of the first initialization). The RabbitMqProducer would just assume every queue/exchange to be properly set up and just publish the message.
(At the moment our client just declares its response queue to be exclusive=false and everything works fine. But I'd really like to use rabbitmq's built-in temporary queues.)
MQ-Client Code, requires simple "SayHello" service:
const string INQ_QUEUE_NAME = "mq:SayHello.inq";
const string EXCHANGE_NAME="mx.servicestack";
var factory = new ConnectionFactory() { HostName = "192.168.179.110" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
// Create temporary queue and setup bindings
// this works (because "mq:tmp:" stops RabbitMqProducer from redeclaring response queue)
string responseQueueName = "mq:tmp:SayHello_" + Guid.NewGuid().ToString() + ".inq";
channel.QueueDeclare(responseQueueName, false, false, true, null);
// this does NOT work (RabbitMqProducer tries to declare queue again => error):
//string responseQueueName = Guid.NewGuid().ToString() + ".inq";
//channel.QueueDeclare(responseQueueName, false, false, true, null);
// this does NOT work either (RabbitMqProducer tries to declare queue again => error)
//var responseQueueName = channel.QueueDeclare().QueueName;
// publish simple SayHello-Request to standard servicestack exchange ("mx.servicestack") with routing key "mq:SayHello.inq":
var props = channel.CreateBasicProperties();
props.ReplyTo = responseQueueName;
channel.BasicPublish(EXCHANGE_NAME, INQ_QUEUE_NAME, props, Encoding.UTF8.GetBytes("{\"ToName\": \"Chris\"}"));
// consume response from response queue
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(responseQueueName, true, consumer);
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
// print result: should be "Hello, Chris!"
Console.WriteLine(Encoding.UTF8.GetString(ea.Body));
}
}
Everything seems to work fine when RabbitMqProducer does not try to declare the queues, like that:
public void PublishMessage(string exchange, string routingKey, IBasicProperties basicProperties, byte[] body)
{
const bool MustDeclareQueue = false; // new config parameter??
try
{
if (MustDeclareQueue && !Queues.Contains(routingKey))
{
Channel.RegisterQueueByName(routingKey);
Queues = new HashSet<string>(Queues) { routingKey };
}
Channel.BasicPublish(exchange, routingKey, basicProperties, body);
}
catch (OperationInterruptedException ex)
{
if (ex.Is404())
{
Channel.RegisterExchangeByName(exchange);
Channel.BasicPublish(exchange, routingKey, basicProperties, body);
}
throw;
}
}
The issue got adressed in servicestack's version v4.0.32 (fixed in this commit).
The RabbitMqProducer no longer tries to redeclare temporary queues and instead assumes that the reply queue already exist (which solves my problem.)
(The underlying cause of the infinite loop (wrong error handling while publishing response message) probably still exists.)
Edit: Example
The following basic mq-client (which does not use ServiceStackmq client and instead depends directly on rabbitmq's .net-library; it uses ServiceStack.Text for serialization though) can perform generic RPCs:
public class MqClient : IDisposable
{
ConnectionFactory factory = new ConnectionFactory()
{
HostName = "192.168.97.201",
UserName = "guest",
Password = "guest",
//VirtualHost = "test",
Port = AmqpTcpEndpoint.UseDefaultPort,
};
private IConnection connection;
private string exchangeName;
public MqClient(string defaultExchange)
{
this.exchangeName = defaultExchange;
this.connection = factory.CreateConnection();
}
public TResponse RpcCall<TResponse>(IReturn<TResponse> reqDto, string exchange = null)
{
using (var channel = connection.CreateModel())
{
string inq_queue_name = string.Format("mq:{0}.inq", reqDto.GetType().Name);
string responseQueueName = channel.QueueDeclare().QueueName;
var props = channel.CreateBasicProperties();
props.ReplyTo = responseQueueName;
var message = ServiceStack.Text.JsonSerializer.SerializeToString(reqDto);
channel.BasicPublish(exchange ?? this.exchangeName, inq_queue_name, props, UTF8Encoding.UTF8.GetBytes(message));
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(responseQueueName, true, consumer);
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
//channel.BasicAck(ea.DeliveryTag, false);
string response = UTF8Encoding.UTF8.GetString(ea.Body);
string responseType = ea.BasicProperties.Type;
Console.WriteLine(" [x] New Message of Type '{1}' Received:{2}{0}", response, responseType, Environment.NewLine);
return ServiceStack.Text.JsonSerializer.DeserializeFromString<TResponse>(response);
}
}
~MqClient()
{
this.Dispose();
}
public void Dispose()
{
if (connection != null)
{
this.connection.Dispose();
this.connection = null;
}
}
}
Key points:
client declares anonymous queue (=with empty queue name) channel.QueueDeclare()
server generates queue and returns queue name (amq.gen*)
client adds queue name to message properties (props.ReplyTo = responseQueueName;)
ServiceStack automatically sends response to temporary queue
client picks up response and deserializes
It can be used like that:
using (var mqClient = new MqClient("mx.servicestack"))
{
var pingResponse = mqClient.RpcCall<PingResponse>(new Ping { });
}
Important: You've got to use servicestack version 4.0.32+.

NetMQ in Publisher and subscriber not working

I'm trying to implement the NetMQ Pub/Sub Model, but the Subscriber is not receiving any messages. What possibly is wrong here?
private static void ServerTask()
{
using (var context = NetMQContext.Create())
{
using (var socket = context.CreateSubscriberSocket())
{
socket.Bind("tcp://10.120.19.109:5000");
socket.Subscribe(string.Empty);
while (true)
{
Thread.Sleep(100);
string receivedMessage = socket.ReceiveString();
Console.WriteLine("Received: " + receivedMessage);
}
}
}
}
public static void ClientTask()
{
using (NetMQContext ctx = NetMQContext.Create())
{
using (var socket = ctx.CreatePublisherSocket())
{
socket.Connect("tcp://10.120.19.109:5000");
string obj = "hi";
socket.Send(obj);
}
}
}
Both are in different apps.
If you are new to NetMQ I suggest reading the zeromq guide http://zguide.zeromq.org/page:all.
Bottom line is that you are sending the message before the subscriber sent the subscription.
Pubsub in zeromq and NetMQ is like radio, you will only get messages from the moment you start listen.
To simple way to do it (not a real life solution) is to sleep for some time after the connect.
For real life solution I need to understand what are you trying to achieve
issue is like
using (NetMQContext ctx = NetMQContext.Create())
{
using (var publisher = ctx.CreatePushSocket())
{
publisher.Bind("tcp://localhost:5000");
int i = 0;
while (true)
{
try
{
publisher.Send(i.ToString(), dontWait:true);
Console.WriteLine(i.ToString());
}
catch (Exception e)
{
}
finally
{
i++;
}
}
}
Now , this code works. But if I move my while(true) loop outside. and call this code from some other function Which forces push socket and context to be created as new everytime.. this doesnot work.

Resources