I have a requirement to process the same set of messages together and for this, I was trying Azure Service Bus Sessions Enabled feature. To test this, I created a very simple application, a message is submitted successfully in a queue, however, while trying to receive the message in "ReceiveSessionMessage" function, a message session is not returned and the program exits after this line.
I am not able to figure out the exact root cause, any help would be much appreciated. Thanks
[var messageSession = await
sessionClient.AcceptMessageSessionAsync();]
Program
using Microsoft.Azure.ServiceBus;
using System;
using System.Text;
using System.Threading.Tasks;
namespace TestSendReceiveMessagesAzure
{
class Program
{
static string connectionString = "";
static string queueName = "demosessionqueue";
static void Main(string[] args)
{
Console.WriteLine("Test Service Bus Session! enable feature");
SendMessage();
Console.WriteLine("Message Pushed");
ReceiveSessionMessage();
}
private static void SendMessage()
{
QueueClient queueClilent = new QueueClient(connectionString, queueName, ReceiveMode.PeekLock);
string msgJson = "{PizzaType:Veggie,SessionID:SessionId0101}";
Message message = new Message(Encoding.UTF8.GetBytes(msgJson))
{
SessionId = "SessionId0101"
};
Console.WriteLine(msgJson);
queueClilent.SendAsync(message).Wait();
}
private static async Task ReceiveSessionMessage()
{
var sessionClient = new SessionClient(connectionString, queueName, ReceiveMode.PeekLock);
Console.WriteLine("Accepting a message session...");
try
{
var messageSession = await sessionClient.AcceptMessageSessionAsync();
Console.WriteLine($"Message.SessionID={messageSession.SessionId}");
}
catch (Exception ex)
{
Console.WriteLine(ex.StackTrace);
}
}
}
}
Console Output
The issue is with the declaration of
static void Main(string[] args) , and the calling method "ReceiveSessionMessage()" in it. The correct way of calling this function from the Program.cs was
static async Task Main(string[] args)
{
Console.WriteLine("Message Session Handler..");
await MessageSessionReceiver();
}
"ReceiveSessionMessage" function was an async function and the calling function did not have the await keyword mentioned due to which the program exited. After changing the syntax to add await, it worked.
Related
When I create a scheduled service bus message, both in Azure Portal and in my app using the Service bus producer code (below) and I receive a sequence number. I save it in my db.
Problem - When my Service bus consumer code is triggered by the dequeue of the scheduled message the sequence number is different than the one that was initially given to me by both the service bus producer code and through the Azure portal.
Shown here, where '13' is the sequnce number shown in Azure Portal screen.
Here is the code that receives the scheduled message and you can see the sequence number is different!
Here is my consumer code (don't think it matters)
private async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
JObject jsonObject = JObject.Parse(body);
var eventStatus = (string)jsonObject["EventStatus"];
await args.CompleteMessageAsync(args.Message);
// fetch row here by sequence number
// edit some data from entity, then save
int result = await dbContext.SaveChangesAsync();
}
Here is my producer code
public async Task<long> SendMessage(string messageBody, DateTimeOffset scheduledEnqueueTimeUtc)
{
await using (ServiceBusClient client = new ServiceBusClient(_config["ServiceBus:Connection"]))
{
ServiceBusSender sender = client.CreateSender(_config["ServiceBus:Queue"]);
ServiceBusMessage message = new ServiceBusMessage(messageBody);
var sequenceNumber = await sender.ScheduleMessageAsync(message, scheduledEnqueueTimeUtc);
return sequenceNumber;
}
}
From the documentation:
The SequenceNumber for a scheduled message is only valid while the message is in this state. As the message transitions to the active state, the message is appended to the queue as if had been enqueued at the current instant, which includes assigning a new SequenceNumber.
This is the code on my side:
using System;
using System.Threading;
using System.Threading.Tasks;
using Azure.Messaging.ServiceBus;
namespace ConsoleApp3
{
class Program
{
static string connectionString = "xxxxxx";
static string queueName = "myqueue";
static ServiceBusClient client;
static ServiceBusProcessor processor;
static async Task Main(string[] args)
{
client = new ServiceBusClient(connectionString);
processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions());
try
{
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.WriteLine("Wait for a minute and then press any key to end the processing");
Console.ReadKey();
Console.WriteLine("\nStopping the receiver...");
await processor.StopProcessingAsync();
Console.WriteLine("Stopped receiving messages");
}
finally
{
await processor.DisposeAsync();
await client.DisposeAsync();
}
}
static async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine($"Received: {body}");
Console.WriteLine($"ID: {args.Message.MessageId}");
await args.CompleteMessageAsync(args.Message);
}
static Task ErrorHandler(ProcessErrorEventArgs args)
{
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}
}
}
And it seems no problem on my side:
Message Id changed should be the message be thrown back by some reasons.
I'm facing a strange issue, and I ran out of the possible causes. The scenario is
Fetch incoming message from queue
Process it and then add new message to another queue
but the thing is, if I finish the long running task for the incoming message, and then try to add new message to another queue, I don't receive it. If I just add a face message to that another queue, then I am able to receive the real message after the long-running operation is finished. But why ? I don't want to put any fake messages to the queue, but without that my scenario doesn't work. Any ideas ?
public class WorkerRole : RoleEntryPoint
{
// QueueClient is thread-safe. Recommended that you cache
// rather than recreating it on every request
Microsoft.ServiceBus.Messaging.QueueClient Client;
ManualResetEvent CompletedEvent = new ManualResetEvent(false);
public override void Run()
{
MyResult result = null;
var queueClient = new Microsoft.Azure.ServiceBus.QueueClient("QueueConnectionString", "QueueName");
Client.OnMessage(async (receivedMessage) =>
{
try
{
using (Stream stream = receivedMessage.GetBody<Stream>())
{
using (StreamReader reader = new StreamReader(stream))
{
string json = reader.ReadToEnd();
OCRQueueItem_Incoming item = JsonConvert.DeserializeObject<IncomingClass>(json);
var someClass = new OCRManager();
var message = new Message(Encoding.UTF8.GetBytes("test 1"));
await queueClient.SendAsync(message);
result = new SomeManager().RunLongRunningTask(item); //it runs for 1-2min
}
}
}
catch (Exception ex) { }
finally
{
var json = JsonConvert.SerializeObject(result);
var message = new Message(Encoding.UTF8.GetBytes(json));
await queueClient.SendAsync(message);
}
});
CompletedEvent.WaitOne();
}
public override bool OnStart()
{
ServicePointManager.DefaultConnectionLimit = 12;
string connectionString = CloudConfigurationManager.GetSetting("Queue.ConnectionString");
Client = Microsoft.ServiceBus.Messaging.QueueClient.Create(connectionString);
return base.OnStart();
}
public override void OnStop()
{
Client.Close();
CompletedEvent.Set();
base.OnStop();
}
}
This is my working code in a console app:
Writer (working code):
static void Main(string[] args)
{
Console.WriteLine("Starting..");
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
QueueClient Client = QueueClient.CreateFromConnectionString(connectionString, "xxx");
// Create message, passing a string message for the body.
BrokeredMessage message = new BrokeredMessage("");
// Set some addtional custom app-specific properties.
message.Properties["UserCode"] = "HELLOOO22353";
message.Properties["UserId"] = "4511";
try
{
// Send message to the queue.
Client.Send(message);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
Console.WriteLine("Complete..");
//Console.ReadKey();
}
Reader: (working code)
static void Main(string[] args)
{
Console.WriteLine("Starting");
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
QueueClient Client = QueueClient.CreateFromConnectionString(connectionString, "xxx");
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromMinutes(1);
Client.OnMessage((message) =>
{
try
{
string sMessage = " UserCode: " + message.Properties["UserCode"];
Console.WriteLine("Found new User - " + sMessage);
// Remove message from queue.
message.Complete();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
// Indicates a problem, unlock message in queue.
message.Abandon();
}
}, options);
Console.ReadKey();
}
Now, this is the code I have in Azure functions which isn't working:
public static void Run(BrokeredMessage myMessage, TraceWriter log)
{
log.Info($"C# ServiceBus queue trigger function processed message: {myMessage}");
log.Info("1111");
log.Info(myMessage.MessageId); // this works
myMessage.Complete(); // fails: Microsoft.ServiceBus: Operation is not valid due to the current state of the object.
log.Info(myMessage.Properties["UserCode"].ToString()); // fails: myMessage.Properties is empty for some reason
}
What I'm failing to understand is why the Reader console app is able to read and complete message correctly but Azure function one (which is essentially based on the same idea) isn't. Both the codes are using the same version of Windows.ServiceBus package.
You do not need to complete the message yourself when using the ServiceBus trigger with Azure Functions. The ServiceBus trigger automatically uses PeekLock mode and will handle automatically completing, abandoning and renewing the message lock for you. You can also take control the finer details of this behavior via host.json settings for your function.
I could just write a long-running CLI app and run it, but I'm assuming it wouldn't comply to all the expectations one would have of a standards-compliant linux daemon (responding to SIGTERM, Started by System V init process, Ignore terminal I/O signals, etc.)
Most ecosystems have some best-practice way of doing this, for example, in python, you can use https://pypi.python.org/pypi/python-daemon/
Is there some documentation about how to do this with .Net Core?
I toyed with an idea similar to how .net core web host waits for shutdown in console applications. I was reviewing it on GitHub and was able to extract the gist of how they performed the Run
https://github.com/aspnet/Hosting/blob/15008b0b7fcb54235a9de3ab844c066aaf42ea44/src/Microsoft.AspNetCore.Hosting/WebHostExtensions.cs#L86
public static class ConsoleHost {
/// <summary>
/// Block the calling thread until shutdown is triggered via Ctrl+C or SIGTERM.
/// </summary>
public static void WaitForShutdown() {
WaitForShutdownAsync().GetAwaiter().GetResult();
}
/// <summary>
/// Runs an application and block the calling thread until host shutdown.
/// </summary>
/// <param name="host">The <see cref="IWebHost"/> to run.</param>
public static void Wait() {
WaitAsync().GetAwaiter().GetResult();
}
/// <summary>
/// Runs an application and returns a Task that only completes when the token is triggered or shutdown is triggered.
/// </summary>
/// <param name="host">The <see cref="IConsoleHost"/> to run.</param>
/// <param name="token">The token to trigger shutdown.</param>
public static async Task WaitAsync(CancellationToken token = default(CancellationToken)) {
//Wait for the token shutdown if it can be cancelled
if (token.CanBeCanceled) {
await WaitAsync(token, shutdownMessage: null);
return;
}
//If token cannot be cancelled, attach Ctrl+C and SIGTERN shutdown
var done = new ManualResetEventSlim(false);
using (var cts = new CancellationTokenSource()) {
AttachCtrlcSigtermShutdown(cts, done, shutdownMessage: "Application is shutting down...");
await WaitAsync(cts.Token, "Application running. Press Ctrl+C to shut down.");
done.Set();
}
}
/// <summary>
/// Returns a Task that completes when shutdown is triggered via the given token, Ctrl+C or SIGTERM.
/// </summary>
/// <param name="token">The token to trigger shutdown.</param>
public static async Task WaitForShutdownAsync(CancellationToken token = default (CancellationToken)) {
var done = new ManualResetEventSlim(false);
using (var cts = CancellationTokenSource.CreateLinkedTokenSource(token)) {
AttachCtrlcSigtermShutdown(cts, done, shutdownMessage: string.Empty);
await WaitForTokenShutdownAsync(cts.Token);
done.Set();
}
}
private static async Task WaitAsync(CancellationToken token, string shutdownMessage) {
if (!string.IsNullOrEmpty(shutdownMessage)) {
Console.WriteLine(shutdownMessage);
}
await WaitForTokenShutdownAsync(token);
}
private static void AttachCtrlcSigtermShutdown(CancellationTokenSource cts, ManualResetEventSlim resetEvent, string shutdownMessage) {
Action ShutDown = () => {
if (!cts.IsCancellationRequested) {
if (!string.IsNullOrWhiteSpace(shutdownMessage)) {
Console.WriteLine(shutdownMessage);
}
try {
cts.Cancel();
} catch (ObjectDisposedException) { }
}
//Wait on the given reset event
resetEvent.Wait();
};
AppDomain.CurrentDomain.ProcessExit += delegate { ShutDown(); };
Console.CancelKeyPress += (sender, eventArgs) => {
ShutDown();
//Don't terminate the process immediately, wait for the Main thread to exit gracefully.
eventArgs.Cancel = true;
};
}
private static async Task WaitForTokenShutdownAsync(CancellationToken token) {
var waitForStop = new TaskCompletionSource<object>();
token.Register(obj => {
var tcs = (TaskCompletionSource<object>)obj;
tcs.TrySetResult(null);
}, waitForStop);
await waitForStop.Task;
}
}
I tried adapting something like a IConsoleHost but quickly realized I was over-engineering it. Extracted the main parts into something like await ConsoleUtil.WaitForShutdownAsync(); that operated like Console.ReadLine
This then allowed the utility to be used like this
public class Program {
public static async Task Main(string[] args) {
//relevant code goes here
//...
//wait for application shutdown
await ConsoleUtil.WaitForShutdownAsync();
}
}
from there creating a systemd as in the following link should get you the rest of the way
Writing a Linux daemon in C#
The best I could come up with is based on the answer to two other questions: Killing gracefully a .NET Core daemon running on Linux and Is it possible to await an event instead of another async method?
using System;
using System.Runtime.Loader;
using System.Threading.Tasks;
namespace ConsoleApp1
{
public class Program
{
private static TaskCompletionSource<object> taskToWait;
public static void Main(string[] args)
{
taskToWait = new TaskCompletionSource<object>();
AssemblyLoadContext.Default.Unloading += SigTermEventHandler;
Console.CancelKeyPress += new ConsoleCancelEventHandler(CancelHandler);
//eventSource.Subscribe(eventSink) or something...
taskToWait.Task.Wait();
AssemblyLoadContext.Default.Unloading -= SigTermEventHandler;
Console.CancelKeyPress -= new ConsoleCancelEventHandler(CancelHandler);
}
private static void SigTermEventHandler(AssemblyLoadContext obj)
{
System.Console.WriteLine("Unloading...");
taskToWait.TrySetResult(null);
}
private static void CancelHandler(object sender, ConsoleCancelEventArgs e)
{
System.Console.WriteLine("Exiting...");
taskToWait.TrySetResult(null);
}
}
}
Implementing Linux Daemon or service for windows quite easy with single codebase using Visual Studio 2019. Just create project using WorkerService template. In my case I have
Coraval library to schedule the tasks.
Program.cs class
public class Program
{
public static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.Enrich.FromLogContext()
.WriteTo.File(#"C:\temp\Workerservice\logfile.txt").CreateLogger();
IHost host = CreateHostBuilder(args).Build();
host.Services.UseScheduler(scheduler =>
{
scheduler
.Schedule<ReprocessInvocable>()
.EveryThirtySeconds();
});
host.Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args).UseSystemd() //.UseWindowsService()
.ConfigureServices(services =>
{
services.AddScheduler();
services.AddTransient<ReprocessInvocable>();
});
}
ReprocessInvocable.cs class
public class ReprocessInvocable : IInvocable
{
private readonly ILogger<ReprocessInvocable> _logger;
public ReprocessInvocable(ILogger<ReprocessInvocable> logger)
{
_logger = logger;
}
public async Task Invoke()
{
//your code goes here
_logger.LogInformation("Information - Worker running at: {time}", DateTimeOffset.Now);
_logger.LogWarning("Warning - Worker running at: {time}", DateTimeOffset.Now);
_logger.LogCritical("Critical - Worker running at: {time}", DateTimeOffset.Now);
Log.Information("Invoke has called at: {time}", DateTimeOffset.Now);
}
}
For linux daemon use UseSystemd and for windows service use UseWindowsService as per the above code.
If you're trying to find something more robust, I found an implementation on Github that looks promising: .NET Core Application blocks for message-based communication. It uses Host, HostBuilder, ApplicationServices, ApplicationEnvironment, etc classes to implement a messaging service.
It doesn't quite look ready for black box reuse, but it seems like it could be a good starting point.
var host = new HostBuilder()
.ConfigureServices(services =>
{
var settings = new RabbitMQSettings { ServerName = "192.168.80.129", UserName = "admin", Password = "Pass#word1" };
})
.Build();
Console.WriteLine("Starting...");
await host.StartAsync();
var messenger = host.Services.GetRequiredService<IRabbitMQMessenger>();
Console.WriteLine("Running. Type text and press ENTER to send a message.");
Console.CancelKeyPress += async (sender, e) =>
{
Console.WriteLine("Shutting down...");
await host.StopAsync(new CancellationTokenSource(3000).Token);
Environment.Exit(0);
};
...
Have you tried Thread.Sleep (Timeout.Infinite) ?
using System;
using System.IO;
using System.Threading;
namespace Daemon {
class Program {
static int Main(string[] args) {
if (Environment.OSVersion.Platform == PlatformID.Win32NT) {
Log.Critical("Windows is not supported!");
return 1;
}
Agent.Init();
Agent.Start();
if (Agent.Settings.DaemonMode || args.FirstOrDefault() == "daemon") {
Log.Info("Daemon started.");
Thread.Sleep(Timeout.Infinite);
}
Agent.Stop();
}
}
}
I'm not sure it is production grade, but for a quick and dirty console app this works well:
await Task.Delay(-1); //-1 indicates infinite timeout
I've tried to come up with something from the example in the WebJobsSDK gitHub
var eventHubConfig = new EventHubConfiguration();
string eventHubName = "MyHubName";
eventHubConfig.AddSender(eventHubName,"Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=SendRule;SharedAccessKey=xxxxxxxx");
eventHubConfig.AddReceiver(eventHubName, "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=ReceiveRule;SharedAccessKey=yyyyyyy");
config.UseEventHub(eventHubConfig);
JobHost host = new JobHost(config);
But I'm afraid that's not far enough for someone of my limited "skillset"!
I can find no instance of JobHostConfiguration that has a UseEventHub property (using the v1.2.0-alpha-10291 version of the Microsoft.AzureWebJobs package), so I can't pass the EventHubConfiguration to the JobHost.
I've used EventHub before, not within the WebJob context. I don't see if the EventHostProcessor is still required if using the WebJob triggering...or does the WebJob trigger essentially act as the EventHostProcessor?
Anyway, if anyone has a more complete example for a simpleton like me that would be really sweet! Thanks
From the documentation here, you should have all the information you need.
What you are missing is a reference of the Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 nuget package.
The UseEventHub is an extension method that is declared in this package.
Otherwise your configuration seems ok.
Here is an example on how to receive or send messages from/to an EventHub:
public class BasicTest
{
public class Payload
{
public int Counter { get; set; }
}
public static void SendEvents([EventHub("MyHubName")] out Payload x)
{
x = new Payload { Counter = 100 };
}
public static void Trigger(
[EventHubTrigger("MyHubName")] Payload x,
[EventHub("MyHubName")] out Payload y)
{
x.Counter++;
y = x;
}
}
EventProcessorHost is still required, as the WebJob just provides the hosting environment for running it. As far as I know, EventProcessorHost is not integrated so deeply into WebJob, so its triggering mechanism cannot be used for processing EventHub messages. I use WebJob for running EventProcessorHost continuously:
public static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
try
{
using (var shutdownWatcher = new WebJobsShutdownWatcher())
{
await Console.Out.WriteLineAsync("Initializing...");
var eventProcessorHostName = "eventProcessorHostName";
var eventHubName = ConfigurationManager.AppSettings["eventHubName"];
var consumerGroupName = ConfigurationManager.AppSettings["eventHubConsumerGroupName"];
var eventHubConnectionString = ConfigurationManager.ConnectionStrings["EventHub"].ConnectionString;
var storageConnectionString = ConfigurationManager.ConnectionStrings["EventHubStorage"].ConnectionString;
var eventProcessorHost = new EventProcessorHost(eventProcessorHostName, eventHubName, consumerGroupName, eventHubConnectionString, storageConnectionString);
await Console.Out.WriteLineAsync("Registering event processors...");
var processorOptions = new EventProcessorOptions();
processorOptions.ExceptionReceived += ProcessorOptions_ExceptionReceived;
await eventProcessorHost.RegisterEventProcessorAsync<CustomEventProcessor>(processorOptions);
await Console.Out.WriteLineAsync("Processing...");
await Task.Delay(Timeout.Infinite, shutdownWatcher.Token);
await Console.Out.WriteLineAsync("Unregistering event processors...");
await eventProcessorHost.UnregisterEventProcessorAsync();
await Console.Out.WriteLineAsync("Finished.");
}
catch (Exception ex)
{
await HandleErrorAsync(ex);
}
}
}
private static async void ProcessorOptions_ExceptionReceived(object sender, ExceptionReceivedEventArgs e)
{
await HandleErrorAsync(e.Exception);
}
private static async Task HandleErrorAsync(Exception ex)
{
await Console.Error.WriteLineAsync($"Critical error occured: {ex.Message}{ex.StackTrace}");
}