I have an asp.net MVC 5 application. Now I want to have a new feature allow me to receive GPS signal from devices and update devices real time in web client side.
My plan is to use RabbitMQ to handle message queue and SignalR to notify clients for postion update
My code in Rabbit consumer console application like this
var client = GlobalHost.ConnectionManager.GetHubContext<MyHub>().Clients;
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
var queueName = channel.QueueDeclare().QueueName;
channel.ExchangeDeclare("mychannel", "fanout");
channel.QueueBind(queueName, "mychannel", "");
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine("[x] Receive {0}", message);
client.All.hello(message);
};
channel.BasicConsume(queue: queueName,
autoAck: true,
consumer: consumer);
Console.WriteLine("Press Enter to quit");
Console.ReadLine();
}
}
But the code is not fired when I produce a message to RabbitMQ. If I remove code of signalR client:
var client = GlobalHost.ConnectionManager.GetHubContext<MyHub>().Clients;
things work as expected and I can receive message as usual
My question is:
Is my approach correct
if the approach correct then how to make it work?
Many thanks
Related
I created a simple Azure Service bus (Queue) and a client that is sending message to service bus. Using below code to send message:
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Configuration;
public async Task SendMessageAsync<T>(T message, string queueName)
{
try
{
var queueClient = new QueueClient(_config.GetConnectionString("AzureServiceBus"), queueName);
string messageBody = JsonSerializer.Serialize(message);
var byteMessage = new Message(Encoding.UTF8.GetBytes(messageBody));
queueClient.SendAsync(byteMessage);
Console.WriteLine((message as Employee).FirstName);
}
catch (Exception ex)
{
var c = ex;
}
}
Sending message using:
using SenderApp;
Console.WriteLine("Hello, World!");
QueueService service = new QueueService();
for (int i = 0; i < 100; i++)
{
Employee e = new Employee();
e.FirstName = "1 " + i.ToString();
e.LastName = "2 " + i.ToString();
service.SendMessageAsync<Employee>(e, "employeequeue");
}
When I try to see active messages, There is nothing in the queue:
However I do see some traffic. But the number of message I sent (over 100) is not equal to number of incoming request show (62) at the bottom of the image. I am not sure what is happening to my messages? This defeats the purpose of the queue.
Please guide me why I am not seeing any messages. What is the best way to handle this ?
I am using following nuget packages:
<PackageReference Include="Microsoft.Azure.ServiceBus" Version="5.2.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="6.0.0" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.1" />
A message sent to an Azure Service Bus queue will be delivered to the queue unless operation is failing. In that case, an exception will be thrown. Check the following:
Exception handling doesn't swollow exceptions
Await asynchronous send operations to ensure messages are dispatched
Namespace/queue used for sending is what you use to receive
There are no competing consumers, actively receiving messages from the queue.
Validate TCP ports needed for AMQP are not blocked. If those ports are blocked, you could configure your client to use WebSockets.
So I still dont know what caused this issue. But I realized Microsoft.Azure.ServiceBus package was deprecated and later I started using Azure.Messaging.ServiceBus package to connect to service bus and things started to work.
I used following code to send message to queue:
string connectionString = "Endpoint=sb://test.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=f3f+qMYTyVwE18YNl5J6ygJFi30v6J/Smph5HZvyQyE=";
string queueName = "employeequeue";
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);
// create the sender
ServiceBusSender sender = client.CreateSender(queueName);
// create a message that we can send. UTF-8 encoding is used when providing a string.
ServiceBusMessage message = new ServiceBusMessage("Hello world! " + id);
// send the message
await sender.SendMessageAsync(message);
return "Sent";
Used following code to receive message:
string queueName = "employeequeue";
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);
// create a receiver that we can use to receive and settle the message
ServiceBusReceiver receiver = client.CreateReceiver(queueName);
// the received message is a different type as it contains some service set properties
ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();
string body = receivedMessage.Body.ToString();
// complete the message, thereby deleting it from the service
await receiver.CompleteMessageAsync(receivedMessage);
More info is available # https://github.com/Azure/azure-sdk-for-net/blob/Azure.Messaging.ServiceBus_7.7.0/sdk/servicebus/Azure.Messaging.ServiceBus/README.md
I have an IoT device that is connected to Azure event hub. I'm trying to make this device communicate with azure databricks and azure event hub is placed in between as a middleware. The problem is that after we are able to send messages via ".NET framework", it is never shown in messages received in "python" command line (we should do that as we work separately for each part)
I followed the guidelines .NET framework as sender and python as receiver, and this doesn't work.
I am seeing that there are spikes in the request and message graphs under event hub stream instances, but it just never shows in the receiver
==================================UPDATE==================================
Just deleted the eventhub and recreated and it seems work.
However, messages are received in the form of long strings something like this below:
Received: 662a2a44-4414-4cb5-a9e9-a08d12a417e0
Received: b68ef8f8-305f-4726-84e4-f35b76de30c5
Received: e94dfb73-972c-47b4-baef-1ab41b06be28
Received: 8eda384d-f79d-4cdf-9db3-fe5c2156553b
Received: d154283f-a8c2-4a4c-a7d5-e8d8129b568d
Received: 3e3e190e-f883-416c-a5be-b8cd8547d152
Received: e218c63b-85b3-4f4f-8f04-cb5ffc6d8921
Received: 0adec5ad-e351-4151-ba56-01093e0f383d
Received 8 messages in 0.05406975746154785 seconds
This happens when I read the messages in format below:
print("Received: {}".format(event_data.body_as_str(encoding='UTF-8')))
I just give it a try, and I can repro your issue. And here are something you need to check.
1.In you sender(in c#), you should make sure your message to send is correct. Like below:
static void SendingRandomMessages()
{
var eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);
int i = 0;
while (true)
{
try
{
// make sure the message is correct.
var message = i+":"+Guid.NewGuid().ToString()+":"+DateTime.Now;
Console.WriteLine("{0} > Sending message: {1}", DateTime.Now, message);
var myeventdata = new EventData(Encoding.UTF8.GetBytes(message));
eventHubClient.Send(myeventdata);
i++;
//eventHubClient.Send(new EventData(Encoding.UTF8.GetBytes(message)));
}
catch (Exception exception)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("{0} > Exception: {1}", DateTime.Now, exception.Message);
Console.ResetColor();
}
Thread.Sleep(2000);
}
}
2.There seems some delay for the receiver(in python), so I execute the python receiver about 3 times, and I can see the expected output. The screenshot as below:
Update 1022: as we discussed in the comment, there is a solution for fixing just receiving even / odd number event data.
In you sender(in c#), use the code below, which sends event data to partition 0:
static void SendingRandomMessages()
{
var eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);
var myclient = eventHubClient.CreatePartitionedSender("0");
int i = 30;
while (true)
{
var message = i + ":" + Guid.NewGuid().ToString() + ":" + DateTime.Now;
Console.WriteLine("{0} > Sending message: {1}", DateTime.Now, message);
var myeventdata = new EventData(Encoding.UTF8.GetBytes(message));
myclient.Send(myeventdata);
i++;
Thread.Sleep(2000);
}
}
then in your receiver(in python), specify the partition to 0(use this PARTITION = "0"), then you can get all the event data.
There is some documentation on using Amazon SQS as an MQ Server forServiceStack Messaging API
But the message publisher is frontend web code and when you dig into the Email Contacts demo app, it is using Rabbit MQ.
There is a ServiceStack Email Contacts AWS App demo but it doesn't use the Messaging API.
Trying to use the Rabbit MQ Integration Test in Email Contacts as an example:
[Test]
public void Can_Send_Email_via_MqClient()
{
var mqFactory = new RabbitMqMessageFactory();
using (var mqClient = mqFactory.CreateMessageQueueClient())
{
mqClient.Publish(new EmailContact { ContactId = 1, Subject = "UnitTest MQ Email #1", Body = "Body 1" });
mqClient.Publish(new EmailContact { ContactId = 1, Subject = "UnitTest MQ Email #2", Body = "Body 2" });
}
}
I quickly confused and lead astray and the ServiceStack API for Sqs seems very different than RabbitMQ. I cannot even seem to be able to use a strongly type POCO as a Message:
[Fact(DisplayName = "Tests that a successful message is published and received")]
public async void TestMessage()
{
var mqFactory = new SqsConnectionFactory("awskey", "awssecret", RegionEndpoint.USWest1);
using (IAmazonSQS mqClient = mqFactory.GetClient())
{
var req = new SendMessageRequest("query", "hello");
await mqClient.SendMessageAsync(req);
//mqClient.Publish(new Hello { Name = "World" });
//var rec = new ReceiveMessageRequest();
//await mqClient.Re
//var responseMsg = mqClient.Get<HelloResponse>(QueueNames<HelloResponse>.In);
//mqClient.Ack(responseMsg);
//responseMsg.GetBody().Result //= Hello, World!
}
}
Is there an example app using the ServiceStack Messaging API with SQS as the MQ Server?
There is a ServiceStack Email Contacts AWS App demo but it doesn't use the Messaging API.
Note AWS Apps does register the AWS SqsMqServer:
//EmailContacts
ConfigureSqsMqServer(container);
//..
private void ConfigureSqsMqServer(Container container)
{
container.Register<IMessageService>(c => new SqsMqServer(
AwsConfig.AwsAccessKey, AwsConfig.AwsSecretKey, RegionEndpoint.USEast1) {
DisableBuffering = true,
});
var mqServer = container.Resolve<IMessageService>();
mqServer.RegisterHandler<EmailContacts.EmailContact>(ExecuteMessage);
mqServer.Start();
}
There's also a number of examples in SqsMqServerTests.cs.
If you want to use ServiceStack MQ's high-level APIs, you'd need to use ServiceStack's MQ classes instead of AWS's SQS classes directly.
Basically it works like every other MQ Server, you can fetch an MQ Client from the IMessageFactory or IMessageService (registered in your AppHost) and use it to publish DTOs:
var mqFactory = HostContext.TryResolve<IMessageFactory>(); //or
//var mqFactory = HostContext.TryResolve<IMessageService>().MessageFactory;
using (var mqClient = mqFactory.CreateMessageQueueClient())
{
mqClient.Publish(new Hello { Name = "World" });
}
Although the preferred API within a ServiceStack Service is to use PublishMessage():
PublishMessage(new Hello { Name = "World" });
Note requests to the /oneway pre-defined endpoint are automatically published to the registered MQ Server.
In client Apps without a registered SqsMqServer you'd create a SqsMqMessageFactory:
var mqFactory = new SqsMqMessageFactory(new SqsQueueManager(...));
using (var mqClient = mqFactory.CreateMessageQueueClient())
{
mqClient.Publish(new Hello { Name = "World" });
}
Given this snippet of code:
//DirectApi
mqServer.RegisterHandler<LeadInformationInfo>(m =>
{
repository.SaveMessage(m as Message);
LeadInformationInfoResponse response = new LeadInformationInfoResponse();
try
{
var client = new JsonServiceClient(settingsFactory.GetMasterSetting("ProcessorApi:baseUri"));
response = client.Post(m.GetBody());
}
catch (WebServiceException webServiceException)
{
_log.Error("RegisterHandler<LeadInformationInfo>", webServiceException);
response = ((LeadInformationInfoResponse) webServiceException.ResponseDto);
response.CorrelationId = m.Id;
}
// Log response message here
return response;
}, 1);
I've gone to great lengths to make sure that a correlationId based off the original message Id property is propagated through the life of this message and any child messages spawned from this action. How do I get a handle on the response message so that I may log it in the handler? I only have access to the ResponseDto and not the message.
One of the reasons for this request is that the message queue client does not have access to the database, only the process that has the handler registered does. Hope that explains the situation better.
Just to clarify, this question is about persisting a MQ Response Message in the handler, the correlation Id is something that all messages in 1 request/response workflow will share. I'm also using ServiceStack ORMlite to persist the Message object, so querying this table by ID for troubleshooting is paramount.
Thank you,
Stephen
You're calling a WebService from within your MQ Handler:
var client = new JsonServiceClient(...);
response = client.Post(m.GetBody());
So there is no MQ Response which is only available in MQ Services. Although the WebService will return the response for the request that's sent so you can either use the CorrelationId on the MQ Request, otherwise you can have your Response DTO implement an interface like IHasCorrelationId and get it that way, e.g:
var correlationResponse = response as IHasCorrelationId;
if (correlationResponse != null)
{
var correlationId = correlationResponse.CorrelationId;
}
Create your own Instance of Message
As the Message<T> class is just a POCO if you wanted to create your own you can intialize your own instance:
var mqResponse = new Message<Response>(response);
If you only had the runtime late-bound type info, you can create one with:
var mqResponse = MessageFactory.Create(response);
Use RabbitMQ Message Filters
If you just wanted to log incoming and outgoing messages you can use the RabbitMQ Message Filters, e.g:
var mqServer = new RabbitMqServer("localhost")
{
PublishMessageFilter = (queueName, properties, msg) => {
properties.AppId = "app:{0}".Fmt(queueName);
},
GetMessageFilter = (queueName, basicMsg) => {
var props = basicMsg.BasicProperties;
receivedMsgType = props.Type; //automatically added by RabbitMqProducer
receivedMsgApp = props.AppId;
}
};
Is there way to make method
myMessageService.CreateMessageQueueClient().Publish(myMessage);
broadcast messages to all receivers?
The problem is, that RegisterHandler<T> internally uses the type of T to build the queue name it listens to. So the only chance you have is to go off the track with the following solution by utilizing a custom fanout exchange to multiple queues:
var fanoutExchangeName = string.Concat(QueueNames.Exchange,
".",
ExchangeType.Fanout);
At some point of your system you have to ensure the exchange with the following code:
var rabbitMqServer = new RabbitMqServer();
var messageProducer = (RabbitMqProducer) rabbitMqServer.CreateMessageProducer();
var channel = messageProducer.Channel; // you can use any logic to acquire a channel here - this is just a demo
channel.ExchangeDeclare(fanoutExchangeName,
ExchangeType.Fanout,
durable: true,
autoDelete: false,
arguments: null);
Now we can publish messages to this fanout:
var message = new Message<T>(yourInstance);
messageProducer.Publish(QueueNames<T>.In, // routing key
message, // message
fanoutExchangeName); // exchange
So now the message gets published to our exchange, but we need to bind queues to the exchange in the consuming components, which we do with:
var rabbitMqServer = new RabbitMqServer();
var messageQueueClient = (RabbitMqQueueClient) rabbitMqServer.CreateMessageQueueClient();
var channel = messageQueueClient.Channel; // you just need to get the channel
var queueName = messageQueueClient.GetTempQueueName();
channel.QueueBind(queueName, // queue
fanoutExchangeName, // exchange
QueueName<T>.In); // routing key
The queue is automatically deleted after the last (and only) consumer disconnects and will not survive a restart of RabbitMq.
The hacky part is now the listening though ...
var consumer = new RabbitMqBasicConsumer(channel);
channel.BasicConsume(queueName,
false,
consumer);
Task.Run(() =>
{
while (true)
{
BasicGetResult basicGetResult;
try
{
basicGetResult = consumer.Queue.Dequeue();
}
catch (EndOfStreamException endOfStreamException)
{
// this is ok
return;
}
catch (OperationInterruptedException operationInterruptedException)
{
// this is ok
return;
}
catch (Exception ex)
{
throw;
}
var message = basicGetResult.ToMessage<T>();
// TODO processing
}
});
This solution does not provide any auto-reconnect, filters, or other stuff though.
A basic walkthrough is available here.
Edit:
One thing that just got to my mind: You can use a ServiceStack.Messaging.MessageHandler<T> instance to provide replies and retries with ease.