Service bus Namespace: namespace1
Topic: source
Service bus Namespace: namespace2
Topic: Target
we receive messages to source as byte[]. I would like to copy the same message in source and move it to target. below is azure function i am using to copy.
but when writing message to target topic i am unable to get the same message I have received from source.
My target queueitem is coming in as System.Byte[] instead of the message.
[Function("CopyTopic")]
[ServiceBusOutput("TargetTopicName", ServiceBusEntityType.Topic, Connection = "TargetServiceBus")]
public async Task<byte[]> RunAsync([ServiceBusTrigger("%SourceTopicName%", "%SourceSubscriptionName%",Connection= "AzureWebJobsServiceBus")] byte[] myQueueItem,
FunctionContext context)
{
return myQueueItem;
}
Related
with servicebus_client:
receiver = servicebus_client.get_subscription_receiver(topic_name=TOPIC_NAME, subscription_name=SUBSCRIPTION_NAME, max_wait_time=5)
with receiver:
for msg in receiver:
print("Received: " + str(msg))
receiver.complete_message(msg)
This code is taken from the Azure service bus code snippet in Microsoft portal. The code works fine till the it sends the messages, but when it reaches this part of the code, it is not getting in the for loop. The print statement is not getting execute. But I can see the Incoming and outcoming messages on the particular metrics page of that topic. Can I get some help here as I am new to Azure Service Bus?
Code for receive message from topic. its working as expected. you can try again with all library reference and follow all steps.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Azure.Messaging.ServiceBus;
namespace SubscriptionReceiver
{
internal class Program
{
// connection string to your Service Bus namespace
static string connectionString = "Endpoint=sb://xxx/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx";
// name of the Service Bus topic
static string topicName = "mytopic";
// name of the subscription to the topic
static string subscriptionName = "S1";
// the client that owns the connection and can be used to create senders and receivers
static ServiceBusClient client;
// the processor that reads and processes messages from the subscription
static ServiceBusProcessor processor;
static async Task Main()
{
// The Service Bus client types are safe to cache and use as a singleton for the lifetime
// of the application, which is best practice when messages are being published or read
// regularly.
//
// Create the clients that we'll use for sending and processing messages.
client = new ServiceBusClient(connectionString);
// create a processor that we can use to process the messages
processor = client.CreateProcessor(topicName, subscriptionName, new ServiceBusProcessorOptions());
try
{
// add handler to process messages
processor.ProcessMessageAsync += MessageHandler;
// add handler to process any errors
processor.ProcessErrorAsync += ErrorHandler;
// start processing
await processor.StartProcessingAsync();
Console.WriteLine("Wait for a minute and then press any key to end the processing");
Console.ReadKey();
// stop processing
Console.WriteLine("\nStopping the receiver...");
await processor.StopProcessingAsync();
Console.WriteLine("Stopped receiving messages");
}
finally
{
// Calling DisposeAsync on client types is required to ensure that network
// resources and other unmanaged objects are properly cleaned up.
await processor.DisposeAsync();
await client.DisposeAsync();
}
}
// handle received messages
static async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine($"Received: {body} from subscription: {subscriptionName}");
// complete the message. messages is deleted from the subscription.
await args.CompleteMessageAsync(args.Message);
}
// handle any errors when receiving messages
static Task ErrorHandler(ProcessErrorEventArgs args)
{
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}
}
}
Output :- Receive Message from Topic Subscription
I have created an Event Hub Namespace and 2 event hubs. I defined a Shared Access Policy (SAP) on the Event Hub Namespace. However, when I use the connection string defined on the namespace, I am able to send events to only one of the hubs even though I create the client using the correct event hub name
function void SendEvent(connectionString, eventHubName){
await using(var producerClient = new EventHubProducerClient(connectionString, eventHubName)) {
// Create a batch of events
using EventDataBatch eventBatch = await producerClient.CreateBatchAsync();
var payload = GetEventModel(entity, entityName);
// Add events to the batch. An event is a represented by a collection of bytes and metadata.
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes(payload.ToString())));
// Use the producer client to send the batch of events to the event hub
await producerClient.SendAsync(eventBatch);
System.Diagnostics.Debug.WriteLine($"Event for {entity} sent to Hub {eventHubName}");
}
}
The above code is called for sending events to Hub1 and Hub2. When I use the connection string from the SAP defined on the Namespace, I can only send events to Hub1 or Hub2 whichever happens to be called first. I am specifying the eventHubName as Hub1 or Hub2 as appropriate.
I call the function SendEvent in my calling code.
The only way I can send to both hubs is to define SAP on each hub and use that connection string when creating the EventHubProducer
Am I missing something or is this by design?
I did a quick test at my side, and it can work well at my side.
Please try the code below, and let me know if it does not meet your need:
class Program
{
//the namespace level sas
private const string connectionString = "Endpoint=sb://yyeventhubns.servicebus.windows.net/;SharedAccessKeyName=mysas;SharedAccessKey=xxxx";
//I try to send data to the following 2 eventhub instances.
private const string hub1 = "yyeventhub1";
private const string hub2 = "yyeventhub2";
static async Task Main()
{
SendEvent(connectionString, hub1);
SendEvent(connectionString, hub2);
Console.WriteLine("**completed**");
Console.ReadLine();
}
private static async void SendEvent(string connectionString, string eventHubName)
{
// Create a producer client that you can use to send events to an event hub
await using (var producerClient = new EventHubProducerClient(connectionString, eventHubName))
{
// Create a batch of events
using EventDataBatch eventBatch = await producerClient.CreateBatchAsync();
// Add events to the batch. An event is a represented by a collection of bytes and metadata.
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("First event: "+eventHubName)));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Second event: "+eventHubName)));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Third event: "+eventHubName)));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Fourth event: " + eventHubName)));
eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Fifth event: " + eventHubName)));
// Use the producer client to send the batch of events to the event hub
await producerClient.SendAsync(eventBatch);
Console.WriteLine("A batch of 3 events has been published to: "+ eventHubName);
}
}
}
After running the code, I can see the data are sent to both of the 2 eventhub instances. Here is the screenshot:
We followed this example (http://masstransit-project.com/MassTransit/usage/azure-functions.html) to try to set up Azure Functions as Azure Service Bus event (topic) subscribers using MassTransit (for .Net CORE 2.1, Azure Functions 2.0).
When using Azure Webjobs this is as simple as using RabbitMQ, configure the publisher, let the subscriber configure and set up its queue, and have Masstransit automatically create one topic per event, redirect to queue and to "queue_error" after all retries have failed. You do not have to setup anything manually.
But with Azure Functions we seem to manually (through Service Bus Explorer or ARM templates) have to add the subscribers to the topic (which is created by the publisher on the first event it publishes) and the queues as well (though these don't even seem to be necessary, the events are handled directly by the consuming Azure Function topic subscribers.).
Maybe we are doing something wrong, I cannot see from the docs that MT will not, as it normally does, set up the subscriber andd creating queues when using Azure Functions. But it works, except for when the consumer throws an exception and after all setup retries have been executed. We simply do not get the event in the deadletter queue and the normally MT-generated error queue does not even get generated.
So how do we get MT to create the error queues, and MOVE the failed events there?
Our code:
[FunctionName("OrderShippedConsumer")]
public static Task OrderShippedConsumer(
[ServiceBusTrigger("xyz.events.order/iordershipped", "ordershippedconsumer-queue", Connection = "AzureServiceBus")] Message message,
IBinder binder,
ILogger logger,
CancellationToken cancellationToken,
ExecutionContext context)
{
var config = CreateConfig(context);
var handler = Bus.Factory.CreateBrokeredMessageReceiver(binder, cfg =>
{
var serviceBusEndpoint = Parse.ConnectionString(config["AzureServiceBus"])["Endpoint"];
cfg.CancellationToken = cancellationToken;
cfg.SetLog(logger);
cfg.InputAddress = new Uri($"{serviceBusEndpoint}{QueueName}");
cfg.UseRetry(x => x.Intervals(TimeSpan.FromSeconds(5)));
cfg.Consumer(() => new OrderShippedConsumer(cfg.Log, config));
});
return handler.Handle(message);
}
And the Consumer code:
public OrderShippedConsumer(ILog log, IConfigurationRoot config)
{
this.config = config;
this.log = log;
}
public async Task Consume(ConsumeContext<IOrderShipped> context)
{
// Handle the event
}
}
Iam trying to make a simple app for my raspberry pi that will send a message to the IOThub and then try to receive a response however nothing is happening.
I copied the connectionstring from the device controller. Ofcource I hidden it for this question.
I see it printing out that the message was succesfully sended to the iothub but when I check the iothub I see 0 received messages.
Iam using the free tier of the iothub is this a limitation?
public sealed partial class MainPage : Page
{
private const string DeviceConnectionString = "Hidden";
private readonly DeviceClient _deviceClient;
public MainPage()
{
this.InitializeComponent();
_deviceClient = DeviceClient.CreateFromConnectionString(DeviceConnectionString, TransportType.Amqp); //Already tried using different transport types but no succes.
}
public async Task SendEvent()
{
Debug.WriteLine("\t{0}> Sending message", DateTime.Now.ToLocalTime());
var commandMessage = new Message(Encoding.ASCII.GetBytes("Cloud to device message."));
await _deviceClient.SendEventAsync(commandMessage);
Debug.WriteLine("Succesfully sended message to IotHub");
}
public async Task ReceiveCommands()
{
Debug.WriteLine("\nDevice waiting for commands from IoTHub...\n");
while (true)
{
var receivedMessage = await _deviceClient.ReceiveAsync();
if (receivedMessage != null)
{
var messageData = Encoding.ASCII.GetString(receivedMessage.GetBytes());
Debug.WriteLine("\t{0}> Received message: {1}", DateTime.Now.ToLocalTime(), messageData);
var propCount = 0;
foreach (var prop in receivedMessage.Properties)
{
Debug.WriteLine("\t\tProperty[{0}> Key={1} : Value={2}", propCount++, prop.Key, prop.Value);
}
await _deviceClient.CompleteAsync(receivedMessage);
Debug.WriteLine("Finishing recieving message");
}
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
private async void Button_Click(object sender, Windows.UI.Xaml.RoutedEventArgs e)
{
Debug.WriteLine("Sending event");
await SendEvent();
await ReceiveCommands();
Debug.WriteLine("Received commands");
}
}
It is nothing to do with free tier of the iothub. There is no such limitation.
You can't receive Device-To-Cloud(D2C) messages using DeviceClient that you used in ReceiveCommands(). It is by designed. You look like having some misunderstanding of Azure IoT Hub message types and SDKs.
There are two kinds of message types: Device-To-Cloud(D2C) message and Cloud-To-Device(C2D) message.
And two kinds of SDKs: device SDK and service SDK.
Device SDK is used to connect to and send D2C messages to Azure IoT Hub.
While service SDK is used to manage and send C2D messages to devices.
So, if you send C2D messages to device in Device Explorer you will receive these messages in your ReceiveCommands method.
If you want to receive D2C message you can utilize Event Hub-compatible endpoint(messages/events). Here is a console sample you can reference. But this is can't be done in UWP due to service bus not supported in UWP.
I send the message to azure event hub. But I am not able to download the message from event hub.
enter code here
string eventHubConnectionString = "<connection string>";
string eventHubName = "<event Hub name>";
string storageAccountName = "<event hub storage>";
string storageAccountKey = "<storage Key>";
string storageConnectionString = string.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}",storageAccountName, storageAccountKey);
EventProcessorHost eventProcessorHost = new EventProcessorHost("message", eventHubName, EventHubConsumerGroup.DefaultGroupName, eventHubConnectionString, storageConnectionString);
eventProcessorHost.RegisterEventProcessorAsync<SimpleEventProcessor>().Wait();
IEventProcessor:
enter code here
class SimpleEventProcessor : IEventProcessor
{
Stopwatch checkpointStopWatch;
async Task IEventProcessor.CloseAsync(PartitionContext context, CloseReason reason)
{
Console.WriteLine(string.Format("Processor Shuting Down. Partition '{0}', Reason: '{1}'.", context.Lease.PartitionId, reason.ToString()));
if (reason == CloseReason.Shutdown)
{
await context.CheckpointAsync();
}
}
Task IEventProcessor.OpenAsync(PartitionContext context)
{
Console.WriteLine(string.Format("SimpleEventProcessor initialize. Partition: '{0}', Offset: '{1}'", context.Lease.PartitionId, context.Lease.Offset));
this.checkpointStopWatch = new Stopwatch();
this.checkpointStopWatch.Start();
return Task.FromResult<object>(null);
}
async Task IEventProcessor.ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (EventData eventData in messages)
{
string data = Encoding.UTF8.GetString(eventData.GetBytes());
Console.WriteLine(string.Format("Message received. Partition: '{0}', Data: '{1}'",
context.Lease.PartitionId, data));
}
//Call checkpoint every 5 minutes, so that worker can resume processing from the 5 minutes back if it restarts.
if (this.checkpointStopWatch.Elapsed > TimeSpan.FromMinutes(5))
{
await context.CheckpointAsync();
lock (this)
{
this.checkpointStopWatch.Reset();
}
}
}
}
It show the following error
Aggregate exception handling. One or more error occurred.
Message details: No such host is known
what is EventProcessor host name?
It show error at this line: eventProcessorHost.RegisterEventProcessorAsync().Wait();
it is not calling IEventprocessor. Is it have any other method for consuming message from event hub?
You can trace your exception and look for inner exception while debugging so this should give you a clue whats the real reason.. I had this stupid exception too and it was because when you use the eventHubName variable with EventProcessorHost it should be in lowercase, (containing only letters/numbers and '-' which has to be followed by letter or number which means that '--' is not supported. eventHubName should also begin with a letter)
Even if the event hub name is "myEventHub123" your variable has to be like:
string eventHubName = "myeventhub123";
Hope this will help someone..
I have had success building a event processor using the sample code located here.
It is hard to tell from your sample code what the error is because it could be related to a typo in your connection string/eventhub/storage account name since it is not provided (you did right, don't post your connection string with sensitive data).
The difference between the way the example loads the event hub information from the connection string and the way the code you provided is how the information is provided through the Evenhub Client. Try update the way you build your EventProcessorHost like the example below:
EventHubClient eventHubClient = EventHubClient.CreateFromConnectionString(eventHubConnectionString, this.eventHubName);
// Get the default Consumer Group
defaultConsumerGroup = eventHubClient.GetDefaultConsumerGroup();
string blobConnectionString = ConfigurationManager.AppSettings["AzureStorageConnectionString"]; // Required for checkpoint/state
eventProcessorHost = new EventProcessorHost("singleworker", eventHubClient.Path, defaultConsumerGroup.GroupName, this.eventHubConnectionString, blobConnectionString);
eventProcessorHost.RegisterEventProcessorAsync<SimpleEventProcessor>().Wait();