I am implementing an azure event hub subscriber which is successfully able to connect but it's not receiving the message for a particular event hub in an event hub namespace. But it's working as expected for the other event hub in the same namespace. there is no error in the logs. Is there any way to find out the root cause of the issue?
I am successfully able to publish the message with the same configuration.
Publisher and Subscriber are running on the same machine.
I have implemented the publisher and subscriber in Dapr.
Both publisher and subscriber logs are exactly the same in Dapr, So I think it's an issue with the storage account.
Adding more information:
I have tried my application with a different storage account and it's working as expected. So what is the issue? How can I resolve it? I want to work with my old storage account.
All issues
There are multiple problems put together in your question, which can be tested separately. (E.g. event hubs, dapr, storage account ...etc.) It is critical to have a granular understanding of these problems, so I would try to test them one-by-one.
Event Hub issues
You can test your Event Hub flow separately by creating simple "hello world" clients locally and reproducing a relationship between them based on your production scenario.
E.g.
try to connect to an event hub with exactly one publisher and exactly one consumer,
try to publish the same message with one publisher to multiple event hubs and see if you can consume them with separate consumers,
...etc.
Based on your comment, I will list 2 example problem scenarios:
Problem 1: "I can access one Event Hub in the namespace, but can't access another in the same namespace."
You can configure "Shared Access Policies" for the scope of the Event Hub Namespace, which will be applied to all event hubs in that namespace.
You can also configure the same policies for the scope of the Event Hub, which will be applied to that single hub only.
Make sure to use the appropriate scope in Azure portal while trying to use the connection strings from your code.
Problem 2: "I can publish messages from a client to an Event Hub, but cannot consume messages with the same client, from the same machine, with the same connection string."
You can configure an infinite number of "Shared Access Policies", each having a custom key and connection string.
You can also configure client access privileges for each policy, chosen from the following 3 options:
Manage
Send
Listen
Make sure to use the appropriate client access privileges. If you want to use the same exact connection string for sending and listening, add both "Send" and "Listen" access to that policy. (You can also have a different policy and connection string for sending and listening...etc.)
Related
I want to use the auto-forwarding feature of the Azure service bus. I have a topic called "trip" and has a subscription called "test".
I have set the auto-forwarding enabled and set to forward the message to another Topic called "trip_elaborated". This is working fine. But, It does not wait for the message to complete and then auto-forward to another topic.
e.g the "test" subscription takes 30 seconds to process the message and before it completed it forwards the message to the "trip_elaborated" topic. I want this operation do in sync.
Is there any configuration needed? Or any other way to achieve this kind of scenario?
I would prefer to manage this using service bus explorer(without explicitly do in the consumer using code).
When Auto forwarding is enabled on an entity, messages will be forwarded automatically, and cannot be processed from the entity they were originally sent to. If you want to process the message and forward it in a synchronous manner, you'd need to do it in your processer. Azure service bus will forward the message from the subscription straight to the destination the moment the message arriving at the topic meets the filter criteria.
To achieve processing and forwarding, you can process the incoming message in a transactional manner, something Azure Service Bus supports. See documentation for more details.
In case you can tolerate processing and forwarding in parallel, you'd have two subscriptions, one for processing and another for solely auto-forwarding.
I have tried to find out how I can notify multiple microservice (MS) instances via eventdriven approach if their Azure App Configuration values are changed. I have found that I via Eventgrid can listen on changes in Azure App Configuration but I don't find any built in method to distribute event to multiple instances (many MS instances)... I can choose webhook but then it will be to one instance, I can choose other eventdriven approaches as Event Hub but then I have to setup that and I wonder what is best practice for this? I don't want each MS to poll for changes rather to be notified and receive what has been changed or is there a better built in approach/strategy?
For push-based configuration updates from Azure App Configuration, the suggested approach is to forward events to a Service Bus Topic. Azure Service Bus SDK provides RegisterMessageHandler method that allows clients to register a message handler that would be triggered for every message recieved in the topic. Each instance of the microservice could set up a subscription to this Service Bus Topic and register a message handler during service initialization to receive configuration updates.
The instructions to set up a service bus topic can be found here. Details on the protocols available to subscribe to service bus topics and the required firewall configuration can be found here. Since a single topic can support up to 2000 subscriptions, this approach would allow up to 2000 service instances.
i want to develop a Game using Azure Cloud but i am not sure which ressources i could use for it. Folowing is my problem:
There is one publisher which is providing some messages. I need to find a way to add subscriber on demand. That means whenever i need more subscriber i need to add them without changing any settings on the portal.
All subscriber needs to get the published message at the same time.
Received message from the publisher needs to be deleted for all subscriber at the same time.
Is there a ressource or a way which fits theese needs?
Thanks.
Azure SignalR Service is WebSocket Gateway service that covers all your requirements.
While you could always host your own server, the service supports scaling out.
I was given the iothubower SharedAccessKey for an IoT Hub instance I don't have otherwise access.
Using Visual Studio Code I can Set IoT Hub Connection String using the iothubowner credentials and then Create Device to create a device on the IoT Hub.
However, when I try to Start Monitoring Built-in Event Endpoint on that device, I am asked for additional information, namely the Event Hub compatible connection string.
Is this string something I can retrieve using the iothubowner credentials that I've been given? I don't have access to the Azure Portal.
I see the structure of the Event Hub compatible connection string consists of a namespace (which I don't know) and the event hub name, in addition to the iothubowner credentials.
Is the namespace and event hub name something I can retrieve using the my credentials?
Is there any other way I can monitor events?
Event Hub compatible endpoint with iothubowner credentials in my case looks something like this:
Endpoint=sb://iothub-ns-{part_of_iot_hub_name}-2052657-6507c44fda.servicebus.windows.net/;SharedAccessKeyName=iothubowner;SharedAccessKey={shared_access_key}=;EntityPath={iot_hub_name}
The namespace looks like this and is not a part of iothubowner credentials(connection string):
iothub-ns-{part_of_iot_hub_name}-2052657-6507c44fda
As you can see, this endpoint cuts the IoT hub name and contains some numbers - which would be really hard to guess. So if you have the namespace, you can construct the endpoint, otherwise, I would say no and you need access to the Azure portal or to log in with 'az command' and retrieve it with Powershell.
Even though there is an accepted answer, I could not quite figure out how to get my Endpoint.
You need to go to your IoT Hub and click on "Built-in endpoints" and there you will find the Entry:
I'm looking to use Azure Service Bus with topics but need to handle the scenario where a subscriber might not be listening for a message it's interested in (e.g. server being rebooted etc.). This is the typical durable subscriber pattern as described here http://www.eaipatterns.com/DurableSubscription.html.
What I can't work out is how to apply this with Azure Service Bus and I can't seem to find any examples or discussion of this in the documentation. Is this something that Azure service bus provides or should I start looking at alternatives to Azure Service Bus?
This is built straight into Service Bus. As long as a subscription is created it is durable. You create a topic and then create one or more subscriptions. One or more consumers then listen to a subscription when they are active. If they go inactive, such as the server being rebooted, then the subscription stores the messages until a consumer comes back up and asks for one.
Service Bus would only be nondurable if you were creating and destroying subscriptions on the fly as each consumer becomes active or becomes inactive. If there are no subscriptions then messages sent to a topic are lost. Once you create a subscription, any messages sent to the topic (if they pass any filters applied) will be available on the subscription regardless if there are any active consumers using that subscription. Subscriptions exist until you remove them or, if you have the idle removal feature turned on, they surpass the idle deletion time.
You can verify this with a simple console application, or using LinqPad to set up code that does the following:
Create a topic.
Create a subscription on that topic (no filters)
Send a few messages to the topic.
In a different script or console app, create a MessageReceiver for that subscription and pull down the messages.
The messages within a subscription are durable for the life of that subscription, until they are processed (completed, etc.), they are forwarded somewhere else or they expire.
I am not sure where you looked for documentation, following are good to read:
1) http://azure.microsoft.com/en-us/documentation/articles/service-bus-dotnet-how-to-use-topics-subscriptions/
2) http://code.msdn.microsoft.com/windowsazure/Simple-Publish-Subscribe-d406eb03