I am running a set up, where Stream Analytics job is reading from Azure Event Hub. Event Hub is contained within a namespace and there is consumer group created for SA specifically.
Recently I noticed User Error on Event Hub metrics.
Event Hub Namespace is configured to push all logs to OMS:
ArchiveLogs
OperationalLogs
AutoScaleLogs
KafkaCoordinatorLogs
EventHubVNetConnectionEvent
CustomerManagedKeyUserLogs
KafkaUserErrorLogs
But that does not seem to cover that particular case, since I am unable to find that error in Log Analytics Workspace (e.g. by doing search "UserError" over last 24 hours).
Searching through docs is not to helpful either. Closest I have found was in .NET SDK for Azure:
Namespace: Microsoft.Azure.EventHubs
Assembly: Microsoft.Azure.EventHubs.dll
UserError Identifies the exception as a user error and user needs to take an action to address the failure.
But that is not very helpful either, as it does not even indicate whether user is the sender or receiver of message.
What is the UserError in that case and how to pinpoint, what is actually happening?
User errors generally occur due to mistakes in the client application like,
User Errors in Event hub and service bus occurs due to similar reasons .. You can refer Azure Service Bus Message Queue User Errors Metric
Related
The message is posted to the service bus from Java code and while checking in Azure portal -> service bus -> Possible to see the incoming and successful request.
I have been trying to see the Azure posted message using 'Service Bus Explorer'.
But not able to see the message though there is no issues in Azure portal -> service bus.
I am suspecting about configuration or access restriction. Since I am new to Azure Service Bus, not able to proceed how to process further.
Any help will be greatly appreciated.
I do agree with #Anand Sowmithiran that,
If that consumer application has received the message and marked it as 'consumed', it will no longer appear in Azure portal.
If the Consumer didn't do that then we can check messages as below process:
Firstly, open Service bus then click on Topic as below:
Then click on peek messages as below :
If the message is not sent/failed because of any reason you can check as below:
if the message is in receive mode then you should give session id as correctly as below:
After Clicking on Receive:
I am implementing an azure event hub subscriber which is successfully able to connect but it's not receiving the message for a particular event hub in an event hub namespace. But it's working as expected for the other event hub in the same namespace. there is no error in the logs. Is there any way to find out the root cause of the issue?
I am successfully able to publish the message with the same configuration.
Publisher and Subscriber are running on the same machine.
I have implemented the publisher and subscriber in Dapr.
Both publisher and subscriber logs are exactly the same in Dapr, So I think it's an issue with the storage account.
Adding more information:
I have tried my application with a different storage account and it's working as expected. So what is the issue? How can I resolve it? I want to work with my old storage account.
All issues
There are multiple problems put together in your question, which can be tested separately. (E.g. event hubs, dapr, storage account ...etc.) It is critical to have a granular understanding of these problems, so I would try to test them one-by-one.
Event Hub issues
You can test your Event Hub flow separately by creating simple "hello world" clients locally and reproducing a relationship between them based on your production scenario.
E.g.
try to connect to an event hub with exactly one publisher and exactly one consumer,
try to publish the same message with one publisher to multiple event hubs and see if you can consume them with separate consumers,
...etc.
Based on your comment, I will list 2 example problem scenarios:
Problem 1: "I can access one Event Hub in the namespace, but can't access another in the same namespace."
You can configure "Shared Access Policies" for the scope of the Event Hub Namespace, which will be applied to all event hubs in that namespace.
You can also configure the same policies for the scope of the Event Hub, which will be applied to that single hub only.
Make sure to use the appropriate scope in Azure portal while trying to use the connection strings from your code.
Problem 2: "I can publish messages from a client to an Event Hub, but cannot consume messages with the same client, from the same machine, with the same connection string."
You can configure an infinite number of "Shared Access Policies", each having a custom key and connection string.
You can also configure client access privileges for each policy, chosen from the following 3 options:
Manage
Send
Listen
Make sure to use the appropriate client access privileges. If you want to use the same exact connection string for sending and listening, add both "Send" and "Listen" access to that policy. (You can also have a different policy and connection string for sending and listening...etc.)
We have an Azure Function that is publishing messages to an Azure Event Hub via the built-in event hub integration binding that is then used to feed a Stream Analytics job. This has all been running fine then suddenly last night we started getting the following error in our Azure function that writes to the Event Hub:
Exception while executing function: Functions.LocationEventPublish. Microsoft.Azure.WebJobs.Script: One or more errors occurred. Microsoft.ServiceBus: Cannot allocate more handles to the current session or connection. The maximum number of handles allowed is 4999. Please free up resources and try again.
Found the SO post below and understand why we are getting this and the suggested solution. However how are we supposed to only allocate a single instance of the client when using the built-in integration binding to publish from an Azure function to an event hub? Stopping and starting our Azure function resolved this issue but it seems this is a pretty major issue with using an Azure Function to populate an Event Hub.
Event Hub Exception Cannot Allocate More Handles To The Current Session or connection
The EventHub binding for Functions already does some caching of the clients (see https://github.com/Azure/azure-webjobs-sdk/blob/ffae7c86ea87fd73748186bac2c38c5401b80f68/src/Microsoft.Azure.WebJobs.ServiceBus/EventHubs/EventHubConfiguration.cs#L24 ) .
Can you share your function.json? Are you binding against many different event hubs?
I am using azure service bus subscription and Topics for sending orders and getting orders. We have a windows app which pulls orders using web api from azure service bus subscription.
Suppose if user login and got 5 orders. If user log out from the windows app then we unlocks the messages from azure service bus. and if user logs again after 5 min or less then we are not getting all the orders from azure service bus. Its coming like 3 or 4 sometimes.
Can anyone help me on this issue, why I am not getting all the 5 orders again. I am using ReceiveBatch method of subscriptionclinet for pulling messages.
I suspect you're running into a delivery count issue here. Assuming I understood correctly what you're saying, the windows app pulls the messages in a PeekLock mode. Once message is processed, you complete it. Otherwise, upon exit from the application, you abandon the messages. If you not done processing messages, and user quits, the message is abandoned while DeliveryCount is increased. Once DeliveryCount exceed the maximum set on the entity, the message is deadlettered and will not be delivered anymore.
Check your MaxDeliveryCount on the entity
Check you entity`s DLQ to verify if this is the behaviour
In case they are DLQed, I suggest to look into centralising your DLQs for easier error handling
I'm looking to use Azure Service Bus with topics but need to handle the scenario where a subscriber might not be listening for a message it's interested in (e.g. server being rebooted etc.). This is the typical durable subscriber pattern as described here http://www.eaipatterns.com/DurableSubscription.html.
What I can't work out is how to apply this with Azure Service Bus and I can't seem to find any examples or discussion of this in the documentation. Is this something that Azure service bus provides or should I start looking at alternatives to Azure Service Bus?
This is built straight into Service Bus. As long as a subscription is created it is durable. You create a topic and then create one or more subscriptions. One or more consumers then listen to a subscription when they are active. If they go inactive, such as the server being rebooted, then the subscription stores the messages until a consumer comes back up and asks for one.
Service Bus would only be nondurable if you were creating and destroying subscriptions on the fly as each consumer becomes active or becomes inactive. If there are no subscriptions then messages sent to a topic are lost. Once you create a subscription, any messages sent to the topic (if they pass any filters applied) will be available on the subscription regardless if there are any active consumers using that subscription. Subscriptions exist until you remove them or, if you have the idle removal feature turned on, they surpass the idle deletion time.
You can verify this with a simple console application, or using LinqPad to set up code that does the following:
Create a topic.
Create a subscription on that topic (no filters)
Send a few messages to the topic.
In a different script or console app, create a MessageReceiver for that subscription and pull down the messages.
The messages within a subscription are durable for the life of that subscription, until they are processed (completed, etc.), they are forwarded somewhere else or they expire.
I am not sure where you looked for documentation, following are good to read:
1) http://azure.microsoft.com/en-us/documentation/articles/service-bus-dotnet-how-to-use-topics-subscriptions/
2) http://code.msdn.microsoft.com/windowsazure/Simple-Publish-Subscribe-d406eb03