Here is the structure, a data-drift-detected event in ML Workspace sends the event into event grid which triggers a function in Azure Function App. I want it to run only once after the data drift detection. However, I got this:
image
It runs every ~20s for a few times ://
Here is my host.json:
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[3.*, 4.0.0)"
}
}
and function.json:
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventGridTrigger",
"name": "event",
"direction": "in"
}
]
}
Tried changing default options in "singleton" field in host.json, but nothing changed.
Do you have any idea?
When you create an event grid trigger there you will have retry policies where you can change it 1.
Event grid trigger waits of response if it doesnt get it triggers again until it gets a response, so change it to 1. To only trigger once.
So if event grid doesnt get response it triggers again with some interval of time
If not you are sending responses, so its triggering try not to send more responses to your end point.
References taken from:
Azure Event Grid delivery and retry - Azure Event Grid | Microsoft Learn
Related
I have an azure function in the premium plan and I have deployed a queue trigger function inside the docker image.
Here is the reference -> https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-function-linux-custom-image?tabs=in-process%2Cbash%2Cazure-cli&pivots=programming-language-javascript
host.json
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[3.*, 4.0.0)"
},
"functionTimeout": "00:30:00",
"extensions": {
"queues": {
"batchSize": 1,
"newBatchThreshold": 0,
"maxDequeueCount": 1
}
}
}
It was working perfectly after some days it suddenly stopped processing message. When i went into azure portal and checked the storage queue, got to see that there are more 100+ message in the queue and function is not picking up any message.
I just restarted the function and it starts working.
I went through this -> https://github.com/Azure/azure-functions-host/issues/3721#issuecomment-441186710
There someone was trying to set maxPollingInterval up to 2 mins. I just check this doc it is mentioned that the default value of this property is 1 min then setting it to 2 mins doesn’t make any sense.
This is an acutely weird behaviour that is why this is happening and it happened to me a couple of times.
The queue is receiving 20 messages maximum in a day, hence the frequency of messages is very low. Those 20 messages can be pushed into the storage queue at once or every hour depending on the requirement.
Any help would be appreciated.
Thanks in advance.
"triggers": {
"manual": {
"inputs": {
"schema": {}
},
"kind": "Http",
"runtimeConfiguration": {
"concurrency": {
"maximumWaitingRuns": 99,
"runs": 1
}
},
"type": "Request"
}
}
From the above code can I increase the trigger amount from "maximumWaitingRuns": 99 to 2000+?
If yes then how?
I also want to trigger it 2000+ times using single click.
One of the workaround to make the workflow run multiple times is to use a recurrence trigger instead of an HTTP trigger.
Another workaround would be mentioning the iterations and run the workflow through Postman. After saving your request to one of your collections, navigate to your collection >> Run >> Mention the number of iterations >> Run Sample.
I've been trying to get my logging output in order so I'd be able to fix any errors resulting from the Function working improperly. But all the logging says is ResultCode 0.
As I looked at the initial examples of the docs I thought maybe i am missing a return, just like here LINK. But I am misunderstanding how it works, because when I add them they only generate errors. They can be found in below snippet where I return the statuscode to the output binding.
{
"scriptFile": "__init__.py",
"bindings": [
{
"name": "mytimer",
"type": "timerTrigger",
"direction": "in",
"schedule": "0 */10 * * * *"
},
{
"name": "$return",
"direction": "out",
"type": "http"
}
]
}
init.py
def main(mytimer: func.TimerRequest)-> func.HttpResponse:
logging.info('Getting pre-requisite data from Azure RM and CosmosDB ...')
azure_nsg_list = get_full_nsg_list()
cosmosdb_nsg_entities_list = get_list_of_entities()
nsg_stack_reference_list = get_nsg_number_references()
logging.info('Checking for unmanaged Network Security Groups ..')
unmanaged_nsg_list = [item for item in azure_nsg_list if item not in cosmosdb_nsg_entities_list]
if unmanaged_nsg_list:
logging.info('Unmanaged NSGs found, adding to CosmosDB ..')
for nsg in unmanaged_nsg_list:
logging.info('Adding NSG %s ..')
create_azure_table_entity()
logging.info('Finished adding to CosmosDB ..')
return func.HttpResponse("All found NSGs have been added to CosmosDB.", status_code=200)
else:
logging.info('No unmanaged NSGs found ...')
return func.HttpResponse("No unmanaged NSGs found ...", status_code=101)
In the end i want to be able to get alerting on the moments when my function would actually give an error 4xx.
Is there some way I can get the ResultCodes to show the actual statuscode of the code? I have three functions, the other two have eventhub inputs.
Probably also important, this is my host.json:
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[2.*, 3.0.0)"
},
"extensions": {
"eventHubs": {
"batchCheckpointFrequency": 1,
"eventProcessorOptions": {
"maxBatchSize": 10,
"prefetchCount": 20
}
}
}
}
I actually there are other things I am missing, as the metrics are also not showing all the metrics which these logs should be based on. For example:
We can check all the logging.info messages from Application Insights. Make sure that Applications Insights are enabled for Function App.
Steps to get the log information: Application Insights (Our function App) -> Performance -> Select “Overall” under “Operation Name” column -> Select Function Name from the under “All” logs -> Click on “View all telemetry”
Here we will be able to see the message, have a look at below screenshot for reference from my function message:
My Python code:
Coming to 4XX error, mostly we’ll be getting 403 errors while running the function app.
We need to check two points here:
Need to make sure that we add all the values from Local.Settings.json file to Application settings (FunctionApp -> Configuration -> Application Settings)
Check for CORS. Try adding “*” (Any request made against a storage resource when CORS is enabled must either have a valid authorization header or must be made against a public resource.)
I started playing around with Azure Functions and am running in the issue that my Function is not being triggered by events entering my eventhub.
This is the code for my Function:
host.json:
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[2.*, 3.0.0)"
}
}
function.json:
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "events",
"direction": "in",
"eventHubName": "eventhub",
"connection": "eventhub_connection",
"cardinality": "many",
"consumerGroup": "$Default",
"dataType": "stream"
}
]
}
init.py:
import logging
import azure.functions as func
def main(events: List[func.EventHubEvent]):
for event in events:
logging.info('Python EventHub trigger processed an event: %s',
event.get_body().decode('utf-8'))
logging.info(f'Function triggered to process a message: {event.get_body().decode()}')
logging.info(f' EnqueuedTimeUtc = {event.enqueued_time}')
logging.info(f' SequenceNumber = {event.sequence_number}')
logging.info(f' Offset = {event.offset}')
# def main(event: func.EventHubEvent):
# logging.info(f'Function triggered to process a message: {event.get_body().decode()}')
# logging.info(f' EnqueuedTimeUtc = {event.enqueued_time}')
# logging.info(f' SequenceNumber = {event.sequence_number}')
# logging.info(f' Offset = {event.offset}')
# # Metadata
# for key in event.metadata:
# logging.info(f'Metadata: {key} = {event.metadata[key]}')
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "python",
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=storageaccount;AccountKey=storageacciuntaccesskey=;EndpointSuffix=core.windows.net",
"eventhub_connection": "Endpoint=sb://eventhub01.servicebus.windows.net/;SharedAccessKeyName=function;SharedAccessKey=0omitted;EntityPath=eventhub"
}
}
I started out with the basic eventhub python code provided by the Azure Function Core tools. And have been testing different pieces of code found in online examples from people's blogs and the Microsoft docs.
When switching to cardinality: one -> I switch to the code which is currently commented out. I don't know if that is supposed to go like that, it just feels right to me.
In any case, regardless of the cardinality setting, or the datatype being changed between binary, stream or string. My Function simply does not trigger.
I can query my eventhub and see/read the events. So I know my policy, and the sharedkey and such, work fine. I am also only using the $Default consumer group.
I also tried setting up a HTTP triggered function, and this function gets triggered from Azure Monitor. I can see in the logs each request entering the function.
Am I doing something wrong in the code for my eventhub function?
Am I missing some other configuration setting perhaps? I already checked the Access Rules on the function, but that realy doesn't matter does it? The function is pulling the event from the eventhub. It's not being sent data by an initiator.
Edit: Added the local.settings.json file configuration and updated the function.json
Edit 2: solution to my specific issue is in the comments of the answer.
Update:
__init__.py of the function:
from typing import List
import logging
import azure.functions as func
def main(events: List[func.EventHubEvent]):
for event in events:
logging.info('Python EventHub trigger processed an event: %s',
event.get_body().decode('utf-8'))
Send message to event hub:
import asyncio
from azure.eventhub.aio import EventHubProducerClient
from azure.eventhub import EventData
async def run():
# Create a producer client to send messages to the event hub.
# Specify a connection string to your event hubs namespace and
# the event hub name.
producer = EventHubProducerClient.from_connection_string(conn_str="Endpoint=sb://testbowman.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxxxxx;EntityPath=test", eventhub_name="test")
async with producer:
# Create a batch.
event_data_batch = await producer.create_batch()
# Add events to the batch.
event_data_batch.add(EventData('First event '))
event_data_batch.add(EventData('Second event'))
event_data_batch.add(EventData('Third event'))
# Send the batch of events to the event hub.
await producer.send_batch(event_data_batch)
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
And please make sure you give the right event hub name:
It seems your function.json has a problem, the connection string should not directly put in the binding item.
It should be like below:
function.json
{
"scriptFile": "__init__.py",
"bindings": [
{
"type": "eventHubTrigger",
"name": "events",
"direction": "in",
"eventHubName": "test",
"connection": "testbowman_RootManageSharedAccessKey_EVENTHUB",
"cardinality": "many",
"consumerGroup": "$Default",
"dataType": "binary"
}
]
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=0730bowmanwindow;AccountKey=xxxxxx;EndpointSuffix=core.windows.net",
"FUNCTIONS_WORKER_RUNTIME": "python",
"testbowman_RootManageSharedAccessKey_EVENTHUB": "Endpoint=sb://testbowman.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxxxxx;EntityPath=test"
}
}
check for configuration of function app and EventHub. pre-warned instance of function app should be lesser/equal to partition count of EventHub. Worked for me ad was able to receive events properly after this configuration.
I have been trying to put a message in azure service bus topic. My azure function test says that the message has been accepted(202). But there is no message on the subscription side, Could you please help me in this POC. Here is my code snippet. It's a sample that got generated from VS code, I am using serviceBusTrigger.
const { ServiceBusClient, ReceiveMode } = require("#azure/service-bus");
module.exports = async function(context, mySbMsg) {
context.log('JavaScript ServiceBus topic trigger function processed message', mySbMsg);
context.done();
};
Is there any way that I can check if the service bus topic is working as expected?
I do not see a connection string and a binding associated with the code, Inorder to save the message to the queue, you need to have the Queue connection strings in the settings file. Follow the docs,
{
"bindings": [
{
"schedule": "0/15 * * * * *",
"name": "myTimer",
"runsOnStartup": true,
"type": "timerTrigger",
"direction": "in"
},
{
"name": "outputSbQueue",
"type": "serviceBus",
"queueName": "testqueue",
"connection": "MyServiceBusConnection",
"direction": "out"
}
],
"disabled": false
}
Okay, So the thing was, I tried calling the function after pushing messages onto the topic. But the purpose of ServiceBusTriger is reverse. The function with this trigger processes the message when there is message on queue. There is no need to Call the function separately. Yeah, so basics, just create the function with your available topic and subscription, and then try putting message onto the topic. You can see the message in the log of the function.