Cannot get rid of logs in Azure Function for Web Storage - azure

I have tried so many things to do this and nothing works, but when I run my Azure function I get all these logs its far too much noise
What is the correct way to get rid of them once and for all?
I have this in my host.json
"logging": {
"logLevel": {
"default": "Information",
"Microsoft": "Warning",
"System": "Warning",
"Host": "Error",
"Function": "Error",
"Host.Aggregator": "Information"
},
"Serilog": {
"MinimumLevel": "Information",
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "{Timestamp:HH:mm:ss} {Level} | {RequestId} - {Message}{NewLine}{Exception}"
}
}
]
}
},
and this is in my localsettings.json
"logging": {
"logLevel": {
"Microsoft.Azure.WebJobs.Script.WebHost.Middleware.SystemTraceMiddleware": "Error",
"Worker.rpcWorkerProcess": "Error"
}
},

You have to modify the host.json for configuring minimum level of logs either locally or in Azure Cloud.
I can see you have defined the log values as Information, Warning to the Log Level attributes where information gives the general flow of the Application Execution from Start to End like Host level flow, Application-Level Flow logs.
Host.Aggregator generates more than the trace level metrics if assigned with the value of Information.
You have to remove/disable the unnecessary modules which are not required for logs or keep that modules log level to None if not required for the current situation and change the log level to minimum which also reduces your Logs Consumption if deployed to Azure Cloud.
I found a similar SO Issue 70690850 that shows the minimum log levels should be modified in the host.json which reduces the number of logs both locally and given the techniques of Application Insights and this MS Doc for more information on Log Levels Configuration.

Related

Microsoft.Azure.WebJobs.Host.FunctionTimeoutException

We have a durable function, which got timeout exception
problemid: Microsoft.Azure.WebJobs.Host.FunctionTimeoutException at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryHandleTimeoutAsync>d__35.MoveNext
outerType: Microsoft.Azure.WebJobs.Host.FunctionTimeoutException
outerMessage: Timeout value of 02:00:00 was exceeded by function: Functions.MultiRecipientActivity
This app is running on App Service Plan (P1v2)
This is my host.json
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"functionTimeout": "02:00:00",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[2.*, 3.0.0)"
}
}
Please help me resolve this problem.
outerMessage: Timeout value of 02:00:00 was exceeded by function: Functions.MultiRecipientActivity
If you are hosting the Azure Functions Project in a Dedicated App Service Plan or Premium plan and if timeout property is required more than the default (30 minutes) then it is recommended to use -1 value as mentioned in this MS Doc of Azure Functions Host Configuration.
I found the similar issue on the GitHub Repository of Azure Functions timeout issue were specifying the timeout property more than the default would fail but as recommended by Microsoft i.e., -1 (unbounded value) for that property is working pretty fine for many customers.
Syntax:
"functionTimeout": "-1"
The above-mentioned timeout property will be applied on the host.json file for the .csx functions (Functions created in the portal environment) and for remaining types of Function projects like C# Class libraries, we can use the timeout property in Function code declaration level:
[Timeout("-1")]
[Function("Your_Function_Name")]
{
xxxxxxx
}
Also, update the extensionBundle to the recent version which makes the functions bindings to right compatibility, replace "version": "[2.*, 3.0.0)" to "version": "[3.3.0, 4.0.0)".

Azure Insights: webapp request time

I have enabled http request logging for my webapp in Azure insights. I don't understand there is such a large response time disparity between identical requests, received just a few seconds or a few minutes apart. A log example with three records:
{ "time": "2022-07-20T06:08:41.7548330Z", "EventTime": "2022-07-20T06:08:41.7548330Z", "resourceId": "***", "properties": "{\"CsHost\":"***",\"CIp\":\"195.235.205.153\",\"SPort\":\"80\",\"CsUriStem\":\"\\/mensajes\",\"CsUriQuery\":\"desde=20220705T125027\",\"CsMethod\":\"GET\",\"TimeTaken\":1426,\"ScStatus\":\"200\",\"Result\":\"Success\",\"CsBytes\":\"976\",\"ScBytes\":\"302\",\"UserAgent\":\"RestSharp 104.2.0.0\",\"Cookie\":\"--\",\"CsUsername\":\"\",\"Referer\":\"\",\"ComputerName\":\"RD501AC5BF5D04\"}", "category": "AppServiceHTTPLogs", "EventStampType": "Stamp", "EventPrimaryStampName": "waws-prod-am2-325", "EventStampName": "waws-prod-am2-325d", "Host": "RD501AC5BF5D04", "EventIpAddress": "1*.*.*.*"}
{ "time": "2022-07-20T06:09:42.2283150Z", "EventTime": "2022-07-20T06:09:42.2283150Z", "resourceId": "***", "properties": "{\"CsHost\":"***",\"CIp\":\"195.235.205.153\",\"SPort\":\"80\",\"CsUriStem\":\"\\/mensajes\",\"CsUriQuery\":\"desde=20220705T125027\",\"CsMethod\":\"GET\",\"TimeTaken\":279,\"ScStatus\":\"200\",\"Result\":\"Success\",\"CsBytes\":\"976\",\"ScBytes\":\"302\",\"UserAgent\":\"RestSharp 104.2.0.0\",\"Cookie\":\"--\",\"CsUsername\":\"\",\"Referer\":\"\",\"ComputerName\":\"RD501AC5BF5D04\"}", "category": "AppServiceHTTPLogs", "EventStampType": "Stamp", "EventPrimaryStampName": "waws-prod-am2-325", "EventStampName": "waws-prod-am2-325d", "Host": "RD501AC5BF5D04", "EventIpAddress": "*.*.*.*"}
{ "time": "2022-07-20T06:10:15.0636460Z", "EventTime": "2022-07-20T06:10:15.0636460Z", "resourceId": "***", "properties": "{\"CsHost\":"***",\"CIp\":\"195.235.205.153\",\"SPort\":\"80\",\"CsUriStem\":\"\\/mensajes\",\"CsUriQuery\":\"desde=20220705T125027\",\"CsMethod\":\"GET\",\"TimeTaken\":2629,\"ScStatus\":\"200\",\"Result\":\"Success\",\"CsBytes\":\"976\",\"ScBytes\":\"302\",\"UserAgent\":\"RestSharp 104.2.0.0\",\"Cookie\":\"--\",\"CsUsername\":\"\",\"Referer\":\"\",\"ComputerName\":\"RD501AC5BF5D04\"}", "category": "AppServiceHTTPLogs", "EventStampType": "Stamp", "EventPrimaryStampName": "waws-prod-am2-325", "EventStampName": "waws-prod-am2-325d", "Host": "RD501AC5BF5D04", "EventIpAddress": "*.*.*.*"}
The three requests are the same and therefore trigger the same process on the server side (same endpoint, made from the same origin). The result can be seen in the 'timeTaken' field: 1426ms/279ms/2629ms Any suggestion is appreciated.
According to Microsoft-Documentation it say,
If they remain idle for a predetermined amount of time, web apps are by default unloaded. You can activate the Always On feature in Basic and Standard service levels to keep the app constantly loaded.This eliminates longer load times after the app is idle.
Open your Web app=> Then click on Configuration=>Then click on General settings=> then on.
References:
Smart detection - performance anomalies - Azure Monitor | Microsoft Docs
Exploring performance issues with Azure Application Insights | by Thomas Weiss | Medium

Not sending data to Application Insights from Function App

I have Function App and Application Insight services. I've noticed that amount of data which my application sends is big and generates big costs. Can I disable/completely stop sending data to AI without deleting APPINSIGHTS_INSTRUMENTATIONKEY or APPLICATIONINSIGHTS_CONNECTION_STRING? But of course I want keep the both services alive.
Should host.json be configured in some way?
Here is the workaround I did for Optimizing the cost generated by Application Insights Logs:
To minimize the number of logs, you can use the higher logging level of logs in host.json as you can see in the below screenshot:
As you can see the minimization of logs here function information log is not produced and the output log is shown in the browser, only the manual logging is shown in the logs/terminal.
And the other ways of reducing the logs and optimizing the AI Cost for Azure Functions:
- Disable unneeded modules: Edit the ApplicationInsights.config to turn off collection modules which are not required.
- Disable telemetry dynamically: To disable telemetry conditionally and dynamically anywhere in the code, set the DisableTelemetry flag on it using TelemetryConfiguration instance.
This code sample prevents the sending of telemetry to Application Insights but not the automatic collection modules from colleting telemetry and in order to remove auto collection modules also, please refer this Microsoft Documentation.
- . Customize Logs Collection:
{
"Logging": {
"LogLevel": {
"Default": "Warning"
},
"ApplicationInsights": {
"LogLevel": {
"Default": "Information"
}
}
}
}
The following above configuration allows Application Insights to capture all Information logs and severe warning logs. To change this behavior, explicitly override the logging configuration for the provider ApplicationInsights as shown below:
{
"Logging": {
"LogLevel": {
"Default": "Warning"
}
}
}
There are few more techniques to manage the data volume used for telemetry data optimizing also like:
Sampling:
Daily Cap:
Pre-aggregate metrics
Please check these references for more information:
Resolve if logs shows twice in Application Insights
Optimizing logging costs of Azure Functions
Configuring or removing the necessary Telemetry Initializers
Also, Please visit my practical workarounds (Ref1, Ref2)to reduce the unnecessary logs and optimize the cost.

Azure Function silently fails to start if dependency is misconfigured

When developing locally on my machine, if I do not start the Azure Cosmos Emulator, my Function fails to start correctly. There are no errors in the console output, however if I try to call a HttpTrigger function the TCP connection is refused. No exceptions are trapped by the Visual Studio debugger, and no errors are shown in the console.
How can I get an error to be logged to console?
The only difference I've seen is in the console output is the following output not showing when Cosmos is switched off:
[2020-10-19T19:20:29.487] Host started (2444ms)
[2020-10-19T19:20:29.490] Job host started
Hosting environment: Production
Content root path: C:\Code\MyApp.Functions\bin\Debug\netcoreapp3.1
Now listening on: http://0.0.0.0:7071
Application started. Press Ctrl+C to shut down.
[2020-10-19T19:20:34.554] Host lock lease acquired by instance ID '000000000000000000000000C51E9459'.
I'm not seeing much when using enhanced logging either, as configured below:
{
"version": "2.0",
"logging": {
"logLevel": {
"Host.Triggers.Warmup": "Trace",
"Host.General": "Trace",
"Host": "Trace",
"Function": "Trace",
"MyApp": "Trace",
"default": "Trace"
}
}
}
Below is a diff of the logs after normalizing meaningless differences, including timestamps, guids and execution-times.

functionTimeout in hosts.json not recognized by function app host runtime version ~3

With below mentioned host.json, we see that function timeout is set to default of 30 mins. Some of our workflows run beyond 30 mins. This was working in runtime ~2. Also by checking the logs from host startup, there is no way to see the timeout value picked by the runtime. Is it logged ?
{
"version": "2.0",
"functionTimeout": "02:00:00",
"logging": {
"logLevel": {
"Default": "Debug"
}
}
}
The host.json is correct, the version is just for the schema.
I just had a similar issue, and it turns out registering IConfiguration as a singleton re-instates the default values.
So do not do this in Startup.cs:
builder.Services.AddSingleton<IConfiguration>(config);

Resources