I have an API App running under Azure App Service, with Application Insights installed to track server side telemetry of API calls. When viewing Application Insights in the Azure portal, I am seeing two events for every one server call. Each event has an exact duplicate with the same timestamp, response time, telemetry, etc. I have verified that only one event is in the web server logs, so I'm not accidentally calling the same function twice from the client.
Here are a couple of screenshots to illustrate:
What could be causing this? And how can I fix it?
There is a one known scenario that may lead to the data duplication:
Application is not onboarded to AI SDK is deployed as an Azure
Web App
AI Extension is installed to the app -> after this step
you start to receive data without need to modify your code
Later on you decided to use more powerful features of AI let's say custom
event tracking and on-boarded your application to AI from VS and
re-deployed.
Now you may end up in the situation when HTTP module is registered twice and you start to receive duplicate request data. It happens because AI nuget packages add HTTP module definition in web.config, but extension installation drops additional assembly into your application bin folder that registers HTTP module dynamically during app start - Microsoft.AI.HttpModule.dll (Microsoft.ApplicationInsights.Extensibility.HttpModule.dll in previous versions). To correctly handle this case you need to remove extension leftovers during your application deployment by choosing "Settings->Remove additional files from destination" in case of deploying from VS.
Related
An extension called AppStateTracker is causing issues on my Azure web app, what extension is this?
what is it, and why are we only seeing it on one services. What differs that service from the rest of our services. I see it in the Activity Log when I check the JSON for the "Update website extensions"
AppStateTracker is an update which enables config level tracking for your web app from Application Change Analysis blade and simply collects data from the environment. So frequent changes to your application will create frequent updates but will have no adverse effects on your application.
AppStateTracker is a dormant extension, it gets activated when Azure makes PUT calls to the application. It would wake up your process if its not always on, however in terms of actual impact on the application - there is nothing invasive that can affect anything. It only scans environment variables and settings and never attaches or does anything with running process nor modify anything. It is part of Change Analysis which is a completely independent product.
If you want to see less of these updates, you can choose to disable file and configuration change tracking on web app following the instructions below:
https://learn.microsoft.com/en-us/azure/azure-monitor/app/change-analysis#application-change-analysis-in-the-diagnose-and-solve-problems-tool.
Situation 1
I wrote my own bot framework and deploy to Azure,
the Test in Web Chat shown me "Waiting for bot to be ready after deployment".
Situation 2
I downloaded the source code from Azure bot Services,after that I deploy back to Azure, the Web Chat shown me "Waiting for bot to be ready after deployment".
I also faced this issue and this is how I solved it.
Changed my publish settings to have Remove additional files at destination checked. This is an important step because when you provisioned a new web app bot, it will already have some files in the location. So when you publish your code, it will be placed along with the code files which already exists. You can also test it by navigating to the /api/messages url. If this gives you error, then this is most likely the case.
I also updated all the NuGet dependencies.
The other thing that caused me a bit of issue was the App Password. The app password in your web.config should match the one which is already generated for your BOT. Do not create a new password from the bot management page. If you did that then make sure that you change/update the password in web app bot application settings and also in the web.config file. When you host the bot, by default it will use the Application Id and Application Password which were generated automatically. You can see these values in the Web App Bot service type which is under the section App Service Settings>Application Settings. Scroll down to the bottom and you will see these two settings. Copy the values of Application Id and Application Password and update them in the Web.config file.
Hope this help!
What fixed it for me was changing Destination URL in Publish Settings to https. For some reason default is http.
I had missing node packages that I wasn't aware of. I found the errors in the output logs at https://[bot-name-here].scm.azurewebsites.net/dev/wwwroot/:vs.output
Or you can Navigate to this by going to Build -> Click 'Show output' (or Shift+Command+U) (its the 5th icon in the left hand side menu on the Build window)
Hope that helps!
Some ideas to troubleshoot for people having the same error:
Check that the server hosting your bot is actually running. If https://YOUR_BOT.azurewebsites.net is timing out, then your server is not running, check your config
Check if you have logs in the App Service: Enable everything under App Service > Diagnostic logs, then go to App Service > Log Stream.
Check potential errors under Web App Bot > Channels > Web Chat > Issues
Check that the files on the server look ok by going to App Service > App Service Editor (Preview)
Close all your browsers, especially if they have been running for a long time and/or you have been changing a lot of config/files recently
I fixed the issue by using a combination of those points
I had the window open for a few days, and "Test in Web Chat" would just show "Waiting for bot to be ready" and the juggling progress indicator, forever. I duplicated the tab and it came right up. I think my session may have expired or something, because any new, duplicated tab, I could bring the bot up in just fine.
I solved the problem by, in Visual Studio's Solution Explorer, setting the .bot file's property "Copy to Output Directory" to "Copy always"."
The details: The bot worked fine locally in the emulator. After deploying to Azure (continuous deployment), "Test in Web Chat" hung with "Waiting for bot to be ready". (Would be nice if it detected and reported errors.)
After setting up Azure storage, turning on Azure logging, discovering the FileNotFoundException, inspecting with Resource Explorer, and deploying multiple bot versions, I isolated the cause. This document really helped me troubleshoot:
https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-dotnet-visual-studio#a-nameapptracelogsacreate-and-view-application-trace-logs
I also experienced this problem but the solution I have is totally different from the suggestions above.
I created my Knowledge Base and Web App Bot purely using the portal. The problem lies with the configuration. As of writing, the Bot Template: Node.js > Question and Answer does not work. I needed to create a new Web App Bot and I selected instead C# > Question and Answer.
That fixed my problem and allowed me to use the Test in Web Chat feature.
I have a .net core application deployed to IIS. There is a signalr hub in the application.
My problem is that when my client hits the hub it receives a 500 error back.
I have put logging in the hub and can see that it is being hit correctly and no exceptions are being thrown.
The setup works perfectly when run from Visual Studio.
I am thinking something might not be set up in IIS correctly.
Does anyone have any ideas that this might be?
I enabled the generic error page in the api and looking at the network tab in the browser I was able to see the call that signalr was making.
I looked at the content of this request and found that an error with the following message was being returned by the server...
The data protection operation was unsuccessful
After some searching I found that this was being caused due to a setting on the IIS application pool.
The steps to change the setting were...
Open IIS Manager
Select Applications Pools, and go ahead and select the App Pool used by your app
Right-click on it, and select Advanced Settings, Go to the Process Model Section and Find the “Load User Profile” Option and set it to True
These steps were taken from http://puresourcecode.com/dotnet/post/ASPNET-The-data-protection-operation-was-unsuccessful
We are experimenting Azure Application Insight within a SharePoint 2013 solution. We managed to configure it manually, and we are collecting data from the development farm.
However, the same configuration looks like not to be working in the test farm, as only a few requests are tracked.
We tried two ways of diagnosing the issue:
With a custom command-line tool that traces an exception, on the test front-end. No errors are shown, and enabling diagnostic tracing on System.Net shows no problem actually occurs in transmission. Eventually, the exception is logged on AppInsight, but not every time.
Implementing a custom WebApi controller that performs the same action, but within of our solution code. This never gets collected.
In the end, what looks very weird is only certain URL are tracked server-side (no problems with client JS), which interestingly are the slowest requests.
I've decompiled all Application Insight assemblies, and of course telemetry transmission in done asynchronusly, but within w3wp.exe there's no chance that the process is aborted and the transmission never occurs...
What else can I try?
You can install Fiddler or Other HTTP Debugging tool on the box to see the data that AI sends.
To View internal Application Insights traces you can Install PerfView and collect traces:
PerfView.exe /onlyProviders=*Microsoft-ApplicationInsights- collect
http://www.microsoft.com/en-us/download/details.aspx?id=28567
http://channel9.msdn.com/Series/PerfView-Tutorial
I have two servers sitting behind a loadbalancer in my service tier. Both of them should be identical - IIS setup the same, AppFabric (to keep two services warmed up), app pools running under either a service account or the app pool identity. On one server, everything works. On the other server, three of my app pools (the two that AppFabric is warming up, under the service accounts, and one that's just a standard app pool with no changes made from default settings) stop running almost as soon as I start them up (sometimes on the first request).
I get five of the following error in the Application log each time I try to start one of the app pools:
There was an error during processing of the managed application service auto-start for configuration path: 'MACHINE/WEBROOT/APPHOST/Site/App'. The error message returned is: ''. The worker process will be marked unhealthy and be shutdown. The data field contains the error code.
The error code referenced is 80070005.
This is actually for the same Site/App regardless of the app pool being started (though it may change after recreating the app pools).
In the System log, I get the following warning five times before it errors (Application pool 'AppPool' is being automatically disabled due to a series of failures in the process(es) serving that application pool.):
A process serving application pool 'AppPool' reported a failure during application preloading or service loading. The process id was '2396'. Please ensure that all application preload or service settings in the application pool are configured properly. The data field contains the error number.
The error code referenced is 80004005.
The AppPool here is the one being started.
I've tried recreating; I've tried uninstalling AppFabric (but we need it, so reinstalled and still no go). I'm out of ideas. Any suggestions?
EDIT: I tried copying the applicationHost.config over from the working server, but that didn't work either..
EDIT2: One of the app pools works when running under a real user account but doesn't when running under the ApplicationPoolIdentity....
(Also, we had an issue where the site was running under 2.0 and the apps were running under 4.0. That may have resolved the ones that are running as the service accounts.)
I was just wrestling with this same problem for a few hours and found a different culprit.
I had added a new configuration section to my Web.config in a recent commit. I also added this section to a separate ERB file used by Puppet to generate a custom Web.config at the point of deployment. In this template file, I added the new section but forgot to include its declaration in <configSections>.
Once I added the declaration to the template, our app's test VMs were able to start up again and this error went away.
While the app pools for the applications were 4.0, the app pool for the site itself was 2.0, causing some of the issues. We also had inetpub on a different drive, and we had to grant access to SERVER\Users.