Since yesterday (2016-02-11 18:32 EST), we get this mysterious error in our Azure Webapps' "New Relic" monitoring logs. It seems to happens every 5 minutes We have five applications, all were deployed yesterday around that exact time, but the deployment process it the same as before...
The applications are all functional. I do not know where this error comes from, but the Stack trace mentions Kudu.Services.Web.Tracing.TraceModule.OnBeginRequest
Error message
System.UriFormatException: Invalid URI: The hostname could not be parsed.
Stack trace
at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)
at System.Web.Util.UriUtil.BuildUriImpl(String scheme, String serverName, String port, String path, String queryString, Boolean useLegacyRequestUrlGeneration)
at System.Web.Util.UriUtil.BuildUri(String scheme, String serverName, String port, String path, String queryString)
at System.Web.HttpRequest.BuildUrl(Func`1 pathAccessor)
at System.Web.HttpRequest.get_Url()
at Kudu.Services.Web.Tracing.TraceModule.TryConvertSpecialHeadersToEnvironmentVariable(HttpRequestWrapper request)
at Kudu.Services.Web.Tracing.TraceModule.OnBeginRequest(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Any ideas what is causing this? Is it a Kudu / Azure issue? Can I do anything about it?
There was indeed a new build of Kudu yesterday that caused this regression (build 51.50211.2078). When it was found, and additional build was deployed (build 51.50212.2079) to address it. However, it doesn't take effect until the site is restarted.
Please try restarting your site and confirm that the issue is gone. If you go to the Kudu home page, you can confirm the version before and after the restart.
Related
When on the server where the site is running, it works fine, and no errors show in event log. When hitting via DNS or IP (which is to the load balancer), server error occurs regarding geolitedbpath
I've commented out the reference to this library in the app settings, commented out the calls in the packages.config file, and afaik our application isn't specifically using this. We've run our website for 10+ years and from the point I started, have always seen this error, but never to the point of causing a server error.
This is a new build of K12 portal on new Windows 2016 servers/IIS 10 with plenty of resources etc. The site is a copy of the filesystem of current production which is working.
This is the error in the event logs.
Description:
Message: geolitedbpath ain't presented in web.comfig
Parameter name: geolitedbpath
Exception type: System.ArgumentException
Stack trace:
at CMS.Mvc.Core.GeoProviders.GeoInfoProvider..ctor()
at Global.WriteStateCodeToTheSession(Object sender, EventArgs e)
at CMS.Base.AbstractHandler.CallEventHandler[TArgs](EventHandler1 h, TArgs e) at CMS.Base.AbstractHandler.Raise[TArgs](String partName, List1 list, TArgs e, Boolean important)
at CMS.Base.SimpleHandler2.RaiseExecute(TArgs e) at CMS.Base.SimpleHandler2.StartEvent(TArgs e)
at System.Web.SessionState.SessionStateModule.CompleteAcquireState()
at System.Web.SessionState.SessionStateModule.BeginAcquireState(Object source, EventArgs e, AsyncCallback cb, Object extraData)
at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Turns out the problem was with the load balancer assigning a profile with no configuration .. setting LB to pass-thru resolved the issue.
our web application is experiencing permission errors, from time to time we are receiving this exception :
Access to the path 'D:\home\site\wwwroot\App_Data\TEMP\PluginCache\..' is denied.
this is the call stack
System.UnauthorizedAccessException: Access to the path 'D:\home\site\wwwroot\App_Data\TEMP\PluginCache\umbraco-plugins.RD0003FF29ADB0.list' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalDelete(String path, Boolean checkHost)
at Umbraco.Core.PluginManager..ctor(IServiceProvider serviceProvider, IRuntimeCacheProvider runtimeCache, ProfilingLogger logger, Boolean detectChanges)
at Umbraco.Core.CoreBootManager.Initialize()
at Umbraco.Web.WebBootManager.Initialize()
at Umbraco.Core.UmbracoApplicationBase.StartApplication(Object sender, EventArgs e)
however we are not capable of reproducing the error, we tried restarting, up-scaling, down-scaling, changing instance size - it newer.
We do not have any evidence to prove that it is OS-Update, the main reason why we believe it is related to OS Update is as it is happening on all running instances at the same time (+5/-5 seconds). It comes out of nowhere periodically, and usually on Tuesdays or Wednesdays, however we can not ever reproduce it.
our only assumption is that it is caused by OS-Update (or similar system event - any idea which?) in azure web application. To get a better handle on the issue we would like to know.
can I get last windows update time?
if not can I at least get
system uptime ?
and is it by any means possible to initiate windows update or postpone it until the moment I need it ?
thanks
almir
We have an Umbraco website (version 7.5.11) hosted on Azure Web Apps.
We are experiencing the following exception intermittently (3 times within the past 3 weeks). Once the exception occurs it brings the website down until we republish the home node in Umbraco. At all other times the website is working as expected, including retrieving image files from the server.
Exception type: IOException
Exception message: An unexpected network error occurred. at Umbraco.Core.Cache.HttpRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, CacheDependency dependency) at Umbraco.Core.Cache.HttpRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, String[] dependentFiles) at Umbraco.Core.Cache.DeepCloneRuntimeCacheProvider.GetCacheItem(String cacheKey, Func1 getCacheItem, Nullable1 timeout, Boolean isSliding, CacheItemPriority priority, CacheItemRemovedCallback removedCallback, String[] dependentFiles) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetCacheValues(Int32 id, Func2 func) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetUmbracoMedia(Int32 id) at Umbraco.Web.PublishedCache.XmlPublishedCache.PublishedMediaCache.GetById(UmbracoContext umbracoContext, Boolean preview, Int32 nodeId) at Umbraco.Web.PublishedCache.ContextualPublishedCache1.GetById(Boolean preview, Int32 contentId) at Umbraco.Web.PublishedContentQuery.DocumentById(Int32 id, ContextualPublishedCache cache, Object ifNotFound) at Umbraco.Web.PublishedContentQuery.Media(Int32 id) at Umbraco.Web.UmbracoHelper.Media(String id)
The media file exists, and republishing the home node brought the site back online.
At the time of the exception, no code changes were deployed and no pages were updated / published within Umbraco.
Has anyone experienced something similar, or any ideas what the root cause is?
According to the source code of PublishedMediaCache.cs, the exception is often caused by following issue.
Examine index is corrupted.
Here is a thread on umbraco forum which related to your issue.
Examine corruption issues
And here is the solution for this issue from #Shannon Deminick.
If you are using Azure web apps and are NOT auto-scaling, you should use these settings:
useTempStorage="Sync"
use this feature to store local index files: http://issues.umbraco.org/issue/U4-7614
Remove the {machinename} token from your index path
RebuildOnAppStart="true" - since this should only happen one time
If you are using Azure web apps and are load balancing w/ auto-scaling your front-end workers then:
useTempStorage="Sync"
use this feature to store local index files: http://issues.umbraco.org/issue/U4-7614
You must have the {machinename} token from your index path
RebuildOnAppStart="true" - so that when new sites come online, their indexes are built
... yes in some cases this might not be ideal, please see: https://our.umbraco.org/forum/extending-umbraco-and-using-the-api/74731-examine-corruption-issues#comment-244293
After a deploy to our Azure Web App, we are getting 500 timeouts on any request to the service:
500 - The request timed out.
The web server failed to respond within the specified time.
This has come out of the blue and we cannot determine what's causing it. It seems to take around 230s consistently to time out.
I've enabled all the diagnostic logs in the portal:
But I honestly don't quite know what to look for in the logs. I've scoured through all the files in the following folders but nothing jumps out.
How can I troubleshoot this problem?
The trick to get debug messages is to set stdoutLogFile="D:\home\LogFiles\stdout.log" in your config, instead of the ..\logs path that you had. After changing that, you get an error file under D:\home\LogFiles. Here is the error you get:
Application startup exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentNullException: SMTP server password cannot be null or empty.
Parameter name: smtpPassword
at TransitApi.Infrastructure.Modules.Logging.EmailOutput.EmailLogger..ctor(String recipient, String sender, String smtpUsername, String smtpPassword, String smtpHost, Int32 smtpPort, String environmentName, LogLevel minimumLevel)
at TransitApi.Infrastructure.Modules.Logging.EmailOutput.EmailLoggerProvider.CreateLogger(String name)
at Microsoft.Extensions.Logging.Logger.AddProvider(ILoggerProvider provider)
at Microsoft.Extensions.Logging.LoggerFactory.AddProvider(ILoggerProvider provider)
at TransitApi.Api.Startup.Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
So some kind of issue setting up the mail server. That causes the process to crash, and that it behaves poorly.
But I highly suggest that you upgrade to Core RC2, as RC1 is quite obsolete.
We have some solution that we built against a MOSS farm one of which includes a timer job. This job has been working just fine for months. Recently the administrator enlisted another server into the farm, and our timer job automatically started running on this new machine. As soon as this switch happened our timer job started yielding the error below (found this in the SP logs).
At first I thought it was a rights issue, but the timer service on the machine where it worked before and the new one are running under the same domain account. It seems to be failing while looping the site list in a site collection, on just one of the sites/webs (code snippet below). I know that this domain account has access to this because it works on the other box under same account. Does anyone have any ideas on why this cryptic error is occurring? Or if any special procedure needs to be done on this new machine to ensure it has proper ACL's for all databases in the MOSS farm?
Code:
public static void Main(string[] args)
{
SPSecurity.RunWithElevatedPrivileges(delegate() { setInputParameters(); });
}
private static void setInputParameters()
{
SPFarm farm = SPFarm.Local;
SPWebService service = farm.Services.GetValue<SPWebService>("");
foreach (SPWebApplication webApp in service.WebApplications)
{
foreach (SPSite siteCollection in webApp.Sites)
{
using(siteCollection)
{
siteCollection.CatchAccessDeniedException = false;
try
{
/* Here is the line that it fails on */
foreach (SPWeb web in siteCollection.AllWebs)
Exception:
The Execute method of job definition LMSDataImport (ID 4b37b285-ef8a-407c-8652-391639449790) threw an exception.
More information is included below.
The type initializer for 'Microsoft.SharePoint.Administration.SPPersistedObjectCollection`1' threw an exception.
Exception stack trace:
at Microsoft.SharePoint.Administration.SPPersistedObjectCollection`1.get_BackingList()
at Microsoft.SharePoint.Administration.SPPersistedObjectCollection`1.GetEnumerator()
at Microsoft.SharePoint.Administration.SPAlternateUrlCollectionManager.LookupAlternateUrl(Uri canonicalRequestUri)
at Microsoft.SharePoint.Administration.SPAlternateUrl.LookupCore(Uri uri, SPFarm farm)
at Microsoft.SharePoint.Administration.SPWebApplication.Lookup(SPFarm farm, Uri requestUri, Boolean fallbackToHttpContext, SPAlternateUrl& alternateUrl, SiteMapInfo& hostHeaderSiteInfo, Boolean& lookupRequiredContext)
at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite, SPUserToken userToken)
at Microsoft.SharePoint.SPSite..ctor(SPFarm farm, Uri requestUri, Boolean contextSite)
at Microsoft.SharePoint.Administration.SPSiteCollection.get_Item(String strSiteName)
at Microsoft.SharePoint.Administration.SPSiteCollection.get_Item(Int32 index)
at Microsoft.SharePoint.Administration.SPSiteCollection.ItemAtIndex(Int32 iIndex)
at Microsoft.SharePoint.SPBaseCollection.SPEnumerator.System.Collections.IEnumerator.get_Current()
at LMSDataImporter.setInputParameters()
at Microsoft.SharePoint.SPSecurity.CodeToRunElevatedWrapper(Object state)
at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass4.<RunWithElevatedPrivileges>b__2()
at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)
at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param)
at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode)
at Axian.AxianCalendar.LMSDataImporter.Main(String[] args)
at Microsoft.SharePoint.Administration.SPTimerJobInvoke.Invoke(TimerJobExecuteData& data, Int32& result)
Check the DLLs for SharePoint, do all exists and all are the same version? Try putting a catch for the TypeInitializationException, and see what is wrong inside that exception.
It's not a solution, but as a workaround in the interim, I think my suggestion to one of your other questions here:
How do you instruct a SharePoint Farm to run a Timer Job on a specific server?
will keep you running while you investigate further.
Check the NLB (Network Load Balancing) configuration. Most of the time SharePoint and applications integrated to it fails when NLB changes its state. There is a patch available to solve this problem. Just a suggestion. Not sure if this is the reason. But I have faced a similar issue and the root cause was an NLB bug
A Type init exception just means an exception occurred in the .ctor of the class (as you probably know). The real exception should be in the InnerException property - can you get your hands on this? Likely it's stemming from the database alright, from Microsoft.SharePoint.Administration.SPPersistedChildCollection InitializeFromDatabse method.
Can you look into the sharepoint logs (on that errant server) for information about the database error, it will be there. Reading logs are a pain, but not if you install the ULS Log Viewer feature from http://www.codeplex.com/features
Since the stacktrace has SPAlternateUrl tinkering furhter up the stack, perhaps your zones are misconfigured (and do not include a mapping for this new server's machine name) - granted, it shouldn't fail this bad, but what can you do.
You can filter the ULS logs by source.
-Oisin