What is the best way to centralize logging with NLog? - sharepoint

I have been assigned a project with a lot of poorly written code that is based around SharePoint.
It consists of about 15 subprojects, some of them being windows services, some web services, some web applications running inside of SharePoint, some being webparts and even console applications. They all run on the same server and call each other.
There are already many issues in production but they are hard to trace down.
The original developer must have been a fan of either Salinger or Pokémon series judging by his tireless effort to catch all exceptions. Unfortunately, none of them get reported or logged, ever.
My current task is to introduce logging into the whole project so I could find now-invisible exceptions, follow tangled recurring calls and have some stack traces at least. I decided to go with NLog, seeing it's active and cool, as opposed to log4net which is perfectly fine but somewhat not as fancy to my taste.
Because the components are tightly coupled, I want to centralize logging in one file so related errors don't get scattered across the hard drive. Therefore, I am looking to have two or three different log files with five or more projects writing to each of them more or less simultaneously.
What is the best way to configure NLog to centralize logging? Should I have a config file for each project, or should related projects share them? Where should I put config file to log from SharePoint webparts? Am I going to face any permission issues?
I'm using SharePoint 2007.

The easiest way to centralize is probably to simply log to a database, one benefit being that multiple applications and write to the database easier than to the same log file. For each application, configure NLog to log to the Database target, using the same Database target configuration parameters for each. Your NLog.config file might look something like this:
<?xml version="1.0" encoding="utf-8" ?>
<!--
This file needs to be put in the application directory. Make sure to set
'Copy to Output Directory' option in Visual Studio.
-->
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
internalLogLevel="Debug"
internalLogFile="nlog_log.log">
<targets async="true">
<target name="sqlexpress" xsi:type="Database">
<connectionString>
Data Source=.\SQLEXPRESS;Initial Catalog=LoggingDB;Integrated Security=True;
</connectionString>
<commandText>
insert into LogTable(DateTime,Logger,LogLevel,Message,ProcessId,ManagedThreadId) values (#DateTime,#Logger,#LogLevel,#Message,#ProcessId,#ManagedThreadId);
</commandText>
<parameter name="#DateTime" layout="${date:format=yyyy\-MM\-dd HH\:mm\:ss.fff}"/>
<parameter name="#Logger" layout="${logger}"/>
<parameter name="#LogLevel" layout="${level}"/>
<parameter name="#Message" layout="${message}"/>
</target>
</target>
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="sqlexpress" />
</rules>
</nlog>
You could certainly log to files in addition to (or instead of) logging to a database.
I am not familiar with doing this from SharePoint, so I can't comment on any configuration or permission issues that you might run into there.
Here is a link I found where there is a discussion of getting NLog to work in a SharePoint environment:
http://nlog-forum.1685105.n2.nabble.com/Is-anyone-using-NLog-with-SharePoint-td2171451.html
That link appears to put you at the top of the NLog forum instead of the specific post. Search for this text in the forum "Is anyone using NLog with SharePoint" and you should find the right post.
Good luck!

You could also just leverage the existing logging infrastructure in SharePoint and write to the ULS logs. This way your log information can be viewed in a complete context using the ULS log viewer. For SharePoint 2007 see this blog how to write to the ULS log:
SharePoint Trace Logs and the Unified Logging Service (ULS)
With SharePoint 2010 it has become even easier with improvements to the SPDiagnosticsService class where you can use the new WriteTrace method.

Personally I log Exceptions to the Event Logger. And I use NLog for logging details, debug information or tracing.
Since NLog can be easily switched on and off I only activate it when I'm debugging or when I need to inspect an exception in production. I never was a big fan of the default tracing functionality in .NET.
I prefer simple plain text log files. Although logging to a database works great if you don't have too many "log lines" implemented in your code.

I feel like we are working on the same project! Multiple projects consisting of web projects, core dll projects, console apps, services, etc. Unfortunately I'm not working in sharepoint like you are, but I can describe how I am trying to centralize our logging.
We have 1 core .Net framework project.This is where I placed our wrapper class of the log. This project also holds the nlog dlls and the nlog config file. In this core project file you can add this which automatically moves the config when you build projects with a dependency on this core project.
<None Include="Logging\NLog.config">
<link>NLog.config</link>
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
We found that some web projects that don't compile a dll don't automatically pull in the config file, so we will leave that up to the build process. This helps centralize our logging so you only have to manage a single config across them all.
In addition remember when you create a logger per class, the log name should have the namespace in it, so you can make specific targets that filter based off namespace if you want different settings for particular projects.
As for centralizing where the logs end up, we chose to use a file target and specify the full path. This is because on our servers the applications run off a C:\ but we have larger D:\ which can store the logs. In our production servers, we also have multiple servers, so we are using splunk to aggregate all of our logs.
If splunk is out of the question and you are on a distributed system, a database sounds like a good idea as suggested above. If you don't want to stand up an sql instance, there are target wrappers for mongo db as well.
Hopefully helpful, I'm curious if anyone has suggestions or opinions on how I'm doing it as well!

Related

what are all the locations in which connectionStrings can be defined for an iis site?

I'm trying to find all the locations in which a connectionString can be defined for an iis site (to write a script to extract them all).
I know it can be part of a web.config. I would like to have a complete list of files it can be configured in.
Does it make sense for it to be configured in the site code?
Which other configuration files can define a site's connectionStrings?
And a bonus question - how do I know the order of the files in which the connectionString is searched in ?
Thanks,
EDIT:
Additional info - all IIS sites are pure dotnet sites.
Also, specifying the general location of files, rather then file names, is also helpful.
E.g. - connectionStings can be located in external configuration files, whose location is defined at a in an appSettings element in the "%runtime install path%\config\machine.config" file.
Another option is to just link to the relevant docs.
My issue is that I haven't found anything conclusive.
As far as I know, there are several ways to configure connectionstring in ASP.NET applications.
Define it in code. This is the method used by many beginners. Because at this time they focus on code learning and logical understanding. But some people are accustomed to using it if the database is fixed, it does not need to be modified.
SqlConnection connection = new SqlConnection("Data Source=.\\SQLEXPRESS;Initial Catalog=mytest;Integrated Security=True");
Define it in web.config or App.config. The benefit of it is easy to modified connectionstring after publishing application. Developers can change web.config, no need to change code and deploy application again.
<connectionStrings>
<add name="mytest" connectionString="Data Source=.\SQLEXPRESS;Initial Catalog=mytest;Integrated Security=True;" providerName="System.Data.SqlClient" />
</connectionStrings>
Using External Configuration Files. ConnectionString is stored in a independent file for example connections.config. The benefit of it is modifying an external configuration file does not cause an application restart.
<?xml version='1.0' encoding='utf-8'?>
<configuration>
<connectionStrings configSource="connections.config"/>
</configuration>
About list all connectionstrings, you can use ConnectionStringSettingsCollection. It can get a connection by name and provider name.
I found a pretty good source for the locations of the IIS dotnet Framework configuration files - https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc754617(v=ws.10)?redirectedfrom=MSDN#inheritance
You could define additional connectionStrings
In dotnet code.
In env.
In external config files.
For non-IIS scopes - such as user scopes and role scopes.
These 4 options are not specific to IIS.
I don't know where options (3) and (4) are defined, and I'm not sure if this list is complete. But by combing this list and the one in the doc, I think we have 99% coverage of defined connectionStrings.

.NET Core 2 Service Fabric console apps common configuration file

I'm trying to build out a bunch of stateful microservices written in .NET Core 2.1. These are not ASP.NET applications, no http interface, but rather designed as console apps that are consuming messages off a service bus.
I'm having a deuce of a time trying to figure out how to reference my appsettings.json file into the service packages. There's a bunch of articles related to using the appsettings.json file in an ASP.NET service fabric app (basically spelling it out in a webhost builder) but that's not applicable for me, since I'm not using ASP.NET.
I have a single appsettings.json file that all the microservices in this application should be able to use.
I also found this article Service Fabric include additional files which talks about putting it into an sfproj target for MSBuild, but there seems to be no information whatsoever about what the file structure looks like on the target end.
What I tried to do is add targets to my sfproj file like so:
<Target Name="AfterPackage" AfterTargets="Package">
<Copy SourceFiles="appsettings.json" DestinationFolder="$(PackageLocation)\appsettings.json" />
</Target>
and then in my service, I'd try to instantiate an IConfiguration object based upon ..\appsettings.json but as one might guess by now, the service, upon attempting to publish, throws a FileNotFoundException looking for the appsettings.json file, so clearly the file is in a different location relative to the micro service. (Of course, the publish failure just says there was a problem and rolls back, you have to dig into the the SF Explorer to actually see that message)
What should this configuration look like? I can't find any documentation describing what the file system of a stateful microservice looks like, I'm not clear on where I should be putting this common file, or how the services should be accessing the file. Some help would be appreciated.
I hope I got everything right.
Here are the steps:
Put appsettings.json into shared folder in *.sfproj and include it in project using Visual Studio.
Unload *.sfproj and start editing it as .xml
Add the bellow code right after the last <Target> element.
<Target Name="AfterPackage" AfterTargets="Package">
<Copy SourceFiles="appsettings.json"
DestinationFolder="$(PackageLocation)\%(_DeployableServiceProjectReference.ServiceManifestName)\%(_DeployableServiceProjectReference.CodePackageName)" />
</Target>
The _DeployableServiceProjectReference is the internal <ItemGroup> initialized by the previous targets.
These should force the package command to copy appsettings.json file to service's code package directory for each service references in *.sfproj.
Hope this helps.

Web based log4net viewer

We have an asp.net application that logs errors and warnings to xml files using log4net.
We have tried quite a few log4net viewers but didn't find any decent ones.
The latest one we have tried is log4net dashboard which is pretty good but doesn't do exactly what we need.
Our logs are saved in C:\MyApp\Logs[year][month]\log[date].xml
Is there any decent web based viewer that can allow us to view all the logs under C:\MyApp\Logs\
The viewer doesn't need to have a free license for use.
I see a post on their blog for this issue and how to work around it while a new build is dropped.
http://blog.reflectsoftware.com/2013/04/17/directorynotfoundexception/
I can confirm that creating this directory manually resolves the exception that is thrown.
I've used l4ndash in the past and it is a good tool.
Have you tried changing this in the web.config?
This should load all files in that folder.
<datasource name="XML Text file" Description="XML Text file (output from FilAppender or RollingFileAppender)">
<provider type="L4NDashXmlTextFileProvider.XmlTextFileProvider" assembly="L4NDashXmlTextFileProvider, Version=2.4, Culture=neutral"/>
<predefinedsearchfilter value="Filters\DefaultSearchFilters.xml"/>
<providersettings>
<filename value="C:\MyApp\Logs\*.*" />
<SleepOnRename value="100"/>
</providersettings>

Switching from one connectionstring to another when moving from development to cloud

I am working on a cloud application. When I test out the application on my computer I want to have my connection string set as follows in ServiceConfiguration.cscfg:
<Setting name="DataConnectionString" value="UseDevelopmentStorage=true" />
When I publish to the cloud I need to have it set as follows:
<Setting name="DataConnectionString" value="DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=yyy" />
I keep going from one environment to the other and keep having to change the DataConnectionString.
Is there a way that I can automate this? I looked around and can't see any examples but I'm sure some others have the same problem as me.
Thanks,
Nancy
I've answered a similar question here:
Visual Studio 2010 can apply Debug or Release transformations to Web.config, but what about the Azure settings?
While Igorek's approach works perfectly we prefer to make such kind of transformations on our CI server which is also responsible for automatic deployment. The main reason is to restrict access to sensitive data, since only limited number of trusted developers have access to CI server. Also, it helps to keep our code cleaner, since it doesn't include redundant config project. In our case we use Hudson-CI (Jenkins-CI) and MSBuild.ExtensionPack.FileSystem.File (replace action) to transform config files.
In the part of your code that gets the connection string, you can use the #if(DEGUG) preprocessor directive in order to use something or not depending if the compilation is Debug or not, or the if(Debugger.IsAttached) (MSDN) in order to know if the debugger is attached.
You can use CloudConfigurationManager in Azure SDK 1.7 http://msdn.microsoft.com/en-us/LIBRARY/microsoft.windowsazure.cloudconfigurationmanager
This starts by looking in the ServiceConfiguration.cscfg e.g. ServiceConfiguration.Cloud.cscfg for config setting. If it isn't there it falls back to web.config and app.config
For example
CloudConfigurationManager.GetSetting("StorageConnectionString")
Will look in the appropriate cscfgfile for StorageConnectionString setting, then it will search the web.config and then app.config.

SharePoint and Log4Net

I'm looking for best practices to integrate log4net to SharePoint for web request, feature activation and all timer stuff.
I have several subprojects in my farm, and I would like to have only one Log4Net.config file.
[Edit]
Not only I need to configure log4net for the web application, which is easy to do (I use global.asax, and a log4net.config file, so I can modify log settings withtout reloading the webapp), but I also need to log asynchronous events:
Event Handler (like ItemAdded)
Timer Jobs
...
I implemented this recently and came up with a solution that worked for me.
Deploy your log4net config file to the 12 hive and the log4net dll into the GAC using a globally scoped solution. Then in your application code explicitly initialize log4net from the location of your global file. This allows you to log feature receiver, timer jobs and web application code.
[assembly: log4net.Config.XmlConfigurator(ConfigFile =
#"C:\Program Files\Common Files\Microsoft Shared\" +
#"Web Server Extensions\12\CONFIG\log4net.config", Watch = true)]
see here http://www.codeproject.com/KB/sharepoint/SharepointLog4Net.aspx
Firstly, you will need to modify the web.config where your SharePoint virtual directory resides. This is because you'll need to add SafeControl entries to trust the log4net assembly. You can update the web.config programmatically using the SPWebConfigModification class in a feature receiver. As you have to modify web.config anyway, you may want to consider including your log4net config inside and not set up an external log4net config.
However, if you'd still like to do this, it may work if you add the following to the web.config file:
<configuration ...>
...
<configSections>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" />
</configSections>
<log4net configSource="log4Net.config">
...
</configuration>
The log4net.config file should then be able to live alongside your web.config. As Nat says, you could deploy this file as a solution package.
Assuming you are attempting to run a minimal trust, you will need to update your Code Access Security file to include the log4net assemblies as well. All of your custom SharePoint code should then automatically use your log4net configuration.
You could release the config file as part of the solution package(s) to the 12 hive (use STSDev) to create any packages). This would give you a set location for the config and any changes to it can be released in a controlled manner (i.e. no need for manual editm, just roll back and re-install the solution).
I developed a log4net feature and packaged it in a wsp file. The feature receiver adds an httpmodule to the the web.config and the httpmodule loads the log4net.config from the layouts direcory when the application start event is raised in the http module.

Resources