TFS and MSBuild projects Logging for each project - visual-studio-2012

I'm going build large number (300+) of projects on TFS. I'm considering a proj file to manage these as follows:
<ItemGroup Label="GGX_Projects" >
<MyProj Include="$(Source)Proj1.vcxproj" />
<MyProj Include="$(Source)Proj2.vcxproj" />
<MyProj Include="$(Source)Proj3.csproj" />
<MyProj Include="$(Source)Proj4.csproj" />
<MyProj Include="$(Source)Proj5.vcxproj" />
<MSBuild Projects="#(MyProj)"
Targets="ReBuid"
Properties="Configuration='$(Configuration)'"
RebaseOutputs="true"
ContinueOnError="true"
/>
This actually working fine, but one thing I'm unable to achieve i.e. to get Log for each of the projects mentioned in ItemGroup.
I need separate logs, so that if any project fails, that log can be sent via e-mail.
In addition, on msbuild command line, I tried /distributedFileLogger, but unable to understand how to use this to get log file for each project separately.
Is that a correct approach?
Can anyone please suggest some better solution which could provide separate logging for each project??

Distributed file logger is, as the name suggests, for logging from individual MSBuild nodes within a "distributed" build system, not for logging from individual projects, targets or tasks. You can try creating a simple custom logger, it is fairly trivial, I've written a sample below. Although, you might want to play around with event types for verbosity and use something like NLog rather than appending to files as you might get locks.
msbuild PerProjectLogger.sln /logger:PerProjectLogger\bin\Debug\PerProjectLogger.dll
using System.Collections.Concurrent;
using System.IO;
using Microsoft.Build.Framework;
namespace PerProjectLogger
{
public class Logger : Microsoft.Build.Utilities.Logger
{
public override void Initialize(IEventSource eventSource)
{
var loggers = new ConcurrentDictionary<string, FileInfo>();
eventSource.AnyEventRaised += (s, a) =>
{
var property = a.GetType().GetProperty("ProjectFile");
if (property == null)
return;
var project = property.GetValue(a) as string;
if (project == null)
return;
var logger = loggers.GetOrAdd(Path.GetFileName(project), name => new FileInfo(name + ".log"));
using (var writer = logger.AppendText())
writer.WriteLine(a.Message);
};
}
}
}

The simplest way to achieve this with TFS is to create a single build detention and then on the Process tab you should see an option to pick solutions to build. Here you can add each Project individually. When you do a build the default build process will do a foreach project and build them. You will get a log file in the drop location for each project.

Related

Azure Logic App, Cant get data from CreateFile Function

So I've noticed a strange behavior which I would like to share and see if anyone has had the similar problem.
We are using on Prem solution where we pickup a file or a http event request, map it to an outgoing xml xsd/schema and then create the file later on prem.
The problem was that the system where we save the file does not cooperate so good with the logic app, the logic app failes sometime because the system takes the file before the logic app can finish writing the full content.
The system receiving the files only read .xml files, so we though we should first rename the files to tmp, let logic app create the files and then rename them.
This solution sounded quite simple before we started actually applying it to the logic app.
If we take FileSystem function which has Rename File function and use the parameters “Name” from the create file on prem
{
"statusCode": 404,
"message": "Resource not found"
}
We get the message 404 that the resource is not found, now this complicates a lot of things, I’ve checked the privileges on the account that should not be an issue.
What we also have tried is listing all files in the folder, creating a foreach and then adding a rule and the Rename File function. This makes it work but the logic app does not cope well with receiving a lof of files at ones with that solution.
But the Rename Files works when it’s in a foreach loop and we extract the file names in a list from root folder or normal folder.
But why does it not work with just using the Rename Function? Is this perhaps an azure function bug in the Logic app Rename File Function?
So after discussing with Microsoft support on Azure they have actually confirmed that there is a bug with the “Create File” function.
It looks like all the data and information is actually lost during that functions, the support technicians do not know why that is happening but they have had similar cases which people have reported.
I have not stumbled across any of those posts, but I will post how we solved the problem with a work around.
FYI, The support team has taken the case further so that the developers at azure should look into it, because it’s not just “name” tag which is lost from Create a File, ( it’s all valuable options are actually lost ).
So first we initialize a variable and then actually set the variable name with two steps before we create the file:
The name is set with a temp name and a GUID.
Next step is creating the file with the temp-name used in function “Set Variable Temp FileName”
And on the Rename File function we use the Path from where we store the temp file and add \”FILENAME”
And add the “New Name” which we want to use.
This proved to work but is a workaround, support confirmed that you should be able to just use the “RenameFile” after creating the file with a temp name and changing it to the desired name.
But since Create a File does not send or pass any information at all from this list we have to initialize Variables to make it work.
If anyone has stumbled on the same problem where the Backend system reads the files before they are managed to be created by the logic app and you need some workaround this worked good for me.
Hope it helps!
We recently had the same issue; and the workaround of renaming the file also failed.
The cause seems to be that the Azure On Prem Gateway creates a file (or renames a file), then releases its lock, before checking that the file exists. In the gap between releasing the lock and checking that the file exists, the file may be picked up (deleted) thus causing LogicApps to think the step failed (reporting a 404 error), and thus confusion.
Our workaround was to create a Windows service which we hosted on the file servers (so they'd be able to respond to file changes before anything else on the network). This service has a configuration file which accepts a list of paths and file filters, and it uses the FileSystemWatcher to monitor for new files, or renamed files. When it detects a match it takes out a read lock on the file. This ensure it's not blocked by anything writing to the file (i.e. so it doesn't have to wait for the On Prem Gateway's write aciton to complete before obtaining its own lock), but whilst our service holds its lock the file can't be deleted (so the consumer can't remove the file / buying time for the On Prem Gateway to perform it's post-write read and report success). Our service releases its own lock after a defined period (we've gone with 30 seconds, though you could likely get away with much less). At that point, the consumer can successfully consume the file.
Basic code for the file watch & locking logic below:
sing System;
using System.IO;
using System.Diagnostics;
using System.Threading.Tasks;
namespace AzureFileGatewayHelper
{
public class Interceptor: IDisposable
{
object lockable = new object();
bool disposed = false;
readonly FileSystemWatcher watcher;
readonly int lockTimeInMS;
public Interceptor(string path, string filter, int lockTimeInSeconds)
{
lockTimeInMS = lockTimeInSeconds * 1000;
watcher = new FileSystemWatcher();
watcher.Path = path;
watcher.Filter = filter;
watcher.NotifyFilter = NotifyFilters.LastAccess
| NotifyFilters.LastWrite
| NotifyFilters.FileName
| NotifyFilters.DirectoryName;
watcher.Created += OnIncercept;
watcher.Renamed += OnIncercept;
}
public Interceptor(InterceptorConfigElement config) : this(config.Path, config.Filter, config.TimeToLockInSeconds) { Debug.WriteLine($"Loaded config ${config.Key}: Path: '${config.Path}'; Filter: '${config.Filter}'; LockTime: : '${config.TimeToLockInSeconds}'."); }
public void Start()
{
watcher.EnableRaisingEvents = true;
}
public void Stop()
{
if (watcher != null)
watcher.EnableRaisingEvents = false;
}
private async void OnIncercept(object source, FileSystemEventArgs e)
{
using (var fs = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
Debug.WriteLine($"Locked: {e.FullPath} {e.ChangeType}");
await Task.Delay(lockTimeInMS);
}
Debug.WriteLine($"Unlocked {e.FullPath} {e.ChangeType}");
}
public void Dispose()
{
if (disposed) return;
lock (lockable)
{
if (disposed) return;
Stop();
watcher?.Dispose();
disposed = true;
}
}
}
}

How to create predictable logging locations with sbt-native-packager

I am using sbt-native-packager with the experimental Java Server archetype. I am trying to identify a conventional way to access my log files, and I'm wondering if anyone knows of a common approach here. Since I am using the Java Server archetype, I am getting a symlink /var/log/$app -> install_dir/$app/log, but it feels a little dirty and less portable to just have log4j open /var/log/$app/error.log directly.
[Update]
I ended up creating an object with run time path information:
object MakaraPaths {
def getLogPath = new File(getJarPath, "../logs").getPath
def getConfigPath = new File(getJarPath, "../conf").getPath
def getJarPath = {
val regex = new Regex("(/.+/)*")
val jarPath = Makara.getClass.getProtectionDomain.getCodeSource.getLocation.getPath
(regex findAllIn jarPath).mkString("")
}
}
In my main method, I established a system property based on the new MakaraPaths object:
System.setProperty("logPath", MakaraPaths.getLogPath)
I also used this for my config file:
val config = ConfigFactory.parseFile(new File(MakaraPaths.getConfigPath, "application.conf"))
Ultimately, to load the log file, I used a System Property lookup:
<RollingFile name="fileAppender" fileName="${sys:logPath}/server.log" filePattern="${sys:logPath}/server_%d{yyMMdd}.log">
This gets me most of the way where I needed to be. It's not completely portable, but it does technically support my use case. (Deploying to Ubuntu)
You could use relative path in log4j configuration. Just write logs in logs/filename.log.
During installation symlink install_dir/$app/logs -> /var/log/$app will be created, and all logs will be written in /var/log/$app/filename.log

Entity Framework my DB Context does not have connection when Reference in other Project

So here is my problem Guys
In my Solution,
I have ORM Class Liberary where I've added EntityFramework 5 (so has .edmx containing Context.tt
Designer.cs, edmx.diagram and .tt) files.. So far so good
And I have Project called Repositories and has reference of ORM project above.
In HeaderRepository class of Repositories Project, when I write following code,
using(UFPEntities ufpEntities = new UFPEntities())
{
try{
Header header = ufpEntities.Headers.Single(x => x.VendorId == id);
}catche(Exception e)
{
}
}
Note: intellisense works fine COMPILER DOES NOT GIVE ERROR while writing above code, it happens at Run time
But, I get "No connection string named 'UFPEntities' could be found in the application config file."
App.config is in ORM Project, not in Repository Project where I am dealing with Data as Above.
Can you please help me so that I can CREATE MY MODEL class (such as Header) from Repository Project? or What I am doing wrong so it gives me Exceptions?
Thx in Advance.
The connection string must be in config of entry assembly - it is either web.config for web application or app.config for executable or unit test library. App.config for arbitrary library which is just referenced by executed code is completely ignored.

How to retrieve data from an XML file in a Sharepoint 2010 Feature Event Receiver?

I am following this tutorial and I am trying to setup the code in a Event Receiver.
I need 2 properties to send into their method a SPWeb and string.
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
// is there a way to make this non hardcoded?
SPSite site = new SPSite("http://localhost.com");
SPWeb web = site.OpenWeb("/");
string XMlPath = // get xml file path
CreateGroups(web, path);
}
private void CreateGroups(SPWeb currentSite, string groupsFilename)
{
}
So I tried to use getFullPath but that did not work. I also tried to use MapPath but I did not seem to have access to that.
So how do I get XML file (I think thats what I need)?
You need to dispose of the SPSite / SPWeb object, this is usually done in a using clause.
You don't need to use an absolute path (hard code) in a feature receiver, as the feature is already web/site scoped
your XmlPath usually needs to point to a file on the Sharepoint server, which you also deployed in your Feature - as the feature receiver is running after all the normal files have been deployed, you're good.
Without further ado, slightly different code:
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
//Web scoped feature?
//spWeb = (SPWeb) properties.Feature.Parent;
//assuming Site scoped feature
spWeb = ((SPSite) properties.Feature.Parent).RootWeb;
using (spWeb)
{
string XmlPath = properties.Definition.RootDirectory + #"\Xmlfile\groups.xml"
CreateGroups(spWeb, XmlPath);
}
}
So how do you get your XML file into "\Xmlfile\groups.xml"? Just create a module! (Add new item > Module)
The elements.xml of your module should look something like this:
<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<Module Name="Xmlfile" Path="Xmlfile">
<File Path="groups.xml" Url="Xmlfile/groups.xml" />
</Module>
</Elements>
Of course you will need to add your groups.xml file into that module (Context menu > Add existing item) for this to work.
Also note that you can easily debug feature receivers, just make sure that the deployment configuration is set to "No Activation" (Project Properties > Sharepoint > Active Deployment Configuration) - this way you will need to manually activate the feature on the site (instead of Visual Studio doing it automatically for you in debug mode) - but debugging will work flawlessly.

Using log4net as a logging mechanism for SSIS?

Does anyone know if it is possible to do logging in SSIS (SQL Server Integration Services) via log4net? If so, any pointers and pitfalls to be aware of? How's the deployment story?
I know the best solution to my problem is to not use SSIS. The reality is that as much as I hate this POS technology, the company I work with encourages the use of these apps instead of writing code. Meh.
So to answer my own question: it is possible. I'm not sure how our deployment story will be since this will be done in a few weeks from now.
I pretty much took the information from these sources and made it work. This one explains how to make referencing assemblies work with SSIS, click here. TLDR version: place it in the GAC and also copy the dll to the folder of your targetted framework. In my case, C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727. To programmatically configure log4net I ended up using this link as reference.
This is how my logger configuration code looks like for creating a file with the timestamp on it:
using log4net;
using log4net.Config;
using log4net.Layout;
using log4net.Appender;
public class whatever
{
private ILog logger;
public void InitLogger()
{
PatternLayout layout = new PatternLayout("%date [%level] - %message%newline");
FileAppender fileAppenderTrace = new FileAppender();
fileAppenderTrace.Layout = layout;
fileAppenderTrace.AppendToFile = false;
// Insert current date and time to file name
String dateTimeStr = DateTime.Now.ToString("yyyyddMM_hhmm");
fileAppenderTrace.File = string.Format("c:\\{0}{1}", dateTimeStr.Trim() ,".log");
// Configure filter to accept log messages of any level.
log4net.Filter.LevelMatchFilter traceFilter = new log4net.Filter.LevelMatchFilter();
traceFilter.LevelToMatch = log4net.Core.Level.All;
fileAppenderTrace.ClearFilters();
fileAppenderTrace.AddFilter(traceFilter);
fileAppenderTrace.ImmediateFlush = true;
fileAppenderTrace.ActivateOptions();
// Attach appender into hierarchy
log4net.Repository.Hierarchy.Logger root = ((log4net.Repository.Hierarchy.Hierarchy)LogManager.GetRepository()).Root;
root.AddAppender(fileAppenderTrace);
root.Repository.Configured = true;
logger = log4net.LogManager.GetLogger("root");
}
}
Hopefully this might help someone in the future or at least serve as a reference if I ever need to do this again.
Sorry, you didn't dig deep enough. There are 5 different destinations that you can log to, and 7 columns you can choose to include or not include in your logging as well as between 18 to 50 different events that you can capture logging on. You appear to have chosen the default logging, and dismissed it because it didn't work for you out of the box.
Check these two blogs for more information on what can be done with SSIS logging:
http://consultingblogs.emc.com/jamiethomson/archive/2005/06/11/SSIS_3A00_-Custom-Logging-Using-Event-Handlers.aspx
http://www.sqlservercentral.com/blogs/michael_coles/archive/2007/10/09/3012.aspx

Resources