How to search for a string in files with Ant (make sure certain string isn't found in source files) - search

I'd like to search for a string within my source files with Ant. (I'd like my build to fail in case certain string is found within my source files).
So, I should be able to recursively search for a certain string within a file set.
I already found that I can use loadfile task to check whether a string pattern is found within one file. But that seems to be working & sensible only with a single file.
On the other hand, replace task would provide recursively search-and-replace. I guess I could do that before build and replace my string with something that would break the build but I wonder if there is some cleaner solution?
br, Touko

You might consider using fileset selectors to do this. Selectors allow you to choose files based on content, size, editability and so on. You can combine selectors with name-based includes and excludes, or patternsets.
Below is an example. The second fileset is derived from the first, with a selector that simply matches on file content. For more sophisticated matching there is the containsregexp selector. The result is a fileset containing only files matching the string. A fail task with a resourcecount condition is then used to fail the build, unless that fileset is empty.
<property name="src.dir" value="src" />
<property name="search.string" value="BAD" />
<fileset id="existing" dir="${src.dir}">
<patternset id="files">
<!-- includes/excludes for your source here -->
</patternset>
</fileset>
<fileset id="matches" dir="${src.dir}">
<patternset refid="files" />
<contains text="${search.string}" />
</fileset>
<fail message="Found '${search.string}' in one or more files in '${src.dir}'">
<condition>
<resourcecount when="greater" count="0" refid="matches" />
</condition>
</fail>
(Old answer): If adjusting or reusing filesets might be problematic, here's an illustration of a relatively simple alternative.
The idea is to make a copy of the files,
then replace the string you wish to search for
with some flag value in the copied files.
This will update the last modified time on any matching file.
The uptodate task can then be used to look for affected files.
Finally, unless no files matched, you can fail the build.
<property name="src.dir" value="src" />
<property name="work.dir" value="work" />
<property name="search.string" value="BAD" />
<delete dir="${work.dir}" />
<mkdir dir="${work.dir}" />
<fileset dir="${src.dir}" id="src.files">
<include name="*.txt" />
</fileset>
<copy todir="${work.dir}" preservelastmodified="true">
<fileset refid="src.files" />
</copy>
<fileset dir="${work.dir}" id="work.files">
<include name="*.txt" />
</fileset>
<replaceregexp match="${search.string}"
replace="FOUND_${search.string}">
<fileset refid="work.files" />
</replaceregexp>
<uptodate property="files.clean">
<srcfiles refid="work.files" />
<regexpmapper from="(.*)" to="${basedir}/${src.dir}/\1" />
</uptodate>
<fail message="Found '${search.string}' in one or more files in dir '${src.dir}'"
unless="files.clean" />

This was very helpful as a start, but I have a list of strings which should be checked in a fileset.
My current code sofar is:
<property name="search4" value="XYZ"/>
<fileset id="existing" dir="../src">
<patternset id="files">
<include name="content/**/*.txt"/>
</patternset>
</fileset>
<resourcecount property="count">
<fileset id="matches" dir="../src">
<patternset refid="files" />
<contains text="${search4}" />
</fileset>
</resourcecount>
<echo message="Found '${search4}' in files : '${count}'"/>
That works well, but how to expand that so the ${search4} is read from a list. Actually the list can be read from a file containing each search item is on a separate line.

Slightly more concise variation on the first part of #martinclayton's answer:
<property name="log.dir" value="logs" />
<property name="fail.string" value=" FAILED " />
<fileset id="build.failures" dir="${log.dir}" includes="*.log">
<contains text="${fail.string}"/>
</fileset>
<fail status="1" message="One or more failures detected">
<condition>
<resourcecount when="greater" count="0" refid="build.failures" />
</condition>
</fail>

Related

How to integrate xUnit.net with CruiseControl.net

I have a continuous integration server that discovers and runs assemblies with NUnit tests. I would like to add some assemblies with xUnit.net tests to the mix. How would I do that?
Download xunit-build-xyzw.zip from xUnit.net on CodePlex and extract it to, for example, C:\Program Files\xUnit.net. Add this location to PATH environment variable
be sure to have no trailing semicolon
Modify your CC.NET *.build script to discover assemblies by convention, as outlined below
note that command line argument syntax no longer has equals sign
In C:\Program Files\CruiseControl.NET\server\ccnet.config, Merge XML files produced by NUnit runner and by xUnit.net runner, as outlined below
merging happens after build, irrespective of build status
be sure results of test run get deleted in the beginning of build script
Restart CC.NET
Download xUnitSummary.xsl from xUnit.net on GitHub and place it in C:\Program Files (x86)\CruiseControl.NET\WebDashboard\xsl
In C:\Program Files\CruiseControl.NET\WebDashboard\dashboard.config, modify buildPlugins element as outlined below
Restart IIS
Additional info:
CruiseControl.Net – Server Installation at Neal's Blog
Step 2:
<project default="RunTests_xUnit">
<target name="RunTests_xUnit" description="Runs the discovered xUnit.net unit tests" depends="someCompileStep">
<!-- Outer loop to search through a list of different locations -->
<!-- Folders to be searched should listed as a semicolon deliminated list in the 'in' attribute -->
<foreach item="String" in="${TestAssemblyOutputPath}" delim=" ;" property="testsPath">
<echo message="Searching for xUnit.net test suites in ${testsPath}" />
<!-- Inner loop to search for dlls containing unit tests -->
<foreach item="File" property="filename">
<in>
<items basedir="${testsPath}">
<!-- see http://nant.sourceforge.net/release/0.91/help/types/fileset.html for how to include or exclude specific files or file patterns -->
<!-- attempt to run tests in any dlls whose name ends with UnitTestSuite.dll' -->
<include name="**UnitTestSuite.dll" />
</items>
</in>
<do>
<property name="testDLLName" value="${path::get-file-name-without-extension(filename)}" />
<echo message="Testing ${testDLLName} with xUnit.net" />
<exec program="${xunit-console.exe}" failonerror="true" resultproperty="resultVal">
<arg line="${testsPath}\${testDLLName}.dll /xml ${xUnitTestLogsFolder}${testDLLName}-xUnitResults.xml" />
</exec>
<fail message="Failures reported in ${testDLLName}." failonerror="true" unless="${int::parse(resultVal)==0}" />
</do>
</foreach>
</foreach>
</target>
</project>
Step 3:
<publishers>
<merge>
<files>
<file>C:\logs-location\xUnitTestLogs\*UnitTestSuite-xUnitResults.xml</file>
<file>C:\logs-location\TestLogs\*Tests-Results.xml</file>
</files>
</merge>
<xmllogger />
<statistics />
</publishers>
Step 5:
<buildPlugins>
<buildReportBuildPlugin>
<xslFileNames>
...
<xslFile>xsl\xUnitSummary.xsl</xslFile>
</xslFileNames>
</buildReportBuildPlugin>
...
<xslReportBuildPlugin description="xUnit.net Report" actionName="xUnitReport" xslFileName="xsl\xUnitSummary.xsl" />
...
</buildPlugins>

Create file that keeps time and revision number of my project when I compile

Currently the file is only keeping the latest revision. I want to use a different file where the history is kept. Each time ANT is used, I want it to append the time and revision number on that file. On the home screen I will just have a link to that file.
This is how its written now:
<target name="compile-java" depends="prepare,compile">
<exec dir="${project.dir}" executable="tools/version.sh" output="${src.web.dir}/date_compile.jsp">
<arg line="" />
</exec>
<propertyfile file="${src.web.dir}/date_compile.jsp">
</propertyfile>
</target>
I want to append new revision while keeping old revision data too.
This is just a matter of using the BuildNumber or PropertyFile task to create the build version number file and then read the file to get that number, create also a timestamp of the build and then append them both to another file.
Here is a basic idea of how to do it. Starting from that you should be able to write something of the likes of this:
<project default="increment">
<target name="increment">
<tstamp>
<format property="build.time" pattern="yyyy-MM-dd HH:mm:ss" />
</tstamp>
<propertyfile file="build.properties">
<entry key="build.number" type="int" operation="+" default="0" />
</propertyfile>
<property file="build.properties" />
<echo message="Build ${build.number} on ${build.time}
" append="true" file="build.history" />
</target>
</project>
This creates two files: build.properties with your build number (always the last number as it is overwritten on each build) and build.history that contains a list of build numbers and a timestamp of each build.
EDIT : Based on the comments, if the version.sh outputs the revision and date to date_compile.jsp and overrides it on each build then - to keep history - you just need to load date_compile.jsp in your build and append its content to another file, something like this:
<target name="compile-java" depends="prepare,compile">
<exec dir="${project.dir}" executable="tools/version.sh" output="${src.web.dir}/date_compile.jsp">
<arg line="" />
</exec>
<loadfile property="revision" srcfile="${src.web.dir}/date_compile.jsp" />
<echo message="${revision}<br>
" append="true" file="${src.web.dir}/compile_history.jsp" />
</target>
Not 100% sure what you want, but it looks like a combination of <tstamp/>, <echo>, and <propertyfile/>.
You can use <propertyfile> to specify the property file where the build number is stored, and for editing it:
<propertyfile file="${build.prop.file}">
<entry key="build.number"
value="1"
default="0"
operation="+"/>
</propertyfile>
Now, you can include that property file in your build:
<property file="${build.prop.file}"/>
Which will set ${build.number}.
Next, you're going to get the date and time:
<tstamp>
<format property="build.time.stamp"
pattern="yyyy-mmm-dd.hh:mm:ss-zzzzz"/>
</tstamp>
Now, you'll append this to your log file
<echo append="true" file="${build.log}"
message="Building build # ${build.number} on ${build.time.stamp}"/>

CCnet queue priorities not sequencing

I am setting up a nightly build using ccnet 1.6 to run projects sequentially and exclusively, but however I set up the queues when the build is triggered the projects do not start in the expected order as defined by the queue priority. The config below randomly starts queues 1, 2, 3 or 4 first then completes the remaining projects in the defined sequence order (ie 2, 1, 3, 4 or 3, 1, 2, 4 etc). I've tried xlmns 1.6 and 1.5 with same results. I can't see why this is not working, am I missing something?
Thanks
<cb:define name="Common">
<webURL>http://hostname/ccnet/server/local/project\$(CCProjectName)\ViewLatestBuildReport.aspx</webURL>
<workingDirectory>D:\source\$(ProjectName)</workingDirectory>
<sourcecontrol type="svn"/>
<modificationDelaySeconds>60</modificationDelaySeconds>
<triggers>
<scheduleTrigger time="11:05" buildCondition="ForceBuild "/>
</triggers>
<state type="state" directory="C:\Program Files\CruiseControl.NET\server" />
</cb:define>
<cb:define name="BuildProject">
<project name="$(CCProjectName)" queue="SequentialQueue" queuePriority="$(QueuePriority)">
<cb:Common />
<tasks>
<exec>
<executable>$(ANT_HOME)\bin\ant.bat</executable>
<buildArgs>-logger org.apache.tools.ant.XmlLogger -logfile build_log.xml -f build\$(ProjectName).xml</buildArgs>
</exec>
</tasks>
<publishers>
<merge>
<files>
<file>D:\source\$(ProjectName)\build\*.xml</file>
</files>
</merge>
<xmllogger />
</publishers>
</project>
</cb:define>
<cb:BuildProject CCProjectName="API" ProjectName="api" QueuePriority="1" />
<cb:BuildProject CCProjectName="Ack" ProjectName="acknowledgement" QueuePriority="2" />
<cb:BuildProject CCProjectName="Error" ProjectName="error" QueuePriority="3" />
<cb:BuildProject CCProjectName="Monitoring" ProjectName="monitoring" QueuePriority="4" />
I'm guessing this is a bug in CruiseControl.Net, your scenario (variable priority projects in one queue triggered by the same schedule trigger) is not a common one. You should file a bug on cruisecontrolnet.org.
A more usual solution for this kind of setup is having the projects trigger each other using <projectTrigger/>, which also results in a sequential build. You'll probably want to change the triggerStatus parameter so that a failed build does not stop the chain.

JAXB: how to get sources annotated by #generated by ant task?

Is there a way to pass the option -mark-generated, which is applicable to xjc.bat:
%JAXB_HOME%\bin\xjc.bat -mark-generated c:\TEMP\my.xsd
to the corresponding ant task?
<taskdef name="xjc" classname="com.sun.tools.xjc.XJCTask">
<classpath refid="classpath" />
</taskdef>
<xjc schema="my.xsd" destdir="src" package="gen.example">
<produces dir="src/gen" includes="**/*.java" />
</xjc>
You can pass -mark-generated and other options which are not directly supported in an tag nested under the tag, like this:
<xjc schema="simple.xsd" destdir="src" package="gen.example">
<produces dir="src/gen" includes="**/*.java" />
<arg line="-mark-generated"/>
</xjc>
See the Ant Task reference for details. Happy marshalling!

How to search for files containing a particular text string?

How to search for files containing a particular text string using MSBuild?
Thanks guys! I appreciate all of your quick replies!
I've try Grep but I need to read the xml file to see the result.
I've just found out that we can use the task FilterByContent in MSBuild Extension Pack which gives us a direct result in properties & items. I'd like to share it back to you in case you may need it. An example of usage is as below:
<Target Name="ttt">
<ItemGroup>
<files Include="d:\temp\test\**" />
</ItemGroup>
<MSBuild.ExtensionPack.FileSystem.File TaskAction="FilterByContent" RegexPattern="abbcc" Files="#(files)" >
<Output TaskParameter="IncludedFileCount" PropertyName="out"/>
</MSBuild.ExtensionPack.FileSystem.File>
<Message Text="ttt:$(out)" />
</Target>
Nam.
You can find a "grep" task in the MSBuild Contrib project on CodePlex. Haven't used it myself though.
It's not clear whether you want to search of the text in the name or in the file itself.
If you simply want a list of files that their name match particular (simple) criteria I would suggest using the ItemGroup like this:
The Grep taks from the MSBuild Contrib project would look like this
<PropertyGroup>
<MSBuildContribCommonTasksAssembly>$(MSBuildExtensionsPath)\MSBuildContrib\MSBuildContrib.Tasks.dll</MSBuildContribCommonTasksAssembly>
</PropertyGroup>
<UsingTask TaskName="MSBuildContrib.Tasks.Grep" AssemblyFile="$(MSBuildContribCommonTasksAssembly)" Condition="Exists('$(MSBuildContribCommonTasksAssembly)')" />
<ItemGroup>
<FilesToSearch Include="**\*.cs" />
</ItemGroup>
<!-- very simple search -->
<Grep InputFiles="#(FilesToSearch )" OutputFile="out.xml" Pattern="Error" />
<!-- slightly more complicated search (search and extract info) -->
<Grep InputFiles="#(FilesToSearch )"
OutputFile="out.xml"
Pattern="// (?'Type'TODO|UNDONE|HACK): (\[(?'Author'\w*),(?'Date'.*)\])? (?'Text'[^\n\r]*)" />
The Grep task will generate the out.xml file that can subsequently be used to read information from it and use in the build process.

Resources