We have several projects in CruiseControl and I noticed that often 2 or more projects are building at the same time. This seems to be causing conflicts. Often, a build fails and if I look at the reason, it's failing to access some file, and if I wait until the other build completes, I can just force-build the failed build and it will succeed.
How can I configure CC so that it won't run builds concurrently if they are going to conflict with each other?
You have to do all builds that depend on each other, in one queue.
The cruise control project configuration element has two attributes to control concurrency and order of build:
<project name="Project 1" queue="Q1" queuePriority="1">
With queue you can define in which queue the project shall be build, with queuePriority you can define the order of builds when several builds are requested.
So two projects that cannot be build in parallel have to be on the same queue, say queue="Q1". When the second projects is dependend on the first one, the second project must have a higher priority than the first.
You can use the Locking feature to define queus that should not run at the same time.
This is available in version 1.4.3
A sample from the documentation:
<cruisecontrol>
<queue name="Q1" lockqueues="Q2"/>
<queue name="Q2" lockqueues="Q1"/>
<project name="MyFirstProject" queue="Q1" queuePriority="1">
...
</project>
...
<project name="MySecondProject" queue="Q2" queuePriority="1">
...
</project>
...
</cruisecontrol>
Related
I have a solution which builds a number of projects. Some of these projects produce DLLs for redistribution amongst other development teams (for integration). These same DLLs are utilized by our own application and they are primarily modified and updated for the maintenance of this application.
We have run into the problem that unwanted dependencies are sometimes added to the DLLs we provide to third parties. As such, we had created a new configuration within our visual studio solution which only builds the DLLs we provide to third parties. By building this configuration we are able to ensure that our gated-checkin will fail if any new dependencies are added for these projects.
<!-- Build only the Database Access provider DLLs, if a new, unwanted dependency is added, this MsBuild task will fail -->
<Message Text="Build Database Access Providers"/>
<MSBuild Condition="'$(Configuration)'=='Release'" Properties="Configuration=Database Providers Release; Platform=Mixed Platforms" Projects="#(MySolution)" Targets="Build" BuildInParallel="$(BuildInParallel)" />
<!-- ... -->
<!-- Build all the projects in the solution -->
<MSBuild Properties="Configuration=$(Configuration); Platform=Mixed Platforms" Projects="#(MySolution)" Targets="Build" BuildInParallel="$(BuildInParallel)" />
The “Database Providers Release” configuration builds each of the required projects in the project’s “Release” configuration.
Because the first call to MsBuild builds the individual projects in the Release configuration and the full build also builds the individual projects in the Release configuration, we thought that the “Build” target would detect that the build outputs are up to date and would skip rebuilding these projects a 2nd time when we execute the second call to MsBuild. However, the MsBuild script will always rebuild the projects in the second call to the MsBuild task. This is increasing our build time and impacts our build’s code metrics (for instance the number of build warnings increases because the same warnings are reported twice).
If I simply invoke the solution as a whole twice (using the identical build configuration at the solution level), MsBuild will correctly skip all the solution items (I.e. it performs incremental building of the projects). E.g:
<!-- Build all the projects in the solution -->
<MSBuild Properties="Configuration=$(Configuration); Platform=Mixed Platforms" Projects="#(MySolution)" Targets="Build" BuildInParallel="$(BuildInParallel)" />
<!— This 2nd build will skip every project as MsBuild detects the projects have been built. -->
<MSBuild Properties="Configuration=$(Configuration); Platform=Mixed Platforms" Projects="#(MySolution)" Targets="Build" BuildInParallel="$(BuildInParallel)" />
If I run the 2 different build configurations within of Visual Studio directly, it will correctly detect that the various projects have already been built and will skip rebuilding them a 2nd time (I.e. the setup for incremental building of the vcxprojs is correct.)
Is there a way to build 2 solution configurations with MsBuild (where each solution configuration is using the same project configurations) and have MsBuild skip projects which were already built in the first build when they are encountered in the 2nd build?
I'm in the process of moving build projects from CC.Net to Jenkins. However, I need to run a few projects from CC.Net for a while until the migration is complete. Simply put, I want the CC.Net project to do nothing but trigger a job in Jenkins, by using a HTTP request as remote trigger.
I've already set up everything in Jenkins. All I need to do is add the project to CC.Net and have this project trigger my Jenkins job.
I have been unsuccessful in triggering the Jenkins job. I've tried using a HTTP Request as well as an Executable (created a small console application which does the HTTP request). Whenever I try to force the build from my CCTray, the build server tells me "Exception Caught: Input string was not in a correct format.". This should give an idea about what I've tried:
<project name="Test32" queue="venus-02">
<tasks>
<exec>
<executable>C:\key\TestRemoteTrigger.exe</executable>
</exec>
</tasks>
</project>
I don't need any fancy stuff. I just want the CC.Net project to run my executable. Or even better, to simply perform the HTTP Request. But I can't get it to work, and I've come up empty using Google - mostly because every topic I find is about something much more complex which is the opposite of what I need.
Okay, after some stabbing at the keyboard I figured out it was not what I was, or was not, putting inside the section... it was what I was missing outside it. And was apparently not needed.
This will work:
<project name="Test32" queue="venus-02">
<publishers>
<exec>
<executable>C:\key\TestRemoteTrigger.exe</executable>
</exec>
</publishers>
</project>
In CruiseControl.NET, I have two projects set up, one for building and one for deploying build packages.
Our build is largely based around MSBuild, and as it runs the dashboard constantly updates with the latest output from the build. This means that even though a full build may take 15 minutes, you can see exactly where it is, and that it's making progress.
The deploy is run using another tool (VisualBuild, though I'm see the same basic behaviour with other tools like PowerShell). This is another long-running task, but in this case the dashboard is not updated with its output as it progresses. Since a deploy may take a long time, it's hard to tell whether things are progressing or if things have stalled. The output is getting logged to the CruiseControl.NET log, and will display on the dashboard once things are done, but not while the deploy is in progress.
Is there a way to get output from other arbitrary long-running tasks updated on the dashboard in something resembling real time? What makes MSBuild special in this regard?
CruiseControl.Net, since version 1.4, includes support for build listener files: it's a mechanism which allows tracking the execution of long-running tasks by reading from a log file. While this mechanism is generic, and can be used with any tool, CruiseControl.Net by itself ships only with build listeners for MSBuild and NAnt (which means that for those two tools progress is reported automatically, without the need for extra configuration).
For an external tool, such as VisualBuild, called by <exec> task, you would have to plug in your own logger that creates a simple progress file:
<data>
<Item Time="2007-10-14 08:43:12" Data="Starting Build timetester" />
<Item Time="2007-10-14 08:43:16" Data="Starting Target build" />
<Item Time="2007-10-14 08:43:16" Data="Sleeping for 5000 milliseconds." />
</data>
in the location pointed to by the CCNetListenerFile environment variable.
I'm setting up a Cruise Control configuration for one project. I have an <msbuild> task under <tasks>. It seems that I have the option of putting my file deployment under either <tasks> or <publishers>.
Logically I would think it should reside under <publishers> but none of the examples I have seen online work this way.
Should deployment happen within in <tasks> or <publishers>?
It depends. Since CC.Net 1.5 tasks and publishers are quite the same, you can put your any task in the publishers section. The main difference is that, if a publisher fail, your project does not fail (at least it is not shown as failed in CCTray).
For "simple" deployment (for example, copying a dll to a server) I did it under the publishers because this deployment task does not impact the success of the build and it's not that much important if the deploy fails.
If the deployment is a important part of the build (website deployment for example), then I put it in the tasks section to be sure to be notified when it fails.
Deployment task should be in the tasks section.
As deployment part is playing with a final package, the build must succeed.
The publisher section is executed whatever the build result is. If you want to deploy only if all tasks succeed, then write the deployment section as the last task of the tasks section.
So if a task fails, the deployment will not occur.
EDIT: from ccnet documentation:
The publishers section is run after
the build completes (whether it passes
or fails). This is where you aggregate
and publish the build results.
and
Historical Note Publishers and Tasks
were different objects in earlier
version of ccnet. Now they are
interchangeable, and can appear either
in the <prebuild> section, the <tasks>
section, or the <publishers> section
of the ccnet.config file depending on
whether they should be run before,
during or after the build.
reference : http://confluence.public.thoughtworks.org/display/CCNET/Task+And+Publisher+Blocks
I have CruiseControl.NET configured to monitor 10+ .NET projects. All of these projects (web apps, windows services, wcf services, etc...) use shared class library projects so in the ccnet.config I had to set the svn path for each project to the root of the svn repo. If I didn't do it this way then a change to a shared assembly wouldn't trigger projects that depend on it to rebuild.
The issue is that because I've set the svn path for each project to the root of the repo then that means that any change at all triggers a rebuild of every single project, which takes a really long time. How do you get around this issue with using shared assemblies in multiple projects without having every single project rebuild every time a little change is made??
Here's another way to see the issue:
CC Project #1 = svn://repo/WebApps/WebsiteA (references svn://repo/Shared/ClassLibraryA)
CC Project #2 = svn://repo/WebApps/WebsiteB (references svn://repo/Shared/ClassLibraryB)
For CC Project #1, you can't set the svn path to svn://repo/WebApps/WebSiteA, as if you did and ClassLibraryA changed then it wouldn't trigger a build. However if you set the path to svn://repo, then it'll pick up the changes to the ClassLibraryA, but then it'd also trigger CC Project #2.
Any suggestions would be greatly appreciated...
You may want to use the Project Trigger to start a build :
http://confluence.public.thoughtworks.org/display/CCNET/Project+Trigger
If your ClassLibraryA has a project on CruiseControl (whose svnpath is svn://repo/Shared/ClassLibraryA) then your WebsiteA would look like :
<project name="WebSiteA">
<triggers>
<projectTrigger project="ClassLibraryA">
<triggerStatus>Success</triggerStatus>
<innerTrigger name="ClassLibraryA" type="intervalTrigger" seconds="60" buildCondition="ForceBuild" />
</projectTrigger>
<intervalTrigger seconds="300"/>
</triggers>
<cb:svn-block svnpath="repo/WebApps/WebsiteA" />
<tasks>
<...>
</tasks>
<publishers>
<...>
</publishers>
</project>
The answer was to use the sourcecontrol multi block in CruiseControl.NET, which allows you to specify multiple svn paths for each project:
http://ccnet.sourceforge.net/CCNET/Multi%20Source%20Control%20Block.html