We have a website with all the media (css/images) stored in a media folder. The media folder and it's 95 subdirectories contain about 400 total files. We have a Cruiscontrol project that monitors just the media directory for changes and when triggered copies those files to our integration server.
Unfortunately, our integration server is at a remote location and so even when copying 2-3 files the NANT task is taking 4+ minutes. I believe the combination of the sheer number or directories/files and our network latency is causing the NANT task to run slow. I believe it is comparing the modified dates of both the local and remote copy of every file.
I really want to speed this up and my initial thought was instead of trying to copy the whole media folder, can I get the list of file modifications from CruiseControl and specifically copy those files instead, saving the NANT task the work of having to compare them all for changes.
Is there a way to do what I am asking or is there a better way to accomplish the same performance gains?
This sounds like a job for RoboCopy. Use NAnt to bootstrap the execution and let RC do the file synchronization.
Update: digging deeper into the CCNet documentation you'll find the <modificationWriter/> task. Adding this task to your ccnet project will write out an xml file containing information about all the modifications. You should be able to read in the contents of that file in your NAnt script. A suggestion here is to use the NAnt <style/> task to convert the modification xml into a NAnt script containing copy and delete tasks.
Related
This is my first time where I am both learning and implementing automated CICD pipelines in Atlassian bamboo. I have a NodeJS project whose build and deployment plan I configured after much R&D over the net.
In the deployment process, I observed that the deployment is taking very much time as the number of files to be transferred are more in numbers due to node_modules probably. I would like to compress the artifact generated after build steps and want to decompress at server side once the transfer is complete.
I tried finding ZIP in the tool tasks but it is not there. My question is that is it possible in any other way. Is doing it via cmd works & is feasible?
I have a little experience over the Linux commands.
Any help would be highly appreciated.
In my company we use an Ant task including ivy to prepare, zip and publish our projects as artifacts. In the deployment we use an SCP Task to copy the artifact onto our server and an SSH task to unzip it.
So our whole build part is implemented in ant and the only thing our bamboo build does, is checking out a git repository and running the ant script.
That workflow is used for a lot of different projects including nodejs, python, java, c++ or pure text file setups and it works really well.
But a normal script task for zipping should also do the job and depending on the scale of your projects Ant may be an overkill.
I think its possible to use win/linux commands for acheiving your requirement. you would need to write a task to compress the files you can use shell plugin or any other suitable plugin. once the artifact is sent to server you would need a pooling batch program to unzip your artifact at the server end.
In CC project config file I have many MSBuild tasks. Each task is used to build one solution.
Problem is that this requires maintenance if this CC config file each time when new project is added to / deleted from repository.
My idea is to pass to the CC dynamic list of solutions that should be build and execute build one by one as it is done now with "static / old fashion" maintenance of config file.
Does anyone prepare already such configuration?
Presuming you have something akin to the following:
On disk:
./solution1.sln
./solution2.sln
./solutionN.sln
And a single ccnet project:
Msbuild task -> solution1.sln
Msbuild task -> solution1.sln
Msbuild task -> solutionN.sln
What you are asking for is ccnet to react to what is outside of its environment. This isn't possible, however it would be possible to get another tool to do so.
Possible options:
1. Custom Msbuild project
Create a specific msbuild project which finds and invokes msbuild on all solution files it finds. Call msbuild on this project alone. It should be possible to do this with vanilla msbuild, see https://msdn.microsoft.com/en-us/library/z7f65y0d.aspx
2. Batch files
Find all files, and for each file execute msbuild. Ensure the output is logged to an XML file (msbuild switch - I believe) and merge in the result in the publishers section.
See How to do something to each file in a directory with a batch script
3. Single solution
Create a single solution, which contains all the projects from all solutions (1 to N) and call msbuild on this once.
This solution file would be need to be updated each time a new project comes along however.
I am using Jenkins to automate some tasks on a remote server. During these tasks, a script is creating a lot of log files. How can I make these log files available in Jenkins for other? Jenkins won't be creating these files, some script running on my server will. The job will take ~15 days to complete and I would like users to be able to go take a look at the log files anytime in Jenkins.
Jenkins has a mechanism known as "User Content", where administrators can place files inside $JENKINS_HOME/userContent, and these files are served from http://yourhost/jenkins/userContent. This can be thought of as a mini HTTP server to serve images, stylesheets, and other static resources that you can use from various description fields inside Jenkins.
So just put your log files under the userContent directory, others should see them.
I maintain a classic asp web application, and I'd like to try to automate some of the merging to and from branches using CruiseControl (CruiseControl.net specifically). But I don't have any use for a build tool. Basically I would want a commit on /trunk to automagically get merged to a bunch of feature branches that would exist as CC projects.
Is this possible/recommended or is there an easier way to do this?
Some parts of the repository do contain VB6 or .NET code, so I would like to be able to automate builds to those parts in the future, but for the time being I would just basically be using it to automate keeping my static website clean.
It is not the normal responsibility of CruiseControl to affect source control, short of tagging commits. You could have another, say release, branch that CruiseControl monitors. The merging can then be done manually to this branch as required - resolving any merge conflicts at point of merge. CruiseControl would then just 'build' these merge commits.
CruiseControl is a basically a tool for executing commands in a given order based on a trigger. In your case a SourceControlTrigger.
You can setup CruiseControl to execute msbuild, or not, as required.
As you are trying to deploy/package a classic asp site, I guess you only need to copy/zip the source files and deploy them to your web server. This can be achieved as so:
Use nant to copy, and zip, your source files, execute it via an nant task
use curl, executed via an exec task to send the zip file via http(s)
If required you can preceded this with an msbuild task to build your .net projects.
I have a large web project and I would like to publish only the files that were modified by the last checkin. What would be the best way to do this?
I found an article that detailed the process using robocopy. I thought perhaps there was a task in ccnet that did this, but I couldn't find one.
http://msmvps.com/blogs/omar/archive/2008/10/06/asp-net-website-continuous-integration-deployment-using-cruisecontrol-net-subversion-msbuild-and-robocopy.aspx
It depends on what you are doing. Do you use a build script? Your best bet is to let your build script do your publishing, you will have much more control over it.
There is a Modification Writer Task that will create an XML file of the detected modifications in that build. You could then have your build script read that and publish just those files.
There is also a Modification Reader and the doc implies that you can use the read in modifications and publish based on that list:
The ModificationReaderTask can now easily read the modification
file(s) made by project one, into it's
own integration,
making it possible that these can be used by the existing
tasks/publishers of ccnet for project
2
However, it doesn't make it to obvious how you would do this. Perhaps the BuildPublisher will use the Modification Reader info. This might get you started.