So, I'm having this odd issue with my CruiseControl.net project. I have a ccnet.config file that goes and pulls from my bitbucket repo, runs a basic build, then runs some unit tests, and then finally copies the tested files out to the webroot. Seems pretty basic, even for being my first time using CC.Net.
I, of course, would like this to run when I do commits. Failing writing some sort of web page that bitbucket can invoke through it's services framework, I settled for having CC run a scheduled build, and only run the full build if there's source control changes.
After some googling, I found out that in 1.6, it has a cronTrigger. Excactly what I need. So, I add the following chunk to my config:
<cronTrigger>
<cronExpression>*/5 * * * *</cronExpression>
</cronTrigger>
Then it triggers fine, but then I get a loop of this in the console:
[HMS:INFO] Calling hg log -r tip --template {rev} --noninteractive
[HMS:DEBUG] Starting process [C:\Program Files (x86)\TortoiseHg\hg.exe] in working directory [C:\build\hms\src] with arguments [log -r tip --template {rev} --noninteractive]
[HMS:DEBUG] Not setting PriorityClass on [C:\Program Files (x86)\TortoiseHg\hg.exe] to default Normal
[8:DEBUG] [HMS C:\Program Files (x86)\TortoiseHg\hg.exe] standard-error stream closed -- null received in event
[7:DEBUG] [HMS C:\Program Files (x86)\TortoiseHg\hg.exe] 60
[7:DEBUG] [HMS C:\Program Files (x86)\TortoiseHg\hg.exe] standard-output streamclosed -- null received in event
[3:DEBUG] [HMS C:\Program Files (x86)\TortoiseHg\hg.exe] process exited event received
[HMS:INFO] No modifications detected.
This loop continues until I go in and remove the cronTrigger block. I don't really want this thing busy-waiting on my server for a change, and I'm sure BitBucket doesn't smile upon several requests per second.
So, in short, how do I stop this thing from molesting bitbucket? Stop using CC.Net? Resign myself to hitting the "Build" button?
Also: my ccnet.config file for reference: http://pastebin.com/6jSPxVTk
I said screw it, I moved to Hudson instead. A bit heavier on my server, but it supports a URL to post to, so I can drop that into BitBucket's "Services" option, so no polling.
I also get a free Jabber bot, and Chuck Norris.
Related
Background
I have a JetBrains Plugin that I am developing. Recently, I moved from a Windows to a Ubuntu system. I am trying to set up everything correctly as before. Note: I am fairly new to Linux.
Issue
I am having an apparent file permission issue (as can be seen in the error section of this question) when I ran the following Gradle script. Note: Whenever I build the project, this Gradle script is automatically called. It also worked correctly on Windows.
If I comment out the copy{...} closure then everything works correctly. I just need to manually copy over the required file.
tasks.create(name: "copyJar_v${project['version']}") {
group GROUP_CHROMATERIAL
def mostCurrentJarFile = "ChroMATERIAL-${project['version']}.jar"
// comment this out and there are not errors, but I need to do this copy manually
copy {
into '/' // Copy into project's root folder
from 'build/libs', {
include mostCurrentJarFile
rename mostCurrentJarFile, 'ChroMATERIAL.jar'
}
}
}
Error
FAILURE: Build failed with an exception.
* Where:
Build file
'/home/ciscorucinski/IdeaProjects/ChroMATERIAL/ChroMATERIAL/build.gradle' line: 100
* What went wrong:
A problem occurred evaluating project ':ChroMATERIAL'.
> Could not copy file '/home/ciscorucinski/IdeaProjects/ChroMATERIAL/ChroMATERIAL/build/libs/ChroMATERIAL-2.5.1.jar' to '/ChroMATERIAL.jar'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 0.139 secs
/ChroMATERIAL.jar (Permission denied)
오후 12:20:39: Task execution finished 'copyJar_v2.5.1'.
What I tried
I have been using the UI to modify the file permissions of the Plugin Project Folder within IdeaProjects.
I gave everyone Create and delete files permissions and I Change Permissions for Enclosing Files...
Everyone has Read and write access to files and Create and delete files access to folders.
Clicked Change
Run the Gradle script again ... same error message
Thoughts
It appears that Linux is not letting Gradle modify these files. I can comment out the code and do everything myself, but I need to allow Gradle to have higher control. Don't know how though.
I notice that when I go back to Change Permissions for Enclosing Files... the shown permissions are not what I selected! Others has Read-only access to files and Access files access to folders. I don't know if this is common Ubuntu behavior, a bug, or something else.
It would be nice to know how to give as restrictive as possible access while fixing this.
Well, it is true -- you cannot (and should not) copy files into the Root folder in a linux box. You could if you ran the script as sudo, but that is a bad idea.
Edited
Since you want to copy to the root of the project, you can use ${projectDir} or ${rootDir}.
Also, you should be able to do this without the hassle of a closure -- and it makes your script cleaner, IMHO -- by using the built in Copy task:
task copyClientLoc(type: Copy) {
from "build/libs/"
into "${rootDir}"
include "ChroMATERIAL-${project['version']}.jar"
fileMode = 0644
}
Had cruise control.net (v1.8.5.0) up and running for about 6 months on 4 projects, no issues.
In the past week or so, one project reports a failure... even though it compiles correctly and no unit test errors are reported as they were previously on failures. None of similiar reports on SO or elsewhere describe or correct this problem.
Here is dashboard status:
BluePrint Failure Breakers : support2 Failing Tasks : NUnitTask
In the build report log where it would normally show which tests failed, it shows no failures just the unit startup message:
ProcessModel: Default DomainUsage: Single Execution
Runtime: net-3.5
Normally here for an nunit error the erroring test #s and names that failed and the nunit assert failure message. In a passing situation it would display xml for each test.
Can someone point me somewhere else what might be causing this or where else I might look to find out why?
When I run nunit manually on the same tests the cruise control service has checked out the build server, everything works fine.
I've tried restarting both the cruise control service and IIS, same result.
When I look in the ccnet.log file on the server, it doesn't report anything more specific than whats already in the build log above:
Done Building Project
"C:\Users\j\Documents\CIWork\BluePrint\BluePrint.sln" (Build
target(s)).
Build succeeded.
0 Error(s)
Time Elapsed 00:00:13.99
2017-01-20 18:50:36,869 [BluePrint:DEBUG] Running unit tests:
C:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console.exe
/xml=C:\Users\j\Documents\CIWork\BluePrint\logs\LatestTests.BluePrint.xml
/nologo
C:\Users\j\Documents\CIWork\BluePrint\Tests\bin\x64\Release\TestBluePrint.exe
2017-01-20 18:50:36,869 [BluePrint:DEBUG] Starting process [C:\Program
Files (x86)\NUnit 2.6.3\bin\nunit-console.exe] in working directory
[C:\Program Files
(x86)\CruiseControl.NET\server\BluePrint\WorkingDirectory] with
arguments
[/xml=C:\Users\j\Documents\CIWork\BluePrint\logs\LatestTests.BluePrint.xml
/nologo
C:\Users\j\Documents\CIWork\BluePrint\Tests\bin\x64\Release\TestBluePrint.exe]
2017-01-20 18:50:36,953 [BluePrint:DEBUG] Not setting PriorityClass on
[C:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console.exe] to default
Normal 2017-01-20 18:50:37,686 [20:DEBUG] [BluePrint C:\Program Files
(x86)\NUnit 2.6.3\bin\nunit-console.exe] ProcessModel: Default
DomainUsage: Single 2017-01-20 18:50:37,686 [20:DEBUG] [BluePrint
C:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console.exe] Execution
Runtime: net-3.5
My configuration has not changed in 6months, here's the relevant task from ccnet.config:
<nunit>
<path>C:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console.exe</path>
<outputfile>C:\Users\j\Documents\CIWork\BluePrint\logs\LatestTests.BluePrint.xml</outputfile>
<assemblies>
<assembly>C:\Users\j\Documents\CIWork\BluePrint\Tests\bin\x64\Release\TestBluePrint.exe</assembly>
</assemblies>
</nunit>
As I was posting this I recalled that one of these tests downloaded a fair amount of data from internet and cached it. Running it from the console worked, but running it from system account as service must've been slower to exceeded default cruisecontrol timeout.
Weird that no timeout error was in the log. However copying the cached files from the admin account to systemprofile account fixed it. Will also see if the nunit test can be modified to automatically time itself out.
Description
We are in a current project based on MVC4/Umbraco using Azure Websites to host it.
We are using SCM_BUILD_ARGS to change between different build setups depending on which site in Azure we deploy to (Test and Prod).
This is done by defining an app setting in the UI:
SCM_BUILD_ARGS = /p:Environment=Test
Earlier we used Bitbucket Integration to deploy and here this setting worked like a champ.
We have now switched to using Git Deployment, pushing the changes from our build server when tests have passed.
But when we do this, we get a lovely error.
"MSB1008: Only one project can be specified."
Trying to redeploy the same failed deployment from the UI on Azure works though.
After some trial and error I ended going into the deploy.cmd and outputting the %SCM_BUILD_ARGS% value in the script.
It looks like the / gets dropped from SCM_BUILD_ARGS but only when using Git deploy, not Bitbucket Integration or redeploy from UI.
Workaround
As workaround I have for now added a / to the deploy.cmd script in front of the %SCM_BUILD_ARGS%, but this of course breaks redeploy, since we then have //p:Environment=Test in the MSBuild command when the value of %SCM_BUILD_ARGS% has been inserted.
:: 2. Build to the temporary path
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
:: Added / to SCM_BUILD_ARGS
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
) ELSE (
%MSBUILD_PATH% "%DEPLOYMENT_SOURCE%\www\www.csproj" [....] /%SCM_BUILD_ARGS%
)
Question
Anyone know of a better solution for this problem or is it possibly a bug in Kudu?
We would love to have both deploy from Git and Redeploy working.
Could you try changing from "/" to "-"? For instance, AppSettings from /p:Environment=Test to -p:Environment=Test, see if it helps.
-p:Environment=Test did not work for me, the setting which worked for me at the time of this writing (September 2015) was
-p:Configuration=Test
There is clearly a Kudu bug in there, and you should open an issue on https://github.com/projectkudu/kudu. But for now, I can give you a workaround.
Instead of using an App Setting, include a .deployment file at the root of your repo, containing:
[config]
SCM_BUILD_ARGS = /p:Environment=Test
I think this will work in all cases. I suspect the bug has to do with bash messing up the environment in post receive hook scenarios, which only apply to direct git push but not to Bitbucket and Redeploy scenarios.
UPDATE: In fact, it's easy to see such weird bash behavior. Try this:
Open cmd.exe
Run: set foo=/abc to set a variable
Run bash
From bash, run cmd to launch a new cmd on top of bash (so cmd -> bash -> cmd)
Run set foo to get the value of foo
Result:
FOO=C:/Program Files (x86)/git/abc
So the value gets completely messed up. The key also gets upper cases, though that's mostly harmless. Strange stuff...
I have a node.js project that compiles less files to css when I start the app. I do this by modifying the start script in package.json like so:
{
// omitted for brevity
start: { lessc public/stylesheets/styles.less > public/stylesheets/styles.css; node app.js; }
}
This works nicely locally, but not at all on my Windows Azure instance. Either because less needs to be installed globally on the machine for this to work, or because Azure doesn't run npm start. Or both. Either way, I need another solution!
I thought custom deployments was the answer (I'm using git remote deployment) and I tried modifying the deploy.cmd to include
call "lessc public/stylesheets/styles.less > public/stylesheets/styles.css;"
No joy. I even tried
call "%SITE_ROOT%/node_modules/less/bin/lessc %SITE_ROOT%/public/stylesheets/styles.less > %SITE_ROOT%/public/stylesheets/styles.css;
Am I coming at this the wrong way? How can I keep the compiled css files out of my source control and compile them on the server after deployment to Azure?
Thanks!
OK, I finally have this going, I think.
For some reason, even though the physical file is on the disk (I can see them with my FTP client), Azure is not letting me run lessc in the \node_modules\less\bin folder, but it does let me run the version in the \node_modules\.bin folder.
In the end, I added the following lines to my deploy.cmd file, and it worked!
IF NOT DEFINED LESS_COMPILER (
SET LESS_COMPILER=%DEPLOYMENT_TARGET%\node_modules\.bin\lessc
)
call %LESS_COMPILER% %DEPLOYMENT_TARGET%\public\stylesheets\styles.less > %DEPLOYMENT_TARGET%\public\stylesheets\styles.css
We have started migrating our one of several projects to team city as part of CI. Below is how we have setup teamcity build. We are trying to deploy WebSite.
1) Build Step 1 (Package installation)
Using "command line " runner type install required package.
2) Build Step 2 (Build)
Using Runner type "Visual Studio (sln)" (Visual Studio 2010) build website.
3) Build Step 3 (Deploy Web Site)
Using ".Net Process Runner", deployer.exe (x86 built with .Net Framework 4) deploy site.
Deployer.exe reads config file. Config file contains "BuildId", "Environment" and "Servers" where we want build to be pushed.
<buildType id="bt52">
<env name="Debug">
<server path="SERVER1" />
</env>
<env name="QA">
<server path="SERVER2" />
<server path="SERVER3" />
</env>
<env name="UAT">
<server path="SERVER4" />
<server path="SERVER5" />
</env>
</buildType>
Deployer.exe is called with required parameters as below. Which reads config and deploys site to Server2 and Server3.
Deployer.exe "bt52" "QA" "siteQA" "E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" msdeploy.exe
Problem area is step #3.
When we run deployer.exe using .Net process runner as part of team city we see its hanging and not responsind sometime even for 45 minutes. When we try to execute same deployer.exe from build server using command line script executes within couple of seconds.
E:\TeamCity_custom_applications\deployer>Deployer.exe farm1-1 QA siteQA E:\BuildAgent\work\2483052e33e5e1e8\src\diy\ msdeploy.exe
Info
: Processing batch run ... Info : Processing command ...msdeploy.exe
-verb:sync -source:contentPath="E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" -dest:contentPath="siteQA",wmsvc="SERVER2",userName="*****",password="******",authType="Basic"-skip:objectName=filePath,absolutePath=web.config -skip:objectName=dirPath,absolutePath="bin" -enableRule:DoNotDeleteRule -allowUntrusted Info : output >>Total changes: 0 (0 added, 0 deleted, 0 updated, 0 parameters changed, 0
bytes copied) Info : error >>(none) Info : ExitCode >> 0 Info :
Processing command ...msdeploy.exe -verb:sync
-source:contentPath="E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" -dest:contentPath="siteQA",wmsvc="SERVER3",userName="******",password="******",authType="Basic"
-skip:objectName=filePath,absolutePath=web.config -skip:objectName=dirPath,absolutePath="bin" -enableRule:DoNotDeleteRule -allowUntrusted Info : output >>Total changes: 0 (0 added, 0 deleted, 0 updated, 0 parameters ch anged, 0
bytes copied) Info : error >>(none) Info : ExitCode >> 0
Info: Deploy Script Complete.
One more thing we observed is running deployer.exe through teamcity I see that site content gets copied but only for 1 server and teamcity build status stays in "Running" mode. I am wondering if someone can please put little bit of insight on how can I look into this issue.
Update 1:
Thanks for your time looking into it !! What we ended up doing is, Instead of running command "msdeploy.exe" from "cmd.exe" we added "msdeploy.exe" location as Environment variable and executed "msdeploy.exe" in loop for # of servers. This has resolved issue of hanging. Now I am just curious to know why would it behave in such manner where if you execute "msdeploy.exe" from "cmd.exe" it would hang while running directly "msdeploy.exe" it would execute successfully. Any insight into same would be greatly appreciated.
Update 2:
I have added image which explains behavior using process explorer. If we kill msdeploy.exe from process explorer than for next all deployments to that server will not have the issue of build hanging. Please see below image
To be honest, it sounds like you're running into issues with redirecting input/output streams. TeamCity is running your application in a totally headless environment and then you, in turn, are attempting to redirect and parse the output of msdeploy.exe
If that's the case, I'd recommend looking into using the MSDeploy API instead of msdeploy.exe. The latter is just a command line wrapper for the former, so all the functionality is available to you. There's a sample deployment application available on the IIS blog if you need help getting started.
It seems you have NUnit build step configured in TeamCity and invoke cmd.exe from your test. This looks like an issue with the test code then. Most probably it will reproduce without TeamCity if you run the test in question with NUnit directly.
As Richard noted, most probably the issue root cause is related to stdin/stdout processing.
If you want to fix it in your code, you can try to experiment by explicitly closing stdin or the other way around, try writing something into it, etc.
Work around we did is, we observed msdeploy doesn't take more than 3-5 seconds to execute and deploy (Even for our biggest project which is almost 300mb website). So we set timeout of 20 seconds. So far since last 1 weeks we have not seen any issue with it and hopefully it will not cause more trouble but still we are not sure why such behavior.