I'm having some problem with my scheduled build. This is how it goes:
Created the project entry in my cruisecontrol.net config file. At this time, CC dashboard will show the new project entry, with CCNet status "Unknown". This project will have a scheduledTrigger, so that it runs daily at a specific time.
I trigger a build for the project and the Build is Successful. Now the CCNet status is "Running".
The daily builds happen in a consistent manner for some days. The CCNet status still shows "Running" every day.
At some point (after few days), the CCNet status of this project suddenly changes to "Unknown", after which the scheduled build doesn't happen.
In short, CC scheduled build happens only when the CCNet status is "Running". Once it changes to "Unknown", the scheduled build stops triggering.
In my case, the CCNet status is getting changed from "Running" to "Unknown" without any specific reason, and hence the scheduled build stops.
Any idea why this is happening?
Thanks.
$ccnetINSTALLDIR\server\ProjectsState.xml.
Remove any entries of projects in the node in the ProjectState.xml file. This will fix the issue. Restart ccnet server if needed
One possibile cause for this is that state file is getting deleted. State file location is defined by state tag in the ccnet.
You can check if any clean up process is turning ccnet service off and deleting the state files.
I had a similar issue.
Try shortening the name of the project name or just rename for project in file ccnet.config.
I have shortened the project name, then restarted CCNet service and CCNet status immediately went to "Running".
My scheduled tasks ran without issue after that.
Related
We've been using Gitlab on my current job for a while now, and have encountered some instability which express itself in various ways.
The most recent one: projects that should be deleted are flagged as such but actual deletion never occurs.
Some research has allowed me to see the probable cause of the problem, but not how to resolve it: the ProjectDestroyWorker hasn't run for over 10 days.
Could someone point me to some documentation on the mechanism(s) that trigger the workers, and how to monitor them?
Version: GitLab Community Edition 8.5.0 a513e09
You have a few issues with this kind of problem: issue 15334, issue 20984.
Checking the backtrace from sidekiq.log can help
Merge Requests 5695 and 4341 (for GitLab 8.11) should fix some of those issues, like:
There is a race condition in DestroyGroupService now that projects are deleted asynchronously:
User attempts to delete group
DestroyGroupService iterates through all projects and schedules a Sidekiq job to delete each Project
DestroyGroupService destroys the Group, leaving all its projects without a namespace
Projects::DestroyService runs later but the can? (current_user, :remove_project) is false because the user no longer has permission to destroy projects with no namespace.
This leaves the project in pending_delete state with no namespace/group.
I came across this issue during the migration to hashed storage in v13.5.7
Drop to the ruby console & destroy projects that are pending
deletion
In my particular case this allowed gitlab-rake gitlab:storage:legacy_projects to correctly show 0 so further upgrades were possible.
I have a custom deployment script (*.sh script) defined for my azure deployment.
Just today, I have found that I am unable to publish. I updated my bitbucket repository and after a while I get an error similar to the following:
Command 'starter.cmd deploy_pvl_cont ...' was aborted due to no output nor CPU activity for 180 seconds. You can increase the SCM_COMMAND_IDLE_TIMEOUT app setting (or WEBJOBS_IDLE_TIMEOUT if this is a WebJob) if needed.\r\nstarter.cmd deploy_pvl_content.sh
I have tried a number of things to try to diagnose the problem.
Increase SCM_COMMAND_IDLE_TIMEOUT to 300
Run the script locally (Works)
Set up a new fresh deployment slot and try publishing same commit (Same error)
Tried publishing the previously successful commit (Same error)
Look for useful error messages in a diagnostic log dump (Coldn't find anything more useful)
Tried running the deployment script from the Kudu Console (No output returned, like it didn't actually run)
Tried reverting git to a previous version as suggested by #david-ebbo
Tried simplifying my script to a single echo command with the same results
Not sure what I can do to debug this further. Ideally I would like to get the output of the shell script on the azure host but don't know how to get it. Any ideas?
Updated answer
This is a regression caused by the move to git 2.8.x in Azure. The issue is tracked by https://github.com/projectkudu/kudu/issues/2041.
Here is a very simple workaround (and you don't need to bring in the old git tools): instead of setting your COMMAND to deploy_pvl_content.sh, set it to bash deploy_pvl_content.sh
We'll address the issue, but this workaround will get you going.
Original answer (only leaving for context)
You could be running into some flavor of this issue, which is caused by the upgrade to git 2.8.1 that we just did.
While we're trying to get to the bottom of it, please try this workaround to see if that helps:
Go to Kudu Console
Create a d:\home\bin folder
Copy the old Windows git 1.8.x folder in there. You can get the content from here. If you drag and drop the zip into Kudu console, there is a special unzip drop area that will expand it.
Try your deployment again
What happens to a CruiseControl.NET project if a trigger fires to build the project while it's already building due to someone "Force" building it earlier?
Will the build request of the trigger got queued?
We use CCNET 1.5 and 1.6.
Nothing, while a build is running triggers are not executed. However if you configured a sourcecontrol block for example for svn or p4, modifications during forced build will still be detected and cause another build in consequence.
You can see easier this if you add <queueStatusServerPlugin /> to the serverPlugins in dashboard.config.
When the source control repository is unreachable cruise control keeps going back to check for modifications. While the latest build was successful the dashboard reports failure but cctray reports success.
Is there some way I can catch this scenario and have these two agree?
Yes this scenario can occur when the CCTray hangs on the local PC. If the issue is occuring at the dashboard then it means the IIS Server hangs where the CruiseControl server is running.
To resolve this is to identify where the issue is at. If the issue is at the CCTray level then restart the CCTray. If the issue is at the Dashboard level then restarting the IIS should fix it.
This is actually due to an issue in CruiseControl; not CCTray itself.
If source control fails (say because of a timeout or connection failure) the following will be true:
CruiseControl will set project state to Exception as the project is currently in an error state
CruiseControl will NOT modify the last build status, as a build has not occurred
so if the previous build succeeded - the project will report Success for the last build status
CruiseControl only reports - natively - the last build status via the API that CCTray uses. Getting it to inspect the project status is more complicated and ends up being less efficient. As such the CCTray reports the 'status' as the last build status rather than a hybrid of the two.
The WebDashboard shows the project status and the last build status hence true status of the project is better evaluated.
This issue has several other side-effects; such as projectTriggers firing in this circumstance; as these also do not check the project status.
Ideally CCTray - and projectTriggers, et. al - would check both the project status and last build status and report the outcome as a combination of both.
in our build process we commit created files back to the SVN Repository. To avoid an endless build/commit cycle I use the FilteredSourceControl-Block to ignore commit made by the user "CruiseControl".
It works as expected, as long nothing is committed before the build is finished.
This is what is happening:
1st Commit
CC sees the change and begins the build process
2nd Commit, Build process is still running
Build finishes and does a commit
Now the latest and greatest revision is the one made by CC, which is ignored as I told CC.
But my problem is, that the 2nd commit does not trigger an other build.
Has anyone an idea to work around this problem?
The culprit was that the time on the SVN server and the CruiseControl Server were not in sync. Check this if you have the same problem.