For certain jobs, we need to do some cleanup or preparation before that job is run again at another node due to failover. This is important especially if the previous run generates some partial result in db. It needs to be cleaned up before the job is run again.
I found #GridComputeJobBeforeFailover. But it doesn't seem the default GridCompute.run()/call() API support that. It will be very useful to add a GridComputeJobFailoverAware interface similar to GridComputeJobMasterLeaveAware. When an closure is an instance of GridComputeJobFailoverAware, then use a ComputeJobImpl with #GridComputeJobBeforeFailover.
But for now, is it true that my only option is to implement my own Task/Job if we want to have something run before a failover?
Yes, for now you need to implement your own GridComputeTask/GridComputeJob classes. However, your suggestion about supporting this annotation for basic runnables and callables is very valid. I have filed a Jira ticket for it, so it will be added to the product.
Related
I have a Terraform script that, after creating several nodes, installs a piece of software on them by telling one node about the other ones using cloud-init. I want this cloud-init piece to only run when this one node is initially created, and not if it's altered or re-created. Now thanks to Terraform plan, Terraform has the information needed, it's telling the user clearly if the node has to be re-created or if it's created the first time at all. But how do I get this kinda information in my script?
The (boring) and manual way is of course to make it a variable that a human enters after reviewing the plan. But this is hardly scalable, secure or sophisticated.
Maybe Terraform is entirely the wrong tool for this kinda job?
Terraform does not expose the information about what action is planned for an object for use in the configuration itself, because Terraform is a desired state system and so the actions are derived from the configuration, rather than the configuration being derived from the actions.
To achieve what you described I think you'll need to arrange for the initialization process to itself remember that it already run in some persistent location. Cloud-init itself remembers when it has run to completion so that rebooting the system won't re-run initialization tasks, but of course that information cannot survive replacing the VM entirely and so you'd need to create a similar marker yourself in a data store that will outlive that particular VM.
One unanswered question down that path is how that external state would get cleaned up if you were to destroy the VM entirely, since the software running in the VM can't tell whether the system is being shut down in preparation for replacement or being shut down in response to just destroying.
i have to do a reptitive task in nodeJS and i've seen there is existing package like this one.
https://www.npmjs.com/package/node-cron
and the actual platform where i'm hosted propose inside cronjob.
https://www.netlify.com/docs/webhooks/
so my question is when it's more interessant to use the platform or a package.
thanks.
From the URL posted i didn't see any method of setting up a cron job using webhooks. Unless you were thinking of setting up a webhook that listens for a post which is sent using a linux cron job or the like?
Regardless, the actual question about using a platform or a package. They have pros and cons, but based purely on your question I would go with the platform.
If you choose to use a package you will have to write the code to call the package (which you need to test, maintain and run). You need to ensure that the node process is always up and running, if it dies or exits that it is re-spawned, that if the operating system reboots the node process gets kicked off again. All these problems are can be easily solved (PM2 for instance) but the fact is you need to think of the problems and solve them yourself or the cron job might not run when you want it to.
When using the platform you know that it is well tested, that it will work as documented, and that it will be resilient to failure modes that you might not be aware of.
I'm trying to debug why the remote caching doesn't work for my use case.
I wanted to inspect the cache entries related to bazel, but realized that I don't really know and can't find what map names are used.
I found one "hazelcast-build-cache" - this seems to keep some of the build and test actions. I've set up a listener to see what gets put there, but I can't see any of the success actions.
For example,I run a test, and I want to verify that the success of this test gets cached remotely. I have no idea how to do this. I would either want to know how to find it out, or what map names I can inspect in hazelcast to find it out.
Hazelcast Management Center can show you all the maps/caches that you create or get created in the cluster, how data is distributed etc. You can also make use of various types of listeners within Hazelcast: EntryListener, MapListener etc.
Take a look at documentation:
http://docs.hazelcast.org/docs/3.9/manual/html-single/index.html#management-center
http://docs.hazelcast.org/docs/3.9/manual/html-single/index.html#distributed-events
I need to spawn a variable number of jobs from one upstream job.
AFAIK, there is no plugin that can do this. The closest one is MultiJob plugin (https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin).
So I thought to create a build step that would use one of the Jenkins APIs (REST, groovy or jenkins-cli) to trigger those builds.
However, if I do that, those builds become "detached" (meaning they do not have an upstream job) and the main job has no linkage with those builds.
So it boils down to this: is it possible to start a job build and tell it who is its upstream?
There is Build Result Trigger plugin. It is literally the inverse of Parameterized Trigger Plugin. Instead of triggering downstream jobs, like the latter does, the Build Result Trigger lets your "downstream" jobs watch/monitor the progress of an upstream job, and trigger based on that result.
This way, your "upstream" job is actually not aware of downstream jobs that are watching it.
Check out the Groovy Plugin.
It'll let you fire as many jobs as you want, and set the upstream cause.
Code Example: http://wiki.jenkins-ci.org/display/JENKINS/Groovy+plugin#Groovyplugin-Retrievingparametersandtriggeringanotherbuild
job.scheduleBuild2(0, new Cause.UpstreamCause(build), new ParametersAction(params))
A related post is here:
How do I dynamically trigger downstream builds in jenkins?
However, from all the answers that I have read it's clear that using the Groovy/Java class hudson.model.Cause.UpstreamCause(currentBuild) does achieve the goal of programmatically triggering another job, but it does not fully establish an upstream/downstream relationship.
When you examine the builds you do not see any upstream/downstream information. The only way to see those is to open the console output of each.
The application must be desgined in such a way that it must support multiple users while commiting into SVN Repositiry. I'm done with the application and the related stuff, however, i'm stuck with this multi-user thing. How i can achieve this? I saw somewhere that for every thread you have to instantiate a separate SVNRepository driver. This tell it's not thread safe..or may be i'm getting the whole thing wrong. Any help on this issue is appreciated. thanks.
I get the above info (the one in italics) from Here .
I'm SVNKit developer, let me explain how the things work.
SVNRepository class represents one SVN connection with it's own credentials. It is thread-unsafe that means that you can perform only sequential operations on it. See this article for more details:
http://vcs.atspace.co.uk/2012/09/21/are-svnkit-methods-reenterable/
So if your application tries to create several commits at the same time, you should use several independent SVNRepository instances. Good news is that no special synchronization code is required, all synchronization is performed on the server side. Another good news is that when commit for a certain SVNRepository object is finished or cancelled, you can reuse the connection to start another commit. But note that if you use http protocol, you can't reuse the same connection to commit on behalf another user even if you change credentials for the connection (SVNRepository#setAuthenticationManager).
To create a commit without working copy use SVNRepository#getCommitEditor which starts the commit transaction. To stop the transaction use either ISVNCommitEditor#closeEdit or ISVNCommitEditor#abortEdit; you can perform other operations on SVNRepository instance, until commit transaction is finished.
ISVNCommitEditor instance should describe your virtual working copy: it tells to SVNKit about your current knowledge of the latest working copy state. If the description doesn't correspond to the real latest change you get "File or directory is out of date; try updating" error.
http://vcs.atspace.co.uk/2012/07/20/subversion-remote-api-committing-without-working-copy/
You can use -1 instead of real revision in ISVNEditor#openFile/openDir to disable checks, but that can cause another problem: you could overwrite changes without knowing about them.
Another option is to commit using real working copies and real changes on filesystem (using SvnOperationFactory#createCommit). But even in this case have a look at the first link to learn which objects can/can't be reused across threads.
Hope this helps, if you have other questions, feel free to ask on SVNKit mailing list.