Has anyone else come across this issue:
how to make the pipelines run in turn (first test_1, then test_2, then test_3) every hour?
Several pipelines are defined in pipelines.yml
`
- pipeline.id: test_1
path.config: "/etc/logstash/cinf.d/test_1.conf"
pipeline.workers: 1
- pipeline.id: test_2
path.config: "/etc/logstash/cinf.d/test_2.conf"
pipeline.workers: 1
- pipeline.id: test_3
path.config: "/etc/logstash/cinf.d/test_3.conf"
pipeline.workers: 1
`
How to make the pipelines run in turn (first test_1, then test_2, then test_3) every hour?
The problem of order, if I understand correctly, is solved by the settings in logstash.yml
`
pipeline.ordered: true
pipeline.workers: 1
`
but I can't figure out how to make them run according to the schedule every hour.
I thought about the option to use the schedule in the input sections of the configuration files
`
input {
jdbc {
...
schedule => " 0 * * * *"
...
}
}
`
But then, if I understand correctly, the pipelines will be launched simultaneously at the beginning of each hour, and I need the next one to start no earlier than the previous one ends.
Perhaps someone faced a similar problem? I will be very grateful for your help.
I thought about the option to use the schedule in the input sections of the configuration files
`
input {
jdbc {
...
schedule => " 0 * * * *"
...
}
}
`
But then, if I understand correctly, the pipelines will be launched simultaneously at the beginning of each hour, and I need the next one to start no earlier than the previous one ends.
Related
I've created a cron job in Drone CI using the following command
drone cron add --branch master "foo/bar" "every-5-mins" "0 */5 * * * *"
And when I look in the UI, I can see the cron job has been created. However, it doesn't fire, even though I have it defined in my drone yaml file.
- name: cron-job
pull: if-not-exists
image: foo-image
commands:
- *random
- echo "I am testing a cron job"
when: &cron-demo
event:
- cron
cron:
- every-5-mins
Does something else need to be done to get it to work on schedule?
And also, is there a way to manually trigger a cron job? I can't seem to find anything in the docs.
You might find this helpful.
https://docs.drone.io/api/cron/cron_trigger/
Basically, what you need is something like this.
curl -X POST -H "Authorization: Bearer $DRONE_TOKEN" "$DRONE_SERVER/api/repos/foo/bar/cron/every-5-mins"
I want to know if there is any way to create a ansible playbook able to run a cronjob every 10sec.
Thanks all for your answers
Cron only has resolution down to minutes. The best you can get out of cron is once per minute. To get once every ten seconds, you'll need a script that will run your task and then wait ten seconds, and repeat it 6 times. If you have that script, you can use cron to run it every minute. And of course, you can use the Ansible cron job to set it up on your servers.
Task to create a Cron Job
- name: Sets cron job to run script per minute
cron:
name: Sets cron job to run script per minute
minute: "*"
hour: "*"
day: "*"
month: "*"
weekday: "*"
user: "centos"
job: sh sample.sh > sample.out 2>&1
cron module parameters explained:
minute, hour, day, month, and weekday are configured to specify the cronjob interval at which the job should be executed
user: the user for which the cronjob should be configured
job: the operation to be performed
As we can see that the minimum we can set is per minute. So, to run the operation per 10 seconds, we need to add that support in the script.
Sample script sample.sh
#!/bin/bash
while sleep 10; do
echo "Running after a sleep of 10s"
# Add the business logic here
done
Now, this script will get triggered every minute by the cronjob and the script in turn will run the business logic every 10 sec.
I use crontab to run Laravel schedule, this is my cron entry:
* * * * * php /var/www/mailserver/artisan schedule:run >> /dev/null 2>&1
In App/Console/Kernel.php I just write log file:
Log::info('Schedule excuted');
But in my log file, it seems run sequently in 3 seconds instead of 1 minute
Please help me, how to run cronjob slow down?
The cron statement you provide is telling it to run constantly. Chances are there is just a delay of 3 seconds between the log being called. If you read the scheduling docs you would see that to run every minute you need to do the following:
protected function schedule(Schedule $schedule)
{
$schedule->call(function () {
Log::info('Schedule excuted');
})->everyMinute();
}
Source: Scheduling Docs
I have the following .json configuration for a job that I need to run once every 10 minutes and then do nothing for the remaining time...
"name" : "test.10minjob",
"cron_restart": "*/10 * * * *",
"autorestart" : false,
"exec_mode" : "cluster",
"instances" : 1,
If I run it like this, it will run once and never restart.
If I run it with autorestart set to true, it will restart every time the job exits (i.e. is done).
Am I missing something obvious here?
You could try this cron expression 0 0/10 * 1/1 * ? *
I created it using this tool:
http://www.cronmaker.com
The expression looks very different from yours and also if i put yours into a cron parser it complains about an invalid expression.
I want to set a cron run after an other cron. For example: Cron A finishs at 01:00 PM, cron B will start at 01:01 PM. The problem is I don't know when cron A finishs.
I checked the crontab syntax. It doesn't provide any param for that purpose.
My actual situation is:
# This cron must run first.
? ? * * * /usr/local/bin/php -f /path/select_and_print_to_log_file.php
# two these crons runs at the same time.
0 13 * * * /usr/local/bin/php -f /path/update_user.php
0 13 * * * /usr/local/bin/php -f /path/update_image.php
# This cron runs right after two above cron completes.
? ? * * * /usr/local/bin/php -f /path/select_and_print_to_log_file.php
You can use the batch command inside the first cron to have the second thing being scheduled to run.
Your first job could produce a timestamp when finished.
Then you estimate - for example - that job A needs about 60 to 90 minutes. After 60 minutes, you start job B. Job b looks for the timestamp. If it is present, job B starts, else it waits for a minute and looks again.
After finishing, job B deletes the timestamp, or renames it, maybe from 'todo' to 'done'. You could insert the current date inside the file, to check, whether your estimation is still acceptable, or should be adjusted.
What I do in such cases (commonly a backup scenario where I don't want to thrash the disk by having concurrent backups) is to write a script that cron calls, and in the script have the actual tasks run serially.
Something like:
#!/bin/bash
/usr/local/bin/php -f /path/update_user.php
/usr/local/bin/someOtherTaskToRunSecond
YMMV.