We have used a working once per day at 2am backup via crontab for gitlab. A project to make the whole backup/restore process more automated was embarked upon and we're mostly there. The one issue we keep running into is that omnibus updates are fairly frequent as of late, and because the backup then auto-upgrade is staggered, our backups are 24 hours behind, and as such, we get version mis-match errors when restoring on a day immediately following an upgrade.
The more complicated solution is build in a version check to our backup/restore script and postpone 24 hours but that doesn't seem ideal without adding more complexity to both monitor how long the job is postponed compared to the set schedule once per week or every second saturday of the month for tests.
The simpler solution I thought would be the best route was to simply increase the gitlab backup frequency and utilize the STRATEGY=copy option detailed here. That said, after changing the cron to reflect a once every 6 hours backup schedule, I'm not seeing any actual increase in backup count.
Is there a limitation in gitlab in regards to how frequently a backup can be run?
Old Cron (once per day at 2am):
- 0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1 && /root/datadog_success_push.sh
New Cron (that isn't working as expected - “At minute 30 past every 6th hour from 0 through 23.”)
- 30 0/6 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create STRATEGY=copy CRON=1 && /root/datadog_success_push.sh
Related
I tried using both npm packages "cron" and "node-schedule" to make this work, but i couldn't.
Let's say i want to start the cron job at 6.30PM, and starting from 6PM i want it to be executed every 2 hours.
First is that i couldn't make it work if it is not an exact hour , meaning if it is not (1PM,2PM,8PM.....)
So i supposed that it will run at 6PM sharp and not 6.30PM
I tried this pattern:
"0 18-23/2 * * *" which is supposed to work from 18 to 23, escaping two hours [18,20,22,00] but once it's past midnight it will stop til it's 18h again. i want it to keep executing every 2 hours (meaning it executes at 2AM and so on ...)
IS that possible with cron jobs ?
You could do a basic shell script to check if the current date and time are greater than or equal to the date and time you specify before executing the command.
Please note that the date and time are specified in UTC. This will cause the cron to run every 2 hours 30 minutes past the top of the hour but it wont actually activate until after 6PM December 3rd 2019 UTC. Also note that the date command may vary with different flavors of nix* / BSD / OSX.
30 */2 * * * if [ $(date --utc +%s) -ge $(date --utc --date "2019-12-03T18:00:00" +%s) ]; then (command) fi
I believe you are talking about using crontab to schedule jobs.
crontab is designed to let you set the frequency based on many things but it is not designed to become "enabled" a certain time.
so the following will run every two hours
0 0,2,4,6,8,10,12,14,16,18,20,22 * * * (command)
Lets say it is 1st of the month and you want it to run every two hours starting at 6 pm
I would do the following
0 18,20,22 1 * * (command)
0 0,2,4,6,8,10,12,14,16,18,20,22 2 * * (command)
This means that today (based on day of month it would run starting at 6 and tomorrow it will run every two hours. Then sometime tomorrow I would edit the file and remove the first line and change the '2' for the day of month to a '*' (to match the first line I showed you) -- now it will always run every two hours.
tldr; edit crontab to work for two days and then edit it after the go live to work for every day.
I want to create a cron expression which will run the scheduler every 2.5 min of every hour. e.g. 2.5min, 5.0min, 7.5min, 10.0min etc. of every hour.
I am using Spring to create the scheduler. I tried various combination but nothing worked. One of them is as below but it is not working.
#Scheduled(cron = "*/30 */2 * * * *")
Thanks in advance.
That should works for you
0 0/5 0 ? * * *
30 2/5 0 ? * * *
At second :00, every 5 minutes starting at minute :00, at 00am, of every day
At second :30, every 5 minutes starting at minute :02, at 00am, of every day
You are right in this case you need to schedule your task twice using expression like on example.
There is a danger of becoming fixated on the 30 seconds. My problem was that I needed to check 18000 records for updates every month ~ 1 record every 2.5 minutes. I spent too much time trying techniques to run a job at exactly 02:32:30 before I realised that accuracy was not important.
In my situation, I realised I could execute every 2 minutes, updating my full database every 25 days instead of every 31 days.
Alternatively, I could have had 2 cron jobs running every 5 minutes. First, a 2-minute gap, followed by a 3-minute gap.
02:30 02:32 02:35 02:37 02:40 02:42 02:45 02:47
My point is that when the cron job is live, it runs unseen. Obviously, everyone has their own specific problem, but before introducing complexity, consider if it is necessary. As long as the job executes, does it really matter the exact time it ran?
In AWS glue service there is an option to trigger job by custom CRON expression. Before i used this (0/2 * * ? *) cron expression to trigger job for every 2 hours.
Now I need to change the cron expression to trigger every 90 minutes, i.e for every 1 and a half hour. I tried with many cron expressions but that did not triggered for every 90 minutes. Even if i give for 90 minutes, it trigged for every 1 hour.
Can anyone help me out by providing the correct cron expression to trigger job for every 90 minutes ?
You can use the following pattern which was based on Bill Weiss' answer on Server Fault. It was modified to comply with the unique syntax AWS uses (reference here):
0 0-21/3 * * ? *
30 1-22/3 * * ? *
You'll have to define two separate Glue Triggers to accomplish this, each with the same job settings.
If curious, the syntax reads:
Run every 0th minute for every third hour for 0-21 hours
Run every 30th minute for every third hour for 1-22 hours
I'm trying to schedule an app to run every 5 minutes, M-F from 6am-6pm, and every 2 hours, M-F from 6pm-6am. The fixed poll frequency doesn't allow this level of scheduling, so I'm trying to use a cron scheduler.
I set the time zone to America/Chicago and the 5M expression to 0 0/5 6-18 ? * 2-6, and I set the 2H expression to 0 0/120 18-23,0-6 ? * 2-6.
According to both Cron documentation and MuleSoft documentation, this should be setup correctly and should work, and it does work locally. When I publish this to our VPC (US-EAST), I found that the 2H scheduler was working during the day until about 4PM (Central time), so I changed the expressions to this:
5M 0 0/5 0-12 ? * 2-6
2H 0 0/120 12-23 ? * 2-6
Now my 5M scheduler started at 2:15AM and ran until 7AM.
How can I setup two central timezone cron schedules to run 6am-6pm M-F every 5 minutes and 6pm-6am M-F every 2 hours on a VPC server?
MuleSoft confirmed to me that their VPC servers are UTC based, regardless of waht time zone you place on the cron scheduler/poll connector. While not ideal, that means that I needed to back the schedulers up 6 hours (to get them to central time). I suspect that DST will cause the schedule to shift one direction by an hour.
This is a good resource for learning about Mule Quartz/Cron scheduler.
There is also a Free Online cron scheduler formatter.
For the purpose of my scheduling needs, I ended up needing a total of four cron schedulers
0 0/5 12-23 ? * 2-6 - runs M-F, 6am-5:55pm (central) every five minutes.
0 0 0-11/2 ? * 3-6 - runs T-F UTC, but actually runs 6pm-4am M-F every two hours. You only need to run the app up to 4am since it is every two hours, and the five minute schedule will start promptly at 6am. Two things to note here. First, to properly rune very two hours you need to do /2 on the hour slot, instead of /120 on the minute. The minute slot can only handle values from 0-59. Second, the days are set to start on Tuesday because of the 6 hour difference from UTC to Central. If you were to use hours 0-11 (UTC) on Monday, the app would actually start at 6pm on Sunday central time.
This brings us to the two additional schedules, one to capture 12am-4am on Monday (central), and a second to capture 6pm-12am on Friday (central).
0 0 6-11/2 ? * 2 - runs every two hours between 12am-4am (central) on Monday.
0 0 0-6/2 ? * 7 - runs every two hours between 6pm and 12am (central) on Friday.
I am trying to figure out a solution to getting a WebRole to run a Task every morning at 5AM. I have looked at Quartz.Net and the examples on stackoverflow but all of them schedule the task to run every 10 minutes. Are there any examples that show how I can schedule it?
You might also want to check out the Scheduler add-on in the Windows Azure Store (login to the portal at manage.windowsazure.com, head to Add-Ons, then hit App Services and select Scheduler).
Up to 5,000 scheduled jobs/month are free.
Quartz.Net should be good for you.Try to use CronTrigger (or CronTriggerImpl in version 2.x).
Example of cron-expression - "0 0 5 * * ?" - run every day at 5 AM.
Cron trigger sub-expression position meaning:
Seconds - 0 for you (run at 0 second)
Minutes - 0 for you (run at 0 minutes)
Hours - 5 for you (run at 5 hour; it uses 24-hour clock)
Day-of-Month - * - run every day
Month - * for you (run every month)
Day-of-Week - ? - not specified for you (Day-of-Month has been used instead)
Year (optional field) - not used