I have a PHP scraper running every night on a very large site. Crontab launches the script at 2am and pkill it at 7am. Now I am concerned that brutally killing the script might result in data loss. Let's say that crontab calls the script off while the script is busy writing my scraped data into the database, then the next day the database will refuse that last/first record because it is already present (even if not completely).
Is there any way I can freeze the script with crontab? (That is, without adding a sleep() to my script)
Let's say that crontab calls the script off while the script is busy writing my scraped data into the database
That would be a problem, since you will run into some transaction timeout or something, if you stop your process externally. A better way would be to let the script halt/pause on its own. You could for example define some marker file that is checked by the script periodically so that the script can halt/pause in a controlled way.
Having one large cronjob that can't be interrupted is usually a sign of bad design for a number of reasons.
Most notably, you can't interrupt the run for no reason whatsoever or you'll end up with corrupted data. This can become a big problem in case you have an unexpected power loss or server crash.
Also, it doesn't scale. If you need to process more data, you can't scale it to multiple servers. If you have run times of a few hours now, you may end up exhausting a complete server very soon.
I would recommend to seriously rethink the functionality of this cronjob and restructure it so you have a number of smaller tasks that are queued up somewhere. (It can even be the database.) You could then mask the SIGINT and SIGTERM signals when processing a single task and check the received signals in between tasks. This will allow you to notify the process using either of the aforementioned and have it shut down gracefully.
That being said, things do break and servers do crash. I also urge you to work out plans for data recovery in case the cronjob breaks down while working on something.
Related
I need to set a Timeout, in a JCL step that calls a Unix script through bpxbtach. I did it with
//STEPX EXEC PGM=BPXBATCH, PARM='sh /x.sh',TIME=(,10)
However, After some time I realized that does not include the time in the queue. they say " This run time refers to actual execution time only, and does not include the time that the job spends in the INPUT or INPUT HOLD queues" https://supportline.microfocus.com/documentation/books/rd60/cbwjto.htm
That is microfocus JCL, but I verified the behavior is that on IBM Z too.
So even if I set the timeout to 10 seconds, the step can take several minutes if the queue is attending other things. I need a timeout that kills the step no matter the reason it took so long. I haven't been able to find what I need. Please help.
z/OS batch really isn't the best choice for time-critical work. As you figured out, the JCL "TIME" parameter is about CPU time consumption, not an elapsed time control. If this is a business-critical need, then by all means talk to your z/OS administrators - they can certainly configure your system such that your job is very likely to run without delay, but this isn't usually default behavior.
You don't provide a lot of detail as to what else your job might be doing and how it gets submitted. If you have the ability to control how your job is submitted, one option might be to spawn your shell script directly rather than submitting a batch process to run your script.
For example, what you've described is submitting JCL that spawns BPXBATCH, then BPXBATCH spawns your shell script. Instead, you might write a small C program that simply calls "spawn()" to run the shell as a distinct UNIX process - that's not difficult, depending on how you're submitting the JCL you shared. You cut out the need for the batch job - just run your script directly.
If you're running in a TSO environment, the OSHELL command lets you interactively run your script. You can even automate the whole process with a simple REXX script, and none of this requires a pass through a batch initiator.
If your site runs SSH or similar, you might consider launching your script through an SSH command - this even works across a network. SSH lets you launch a shell session and pass a command for execution...again, there's no JCL or input queue here.
If your administrators would allow it, another alternative would be to run your JCL via a "START" command. Unlike batch JCL, when a START command is encountered, the work you're starting runs immediately - there's no input queue for started tasks. Start commands can be issued from JCL too, and since they're issued as the JCL is scanned and not when the job starts, these are fairly immediate too.
Inside your shell script, it's pretty easy to setup an elapsed time limit - there are examples here.
I see a couple of problems in your code...
//STEPX EXEC PGM=BPXBATCH, PARM='sh /x.sh',TIME=(,10)
First, you have a space between BPXBATCH, and PARM= which will not execute your shell script and may result in a JCL error.
Second, you are using the TIME parameter of the EXEC statement, which limits CPU time, yet you reference a desire to cancel the job step if it waits more than some amount of time in the input queue, which is a clock time limitation.
There is no way to cancel the job from the job itself via JCL parameters based on clock time, either including or excluding time spent in the input queue.
If you really need to do this, I suggest you look into capabilities of your shop's job scheduler package. You might want to reexamine why you need to cancel a job if it doesn't run to completion within 10 clock seconds after you submit it.
I have a background task that needs to be run repeatedly, every hour or so, sending me an email whenever the task emitted non-trivial output.
I'm currently using cron for that, but it's somewhat ill-suited: it forces me to choose exact times at which the command is run, and it doesn't prevent overlap.
An alternative would be to run the script in a loop with sleep 3600 at the end of each iteration but this then needs extra work to make sure the script is always restarted after boot and such.
Ideally, I'd like a cron-like tool where I can give a set of commands to run repeatedly with approximate execution rates and the tool will run them "when convenient" and without overlapping execution of different iterations of a command (or even without overlapping execution of any command).
Short of writing such a tool myself, what would be the recommended approach?
hey getting used to groovy and i wanted to have a loop such as a do while loop in my groovy script which is ran every hour or 2 for until a certain condition inside the loop is met (variable = something). So I found the sleep step but was wondering if it would be ok to sleep for such a long time. The sleep function will not mess up right?
The sleep function will not mess up. But that isn't your biggest problem.
If all your script is doing is sleeping, it would be better to have a scheduler like Cron launch your script. This way is simpler and more resilient, it reduces the opportunities for the script to be accumulating garbage, leaking memory, having its JVM get killed by another process, or otherwise just falling into a bad state from programming errors. Cron is solid and there is less that can go wrong that way. Starting up a JVM is not speedy but if your timeframe is in hours it shouldn't be a problem.
Another possible issue is that the time your script wakes up may drift. The OS scheduler is not obliged to wake your thread up at exactly the elapsed time. Also the time on the server could be changed while the script is running. Using Cron would make the time your script acts more predictable.
On the other hand, with the scheduler, if a process takes longer than the time to the next run, there is the chance that multiple instances of the process can exist concurrently. You might want to have the script create a lock file and remove it once it's done, checking to see if the file exists already to let it know if another instance is still running.
First of all there's not do {} while() construct in groovy. Secondly it's a better idea to use a scheduler e.g. QuartzScheduler to run a cron task.
I have a modern webapp running under Tomcat, which often needs to call some legacy perl code to get some results. Right now, we wrap these in a call to Runtime.getRuntime().exec() which is working fine.
However, as the webapp gets busier we are noticing that often the perl is timing out and we need to control this.
I am using commons-pool to ensure that only X number of copies can be run at a time, and threads will queue up nicely for a perl instance when they need one, timing out after Y seconds and returning an error (this is fine, the client will just retry).
However we still have the problem that Perl takes a long time to start up, interpret the script, execute and return. At busy times we are doing this 30-50 times per second. It's a beefy machine but it's starting to struggle.
I have read up on Speedy and PersistentPerl and am considering holding open a copy of this in memory for each object in my pool, so that we do not need to open and close the Perl each time.
Is this a good idea? Any tips for how to go about doing this?
Those approaches should reduce the overhead from the start up time of your script. If the script is something that can be run as a CGI program then you might be better offer making it work with Plack and running it with a PSGI server. Your Tomcat application could collect and send the request parameters to your script and/or "web application" running in the background.
I coded a monitoring program in RPG that checks if the fax/400 is operational.
And now I want this program to check every 15 minutes.
Instead of placing a job every 15 minutes in the job scheduler (which would be ugly to manage), I made the program wait between checks using DLYJOB.
Now how can I make this program "place itself" in memory so it keeps running?
(I thought of using SBMJOB, but I can't figure in which job queue I could place it.)
A good job queue to use for an endlessly running job would be QSYSNOMAX. That allows unlimited numbers of jobs to be running.
You could submit the job to that queue in your QSTRTUP program and it will simply remain running all the time.
Here what I have done in the past. There are two approaches to this.
Submit a new job every time the program runs with DLYJOB before it runs.
Create a loop and only end given a certain condition.
What I did with a Monitor MSGW program was the following:
PGM
DCL VAR(&TIME) TYPE(*CHAR) LEN(6)
DCL VAR(&STOPTIME) TYPE(*CHAR) LEN(6) +
VALUE('200000')
/* Setup my program (run only once) */
START:
/* Perform my actions */
RTVSYSVAL SYSVAL(QTIME) RTNVAR(&TIME)
IF COND(&TIME *GE &STOPTIME) THEN(GOTO CMDLBL(END))
DLYJOB DLY(180)
GOTO CMDLBL(START)
END:
ENDPGM
This will run continuously until 8:00 pm. Then I add this to the job scheduler to submit every morning.
As far as which jobq. I am using QINTER, but it could really be run anywhere. Make sure you choose a subsystem with enough available running jobs as this will take one.
The negative of running in QINTER if the program starts to hit 100% CPU, that will use up all of your interactive CPU and effectively locks up your system.
i know of 3 ways to that.
1) using Data queue, there is parm to tell it to wait endlessly and at time-interval.
2) using OVRDBF cmd, there is parm there to tell that it should not end or EOF, making your pgm to keep on waiting.
3) easiest to implement, sbmjob to call a pgm that loops forever eg with DOW 1=1, you can insert a code to check for certain time interval before it iterates. You can have your logic inside the loop that checks for fax, process it and then back to waiting.