I have a trigger on one of the DB2 tables. What i need is, every time that trigger runs, it should invoke a Linux shell script.
How to do this-- same as any other process? If my trigger was to invoke a Java process instead of a shell script, putting the bytecode (.class file) of that process to ..SQLLIB/function and defining a function for it would do the job.
Is this much different for Linux script, any subtleties?
I don't have Linux for another few days but deployment is right around the corner and nervous at it just the same.
TIA.
You cannot invoke a shell script from SQL, including from a trigger. What you can do is create a Java UDF the way you described, then use Runtime.exec() to invoke the script.
Note that your approach, apart from introducing security risks, can affect data consistency (a transaction can be rolled back, but you cannot "unrun" the script) and concurrency (a transaction with all its acquired locks will have to wait until your script returns).
A better approach would be to use an asynchronous process to decouple the external action from the database transaction. As an example, your trigger might insert a record into a log table, and the external process would read that log table on schedule, perform the required action when a new record is found, then delete the processed record.
There is an good article about
Making Operating System Calls from SQL
which includes sample code.
Related
I need to set a Timeout, in a JCL step that calls a Unix script through bpxbtach. I did it with
//STEPX EXEC PGM=BPXBATCH, PARM='sh /x.sh',TIME=(,10)
However, After some time I realized that does not include the time in the queue. they say " This run time refers to actual execution time only, and does not include the time that the job spends in the INPUT or INPUT HOLD queues" https://supportline.microfocus.com/documentation/books/rd60/cbwjto.htm
That is microfocus JCL, but I verified the behavior is that on IBM Z too.
So even if I set the timeout to 10 seconds, the step can take several minutes if the queue is attending other things. I need a timeout that kills the step no matter the reason it took so long. I haven't been able to find what I need. Please help.
z/OS batch really isn't the best choice for time-critical work. As you figured out, the JCL "TIME" parameter is about CPU time consumption, not an elapsed time control. If this is a business-critical need, then by all means talk to your z/OS administrators - they can certainly configure your system such that your job is very likely to run without delay, but this isn't usually default behavior.
You don't provide a lot of detail as to what else your job might be doing and how it gets submitted. If you have the ability to control how your job is submitted, one option might be to spawn your shell script directly rather than submitting a batch process to run your script.
For example, what you've described is submitting JCL that spawns BPXBATCH, then BPXBATCH spawns your shell script. Instead, you might write a small C program that simply calls "spawn()" to run the shell as a distinct UNIX process - that's not difficult, depending on how you're submitting the JCL you shared. You cut out the need for the batch job - just run your script directly.
If you're running in a TSO environment, the OSHELL command lets you interactively run your script. You can even automate the whole process with a simple REXX script, and none of this requires a pass through a batch initiator.
If your site runs SSH or similar, you might consider launching your script through an SSH command - this even works across a network. SSH lets you launch a shell session and pass a command for execution...again, there's no JCL or input queue here.
If your administrators would allow it, another alternative would be to run your JCL via a "START" command. Unlike batch JCL, when a START command is encountered, the work you're starting runs immediately - there's no input queue for started tasks. Start commands can be issued from JCL too, and since they're issued as the JCL is scanned and not when the job starts, these are fairly immediate too.
Inside your shell script, it's pretty easy to setup an elapsed time limit - there are examples here.
I see a couple of problems in your code...
//STEPX EXEC PGM=BPXBATCH, PARM='sh /x.sh',TIME=(,10)
First, you have a space between BPXBATCH, and PARM= which will not execute your shell script and may result in a JCL error.
Second, you are using the TIME parameter of the EXEC statement, which limits CPU time, yet you reference a desire to cancel the job step if it waits more than some amount of time in the input queue, which is a clock time limitation.
There is no way to cancel the job from the job itself via JCL parameters based on clock time, either including or excluding time spent in the input queue.
If you really need to do this, I suggest you look into capabilities of your shop's job scheduler package. You might want to reexamine why you need to cancel a job if it doesn't run to completion within 10 clock seconds after you submit it.
I'm going to start writing my first mass update script and am currently using a Sandbox account. I am wondering what is the best way to test the script considering it will make changes on a lot of items? Is it possible to test the script on a small sample?
When you execute a Mass Update, you have to build a search in the UI first, then you manually select which search results the script should process. Thus, the best way to test your script is to only select one result at a time, or just a few results. Perhaps build a few test records that match your search criteria and process those.
At any rate, with Mass Updates, you are in full control over which results are processed each time you execute the script.
What I often do ( this dead simple in SS 1) is to create a companion suitelet interface that just calls the mass update function
That way you pass a record id, via the suitelet params, to the mass update function and it runs immediately. In another tab you can make changes etc and then just refresh the suitelet when you want to run it again.
I find this more convenient when developing than going through the mass update interface for each development iteration.
I always test it by simply feeding in an internal ID to the script, in the debugger. That is pretty much how things will run with a mass update.
I want to replace MULTI/EXEC operation in Redis, with Lua script call, and according to documentation of scripting in Redis:
Redis uses the same Lua interpreter to run all the commands. Also
Redis guarantees that a script is executed in an atomic way: no other
script or Redis command will be executed while a script is being
executed. This semantics is very similar to the one of MULTI / EXEC.
From the point of view of all the other clients the effects of a
script are either still not visible or already completed.
As I understand I can drop MULTI/EXEC in this case and simplify my procedure, or is it more complicated, then this, and I still need to use it just in case? (For example for cluster environment)
In general, with a Lua script, you can drop MULTI/EXEC, provided the MULTI/EXEC block was only used to enforce isolation (isolation as in ACID).
A cluster environment does not change anything, since MULTI/EXEC blocks only work at the instance level.
I have a PHP scraper running every night on a very large site. Crontab launches the script at 2am and pkill it at 7am. Now I am concerned that brutally killing the script might result in data loss. Let's say that crontab calls the script off while the script is busy writing my scraped data into the database, then the next day the database will refuse that last/first record because it is already present (even if not completely).
Is there any way I can freeze the script with crontab? (That is, without adding a sleep() to my script)
Let's say that crontab calls the script off while the script is busy writing my scraped data into the database
That would be a problem, since you will run into some transaction timeout or something, if you stop your process externally. A better way would be to let the script halt/pause on its own. You could for example define some marker file that is checked by the script periodically so that the script can halt/pause in a controlled way.
Having one large cronjob that can't be interrupted is usually a sign of bad design for a number of reasons.
Most notably, you can't interrupt the run for no reason whatsoever or you'll end up with corrupted data. This can become a big problem in case you have an unexpected power loss or server crash.
Also, it doesn't scale. If you need to process more data, you can't scale it to multiple servers. If you have run times of a few hours now, you may end up exhausting a complete server very soon.
I would recommend to seriously rethink the functionality of this cronjob and restructure it so you have a number of smaller tasks that are queued up somewhere. (It can even be the database.) You could then mask the SIGINT and SIGTERM signals when processing a single task and check the received signals in between tasks. This will allow you to notify the process using either of the aforementioned and have it shut down gracefully.
That being said, things do break and servers do crash. I also urge you to work out plans for data recovery in case the cronjob breaks down while working on something.
I am new to VC++ and programming. I have a task in which I am supposed to design a file watcher in VC++.
The problem goes this way:
I have to monitor some log files continously; whenever a particular log file gets deleted(this deletion is done by some other program), I have to open a TextFile and write some data and the timestamp into it.
How do I go about it? Please help!!
First, you need to setup a system to monitor for file events from that folder.
To get started, take a look at FindFirstChangeNotification().
You'll basically get a waitable handle from that.
Then, were it me, I'd have a thread that waited on that event. Each time the event triggers, the thread resumes, queries for the change details (what file), then perform the needed actions, and resume sleeping on that handle again.
You'll need some additional semaphore or something to use to interrupt this worker-thread and wake it so that you can tell it to quit. Simple to do: have your thread's main loop do a WaitForMultipleObjects - the "wake up semaphore" and the FindFirstChangeNotification handle. When you wake up, check which even notified you, then either process the file change or quit.
MFC has a slightly different way of handling it (slightly) but to do this using the Win32 API what you'd typcially do is use the Directory Management Functions to set up a change notification handle for the directory the file goes in. Then you can wait on the handle and when something happens inside that directory your wait completes, and you can check to see if it was a change to the file that you care about.
Look at the docs for FindFirstChangeNotification and ReadDirectoryChangesW for more information.
Try the Windows Management Instrumentation (WMI) if you have enough privileges. AFAIK it is also the most efficient way to handle the filesystem events.
Handle or query the __InstanceDeletionEvent, __InstanceModificationEvent or __InstanceCreationEvent for the deletion, modification or creation events respectively and filter the files and target path that you want.
Take a look at the WMI Reference/C++ invocation.
For a full-scale example take a look at codeproject querying example.
I strongly recommmend you consider using the implementation here. This API is not 100% reliable, but this code does a good job of wrapping it. If your filesystem traffic is local and not too frequent, it should work well for you.