I'm working on a program that would simulate scheduling from creation to completion of processes. I need assistance to know can a process move back from ready queue to the job queue (in any case - may be an exception).
I'm not sure what you mean by "queue job". A process is either :
running (in that case no need to do anything)
sleeping, that means that the process is waiting for an input or an output. You can't force it to “wake up”. It'll wake up when the input or output operation it wants to make is possible.
stopped, that means that the process is currently suspended. There four different kind for it.
SIGTSTP, (most of the time triggered by CTRL + Z. Can be unstopped with fg command
SIGSTOP, meaning it has been roughly stopped. Can't do many things about that one.
SIGTTIN and SIGTTOU, but I don't have the knowledge for those two.
So you can dive in fg command that might helps you.
NB : sorry for bad english.
Related
I am dealing with an odd problem which I couldn't find the answer to online, nor through a lot of trial and error.
In a multi-multi process cluster, forked worker processes can run arbitrarily long commands, but the parent process listens for keepalive messages sent by workers, and kills workers that are stuck for longer than X seconds.
Worker processes can asynchronously communicate with the rest of the world (using http, or process.send ipc communication), but on exit, I'd like to be able to communicate some things (typically, queued logs or error details).
Most online documentation for process.on('exit', handler) indicates usage of console.log, however it seems like forked processes don't inherit a normal stdout, and the console.log isn't a direct tty, it's a stream (the ipc stream, I presume?).
Because of this, the process exit handler doesn't let me use console.log to log extra lines (or if it does, I'm not sure where these lines end up)
I tried various combinations of fork options (silent/not silent, non-default stdio options like inherit), using fs.write to write to tty or a real file, using process.send, or but in no case, was I able to get the on-exit handler to log anywhere visible.
How can I get the forked process to successfully log on exit?
small additional points - all this testing is on unix-like systems (macos , amazon linux...) and both parent and child processes are fired with --sigint-trace so that we can get at least the top 10 stack frames of the interrupted process on exit. These frames do make it out to the terminal successfully
This was a bit of a misunderstanding about how SIGINT is handled, and I believe that it's impossible to accomplish what I want here, but I'd love to hear if someone else found a solution.
Node has its own SIGINT handler which is "more powerful" than custom SIGINT handlers - typically it interrupts infinite loops, which is extremely useful in the case where code is blocked by long-running operations.
Node allows one-upping its own SIGINT debugging capabilities by attaching a --trace-sigint flag which captures the last frames of execution.
If I understood this correctly, there are 4 cases with different behavior
No custom handler, event loop blocked
process is terminated without any further code execution. (and --trace-sigint can give a few stack traces)
No custom handler, event loop not blocked
normal exit flow, process.on('exit') event fires.
Custom handler, event loop blocked
nothing happens until event loop unblocks (if it does), then normal exit flow
Custom handler, event loop not blocked
normal exit flow.
This happens regardless of the way the process is started, and it's not a problem about pipes or exit events - in the case where the event loop is blocked and the native signal handler is in place, the process terminates without any further execution.
It would seem like there is no way to both get a forced process exit during a blocked event loop, AND still get node code to run on the same process after the native interruption to recover more information.
Given this, I believe the best way to recover information from the stuck process is to stream data out of it before it freezes (sounds obvious, but brings a lot of extra considerations in production environments).
I am working on a big project that puts performance as a high priority. I have a little bit of experience using wxPython to create windows and dialog boxes for software, but I have no experience in getting processes to work in parallel during the course of a single program.
So basically, what I want to accomplish is the following:
I want one main class that controls the high level program. It sets up a configuration either from a config file or from user input. This much I have accomplished on my own.
I need PROCESS #1 to read in a file and a list of commands, execute the commands, and then pass the modified file to PROCESS #2 (this requires that PROCESS #2 is ready to accept new input.) Once the file is passed, PROCESS #1 would begin work on the next set of inputs and wait for PROCESS #2 to finish before the cycle repeats.
PROCESS #2 takes input from PROCESS #1 and writes output to a log file. Once the output is complete, it waits for the next set of output from PROCESS #1.
I know how to use wxTimers and the events associated with that, but what I have found is that a timer event will not execute if the program is otherwise occupied (like in the middle of a method.)
I have seen threads about "threading" and "Pool", but the terminology tends to go over my head, and I haven't gotten any of that sort of stuff to work.
If anybody can point me in the right direction, I would be greatly appreciative.
If you use threads, then I think this would be fairly easy to do. Here's what I would suggest:
Create a button (or some other widget) to execute process #1 in a thread. The thread itself will run BOTH processes. Here's some psuedo-code that might help:
# this is in your thread code:
result = self.call_process_1(args)
self.call_process_2(result)
This will allow you to start another process #1/2 with a new set of commands every time you press the button. Since the two processes are encapsulated in the thread, they don't have to wait for process #2 to finish. You will probably need to log to separate logs for the logs to make sense, but you can label the logs with a timestamp and a thread number or a uuid.
Depending on how many of these processes you need to do, you might need to look into setting up a cluster that's driven with celery or some such. But I think this is a good starting place.
I want to run a small clean up process every few hours on an Erlang server.
I know of the timer module. I saw an example in a tutorial used chained timer:sleep commands to wait for an event that would occur multiple days later, which I found strange. I understand that Erlang process are unique compared to those in other languages, but the idea of a process/thread sleeping for days, weeks, and even months at a time seemed odd.
So I set out to find out the details of what sleeping actually does. The closest I found was a blog post mentioning that sleep is implemented with a receive timeout, but that still left the question:
What do these sleep/sleep-like functions actually do?
Is my process taking up resources as it sleeps? Would having thousands of sleeping process use as many resources, as say, thousands of process servicing a recursive call that did nothing? Is there any performance penalty from repeatedly sleeping within processes, or sleeping for long periods of time? Is the VM constantly expending resources to see if the conditions to end the processes' sleep are up?
And as a side note, I'd appreciate if someone could comment on if there is a better way than sleeping to pause for hours or days at a time?
That is the Karma of any erlang process: it waits or dies :o)
when a process is spawned, it start executing until the last execution line, and die, returning the last evaluation.
To keep a process alive, there is no other solution to recursively loop in a never ending succession of calls.
of course there are several conditions that make it stop or sleep:
end of the loop: the process received a message which tell him to
stop recursion
a receive bloc: the process will wait until a message
matching one entry in the receive bloc is posted in the message
queue.
The VM scheduler stop it temporarily to let access to the CPU
to other processes
in the 2 last cases the execution will restart under the responsibility of the VM scheduler.
while waiting it uses no CPU bandwidth, but keeps the exact same memory layout it had when it started waiting. The Erlang OTP offers some means to reduce this memory layout to the minimum using the hibernate option (see the documentation of gen_serevr or gen_fsm, but it is for advanced usage only in my mind).
a simple way to create a "signal" that will fire a process at regular (or almost regular) interval is effectively to use receive block with timout (The timeout is limited to 65535 ms), for example:
on_tick_sec(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,1000,Period,0).
on_tick_mn(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,60000,Period,0).
on_tick_hr(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,60000,Period*60,0).
on_tick(Module,Function,Arglist,TimeBase,Period,Period) ->
apply(Module,Function,Arglist),
on_tick(Module,Function,Arglist,TimeBase,Period,0);
on_tick(Module,Function,Arglist,TimeBase,Period,CountTimeBase) ->
receive
stop -> stopped
after TimeBase ->
on_tick(Module,Function,Arglist,TimeBase,Period,CountTimeBase+1)
end.
and usage:
1> Pid = spawn(util,on_tick_sec,[io,format,["hello~n"],5]).
<0.40.0>
hello
hello
hello
hello
2> Pid ! stop.
stop
3>
[edit]
The timer module is a standard gen_server running in a separate process. All the function in the timer module are public interfaces that execute a hidden gen_server:call or gen_server:cast to the timer server. This is a common usage to hide the internal of a server and allow further evolutions without impact on existing applications.
The server uses internally a table (ets) to store all the actions it has to do along with each timer reference and it uses its own function to be awaken when needed (at the end, the VM must take care of this ?).
So you can hibernate a process without any effect on the timer server behavior. The hibernation mechanism is
tricky, see documentation at hibernate/3 definition, you will see that yo have to "rebuild" the context by yourself since everything was removed from the process context, and a tuple(Module,Function,Arguments} is stored by the system to restart your process when needed.
cost some time in garbage collecting and process restart
It is why I said that it is really an advance feature that need good reason to be used.
There is also erlang:hibernate/3 that puts a process in "deep sleep", minimizing memory usage for it.
I coded a monitoring program in RPG that checks if the fax/400 is operational.
And now I want this program to check every 15 minutes.
Instead of placing a job every 15 minutes in the job scheduler (which would be ugly to manage), I made the program wait between checks using DLYJOB.
Now how can I make this program "place itself" in memory so it keeps running?
(I thought of using SBMJOB, but I can't figure in which job queue I could place it.)
A good job queue to use for an endlessly running job would be QSYSNOMAX. That allows unlimited numbers of jobs to be running.
You could submit the job to that queue in your QSTRTUP program and it will simply remain running all the time.
Here what I have done in the past. There are two approaches to this.
Submit a new job every time the program runs with DLYJOB before it runs.
Create a loop and only end given a certain condition.
What I did with a Monitor MSGW program was the following:
PGM
DCL VAR(&TIME) TYPE(*CHAR) LEN(6)
DCL VAR(&STOPTIME) TYPE(*CHAR) LEN(6) +
VALUE('200000')
/* Setup my program (run only once) */
START:
/* Perform my actions */
RTVSYSVAL SYSVAL(QTIME) RTNVAR(&TIME)
IF COND(&TIME *GE &STOPTIME) THEN(GOTO CMDLBL(END))
DLYJOB DLY(180)
GOTO CMDLBL(START)
END:
ENDPGM
This will run continuously until 8:00 pm. Then I add this to the job scheduler to submit every morning.
As far as which jobq. I am using QINTER, but it could really be run anywhere. Make sure you choose a subsystem with enough available running jobs as this will take one.
The negative of running in QINTER if the program starts to hit 100% CPU, that will use up all of your interactive CPU and effectively locks up your system.
i know of 3 ways to that.
1) using Data queue, there is parm to tell it to wait endlessly and at time-interval.
2) using OVRDBF cmd, there is parm there to tell that it should not end or EOF, making your pgm to keep on waiting.
3) easiest to implement, sbmjob to call a pgm that loops forever eg with DOW 1=1, you can insert a code to check for certain time interval before it iterates. You can have your logic inside the loop that checks for fax, process it and then back to waiting.
I can't find a good way to find out when a process exits in Linux. Does anyone have a solution for that?
One that I can think of is check process list periodically, but that is not instant and pretty expensive (have to loop over all processes each time).
Is there an interface for doing that on Linux? Something like waitpid, except something that can be used from unrelated processes?
Thanks,
Boda Cydo
You cannot wait for an unrelated process, just children.
A simpler polling method than checking the process list, if you have permission, you can use the kill(2) system call and "send" signal 0.
From the kill(2) man page:
If sig is 0, then no signal is sent,
but error checking is still performed;
this can be used to check for the
existence of a process ID or process
group ID
Perhaps you can start the program together with another program, the second one doing whatever it is you want to do when the first program stops, like sending a notification etc.
Consider this very simple example:
sleep 10; echo "finished"
sleep 10 is the first process, echo "finished" the second one (Though echo is usually a shell plugin, but I hope you get the point)
Another option is to have the process open an IPC object such as a unix domain socket; your watchdog process can detect when the process quits as it will immediately be closed.
If you know the PID of the process in question, you can check if /proc/$PID exists. That's a relatively cheap stat() call.