A process in Erlang will either call link/1 or spawn_link to create a link with another process. In a recent application i am working on i got curious on whether its possible for a process to know at a given instance, the number of other processes its linked to. is this possible ? is their a BIF ?
Then, also, when a linked process dies, i guess that if it were possible to know the number of linked processes, this number would be decremented automatically by the run-time system. Such a mechanism would be ideal in dealing with Parent-Child relationships in Erlang concurrent programs, even in simple ones which do not involve supervisors.
Well, is it possible for an Erlang process to know out-of-the-box perhaps via a BIF, the number of processes linked to it, such that whenever a linked process dies, this value is decremented automatically under-the-hood :)?
To expand on this question a little bit, consider a gen_server, which will handle thousands of messages via handle_info. In this part, its job is to dispatch child processes to handle the task as soon as it comes in. The aim of this is to make sure the server loop returns immediately to take up the next request. Now, the child process handles the task asynchronously and sends the reply back to the requestor before it dies. Please refer to this question and its answer before you continue. Now, what if, for every child process spawned off by the gen_server, a link is created, and i would like to use this link as a counter. I know, i know, everyone is going to be like " why not use the gen_server State, to carry say, a counter, and then increment or decrement it accordingly ? " :) Somewhere in the gen_server, i have:
handle_info({Sender,Task},State)->
spawn_link(?MODULE,child,[Sender,Task]),
%% At this point, the number of links to the gen_server is incremented
%% by the run-time system
{noreply,State};
handle_info( _ ,State) -> {noreply,State}.
The child goes on to do this:
child(Sender,Task)->
Result = (catch execute_task(Task)),
Sender ! Result,
ok. %% At this point the child process exits,
%% and i expect the link value to be decremented
Then finally, the gen_server has an exposed call like this:
get_no_of_links()-> gen_server:call(?MODULE,links).
handle_call(links, _ ,State)->
%% BIF to get number of instantaneous links expected here
Links = erlang:get_links(), %% This is fake, do not do it at home :)
{reply,Links,State};
handle_call(_ , _ ,State)-> {reply,ok,State}.
Now, some one may ask them selves, really, Why would anyone want to do this ?
Usually, its possible to create an integer in the gen_server State and then we do it ourselves, or at least make the gen_server handle_info of type {'EXIT',ChildPid,_Reason} and then the server would act accordingly. My thinking is that if it were possible to know the number of links, i would use this to know ( at a given moment in time), how many child processes are still busy working, this in turn may actually assist in anticipating server load.
From manual for process_info:
{links, Pids}:
Pids is a list of pids, with processes to which the process
has a link
3> process_info(self(), links).
{links,[<0.26.0>]}
4> spawn_link(fun() -> timer:sleep(100000) end).
<0.38.0>
5> process_info(self(), links).
{links,[<0.26.0>,<0.38.0>]}
I guess it could be used to count number of linked processes
Your process should run process_flag(trap_exit, true) and listen for messages of the form {'EXIT', Pid, Reason} which will arrive whenever a linked process exits. If you don't trap exits, the default behaviour will be for your linked process to exit when the other side of the link exits.
As for listening to when processes add links, you can use case process_info(self(), links) of {links, L} -> length(L) end or length(element(2, process_info(self(), links)), but you have to re-run this regularly as there is no way for your process to be notified whenever a link is added.
A process following OTP guidelines never needs to know how many processes are linked to it.
Related
I am working on a big project that puts performance as a high priority. I have a little bit of experience using wxPython to create windows and dialog boxes for software, but I have no experience in getting processes to work in parallel during the course of a single program.
So basically, what I want to accomplish is the following:
I want one main class that controls the high level program. It sets up a configuration either from a config file or from user input. This much I have accomplished on my own.
I need PROCESS #1 to read in a file and a list of commands, execute the commands, and then pass the modified file to PROCESS #2 (this requires that PROCESS #2 is ready to accept new input.) Once the file is passed, PROCESS #1 would begin work on the next set of inputs and wait for PROCESS #2 to finish before the cycle repeats.
PROCESS #2 takes input from PROCESS #1 and writes output to a log file. Once the output is complete, it waits for the next set of output from PROCESS #1.
I know how to use wxTimers and the events associated with that, but what I have found is that a timer event will not execute if the program is otherwise occupied (like in the middle of a method.)
I have seen threads about "threading" and "Pool", but the terminology tends to go over my head, and I haven't gotten any of that sort of stuff to work.
If anybody can point me in the right direction, I would be greatly appreciative.
If you use threads, then I think this would be fairly easy to do. Here's what I would suggest:
Create a button (or some other widget) to execute process #1 in a thread. The thread itself will run BOTH processes. Here's some psuedo-code that might help:
# this is in your thread code:
result = self.call_process_1(args)
self.call_process_2(result)
This will allow you to start another process #1/2 with a new set of commands every time you press the button. Since the two processes are encapsulated in the thread, they don't have to wait for process #2 to finish. You will probably need to log to separate logs for the logs to make sense, but you can label the logs with a timestamp and a thread number or a uuid.
Depending on how many of these processes you need to do, you might need to look into setting up a cluster that's driven with celery or some such. But I think this is a good starting place.
I want to run a small clean up process every few hours on an Erlang server.
I know of the timer module. I saw an example in a tutorial used chained timer:sleep commands to wait for an event that would occur multiple days later, which I found strange. I understand that Erlang process are unique compared to those in other languages, but the idea of a process/thread sleeping for days, weeks, and even months at a time seemed odd.
So I set out to find out the details of what sleeping actually does. The closest I found was a blog post mentioning that sleep is implemented with a receive timeout, but that still left the question:
What do these sleep/sleep-like functions actually do?
Is my process taking up resources as it sleeps? Would having thousands of sleeping process use as many resources, as say, thousands of process servicing a recursive call that did nothing? Is there any performance penalty from repeatedly sleeping within processes, or sleeping for long periods of time? Is the VM constantly expending resources to see if the conditions to end the processes' sleep are up?
And as a side note, I'd appreciate if someone could comment on if there is a better way than sleeping to pause for hours or days at a time?
That is the Karma of any erlang process: it waits or dies :o)
when a process is spawned, it start executing until the last execution line, and die, returning the last evaluation.
To keep a process alive, there is no other solution to recursively loop in a never ending succession of calls.
of course there are several conditions that make it stop or sleep:
end of the loop: the process received a message which tell him to
stop recursion
a receive bloc: the process will wait until a message
matching one entry in the receive bloc is posted in the message
queue.
The VM scheduler stop it temporarily to let access to the CPU
to other processes
in the 2 last cases the execution will restart under the responsibility of the VM scheduler.
while waiting it uses no CPU bandwidth, but keeps the exact same memory layout it had when it started waiting. The Erlang OTP offers some means to reduce this memory layout to the minimum using the hibernate option (see the documentation of gen_serevr or gen_fsm, but it is for advanced usage only in my mind).
a simple way to create a "signal" that will fire a process at regular (or almost regular) interval is effectively to use receive block with timout (The timeout is limited to 65535 ms), for example:
on_tick_sec(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,1000,Period,0).
on_tick_mn(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,60000,Period,0).
on_tick_hr(Module,Function,Arglist,Period) ->
on_tick(Module,Function,Arglist,60000,Period*60,0).
on_tick(Module,Function,Arglist,TimeBase,Period,Period) ->
apply(Module,Function,Arglist),
on_tick(Module,Function,Arglist,TimeBase,Period,0);
on_tick(Module,Function,Arglist,TimeBase,Period,CountTimeBase) ->
receive
stop -> stopped
after TimeBase ->
on_tick(Module,Function,Arglist,TimeBase,Period,CountTimeBase+1)
end.
and usage:
1> Pid = spawn(util,on_tick_sec,[io,format,["hello~n"],5]).
<0.40.0>
hello
hello
hello
hello
2> Pid ! stop.
stop
3>
[edit]
The timer module is a standard gen_server running in a separate process. All the function in the timer module are public interfaces that execute a hidden gen_server:call or gen_server:cast to the timer server. This is a common usage to hide the internal of a server and allow further evolutions without impact on existing applications.
The server uses internally a table (ets) to store all the actions it has to do along with each timer reference and it uses its own function to be awaken when needed (at the end, the VM must take care of this ?).
So you can hibernate a process without any effect on the timer server behavior. The hibernation mechanism is
tricky, see documentation at hibernate/3 definition, you will see that yo have to "rebuild" the context by yourself since everything was removed from the process context, and a tuple(Module,Function,Arguments} is stored by the system to restart your process when needed.
cost some time in garbage collecting and process restart
It is why I said that it is really an advance feature that need good reason to be used.
There is also erlang:hibernate/3 that puts a process in "deep sleep", minimizing memory usage for it.
I'm implementing a simple job scheduler, which spans a new process for every job to run. When a job exits, I'd like it to report the number of actions executed to the scheduler.
The simplest way I could find, is to exit with the number of actions as a return code. The process would for example exit with return code 3 for "3 actions executed".
But the standard (AFAIK) being to use the return code 0 when a process exited successfully, and any other value when there was en error, would this approach risk to create any problem?
Note: the child process is not an executable script, but a fork of the parent, so not accessible from the outside world.
What you are looking for is inter process communication - and there are plenty ways to do it:
Sockets
Shared memory
Pipes
Exclusive file descriptors (to some extend, rather go for something else if you can)
...
Return convention changes are not something a regular programmer should dare to violate.
The only risk is confusing a calling script. What you describe makes sense, since what you want really is the count. As Joe said, use negative values for failures, and you should consider including a --help option that explains the return values ... so you can figure out what this code is doing when you try to use it next month.
I would use logs for it: log the number of actions executed to the scheduler. This way you can also log datetimes and other extra info.
I would not change the return convention...
If the scheduler spans a child and you are writing that you could also open a pipe per child, or a named pipes or maybe unix domain sockets, and use that for inter process communication and writing the processed jobs there.
I would stick with conventions, namely returning 0 for success, expecially if your program is visible/usable around by other people, or anyway document well those decisions.
Anyway apart from conventions there are also standards.
I'm currently learning Erlang at a reasonable clip but have a question about gen_server with supervisors. If a gen_server process crashes and is consequentially restarted by a supervisor, it receives a new pid. Now, what if I want other processes to refer to that process by Pid? What are some good idiomatic ways to 'update' the Pid in those processes?
As an exercise with some practical application, I'm writing a lock server where a client can request a lock with an arbitrary key. I ideally would like to have a separate processes handle the locking and releasing of a particular lock, the idea being that I can use the timeout argument in gen_server to terminate the process if no one has requested it after N amount time, so that only currently relevant locks will stay in memory. Now, I have a directory process which maps the lock name to the lock process. When the lock process terminates, it deletes the lock from the directory.
My concern is how to handle the case where a client requests a lock while the lock process is in the middle of terminating. It hasn't shutdown yet, so sniffing that the pid is alive won't work. The lock process hasn't reached the clause that deletes it from the directory yet.
Is there a better way to handle this?
EDIT
There are two gen_servers currently: the 'directory' which maintains an ETS table from LockName -> Lock Process, and the 'lock servers' which are added dynamically to the supervision tree using start_child. Ideally I would like each lock server to handle talking with the clients directly, but am worried about the scenario of a request to acquire/release getting issued with call or cast when the process is in the middle of crashing (and thus won't respond to the message).
Starting with {local} or {global} won't work since there can be N amount of them.
The trick is to name the process and don't refer to it by its pid. You generally have 3 viable options,
Use registered names. This is what andreypopp suggests. You refer to the server by its registered name. locally registered names have to be atoms, which may somewhat limit you. globally registered names do not have this limitation, you can register any term.
The Supervisor knows the Pid. Ask it. You will have to pass the Supervisor Pid to the process.
Alternatively, use the gproc application (exists on http://github.com). It allows you to create a generic process registry - you could have done that by ETS, but steal good code rather than implement yourself.
The pid is usable if all processes are part of the same supervision tree. So the death of one of them means the death of the others. Thus, the Pids recycling doesn't matter.
Don't refer to gen_server process by pid.
You should provide API for your gen_server via gen_server:call/2 or gen_server:call/3 functions. They are accept ServerRef as first argument, which can be Name | {Name,Node} | {global,GlobalName} | pid(). So, you API would look like:
lock(Key) ->
gen_server:call(?MODULE, {lock, Key}).
release(Key) ->
gen_server:call(?MODULE, {release, Key}).
Note that this API is defined in the same module as your gen_server and I assume you starting you server with something like:
gen_server:start_link({local, ?MODULE}, ?MODULE, [], [])
So your API methods can lookup server not by pid, but by server name, which is equal to ?MODULE.
For more information, please see gen_server documentation.
You can completely avoid the use of your "lock_server" process by using the "erlang:monitor/demonitor" API.
When a client requests a lock, you issue the lock.. and do a erlang:monitor on the client.. This will return you a Monitor Reference.. You can then store this Reference along with your lock.. The beauty of this is that your directory server WILL be notified when the client dies.. you could implement the TIMEOUT thing in the client.
Here is a snippet from code I had written recently..
https://github.com/xslogic/phoebus/blob/master/src/table_manager.erl
Basically, the table_manager is a process that issues lock on a particular table resource to client.. if the client dies, the table is returned to the pool..
I can't find a good way to find out when a process exits in Linux. Does anyone have a solution for that?
One that I can think of is check process list periodically, but that is not instant and pretty expensive (have to loop over all processes each time).
Is there an interface for doing that on Linux? Something like waitpid, except something that can be used from unrelated processes?
Thanks,
Boda Cydo
You cannot wait for an unrelated process, just children.
A simpler polling method than checking the process list, if you have permission, you can use the kill(2) system call and "send" signal 0.
From the kill(2) man page:
If sig is 0, then no signal is sent,
but error checking is still performed;
this can be used to check for the
existence of a process ID or process
group ID
Perhaps you can start the program together with another program, the second one doing whatever it is you want to do when the first program stops, like sending a notification etc.
Consider this very simple example:
sleep 10; echo "finished"
sleep 10 is the first process, echo "finished" the second one (Though echo is usually a shell plugin, but I hope you get the point)
Another option is to have the process open an IPC object such as a unix domain socket; your watchdog process can detect when the process quits as it will immediately be closed.
If you know the PID of the process in question, you can check if /proc/$PID exists. That's a relatively cheap stat() call.