I have this code and trying to understand how many process and threads will be created from this:
pid t pid;
pid = fork();
if (pid == 0) { /* child process */
fork();
thread create( . . .);
}
fork();
I think it creates 2 threads, from the fork inside the if loop. and 8 processes? But Im not sure if thats right
Actually, there should be 8 threads and 6 processes.
Here's the diagrams to make it clear:
1) after first fork():
|------------------- child of p0 [p1]
---|------------------- parent [p0]
2) after second fork():
|--------------- child of p1 [p2]
|---|--------------- [p1]
---|------------------- [p0]
3) after pthread_create():
----------- thread 1 of p2 [p2t1]
|---/----------- thread 0 of p2 [p2t0]
| ----------- thread 1 of p1 [p1t1]
|---|---/----------- thread 0 of p1 [p1t0]
---|------------------- [p0]
4) after third fork():
|------------ child of p2 [p5]
| ------ [p2t1]
|-|-----/------ [p2t0]
| |---------- child of p1 [p4]
| | ------ [p1t1]
|---|---|---/------ [p1t0]
| |------------ child of p0 [p3]
---|-----|------------ [p0]
important: Remember that the fork(2) call clones just the thread which executed it, thus process 4 [p4] has only one thread (same apply to process 5[p5]).
One extra process will get created each time fork is called.
On first call to fork, parent process P creates sub-process SP1.
After fork, parent process calls fork again (skipping the if), creating sub-process SP2.
SP1 after fork calls fork inside if, creates sub-sub-process SSP1.
SP1 then spawns a thread.
SP1 leaves the if. and calls fork again, creating sub-sub-process SSP2.
SSP1 spawns a thread.
SSP1 leaves the if, and calls fork, creating sub-sub-sub-process SSSP.
So, processes created: SP1, SP2, SSP1, SSP2, SSSP = 5 processes.
If you count the original process P, there are 6 processes.
Only SP1 and SSP1 spawn threads, so there are 2 threads created. If you count all the main threads of all the processes, there are 7 or 8 threads, depending on whether or not you count the original process P.
An illustration of the processes and threads being created correlated to the code.
P
pid t pid; |
pid = fork(); +------SP1
if (pid == 0) { | |
fork(); | +---------------SSP1
thread create(...); | |-SP1's thread |-SSP1's thread
} | | |
fork(); +-SP2 +-SSP2 +-SSSP
| | | | | |
shouldn't it be 2 threads and 6 processes?
M
| ↘
M A
| |↘
M A* B*
| | |
| ↘ | ↘ |↘
M C A D B E
as I use * to represent thread.
Total fork process is=5
Thread create is=2
Related
How many processes are created when running the following program ? I can not solve. I would appreciate if you help
int main()
{
int i;
for (i=fork(); i<2; i++ )
fork();
}
fork() creates a child process, creating an another instance of the parent process. It returns 0 to the child and PID of the child to the parent.
In this case, when i = fork() is executed,
The parent gets i assigned as the PID of the child process, which is most likely greater than 1. The for loop in the parent will run not run even once as i < 2 will fail. At this point of time there are two processes P and C1 (Child)
After the fork is executed by parent, child gets a 0 as return value, i = 0. This means the condition i < 2 is successful. Child executes the fork() in the loop body, creating C2. Value of i in both C1 and C2 are 0. i gets incremented to 1.
C1 and C2 execute i < 2, This condition is successful. Fork is executed by both. C3 is spawned off by C1 and C4 by C2.
i's value gets incremented to 2. i < 2 fails. All of them get out of the loop
To summarise, there are 4 child processes created in this program. You can try this with the following program where you will see 5 PIDs getting printed.
#include <stdio.h>
main()
{
int i = 0;
for (i = fork(); i < 2; i++)
fork();
printf("Hello World i %d\n", getpid());
}
I'm a C programmer learning about fork(), exec(), and wait() for the first time. I'm also whiteboarding a Standard C program which will run on Linux and potentially need a lot of child processes. What I can't gauge is... how many child processes are too many for one parent to spawn and then wait upon?
Suppose my code looked like this:
pid_t status[ LARGENUMBER ];
status[0] = fork();
if( status[0] == 0 )
{
// I am the child
exec("./newCode01.c");
}
status[1] = fork();
if( status[1] == 0 )
{
// child
exec("./newCode02.c");
}
...etc...
wait(status[0]);
wait(status[1]);
...and so on....
Obviously, the larger LARGENUMBER is, the greater the chance that the parent is still fork() ing while children are segfaulting or becoming zombies or whatever.
So this implementation seems problematic to me. As I understand it, the parent can only wait() for one child at a time? What if LARGENUMBER is huge, and the time gap between running status[0] = fork(); and wait(status[0]); is substantial? What if the child has run, becomes a zombie, and been terminated by the OS somehow in that time? Will the parent then wait(status[0]) forever?
In the above example, there must be some standard or guideline to how big LARGENUMBER can be. Or is my approach all wrong?
#define LARGENUMBER 1
#define LARGENUMBER 10
#define LARGENUMBER 100
#define LARGENUMBER 1000
#define LARGENUMBER ???
I want to play with this, but my instinct is to ask for advice before I invest the development time into a program which may or may not turn out to be infeasible. Any advice/experience is appreciated.
If you read the documentation of wait, you would know that
If status information is available prior to the call to wait(), return will be immediate.
That means, if the child has already terminated, wait() will return immediately.
The OS will not remove the information from the process table until you have called wait¹ for the child process or your program exits:
If a parent process terminates without waiting for all of its child processes to terminate, the remaining child processes will be assigned a new parent process ID corresponding to an implementation-dependent system process.
Of course you still can't spawn an unlimited amount of children, for more detail on that see Maximum number of children processes on Linux (as far as Linux is concerned, other OS will impose other limits).
¹: https://en.wikipedia.org/wiki/Zombie_process
I will try my best to explain.
First a bad example: where you fork() one child process, then wait for it to finish before forking another child process. This kills the multiprocessing degree, bad CPU utilization.
pid = fork();
if (pid == -1) { ... } // handle error
else if (pid == 0) {execv(...);} // child
else (pid > 0) {
wait(NULL); // parent
pid = fork();
if (pid == -1) { ... } // handle error
else if (pid == 0) {execv(...);} // child
else (pid > 0) {wait(NULL); } // parent
}
How should it be done ?
In this approach, you first create the two child process, then wait. Increase CPU utilization and multiprocessing degree.
pid1 = fork();
if (pid1 == -1) { ... } // handle error
if (pid1 == 0) {execv(...);}
pid2 = fork();
if (pid2 == -1) { ... } // handle error
if (pid2 == 0) {execv(...);}
if (pid1 > 0) {wait(NULL); }
if (pid2 > 0) {wait(NULL); }
NOTE:
even though it seems as parent is waiting before the second wait is executed, the child is still running and is not waiting to execv or being spawned.
In your case, you are doing the second approach, first fork all processes and save return value of fork then wait.
the parent can only wait() for one child at a time?
The parent can wait for all its children one at a time!, whether they already finished and became zombie process or still running. For more explained details look here.
How many child processes can a parent spawn before becoming infeasible?
It might be OS dependent, but one acceptable approach is to split the time given to a process to run in 2, half for child process and half for parent process.
So that processes don't exhaust the system and cheat by creating child processes which will run more than the OS wanted to give the parent process in first place.
I'm playing with the Linux kernel, and one thing that I don't understand is the pid of the init_task task.
As far as I know, there are two special pids: pid 0 for the idle/swapper task, and pid 1 for the init task.
Every online resource (e.g. one, two) I could find say that the init_task task represents the swapper task, i.e. it should have pid 0.
But when I print all the pids using the for_each_process macro, which starts from init_task, I get pid 1 as the first process. I don't get pid 0 at all. Which means that init_task has pid 1, and that it's the init task (?!).
Please help me resolve this confusion.
P.S. the kernel version is 2.4.
The reason for my confusion was the tricky definition of the for_each_task macro:
#define for_each_task(p) \
for (p = &init_task ; (p = p->next_task) != &init_task ; )
Even though it seems that p starts from init_task, it actually starts from init_task.next_task because of the assignment in the condition.
So for_each_task(p) { /* ... */ } could be rewritten as:
p = init_task.next_task;
while(p != &init_task)
{
/* ... */
p = p->next_task;
}
As it can be seen, the swapper process is not part of the iteration.
I have an api, its working process is like this:
doing some logic, using 1 second's CPU time
wait for network IO, and this IO need 1 second too.
So, normally this api will need about 2 seconds to respond
Then I did a test.
I start 10 requests at the same time.
EVERY ONE OF THEM need more than 10 seconds to respond
This test means
Node will finish all the cpu costly part of all the 10 requests first.
WHY?
why doesn't it respond to one request immediately after one IO is done.
Thanks for the comments. I think I need to do some explanation about my concern.
What i concern is if the request count is not 10, if there are 100 request at the same time.
All of them will timeout!!
If the Node respond to the child IO event immediately, I think at least 20% of them will not time out.
I think node need some Event Priority mechanism
router.use('/test/:id', function (req, res) {
var id = req.param('id');
console.log('start cpu code for ' + id);
for (var x = 0; x < 10000; x++) {
for (var x2 = 0; x2 < 30000; x2++) {
x2 -= 1;
x2 += 1;
}
}
console.log('cpu code over for ' + id);
request('http://terranotifier.duapp.com/wait3sec/' + id, function (a,b,data) {
// how can I make this code run immediately after the server response to me.
console.log('IO over for ' + data);
res.send('over');
});
});
Node.js is single threaded. Therefore as long as you have a long running routine it cannot process other pieces of code. The offending piece of code in this instance is your double for loop which takes up a lot of CPU time.
To understand what you're seeing first let me explain how the event loop works.
Node.js event loop evolved out of javascript's event loop which evolved out of web browsers event loop. The web browser event loop was originally implemented not for javascript but to allow progressive rendering of images. The event loop looks a bit like this:
,-> is there anything from the network?
| | |
| no yes
| | |
| | '-----------> read network data
| V |
| does the DOM need updating? <-------------'
| | |
| no yes
| | |
| | v
| | update the DOM
| | |
'------'--------------'
When javascript was added the script processing was simply inserted into the event loop:
,-> is there anything from the network?
| | |
| no yes
| | |
| | '-----------> read network data
| V |
| any javascript to run? <------------------'
| | |
| no yes
| | '-----------> run javascript
| V |
| does the DOM need updating? <-------------'
| | |
| no yes
| | |
| | v
| | update the DOM
| | |
'------'--------------'
When the javascript engine is made to run outside of the browser, as in Node.js, the DOM related parts are simply removed and the I/O becomes generalized:
,-> any javascript to run?
| | |
| no yes
| | |
| | '--------> RUN JAVASCRIPT
| V |
| is there any I/O <------------'
| | |
| no yes
| | |
| | v
| | read I/O
| | |
'------'--------------'
Note that all your javascript code is executed in the RUN JAVASCRIPT part.
So, what happens with your code when you make 10 connections?
connection1: node accepts your request, processes the double for loops
connection2: node is still processing the for loops, the request gets queued
connection3: node is still processing the for loops, the request gets queued
(at some point the for loop for connection 1 finishes)
node notices that connection2 is queued so connection2 gets accepted,
process the double for loops
...
connection10: node is still processing the for loops, the request gets queued
(at this point node is still busy processing some other for loop,
probably for connection 7 or something)
request1: node is still processing the for loops, the request gets queued
request2: node is still processing the for loops, the request gets queued
(at some point all connections for loops finishes)
node notices that response from request1 is queued so request1 gets processed,
console.log gets printed and res.send('over') gets executed.
...
request10: node is busy processing some other request, request10 gets queued
(at some point request10 gets executed)
This is why you see node taking 10 seconds answering 10 requests. It's not that the requests themselves are slow but their responses are queued behind all the for loops and the for loops get executed first (because we're still in the current loop of the event loop).
To counter this, you should make the for loops asynchronous to give node a chance to process the event loop. You can either write them in C and use C to run independent threads for each of them. Or you can use one of the thread modules from npm to run javascript in separate threads. Or you can use worker-threads which is a web-worker like API implemented for Node.js. Or you can fork a cluster of processes to execute them. Or you can simply loop them with setTimeout if parallelism is not critical:
router.use('/test/:id', function (req, res) {
var id = req.param('id');
console.log('start cpu code for ' + id);
function async_loop (count, callback, done_callback) {
if (count) {
callback();
setTimeout(function(){async_loop(count-1, callback)},1);
}
else if (done_callback) {
done_callback();
}
}
var outer_loop_done=0;
var x2=0;
async_loop(10000,function(){
x1++;
async_loop(30000,function(){
x2++;
},function() {
if (outer_loop_done) {
console.log('cpu code over for ' + id);
request('http://terranotifier.duapp.com/wait3sec/' + id,
function (a,b,data){
console.log('IO over for ' + data);
res.send('over');
}
);
}
});
},function(){
outer_loop_done = 1;
});
});
The above code will process a response from request() as soon as possible rather than wait for all the async_loops to execute to completion without using threads (so no parallelism) but simply using event queue priority.
I'm having a problem trying to implement a solution to the producer-consumer problem in linux.
I'm supposed to use a shared semaphore to synchronize across different processes.
I understand the concept of signalling alternate semaphores like this (I'm also using sem_open() to make the semaphores shared across the two processes:
process 1
----------------------------------------------
semaphore s1 = sem_open("/s1", O_CREAT, 0666, 0);
semaphore s2 = sem_open("/s2", O_CREAT, 0666, BUFFSIZE);
sem_wait(s2);
/* do stuff */
printf("This is process 1!\n");
sem_post(s1);
process 2
----------------------------------------------
semaphore s1 = sem_open("/s1", 0);
semaphore s2 = sem_open("/s2", 0);
sem_wait(s1);
/* do stuff */
printf("This is process 2!\n");
sem_post(s2);
The problem I'm having is that both processes are deadlocked. From what I understand, process one should enter the critical section first (since the semaphore has an initial value of BUFFSIZE), then signal s1 so process 2 can proceed.
This isn't the case; both processes just sit there blankly, with no output to the screen.