The assignment I have to do requiers working with processes in linux. The task sounds like this: calculate the sum of the elements of an array by using the divide and conquer method in the following way- a parent proccess would split the array in two different subbarrays, which are passed to two child processes. Every child process must calculate the sum of the elements from their own subarray and the results(s1 and s2) would then be added; the child processes should repeat the same "technique" till the final sum is returned.
I must admit that I really don't know much about unix procceses, as I've just started studying this chapter, but I know how to use fork() in order to create two child procceses from a parent proccess and also who to write a C program which uses the divide and conquer method. My problem is that I am struggling bringing these two aspects toghether, which means integrating the divide and conquer algorithm-for calculating the sum of the elements of an array- in a program that creates two child proccesses(such as the following one:)
pid_t child_a, child_b;
child_a = fork();
if (child_a == 0) {
/* Child A code */
} else {
child_b = fork();
if (child_b == 0) {
/* Child B code */
} else {
/* Parent Code */
}
}
You can try to use a for loop to do fork().
First, divide the array into multiple subarrays.
In the first for() loop, you just create a child by fork(), and assign a subarray to it.(just do the calculation in the if (pid == 0) section.).
After that, all child processes are ready to do calculations.
You need another for() loop to sum up the result from all child processes. But be careful, you have to wait for all child process until all of their results come out.
Finally, the work is done.
Related
I'm a C programmer learning about fork(), exec(), and wait() for the first time. I'm also whiteboarding a Standard C program which will run on Linux and potentially need a lot of child processes. What I can't gauge is... how many child processes are too many for one parent to spawn and then wait upon?
Suppose my code looked like this:
pid_t status[ LARGENUMBER ];
status[0] = fork();
if( status[0] == 0 )
{
// I am the child
exec("./newCode01.c");
}
status[1] = fork();
if( status[1] == 0 )
{
// child
exec("./newCode02.c");
}
...etc...
wait(status[0]);
wait(status[1]);
...and so on....
Obviously, the larger LARGENUMBER is, the greater the chance that the parent is still fork() ing while children are segfaulting or becoming zombies or whatever.
So this implementation seems problematic to me. As I understand it, the parent can only wait() for one child at a time? What if LARGENUMBER is huge, and the time gap between running status[0] = fork(); and wait(status[0]); is substantial? What if the child has run, becomes a zombie, and been terminated by the OS somehow in that time? Will the parent then wait(status[0]) forever?
In the above example, there must be some standard or guideline to how big LARGENUMBER can be. Or is my approach all wrong?
#define LARGENUMBER 1
#define LARGENUMBER 10
#define LARGENUMBER 100
#define LARGENUMBER 1000
#define LARGENUMBER ???
I want to play with this, but my instinct is to ask for advice before I invest the development time into a program which may or may not turn out to be infeasible. Any advice/experience is appreciated.
If you read the documentation of wait, you would know that
If status information is available prior to the call to wait(), return will be immediate.
That means, if the child has already terminated, wait() will return immediately.
The OS will not remove the information from the process table until you have called wait¹ for the child process or your program exits:
If a parent process terminates without waiting for all of its child processes to terminate, the remaining child processes will be assigned a new parent process ID corresponding to an implementation-dependent system process.
Of course you still can't spawn an unlimited amount of children, for more detail on that see Maximum number of children processes on Linux (as far as Linux is concerned, other OS will impose other limits).
¹: https://en.wikipedia.org/wiki/Zombie_process
I will try my best to explain.
First a bad example: where you fork() one child process, then wait for it to finish before forking another child process. This kills the multiprocessing degree, bad CPU utilization.
pid = fork();
if (pid == -1) { ... } // handle error
else if (pid == 0) {execv(...);} // child
else (pid > 0) {
wait(NULL); // parent
pid = fork();
if (pid == -1) { ... } // handle error
else if (pid == 0) {execv(...);} // child
else (pid > 0) {wait(NULL); } // parent
}
How should it be done ?
In this approach, you first create the two child process, then wait. Increase CPU utilization and multiprocessing degree.
pid1 = fork();
if (pid1 == -1) { ... } // handle error
if (pid1 == 0) {execv(...);}
pid2 = fork();
if (pid2 == -1) { ... } // handle error
if (pid2 == 0) {execv(...);}
if (pid1 > 0) {wait(NULL); }
if (pid2 > 0) {wait(NULL); }
NOTE:
even though it seems as parent is waiting before the second wait is executed, the child is still running and is not waiting to execv or being spawned.
In your case, you are doing the second approach, first fork all processes and save return value of fork then wait.
the parent can only wait() for one child at a time?
The parent can wait for all its children one at a time!, whether they already finished and became zombie process or still running. For more explained details look here.
How many child processes can a parent spawn before becoming infeasible?
It might be OS dependent, but one acceptable approach is to split the time given to a process to run in 2, half for child process and half for parent process.
So that processes don't exhaust the system and cheat by creating child processes which will run more than the OS wanted to give the parent process in first place.
#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
#include<stdlib.h>
int main(void)
{
pid_t pids[10];
int i;
for (i = 9; i >= 0; --i) {
pids[i] = fork();
if (pids[i] == 0) {
printf("Child%d\n",i);
sleep(i+1);
_exit(0);
}
}
for (i = 9; i >= 0; --i){
printf("parent%d\n",i);
waitpid(pids[i], NULL, 0);
}
return 0;
}
What is happening here? How is sleep() getting executed in the for loop? When is it getting called? Here is the output:
parent9
Child3
Child4
Child2
Child5
Child1
Child6
Child0
Child7
Child8
Child9 //there is a pause here
parent8
parent7
parent6
parent5
parent4
parent3
parent2
parent1
parent0
Please explain this output. I am not able to understand how it's working.
Step by step analysis would be great.
In the first loop, the original (parent) process forks 10 copies of itself. Each of these child processes (detected by the fact that fork() returned zero) prints a message, sleeps, and exits. All of the children are created at essentially the same time (since the parent is doing very little in the loop), so it's somewhat random when each of them gets scheduled for the first time - thus the scrambled order of their messages.
During the loop, an array of child process IDs is built. There is a copy of the pids[] array in all 11 processes, but only in the parent is it complete - the copy in each child will be missing the lower-numbered child PIDs, and have zero for its own PID. (Not that this really matters, as only the parent process actually uses this array.)
The second loop executes only in the parent process (because all of the children have exited before this point), and waits for each child to exit. It waits for the child that slept 10 seconds first; all the others have long since exited, so all of the messages (except the first) appear in quick succession. There is no possibility of random ordering here, since it's driven by a loop in a single process. Note that the first parent message actually appeared before any of the children messages - the parent was able to continue into the second loop before any of the child processes were able to start. This again is just the random behavior of the process scheduler - the "parent9" message could have appeared anywhere in the sequence prior to "parent8".
I see that you've tagged this question with "zombie-processes". Child0 thru Child8 spend one or more seconds in this state, between the time they exited and the time the parent did a waitpid() on them. The parent was already waiting on Child9 before it exited, so that one process spent essentially no time as a zombie.
This example code might be a bit more illuminating if there were two messages in each loop - one before and one after the sleep/waitpid. It would also be instructive to reverse the order of the second loop (or equivalently, to change the first loop to sleep(10-i)).
Consider de following code fragment:
for(i = 0; i < 5; i++)
if(fork() == 0) {
printf("%d\n", i);
wait(0);
}
What will be the result and how many new processes will be created?
(1) is this a homework assignment? - A question about a homework assignment is ok, getting someone to do your homework assignment for you is not.
(2) why don't you try it and see?
(3) if a process itself has no child processes, wait will (immediately) return -1.
(4) Be warned that each child process, after wait(0);, will continue the loop - that is, the parent will fork and the child will print 0; the second time round, both processes will fork, and their children will print 1; the third time round, all four processes will fork, and their four children will print 2, and so forth.
(5) also, be warned that the processes each run independently - the first child may go round the loop several times before the parent does even one, or vice-versa.
If you have a computer to access the internet, you have a computer to try things on - if you are going to be doing C homework in the future, it would be well worth the effort to download a free C compiler for your computer. Try Tiny C at http://bellard.org/tcc/
I have a function that boils down to:
while(doWork)
{
config = generateConfigurationForTesting();
result = executeWork(config);
doWork = isDone(result);
}
How can I rewrite this for efficient asynchronous execution, assuming all functions are thread safe, independent of previous iterations, and probably require more iterations than the maximum number of allowable threads ?
The problem here is we don't know how many iterations are required in advance so we can't make a dispatch_group or use dispatch_apply.
This is my first attempt, but it looks a bit ugly to me because of arbitrarily chosen values and sleeping;
int thread_count = 0;
bool doWork = true;
int max_threads = 20; // arbitrarily chosen number
dispatch_queue_t queue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
while(doWork)
{
if(thread_count < max_threads)
{
dispatch_async(queue, ^{ Config myconfig = generateConfigurationForTesting();
Result myresult = executeWork();
dispatch_async(queue, checkResult(myresult)); });
thread_count++;
}
else
usleep(100); // don't consume too much CPU
}
void checkResult(Result value)
{
if(value == good) doWork = false;
thread_count--;
}
Based on your description, it looks like generateConfigurationForTesting is some kind of randomization technique or otherwise a generator which can make a near-infinite number of configuration (hence your comment that you don't know ahead of time how many iterations you will need). With that as an assumption, you are basically stuck with the model that you've created, since your executor needs to be limited by some reasonable assumptions about the queue and you don't want to over-generate, as that would just extend the length of the run after you have succeeded in finding value ==good measurements.
I would suggest you consider using a queue (or OSAtomicIncrement* and OSAtomicDecrement*) to protect access to thread_count and doWork. As it stands, the thread_count increment and decrement will happen in two different queues (main_queue for the main thread and the default queue for the background task) and thus could simultaneously increment and decrement the thread count. This could lead to an undercount (which would cause more threads to be created than you expect) or an overcount (which would cause you to never complete your task).
Another option to making this look a little nicer would be to have checkResult add new elements into the queue if value!=good. This way, you load up the initial elements of the queue using dispatch_apply( 20, queue, ^{ ... }) and you don't need the thread_count at all. The first 20 will be added using dispatch_apply (or an amount that dispatch_apply feels is appropriate for your configuration) and then each time checkResult is called you can either set doWork=false or add another operation to queue.
dispatch_apply() works for this, just pass ncpu as the number of iterations (apply never uses more than ncpu worker threads) and keep each instance of your worker block running for as long as there is more work to do (i.e. loop back to generateConfigurationForTesting() unless !doWork).
I am working on multithread programming and I am stuck on something.
In my program there are two tasks and two types of robots for carrying out the tasks:
Task 1 requires any two types of robot and
task 2 requires 2 robot1 type and 2 robot2 type.
Total number of robot1 and robot2 and pointers to these two types are given for initialization. Threads share these robots and robots are reserved until a thread is done with them.
Actual task is done in doTask1(robot **) function which takes pointer to a robot pointer as parameter so I need to pass the robots that I reserved. I want to provide concurrency. Obviously if I lock everything it will not be concurrent. robot1 is type of Robot **. Since It is used by all threads before one thread calls doTask or finish it other can overwrite robot1 so it changes things. I know it is because robot1 is shared by all threads. Could you explain how can I solve this problem? I don't want to pass any arguments to thread start routine.
rsc is my struct to hold number of robots and pointers that are given in an initialization function.
void *task1(void *arg)
{
int tid;
tid = *((int *) arg);
cout << "TASK 1 with thread id " << tid << endl;
pthread_mutex_lock (&mutexUpdateRob);
while (rsc->totalResources < 2)
{
pthread_cond_wait(&noResource, &mutexUpdateRob);
}
if (rsc->numOfRobotA > 0 && rsc->numOfRobotB > 0)
{
rsc->numOfRobotA --;
rsc->numOfRobotB--;
robot1[0] = &rsc->robotA[counterA];
robot1[1] = &rsc->robotB[counterB];
counterA ++;
counterB ++;
flag1 = true;
rsc->totalResources -= 2;
}
pthread_mutex_unlock (&mutexUpdateRob);
doTask1(robot1);
pthread_mutex_lock (&mutexUpdateRob);
if(flag1)
{
rsc->numOfRobotA ++;
rsc->numOfRobotB++;
rsc->totalResources += 2;
}
if (totalResource >= 2)
{
pthread_cond_signal(&noResource);
}
pthread_mutex_unlock (&mutexUpdateRob);
pthread_exit(NULL);
}
If robots are global resources, threads should not dispose of them. It should be the duty of the main thread exit (or cleanup) function.
Also, there sould be a way for threads to locate unambiguously the robots, and to lock their use.
The robot1 array seems to store the robots, and it seems to be a global array. However:
its access is not protected by a mutex (pthread_mutex_t), it seems now that you've taken care of that.
Also, the code in task1 is always modifying entries 0 and 1 of this array. If two threads or more execute that code, the entries will be overwritten. I don't think that it is what you want. How will that array be used afterwards?
In fact, why does this array need to be global?
The bottom line is this: as long as this array is shared by threads, they will have problems working concurrently. Think about it this way:
You have two companies using robots to work, but they're using the same truck (robot1) to move the robots around. How are these two companies supposed to function properly, and efficiently with only one truck?