I need for a specific reason of my work to configure a process in Ubuntu 14.10 Linux distro to use alternately different CPUs.
I know with taskset I can pin a process on a specified CPUs but how I tell that process to use let say 50% of CPU 0 and 50% of CPU 1.
UPDATE
The process I'm runing is single threaded, maybe I'm wrong to try to tell a single threaded process to use two CPUs with roundrobin algorithm.
Thank you for your insights and helps.
Regards.
I would use task set to pin two processes to the two different cores, then use a message passing protocol to switch the processing from one core to the other and back again. I'm assuming you don't want to concurrently execute a task across two cores.
Related
I have VM which builds on cantos 7,so I want to make every process in the system to use more than one core in parallel
I have a 24 CPU on my server, but every process can't use more than 1 core, after that the CPU is always 100%. I need to make more utilization of CPU with other cores.
I need to make a huge process to use multipe cores instead of one core?
How can I do such a thing like that?
Does fork always create a process in a separate processor?
Is there a way, I could control the forking to a particular processor. For example, if I have 2 processors and want the fork to create a parallel process but in the same processor that contains the parent. Does NodeJS provide any method for this? I am looking for a control over the allocation of the processes. ... Is this even a good idea?
Also, what are the maximum number of processes that could be forked and why?
I've no Node.js wisdom to impart, simply some info on what OSes generally do.
Any modern OS will schedule processes / threads on CPUs and cores according to the prevailing burden on the machine. The whole point is that they're very good at this, so one is going to have to try very hard to come up with scheduling / core affinity decisions that beat the OS. Almost no one bothers. Unless you're running on very specific hardware (which perhaps, perhaps one might get to understand very well), you're having to make a lot of complex decisions for every single different machine the code runs on.
If you do want to try then I'm assuming that you'll have to dig deep below node.JS to make calls to the underlying C library. Most OSes (including Linux) provide means for a process to control core affinity (it's exposed in Linux's glibc).
I love node.js' evented model, but it only takes you so far - when you have a function (say, a request handler for HTTP connections) that does a lot of heavy work on the CPU, it's still "blocking" until its function returns. That's to be expected. But what if I want to balance this out a bit, so that a given requests takes longer to process but the overall response time is shorter, using the operarting system's ability to schedule the processes?
My production code uses node's wonderfully simple Cluster module to fork a number of workers equal to the number of cores the system's CPU has. Would it be bad to fork more than this - perhaps two or three workers per core? I know there'll be a memory overhead here, but memory is not my limitation. What reading I did mentioned that you want to avoid "oversubscribing", but surely on a modern system you're not going crazy by having two or three processes vying for time on the processor.
I think your idea sounds like a good one; especially because many processors support hyperthreading. Hyperthreading is not magical and won't suddenly double your application's speed or throughput but it can make sense to have another thread ready to execute in a core when the first thread needs to wait for a memory request to be filled.
Be careful when you start multiple workers: the Linux kernel really prefers to keep processes executing on the same processor for their entire lifetime to provide for strong cache affinity. This makes enough sense. But I've seen several CPU-hungry processes vying for a single core or worse a single hyperthread instance rather than the system re-balancing the processes across all cores or all siblings. Check your processor affinities by running ps -eo pid,psr,comm (or whatever your favorite ps(1) command is; add the psr column).
To combat this you might want to start your workers with an explicitly limited CPU affinity:
taskset -c 0,1 node worker 1
taskset -c 2,3 node worker 2
taskset -c 4,5 node worker 3
taskset -c 6,7 node worker 4
Or perhaps start eight, one per HT sibling, or eight and confine each one to their own set of CPUs, or perhaps sixteen, confine four per core or two per sibling, etc. (You can go nuts trying to micromanage. I suggest keeping it simple if you can.) See the taskset(1) manpage for details.
I'm studying about threads and slightly confused about 1 thing.
If I have a single process with multiple threads running on a dual/quad core CPU, will different threads run concurrently on different cores?
Thanks in advance.
It Depends.
At least on Linux, each task gets assigned to a set of CPUs that it can execute on (processor affinity). And, at least on Linux, the scheduler will try to schedule the task on the same processor as last time, so that it gets the best benefit of CPU cache re-use. The hilarious thing is that it doesn't always rebalance when the system is under load, so it is possible to run one core quite hot and contested and leave three cores cool and relatively idle. (I've seen this exact behavior with the Folding # Home client.)
You can force the affinity you need with the pthread_setaffinity_np(3) routine for threaded applications or sched_setaffinity(2) for more traditional Unix-style fork(2)ed applications. Or you can use the taskset(1) program to set the affinity before or after starting an application. (Which is the approach I took with my silly Folding # Home client -- it was easy to modify the initscript to call taskset(1) to set the affinity of each client process correctly, so each client got its own core and didn't compete for resources with the other clients on different sibling HyperThreaded 'faked' execution cores.)
Yes
It depends on the language, the library, and the operating system, and whether the threaded application ever actually has multiple runnable threads at the same point in time, but usually the answer is "yes".
You can never be sure of that fact, but if it is processor-intensive (such as a game) then most likely yes.
In that case you need to synchronized your every core of processor with memory by using volatile keyword which ensure that every core of processor getting new updated value from memory.
Somnetimes the threads will run concurrently, sometimes not. Its all up to the package you use and the operating system and how CPU intensive each thread each.
I think that you are loosing the idea behind concurrency; it's not that you are looking to run processes on multiple cores. Instead, you're needing to not block on one process the entire time. A perfect example of this is with threading network listeners. You want to perform an accept which will actually create a new client->server socket. After this you want to do some processing with that socket while still be able to take new connections. This is where you would want to generate a thread to perform the processing so that the accept can get back on track to waiting for a new connection.
In my server, there exists several CPUs (0-7). I need run parallel code, and each process affiliate with one CPU, so how do I know the CPU information for each process?
For example, if two processes (#0 and #1) exist, and process #0 uses CPU 5 and process #1 uses CPU 7.
how do I know that by programming in C or Fortran?
Use the sched_getcpu() call.
Keep in mind that a process/thread can be scheduled freely to run on any available cpu/core, so one of your processes could run on core 1 one second, and on core 2 the next milisecond. You can restrict which processors a process is allowed to run on with sched_setaffinity()
I'm not aware of any system call on Linux that will give you general information about what CPU a thread in running on. #nos is correct that sched_getcpu() will tell you which CPU a thread is running on, but only for the calling context.
You can do this by querying the /proc file system. However, if you find yourself building your application around this functionality, it is likely that you need to reexamine your design.
The file /proc/<pid>/stats contains a field that provides you with the last CPU the process ran on. You would just need to parse the output. (use man proc to see the field list).
In general it is the task of the operating system to abstract such things from applications.
Normally I see my applications (as simple as doing a grep on a huge file) change CPU core every once in a while.
Now if you want to force an application on a specific core you can manually set the CPU affinity.
I've written some pretty strange software in the past and I've never had the desire to know and/or control this.
Why would you want to know?
More generally, why do you want to know? The Linux kernel is very good at scheduling processes/threads to make the best use of the available cores.
Generally, you have to change the CPU affinity, because a process can migrate between processors: CPU Affinity (Linux Journal, 2003).