Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I'm recompiling my kernel, and I want to choose an I/O scheduler. What's the difference between these?
If you compile them all, you can select at boot time or per-device which scheduler to use. No need to pick at compile time, unless you are targeting an embedded device where every byte counts. See Documentation/block/switching-sched.txt for details on switching per-device or system-wide at boot.
The CFQ scheduler allows you to set priorities via the ionice(1) tool or the ioprio_set(2) system call. This allows giving precedence to some processes or forcing others to do their IO only when the system's block devices are relatively idle. The queues are implemented by segregating the IO requests from processes into queues, and handling the requests from each queue similar to CPU scheduling. Details on configuring it can be found in Documentation/block/cfq-iosched.txt.
The deadline scheduler by contrast looks at all writes from all processes at once; it sorts the writes by sector number, and writes them all in linear fashion. The deadlines means that it tries to write each block before its deadline expires, but within those deadlines, is free to re-arrange blocks as it sees fit. Details on configuring it can be found in Documentation/block/deadline-iosched.txt.
Probably very little in practice.
In my testing, I found that in general NOOP is a bit better if you have a clever RAID controller. Others have reported similar results, but your workload may be different.
However, you can select them at runtime (without reboot) so don't worry about it at compile-time.
My understanding was that the "clever" schedulers (CFQ and deadline) are only really helpful on traditional "spinning disc" devices which don't have a RAID controller.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 months ago.
Improve this question
I am writing a Rust script that needs to brute force the solution to some calculation and is likely to run 2^80 times. That is a lot! I am trying to make it run as fast as possible and thus want to divide the burden to multiple threads. However if I understand correctly this only accelerates my script if the threads actually run on different cores, otherwise they will not truly run simultaneously but switch between one another when running..
How can I make sure they use different cores, and how can I know that no more cores are available?
TL;DR: Use std::thread::available_parallelism (or alternatively the num-cpus crate) to know how many threads to run and let your OS handle the rest.
Typically when you create a thread, the OS thread scheduler is given free liberty to decide where and when those threads execute, however it will do so in a way that best takes advantage of CPU resources. So of course if you use less threads than the system has available, you are potentially missing out on performance. If you use more than the number of available threads, that's not particularly a problem since the thread scheduler will try its best to balance the threads that have work to do, but more than the available threads would be a mall waste of memory, OS resources, and context-switches. Creating your threads to match the number of logical CPU cores on your system is the sweetspot, and the above function will get that.
You could tell the OS exactly which cores to run which threads by setting their affinity, however that isn't really advisable since it wouldn't particularly make anything faster unless you start really configuring your kernel or are really taking advantage of your NUMA nodes.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 3 years ago.
Improve this question
I am seeing some slow Marklogic cluster logs like below
2020-01-14 05:55:22.649 Info: Slow background cluster.background.clusters, 5.727 sec
2020-01-14 05:55:22.649 Info: Slow background cluster.background.hosts.AssignmentManager, 5.581 sec
I suspect MarkLogic filesystem is running slow and does not able to keep up with MarkLogic. I am seeing below log entry also:-
2020-01-14 05:55:53.380 Info: Linux file system mount option 'barrier' is default; recommend faster 'nobarrier' if storage has non-volatile write cache
I want to know what is the meaning of the above log entry in MarkLogic? How can I be sure that filesystem is having slowness problems or not?
The meaning of "slow messages" is that a background activity takes longer time than expected. It is an indicator of starvation.
From your question it's impossible to say what is causing it. Typically, it's related to underlying physical infrastructure where MarkLogic is running. MarkLogic doesn't have its filesystem or other resources - it uses the OS's filesystem, memory etc. and if available physical resources are not enough for MarkLogic to serve the requested load, background operations will take longer time than expected. This will always be reflected in the log.
You can read more here:
Understanding "slow background" messages
https://help.marklogic.com/Knowledgebase/Article/View/508/0/understanding-slow-infrastructure-notifications
29 August 2019 10:54 AM
Introduction
In more recent versions of MarkLogic Server, "slow background" error log messages were added to note and help diagnose slowness.
Details
For "Slow background" messages, the system is timing how long it took to do some named background activity. These activities should not take long and the "slow background" message is an indicator of starvation. The activity can be slow because:
it is waiting on a mutex or semaphore held by some other slow thread;
the operating system is stalling it, possibly because it is thrashing because of low memory.
Looking at the "slow background" messages in isolation is not sufficient to understand the reason - we just know a lot of time passed since the last time we read the time of day clock. To understand the actual cause, additional evidence will need to be gathered from the time of the incident.
Notes:
In general, we do not time how long it takes to acquire a mutex or semaphore as reading the clock is usually more expensive than getting a mutex or semaphore.
We do not time things that usually take about a microsecond.
We do time things that usually take about a millisecond.
Related Articles
Knowledgebase: Understanding Slow Infrastructure Notifications
Knowledgebase: (Understanding slow 'journal frame' entries in the ErrorLog)[https://help.marklogic.com/Knowledgebase/Article/View/460/0/understanding-slow-journal-frame-entries-in-the-errorlog]
Knowledgebase: (Hung Messages in the ErrorLog)[https://help.marklogic.com/Knowledgebase/Article/View/35/0/hung-messages-in-the-errorlog]
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
user threads cannot take advantage of multithreading or multiprocessing
source : wikipedia
Does this mean a CPU cannot efficiently execute multiple user threads simultaneously ?
Does this mean a CPU cannot switch between two or more user threads ?
For example : there are two user threads t0 and t1. t0 is the first one to execute. Will t1 only begin execution when t0 has finished or can switching take place ?
PS : This question might look like more than one question but I guess it is just one.
Here's what the page currently says:
Threads are sometimes implemented in userspace libraries, thus called user threads. The kernel is unaware of them, so they are managed and scheduled in userspace. Some implementations base their user threads on top of several kernel threads, to benefit from multi-processor machines (M:N model). In this article the term "thread" (without kernel or user qualifier) defaults to referring to kernel threads. User threads as implemented by virtual machines are also called green threads. User threads are generally fast to create and manage, but cannot take advantage of multithreading or multiprocessing and get blocked if all of their associated kernel threads get blocked even if there are some user threads that are ready to run.
As you can see, in one paragraph it is stating BOTH that user threads both can take advantage of multiprocessors (via associated kernel threads), AND that it cannot.
I suggest that you ask your question on the Wikipedia page's Talk page, and see if they authors can enlighten you as to what they mean ... and why they are saying it.
But what I think they are saying that user(-space) threads that aren't backed by multiple kernel threads typically cannot execute simultaneously on multiple cores.
However, I would hesitate to say that this is inherent to user threads per se; i.e. that it would be impossible to implement an OS in which a application could exploit multiple cores without any kernel assistance.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
As we all know that init process is the first process which is being executed and results in the creation of further inevitable processes.Now, the question is that ,what could be the matter of primarily concern that would have lead the init process to create processes rather than threads.
Sounds like you are talking about how most Unix systems start up. Well, let's forget about init for a moment and ask, why would any process create another process instead of creating another thread?
The whole point of having an operating system at all is to allow different "programs" to co-exist and share the hardware without clobbering one another. Threads, by definition, exist in the same address space. There is no way to protect one thread's data structures from being clobbered by another thread in the same process except to write the code in such a way that the threads cooperate with one another. Threads within a process are always part of the same "program".
The Unix init process spawns other processes because many of the services in a Unix system are provided by different programs. And why is that? Partly it's for historical reasons. Nobody had ever heard of "threads" when Unix was new. But also, it's a convenient way to organize components that are not related to one another. The person who writes the ssh daemon does not have to worry about whether it plays nicely with the cron daemon or not. Since they are separate programs, the operating system automatically hides them/protects them from each other the same way it hides/protects user programs.
The main problem is that threads in the same address space are not protected from other threads. If however, such a protection exists (like in Java), then using threads instead of processes make sense.
I know at least one operating system where all system activities were performed by threads in single system process - the one for Elbrus1 and 2 computers (modern Elbruses are operated by Linux). This was possible because of tagged memory architecture, inherited from Burrows machines. Probably Burrows machines worked that way.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Hi i am an embedded programmer. Recently we came across a project where we are forced to use multi threading. I have used the same in java but i could not implement it my embedded code for 8051. Could any body please help me?
Threading requires that there be some mechanism to switch threads, typically called a scheduler.
Broadly speaking, there are two types of threading: cooperative, and pre-emptive.
In cooperative threading, each thread does some work and then transfers control back to the scheduler. This is almost like having a grand while(1) {} loop as a program structure, only with more independence (only during development) of the tasks. It still suffers from the risk of one task hogging the CPU, or even locking up and preventing anything else from running. In effect, the independence between tasks is only an illusion or organizational abstraction for the developer.
In pre-emptive multi-tasking, the scheduler (likely driven from a timer interrupt) periodically forces a change of tasks by grabbing execution out of one thread, saving its state, and restarting a different frozen thread. This is a little trickier to set up, but a lot more reliable.
Often with either scheme, you would not write the infrastructure from scratch, but instead would use a primitive operating system or at least scheduler routine developed by others.
For a very small embedded system though, you can also consider that interrupt service routines can themselves provide something akin to alternate threads for handling certain brief and/or urgent tasks. If your serial interrupt fires, you grab some character(s) and store them for later interpretation at a convenient time by something else. Many tasks can be implemented by using interrupts to deal with the immediate part, and then doing resulting work at a later point in a while(1) {} type program structure.
Some might properly laugh at the idea of a scheduler running on an 8051 - though for an oddity of reasons, inexpensive little 8051-equivalent cores end up in some fairly complicated special purpose chips today (typically accessorized by huge amounts of banked memory, and powerful peripheral engines to do the real work), so it's actually not uncommon to see multithreading solutions with dynamic task creation implemented on them in order to manage everything which the device does.
The architecture of the 8051 is not amenable to any reasonable preemptive scheduling. At least the stack, and probably more, in the on-chip RDATA/IDATA has to swapped out to XDATA and it gets very messy.
8051 is good for toaster/washing-machine controllers.
If you want/need such functionality as a premptive scheduler, move to ARM.