Do I need to protect my interrupt handler being called many times for the same interrupt?
Given the following code, I am not sure on the system calls I should make. I am getting rare, random dead-locks with this current implementation :-
void interrupt_handler(void)
{
down_interruptible(&sem); // or use a lock here ?
clear_intr(); // clear interrupt source on H/W
wake_up_interruptible(...);
up(&sem); // unlock?
return IRQ_HANDLED;
}
void set/clear_intr()
{
spin_lock_irq(&lock);
RMW(x); // set/clear a bit by read/modify/write the H/W interrupt routing register
spin_unlock_irq(&lock);
}
void read()
{
set_intr(); // same as clear_intr, but sets a bit
wait_event_interruptible(...);
}
Should interrupt_handler:down_interruptible be spin_lock_irq / spin_lock_irqsave / local_irq_disable?
Should set/clear_intr:spin_lock_irq be spin_lock_irqsave / local_irq_disable?
Can it (H/W -> kernel -> driver handler) keep generating/getting interrupts until its cleared? Can the interrupt_handler keep getting called while within it?
If as currently implemented the interrupt handler is reentrant then will it block on the down_interruptible?
From LDD3 :-
must be reentrant—it must be capable of running in more than one context at the same time.
Edit 1) after some nice help, suggestions are :-
remove down_interruptible from within interrupt_handler
Move spin_lock_irq outside set/clear methods (no need for spin_lock_irqsave you say?) I really don't see the benefit to this?!
Code :-
void interrupt_handler(void)
{
read_reg(y); // eg of other stuff in the handler
spin_lock_irq(&lock);
clear_intr(); // clear interrupt source on H/W
spin_unlock_irq(&lock);
wake_up_interruptible(...);
return IRQ_HANDLED;
}
void set/clear_intr()
{
RMW(x);
}
void read()
{
error_checks(); // eg of some other stuff in the read method
spin_lock_irq(&lock);
set_intr(); // same as clear_intr, but sets a bit
spin_unlock_irq(&lock);
wait_event_interruptible(...);
// more code here...
}
Edit2) After reading some more SO posts : reading Why kernel code/thread executing in interrupt context cannot sleep? which links to Robert Loves article, I read this :
some interrupt handlers (known in
Linux as fast interrupt handlers) run
with all interrupts on the local
processor disabled. This is done to
ensure that the interrupt handler runs
without interruption, as quickly as
possible. More so, all interrupt
handlers run with their current
interrupt line disabled on all
processors. This ensures that two
interrupt handlers for the same
interrupt line do not run
concurrently. It also prevents device
driver writers from having to handle
recursive interrupts, which complicate
programming.
And I have fast interrupts enabled (SA_INTERRUPT)! So no need for mutex/locks/semaphores/spins/waits/sleeps/etc/etc!
Don't use semaphores in interrupt context, use spin_lock_irqsave instead. quoting LDD3:
If you have a spinlock that can be
taken by code that runs in (hardware
or software) interrupt context, you
must use one of the forms of spin_lock
that disables interrupts. Doing
otherwise can deadlock the system,
sooner or later. If you do not access
your lock in a hardware interrupt
handler, but you do via software
interrupts (in code that runs out of a
tasklet, for example, a topic covered
in Chapter 7), you can use
spin_lock_bh to safely avoid deadlocks
while still allowing hardware
interrupts to be serviced.
As for point 2, make your set_intr and clear_intr require the caller to lock the spinlock, otherwise you'll find your code deadlocking. Again from LDD3:
To make your locking work properly,
you have to write some functions with
the assumption that their caller has
already acquired the relevant lock(s).
Usually, only your internal, static
functions can be written in this way;
functions called from outside must
handle locking explicitly. When you
write internal functions that make
assumptions about locking, do yourself
(and anybody else who works with your
code) a favor and document those
assumptions explicitly. It can be very
hard to come back months later and
figure out whether you need to hold a
lock to call a particular function or
not.
Use spinlock in interrupt context because you don't want to sleep in interrupt context if you didn't acquired a lock.
The code you posted does not look like a device driver irq handler.
The irq handlers in kernel drivers return irqreturn_t and take in int irq_no, void * data as arguements.
You have also not specified if you are registering a threaded handler or a non threaded handler.
A non threaded irq handler cannot have any sleeping calls whether or not you hold any spinlocks.
wait_event, mutex, semaphore, etc, are all sleeping calls and must not be used in a non threaded irq handler. You can however, hold a spinlock to prevent interruption to your interrupt handler. This will ensure that maskable irqs and scheduler do not interrupt your irq handler in the middle.
In a threaded irq handler, such things as sleeping calls (wait queues, mutex, etc) can be used but are still not recommended.
Related
i have passed through this post and i noticed that in Clifford's answer he said that we shouldn't use mutex in an interrupt, i know that in an interrupt we have to avoid too much instructions and delays ext... but am not very clear about the reasons could anyone clarify me for which reason we have to avoid this?
In case that we want establish a synchronous communication between 2 interrupt driven threads what are the other mecahnism to use if using mutex is not allowed?
The original question you cite refers to code on an Atmel ATMegaAVR - a simple 8 mit microcontroller. In that context, one can assume that the mutex machanism is part of a simple RTOS.
In such a system, there is a thread context and an interrupt context. Interrupts are invoked by the hardware, while threads are scheduler by the RTOS scheduler. Now when an interrupt occurs, any thread will be immediately pre-empted; the interrupt must run to completion and can only be preempted by a higher priority interrupt (where nested interrupts are supported). All pending interrupts will run to completion before the scheduler can run.
Blocking on a mutex (or indeed any blocking kernel object) is a secheduling event. If you were to make any blocking call in an interrupt, the scheduler will never run. In prectice an RTOS would either ignore the blocking call, raise an exception, or enter a terminal error handler.
Some OS's such as SMX, Velocity or even WinCE have somewhat more complex interrupt architectures and support variety of deferred interrupt handler. Deferred interrupt handlers are run-to-completion scheduled from an interrupt but running outside of the interrupt context; the rules for blocking in such handlers may differ, but you would need to refer to the specific OS documentation. Without deferred interrupt handlers, the usual solution is to have a thread wait on a some blocking object such as a semaphore, and have the interrupt itself do little more that cause the object to unblock (such as giving a semaphore for example).
Multi-processor/core and parallel processing systems are another issue altogether, such systems are way beyond the scope of the question where the original comment was made, and beyond my experience - my comment may not apply in such a system, but there are no doubt additional complexities and considerations in any case
A mutex is typically used to ensure that a resource is used by only one user at any given time.
When a thread needs to use a resource it attempts to get the mutex first to ensure the resource is available. If the mutex is not available then the thread typically blocks to wait for the mutex to become available.
While a thread owns the mutex, it prevents other threads from obtaining the mutex and interfering with its use of the resource. Higher priority threads are often the concern here because those are the threads that may preempt the mutex owner.
The RTOS kernel assigns ownership of the mutex to a particular thread and typically only the mutex owner can release the mutex.
Now lets imagine this from an interrupt handler's point of view.
If an interrupt handler attempts to get a mutex that is not available, what should it do? The interrupt handler cannot block like the thread (the kernel is not equipped to push the context of an interrupt handler or switch to a thread from an interrupt handler).
If the interrupt handler obtains the mutex, what higher priority code is there that could interrupt the interrupt handler and attempt to use the mutex? Is the interrupt handler going to release the mutex before completing?
How does the kernel assign ownership of the mutex to an interrupt handler? An interrupt handler is not a thread. If the interrupt handler does not release the mutex then how will the kernel validate that the mutex is being released by the owner?
So maybe you have answers for all those questions. Maybe the you can guarantee that the interrupt handler runs only when the mutex is available or that the interrupt handler will not block on the mutex. Or maybe you're trying to protect the resource access from an even higher priority nested interrupt handler that also wants to use the resource. And maybe your kernel doesn't have any hangup with assigning ownership or restricting who releases the mutex. I guess if you've got all these questions answered then maybe you have a case for using a mutex within an interrupt handler.
But perhaps what you really need is a semaphore instead. One common application of a semaphore is to signal an event. Semaphores are very often used this way within interrupt handlers. The interrupt handler posts or sets the semaphore to signal that an event has occurred. The threads pend on the semaphore to wait for the event condition. (A semaphore doesn't have that ownership restriction that a mutex has.) Event signalling semaphores is one common way to establish synchronous communication between 2 interrupt driven threads.
The term "mutex" is often defined both as being the simplest form of synchronization between execution contexts, and also as being a construct that will not only check whether a resource is available, but wait for it to become available if it isn't, acquiring it as soon as it becomes available. These definitions are inconsistent, since the simplest forms of synchronization merely involve testing whether one has been granted ownership of a resource, and don't provide any in-built mechanism to wait for it if it isn't.
It is almost never proper to have code within an interrupt handler that waits for a resource to become available, unless the only things that could hold the resource would be higher-priority interrupts or hardware that will spontaneously release it. If the term "mutex" is only used to describe such constructs, then there would be very few cases where one could properly use a mutex within an interrupt handler. If, however, one uses the term "mutex" more broadly to refer to the simplest structures that will ensure that a piece of code that accesses a resource can only execute at times when no other piece of code anywhere in the universe will be accessing that resource, then the use of such constructs within interrupts is often not only proper, but required.
While there might be unusual cases where there's some problem with using a mutex in an interrupt handler, it's quite common practice and there's nothing wrong with it.
It really only makes sense on systems with more than one core. With just a single core (and no hyper-threading), the mutex would never do anything anyway. If the core is running code that acquires a mutex that interrupt code can acquire, interrupts (or the subset of them that matter) are disabled anyway. So with just one core, the mutex would never see any contention.
However, with multiple cores, it's common to use mutexes to protect structures that communicate between interrupt and non-interrupt code. So long as you know what you're doing, and you have to if you're going to write interrupt handlers, there's nothing wrong with it.
How the mutex blocks and unblocks is heavily implementation dependent. It can put the CPU to sleep and be woken by an inter-process interrupt. It can spin the CPU in some CPU-specific way.
Note that a totally unrelated concept that is often confused with this is using user-space mutexes in user-space signal handlers. That's a completely different question.
I have some HW resource which triggers an interrupt when finished.
I want to implement a function activateHw() which would return only when the action is finished (meaning, when the interrupt is triggered) but I don't want it to waste CPU time (no interrupt polling). Basically, what I want is:
bool activateHw() {
trigger_hw();
sleep_until_interrupt_arrived();
return true;
}
How can I achieve that in Linux?
If you are writing a kernel module, you can use request_irq() to set up a callback / interrupt handler. Your interrupt handler can write the data to character device. Your blocking function just needs to call the poll syscall on the character device. It will block and sleep till data is available.
Have a look at gpio.txt , specifically how gpio pins can be setup to write to /sys/.../gpioxx/value by configuring what edge should trigger the interupt. This can be used for userspace code as well if the programme is not too time critical.
here is a SO question that seemed related.
I suggest you use a wait queue. In the interrupt handler you wake up the waiting thread. That way you will not waste resources (eg. using a spin lock).
Take a look at this tutorial.
Take a look at the linux source, an example usage would be /drivers/char/hpet.c or drivers/char/rtc.c.
I have scoured the web, but haven't found a convincing answer to a couple of related questions I have, with regard to the "request_threaded_irq" feature.
Question1:
Firstly, I was reading this article, regarding threaded IRQ's:
http://lwn.net/Articles/302043/
and there is this one line that isn't clear to me:
"Converting an interrupt to threaded makes only sense when the handler
code takes advantage of it by integrating tasklet/softirq
functionality and simplifying the locking."
I understand had we gone ahead with a "traditional", top half/bottom half approach, we would have needed either spin-locks or disable local IRQ to meddle with shared data. But, what I don't understand is, how would threaded interrupts simplify the need for locking by integrating tasklet/softirq functionality.
Question2:
Secondly, what advantage (if any), does a request_threaded_handler approach have over a work_queue based bottom half approach ? In both cases it seems, as though the "work" is deferred to a dedicated thread. So, what is the difference ?
Question3:
Lastly, in the following prototype:
int request_threaded_irq(unsigned int irq, irq_handler_t handler, irq_handler_t thread_fn, unsigned long irqflags, const char *devname, void *dev_id)
Is it possible that the "handler" part of the IRQ is continuously triggered by the relevant IRQ (say a UART receving characters at a high rate), even while the "thread_fn"(writing rx'd bytes to a circular buffer) part of the interrupt handler is busy processing IRQ's from previous wakeups ? So, wouldn't the handler be trying to "wakeup" an already running "thread_fn" ? How would the running irq thread_fn behave in that case ?
I would really appreciate if someone can help me understand this.
Thanks,
vj
For Question 2,
An IRQ thread on creation is setup with a higher priority, unlike workqueues.
In kernel/irq/manage.c, you'll see some code like the following for creation of kernel threads for threaded IRQs:
static const struct sched_param param = {
.sched_priority = MAX_USER_RT_PRIO/2,
};
t = kthread_create(irq_thread, new, "irq/%d-%s", irq,
new->name);
if (IS_ERR(t)) {
ret = PTR_ERR(t);
goto out_mput;
}
sched_setscheduler_nocheck(t, SCHED_FIFO, ¶m);
Here you can see, the scheduling policy of the kernel thread is set to an RT one (SCHED_FIFO) and the priority of the thread is set to MAX_USER_RT_PRIO/2 which is higher than regular processes.
For Question 3,
The situation you described can also occur with normal interrupts. Typically in the kernel, interrupts are disabled while an ISR executes. During the execution of the ISR, characters can keep filling the device's buffer and the device can and must continue to assert an interrupt even while interrupts are disabled.
It is the job of the device to make sure the IRQ line is kept asserted till all the characters are read and any processing is complete by the ISR. It is also important that the interrupt is level triggered, or depending on the design be latched by the interrupt controller.
Lastly, the device/peripheral should have an adequately sized FIFO so that characters delivered at a high rate are not lost by a slow ISR. The ISR should also be designed to read as many characters as possible when it executes.
Generally speaking what I've seen is, a controller would have a FIFO of a certain size X, and when the FIFO is filled X/2, it would fire an interrupt that would cause the ISR to grab as much data as possible. The ISR reads as much as possible and then clears the interrupt. Meanwhile, if the FIFO is still X/2, the device would keep the interrupt line asserted causing the ISR to execute again.
Previously, the bottom-half was not a task and still could not block. The only difference was that interrupts were disabled. The tasklet or softirq allow different inter-locks between the driver's ISR thread and the user API (ioctl(), read(), and write()).
I think the work queue is near equivalent. However, the tasklet/ksoftirq has a high priority and is used by all ISR based functionality on that processor. This may give better scheduling opportunities. Also, there is less for the driver to manage; everything is already built-in to the kernel's ISR handler code.
You must handle this. Typically ping-pong buffers can be used or a kfifo like you suggest. The handler should be greedy and get all data from the UART before returning IRQ_WAKE_THREAD.
For Question no 3,
when an threadedirq is activated the corresponding interrupt line is masked / disabled. when the threadedirq runs and completes it enables it towards the end of the it. hence there won't be any interrupt firing while the respective threadedirq is running.
The original work of converting "hard"/"soft" handlers to threaded handlers was done by Thomas Gleixner & team when building the PREEMPT_RT Linux (aka Linux-as-an-RTOS) project (it's not part of mainline).
To truly have Linux run as an RTOS, we cannot tolerate a situation where an interrupt handler interrupts the most critical rt (app) thread; but how can we ensure that the app thread even overrides an interrupt?? By making it (the interrupt) threaded, schedulable (SCHED_FIFO) and have a lower priority than the app thread (interrupt threads rtprio defaults to 50). So a "rt" SCHED_FIFO app thread with a rtprio of 60 would be able to "preempt" (closely enough that it works) even an interrupt thread. That should answer your Qs. 2.
Wrt to Qs 3:
As others have said, your code must handle this situation.
Having said that, pl note that a key point to using a threaded handler is so that you can do work that (possibly) blocks (sleeps). If your "bottom half" work is guaranteed to be non-blocking and must be fast, pl use the traditional style 'top-half/bh' handlers.
How can we do that? Simple: don't use request_threaded_irq() just call request_irq() - the comment in the code clearly says (wrt 3rd parameter):
* #thread_fn: Function called from the irq handler thread
* If NULL, no irq thread is created"
Alternatively, you can pass the IRQF_NO_THREAD flag to request_irq.
(BTW, a quick check with cscope on the 3.14.23 kernel source tree shows that request_irq() is called 1502 times [giving us non-threaded interrupt handling], and request_threaded_irq() [threaded interrupts] is explicitly called 204 times).
I'm writing a simple key logger as a means to understanding interrupts and IO ports for linux drivers. Now from what I've read, I need to register interrupt handlers via ...
int request_irq(unsigned int irq, irqreturn_t (*handler)(int, void *, struct pt_regs *), unsigned long flags, const char *dev_name, void *dev_id);
However from what I also read, if I call request_irq during module_init() I will hog the interrupt. My problem is that I want to share the interrupt with other resources but I'm not sure how I can call this interrupt. I wish I could just call the function with module_init() and set my flag to SA_SHIRQ. I am told the best practice is to call during open file which allow for me to set the interrupt, but I don't want to rely on writing a user space program calling open "my_dev" just so that I can run the interrupt handler.
Thanks
Don't worry. request_irq doesn't "hog" the interrupt. It puts the handler on a list of callbacks and calls all handlers (for that IRQ) whenever that interrupt is signaled.
If what you do during the interrupt is heavy lifting, you might want to enable processing IRQs only when the driver is open to minimize system disruption when the driver is not in use. To do that, implement the driver open and close calls, and keep a reference count: only when the ref count > 0 should the ISR be registered
Get it working first, and worry about best practice later. Requesting the IRQ in module_init isn't a big deal - so long as all the resources the interrupt handler needs are available for it to be called immediately...
I'm trying to implement a simulation of a microcontroller. This simulation is not meant to do a clock cycle precise representation of one specific microcontroller but check the general correctness of the code.
I thought of having a "main thread" executing normal code and a second thread executing ISR code. Whenever an ISR needs to be run, the ISR thread suspends the "main thread".
Of course, I want to have a feature to block interrupts.
I thought of solving this with a mutex that the ISR thread holds whenever it executes ISR code while the main thread holds it as long as "interrupts are blocked".
A POR (power on reset) can then be implemented by not only suspending but killing the main thread (and starting a new one executing the POR function).
The windows API provides the necessary functions.
But it seems to be impossible to do the above with posix threads (on linux).
I don't want to change the actual hardware independent microcontroller code. So inserting anything to check for pending interrupts is not an option.
Receiving interrupts at non well behaved points is desirable, as this also happens on microcontrollers (unless you block interrupts).
Is there a way to suspend another thread on linux? (Debuggers must use that option somehow, I think.)
Please, don't tell me this is a bad idea. I know that is true in most circumstances. But the main code does not use standard libs or lock/mutexes/semaphores.
SIGSTOP does not work - it always stops the entire process.
Instead you can use some other signals, say SIGUSR1 for suspending and SIGUSR2 for resuming:
// at process start call init_pthread_suspending to install the handlers
// to suspend a thread use pthread_kill(thread_id, SUSPEND_SIG)
// to resume a thread use pthread_kill(thread_id, RESUME_SIG)
#include <signal.h>
#define RESUME_SIG SIGUSR2
#define SUSPEND_SIG SIGUSR1
static sigset_t wait_mask;
static __thread int suspended; // per-thread flag
void resume_handler(int sig)
{
suspended = 0;
}
void suspend_handler(int sig)
{
if (suspended) return;
suspended = 1;
do sigsuspend(&wait_mask); while (suspended);
}
void init_pthread_suspending()
{
struct sigaction sa;
sigfillset(&wait_mask);
sigdelset(&wait_mask, SUSPEND_SIG)
sigdelset(&wait_mask, RESUME_SIG);
sigfillset(&sa.sa_mask);
sa.sa_flags = 0;
sa.sa_handler = resume_handler;
sigaction(RESUME_SIG, &sa, NULL);
sa.sa_handler = suspend_handler;
sigaction(SUSPEND_SIG, &sa, NULL);
}
I am very annoyed by replies like "you should not suspend another thread, that is bad".
Guys why do you assume others are idiots and don't know what they are doing? Imagine that others, too, have heard about deadlocking and still, in full consciousness, want to suspend other threads.
If you don't have a real answer to their question why do you waste your and the readers' time.
An yes, IMO pthreads are very short-sighted api, a disgrace for POSIX.
The Hotspot JAVA VM uses SIGUSR2 to implement suspend/resume for JAVA threads on linux.
A procedure based on on a signal handler for SIGUSR2 might be:
Providing a signal handler for SIGUSR2 allows a thread to request a lock
(which has already been acquired by the signal sending thread).
This suspends the thread.
As soon as the suspending thread releases the lock, the signal handler can
(and will?) get the lock. The signal handler releases the lock immediately and
leaves the signal handler.
This resumes the thread.
It will probably be necessary to introduce a control variable to make sure that the main thread is in the signal handler before starting the actual processing of the ISR.
(The details depend on whether the signal handler is called synchronously or asynchronously.)
I don't know, if this is exactly how it is done in the Java VM, but I think the above procedure does what I need.
Somehow I think sending the other thread SIGSTOP works.
However, you are far better off writing some thread communication involving senaogires.mutexes and global variables.
You see, if you suspend the other thread in malloc() and you call malloc() -> deadlock.
Did I mention that lots of C standard library functions, let alone other libraries you use, will call malloc() behind your back?
EDIT:
Hmmm, no standard library code. Maybe use setjmp/longjump() from signal handler to simulate the POR and a signal handier to simulate interrupt.
TO THOSE WHO KEEP DOWNVOTING THIS: The answer was accepted for the contents after EDIT, which is a specific scenario that cannot be used in any other scenario.
Solaris has the thr_suspend(3C) call that would do what you want. Is switching to Solaris a possibility?
Other than that, you're probably going to have to do some gymnastics with mutexes and/or semaphores. The problem is that you'll only suspend when you check the mutex, which will probably be at a well-behaved point. Depending on what you're actually trying to accomplish, this might now be desirable.
It makes more sense to have the main thread execute the ISRs - because that's how the real controller works (presumably). Just have it check after each emulated instruction if there is both an interrupt pending, and interrupts are currently enabled - if so, emulate a call to the ISR.
The second thread is still used - but it just listens for the conditions which cause an interrupt, and mark the relevant interrupt as pending (for the other thread to later pick up).