kthread_work and kthread_worker functionality - multithreading

I need to use the kthread functions for a work processor and the distinction between the two is not very clear to me. This is my understanding of the fields.
struct kthread_worker {
spinlock_t lock; // lock to update the work queue
struct list_head work_list; // list kthread_work items
struct task_struct *task; // handle for thread
struct kthread_work *current_work; // ?
};
struct kthread_work {
struct list_head node; // list of threads?
kthread_work_func_t func; // func to execute?
struct kthread_worker *worker; // worker associated with this work item
};
My questions are:
Any clarification on the unclear fields.
kthread_work_func_t is a func ptr expecting an argument of kthread_work. How does that work? It should point to the function you want the thread to execute, right?

kthread_worker is a worker, which can execute works (kthread_work). Work can be added to worker at any time. Worker executes works one by one. If no work is currently available, worker waits.
kthread_work_func_t is a func ptr expecting an argument of kthread_work. How does that work? It should point to the function you want the thread to execute, right?
Yes, it is just function you want to execute as work.
If only one work uses this function(e.g., this is some sort of garbage collector), function may simply ignore its argument.
If you want to have several works, which uses same functionality but with different parameters, you may embed kthread_work structure into your structure, which contain these parameters:
struct my_work
{
struct kthread_work work; //Base work object
int i; // Your parameter
}
// Parametrized work function
void my_func(struct kthread_work* work)
{
// Extract actual work object
struct my_work* my_work = container_of(work, struct my_work, work);
// Parameter for the work
int i = my_work->i;
// Do something
...
// Free memory used for work object
kfree(my_work);
}
// Helper for add your work with given parameter
void add_my_work(struct kthread_worker* worker, int i)
{
// Allocate your work object on the heap
struct my_work* my_work = kmalloc(sizeof(struct my_work), GFP_KERNEL);
// Initialize base work structure
init_kthread_work(&my_work->work, &my_func);
// Store parameter
work->i = i;
queue_kthread_work(worker, &my_work->work);
}
Any clarification on the unclear fields.
As you can see from previous example, knowing fields of struct kthread_worker and struct kthread_work is rarely useful for just using it. But actually semantic is simple:
struct kthread_worker {
spinlock_t lock; // lock to update the work queue
struct list_head work_list; // list kthread_work items
struct task_struct *task; // handle for thread
struct kthread_work *current_work; // (+) currently executed work object
};
struct kthread_work {
struct list_head node; // (+) element in the kthread_worker.work_list
kthread_work_func_t func; // func to execute
struct kthread_worker *worker; // worker associated with this work item
};

Related

Accessing disjoint entries in global HashMap for lifetime of thread in Rust

my current project requires recording some information for various events that happen during the execution of a thread. These events are saved in a global struct index by the thread id:
RECORDER1: HashMap<ThreadId, Vec<Entry>> = HashMap::new();
Every thread appends new Entry to its vector. Therefore, threads access "disjoint" vectors. Rust requires synchronization primitives to make the above work of course. So the real implementation looks like:
struct Entry {
// ... not important.
}
#[derive(Clone, Eq, PartialEq, Hash)]
struct ThreadId;
// lazy_static necessary to initialize this data structure.
lazy_static! {
/// Global data structure. Threads access disjoint entries based on their unique thread id.
/// "Outer" mutex necessary as lazy_static requires sync (so cannot use RefCell).
static ref RECORDER2: Mutex<HashMap<ThreadId, Vec<Entry>>> = Mutex::new(HashMap::new());
}
This works, but all threads contend on the same global lock. It would be nice if a thread could "borrow" its respective vector for the lifetime of the thread so it could write all the entries it needs without needing to lock every time (I understand the outer lock is necessary for ensuring threads don't insert into the HashMap at the same time).
We can do this by adding an Arc and some more interior mutability via a Mutex for the values in the HashMap:
lazy_static! {
static ref RECORDER: Mutex<HashMap<ThreadId, Arc<Mutex<Vec<Entry>>>>> = Mutex::new(HashMap::new());
}
Now we can "check out" our entry when a thread is spawned:
fn local_borrow() {
std::thread::spawn(|| {
let mut recorder = RECORDER.lock().expect("Unable to acquire outer mutex lock.");
let my_thread_id: ThreadId = ThreadId {}; // Get thread id...
// Insert entry in hashmap for our thread.
// Omit logic to check if key-value pair already existed (it shouldn't).
recorder.insert(my_thread_id.clone(), Arc::new(Mutex::new(Vec::new())));
// Get "reference" to vector
let local_entries: Arc<Mutex<Vec<Entry>>> = recorder
.get(&my_thread_id)
.unwrap() // We just inserted this entry, so unwrap.
.clone(); // Clone on the Arc to acquire a "copy".
// Lock once, use multiple times.
let mut local_entries: MutexGuard<_> = local_entries.lock().unwrap();
local_entries.push(Entry {});
local_entries.push(Entry {});
});
}
This works and is what I want. However, due to API constraints I have to access the MutexGuard from widely different places across the code without the ability to pass the MutexGuard as an argument to functions. So instead I use a thread local variable:
thread_local! {
/// This variable is initialized lazily. Due to API constraints, we use this thread_local! to
/// "pass" LOCAL_ENTRIES around.
static LOCAL_ENTRIES: Arc<Mutex<Vec<Entry>>> = {
let mut recorder = RECORDER.lock().expect("Unable to acquire outer mutex lock.");
let my_thread_id: ThreadId = ThreadId {}; // Get thread id...
// Omit logic to check if key-value pair already existed (it shouldn't).
recorder.insert(my_thread_id.clone(), Arc::new(Mutex::new(Vec::new())));
// Get "reference" to vector
recorder
.get(&my_thread_id)
.unwrap() // We just inserted this entry, so unwrap.
.clone() // Clone on the Arc to acquire a "copy".
}
}
I cannot make LOCAL_ENTRIES: MutexGuard<_> since thread_local! requires a 'static lifetime. So currently I have to .lock() every time I want to access the thread-local variable:
fn main() {
std::thread::spawn(|| {
// Record important message.
LOCAL_ENTRIES.with(|entries| {
// We have to lock every time we want to write to LOCAL_ENTRIES. It would be nice
// to lock once and hold on to the MutexGuard for the lifetime of the thread, but
// this is not possible to due the lifetime on the MutextGuard.
let mut entries = entries.lock().expect("Unable to acquire lock");
entries.push(Entry {});
});
});
}
Sorry for all the code and explanation but I'm really stuck and wanted to show why it doesn't work and what I'm trying to get working. How can one get around this in Rust?
Or am I getting hung up on cost of the mutex locking? For any Arc<Mutex<Vec<Entry>>>, the lock will always be unlocked so the cost of doing the atomic locking will be tiny?
Thanks for any thoughts. Here is the complete example in Rust Playground.

How can I create "C Blocks" when using FFI?

I'm working with the CoreFoundation framework on OS X, but I don't know how to map this function in Rust:
void CFRunLoopPerformBlock(CFRunLoopRef fl, CFTypeRef mode, void (^block)(void));
The last parameter is void(^block)(void) — how can I create arguments of this type?
Short, probably helpful answer: there's the block crate, which looks like it might do the job.
Short, unhelpful answer: Insofar as I am aware, Rust doesn't have any support for Apple's block extension. There is no equivalent Rust type, assuming you want to call an API that expects a block.
Longer, marginally less unhelpful answer: From what I can gather from some Clang documentation on the Apple Block ABI, void(^)(void) would be the same size as a regular pointer.
As such, my advice is as follows: treat blocks as opaque, pointer-sized values. To invoke one, write a function in C which calls it for you.
The following is untested (I don't have a Mac), but should at least get you going in the right direction. Also, I'm marking this community wiki so anyone who can test it can fix it if need-be.
In Rust:
// These are the "raw" representations involved. I'm not using std::raw
// because that's not yet stabilised.
#[deriving(Copy, Clone)]
struct AppleBlock(*const ());
#[deriving(Copy, Clone)]
struct RustClosure(*const(), *const());
// Functions that we need to be written in C:
extern "C" {
fn rust_closure_to_block(closure_blob: RustClosure) -> AppleBlock;
fn block_release(block_blob: AppleBlock);
}
// The function that the C code will need. Note that this is *specific* to
// FnMut() closures. If you wanted to generalise this, you could write a
// generic version and pass a pointer to that to `rust_closure_to_block`.
extern "C" fn call_rust_closure(closure_blob: RustClosure) {
let closure_ref: &FnMut() = unsafe { mem::transmute(closure_blob) };
closure_ref();
}
// This is what you call in order to *temporarily* turn a closure into a
// block. So, you'd use it as:
//
// with_closure_as_block(
// || do_stuff(),
// |block| CFRunLoopPerformBlock(fl, mode, block)
// );
fn with_closure_as_block<C, B, R>(closure: C, body: B) -> R
where C: FnMut(), B: FnOnce(block_blob) -> R {
let closure_ref: &FnMut() = &closure;
let closure_blob: RustClosure = unsafe { mem::transmute(closure_ref) };
let block_blob = unsafe { rust_closure_to_block(closure_blob) };
let r = body(block_blob);
unsafe { block_release(block_blob) };
r
}
In C:
typedef struct AppleBlock {
void *ptr;
} AppleBlock;
typedef struct RustClosure {
void *ptr;
void *vt;
} RustClosure;
void call_rust_closure(RustClosure closure_blob);
AppleBlock rust_closure_to_block(RustClosure closure_blob) {
return (AppleBlock)Block_copy(^() {
call_rust_closure(closure_blob);
});
}
// I'm not using Block_release directly because I don't know if or how
// blocks change name mangling or calling. You might be able to just
// use Block_release directly from Rust.
void block_release(AppleBlock block) {
Block_release((void (^)(void))block);
}

Kernel module, multiple high resolution timers

I want to implement multiple hrtimers, but I'm not sure how to use all of them with same callback function. For example I have array of type my_struct where one of the field is a struct hrtimer.
When I enter the callback function how to determine which element of the array is calling it?
Use the container_of macro:
struct my_struct {
int my_something;
struct hrtimer my_timer;
...
};
enum hrtimer_restart my_callback(struct hrtimer *hrtimer)
{
struct my_struct my = container_of(hrtimer, struct my_struct, my_timer);
my->my_something = 42;
...
}

Is it possible to find the corresponding task_struct from sched_entity?

I know if we have task_struct, surly we can get the contained sched_entity because it's one field in the task struct. But can we get the pointer to the task_struct given the shed_entity? Following is the sched_entity structure:
struct sched_entity {
struct load_weight load; /* for load-balancing */
struct rb_node run_node;
struct list_head group_node;
unsigned int on_rq;
u64 exec_start;
u64 sum_exec_runtime;
u64 vruntime;
u64 prev_sum_exec_runtime;
u64 nr_migrations;
#ifdef CONFIG_SCHEDSTATS
struct sched_statistics statistics;
#endif
#ifdef CONFIG_FAIR_GROUP_SCHED
struct sched_entity *parent;
/* rq on which this entity is (to be) queued: */
struct cfs_rq *cfs_rq;
/* rq "owned" by this entity/group: */
struct cfs_rq *my_q;
#endif
};
It seems that there is no place where I can get the task_struct. My final goal is to get the sched_entity of the task group_leader containing the task with this shed_entity :>
The Linux kernel code provides a standard way to take a pointer to an element contained within a structure, and get back a pointer to the containing structure: the container_of macro, which is used extensively throughout the kernel.
In this case, if you have a struct sched_entity *foo, you can get the enclosing task_struct with:
struct task_struct *task = container_of(foo, struct task_struct, se);
(Obviously this is only safe if you know for sure that the original struct sched_entity * pointer is pointing to a struct sched_entity which is inside a struct task_struct, so be careful...)

Linux kernel rb tree

Is it valid to do the following
struct foo {
int data;
struct rb_node node
};
struct rb_root root;
/* Filling tree with kalloc'ed foo nodes */
struct rb_node *node=rb_first(&root);
while (node)
{
struct rb_node *next=rb_next(node);
kfree(node);
node = next;
}
root=RB_ROOT;
In fact, I just want do foreach and clear at same time with linear time.
Explored rb_next implementation. It returns parent before right children.
So, It is impossible to clear list this way.

Resources