Unbound workqueue's kthreads CPU affinity - linux

Is there a way to set CPU affinity for unbound workqueue's kthreads (those that named kthread/uXX:y)? Something like cpu mask for regular workqueues.
Is it a good idea to set it for each kthread using taskset?

Workqueue subsystem exports sysfs attribute for setting cpu affinity for unbound workers.
Code can be found in Workqueue.c:
5040 static ssize_t wq_unbound_cpumask_store(struct device *dev,
5041 struct device_attribute *attr, const char *buf, size_t count)
5042 {
5043 cpumask_var_t cpumask;
5044 int ret;
5045
5046 if (!zalloc_cpumask_var(&cpumask, GFP_KERNEL))
5047 return -ENOMEM;
5048
5049 ret = cpumask_parse(buf, cpumask);
5050 if (!ret)
5051 ret = workqueue_set_unbound_cpumask(cpumask);
5052
5053 free_cpumask_var(cpumask);
5054 return ret ? ret : count;
5055 }
5056
5057 static struct device_attribute wq_sysfs_cpumask_attr =
5058 __ATTR(cpumask, 0644, wq_unbound_cpumask_show,
5059 wq_unbound_cpumask_store)
So any user space application can write to sysfs descriptor to set unbound workqueue cpu mask.
I hope this answers your query.

Related

Linux interrupt notification to user-space

I am using Linux kernel with preempt-rt patch, I did a a simple test to measure time between interrupt and notification of the interrupt to userspace.
The idea is quite simple I have a module, that implements a read:
static ssize_t my_read(struct file *filp, char __user *buf, size_t count,loff_t *f_pos)
{
atomic_set(&intflag, 0);
if (wait_event_interruptible(my_hack, atomic_read(&intflag) != 0))
return -ERESTARTSYS;
do_gettimeofday(&tv2);
trace_printk("wait_event_interruptible% d\n",(int) (tv2.tv_usec - tv1.tv_usec));
return 0;
}
then from the interrupt routine I wake the read:
static irqreturn_t
my_interrupt(int irq, void *dev_id)
{
...
atomic_set(&intflag,1);
do_gettimeofday(&tv1);
wake_up_interruptible(&mm_fec_hack);
return IRQ_HANDLED;
}
The value printed is around 65 and 70 e.g.
myproc-532 [002] ....... 8049.789350: 0xbf0c81f0: wake_up_interruptible 65
My CPU is a 4 core:
model name : ARMv7 Processor rev 10 (v7l) BogoMIPS : 7.54
and frequency is
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq 996000
I think that 65-70 us is pretty much to wakeup a process.
I tried to change irq and process priority with chrt and to relegate the irq to a specific core with smp_affinity and process to the same core or different core with taskset but the results basically are not changing.
What am I doing wrong?

How to accelerate Reading/Writing Attribute Data?

I used File attributes in the sys filesystem which allow userspace to communicate with the driver using SysFS. The problem is that I have much data to be transfered so I lost much time. This is a very slow process!
In the kennel module I have this part of the code:
static DEVICE_ATTR(writeFile,S_IWUSR,NULL,writeFunction);
static DEVICE_ATTR(resultFile,S_IRUSR, readFunction, NULL);
static ssize_t writeFunction(struct device *dev, struct device_attribute *attr,
const char * buf, size_t count) {
if(count >0){
resp_ready = 1;
sscanf(buf,"%2x,",&src_vect[0]);
for (i=0;i<80;i++)
{
sscanf(buf+3+12*i,"%2x,%2x,%2x,%2x,",&src_vect[4*i+1],&src_vect[4*i+2],&src_vect[4*i+3],&src_vect[4*i+4]);
}
.
.
.
}
static ssize_t readFunction(struct device *dev, struct device_attribute *attr, char *buf) {
//write the result to buf
for (i=0;i<6;i++)
{
sprintf(buf+72*i,"%8x,%8x,%8x,%8x,%8x,%8x,%8x,%8x,",dst_f[8*i],dst_f[8*i+1],dst_f[8*i+2],dst_f[8*i+3],dst_f[8*i+4],dst_f[8*i+5],dst_f[8*i+6],dst_f[8*i+7]);
}
.
.
.
}
How can I reduce data transfer time from/to user/kennel space?
Is there any method faster than this one?

Writing interrupt handler for external interrupt (IRQ 0 - 5) of ARM processor

I am trying to write an interrupt handler for LS1021A external interrupt IRQ5 using NXP QorIQ SDK 2.0. I have created a kernel module to handle the interrupt but the reset_irq function is always returning -22 (Invalid argument).
This is my code:
static unsigned int irqNum;
static irq_handler_t irq5_handler(unsigned int irq, void *dev_id, struct pt_regs *regs){
printk(KERN_INFO "Interrupt received\n");
return (irq_handler_t) IRQ_HANDLED;
}
static int __init irq_init(void){
int result = 0;
irqNum = 201; // IRQ number of IRQ5
result = request_irq(irqNum, (irq_handler_t) irq5_handler, IRQF_TRIGGER_RISING, "irq5_handler", NULL);
printk(KERN_INFO "request_irq result is %d\n", result);
return result;
}
module_init(irq_init);
Is creating a kernel module the right approach to handle the interrupt?
Do I need to perform any actions like initializing the processor pin before requesting the interrupt (request_irq)?

Shared memory across processes on Linux/x86_64

I have a few questions on using shared memory with processes. I looked at several previous posts and couldn't glean the answers precisely enough. Thanks in advance for your help.
I'm using shm_open + mmap like below. This code works as intended with parent and child alternating to increment g_shared->count (the synchronization is not portable; it works only for certain memory models, but good enough for my case for now). However, when I change MAP_SHARED to MAP_ANONYMOUS | MAP_SHARED, the memory isn't shared and the program hangs since the 'flag' doesn't get flipped. Removing the flag confirms what's happening with each process counting from 0 to 10 (implying that each has its own copy of the structure and hence the 'count' field). Is this the expected behavior? I don't want the memory to be backed by a file; I really want to emulate what might happen if these were threads instead of processes (they need to be processes for other reasons).
Do I really need shm_open? Since the processes belong to the same hierarchy, can I just use mmap alone instead? I understand this would be fairly straightforward if there wasn't an 'exec,' but how do I get it to work when there is an 'exec' following the 'fork?'
I'm using kernel version 3.2.0-23 on x86_64 (Intel i7-2600). For this implementation, does mmap give the same behavior (correctness as well as performance) as shared memory with pthreads sharing the same global object? For example, does the MMU map the segment with 'cacheable' MTRR/TLB attributes?
Is the cleanup_shared() code correct? Is it leaking any memory? How could I check? For example, is there an equivalent of System V's 'ipcs?'
thanks,
/Doobs
shmem.h:
#ifndef __SHMEM_H__
#define __SHMEM_H__
//includes
#define LEN 1000
#define ITERS 10
#define SHM_FNAME "/myshm"
typedef struct shmem_obj {
int count;
char buff[LEN];
volatile int flag;
} shmem_t;
extern shmem_t* g_shared;
extern char proc_name[100];
extern int fd;
void cleanup_shared() {
munmap(g_shared, sizeof(shmem_t));
close(fd);
shm_unlink(SHM_FNAME);
}
static inline
void init_shared() {
int oflag;
if (!strcmp(proc_name, "parent")) {
oflag = O_CREAT | O_RDWR;
} else {
oflag = O_RDWR;
}
fd = shm_open(SHM_FNAME, oflag, (S_IREAD | S_IWRITE));
if (fd == -1) {
perror("shm_open");
exit(EXIT_FAILURE);
}
if (ftruncate(fd, sizeof(shmem_t)) == -1) {
perror("ftruncate");
shm_unlink(SHM_FNAME);
exit(EXIT_FAILURE);
}
g_shared = mmap(NULL, sizeof(shmem_t),
(PROT_WRITE | PROT_READ),
MAP_SHARED, fd, 0);
if (g_shared == MAP_FAILED) {
perror("mmap");
cleanup_shared();
exit(EXIT_FAILURE);
}
}
static inline
void proc_write(const char* s) {
fprintf(stderr, "[%s] %s\n", proc_name, s);
}
#endif // __SHMEM_H__
shmem1.c (parent process):
#include "shmem.h"
int fd;
shmem_t* g_shared;
char proc_name[100];
void work() {
int i;
for (i = 0; i &lt ITERS; ++i) {
while (g_shared->flag);
++g_shared->count;
sprintf(g_shared->buff, "%s: %d", proc_name, g_shared->count);
proc_write(g_shared->buff);
g_shared->flag = !g_shared->flag;
}
}
int main(int argc, char* argv[], char* envp[]) {
int status, child;
strcpy(proc_name, "parent");
init_shared(argv);
fprintf(stderr, "Map address is: %p\n", g_shared);
if (child = fork()) {
work();
waitpid(child, &status, 0);
cleanup_shared();
fprintf(stderr, "Parent finished!\n");
} else { /* child executes shmem2 */
execvpe("./shmem2", argv + 2, envp);
}
}
shmem2.c (child process):
#include "shmem.h"
int fd;
shmem_t* g_shared;
char proc_name[100];
void work() {
int i;
for (i = 0; i &lt ITERS; ++i) {
while (!g_shared->flag);
++g_shared->count;
sprintf(g_shared->buff, "%s: %d", proc_name, g_shared->count);
proc_write(g_shared->buff);
g_shared->flag = !g_shared->flag;
}
}
int main(int argc, char* argv[], char* envp[]) {
int status;
strcpy(proc_name, "child");
init_shared(argv);
fprintf(stderr, "Map address is: %p\n", g_shared);
work();
cleanup_shared();
return 0;
}
Passing MAP_ANONYMOUS causes the kernel to ignore your file descriptor argument and give you a private mapping instead. That's not what you want.
Yes, you can create an anonymous shared mapping in a parent process, fork, and have the child process inherit the mapping, sharing the memory with the parent and any other children. That obvoiusly doesn't survive an exec() though.
I don't understand this question; pthreads doesn't allocate memory. The cacheability will depend on the file descriptor you mapped. If it's a disk file or anonymous mapping, then it's cacheable memory. If it's a video framebuffer device, it's probably not.
That's the right way to call munmap(), but I didn't verify the logic beyond that. All processes need to unmap, only one should call unlink.
2b) as a middle-ground of a sort, it is possible to call:
int const shm_fd = shm_open(fn,...);
shm_unlink(fn);
in a parent process, and then pass fd to a child process created by fork()/execve() via argp or envp. since open file descriptors of this type will survive the fork()/execve(), you can mmap the fd in both the parent process and any dervied processes. here's a more complete code example copied and simplified/sanitized from code i ran successfully under Ubuntu 12.04 / linux kernel 3.13 / glibc 2.15:
int create_shm_fd( void ) {
int oflags = O_RDWR | O_CREAT | O_TRUNC;
string const fn = "/some_shm_fn_maybe_with_pid";
int fd;
neg_one_fail( fd = shm_open( fn.c_str(), oflags, S_IRUSR | S_IWUSR ), "shm_open" );
if( fd == -1 ) { rt_err( strprintf( "shm_open() failed with errno=%s", str(errno).c_str() ) ); }
// for now, we'll just pass the open fd to our child process, so
// we don't need the file/name/link anymore, and by unlinking it
// here we can try to minimize the chance / amount of OS-level shm
// leakage.
neg_one_fail( shm_unlink( fn.c_str() ), "shm_unlink" );
// by default, the fd returned from shm_open() has FD_CLOEXEC
// set. it seems okay to remove it so that it will stay open
// across execve.
int fd_flags = 0;
neg_one_fail( fd_flags = fcntl( fd, F_GETFD ), "fcntl" );
fd_flags &= ~FD_CLOEXEC;
neg_one_fail( fcntl( fd, F_SETFD, fd_flags ), "fcntl" );
// resize the shm segment for later mapping via mmap()
neg_one_fail( ftruncate( fd, 1024*1024*4 ), "ftruncate" );
return fd;
}
it's not 100% clear to me if it's okay spec-wise to remove the FD_CLOEXEC and/or assume that after doing so the fd really will survive the exec. the man page for exec is unclear; it says: "POSIX shared memory regions are unmapped", but to me that's redundant with the general comments earlier that mapping are not preserved, and doesn't say that shm_open()'d fd will be closed. any of course there's the fact that, as i mentioned, the code does seem to work in at least one case.
the reason i might use this approach is that it would seem to reduce the chance of leaking the shared memory segment / filename, and it makes it clear that i don't need persistence of the memory segment.

how can I determine a network device speed from a linux kernel module

I have a linux kernel module that needs to find the speed of a given network interface (i.e. "eth0"). For linux 2.6.31 how would I find the speed (configured/negotiated)?
Every network driver has a "ethtool" implementation for such features. But you probably need a generic function that can give you the speed for a generic netdev struct. You can have a look at net/core/net-sysfs.c and see how it implements the /sys/class/net interface. For example :
static ssize_t show_speed(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct net_device *netdev = to_net_dev(dev);
int ret = -EINVAL;
if (!rtnl_trylock())
return restart_syscall();
if (netif_running(netdev) &&
netdev->ethtool_ops &&
netdev->ethtool_ops->get_settings) {
struct ethtool_cmd cmd = { ETHTOOL_GSET };
if (!netdev->ethtool_ops->get_settings(netdev, &cmd))
ret = sprintf(buf, fmt_dec, ethtool_cmd_speed(&cmd));
}
rtnl_unlock();
return ret;
}

Resources