Why does cat call read() twice when once was enough? - linux

I am new to Linux kernel module. I am learning char driver module based on a web course. I have a very simple module that creates a /dev/chardevexample, and I have a question for my understanding:
When I do echo "hello4" > /dev/chardevexample, I see the write execute exactly once as expected. However, when I do cat /dev/chardevexample, I see the read executed two times.
I see this both in my code and in the course material. All the data was returned in the first read(), so why does cat call it again?
All the things I did so far are as follows:
insmod chardev.ko to load my module
echo "hello4" > /dev/chardevexample. This is the write and I see it happening exactly once in dmesg
cat /dev/chardevexample. This is the read, and dmesg shows it happening twice.
I did strace cat /dev/chardevexample, and I indeed see the function call being called twice for read. There is a write in between as well
read(3, "hello4\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 131072) = 4096
write(1, "hello4\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096hello4) = 4096
read(3, "", 131072)
dmesg after read (cat command)
[909836.517402] DEBUG-device_read: To User hello4 and bytes_to_do 4096 ppos 0 # Read #1
[909836.517428] DEBUG-device_read: Data send to app hello4, nbytes=4096 # Read #1
[909836.519086] DEBUG-device_read: To User and bytes_to_do 0 ppos 4096 # Read #2
[909836.519093] DEBUG-device_read: Data send to app hello4, nbytes=0 # Read #2
Code snippet for read, write and file_operations is attached. Any
guidance would help. I searched extensively and couldn't understand.
Hence the post.
/*!
* #brief Write to device from userspace to kernel space
* #returns Number of bytes written
*/
static ssize_t device_write(struct file *file, //!< File pointer
const char *buf,//!< from for copy_from_user. Takes 'buf' from user space and writes to
//!< kernel space in 'buffer'. Happens on fwrite or write
size_t lbuf, //!< length of buffer
loff_t *ppos) //!< position to write to
{
int nbytes = lbuf - copy_from_user(
buffer + *ppos, /* to */
buf, /* from */
lbuf); /* how many bytes */
*ppos += nbytes;
buffer[strcspn(buffer, "\n")] = 0; // Remove End of line character
pr_info("Recieved data \"%s\" from apps, nbytes=%d\n", buffer, nbytes);
return nbytes;
}
/*!
* #brief Read from device - from kernel space to user space
* #returns Number of bytes read
*/
static ssize_t device_read(struct file *file,//!< File pointer
char *buf, //!< for copy_to_user. buf is 'to' from buffer
size_t lbuf, //!< Length of buffer
loff_t *ppos)//!< Position {
int nbytes;
int maxbytes;
int bytes_to_do;
maxbytes = PAGE_SIZE - *ppos;
if(maxbytes >lbuf)
bytes_to_do = lbuf;
else
bytes_to_do = maxbytes;
buffer[strcspn(buffer, "\n")] = 0; // Remove End of line character
printk("DEBUG-device_read: To User %s and bytes_to_do %d ppos %lld\n", buffer + *ppos, bytes_to_do, *ppos);
nbytes = bytes_to_do - copy_to_user(
buf, /* to */
buffer + *ppos, /* from */
bytes_to_do); /* how many bytes*/
*ppos += nbytes;
pr_info("DEBUG-device_read: Data send to app %s, nbytes=%d\n", buffer, nbytes);
return nbytes;} /* Every Device is like a file - this is device file operation */ static struct file_operations device_fops = {
.owner = THIS_MODULE,
.write = device_write,
.open = device_open,
.read = device_read,};

The Unix convention for indicating end-of-file is to have read return 0 bytes.
In this case, cat asks for 131072 bytes and only receives 4096. This is normal and not to be interpreted as having reached the end of the file. For example, it happens when you read from the keyboard but the user only inputs a small amount of data.
Because cat has not yet seen EOF (i.e. read did not return 0), it continues to issue read calls until it does. This means that if there's any data, you will always see a minimum of two read calls: one (or more) for the data, and one final one that returns 0.

Related

Write from mmapped buffer to `O_DIRECT` output file

I have a device which writes to a video buffer. This buffer is allocated in system memory using CMA and I want to implement streaming write from this buffer to a block device. My application opens video buffer with mmap and I would like to use O_DIRECT write to avoid page cache related overhead. Basically, the pseudo-code of the application looks like this:
f_in = open("/dev/videobuf", O_RDONLY);
f_mmap = mmap(0, BUFFER_SIZE, PROT_READ, MAP_SHARED, f_in, 0);
f_out = open("/dev/sda", O_WRONLY | O_DIRECT);
write(f_out, f_mmap, BLOCK_SIZE);
where BLOCK_SIZE is sector aligned value. f_out is opened without any errors, but write results in EFAULT. I tried to track down this issue and it turned out that mmap implementation in video buffer's driver uses remap_pfn_range(), which sets VM_IO and VM_PFNMAP flags for VMA. The O_DIRECT path in block device drivers checks these flags and returns EFAULT. As far as I understand, O_DIRECT writes need to pin the memory pages, but VMA flags indicate the absence of struct page for underlying memory which causes an error. Am I right here?
And the main question is how to correctly implement O_DIRECT write from mmapped buffer? I have video buffer driver and can modify it appropriately.
I found similar question, but these is no clear answer there.
remap_pfn_range will set your vma as special via pte_mkspecial and add VM_IO/VM_PFNMAP to vma, so you cannot pass the following checks when do Direct I/O.
You say your memory comes from CMA, that's good because cma memory already has struct page support, so you can just use vm_insert_pages as the following steps:
declare cma region from kernel argument or dts
get struct pages from cma:
dma_page = dma_alloc_contiguous(&pdev->dev, size, GFP_KERNEL);
if (!dma_page) {
pr_err("%s %d, dma_alloc_contiguous fail\n", __func__, __LINE__);
return -ENOMEM;
}
nr_pages = DIV_ROUND_UP(size, PAGE_SIZE);
pages = kvmalloc_array(nr_pages, sizeof(*pages), GFP_KERNEL);
for (i = 0; i < nr_pages; i++)
pages[i] = &dma_page[i];
insert pages to vma when mmap
int your_mmap(struct file *file, struct vm_area_struct *vma) {
int ret = 0;
unsigned long temp_nr_pages;
if (vma->vm_end - vma->vm_start > size)
return -EINVAL;
/* duplicitate nr_pages in that vm_insert_pages can change nr_pages */
temp_nr_pages = nr_pages;
ret = vm_insert_pages(vma, vma->vm_start, pages, &temp_nr_pages);
if (ret < 0)
pr_err("%s vm_insert_pages fail, error is %d\n", __func__, ret);
return ret;
}
export dma_alloc_contiguous(the only mm code change, but not so bad).
modified kernel/dma/contiguous.c
## -332,6 +332,7 ## struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
return cma_alloc_aligned(dma_contiguous_default_area, size, gfp); }
+EXPORT_SYMBOL(dma_alloc_contiguous);
/**
* dma_free_contiguous() - release allocated pages

Accessing another process virtual memory in Linux (debugging)

How does gdb access another process virtual memory on Linux? Is it all done via /proc?
How does gdb access another process virtual memory on Linux? Is it all done via /proc?
On Linux for reading memory:
1) If the number of bytes to read is fewer than 3 * sizeof (long) or the filesystem /proc is unavailable or reading from /proc/PID/mem is unsuccessful then ptrace is used with PTRACE_PEEKTEXT to read data.
These are these conditions in the function linux_proc_xfer_partial():
/* Don't bother for one word. */
if (len < 3 * sizeof (long))
return 0;
/* We could keep this file open and cache it - possibly one per
thread. That requires some juggling, but is even faster. */
xsnprintf (filename, sizeof filename, "/proc/%d/mem",
ptid_get_pid (inferior_ptid));
fd = gdb_open_cloexec (filename, O_RDONLY | O_LARGEFILE, 0);
if (fd == -1)
return 0;
2) If the number of bytes to read is greater or equal to 3 * sizeof (long) and /proc is available then pread64 or (lseek() and read() are used:
static LONGEST
linux_proc_xfer_partial (struct target_ops *ops, enum target_object object,
const char *annex, gdb_byte *readbuf,
const gdb_byte *writebuf,
ULONGEST offset, LONGEST len)
{
.....
/* If pread64 is available, use it. It's faster if the kernel
supports it (only one syscall), and it's 64-bit safe even on
32-bit platforms (for instance, SPARC debugging a SPARC64
application). */
#ifdef HAVE_PREAD64
if (pread64 (fd, readbuf, len, offset) != len)
#else
if (lseek (fd, offset, SEEK_SET) == -1 || read (fd, readbuf, len) != len)
#endif
ret = 0;
else
ret = len;
close (fd);
return ret;
}
On Linux for writing memory:
1) ptrace with PTRACE_POKETEXT or PTRACE_POKEDATA is used.
As for your second question:
where can I find information about ... setting hardware watchpoints
gdb, Internals Watchpoint:s http://sourceware.org/gdb/wiki/Internals%20Watchpoints
Reference:
http://linux.die.net/man/2/ptrace
http://www.alexonlinux.com/how-debugger-works

Is the sscanf function in the Linux kernel susceptible to buffer overflow attacks?

From what I understand, a typical buffer overflow attack occurs when an attack overflows a buffer of memory on the stack, thus allowing the attacker to inject malicious code and rewrite the return address on the stack to point to that code.
This is a common concern when using functions (such as sscanf) that blindly copy data from one area to another, checking one for a termination byte:
char str[8]; /* holds up to 8 bytes of data */
char *buf = "lots and lots of foobars"; /* way more than 8 bytes of data */
sscanf(buf, "%s", str); /* buffer overflow occurs here! */
I noticed some sysfs_ops store functions in the Linux kernel are implemented with the Linux kernel's version of the sscanf function:
static char str[8]; /* global string */
static ssize_t my_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t size)
{
sscanf(buf, "%s", str); /* buf holds more than 8 bytes! */
return size;
}
Suppose this store callback function is set to a writable sysfs attribute. Would a malicious user be able to intentionally overflow the buffer via a write call?
Normally, I would expect guards against buffer overflow attacks -- such as limiting the number of bytes read -- but I see none in a good number of functions (for example in drivers/scsi/scsi_sysfs.c).
Does the implementation of the Linux kernel version of sscanf protect against buffer overflow attacks; or is there another reason -- perhaps buffer overflow attacks are impossible given how the Linux kernel works under the hood?
The Linux sscanf() is vulnerable to buffer overflows; inspection of the source shows this. You can use width specifiers to limit the amount a %s is allowed to write. At some point your str must have had copy_from_user() run on it as well. It is possible the user space to pass some garbage pointer to the kernel.
In the version of Linux you cited, the scsi_sysfs.c does have a buffer overflow. The latest version does not. The committed fix should fix the issue you see.
Short answer:
sscanf, when well called, will not cause buffer overflow, especially in sysfs xxx_store() function. (There are a lot sscanf in sysfs XXX_store() examples), because Linux kernel add a '\0' (zero-terminated) byte after the string (buf[len] = 0;) for your XXX_store() function.
Long answer:
Normally, sysfs are defined to have a strict formatted data. Since you expect 8 bytes at most, it's reasonable to limit the size you get like this:
static char str[8]; /* global string */
static ssize_t my_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t size)
{
if (size > 8) {
printk("Error: Input size > 8: too large\n");
return -EINVAL;
}
sscanf(buf, "%s", str); /* buf holds more than 8 bytes! */
return size;
}
(Note: use 9 rather than 8, if you expect a 8-bytes string plus '\n')
(Note that you do reject some inputs such as those with many leading white spaces. However, who would send a string with many leading white spaces? Those who want to break your code, right? If they don't follow your spec, just reject them.)
Note that Linux kernel purposely inserts a '\0' at offset len (i.e. buf[len] = 0;) when the user write len bytes to sysfs purposely for safe sscanf, as said in a comment in kernel 2.6: fs/sysfs/file.c:
static int
fill_write_buffer(struct sysfs_buffer * buffer, const char __user * buf, size_t count)
{
int error;
if (!buffer->page)
buffer->page = (char *)get_zeroed_page(GFP_KERNEL);
if (!buffer->page)
return -ENOMEM;
if (count >= PAGE_SIZE)
count = PAGE_SIZE - 1;
error = copy_from_user(buffer->page,buf,count);
buffer->needs_read_fill = 1;
/* if buf is assumed to contain a string, terminate it by \0,
so e.g. sscanf() can scan the string easily */
buffer->page[count] = 0;
return error ? -EFAULT : count;
}
...
static ssize_t
sysfs_write_file(struct file *file, const char __user *buf, size_t count, loff_t *ppos)
{
struct sysfs_buffer * buffer = file->private_data;
ssize_t len;
mutex_lock(&buffer->mutex);
len = fill_write_buffer(buffer, buf, count);
if (len > 0)
len = flush_write_buffer(file->f_path.dentry, buffer, len);
if (len > 0)
*ppos += len;
mutex_unlock(&buffer->mutex);
return len;
}
Higher kernel version keeps the same logic (though already completely rewritten).

How to read and write with mmap() on a Linux system

I need to make some stream in and out classes using mmap() in Linux. To do so I tried to make some test code that writes some integers to a file, saves it, loads it again and write the data in the file to cout. If that test code works, then It wont be a problem making stream in and out afterwards.
When I first started out I got segment faults and If I did not get that nothing happened, so I googled a little. I found this book http://www.advancedlinuxprogramming.com/alp-folder/alp-ch05-ipc.pdf where around page 107 there is some usefull code. I copy pasted that code and made some small changes and got this code:
int fd;
void* file_memory;
/* Prepare a file large enough to hold an unsigned integer. */
fd = open ("mapTester", O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
//Make the file big enough
lseek (fd, 4 * 10 + 1, SEEK_SET);
write (fd, "", 1);
lseek (fd, 0, SEEK_SET);
/* Create the memory mapping. */
file_memory = mmap (0, 4 * 10, PROT_WRITE, MAP_SHARED, fd, 0);
close (fd);
/* Write a random integer to memory-mapped area. */
sprintf((char*) file_memory, "%d\n", 22);
/* Release the memory (unnecessary because the program exits). */
munmap (file_memory, 4 * 10);
cout << "Mark" << endl;
//Start the part where I read from the file
int integer;
/* Open the file. */
fd = open (argv[1], O_RDWR, S_IRUSR | S_IWUSR);
/* Create the memory mapping. */
file_memory = mmap (0, 4 * 10, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
close (fd);
/* Read the integer, print it out, and double it. */
scanf ((char *) file_memory, "%d", &integer);
printf ("value: %d\n", integer);
sprintf ((char*) file_memory, "%d\n", 2 * integer);
/* Release the memory (unnecessary because the program exits). */
munmap (file_memory, 4 * 10);
But I get a segment fults after the "mark" cout.
Then I replace the "read part" with this:
fd = open("mapTester", O_RDONLY);
int* buffer = (int*) malloc (4*10);
read(fd, buffer, 4 * 10);
for(int i = 0; i < 1; i++)
{
cout << buffer[i] << endl;
}
That is some working code that shows me that the file is empty. I tries out a couple of ways to write to the mapping without any change in result.
So how am I amble to make my code write?
And does my mmap read code seem okay (just in case you can see some obvious flaws)?
I have found some other resources that did not help me yet, but because I am a new user I may only post max 2 links.
You should test the result of mmap. If it gives MAP_FAILED check out errno to find out why.
And you'll better mmap a multiple of pages, often 4K bytes each, and given by sysconf(_SC_PAGESIZE)
You can use stat to find out the size of (and many other numbers about) some given file.
You could use strace on existing Linux programs to learn what syscalls they are doing.
See also this about /proc/ etc.
Your scanf() call should be sscanf(), and the second open should use "mapTester" instead of argv[1] as the filename. When I fix these bugs, your posted program works (printing out 22 and leaving 44 in the file).

Reading boot sector of a FAT32 file system

I am writing a program and I need to access some information in the boot sector about the FAT32 file system that I have mounted.
Here is what I did, fully commented.
int main (void) {
unsigned short *i; //2 byte unsigned integer pointer
char tmp[1024]; //buffer
int fd; //file descriptor from opening device
fd = open("/dev/sdf", O_RDONLY); //open the device file
lseek(fd, 14, SEEK_SET); //set offset to 14 (0x0E), i.e. storing "no. of reserved sectors" of the FAT32 system
read(fd, tmp, 2); //read off 2 bytes, "no. of reserved sectors" is represented by 2 bytes
i = &tmp; //point j at those 2 bytes
printf("*j: %d\n", *j); //print *j out
close(fd); //close device
return 0;
}
The output of *i is 38, which is nonsense. I formatted the file system with mkfs.vfat. I set the "no. of reserved sectors" to 32.
What I have tried:
i = (unsigned short *) &tmp, do a casting, this removes the warning when I compile, but doesn't help
read(fd, tmp, 512), load the whole boot sector (512 bytes) into tmp, then read from the buffer, but doesn't help, result still 38 though.
fiddle with the offset, i.e. change 14 to 13 or 15, in case I get the index wrong. It prints out 9744 for 13 and 512 for 15 respectively, so doesn't work.
I'm not sure whether I'm doing it the right way. Can someone please point me in the right direction?
Thanks in advance,
Felastine.
Try running:
$ dd if=/dev/sdf of=/tmp/x.bin bs=512 count=1
And then:
$ hd /tmp/x.bin
Or
$ od -tx2 /tmp/x.bin
And post the first lines.
Chances are that your fattools are adding 6 extra reserved sectors of their own. And then they substract them before showing the data.
unsigned short *i; //2 byte unsigned integer pointer
char tmp[1024];
[...]
i = &tmp; //point j at those 2 bytes
tmp is char[], &tmp something of order char**.
Think again, you don't want the & here.

Resources