I have a hash-value database with tags and I want to implement a FUSE interface for it. Because values are indexed by their hashes they must be read-only.
Native interface for this database is very simple:
You can download, upload or tag a file.
You can get the set of all defined tags.
You can search for files tagged in accordance to a boolean combination of tags.
FUSE interface semantics are simple:
Database is viewed as a big synthetic directory hierarchy where values are files named by its hash and tags are directories.
cd-ing inside a directory is semantically equivalent to search for a given tag (naming conventions on paths can be used to implement boolean operations).
read-ing a file is semantically equivalent to download (part of) a value (FUSE allows an stateless read so open and close can be no-ops).
Copying/moving an inexistent file into a given path is equivalent to upload and tag it. Copying/moving an existent file into a given path is equivalent to add new tags.
Any other operation throws an error.
This FUSE interface is quite usable and allows you to easily embed a tag file system inside a hierarchical one without the need of external tools like TagSpaces or Evernote.
My problem arises identifying a file copy or move from any other forbidden operation with FUSE interface: there are endless possible combination of operations with equivalent semantics.
What is the most reliable way to identify a file copy or move with FUSE interface?
Hooking rename of a file should be straightforward by implementing rename() fuse call. In this call, you will get path of both old and new location, so that you can check if the file comes from outside or not. That said, this would work only if user space tool renames a file by invoking rename(2) kernel call.
On the other hand, hooking file copy operation would be harder: it can't be done directly as there is no such fuse call - copying happens in user space completely and so it's not directly detectable in kernel space.
You could try to do some heuristics and process incoming fuse operations to detect rename of already stored file (eg. by hashing content of new file and comparing that with already existing files), but I'm not sure how much it makes sense in your case or if it would be actually practical.
Related
I'm writing a read-only FUSE file system that abstracts access to a remote file system. The remote file system requires an user ID, a token and a file ID to produce a file.
If I have mounted my FUSE file system on /mnt/rfs and an program tries to open the file /mnt/rfs/user/token/fileid FUSE will trigger three getAttr invocations, one for /user, one for /user/token and one for /user/token/fileid.
Is there any way to detect from inside the getAttr invocation that this particular call is for the last element in the path? Obviously, if I know how my service works, I can just fake the directories for the first two elements, then download a file to some local tmp storage using the user/token/fileid info. But this is hacky and fragile.
There's no generic way to do it. Handle it on a case-by-case basis.
I am currently developing a simple kernel module that can steal system calls such as open, read, write and replace them with a simple function which logs the files being opened, read, written, into a file and return the original system calls.
My query is, I am able to get the File Descriptor in read and write system calls, but I am not able to understand how to obtain file name using the same.
Currently I am able to access the file structure associated with given FD using following code:
struct file *file;
file = fcheck(fd);
This file structure has two important entities in it, which are of my concern I believe:
f_path
f_inode
Can anybody help me get dentry or inode or the path name associated with this fd using the file structure associated with it?
Is my approach correct? Or do I need to do something different?
I am using Ubuntu 14.04 and my kernel version is 3.19.0-25-generic, for the kernel module development.
.f_inode is actually an inode.
.f_path->dentry is a dentry.
Traversing this dentry via ->d_parent link, until f_path.mnt.mnt_root dentry will be touched, and collecting dentry->d_name components, will construct the file's path, relative to the mount point. This is done, e.g., with d_path, but in more carefull way.
Instead of fcheck(fd), which should be used inside RCU read section, you can also use fget(fd), which should be paired with fput().
The approach is completely incorrect - see http://www.watson.org/~robert/2007woot/
Linux already has a reliable mechanism for doing this thing (audit). If you want to implement it anyway (for fun I presume), you want to place your hooks roughly where audit is doing that. Chances are LSM hooks are in appropriate places, have not checked.
I am trying to traverse a directory structure and open every file in that structure. To traverse, I am using opendir() and readdir(). Since I already have the entity, it seems stupid to build a path and open the file -- that presumably forces Linux to find the directory and file I just traversed.
Level 2 I/O (open, creat, read, write) require a path. Is there any call to either open a filename inside a directory, or open a file given an inode?
You probably should use nftw(3) to recursively traverse a file tree.
Otherwise, in a portable way, construct your directory + filename path using e.g.
snprintf(pathbuf, sizeof(pathbuf), "%s/%s", dirname, filename);
(or perhaps using asprintf(3) but don't forget to later free the result)
And to answer your question about opening a file in a directory, you could use the Linux or POSIX2008 specific openat(2). But I believe that you should really use nftw or construct your path like suggested above. Read also about O_PATH and O_TMPFILE in open(2).
BTW, the kernel has to access several times the directory (actually, the metadata is cached by file system kernel code), just because another process could have written inside it while you are traversing it.
Don't even think of opening a file thru its inode number: this will violate several file system abstractions! (but might be hardly possible by insane and disgusting tricks, e.g. debugfs - and this could probably harm very strongly your filesystem!!).
Remember that files are generally inodes, and can have zero (a process did open then unlink(2) a file while keeping the open file descriptor), one (this is the usual case), or several (e.g. /foo/bar1 and /gee/bar2 could be hard-linked using link(2) ....) file names.
Some file systems (e.g. FAT ...) don't have real inodes. The kernel fakes something in that case.
When I create a new file (eg. touch file.txt) its size equals to 0B.
I'm wondering where are its informations (size, last modify date, owner name, file name) stored.
These informations are stored on hd and are managed by kernel, of course, but I'd love to know something more about them:
Where and how I may get them, using a programming language as C, for example, and how I may change them.
Are these informations changeable, simply using a programming language, or maybe kernel avoids this operations?
I'm working on Unix based file systems, and I'm asking informations especially about this fs.
On unix system, they're traditionally stored in the metadata part of a file representation called an inode
You can fetch this information with the stat() call, see these fields, you can change the owner and permissions with chown() and chmod()
This information is retrievable using the stat() function (and others in its family). Where it's stored is up to the specific file system and for what should be obvious reasons, you cannot change them unless you have raw access to the drive -- and that should be avoided unless you're ok with losing everything on that drive.
The metadata such as owner, size and dates are usually stored in a structure called index-node (inode), which resides in the filesystem's superblock.
Hello I am a newbie to kernel programming. I am writing a small kernel module
that is based on wrapfs template to implement a backup mechanism. This is
purely for learning basis.
I am extending wrapfs so that when a write call is made wrapfs transparently
makes a copy of that file in a separate directory and then write is performed
on the file. But I don't want that I create a copy for every write call.
A naive approach could be I check for existence of file in that directory. But
I think for each call checking this could be a severe penalty.
I could also check for first write call and then store a value for that
specific file using private_data attribute. But that would not be stored on
disk. So I would need to check that again.
I was also thinking of making use of modification time. I could save a
modification time. If the older modification time is before that time then only
a copy is created otherwise I won't do anything. I tried to use inode.i_mtime
for this but it was the modified time even before write was called, also
applications can modify that time.
So I was thinking of storing some value in inode on disk that indicates its
backup has been created or not. Is that possible? Any other suggestions or
approaches are welcome.
You are essentially saying you want to do a Copy-On-Write virtual filesystem layer.
IMO, some of these have been done, and it would be easier to implement these in userland (using libfuse and the fuse module, e.g.). That way, you can be king of your castle and add your metadata in any which way you feel is appriate:
just add (hidden) metadata files to each directory
use extended POSIX attributes (setfattr and friends)
heck, you could even use a sqlite database
If you really insist on doing these things in-kernel, you'll have a lot more work since accessing the metadata from kernel mode is goind to take a lot more effort (you'd most likely want to emulate your own database using memory mapped files so as to minimize the amount of 'userland (style)' work required and to make it relatively easy to get atomicity and reliability right1.
1
On How Everybody Gets File IO Wrong: see also here
You can use atime instead of mtime. In that case setting S_NOATIME flag on the inode prevents it from updating (see touch_atime() function at the inode.c). The only thing you'll need is to mount your filesystem with noatime option.