This might be a dumb question, but I'm struggling to find resources that clearly explain how a VFS is different from an NFS. Can they both be used for the same purpose?
Bonus question: Can you watch a VFS with inotify like you can an NFS?
"NFS" is a network filesystem that's been around for decades. Wikipedia has you covered on that front.
"VFS" is a more generic term that simply means "virtual filesystem". Within the context of Linux, it refers to the part of the kernel with which your user-space programs actually interact when they interact with "files". The VFS layer then passes requests to a concrete filesystem driver -- such as NFS, for example, or ext4, or others.
Read more here and here.
A virtual file system (VFS) is an abstraction layer on top of a more concrete file system.The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way, Where as Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystem in 1984, allowing a user on a client computer to access files over a computer network much more like local storage is accessed
A VFS can be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, Mac and Unix file systems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing Where as, NFS like many other protocols, builds on the Open Newtork Computing Remote Procedure Call (ONC RPC) system. The NFS is an open standard defined in Request for comments (RFC), allowing anyone to implement the protocol.
"VFS" is the name given to the entire layer in the kernel situated between the system calls and the filesystem drivers; it is not a filesystem in its own right.
Related
so I’m making this software that encrypts the files on a computer. A friend of mine (we're both students so don't be too hard on us) suggested I use a Virtual File System. I asked why, and what even is that, and they gave me some half assed answer that didn't help
[I don't know if this is important but I'm on a linux environment]
so no worries I went on Google and searched. But I still don't get it. The explanations, especially the one on Wikipedia doesn't make sense to me. What is a VFS? What is the actual need, or advantage to using a Virtual File System? As opposed to just, not?
I'm asking because I'm actually curious, and if it is that useful, I'd like to implement it into what I'm making.
Thank you
(also any links or books or something I could read on that would expand on my knowledge would help too)
Very generally speaking, the VFS is a layer of abstraction. Think of the VFS like an abstract base class that has to be used when you want to implement your concrete class of file system like NTFS, Ext3, NFS or others. It offers basic functionality that the concrete file systems can use, but also is an interface that the concrete classes have to implement against.
No idea if that was what you were looking for. Let me know if it wasn't and I can add more detail.
The VFS is part of a kernel and is a unified abstraction layer used by file systems and user applications that presents multiple local or network file systems in a common accessible format, regardless of the file system of the volume the files are on, the location of the volume the files are on (local or network), the bus / controller / storage standard or network protocol, or whether the file system is mounted on a volume or file system + volume is mounted at a mount point, allowing it to be accessible anywhere.
The VFS includes:
File IO / file mapping / file metadata / directory traversal APIs which call the underlying file system that is mounted to the volume no matter what the file system is.
API for file system drivers to be notified of volume arrival such that they can identify whether their file system is on the volume
API for file systems to perform read / write operations on the volume with their file system without knowing the underlying bus / controller / storage transfer standards, or the network storage (block, file) / transport / network / data link / physical protocols, or the physical partition or sector of the volume on the storage medium (only the logical cluster within it), or the operation of the storage medium (other than knowing whether or not external fragmentation matters).
Reparse point functionality such as mount points, directory junctions and symbolic links -- it reparses the filepath (unlike a hard link) to produce a file path for the underlying file system to access
Caching pages of files so they can be fetched from RAM without having to call the file system, and only having to call the file system on a file cache page miss (see comments).
Prefetching parts of a file around a page miss (demand paging) or prefetching associated files or dynamic libraries i.e. prefetch on Windows or even Superfetch.
A file explorer GUI application can then use the API to interact with the virtual file system representation of the volumes, and the VFS calls the underlying file system, which then read/write to their volumes through the VFS. The file explorer can then visually represent the virtual file system representations of the volumes on a common interface
In the past I used a SmartCard to store a randomly generated keyfile, which was used by TrueCrypt to open its volume. TrueCrypt used PKCS#11 to read the SmartCard and access the keyfile.
Now I'd like to do something similar, with EncFS. I wanted to store its xml config in a SmartCard. EncFS xml config has 1KB, so it fits, I can even store a few different config files in one SmartCard.
But EncFS doesn't support PKCS#11, as long as I was able to search for it. So, my idea would be to use the SmartCard as a very small and secure storage pendrive. Its readonly content would be mounted as a normal Windows drive letter, and EncFS would read its files.
Any idea how to achieve that?
This is a very interesting task. PKCS#11 devices usually are capable of holding generic BLOBs, although the size of those blobs is severely limited (usually a few hundred bytes). BLOBs have short text labels, so you can treat them as files.
I know that this task can be accomplished fully user mode with our products (SecureBlackbox for PKCS#11 access and Callback File System to create a virtual drive) and maybe you will alternative options as well. I don't think you'd be able to access PKCS#11 from the pure kernel-mode driver because most PKCS#11 "drivers" are user-mode DLLs.
The only problem with this approach is that common PKCS#11 devices (smartcards, USB tokens) are very slow AND also they require that you login before providing any information.
Another possible problem is when exactly this file is needed to EncFS. If it's to be read after you login, that's ok. But if the file is to be read during system boot, then the task would become much more complicated (if possible at all -- that depends on what exactly stage of system booting the file is read by EncFS).
In Windows for instance, and all operating systems, file priveleges exist that "prevent" a file from being written to if that rule is set.
This is hard to describe but please listen. People coding in a C language obviously would use some form of framework to easily modify a file. Using the built-in .Net framework, Microsoft obviously would put prevention into their classes checking file permissions before writing to a file. Since file permissions are stored via software and not hardware, what really prevents a file from being tampered with?
Let's hop over to Assembly. Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this? I guess what I am trying to ask is how a file permission really stays secure if the compiled program does not check for file permissions before writing to a file?
Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this?
If you write in assembly, your assembly is still run in a CPU mode that prevents direct access to memory and devices.
CPU modes … place restrictions on the type and scope of operations that can be performed by certain processes being run by the CPU. This design allows the operating system to run with more privileges than application software.
Your code still needs to issue system calls to get the OS to interact with memory not owned by your process and devices.
a system call is how a program requests a service from an operating system's kernel. This may include hardware related services (e.g. accessing the hard disk), creating and executing new processes, and communicating with integral kernel services (like scheduling).
The OS maintains security by monopolizing the ability to switch CPU modes and by crafting system calls so that they are safe for user-land code to initiate.
I'm trying to set up a directory that is stored on a local (Linux) server and mirrored on several (windows) computers.
I've looked at options like Unison, but with the type of data being worked on it's very important to have instantaneous sync to the server.
I have also looked at NFS mounting (ie sftp, webdav, etc.), but these clients are taking real-time data and I can't afford any network connectivity problems.
What would be perfect is a combination of the two : NFS mount (and "instantaneous sync") for as long as the network is up, with a local fallback storage location in case there is a network problem. Any ideas?
You might consider using one of the following:
Block device emulation:
DRBD
iSCSI + RAID1
Distributed file systems:
There are a number of file systems in this segment, with varying semantics and feature sets. I'm naming a few, though I haven't used any of those: GlusterFS, XtreemFS, Lustre.
I remember reading about a mirroring FUSE file system driver that was able to direct read traffic to the fastest (local) mirror using latency profiling. You could see if you can find it again here.
Here's the scenario:
A central machine running Linux is a "data store" for a number of instruments.
The instruments each have a PC, and each one has a remote mount (SMB) on the data store to which it write data files captured by the instrument.
The instrument PCs run a wide range of operating system including some really old ones.
The instrument PCs hardware clocks are not synchronized, and synchronizing would be problematic for a number of reasons.
Various other fixes (e.g. not using SMB, upgrading instrument operating systems, developing stuff to run on the instruments) are likewise problematic.
What we want to write is a "grabber" application that notices when an instrument writes a file to the data store via its remote mount, and then quickly copies the file somewhere else. The current plan is to use the Linux inotify subsystem to watch for file system events on the directories / trees that the files are likely to arrive in, and then do the copying.
My concern is that the fact the fact that we don't have synchronized clocks is going to be a problem. Is this concern justified?
The concern is not justified.
Linux inotify is an user space API that exposes the Linux fsnotify sub-system. This kernel file system is wired into the general file system layer of the kernel (called VFS). It gets the notification of a creation of a new file directly from the file system code by a callback function, and not by comparing the creation or access dates of the files in a directory. As such, it will not be influenced at all by the time stamps on the files, so the clocks on the different client machine would not matter at all.
To be sure you audit the code for the inotify_should_send_event() send event function, that checks whether an event needs to sent for a tracked file or directory (see here: http://lxr.linux.no/linux+v3.0.4/fs/notify/inotify/inotify_fsnotify.c#L144). Note there is no reference whatsoever to time. In the same spirit the main fsnotify function (http://lxr.linux.no/linux+v3.0.4/fs/notify/fsnotify.c#L296) in the kernel does not reference time stamps anywhere, so clocks will not affect this interface at all.
I hope this helps.