so I’m making this software that encrypts the files on a computer. A friend of mine (we're both students so don't be too hard on us) suggested I use a Virtual File System. I asked why, and what even is that, and they gave me some half assed answer that didn't help
[I don't know if this is important but I'm on a linux environment]
so no worries I went on Google and searched. But I still don't get it. The explanations, especially the one on Wikipedia doesn't make sense to me. What is a VFS? What is the actual need, or advantage to using a Virtual File System? As opposed to just, not?
I'm asking because I'm actually curious, and if it is that useful, I'd like to implement it into what I'm making.
Thank you
(also any links or books or something I could read on that would expand on my knowledge would help too)
Very generally speaking, the VFS is a layer of abstraction. Think of the VFS like an abstract base class that has to be used when you want to implement your concrete class of file system like NTFS, Ext3, NFS or others. It offers basic functionality that the concrete file systems can use, but also is an interface that the concrete classes have to implement against.
No idea if that was what you were looking for. Let me know if it wasn't and I can add more detail.
The VFS is part of a kernel and is a unified abstraction layer used by file systems and user applications that presents multiple local or network file systems in a common accessible format, regardless of the file system of the volume the files are on, the location of the volume the files are on (local or network), the bus / controller / storage standard or network protocol, or whether the file system is mounted on a volume or file system + volume is mounted at a mount point, allowing it to be accessible anywhere.
The VFS includes:
File IO / file mapping / file metadata / directory traversal APIs which call the underlying file system that is mounted to the volume no matter what the file system is.
API for file system drivers to be notified of volume arrival such that they can identify whether their file system is on the volume
API for file systems to perform read / write operations on the volume with their file system without knowing the underlying bus / controller / storage transfer standards, or the network storage (block, file) / transport / network / data link / physical protocols, or the physical partition or sector of the volume on the storage medium (only the logical cluster within it), or the operation of the storage medium (other than knowing whether or not external fragmentation matters).
Reparse point functionality such as mount points, directory junctions and symbolic links -- it reparses the filepath (unlike a hard link) to produce a file path for the underlying file system to access
Caching pages of files so they can be fetched from RAM without having to call the file system, and only having to call the file system on a file cache page miss (see comments).
Prefetching parts of a file around a page miss (demand paging) or prefetching associated files or dynamic libraries i.e. prefetch on Windows or even Superfetch.
A file explorer GUI application can then use the API to interact with the virtual file system representation of the volumes, and the VFS calls the underlying file system, which then read/write to their volumes through the VFS. The file explorer can then visually represent the virtual file system representations of the volumes on a common interface
Related
In Linux kernel, the virtual file systems lies on top of concrete file systems and device files, to provide a uniform interface to access those lying below it.
Do distributed file systems lie below or above the virtual file system in each computer?
Are distributed file systems implemented in the level of OS kernels, or as software above the OS kernel?
How does the big picture of the layers look like?
Thanks.
Reg: Virtual file system
The virtual file system is an abstraction layer for multiple file system's support. The VFS provides template wherein file system developers needs to fill in the supported operations of a file system.
Do distributed file systems lie below or above the virtual file system in each computer?
It depends on the file system implementation.
(A) It could be below VFS (a.k.a kernel file system) which uses the OS internals and having distributed file system features. It again could be of 2 types
(1) Built on existing on-disk file systems (here called proxy file systems)
(2) Could be real on-disk distributed file system.
(B) It could be above VFS and OS completely (a.k.a user file system) by using the user level API interfaces of file system OR FUSE.
This might be a dumb question, but I'm struggling to find resources that clearly explain how a VFS is different from an NFS. Can they both be used for the same purpose?
Bonus question: Can you watch a VFS with inotify like you can an NFS?
"NFS" is a network filesystem that's been around for decades. Wikipedia has you covered on that front.
"VFS" is a more generic term that simply means "virtual filesystem". Within the context of Linux, it refers to the part of the kernel with which your user-space programs actually interact when they interact with "files". The VFS layer then passes requests to a concrete filesystem driver -- such as NFS, for example, or ext4, or others.
Read more here and here.
A virtual file system (VFS) is an abstraction layer on top of a more concrete file system.The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way, Where as Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystem in 1984, allowing a user on a client computer to access files over a computer network much more like local storage is accessed
A VFS can be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, Mac and Unix file systems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing Where as, NFS like many other protocols, builds on the Open Newtork Computing Remote Procedure Call (ONC RPC) system. The NFS is an open standard defined in Request for comments (RFC), allowing anyone to implement the protocol.
"VFS" is the name given to the entire layer in the kernel situated between the system calls and the filesystem drivers; it is not a filesystem in its own right.
In the past I used a SmartCard to store a randomly generated keyfile, which was used by TrueCrypt to open its volume. TrueCrypt used PKCS#11 to read the SmartCard and access the keyfile.
Now I'd like to do something similar, with EncFS. I wanted to store its xml config in a SmartCard. EncFS xml config has 1KB, so it fits, I can even store a few different config files in one SmartCard.
But EncFS doesn't support PKCS#11, as long as I was able to search for it. So, my idea would be to use the SmartCard as a very small and secure storage pendrive. Its readonly content would be mounted as a normal Windows drive letter, and EncFS would read its files.
Any idea how to achieve that?
This is a very interesting task. PKCS#11 devices usually are capable of holding generic BLOBs, although the size of those blobs is severely limited (usually a few hundred bytes). BLOBs have short text labels, so you can treat them as files.
I know that this task can be accomplished fully user mode with our products (SecureBlackbox for PKCS#11 access and Callback File System to create a virtual drive) and maybe you will alternative options as well. I don't think you'd be able to access PKCS#11 from the pure kernel-mode driver because most PKCS#11 "drivers" are user-mode DLLs.
The only problem with this approach is that common PKCS#11 devices (smartcards, USB tokens) are very slow AND also they require that you login before providing any information.
Another possible problem is when exactly this file is needed to EncFS. If it's to be read after you login, that's ok. But if the file is to be read during system boot, then the task would become much more complicated (if possible at all -- that depends on what exactly stage of system booting the file is read by EncFS).
In Windows for instance, and all operating systems, file priveleges exist that "prevent" a file from being written to if that rule is set.
This is hard to describe but please listen. People coding in a C language obviously would use some form of framework to easily modify a file. Using the built-in .Net framework, Microsoft obviously would put prevention into their classes checking file permissions before writing to a file. Since file permissions are stored via software and not hardware, what really prevents a file from being tampered with?
Let's hop over to Assembly. Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this? I guess what I am trying to ask is how a file permission really stays secure if the compiled program does not check for file permissions before writing to a file?
Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this?
If you write in assembly, your assembly is still run in a CPU mode that prevents direct access to memory and devices.
CPU modes … place restrictions on the type and scope of operations that can be performed by certain processes being run by the CPU. This design allows the operating system to run with more privileges than application software.
Your code still needs to issue system calls to get the OS to interact with memory not owned by your process and devices.
a system call is how a program requests a service from an operating system's kernel. This may include hardware related services (e.g. accessing the hard disk), creating and executing new processes, and communicating with integral kernel services (like scheduling).
The OS maintains security by monopolizing the ability to switch CPU modes and by crafting system calls so that they are safe for user-land code to initiate.
I'm trying to set up a directory that is stored on a local (Linux) server and mirrored on several (windows) computers.
I've looked at options like Unison, but with the type of data being worked on it's very important to have instantaneous sync to the server.
I have also looked at NFS mounting (ie sftp, webdav, etc.), but these clients are taking real-time data and I can't afford any network connectivity problems.
What would be perfect is a combination of the two : NFS mount (and "instantaneous sync") for as long as the network is up, with a local fallback storage location in case there is a network problem. Any ideas?
You might consider using one of the following:
Block device emulation:
DRBD
iSCSI + RAID1
Distributed file systems:
There are a number of file systems in this segment, with varying semantics and feature sets. I'm naming a few, though I haven't used any of those: GlusterFS, XtreemFS, Lustre.
I remember reading about a mirroring FUSE file system driver that was able to direct read traffic to the fastest (local) mirror using latency profiling. You could see if you can find it again here.