In Linux kernel, the virtual file systems lies on top of concrete file systems and device files, to provide a uniform interface to access those lying below it.
Do distributed file systems lie below or above the virtual file system in each computer?
Are distributed file systems implemented in the level of OS kernels, or as software above the OS kernel?
How does the big picture of the layers look like?
Thanks.
Reg: Virtual file system
The virtual file system is an abstraction layer for multiple file system's support. The VFS provides template wherein file system developers needs to fill in the supported operations of a file system.
Do distributed file systems lie below or above the virtual file system in each computer?
It depends on the file system implementation.
(A) It could be below VFS (a.k.a kernel file system) which uses the OS internals and having distributed file system features. It again could be of 2 types
(1) Built on existing on-disk file systems (here called proxy file systems)
(2) Could be real on-disk distributed file system.
(B) It could be above VFS and OS completely (a.k.a user file system) by using the user level API interfaces of file system OR FUSE.
Related
so I’m making this software that encrypts the files on a computer. A friend of mine (we're both students so don't be too hard on us) suggested I use a Virtual File System. I asked why, and what even is that, and they gave me some half assed answer that didn't help
[I don't know if this is important but I'm on a linux environment]
so no worries I went on Google and searched. But I still don't get it. The explanations, especially the one on Wikipedia doesn't make sense to me. What is a VFS? What is the actual need, or advantage to using a Virtual File System? As opposed to just, not?
I'm asking because I'm actually curious, and if it is that useful, I'd like to implement it into what I'm making.
Thank you
(also any links or books or something I could read on that would expand on my knowledge would help too)
Very generally speaking, the VFS is a layer of abstraction. Think of the VFS like an abstract base class that has to be used when you want to implement your concrete class of file system like NTFS, Ext3, NFS or others. It offers basic functionality that the concrete file systems can use, but also is an interface that the concrete classes have to implement against.
No idea if that was what you were looking for. Let me know if it wasn't and I can add more detail.
The VFS is part of a kernel and is a unified abstraction layer used by file systems and user applications that presents multiple local or network file systems in a common accessible format, regardless of the file system of the volume the files are on, the location of the volume the files are on (local or network), the bus / controller / storage standard or network protocol, or whether the file system is mounted on a volume or file system + volume is mounted at a mount point, allowing it to be accessible anywhere.
The VFS includes:
File IO / file mapping / file metadata / directory traversal APIs which call the underlying file system that is mounted to the volume no matter what the file system is.
API for file system drivers to be notified of volume arrival such that they can identify whether their file system is on the volume
API for file systems to perform read / write operations on the volume with their file system without knowing the underlying bus / controller / storage transfer standards, or the network storage (block, file) / transport / network / data link / physical protocols, or the physical partition or sector of the volume on the storage medium (only the logical cluster within it), or the operation of the storage medium (other than knowing whether or not external fragmentation matters).
Reparse point functionality such as mount points, directory junctions and symbolic links -- it reparses the filepath (unlike a hard link) to produce a file path for the underlying file system to access
Caching pages of files so they can be fetched from RAM without having to call the file system, and only having to call the file system on a file cache page miss (see comments).
Prefetching parts of a file around a page miss (demand paging) or prefetching associated files or dynamic libraries i.e. prefetch on Windows or even Superfetch.
A file explorer GUI application can then use the API to interact with the virtual file system representation of the volumes, and the VFS calls the underlying file system, which then read/write to their volumes through the VFS. The file explorer can then visually represent the virtual file system representations of the volumes on a common interface
This might be a dumb question, but I'm struggling to find resources that clearly explain how a VFS is different from an NFS. Can they both be used for the same purpose?
Bonus question: Can you watch a VFS with inotify like you can an NFS?
"NFS" is a network filesystem that's been around for decades. Wikipedia has you covered on that front.
"VFS" is a more generic term that simply means "virtual filesystem". Within the context of Linux, it refers to the part of the kernel with which your user-space programs actually interact when they interact with "files". The VFS layer then passes requests to a concrete filesystem driver -- such as NFS, for example, or ext4, or others.
Read more here and here.
A virtual file system (VFS) is an abstraction layer on top of a more concrete file system.The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way, Where as Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystem in 1984, allowing a user on a client computer to access files over a computer network much more like local storage is accessed
A VFS can be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, Mac and Unix file systems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing Where as, NFS like many other protocols, builds on the Open Newtork Computing Remote Procedure Call (ONC RPC) system. The NFS is an open standard defined in Request for comments (RFC), allowing anyone to implement the protocol.
"VFS" is the name given to the entire layer in the kernel situated between the system calls and the filesystem drivers; it is not a filesystem in its own right.
I am reading Embedded Linux Primer book and The Linux Kernel Module Programming Guide and I am confused about user space application communication with kernel module
User space App --> Device node/proc file --> kernel module ( which resides in /lib/modules/)
1) What is exact difference when we communicate with device node method ( /dev/ - with open,read,write,close calls) and /proc/file method ?
procfs (/proc) should be reserved for process information a module should not put any file there. At some point, procfs was the only available pseudo filesystem, that is why you can find sound system or RTC information. Then, sysfs was created to properly contain those information.
The main difference between using a device file (usually residing in /dev) and a file from procfs is the way it is handled in the kernel.
Operations used for the device files are registered using the file_operations structure usually with cdev_init and cdev_add for a character device. Your module may not do that as quite often, the subsytem is the one registering your device.
While the operations for files in procfs are registered using proc_create
In Windows for instance, and all operating systems, file priveleges exist that "prevent" a file from being written to if that rule is set.
This is hard to describe but please listen. People coding in a C language obviously would use some form of framework to easily modify a file. Using the built-in .Net framework, Microsoft obviously would put prevention into their classes checking file permissions before writing to a file. Since file permissions are stored via software and not hardware, what really prevents a file from being tampered with?
Let's hop over to Assembly. Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this? I guess what I am trying to ask is how a file permission really stays secure if the compiled program does not check for file permissions before writing to a file?
Suppose I create an Assembly program that directly accesses hard drive data and changes the bytes of a file. How could file permissions possibly prevent me from doing this?
If you write in assembly, your assembly is still run in a CPU mode that prevents direct access to memory and devices.
CPU modes … place restrictions on the type and scope of operations that can be performed by certain processes being run by the CPU. This design allows the operating system to run with more privileges than application software.
Your code still needs to issue system calls to get the OS to interact with memory not owned by your process and devices.
a system call is how a program requests a service from an operating system's kernel. This may include hardware related services (e.g. accessing the hard disk), creating and executing new processes, and communicating with integral kernel services (like scheduling).
The OS maintains security by monopolizing the ability to switch CPU modes and by crafting system calls so that they are safe for user-land code to initiate.
I would like to know how to create a root file system for an embedded Linux system that is stored on a hard drive. Would this be the same procedure if it was on a flash card?
No, your boot loader would need to know how to initialize the hard drive. With flash cards the boot loader initializes as an MTD and can understand the file system.
You might be able to make progress with an IDE HD and IDE support in the boot loader.
On a regular computer (e.g., PC) the BIOS takes care of initializing all peripherals, like a primary HD.
Typically Linux embedded system is not operate directly in disk based filesystem, but use a mechanism to load the OS from a persistent storage (hard drive, flash card or memory, etc.) to volatile memory space (RAM). In general, these OS's file (commonly called as firmware) are kernel image file and a initrd (Initial RAM Disk) file, the initrd file contains root filesystem's files and any system's related files, upon boot the initrd will be uncompressed and deployed into a RAM based filesystem such as tmpfs, once completed, the system will use the tmpfs filesystem just like any disk based filesystem (ext3, btrfs), for example to run init program or script to do system initialization. Embedded system is tend to minimize I/O on persistent storage for some advantages: reliability, speed and cost.
You can learn how to accomplish this by learning any general Linux distribution on how to create and modify a initrd file.