I am writing a streaming server for linux that reads files from CIFS mounts and sends
them over a socket. Ideally, linux will cache the file in memory so that subsequent
reads will be faster. Is this the case? Can I tell the kernel to cache
network reads ?
Edit: there will be multiple reads, but no writes, on these files.
Thanks!
Update: I've tested this on a CIFS volume, using fadvise POSIX_FADV_WILLNEED to cache the file locally (using linux-ftools on command line). Turns out that the volume needs to be mounted in read-write mode for this to work. In read only mode, the fadvise seems to be ignored. This must have something to do with the samba oplock mechanism.
Subject to the usual cache coherency rules [1] in CIFS, yes, the kernel CIFS client will cache file data.
[1] Roughly, CIFS is uncached in principle, but by taking oplocks the client can cache data more aggressively. For an explanation of CIFS locking, see e.g. the Samba manual at http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html . If the client(s) open the files in read only mode, then I suspect the client will use level 2 oplocks, and as there's no conflicting access takes place, multiple clients should be able to have level 2 oplocks for the same files. Only when some client requests write access to the files, will the oplocks be broken.
Related
We have our application running in a Linux VM. Application does lot of read/write with config files in disk along with logging. We often notice when VM gets reset(non-graceful), some of the config files/log files in VM which are in use get corrupted. Are there any file system(we use ext3/4) settings/tuning, fs driver settings we can do to avoid file corruption when a abrupt shutdown/restart happens?
Check this documentation:
https://www.kernel.org/doc/Documentation/filesystems/ext4.txt
In summary, you have tree options to mount your partition:
data=journal
data=ordered
data=writeback
Mounting your partition with "data=journal" option is the safest way of writing data to disk. As you can read in the provided link, with this config option enabled, all data are committed into the journal prior to being written into the main file system.
You can automate that option adding it to your /etc/fstab config file, in the 'options' column.
Almost everything I've read says that NFS soft mounts can cause silent corruption. I assume this is because of the following scenario:
user application writes to NFS
NFS client accepts the write request and returns success to the user app
NFS client has data queue/buffered waiting to be written to the NFS server
Some problem prevents the queue/buffered data from being written (eg. NFS server goes down)
My question is, what happens with this scenario with NFS soft mounts:
Same steps as above, but in addition...
The user app continues to write more data on the same file handle
The user app closes the file
Using soft mounts, will NFS flush it's cache for the just closed file? And, if unable to do that, (because the NFS soft mount gives up because of errors), shouldn't the user app get an error back on the close command?
Ie., I'm wondering if a successful close on a soft mount NFS file guarantees that there was no silent corruption.
Later edit:
Looking at http://www.avidandrew.com/understanding-nfs-caching.html, it says,
In NFSv3, the close() will cause the client to flush all data to stable storage. The client will also flush data to stable storage on a chmod, since that could potentially affect its ability to write back the data. It will not bother to do so for rename. An application should normally be able to rely on the data being safely on disk in both these situations provided that the server honours the NFS protocol (with a caveat that an ill-timed 'kill -9' could interrupt the process of flushing).
But then it also says that a NFS "commit" is ignored if the NFS volume was mounted with the async option (the default, as far as I can tell), so maybe this only applies if the NFS volume is explicitly mounted with the sync option? But the NFS man page says the sync option doesn't do caching, which contradicts this. Oh well.
The Linux NFS FAQ states that
A8. What is close-to-open cache consistency?
[...]
When the application closes the file, the NFS client writes back any pending changes to the file so that the next opener can view the changes. This also gives the NFS client an opportunity to report any server write errors to the application via the return code from close(). This behavior is referred to as close-to-open cache consistency.
I (with no proof) do not expect the fclose() causes any explicit flushing, nor to block while any flushing occurs. You've simply relinquished the file handle to the local kernel.
https://serverfault.com/questions/9499/what-are-the-advantages-disadvantages-of-hard-versus-soft-mounts-in-unix
I have two scenarios.
Scenario 1: Machine A contains 1000 documents as folders. This folder of machine A is mounted in machine B. I process documents within these folders in machine B and store the output result in mounted path in machine B.
Scenario 2: The documents in machine A is directly copied into machine B and processed
Scenario 2 is much faster than Scenario 1. I could guess its because there is no data transfer happening over the network between 2 machines. Is there a way I can use mounting and still achieve better performance?
Did you try enabling a cache? - for NFS: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/fscachenfs.html - CIFS should have caching enabled by default (unless you disabled it)
The other option would be to use something like Windows’ offline files, which copies files and folders between client and server in the background, so you don’t need to deal with it. The only thing I’ve found for linux is OFS.
But the performance depends on the size of the files and if you read them randomly or sequentially. For instance when I am encoding videos, I access the file right away via the network from my NFS, because it takes as much time as it would take to read and write the file. This way no additional time is “wasted” on the encoding, as the application can encode the stream which is coming from the network.
So for large files you might want to change the algorithms to a sequential read, on the other hand small files which are copied within seconds, could be also synced between server and client using rsync, bittorrent sync, dropbox or one of the other hundreds of tools. And this is actually quite commonly done.
I am sending some files to a fuse-mounted file system through NFS and CIFS shares.
The intention is, once the file is fully copied, we do some processing on the file, like checksum generation, archival, encryption etc.
The issue in case of NFS i can see is, there are multiple file opens, transfer chunks and releases, and then open again and a release invoked by the NFS daemon and it continues so there is no way to really know that when is the last release.
Is there any way for the NFS server to know that when the NFS client has really closed the connection, and no more transfers possible?
P.S:
When i talk open, i am referring to the fuse callback struct fuse_operations::open( ) which is initiated by the NFS daemon for a file open
P.S: For a new file creation it follows the callback sequence create, open, and finally release
Check the NFS_Unstable entry for /proc/meminfo and when its value is zero, all the pending NFS data have been accepted by the filer.
Application scenario:
I have the (normal/permanent) /var/log mounted on an encrypted partition (/dev/LVG/log). /dev/LVG/log is not accessible at boot time, it needs to be manually activated later by su from ssh.
A RAM drive (using tmpfs) is mounted to /var/log at init time (in rc.local).
Once /dev/LVG/log is activated, I need a good way of appending everything in the tmpfs to /dev/LVG/log, before mounting it as /var/log.
Any recommendations on what would be a good way of doing so? Thanks in advance!
The only thing you can do is block until you somehow verify that /var/log is mounted on an encrypted VG, or queue log entries until that happened if your app must start on boot, which could get kind of expensive. You can't be responsible for every other app on the system and I can't see any reason to encrypt boot logs.
Then again, if you know the machine has heap to spare, a log queue that flushed once some event said it was OK to write to disk would seem sensible. That's no more expensive than the history that most shells keep, as long as you take care to avoid floods of events that could fill up the queue.
This does not account for possible log loss, but could with a little imagination.
There is a risk you could lose logging. You might want to try and write your logs to a file in /tmp which is tmpfs and thus in memory. You could then append the content to your encrypted volume and then remove the file in tmp. Of course if your machine failed to boot and went down again tmp would be erased and so you'd lose a good way of working out why.