Can I configure Vim so it does an atomic write?
The default Vim behavior is:
move file.txt to file.txt~
write buffer contents to file.txt
But this causes some problem with compass watch for example
I would like behavior like:
write buffer contents to file.txt~
move file.txt~ to file.txt
I've looked at the backup, backupcopy and writebackup options but I can't get Vim configured as I want.
:set backupcopy=yes is probably what you want.
You could also use the `backupskip' option on the paths you are particularly worried about.
I'm not 100% sure how atomic writes are typically handled in arbitrary software. I could be wrong, but something like this is probably handled or configurable at the kernel level (and that would seem like the optimal place to handle such a notion as atomic writes for multiple reasons such as: efficiency, direct access to hardware, the DRY principle, application agnosticism, etc.). A more manual and user-space way that I can think of accomplishing this however would be to first write to some new/unique/random file inside of /tmp, then mv that file that you just wrote under /tmp into place. You can use Vim's tempname() function to help you accomplish this in a platform-agnostic manner (see :help tempname()).
Related
I need to tail an input file for commands, ignoring EOF. I've been using inotify(2) to block until changes have been made to the file after reaching EOF, which works fine. However, inotify(2) is a Linux-specific syscall. Are there any alternatives defined in POSIX?
Are there any alternatives defined in POSIX?
No.
Well, it's easy to prove that something exists - it's there. It's harder to prove something is not there.
It's not there. There is no POSIX interface with similar functionality as inotify or kqueue.
If you want to be portable, handle each system separately. Don't reinvent the wheel - libuv and libevent exist.
I want to write a wrapper for memory mapped file io, that either fails to map a file or returns a mapping that is valid until it is unmapped. With plain mmap, problems arise, if the underlying file is truncated or deleted while being mapped, for example. According to the linux man page of mmap SIGBUS is received if memory beyond the new end of the file is accessed after a truncate. It is no option to catch this signal and handle the error this way.
My idea was to create a copy of the file and map the copy. On a cow capable file system, this would impose little overhead.
But the problem is: how do I protect the copy from being manipulated by another process? A tempfile is no real option, because in theory a malicious process could still mutate it. I know that there are file locks on Linux, but as far as I understood they're either optional or don't prevent others from deleting the file.
I'm asking for two kinds of answers: Either a way to mmap a file in a rock solid way or a mechanism to protect a tempfile fully from other processes. But maybe my whole way of approaching the problem is wrong, so feel free to suggest radical solutions ;)
You can't prevent a skilled and determined user from intentionally shooting themselves in the foot. Just take reasonable precautions so it doesn't happen accidentally.
Most programs assume the input file won't change and that's usually fine
Programs that want to process files shared with cooperative programs use file locking
Programs that want a private file will create a temp file, snapshot or otherwise -- and if they unlink it for auto-cleanup, it's also inaccessible via the fs
Programs that want to protect their data from all regular user actions will run as a dedicated system account, in which case chmod is protection enough.
Anyone with access to the same account (or root) can interfere with the program with a simple kill -BUS, chmod/truncate, or any of the fancier foot-guns like copying and patching the binary, cloning its FDs, or attaching a debugger. If that's what they want to do, it's not your place to stop them.
I've just read a handful of man pages: dup, dup2, fcntl, pread/pwrite, mmap, etc.
Currently I am using mmap, but it's not the nicest thing in the world because I have to manage file offset and buffer length myself and basically reimplement read/write in userspace.
From what I gathered:
dup, dup2, fcntl just create aliases for the fds, so their offsets and flags are shared - reading from one advances the offset of the others.
pread/pwrite can be buggy and give inconsistent results.
mmap is buggy on linux when given some uncommon flags, but I don't need them.
Am I missing something or is mmap really the way to go?
(Note that re-open()ing a file is dangerous on POSIX - unlike Windows, POSIX provides no guarantees on the path not being moved/deleted while the file is open. On POSIX, you can open a path, move the file, and still read from it. You can even delete the file sometimes. I also couldn't find anything that can open an inode.)
I'd like answers for at least the most common POSIX variants, if there's no one answer for them all.
On Linux, opening /proc/self/fd/$NUM will work regardless of whether the file still has the same name it had the first time you opened it, and will generate a new open file description (i.e. a new fd with independent offset and flags).
I don't know of any POSIXly portable way of doing this.
(I also don't know what you mean about pread/pwrite being buggy...)
It seems not to me and I found a link that supports my opinion. What do you think?
The content of the link you posted is correct. A regular file socket, opened in non-blocking mode, will always be "ready" for reading; when you actually try to read it, blocking (or more accurately as your source points out, sleeping) will occur until the operation can succeed.
In any case, I think your source needs some sedatives. One angry person, that is.
I've been digging into this quite heavily for the past few hours and can attest that the author of the link you cited is correct. However, the appears to be "better" (using that term very loosely) support for non-blocking IO against regular files in native Linux Kernel for v2.6+. The "libaio" package contains a library that exposes the functionality offered by the kernel, but it has some caveats about the different types of file systems which are supported and it's not portable to anything outside of Linux 2.6+.
And here's another good article on the subject.
You're correct that nonblocking mode has no benefit for regular files, and is not allowed to. It would be nice if there were a secondary flag that could be set, along with O_NONBLOCK, to change this, but due to the way cache and virtual memory work, it's actually not an easy task to define what correct "non-blocking" behavior for ordinary files would mean. Certainly there would be race conditions unless you allowed programs to lock memory associated with the file. (In fact, one way to implement a sort of non-sleeping IO for ordinary files would be to mmap the file and mlock the map. After that, on any reasonable implementation, read and write would never sleep as long as the file offset and buffer size remained within the bounds of the mapped region.)
I am developing under Linux with pretty tight constraints on disk usage. I'd like to be able to point logging to a fixed-size file. For example, if my application outputs all logs to stdout:
~/bin/myApp > /dev/debug1
and then, to see the last amount of output:
cat /dev/debug1
would write out however many bytes debug1 was setup to save (if at least that many had been written there).
This post suggests using expect or its library, but I was wondering if anyone has seen a "pseudo-tty" device driver-type implementation as I would prefer to not bind any more libraries to my executable.
I realize there are other mechanisms like logrotate, but I'd prefer to have a non-cron solution.
Pointers, suggestions, questions welcome!
Perhaps you could achieve what you want using mkfifo and something that reads the pipe with a suitable buffer. I haven't tried, but less --buffers=XXXXXX might work for this.