I just tried to redirect my code output to my code file itself, all contents in my code file are gone, any methods to recover?
What can you do to recover the file you overwrote? Virtually nothing as the disk blocks that were freed when file was truncated are likely the first ones overwritten.
What to do to avoid it? Use Version Control religiously (I prefer mercurial because it is the lightest for single-person projects).
You could also set the "noclobber" option in the shell set -C which will disallow
redirection_to > an_existing_file
with the message "cannot overwrite existing file". But don't do this. It is a generally bad practice because the one time you are in a shell without noclobber set, you will accidentally overwrite a precious file.
Related
With more and more programs installed on my computer, I am tired of seeing lots of dotfiles while I have to access them often. For some reason I won't hide dotfiles when browsing files. Is there a way to move them to a better place I want them to stay (e.g. ~/.config/$PROGCONF) without affecting programs while running?
Symlinks still leave file symbols, which is far from my expectation. I expect that operations like listdirs() won't show the files while opening them uses a redirection.
"For some reason it won't hide dotfiles when browsing files.":
That depends on the file manager you use. nautilus hides it by default and most file managers have an option to "show/hide hidden files". The ls command by default omits out hidden files (files starting with a dot). It lists all files with the option -a.
"Is there a way to move them to a better place":
Programs which have support for "XDG user directories" can store their config files in `~/.config/$PROGRAM_NAME/. If the program doesn't support that and expects the config file to be present in the home directory, there is little you can do (Maybe you can give us a list of what programs' config files you want to move). The process differs for each program.
Let me give an example with vim. Its config file is ~/.vimrc. Lets say you move the file to ~/.config/vim/.vimrc. You can make vim read the file by launching vim using the following command.
vim -u ~/.config/vim/.vimrc
You can modify the .desktop entry or create a new shell script to launch vim using the above command and put it inside /usr/local/bin/ or create shell functions / aliases. You can read more about changing vim's config file location in this SO question.
This arch wiki article has application specific information.
"without affecting programs while running":
It depends on a few factors namely the file system used, the program we are dealing with and so on.
Generally, deleting / moving files only unlinks the file name from an inode and programs read / write files using inodes. Read more here. And most programs read the config file at the start, load the values into memory. They rarely read the config files again. So, if you move your config file while the program is running (assuming the program supports config in both places), you won't see a difference until the program is restarted.
"I expect that operations like listdirs() won't show the files"
I am assuming you are talking about os.listdir() in python. If files are present, os.listdir() will list them, there is little you can change about that. But you can write custom functions to omit out the hidden files from being listed.
This SO question can help with that.
I am looping through the files in my Commands folder and loading them using require.
ESLint tells me this is wrong. What is the safe way to load commands from a set of files that is not static?
The warning exists for security reasons. It is rare that you need to dynamically require files. Quote from the documentation:
detect-non-literal-require
Detects require(variable), which might allow an attacker to load and run arbitrary code, or access arbitrary files on disk.
More information: http://www.bennadel.com/blog/2169-where-does-node-js-and-require-look-for-modules.htm
If you know that in your code it is safe, I see no better way then to suppress the warning for the next line.
I have problem in working with link command in linux mint.
I made file1 and add a new hard link to that file :
link file1 file2
I know when I change the contents of file1 , file2 should change too.
and when I edit file1 with vim or add text to it with redirections it works well but when
I edit file1 with kate editor then it's like that the editor break the link of file2! and after that when
I change the contents of file1 with kate or vim,... file 2 will never change any more.
what's the problem?
I'm one of the Kate developers. The issue is as follows: Whenever Kate saves, it does so by saving into a temporary file in the same folder, and on success just does a move to the desired location.
This move operation is exactly what destroys your hard link: first, the hard link is removed, then the temporary file is renamed.
While this avoids data loss, it also has its own problems as you experience. We are tracking this bug here:
https://bugs.kde.org/show_bug.cgi?id=358457 - QSaveFile: Kate removes a hard link to file when opening a file with several hard links
And in addition, QSaveFile also has two further issues, tracked here:
https://bugs.kde.org/show_bug.cgi?id=333577 - QSaveFile: kate changes file owner
https://bugs.kde.org/show_bug.cgi?id=354405 - QSaveFile: Files are unlinked upon save
The solution would be to just directly write in all these corner cases, then we could avoid this trouble at the expense of loosing data in case of a system crash, so it's a tradeoff. To fix this, we need to patch Qt, which noone did so far.
Different programs save files in different ways. At least two come to mind:
open existing file and overwrite its content
create temporary file, write new content there, then somehow replace original file with new one (remove old file and rename new one; or rename old file, rename new file, then remove old file; or use system function to swap files content (in fact, swap names of files), then remove old file; or ...)
Judging from its current source code, Kate is using the latter approach (using QSaveFile in the end, with direct write fallback though). Why? Usually, to make file saving more or less atomic, and also to make sure that saving won't result in errors caused by e.g. lack of free space (although this also means that it uses the space of sum of old and new file sizes when saving).
I don't have Kate on Linux Mint but I have noticed issues which lead me to suspect you may have come across a "bug".
Here are two 'similar' issues with hard links that have been reported.
Bug 316240 - KSaveFile: Kate/kwrite makes a copy of a file while editing a hard link
"Hard link not supported" with NTFS on Linux #3241
I thought about a concurrency issue (in Solaris), what happen if while reading someone tries to delete the same file. I have a query regarding file existence in the Solaris/Linux. suppose I have a file test.txt, I have open it in vi editor, and then I have open a duplicate session and remove that file, but even after deleting that file I am able to read that file. so here are my questions:
Do I need to thinks about any locking mechanism while reading, so no one able to delete same file while reading.
What is the reason of showing different behavior from windows(like in windows if file is open in in some editor than we can not delete that file)
After removing that file, how I am still able to read that file, if I haven't closed file from vi editor.
I am asking files in general,but yes platform specific i.e. unix. what will happen if I am using a java program (buffer reader) for read file and file is deleted while reading, does buffer reader still able to read the file for next chunk or not?
You have basically 2 or 3 unrelated questions there. Text editors like to read the whole file into memory at the start of the editing session. Imagine every character you type being saved to disk immediately, with all characters after it in the file being rewritten one place further along to make room. That would be awful. Much better that the thing you're actually editing is a memory representation of the file (array of pointers to lines, probably with some metadata attached) which only gets converted back into a linear stream when you explicitly save.
Any relatively recent version of vim will notify you if the file you are editing is deleted from its original location with the message
E211: File "filename" no longer available
This warning is not just for unix. gvim on Windows will give it to you if you delete the file being edited. It serves as a reminder that you need to save the version you're working on before you exit, if you don't want the file to be gone.
(Note: the warning doesn't appear instantly - vim only checks for the original file's existence when you bring it back into the foreground after having switched away from it.)
So that's question 1, the behavior of text editors - there's no reason for them to keep the file open for the whole session because they aren't actually using it except at startup and during a save operation.
Question 2, why do some Windows editors keep the file open and locked - I don't know, Windows people are nuts.
Question 3, the one that's actually about unix, why do open files stay accessible after they're deleted - this is the most interesting one. The answer, guaranteed to shock you when presented directly:
There is no command, function, syscall, or any other method which actually requests deletion of a file.
Underlying rm and any other command that may appear to delete a file there is the system call unlink. And it's called unlink, not remove or deletefile or anything similar, because it doesn't remove a file. It removes a link (a.k.a. directory entry) which is an association between a file and a name in a directory. (Note: ANSI C added remove as a more generic function to appease non-unix people who had no intention of implementing unix filesystem semantics, but on unix, remove is just a rmdir if the target is a directory, and unlink for everything else.)
A file can have multiple links (see the ln command for how they are created), which means that the same file is known by multiple names. If you rm one of them, the others stick around and the file is not deleted. What happens when you remove the last link? Well, now you have a file with no name. But names are only one kind of reference to a file. There are at least 2 others: file descriptors and mmap regions. When the last reference to a file goes away, that's when the file is deleted.
Since references come in several forms, there are many kinds of events that can cause a file to be deleted. Here are some examples:
unlink (rm, etc.)
close file descriptor
dup2 (can implicitly closes a file descriptor before replacing it with a copy of a different file descriptor)
exec (can cause file descriptors to be closed via close-on-exec flag)
munmap (unmap memory region)
mmap (if you create a new memory map at an address that's already mapped, the old mapping is unmapped)
process death (which closes all file descriptors and unmaps all memory mappings of the process)
normal exit
fatal signal generated by the kernel (^C, segfault)
fatal signal sent from another process (kill)
I won't call that a complete list. And I don't encourage anyone to try to build a complete list. Just know that rm is "remove name", not "remove file", and files go away as soon as they're not in use.
If you want to destroy the contents of a file immediately, truncate it. All processes already using it will find that its size has suddenly become 0. (This is destruction as far as the normal file access methods are concerned. To destroy it more thoroughly so that even someone with raw disk access can't read what used to be there, you need to overwrite it. There's a tool called shred for that.)
I think your question has nothing to do with the difference between Windows/Linux. It's about how VI works.
when using VI to edit a file. VI will create a .swp file. And the .swp file is what you are actually editing. At the same time, if other users delete the original file will not effect your editing.
And when you type :w in VI, VI will use .swp file to overwrite the original file.
I believe it's designed to be atomic, and I've heard that vim will save to a temporary directory and then mv the temp file over the old file to ensure that EITHER the old version OR the new version is present, and never half of one or no file at all, even if the power is turned off mid-save.
Does anyone have more details?
How this is done depends on the values of several options.
The main option that affects this is backupcopy, on unix systems this defaults to yes on other systems it defaults to auto. Only if it is set to no can you be assured that writes will be done like in your question. When set to yes vim will instead overwrite the original file contents after making a backup copy. If set to auto, it will behave as in the question if it detects that attributes can be passed on and the file isn't a link (either a symlink or a hard link). There are additional values for the option to have it break links, see the help for that option.
Other options that affect this are backup, writebackup. At least one of those must be set for a backup to be made at all. If no backup is done, the original file contents will be overwritten. The writebackup option defaults to on as long as vim was compiled with support for it. There's also the backupskip option which specifies a list of file patterns for which no backups will be done; if one of those patterns match the file, again no backup will be done and the original file contents will be overwritten.
You're right about the general approach; for more information, see :help backup.
You can fine-tune the approach via settings; this is important when you deal with hard-lined files or use APIs like inotify to react to file system changes.
Additionally, the swapfile (:help swap-file) and persistent undo (:help persistent-undo) help you with avoiding any data loss, even in case of a disk crash.