For example, if I create a directory, check its timestamp, then create a directory inside the first directory a minute later, the timestamp for the mtime will change to when the new directory was created. However if I wait another minute and create a third directory inside the second, the second directory will update the mtime to the third directory's time while the first directory will still have the 2nd directory's original mtime. Despite the contents of the first directory changing when the third directory is created, the mtime doesn't change.
Is there documentation as to how linux looks to change mtime? Or is it only able to see direct children changes to update mtime?
This is the closest I could find. From the stat man page:
The field st_mtime is changed by file modifications, for example, by
mknod(2), truncate(2), utime(2) and write(2) (of more than zero
bytes). Moreover, st_mtime of a directory is changed by the creation
or deletion of files in that directory. The st_mtime field is not
changed for changes in owner, group, hard link count, or mode.
-Source
This wording implies to me that a modification in mtime of a nested directory is not considered a modification by the parent directory, since it does not qualify as a mknod, truncate, utime, or write operation.
Hope this helps!
Related
I have an open std::fs::File, and I want to get it's filename, e.g. as a PathBuf. How do I do that?
The simple solution would be to just save the path used in the call to File::open. Unfortunately, this does not work for me. I am trying to write a program that reads log files, and the program that writes the logs keep changing the filenames as part of it's log rotation. So the file may very well have been renamed since it was opened. This is on Linux, so renaming open files is possible.
How do I get around this issue, and get the current filename of an open file?
On a typical Unix filesystem, a file may have multiple filenames at once, or even none at all. The file metadata is stored in an inode, which has a unique inode number, and this inode number can be linked from any number of directory entries. However, there are no reverse links from the inode back to the directory entries.
Given an open File object in Rust, you can get the inode number using the ino() method. If you know the directory the log file is in, you can use std::fs::read_dir() to iterate over all entries in that directory, and each entry will also have an ino() method, so you can find the one(s) matching your open file object. Of course this approach is subject to race conditions – the directory entry may already be gone again once you try to do anything with it.
On linux, files handles held by the current process can be found under /proc/self/fd. These look and act like symlinks to the original files (though I think they may technically be something else - perhaps someone who knows more can chip in).
You can therefore recover the (possibly changed) file name by constructing the correct path in /proc/self/fd using your file descriptor, and then following the symlink back to the filesystem.
This snippet shows the steps:
use std::fs::read_link;
use std::os::unix::io::AsRawFd;
use std::path::PathBuf;
// if f is your std::fs::File
// first construct the path to the symlink under /proc
let path_in_proc = PathBuf::from(format!("/proc/self/fd/{}", f.as_raw_fd()));
// ...and follow it back to the original file
let new_file_name = read_link(path_in_proc).unwrap();
According to the documentation:
Change time (ctime) This marks when a file's metadata was changed,
such as permissions or ownership. This is also known as the "update"
time in some documentation.
Yet when I create a new file in a directory and run the istat command on the directory afterwards I notice that the ctime aka "update" time of the directory has changed. I thought that the ctime should only change if you change the metadata of the directory?
According to stat() system call specification:
The stat() function shall update any time-related fields (as described in XBD File Times Update), before writing into the stat structure.
In the corresponding File Times Update document:
Each function or utility in POSIX.1-2017 that reads or writes data (even if the data does not change) or performs an operation to change file status (even if the file status does not change) indicates which of the appropriate timestamps shall be marked for update.
The list of POSIX system calls contains the following calls related to creation of objects inside a directory:
link()
Upon successful completion, link() shall mark for update the last file status change timestamp of the file. Also, the last data modification and last file status change timestamps of the directory that contains the new entry shall be marked for update.
mkdir()
Upon successful completion, mkdir() shall mark for update the last data access, last data modification, and last file status change timestamps of the directory. Also, the last data modification and last file status change timestamps of the directory that contains the new entry shall be marked for update.
mkfifo()
Upon successful completion, mkfifo() shall mark for update the last data access, last data modification, and last file status change timestamps of the file. Also, the last data modification and last file status change timestamps of the directory that contains the new entry shall be marked for update.
mknod()
Upon successful completion, mknod() shall mark for update the last data access, last data modification, and last file status change timestamps of the file. Also, the last data modification and last file status change timestamps of the directory that contains the new entry shall be marked for update.
open()
If O_CREAT is set and the file did not previously exist, upon successful completion, open() shall mark for update the last data access, last data modification, and last file status change timestamps of the file and the last data modification and last file status change timestamps of the parent directory.
symlink()
Upon successful completion, symlink() shall mark for update the last data access, last data modification, and last file status change timestamps of the symbolic link. Also, the last data modification and last file status change timestamps of the directory that contains the new entry shall be marked for update.
The subdirectory listing is contained in a file, so when you update a file in the directory, it updates the metadata in the directory listing for that file (mtime, etc). Hence, the subdirectory itself is a file that has been modified.
Background:
On CentOS 7 x86_64. I am using applydeltarpm, and was running out of disk space when creating the new RPM from the delta. I watched the / disk space usage increase during the apply process to 100%, but could not find the working/temp file on the volume using either ls -l /tmp or find /tmp -mmin -1 -type f.
I changed the applydeltarpm source code to use /var/tmp instead of /tmp, and rebuilt the RPM. Now apply works with the modified applydeltarpm, because /var/tmp has a lot more disk space. But I still cannot find the temp file created with mkstemp64.
Question:
The temp file created by mkstemp64 appears to be "non-existent", but still exists as a file descriptor to the creator, and is using considerable disk space when applydeltarpm creates a large RPM (1 hour to apply on a slow disk). The mkstemp64 documentation says an actual file is created. And the source code shows the template file name is /tmp/deltarpmpageXXXXXX. But a file with that template name does not exist.
How is this temporary file able to be created on the system without being findable with the usual directory listing ls, or find. And how can I find these kinds of "non-existent" files in the system?
(I am curious, because I also am monitoring system security)
References:
https://github.com/rpm-software-management/deltarpm/blob/master/applydeltarpm.c
# line 198
if (pagefd < 0)
{
char tmpname[80];
sprintf(tmpname, "/tmp/deltarpmpageXXXXXX");
#ifdef DELTARPM_64BIT
pagefd = mkstemp64(tmpname);
#else
pagefd = mkstemp(tmpname);
#endif
if (pagefd < 0)
{
fprintf(stderr, "could not create page area\n");
exit(1);
}
unlink(tmpname);
}
https://www.mkssoftware.com/docs/man3/mkstemp.3.asp
The mktemp() function returns a unique file name based on the template
parameter. At the time it is generated, no file in the current
directory has that name. No file is actually created, so it is
possible that another application could create a file with this name.
The mkstemp() function is similar to mktemp() in that it create a
unique file name based on template; however, mkstemp() actually
creates the file and returns its file descriptor. The name of the
created file is stored in template.
The mkstemp64() function is identical to the mkstemp() function except
that the file is opened with the O_LARGEFILE flag set.
It is a common practice to unlink the temporary file right after creation, if it's not needed to be accessed through a filesystem anymore. This avoids dangling temporary files if the process crashes or forgets to unlink it later.
unlink() does not delete the file, it only removes the link to the file from the filesystem. Every link to a file in a filesystem increases the link count of the file by one (there can be several links to the same file). Also every process, that calls open() or mmap() to open the file, increases the file count, until it closes the descriptor - then the link count is decreased. File exists as long as there is at least one link to it. When link count reaches zero, then the file is actually deleted.
mkstemp() also calls open() behind the scenes to open the temporary file and return its descriptor.
In order to see the files that are opened, but do not exist in the filesystem anymore, you can use lsof and search for the lines, where there is "(deleted)" after the file name.
lsof | grep '(deleted)'
The (disk)space used by these files will be freed, when the processes they are attached to are finished or close the file descriptor by themselves.
On linux, on a ext4 partition there is a movie, if I open the movie & watch it, then I move or rename the movie.
After the previous cache use up, it need to read more cache from original file, then it still could do that.
The question is:
How file system & inode achieve this?
Using rename() on a file within the same file system just changes the name that points to the inode. You can even use rename() to move that name into another directory - as long as it's in the same file system:
The rename() function shall change the name of a file. The old
argument points to the pathname of the file to be renamed. The new
argument points to the new pathname of the file. ...
Note that the POSIX specification for rename() is a lot more specific than the C Standard rename() specification:
7.21.4.2 The rename function
Synopsis
#include <stdio.h>
int rename(const char *old, const char *new);
Description
The rename function causes the file whose name is the string pointed
to by old to be henceforth known by the name given by the string
pointed to by new . The file named old is no longer accessible by
that name. If a file named by the string pointed to by new exists
prior to the call to the rename function, the behavior is
implementation-defined.
Returns
The
rename
function returns zero if the operation succeeds, nonzero if it fails,
269)
in
which case if the file existed previously it is still known by its original name.
(That's the entire C Standard specification for rename(). Read the POSIX link for a lot more information.)
So how can you rename a file while your watching it in an application? The underlying inode that your movie-watching process's open file descriptor uses to access the file that you just rename()'d doesn't change.
That's the same reason why you can unlink() (delete) a file while it's in use - all you're doing is removing the entry that points to the inode - one of the links to the inode. The space used by the file isn't freed until the last link to the inode is removed - and an open file descriptor counts as a link. That's also why the function that deletes the directory entry for a file is called unlink() - that's all it does. And yes, a file (inode) can have more than one link pointing to it.
I have a file that i want to add to sourcecontrol on linux using cleartool .
I've followed the IBM documentation for this, i've tried this:
cleartool mkelem testScript.sh
I got an error: Can't modify directory "." because it is not checked out.
I also would like to know how can i checkout/checkin files or directories and setting activities.
You need to checkout the parent folder first.
cd /path/to/file/
cleartool mkact newfile
cleartool checkout -c "add file" .
cleartool mkelem testScript.sh
cleartool checkin -nc
The cleartool mkact would work if you are in an UCM view.
It will create and set a new activity, which will record the files and folder you will modify.
Here, the new activity newFile will record the new version of the parent folder, as well as the version 0 and 1 of the file.
You should create separate questions for .. separate questions...
Going back to the original - the reason why it isn't working is, as VonC has pointed out, you haven't checked out the parent of the file. Remember, when you run "cleartool mkelem", you are about to modify the contents of the parent directory (. in this case) by adding a new "pointer" to the element you're now creating. As with everything else in clearcase, when you want to modify the contents of an element, you have to check it out first.
One of ClearCase's greatest strength (and hardest to wrap one's head around) is the concept of an "element", IMO. "Everything" behaves similarly with an element. Making any change to an "element" (file or directory) means you have to check it out first to make that change.
In the case of a file, that's easy to grasp - you're just editing lines in a file. For a directory, it's almost as easy - you can think of a directory as just a list of pointers to data blobs. We make the name of the blob something convenient we can remember (like foo.java or myapplication.cc or README.md). But we can also change the name of the pointer (even though it points to the same data blob) by renaming a file. We can remove the pointer to the blob without impacting the blob itself by using "rmname". That's essentially what "rmname" does.
In ClearCases' case, the mkelem command is a little bit special - it creates the initial datablob, and adds a pointer to that datablob in the current directory (kind of does 2 things at once).