this is a very specific question
I'm mainly interested in the open() system calls the happen when running touch ..
So I ran strace touch . and saw that opennat() is called three times.
but I'm not really understanding whats going on; as touch . does not print anything in the console and does not create a new file named "." since "." is a pointer to the current folder and can be seen by running ls -a so nothing is created since that name is already in use.
this is my assumption:
open() is called to check if the specified file name already exits, if a file descriptor is returned this means that the name is already in use and the operation is canceled.
please correct me if I'm wrong.
GNU touch prefers to use a file descriptor when touching files, since it's possible to write touch - > foo and expect the file foo to be touched. As a result, it always tries to open the specified path as a writable file, and if that's possible, it then uses that file descriptor to update the file timestamp.
In this case, it's not possible to open . for writing, so openat returns EISDIR. touch notices that it's a directory, so its call to its internal fdutimensat function gets an invalid file descriptor and falls back to using utimensat instead of futimens.
It isn't the case that the openat call is used to check that the file exists, but instead that using a file descriptor for many operations means that you don't have to deal with path resolution multiple times or handle symlinks, since all of those are resolved when the file descriptor is opened. This is why many long-lived programs choose to open a file descriptor to their current working directory, then change directories, and then use the file descriptor with fchdir to change back. Any pchanges to permissions after the program starts are not a problem.
Related
I have an open std::fs::File, and I want to get it's filename, e.g. as a PathBuf. How do I do that?
The simple solution would be to just save the path used in the call to File::open. Unfortunately, this does not work for me. I am trying to write a program that reads log files, and the program that writes the logs keep changing the filenames as part of it's log rotation. So the file may very well have been renamed since it was opened. This is on Linux, so renaming open files is possible.
How do I get around this issue, and get the current filename of an open file?
On a typical Unix filesystem, a file may have multiple filenames at once, or even none at all. The file metadata is stored in an inode, which has a unique inode number, and this inode number can be linked from any number of directory entries. However, there are no reverse links from the inode back to the directory entries.
Given an open File object in Rust, you can get the inode number using the ino() method. If you know the directory the log file is in, you can use std::fs::read_dir() to iterate over all entries in that directory, and each entry will also have an ino() method, so you can find the one(s) matching your open file object. Of course this approach is subject to race conditions – the directory entry may already be gone again once you try to do anything with it.
On linux, files handles held by the current process can be found under /proc/self/fd. These look and act like symlinks to the original files (though I think they may technically be something else - perhaps someone who knows more can chip in).
You can therefore recover the (possibly changed) file name by constructing the correct path in /proc/self/fd using your file descriptor, and then following the symlink back to the filesystem.
This snippet shows the steps:
use std::fs::read_link;
use std::os::unix::io::AsRawFd;
use std::path::PathBuf;
// if f is your std::fs::File
// first construct the path to the symlink under /proc
let path_in_proc = PathBuf::from(format!("/proc/self/fd/{}", f.as_raw_fd()));
// ...and follow it back to the original file
let new_file_name = read_link(path_in_proc).unwrap();
I'm implementing a libfuse filesystem. When a file is opened, I read the file attributes and store them in a hash table keyed with the file handle I generate. This serves two purposes: to maintain a collection of open file handles and to cache the information I retrieve during opening.
Of course, nothing is stopping user code from trying to pass an invalid file handle, i.e. read from a file that's not open.
There are a number of error codes that I can return from the read function, but it's not clear to me which is the one that is expected in such situation.
As you can see in the POSIX standard, the correct value to return would be EBADF:
[EBADF] The fildes argument is not a valid file descriptor open for
reading.
That said, if user code passes an invalid file handle, the Linux kernel will return EBADF to the user before you, or fuse, get any say on the matter.
Background:
On CentOS 7 x86_64. I am using applydeltarpm, and was running out of disk space when creating the new RPM from the delta. I watched the / disk space usage increase during the apply process to 100%, but could not find the working/temp file on the volume using either ls -l /tmp or find /tmp -mmin -1 -type f.
I changed the applydeltarpm source code to use /var/tmp instead of /tmp, and rebuilt the RPM. Now apply works with the modified applydeltarpm, because /var/tmp has a lot more disk space. But I still cannot find the temp file created with mkstemp64.
Question:
The temp file created by mkstemp64 appears to be "non-existent", but still exists as a file descriptor to the creator, and is using considerable disk space when applydeltarpm creates a large RPM (1 hour to apply on a slow disk). The mkstemp64 documentation says an actual file is created. And the source code shows the template file name is /tmp/deltarpmpageXXXXXX. But a file with that template name does not exist.
How is this temporary file able to be created on the system without being findable with the usual directory listing ls, or find. And how can I find these kinds of "non-existent" files in the system?
(I am curious, because I also am monitoring system security)
References:
https://github.com/rpm-software-management/deltarpm/blob/master/applydeltarpm.c
# line 198
if (pagefd < 0)
{
char tmpname[80];
sprintf(tmpname, "/tmp/deltarpmpageXXXXXX");
#ifdef DELTARPM_64BIT
pagefd = mkstemp64(tmpname);
#else
pagefd = mkstemp(tmpname);
#endif
if (pagefd < 0)
{
fprintf(stderr, "could not create page area\n");
exit(1);
}
unlink(tmpname);
}
https://www.mkssoftware.com/docs/man3/mkstemp.3.asp
The mktemp() function returns a unique file name based on the template
parameter. At the time it is generated, no file in the current
directory has that name. No file is actually created, so it is
possible that another application could create a file with this name.
The mkstemp() function is similar to mktemp() in that it create a
unique file name based on template; however, mkstemp() actually
creates the file and returns its file descriptor. The name of the
created file is stored in template.
The mkstemp64() function is identical to the mkstemp() function except
that the file is opened with the O_LARGEFILE flag set.
It is a common practice to unlink the temporary file right after creation, if it's not needed to be accessed through a filesystem anymore. This avoids dangling temporary files if the process crashes or forgets to unlink it later.
unlink() does not delete the file, it only removes the link to the file from the filesystem. Every link to a file in a filesystem increases the link count of the file by one (there can be several links to the same file). Also every process, that calls open() or mmap() to open the file, increases the file count, until it closes the descriptor - then the link count is decreased. File exists as long as there is at least one link to it. When link count reaches zero, then the file is actually deleted.
mkstemp() also calls open() behind the scenes to open the temporary file and return its descriptor.
In order to see the files that are opened, but do not exist in the filesystem anymore, you can use lsof and search for the lines, where there is "(deleted)" after the file name.
lsof | grep '(deleted)'
The (disk)space used by these files will be freed, when the processes they are attached to are finished or close the file descriptor by themselves.
Is it possible to open a file knowing its inode?
ls -i /tmp/test/test.txt
529965 /tmp/test/test.txt
I can provide path, inode (above 529965) and I am looking to get in return a file descriptor.
This is not possible because it would open a loophole in the access control rules. Whether you can open a file depends not only on its own access permission bits, but on the permission bits of every containing directory. (For instance, in your example, if test.txt were mode 644 but the containing directory test were mode 700, then only root and the owner of test could open test.txt.) Inode numbers only identify the file, not the containing directories (it's possible for a file to be in more than one directory; read up on "hard links") so the kernel cannot perform a complete set of access control checks with only an inode number.
(Some Unix implementations have offered nonstandard root-only APIs to open a file by inode number, bypassing some of the access-control rules, but if current Linux has such an API, I don't know about it.)
Not exactly what you are asking, but (as hinted by zwol) both Linux and NetBSD/FreeBSD provide the ability to open files using previously created “handles”: These are inode-like persistent names that identify a file on a file system.
On *BSD (getfh and fhopen) using this is as simple as:
#include <sys/param.h>
#include <sys/mount.h>
fhandle_t file_handle;
getfh("<file_path>", &file_handle); // Or `getfhat` for the *at-style API
// … possibly save handle as bytes somewhere and recreate it some time later …
int fd = fhopen(&file_handle, O_RDWR);
The last call requiring the caller to be root however.
The Linux name_to_handle_at and open_by_handle_at system calls are similar, but a lot more explicit and require the caller to keep track of the relevant file system mount IDs/UUIDs themselves, so I'll humbly link to the detailed example in the manpage instead. Beware, that the example is not complete if you are looking to persist the handles across reboots; one has to convert the received mount ID to a persistent filesystem identifier, such as a filesystem UUID, and convert that back to a mount ID later on. In essence they do the same however. And just like on *BSD using the later system call requires elevated privileges (CAP_DAC_READ_SEARCH to be exact).
I have a program that can be ran several times. The program uses a working directory where it saves/manipulates its runtime files and puts results. I want to make sure that if several copies of the program run simultaneously, they won't use the same folder. To do this I add a hidden file in the work directory when it's created, that means that the directory is being used, and delete it when the program exits. When a program wants to use a certain directory as its working directory, it'll check if that file exists, and if not it will use the directory, otherwise, it'll use a directory of the same name with its process id attached. The implementation is: (in Tcl)
upon starting:
if [file exists [db_work_area]/.folder_used] {
reg set work_area_override [db_work_area]_[pid]
}
...
exec touch ${db_wa}/.folder_used
when exiting:
if [file exists [db_work_area]/.folder_used] {
file delete [db_work_area]/.folder_used
}
This works when the copies of the program are opened one at a time, however I am afraid that if several copies of the program will be opened at the same time, there will be a problem with their synchronization. Meaning that two programs will check if the file exists, together, see that it doesn't both chose that directory, and only after that, they will add the file. How can I implement a semaphore that will be able to synchronize between the several different copies of the same program running?
You should not do a [file exists] and later the touch, it works better to use open to do it in a single step with the EXCL permission.
Try to use something like this to create the file and fail if it already exists in an atomic way.
if {[catch {open ${db_wa}/.folder_used {WRONLY EXCL CREAT}} fd]} {
# error happend, file exists
# pick a different area
} else {
# just close it again, like a touch to create the file
close $fd
}