If I've got a handle to an open file, is it possible to create a hard link to that file after all references to it have been removed from the filesystem?
For example, something like this:
fd = fopen("/tmp/foo", "w");
unlink("/tmp/foo");
fwrite(fd, "Hello, world!\n");
create_link_from_fd(fd, "/tmp/hello");
fclose(fd);
Specifically, I'd like to do this so that I can safely write to large data files, then move them into place atomically without having to worry about cleaning up after myself if my program is killed in the middle of writing the file.
The newly released linux 3.11 offers a solution to this problem with the new O_TMPFILE open(2) flag. With this flag you can create an "invisible" file (i.e. an inode with no hardlinks) in some file system (specified by a directory in that file system). Then, after the file is fully set up, you can create a hardlink using linkat. It works like this:
fd = open("/tmp", O_TMPFILE | O_RDWR, 0600);
// write something to the file here
// fchown()/fchmod() it
linkat(fd, "", AT_FDCWD, "/tmp/test", AT_EMPTY_PATH);
Note that aside from the >=3.11 kernel requirement, this also requires support from the underlying file system (I tried the above snippet on ext3 and it worked, but it did not seem to work on btrfs).
Not generally, no. [Edit: since Linux 3.11 there is now linkat; see safsaf32's answer. This does not work on POSIX systems in general since POSIX linkat is restricted to directories only.] There are security considerations here: someone can pass to you an open file descriptor that you could not normally open on your own, e.g.:
mkdir lock; chmod 700 lock
echo secret contents > lock/in
sudoish cmd < lock/in
Here cmd runs as a user who has no permission to open the input file (lock/in) by name, but can still read from it. If cmd could create a new name on the same file system, it could pass the file contents on to a later process. (Obviously it can copy those contents, so this issue is more of a "pass the contents on by mistake" thing than "pass the contents on, on purpose".)
That said, people have come up with ways of "relinking" files by inode/vnode internally (it's pretty easy to do inside most file systems), so you could make your own private system call for it. The descriptor must refer to a real file on the appropriate mount point, of course—there's no way of "relinking" a pipe or socket or device into becoming a regular file.
Otherwise you're stuck with "catch signals and clean up and hope for the best", or a similar trick, "fork off a subprocess, run it, and if it succeeds/fails, take appropriate move/clean-up action".
Edit to add historical note: the above lock example is not particularly good, but back in the days of V6 Unix, MDQS used a fancier version of this trick. Bits and pieces of MDQS survive in various forms today.
On Linux, you might try the unportable trick of using /proc/self/fd by trying to call
char pbuf[64];
snprintf (pbuf, sizeof(pbuf), "/proc/self/fd/%d", fd);
link(pbuf, "/tmp/hello");
I would be surprised if that trick worked after an unlink("/tmp/foo") ... I did not try that.
A more portable (but less robust) way would be to generate a "unique temporary path" perhaps like
int p = (int) getpid();
int t = (int) time(0);
int r = (int) random();
sprintf(pbuf, sizeof(pbuf), "/tmp/out-p%d-r%d-t%d.tmp", p, r, t);
int fd = open (pbuf, O_CREAT|O_WRONLY);
Once the file has been written and closed, you rename(2) it to some more sensible path. You could use atexit in your program to do the renaming (or the removing).
And have some cron job to clean the [old] /tmp/out*.tmp every hour...
Related
I am pretty new to linux kernel.I am trying to generate PWM through linux. The API man talks about a sysfs interface. I want to implement a userspace program in C. But using PWM forces me to use a command line. Furthermore, using read, write is a problem in C as when I am using cd, it is changing path directory.
Thus the path is variable. Is there any way I can pass values to pwm_config() without using sysfs? Perhaps through ioctl? If yes, then what would be the procedure?
Application C code:
void main(){
int export = open("/sys/class/pwm/pmwchip0/export",O_WRONLY);
int period,duty_cycle,enable;
if(export == -1)
{
perror("Export:");
}
and so on for other files like period and duty cycle.
When I try to run my application I get the following error.
Export:: No such file or directory
Export_write: Bad file descriptor
Period_write:: Bad file descriptor
Duty_cycle_write:: Bad file descriptor
Enable_write:: Bad file descriptor
As far as I know, the sysfs is the only standard userspace interface to PWM. But anything you can do from the command line can be done in C (the shell is written in C, after all).
The problem you are having with cd is not actually a problem. Inside sysfs the directories in /sys/class/pwd/* are actually symbolic links to the proper devices. In your case /sys/class/pwm/pwmchip0 is a symlink to /sys/devices/soc0/amba/f8001000.timer/pwm/pwmchip0.
The funny thing is that some shells, when you cd a symbolic link will resolve to the real directory, but other shells will actually keep the symlink name as the current directory.
But that issue with the directory symlinks should not be an issue for you. A C program willing to manage PWM devices should not change the working directory. Instead open the files with the full path:
open("/sys/class/pwm/pwmchip0/npwm", O_RDONLY);
and so on.
I have a rather peculiar file format to work with:
Every line begins with the checksum of its content, followed by a new-line-character.
It looks like this:
[CHECKSUM OF LINE_1][LINE_1]\n
[CHECKSUM OF LINE_2][LINE_2]\n
[CHECKSUM OF LINE_3][LINE_3]\n
...
My goal: To allow any application to work with these files like they would work with any other text file - unaware of the additional checksums at the beginning of each line.
Since I work on a linux machine with debian wheezy (kernel 3.18.26) I want to use the LD_PRELOAD-mechanism to override the relevant file functions.
I have seen something like this with zlibc on https://zlibc.linux.lu/index.html - with an explanation of how it works ( https://zlibc.linux.lu/zlibc.html#SEC8 ).
But I dont get it. They only replace the file-opening functions. No read. No write. no fseek. Nothing. So how does it work?
Or - which functions would I have to intercept to handle every read or write operation on this file and handle them accordingly?
I didn't exactly check how it works but the reason seems to be quite simple.
Possible implementation:
zlibc open:
uncompress file you wanted to open to some temporary file
open this temporary file instead of yours
zlibc close:
Compress temporary file
Override original file
In this case you don't need to override read/write/etc because you can use original ones.
In your case you have two possible solutions:
open, that make a copy of your file with striped checksums. close that calculates checksums and override original file
read and write that are able to skip/calculate checksums.
Ad 2.
From What is the difference between read() and fread()?:
fread() is part of the C library, and provides buffered reads. It is
usually implemented by calling read() in order to fill its buffer
In this case I believe that overriding open and close will be less error prone because you can safely reuse original read, write, fread, fseek etc.
Is it possible to open a file knowing its inode?
ls -i /tmp/test/test.txt
529965 /tmp/test/test.txt
I can provide path, inode (above 529965) and I am looking to get in return a file descriptor.
This is not possible because it would open a loophole in the access control rules. Whether you can open a file depends not only on its own access permission bits, but on the permission bits of every containing directory. (For instance, in your example, if test.txt were mode 644 but the containing directory test were mode 700, then only root and the owner of test could open test.txt.) Inode numbers only identify the file, not the containing directories (it's possible for a file to be in more than one directory; read up on "hard links") so the kernel cannot perform a complete set of access control checks with only an inode number.
(Some Unix implementations have offered nonstandard root-only APIs to open a file by inode number, bypassing some of the access-control rules, but if current Linux has such an API, I don't know about it.)
Not exactly what you are asking, but (as hinted by zwol) both Linux and NetBSD/FreeBSD provide the ability to open files using previously created “handles”: These are inode-like persistent names that identify a file on a file system.
On *BSD (getfh and fhopen) using this is as simple as:
#include <sys/param.h>
#include <sys/mount.h>
fhandle_t file_handle;
getfh("<file_path>", &file_handle); // Or `getfhat` for the *at-style API
// … possibly save handle as bytes somewhere and recreate it some time later …
int fd = fhopen(&file_handle, O_RDWR);
The last call requiring the caller to be root however.
The Linux name_to_handle_at and open_by_handle_at system calls are similar, but a lot more explicit and require the caller to keep track of the relevant file system mount IDs/UUIDs themselves, so I'll humbly link to the detailed example in the manpage instead. Beware, that the example is not complete if you are looking to persist the handles across reboots; one has to convert the received mount ID to a persistent filesystem identifier, such as a filesystem UUID, and convert that back to a mount ID later on. In essence they do the same however. And just like on *BSD using the later system call requires elevated privileges (CAP_DAC_READ_SEARCH to be exact).
Windows (NtQueryObject), Linux (/proc/self/fd/x) and OS X (F_GETPATH) all have methods for retrieving the path of a currently open file descriptor. Supposedly so does FreeBSD via something like the following code:
size_t len;
int mib[4]={CTL_KERN, KERN_PROC, KERN_PROC_FILEDESC, getpid()};
BOOST_AFIO_ERRHOS(sysctl(mib, 4, NULL, &len, NULL, 0));
std::vector<char> buffer(len*2);
BOOST_AFIO_ERRHOS(sysctl(mib, 4, buffer.data(), &len, NULL, 0));
for(char *p=buffer.data(); p<buffer.data()+len;)
{
struct kinfo_file *kif=(struct kinfo_file *) p;
if(kif->kf_fd==fd)
{
lock_guard<pathlock_t> g(pathlock);
_path=path::string_type(kif->kf_path);
return _path;
}
p+=kif->kf_structsize;
}
This irritatingly works for pretty much every kind of file descriptor except regular files which return a null path, at least in FreeBSD 10. I assume this is an oversight as from examining the kernel code it would seem trivial to return the path, though maybe there were performance reasons not to do so.
procstat uses the same API as the above, and therefore also too only returns paths for everything but regular files. Using statfs() I can at least get the path of the mount point for the file, but retrieving the path fragment between the mount point and the actual file is the problem.
So let me ask this question: Is it possible to ask UFS or ZFS to return a path fragment from an inode, maybe using some magic ioctl or sysctl, or even some utility library? The code in question doesn't need the path the file descriptor was opened with, it only needs some canonical path the file descriptor can currently be found at (this is to handle tracking third party file renames for proposed Boost.AFIO).
My thanks in advance for any help. FreeBSD is the only showstopper out of the major operating systems for this feature, and without it it makes writing filing system race safety detection code impossible :(
Edit: I found a discussion about getting ZFS to convert inodes to paths at http://comments.gmane.org/gmane.os.solaris.opensolaris.zfs/38277. Apparently there is a ZFS_IOC_OBJ_TO_PATH ioctl, but this is not a well trodden code path judging from google
It turns out that the fact that KERN_PROC_FILEDESC doesn't work is in fact a bug in the FreeBSD kernel. I've logged it at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197695.
KERN_PROC_FILEDESC does work perfectly for directory file descriptors, including tracking renames and deletions. It imperfectly works for regular file descriptors, sometimes returning a path and then quite suddenly returning a null path, and then randomly later returning a path again. I think somewhere in the 10.x release something got broken somewhere :(
Hope that helps anyone else similarly stuck.
I'm trying to open a file for rewriting. I then close the file, and reopen it for read to validate it was written OK. It is indeed as it should be. But, after I unplug the unit (ARM) and plug it again, I find that the file becomes empty. I also tried copying the file manually (with cp), and the same phenomenon reoccurs.
here is some code:
string fileName = "/home/root/LogiTrackV2/InitialSetup.xml";
ofstream theFile (fileName.c_str());
if (theFile.is_open())
{
theFile.close();
}
theFile.open(fileName.c_str(), ios::out | ios::trunc);
theFile << xmlOUT.c_str();
theFile.close();
As I mentioned after this the file exist and updated as it should. The problem is when I unplug the unit...
The problem is more complex than I thought in C++. There is no way in the standard library to force a POSIX fsync call on an ofstream. You can however use Boost.Iostreams with a file_descriptor_sink (http://www.boost.org/doc/libs/1_55_0/libs/iostreams/doc/classes/file_descriptor.html) and do an fsync on the provided fd to force Linux into writing the file to disk.