As part of a project, the Mono program has to write a series of images to a movie. Therefore the images are first cached in the /tmp/ folder/ since their is a possibility that their are still images of a previous session. I want te remove these images. Therefore I use the following commands:
Process proc = new Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.FileName = "rm";
proc.StartInfo.Arguments = "/tmp/output*";
proc.Start();
proc.WaitForExit();
However when the program is executed I get the following warning: /bin/rm: cannot remove '/tmp/output*': No such file or directory..
However when I executed /bin/rm /tmp/output* in the terminal (in user mode), the command doesn't seem to have a problem recognizing the files.
Why does this command doesn't work?
Spawning an external process for this is terrible. Just use the standard System.IO APIs, for instance:
foreach (var file in Directory.EnumerateFiles ("/tmp", "output*")) {
try {
File.Delete (file);
} catch {
; // optionally report error
}
}
You may also use the overload that takes a SearchOption argument to recursively search in subdirectories. See http://msdn.microsoft.com/en-us/library/dd383571.aspx.
Because you should run the shell to expand the glob pattern /tmp/output* into a an ordered array of file paths.
You could run sh -c "rm /tmp/output*" as a Mono process, but that is ugly
But you don't need a shell. You could for instance use mono readdir to build an array (or a list) of file paths to remove, then remove them by calling a function doing the unlink(2) syscall (I leave you to find how to do that in Mono).
Related
git1 repository have a folder named sports which further have subfolders and files inside it.I want to copy the file given by the output of git diff given by command2.
def command2 = "git diff --stat #{12.hours.ago}"
Process process = command2.execute(null, new File('C:/git1'))
def b=process.text
println b
Output of Groovy Console is Sports/Cricket/Players/Virat.txt.
Now i want to use this file path in sourceDir as shown below.
def sourceDir = "C:/git1/$b"
def destinationDir = "D:/git1/$b"
(new AntBuilder()).copy(file: sourceDir, tofile: destinationDir)
This is giving error as Warning: Could not find file C:\git1\Sports\Cricket\Players\Virat.txt
to copy.
As you can see in your comment with the byte array, the last character of the string is a newline character (10). This of course makes the path invalid but is not easily recognizable in the error message. Add a .trim() after the .text and it should work with both, AntBuilder and Files and on both, Windows and Linux. If you would have done it on Windows there would be a 13 additionally before the 10.
Another maybe even better approach would be not to use git diff as you do, as it is a porcelain command. Porcelain command as their name suggests are not stable and their arguments and output can change any time. They are meant for human usage, not for scripts. Scripts should use plumbing commands which are much more stable in input and output. With the respective plumbing command you might not have had your problem.
I'm not sure why this isn't working, but perhaps I've oversimplified/overcomplicated things
I'm writing a Perl script that ultimately needs to call an external program. The catch is, this program needs a modified version of the LD_LIBRARY_PATH environment variable, in order to find a couple of libraries which the vendor does not install in standard places.
OK, the environment is in %ENV, which can be rewritten, yes?
I thought if I changed LD_LIBRARY_PATH in the parent, it would affect the dynamic linking of the child.
So I have:
use Env qw(#LD_LIBRARY_PATH);
use IPC::System::Simple qw(capturex $EXITVAL);
# We need these to establish the call to rsq later
my ($rsqexe, $rsqhome, $suffix) = fileparse($config->rsq());
push #LD_LIBRARY_PATH, $rsqhome;
eval {
$output = capturex(
$config->rsq(),
qq/"$source"/
);
};
But the child process dies with an error indicating the shared libraries can't be found.
How can I improve this?
I do need to examine the contents of $output after successful execution.
eval {
$output = capturex(
$config->rsq(),
qq/"$source"/
);
};
Here's the problem: I wasn't examining what was in $# (or $EVAL_ERR if you use ENGLISH;)
If I had, I would have seen that the problem was with the quoting qq/"$source"/ - because capturex() doesn't call the shell (which was the desired behaviour) the quotes break the file name (i.e. test.pdf exists, but ""test.pdf"" does not).
I have multiple files with the same pattern: file1.txt, file2.txt, file3.txt, etc. I want to run a java program for each file, something like this:
java Main file[0-9]*.txt
but it doesn't work. Is it possible to do what I want from the terminal? If not, I could change the program to read multiple args, but then again, I'd need to type all 100+ files manually.
As written, the command
java Main file[0-9]*.txt
would pass all of the matching filenames file1.txt, file2.txt, etc., in one command. The OP requested "run a java program for each file", which implies that a series of commands is intended. To do this (in bash or POSIX shell), one could do this:
for file in file[0-9]*.txt; do [ -f "$file" ] && java Main "$file"; done
Breaking it down:
this makes a loop with for file in file[0-9]*.txt using the suggested pattern,
it checks to ensure that the loop variable file has found a file rather than a wildcard expression which found none,
runs the Java class Main for each corresponding file.
I used File::Find to traverse a directory tree and Win32::File's GetAttributes function to look at the attributes of files found in it. This worked in a single-threaded program.
Then I moved the directory traversal into a separate thread, and it stopped working. GetAttributes failed on every file with "The system cannot find the file specified" as the error message in $^E.
I traced the problem to the fact that File::Find uses chdir, and apparently GetAttributes doesn't use the current directory. I could work around this by passing it an absolute path, but then I could run into path length limits, and long paths are definitely going to be present where this script will run, so I really need to take advantage of chdir and relative paths.
To demonstrate the problem, here is a script which creates a file in the current directory, another file in a subdirectory, chdir's to the subdirectory, and looks for the file 3 ways: system("dir"), open, and GetAttributes.
When the script is run without arguments, dir shows the subdirectory, open finds the file in the subdirectory, and GetAttributes returns its attributes successfully. When run with --thread, all the tests are done in a subthread, and the dir and open still work, but the GetAttributes fails. Then it calls GetAttributes on the file that is in the original directory (which we have chdir'ed out of) and it finds that one! Somehow GetAttributes is using the original working directory of the process - or maybe the working directory of the main thread - unlike all the other file operations.
How can I fix this? I can guarantee that the main thread won't do any chdir'ing, if that matters.
use strict;
use warnings;
use threads;
use Data::Dumper;
use Win32::File qw/GetAttributes/;
sub doit
{
chdir("testdir") or die "chdir: $!\n";
system "dir";
my $attribs;
open F, '<', "file.txt" or die "open: $!\n";
print "open succeeded. File contents:\n-------\n", <F>, "\n--------\n";
close F;
my $x = GetAttributes("file.txt", $attribs);
print Dumper [$x, $attribs, $!, $^E];
if(!$x) {
# If we didn't find the file we were supposed to find, how about the
# bad one?
$x = GetAttributes("badfile.txt", $attribs);
if($x) {
print "GetAttributes found the bad file!\n";
if(open F, '<', "badfile.txt") {
print "opened the bad file\n";
close F;
} else {
print "But open didn't open it. Error: $! ($^E)\n";
}
}
}
}
# Setup
-d "testdir" or mkdir "testdir" or die "mkdir testdir: $!\n";
if(!-f "badfile.txt") {
open F, '>', "badfile.txt" or die "create badfile.txt: $!\n";
print F "bad\n";
close F;
}
if(!-f "testdir/file.txt") {
open F, '>', "testdir/file.txt" or die "create testdir/file.txt: $!\n";
print F "hello\n";
close F;
}
# Option 1: do it in the main thread - works fine
if(!(#ARGV && $ARGV[0] eq '--thread')) {
doit();
}
# Option 2: do it in a secondary thread - GetAttributes fails
if(#ARGV && $ARGV[0] eq '--thread') {
my $thr = threads->create(\&doit);
$thr->join();
}
Eventually, I figured out that perl is maintaining some kind of secondary cwd that only applies to perl built-in operators, while GetAttributes is using the native cwd. I don't know why it does this or why it only happens in the secondary thread; my best guess is that perl is trying to emulate the unix rule of one cwd per process, and failing because the Win32::* modules don't play along.
Whatever the reason, it's possible to work around it by forcing the native cwd to be the same as perl's cwd whenever you're about to do a Win32::* operation, like this:
use Cwd;
use Win32::FindFile qw/SetCurrentDirectory/;
...
SetCurrentDirectory(getcwd());
Arguably File::Find should do this when running on Win32.
Of course this only makes the "pathname too long" problem worse, because now every directory you visit will be the target of an absolute-path SetCurrentDirectory; try to work around it with a series of smaller SetCurrentDirectory calls and you have to figure out a way to get back where you came from, which is hard when you don't even have fchdir.
In Unix, it's possible to create a handle to an anonymous file by, e.g., creating and opening it with creat() and then removing the directory link with unlink() - leaving you with a file with an inode and storage but no possible way to re-open it. Such files are often used as temp files (and typically this is what tmpfile() returns to you).
My question: is there any way to re-attach a file like this back into the directory structure? If you could do this it means that you could e.g. implement file writes so that the file appears atomically and fully formed. This appeals to my compulsive neatness. ;)
When poking through the relevant system call functions I expected to find a version of link() called flink() (compare with chmod()/fchmod()) but, at least on Linux this doesn't exist.
Bonus points for telling me how to create the anonymous file without briefly exposing a filename in the disk's directory structure.
A patch for a proposed Linux flink() system call was submitted several years ago, but when Linus stated "there is no way in HELL we can do this securely without major other incursions", that pretty much ended the debate on whether to add this.
Update: As of Linux 3.11, it is now possible to create a file with no directory entry using open() with the new O_TMPFILE flag, and link it into the filesystem once it is fully formed using linkat() on /proc/self/fd/fd with the AT_SYMLINK_FOLLOW flag.
The following example is provided on the open() manual page:
char path[PATH_MAX];
fd = open("/path/to/dir", O_TMPFILE | O_RDWR, S_IRUSR | S_IWUSR);
/* File I/O on 'fd'... */
snprintf(path, PATH_MAX, "/proc/self/fd/%d", fd);
linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file", AT_SYMLINK_FOLLOW);
Note that linkat() will not allow open files to be re-attached after the last link is removed with unlink().
My question: is there any way to re-attach a file like this back into the directory structure? If you could do this it means that you could e.g. implement file writes so that the file appears atomically and fully formed. This appeals to the my compulsive neatness. ;)
If this is your only goal, you can achieve this in a much simpler and more widely used manner. If you are outputting to a.dat:
Open a.dat.part for write.
Write your data.
Rename a.dat.part to a.dat.
I can understand wanting to be neat, but unlinking a file and relinking it just to be "neat" is kind of silly.
This question on serverfault seems to indicate that this kind of re-linking is unsafe and not supported.
Thanks to #mark4o posting about linkat(2), see his answer for details.
I wanted to give it a try to see what actually happened when trying to actually link an anonymous file back into the filesystem it is stored on. (often /tmp, e.g. for video data that firefox is playing).
As of Linux 3.16, there still appears to be no way to undelete a deleted file that's still held open. Neither AT_SYMLINK_FOLLOW nor AT_EMPTY_PATH for linkat(2) do the trick for deleted files that used to have a name, even as root.
The only alternative is tail -c +1 -f /proc/19044/fd/1 > data.recov, which makes a separate copy, and you have to kill it manually when it's done.
Here's the perl wrapper I cooked up for testing. Use strace -eopen,linkat linkat.pl - </proc/.../fd/123 newname to verify that your system still can't undelete open files. (Same applies even with sudo). Obviously you should read code you find on the Internet before running it, or use a sandboxed account.
#!/usr/bin/perl -w
# 2015 Peter Cordes <peter#cordes.ca>
# public domain. If it breaks, you get to keep both pieces. Share and enjoy
# Linux-only linkat(2) wrapper (opens "." to get a directory FD for relative paths)
if ($#ARGV != 1) {
print "wrong number of args. Usage:\n";
print "linkat old new \t# will use AT_SYMLINK_FOLLOW\n";
print "linkat - <old new\t# to use the AT_EMPTY_PATH flag (requires root, and still doesn't re-link arbitrary files)\n";
exit(1);
}
# use POSIX qw(linkat AT_EMPTY_PATH AT_SYMLINK_FOLLOW); #nope, not even POSIX linkat is there
require 'syscall.ph';
use Errno;
# /usr/include/linux/fcntl.h
# #define AT_SYMLINK_NOFOLLOW 0x100 /* Do not follow symbolic links. */
# #define AT_SYMLINK_FOLLOW 0x400 /* Follow symbolic links. */
# #define AT_EMPTY_PATH 0x1000 /* Allow empty relative pathname */
unless (defined &AT_SYMLINK_NOFOLLOW) { sub AT_SYMLINK_NOFOLLOW() { 0x0100 } }
unless (defined &AT_SYMLINK_FOLLOW ) { sub AT_SYMLINK_FOLLOW () { 0x0400 } }
unless (defined &AT_EMPTY_PATH ) { sub AT_EMPTY_PATH () { 0x1000 } }
sub my_linkat ($$$$$) {
# tmp copies: perl doesn't know that the string args won't be modified.
my ($oldp, $newp, $flags) = ($_[1], $_[3], $_[4]);
return !syscall(&SYS_linkat, fileno($_[0]), $oldp, fileno($_[2]), $newp, $flags);
}
sub linkat_dotpaths ($$$) {
open(DOTFD, ".") or die "open . $!";
my $ret = my_linkat(DOTFD, $_[0], DOTFD, $_[1], $_[2]);
close DOTFD;
return $ret;
}
sub link_stdin ($) {
my ($newp, ) = #_;
open(DOTFD, ".") or die "open . $!";
my $ret = my_linkat(0, "", DOTFD, $newp, &AT_EMPTY_PATH);
close DOTFD;
return $ret;
}
sub linkat_follow_dotpaths ($$) {
return linkat_dotpaths($_[0], $_[1], &AT_SYMLINK_FOLLOW);
}
## main
my $oldp = $ARGV[0];
my $newp = $ARGV[1];
# link($oldp, $newp) or die "$!";
# my_linkat(fileno(DIRFD), $oldp, fileno(DIRFD), $newp, AT_SYMLINK_FOLLOW) or die "$!";
if ($oldp eq '-') {
print "linking stdin to '$newp'. You will get ENOENT without root (or CAP_DAC_READ_SEARCH). Even then doesn't work when links=0\n";
$ret = link_stdin( $newp );
} else {
$ret = linkat_follow_dotpaths($oldp, $newp);
}
# either way, you still can't re-link deleted files (tested Linux 3.16 and 4.2).
# print STDERR
die "error: linkat: $!.\n" . ($!{ENOENT} ? "ENOENT is the error you get when trying to re-link a deleted file\n" : '') unless $ret;
# if you want to see exactly what happened, run
# strace -eopen,linkat linkat.pl
Clearly, this is possible -- fsck does it, for example. However, fsck does it with major localized file system mojo and will clearly not be portable, nor executable as an unprivileged user. It's similar to the debugfs comment above.
Writing that flink(2) call would be an interesting exercise. As ijw points out, it would offer some advantages over current practice of temporary file renaming (rename, note, is guaranteed atomic).
Kind of late to the game but I just found http://computer-forensics.sans.org/blog/2009/01/27/recovering-open-but-unlinked-file-data which may answer the question. I haven't tested it, though, so YMMV. It looks sound.