Is it possible to use the array version of perls system command (i.e. the version that takes the first element as the command and the rest of the array as the cmd arguments) while also spawning a new process with Linux so that the system command returns immediately e.g. to run a command like:
mycmd arg1 arg2 &
I've tried using system( ('mycmd', 'arg1', 'arg2', '&') ) but it just interprets the ampersand literally as a third argument.
I know I can just pass the whole command as a scalar to system but I'm specifically wondering if it's possible to use the array version because the parameters for this command will originate from user options in a CGI script.
The & part of the shell command tells the shell to run the process in the background, so bypassing the shell by using the multi-arg form of system makes no sense.
Solution 1: Quote using String::ShellQuote.
use String:ShellQuote qw( shell_quote );
system(shell_quote('mycmd', 'arg1', 'arg2').' &');
Solution 2: Quote using shell interpolation.
system('sh', '-c', '"$#" &', 'sh', 'mycmd', 'arg1', 'arg2');
Solution 3: Launch the program in the background yourself.
use IPC::Open3 qw( open3 );
{
open(local *CHILD_IN, '<', '/dev/null') or die $!;
local $SIG{CHLD} = 'IGNORE';
open3(
'<&CHILD_IN', '>&STDOUT', '>&STDERR',
'mycmd', 'arg1', 'arg2',
);
}
Since you have no interest in the fate of the executed program you can use fork/exec. And you're on Linux which allows using $SIG{CHLD} = 'IGNORE' to avoid waiting on the child process.
sub background {
local $SIG{CHLD} = 'IGNORE';
# fork and then exec if we are the child
exec(#_) or die($!) unless fork;
}
background( 'mycmd', 'arg1', 'arg2' );
Related
I am trying to escape the "$" symbol when executing a echo $! command in java.
static def execSync(String command) throws Exception {
log.info("exec(" + command + ")")
String[] splited = command.split("\\s+")
def listCommand = Arrays.asList(splited)
ProcessBuilder processBuilder = new ProcessBuilder()
processBuilder.command(listCommand)
return processBuilder.start()
}
execSync("echo \$!") // returns $! when i'd like a pid
I have identified the problem to be in the command that is executed (ie: the code above) and not from my way of getting the stdout of the command (outputstream blabla). If your are absolutely sure it's not, i'll show more.
When i execute on my system (centOs 7) "echo $!" i obviously get a pid, for instance : 2626.
I would mostly like to know if there is a way to do a "echo $!" like on the system with a string in my function ? (regex or other stuff)
Otherwise,
ProcessBuilder.start returns a Process but doesn't seem to have a method to get the pid, only exitReturn, out/in/error stream... Since I execute the previous command with the method shown above, i though i could get the pid with a linux command.
So, is there a way to get the pid of the previous process ? (not realy what i seek but i can manage if there is no other way)
I'm stuck with java 8, when java 9 has a method "getPid"
If you .execute() or use the ProcessBuilder directly, you can not directly use shell features. It just allows to spawn processes with arguments. You have to start a shell and make it execute your shell "script" (command). E.g.
def listCommand = ["/bin/sh", "-c", command]
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
I'm trying to write a Perl program which will execute a bash script. The Perl script looks like this
#!/usr/bin/perl
use diagnostics;
use warnings;
require 'userlib.pl';
use CGI qw(:standard);
ReadParse();
my $q = new CGI;
my $dir = $q->param('X');
my $s = $q->param('Y');
ui_print_header(undef, $text{'edit_title'}.$dir, "");
print $dir."<br>";
print $s."<br>";
print "Under Construction <br>";
use Cwd;
my $pwd = cwd();
my $directory = "/Logs/".$dir."/logmanager/".$s;
my $command = $pwd."/script ".$directory."/".$s.".tar";
print $command."<br>";
print $pwd."<br>";
chdir($directory);
my $pwd1 = cwd();
print $pwd1."<br>";
system($command, $directory) or die "Cannot open Dir: $!";
The script fail with the following error:
Can't exec "/usr/libexec/webmin/foobar/script
/path/filename.tar": No such file or directory at /usr/libexec/webmin/foobar/program.cgi line 23 (#3)
(W exec) A system(), exec(), or piped open call could not execute the
named program for the indicated reason. Typical reasons include: the
permissions were wrong on the file, the file wasn't found in
$ENV{PATH}, the executable in question was compiled for another
architecture, or the #! line in a script points to an interpreter that
can't be run for similar reasons. (Or maybe your system doesn't support #! at all.)
I've checked that the permissions are correct, the tar file I'm passing to my bash script exists, and also tried from the command line to run the same command I'm trying to run from the Perl script ( /usr/libexec/webmin/foobar/script /path/filename.tar ) and it works properly.
In Perl, calling system with one argument (in scalar context) and calling it with several scalar arguments (in list context) does different things.
In scalar context, calling
system($command)
will start an external shell and execute $command in it. If the string in $command has arguments, they will be passed to the call, too. So for example
$command="ls /";
system($commmand);
will evaluate to
sh -c "ls /"
where the shell is given the entire string, i.e. the command with all arguments. Also, the $command will run with all the normal environment variables set. This can be a security issue, see here and here for a few examples why.
On the other hand, if you call system with an array (in list context), Perl will not call a shell and give it the $command as argument, but rather try to execute the first element of the array directly and give it the other arguments as parameters. So
$command = "ls";
$directory = "/";
system($command, $directory);
will call ls directly, without spawning a shell in between.
Back to your question: your code says
my $command = $pwd."/script ".$directory."/".$s.".tar";
system($command, $directory) or die "Cannot open Dir: $!";
Note that $command here is something like /path/to/script /path/to/foo.tar, with the argument already being part of the string. If you call this in scalar context
system($command)
all will work fine, because
sh -c "/path/to/script /path/to/foo.tar"
will execute script with foo.tar as argument. But if you call it in list context, it will try to locate an executable named /path/to/script /path/to/foo.tar, and this will fail.
I found the problem.
changed the system command removing the second parameter and now it's working
system($command) or die "Cannot open Dir: $!";
In fairness I did not understand what was wrong on first example but now works fine, if anyone can explain probably it can be interesting understand
There are multiple ways to execute bash command/ scripts in perl.
System
backquate
exec
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
I'm using Thread::Pool::Simple to create a few working threads. Each working thread does some stuff, including a call to chdir followed by an execution of an external Perl script (from the jbrowse genome browser, if it matters). I use capturex to call the external script and die on its failure.
I discovered that when I use more then one thread, things start to be messy. after some research. it seems that the current directory of some threads is not the correct one.
Perhaps chdir propagates between threads (i.e. isn't thread-safe)?
Or perhaps it's something with capturex?
So, how can I safely set the working directory for each thread?
** UPDATE **
Following the suggestions to change dir while executing, I'd like to ask how exactly should I pass these two commands to capturex?
currently I have:
my #args = ( "bin/flatfile-to-json.pl", "--gff=$gff_file", "--tracklabel=$track_label", "--key=$key", #optional_args );
capturex( [0], #args );
How do I add another command to #args?
Will capturex continue die on errors of any of the commands?
I think that you can solve your "how do I chdir in the child before running the command" problem pretty easily by abandoning IPC::System::Simple as not the right tool for the job.
Instead of doing
my $output = capturex($cmd, #args);
do something like:
use autodie qw(open close);
my $pid = open my $fh, '-|';
unless ($pid) { # this is the child
chdir($wherever);
exec($cmd, #args) or exit 255;
}
my $output = do { local $/; <$fh> };
# If child exited with error or couldn't be run, the exception will
# be raised here (via autodie; feel free to replace it with
# your own handling)
close ($fh);
If you were getting a list of lines instead of scalar output from capturex, the only thing that needs to change is the second-to-last line (to my #output = <$fh>;).
More info on forking-open is in perldoc perlipc.
The good thing about this in preference to capture("chdir wherever ; $cmd #args") is that it doesn't give the shell a chance to do bad things to your #args.
Updated code (doesn't capture output)
my $pid = fork;
die "Couldn't fork: $!" unless defined $pid;
unless ($pid) { # this is the child
chdir($wherever);
open STDOUT, ">/dev/null"; # optional: silence subprocess output
open STDERR, ">/dev/null"; # even more optional
exec($cmd, #args) or exit 255;
}
wait;
die "Child error $?" if $?;
I don't think "current working directory" is a per-thread property. I'd expect it to be a property of the process.
It's not clear exactly why you need to use chdir at all though. Can you not launch the external script setting the new process's working directory appropriately instead? That sounds like a more feasible approach.