I want to run a script from another script, use the same version of perl, and reroute IO to a terminal-like textbox - linux

I am somewhat familiar with various ways of calling a script from another one. I don't really need an overview of each, but I do have a few questions. Before that, though, I should tell you what my goal is.
I am working on a perl/tk program that: a) gathers information and puts it in a hash, and b) fires off other scripts that use the info hash, and some command line args. Each of these other scripts are available on the command line (using another command-line script) and need to stay that way. So I can't just put all that into a module and call it good.I do have the authority to alter the scripts, but, again, they must also be usable on the command line.
The current way of calling the other script is by using 'do', which means I can pass in the hash, and use the same version of perl (I think). But all the STDOUT (and STDERR too, I think) goes to the terminal.
Here's a simple example to demonstrate the output:
this_thing.pl
#!/usr/bin/env perl
use strict;
use warnings;
use utf8;
use Tk;
my $mw = MainWindow->new;
my $button = $mw->Button(
-text => 'start other thing',
-command => \&start,
)->pack;
my $text = $mw->Text()->pack;
MainLoop;
sub start {
my $script_path = 'this_other_thing.pl';
if (not my $read = do $script_path) {
warn "couldn't parse $script_path: $#" if $#;
warn "couldn't do $script_path: $!" unless defined $read;
warn "couldn't run $script_path" unless $read;
}
}
this_other_thing.pl
#!/usr/bin/env perl
use strict;
use warnings;
use utf8;
print "Hello World!\n";
How can I redirect the STDOUT and STDIN (for interactive scripts that need input) to the text box using the 'do' method? Is that even possible?
If I can't use the 'do' method, what method can redirect the STDIN and STDOUT, as well as enable passing the hash in and using the same version of perl?
Edit: I posted this same question at Perlmonks, at the link in the first comment. So far, the best response seems to use modules and have the child script just be a wrapper for the module. Other possible solutions are: ICP::Run(3) and ICP in general, Capture::Tiny and associated modules, and Tk::Filehandle. A solution was presented that redirects the output and error streams, but seems to not affect the input stream. It's also a bit kludgy and not recommended.
Edit 2: I'm posting this here because I can't answer my own question yet.
Thanks for your suggestions and advice. I went with a suggestion on Perlmonks. The suggestion was to turn the child scripts into modules, and use wrapper scripts around them for normal use. I would then simply be able to use the modules, and all the code is in one spot. This also ensures that I am not using different perls, I can route the output from the module anywhere I want, and passing that hash in is now very easy.

To have both STDIN & STDOUT of a subprocess redirected, you should read the "Bidirectional Communication with Another Process" section of the perlipc man page: http://search.cpan.org/~rjbs/perl-5.18.1/pod/perlipc.pod#Bidirectional_Communication_with_Another_Process
Using the same version of perl works by finding out the name of your perl interpreter, and calling it explicitly. $^X is probably what you want. It may or may not work on different operating systems.
Passing a hash into a subprocess does not work easily. You can print the contents of the hash into a file, and have the subprocess read & parse it. You might get away without using a file, by using the STDIN channel between the two processes, or you could open a separate pipe() for this purpose. Anyway, printing & parsing the data back cannot be avoided when using subprocesses, because the two processes use two perl interpreters, each having its own memory space, and not being able to see each other's variables.
You might avoid using a subprocess, by using fork() + eval() + require(). In that case, no separate perl interpreter will be involved, the forked interpreter will inherit the whole memory of your program with all variables, open file descriptors, sockets, etc. in it, including the hash to be passed. However, I don't see from where your second perl script could get its hash when started from CLI.

Related

Suppressing output after SSH to another server

When I SSH to another server thare are some blurbs of text that always outputs when you log in. (wheather its SSH or just logging in to its own session)
"Authentification banner" is what it prints out every time i either scp a file over or SSH into it.
My code iterates thru a list of servers and sends a file, each time it does that it outputs a lot of text id like to suppress.
This code loops thru each server printing out what its doing.
for(my $j=0; $j < $#servName+1; $j++)
{
print "\n\nSending file: $fileToTransfer to \n$servName[$j]:$targetLocation\n\n";
my $sendCommand = `scp $fileToTransfer $servName[$j]:$targetLocation`;
print $sendCommand;
}
But then it comes out like this:
Sending file: /JacobsScripts/AddAlias.pl to
denamap2:/release/jscripts
====================================================
Welcome authorized users. This system is company
property and unauthorized access or use is prohibited
and may subject you to discipline, civil suit or
criminal prosecution. To the extent permitted by law,
system use and information may be monitored, recorded
or disclosed. Using this system constitutes your
consent to do so. You also agree to comply with applicable
company procedures for system use and the protection of
sensitive (including export controlled) data.
====================================================
Sending file: /JacobsScripts/AddAlias.pl to
denfpev1:/release/jscripts
====================================================
Welcome authorized users. This system is company
property and unauthorized access or use is prohibited
and may subject you to discipline, civil suit or
criminal prosecution. To the extent permitted by law,
system use and information may be monitored, recorded
or disclosed. Using this system constitutes your
consent to do so. You also agree to comply with applicable
company procedures for system use and the protection of
sensitive (including export controlled) data.
====================================================
I havent tried much, i saw a few forums that mention taking the output into a file and then delete it but idk if thatll work for my situation.
NOTE   This answer assumes that on the system in question the ssh/scp messages go to STDERR stream (or perhaps even directly to /dev/tty)†, like they do on some systems I test with -- thus the question.
If not, then ikegami's answer of course takes care of it: just don't print the captured STDOUT. But even in that case, I also think that all ways shown here are better for capturing output (except for the one involving the shell), specially when both streams are needed.
These prints can be suppressed by configuring the server, or perhaps via a .hushlogin file, but then that clearly depends on the server management.
Otherwise, yes you can redirect standard streams to files or, better yet, to variables, what makes the overall management easier.
Using IPC::Run
use IPC::Run qw(run);
my ($file, $servName, $targetLocation) = ...
my #cmd = ("scp", $file, $servName, $targetLocation);
run \#cmd, '1>', \my $out, '2>', \my $err;
# Or redirect both to one variable
# run \#cmd, '>&', \my $out_err;
This mighty and rounded library allows great control over the external processes it runs; it provides almost a mini shell.
Or using the far simpler, and very handy Capture::Tiny
use Capture::Tiny qw(capture);
...
my ($out, $err, $exit) = capture { system #cmd };
Here output can be merged using capture_merged. Working with this library is also clearly superior to builtins (qx, system, pipe-open).
In both cases then inspect $out and $err variables, what is far less cut-and-dry as error messages depend on your system. For some errors the library routines die/croak but for some others they don't but merely print to STDERR. It is probably more reliable to use other tools that libraries provide for detecting errors.
The ssh/scp "normal" (non-error) messages may print to either STDERR or STDOUT stream, or may even go directly to /dev/tty,† so can be mixed with error messages.
Given that the intent seems to be to intersperse these scp commands with other prints then I'd recommend either of these two ways over the others below.
Another option, which I consider least satisfactory overall, is to use the shell to redirect output in the command itself, either to separate files
my ($out_file, $err_file) = ...
system("#cmd 2> $err_file 1> $out_file" ) == 0
or die "system(#cmd...) error: $?"; # see "system" in perldoc
or, perhaps for convenience, both streams can go to one file
system("#cmd > $out_err_file 2>&1" ) == 0 or die $?;
Then inspect files for errors and remove if there is nothing remarkable. Or, shell redirections can be used like in the question but to capture all output
my $out_and_err = qx(#cmd 2>&1);
Then examine the (possibly multiline) variable for errors.
Or, instead of dealing with individual commands we can redirect streams themselves to files for a duration of a larger part of the program
use warnings;
use strict;
use feature 'say';
# Save filehandles ('dup' them) so to be able to reopen later
open my $saveout, ">&STDOUT" or die "Can't dup STDOUT: $!";
open my $saveerr, ">&STDERR" or die "Can't dup STDERR: $!";#]]
my ($outf, $errf) = qw(stdout.txt stderr.txt);
open *STDOUT, ">", $outf or die "Can't redirect STDOUT to $outf: $!";
open *STDERR, ">", $errf or die "Can't redirect STDERR to $errf: $!";
my ($file, $servName, $targetLocation) = ...
my #cmd = ("scp", $file, $servName, $targetLocation);
system(#cmd) == 0
or die "system(#cmd) error: $?"; # see "system" in perldoc
# Restore standard streams when needed for normal output
open STDOUT, '>&', $saveout or die "Can't reopen STDOUT: $!";
open STDERR, '>&', $saveerr or die "Can't reopen STDERR: $!";
# Examine what's in the files (errors?)
I use system instead of qx (operator form of backticks) since there is no need for output from scp. Most of this is covered in open, and search SO for specifics.
It'd be nice to be able to reopen streams to variables but that doesn't work here
† This is even prescribed ("allowed") by POSIX
/dev/tty
In each process, a synonym for the controlling terminal associated with the process group of that process, if any. It is useful for programs or shell procedures that wish to be sure of writing messages to or reading data from the terminal no matter how output has been redirected. It can also be used for applications that demand the name of a file for output, when typed output is desired and it is tiresome to find out what terminal is currently in use.
Courtesy of this superuser post, which has a substiantial discussion.
You are capturing the text, then printing it out using print $sendCommand;. You could simply remove that statement.

How to write to stderr in Nim?

I know there is echo, which writes to stdout. Is it possible to redirect echo to stderr, or is there another way to write to stderr?
It is not possible to redirect echo, but the same can be achieved by using writeLine on the stderr handle (no special imports required):
stderr.writeLine("Error: ", 42)
Documentation links:
writeLine in streams module
stderr in system module
Another way is to call writeLine. It is listed among the system Exports. The docs for echo suggest also calling flushFile as below.
writeLine(stderr, "my err")
flushFile(stderr)
This was tested with Nim 1.4.2.

returning values in a bash function

I'm working with a growing bash script and within this script I have a number of functions. One of these functions is supposed to return a variables value, but I am running into some issues with the syntax. Below is an example of the code.
ShowTags() {
local tag=0
read tag
echo "$tag"
}
selected_tag=$(ShowTags)
echo "$selected_tag"
pulled this code from a Linux Journal article, but the problem is it doesn't seem to work, or perhaps it does and im missing something. Essentially whenever the function is called the script hangs up and does not output anything, I need to CTRL+C to drop back to CLI.
The article in question is below.
http://www.linuxjournal.com/content/return-values-bash-functions
So my question is this the proper way to return a value? Is there a better or more dependable way of doing this? And if there is please give me an example so I can figure this out without using global variables.
EDIT:
The behavior of this is really getting to me now. I am using the following script.
ShowTags() {
echo "hi"
local tag=0
read tag
echo "$tag"
}
selected_tag=$(ShowTags)
echo "$selected_tag
Basically what happens is bash will act as if the read command is taking place before the echo tag at the top of the function. As soon as I pass something to read though it will run the top echo, and complete the rest of the script. I am not sure why this is happening. This is exactly what is happening in my main script.
Change echo "hi" to echo "hi" >/dev/tty.
The reason you're not seeing it immediately is that $(ShowTags) captures all the standard output of the function, and that gets assigned to selected_tag. So you don't see any of it until you echo that variable.
By redirecting the prompt to /dev/tty, it's always displayed immediately on the terminal, not sent to the function's stdout, so it doesn't get captured by the command substitution.
You are trying to define a function with Name { ... ]. You have to use name() { ... }:
ShowTags() { # add ()
local tag=0
read tag
echo "$tag"
} # End with }
selected_tag=$(ShowTags)
echo "$selected_tag"
It now lets the user type in a string and have it written back:
$ bash myscript
hello world # <- my input
hello world # script's output
You can add a prompt with read -p "Enter tag: " tag to make it more obvious when to write your input.
As #thatotherguy pointed out, your function declaration syntax is off; but I suspect that's a transcription error, as if it was wrong in the script you'd get different problems. I think what's going on is that the read tag command in the function is trying to read a value from standard input (by default that's the terminal), and pausing until you type something in. I'm not sure what it's intended to do, but as written I'd expect it to pause indefinitely until something's typed in.
Solution: either type something in, or use something other than read. You could also add a prompt (read -p "Enter a tag: " tag) to make it more clear what's going on.
BTW, I have a couple of objections to the linux journal article you linked. These aren't relevant to your script, but things you should be aware of.
First, the function keyword is a nonstandard bashism, and I recommend against using it. myfunc() ... is sufficient to introduce a function definition.
Second, and more serious, the article recommends using eval in an unsafe way. Actually, it's really hard to use eval safely (see BashFAQ #48). You can improve it a great deal just by changing the quoting, and even more by not using eval at all:
eval $__resultvar="'$myresult'" # BAD, can evaluate parts of $myresult as executable code
eval $__resultvar='"$myresult"' # better, is only vulnerable to executing $__resultvar
declare $__resultvar="$myresult" # better still
See BashFAQ #6 for more options and discussion.

Passing data into perl script from command line

I have a perl script the creates a report based on an xml definition. Currently these definitions all exist as .xml files.
So I have the script run-report.pl, which can take a path to a definition file and create the report.
Now I want to create run-reports-from-db.pl, which will generate the report definition based on same database entries. I don't want to create temp files to pass to run-report.pl, I would just like to pass in the definition somehow.
So instead of saying:
run-report.pl -def=./path/to/def.xml
I want to be able to say:
run-report.pl --stream
And have the report definition available in <STDIN>
I am sure there is pretty trivial way to do this???
If I understand your question correctly, all you need is one | (pipe).
./generate-xml-from-db.pl | ./run-report.pl --stream
Anything the first process in the pipeline prints to stdout will appear in the second process's stdin.
As long as you read from STDIN, you have it available. Notice what happens with you take the code below name it something like echo.pl run it at the command line and paste reams of text.
#!/usr/bin/perl -w
use 5.010;
use strict;
use warnings;
while ( <> ) {
say;
}
<> is the Perl shorthand for "read from STDIN".
As long as the method you're using to launch the process has a way to get a hold of the standard input and outputs, you can just write it to that handle. You have to use the ways that are available to you. In Java, for example, you'd have to get the input stream of the process, in a batch command you have to pipe it. At a GUI terminal you can cut and paste.

Perl program structure for parsing

I've got question about program architecture.
Say you've got 100 different log files with different formats and you need to parse and put that info into an SQL database.
My view of it is like:
use general config file like:
program1->name1("apache",/var/log/apache.log) (modulename,path to logfile1)
program2->name2("exim",/var/log/exim.log) (modulename,path to logfile2)
....
sqldb->configuration
use something like a module (1 file per program) type1.module (regexp, logstructure(somevariables), sql(tables and functions))
fork or thread processes (don't know what is better on Linux now) for different programs.
So question is, is my view of this correct? I should use one module per program (web/MTA/iptablat)
or there is some better way? I think some regexps would be the same, like date/time/ip/url. What to do with that? Or what have I missed?
example: mta exim4 mainlog
2011-04-28 13:16:24 1QFOGm-0005nQ-Ig
<= exim#mydomain.org.ua** H=localhost
(exim.mydomain.org.ua)
[127.0.0.1]:51127 I=[127.0.0.1]:465
P=esmtpsa
X=TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32
CV=no A=plain_server:spam S=763
id=1303985784.4db93e788cb5c#mydomain.org.ua T="test" from
<exim#exim.mydomain.org.ua> for
test#domain.ua
everything that is bold is already parsed and will be putted into sqldb.incoming table. now im having structure in perl to hold every parsed variable like $exim->{timstamp} or $exim->{host}->{ip}
my program will do something like tail -f /file and parse it line by line
Flexability: let say i want to add supprot to apache server (just timestamp userip and file downloaded). all i need to know what logfile to parse, what regexp shoud be and what sql structure should be. So im planning to have this like a module. just fork or thread main process with parameters(logfile,filetype). Maybe further i would add some options what not to parse (maybe some log level is low and you just dont see mutch there)
I would do it like this:
Create a config file that is formatted like this: appname:logpath:logformatname
Create a collection of Perl class that inherit from a base parser class.
Write a script which loads the config file and then loops over its contents, passing each iteration to its appropriate handler object.
If you want an example of steps 1 and 2, we have one on our project. See MT::FileMgr and MT::FileMgr::* here.
The log-monitoring tool wots could do a lot of the heavy lifting for you here. It runs as a daemon, watching as many log files as you could want, running any combination of perl regexes over them and executing something when matches are found.
I would be inclined to modify wots itself (which its licence freely allows) to support a database write method - have a look at its existing handle_* methods.
Most of the hard work has already been done for you, and you can tackle the interesting bits.
I think File::Tail is a nice fit.
You can make an array of File::Tail objects and poll them with select like this:
while (1) {
($nfound,$timeleft,#pending)=
File::Tail::select(undef,undef,undef,$timeout,#files);
unless ($nfound) {
# timeout - do something else here, if you need to
} else {
foreach (#pending) {
# here you can handle log messages depending on filename
print $_->{"input"}." (".localtime(time).") ".$_->read;
}
(from perl File::Tail doc)

Resources