It is very strange, i have many script like following and all running in crontab but following script running on command (./load.pl) line but not inside crontab
crontab:
0-59/5 * * * * /home/spatel/rrd/load.pl >> /tmp/load.out
Notes:
I also tried following method
0-59/5 * * * * /usr/bin/perl /home/spatel/rrd/load.pl >> /tmp/load.out
0-59/5 * * * * root /usr/bin/perl /home/spatel/rrd/load.pl >> /tmp/load.out
Somewhere i read cron ignore newline end of the script so i look care of that too
I have put print in script and redirect to /tmp/load.out i can see that output in load.out when cron execute but somehow it is not updating data in side load.rrd file.
If i run script on command like ./load.pl it works! but not inside crontab.
I have set crontab PATH whatever root use has. I tried all possible way to debug but it is not running inside cron. Here is the place where i get this script, all other script working file in crontab only following one has issue :( https://github.com/mmitch/rrd
Script:
#!/usr/bin/env perl
#
# RRD script to display system load
# 2003,2011 (c) by Christian Garbs <mitch#cgarbs.de>
# Licensed under GNU GPL.
#
# This script should be run every 5 minutes.
#
# *ADDITIONALLY* data aquisition is done externally every minute:
# rrdtool update $datafile N:$( PROCS=`echo /proc/[0-9]*|wc -w|tr -d ' '`; read L1 L2 L3 DUMMY < /proc/loadavg ; echo ${L1}:${L2}:${L3}:${PROCS} )
#
use strict;
use warnings;
#use 5.010;
use RRDs;
# parse configuration file
my %conf;
eval(`/bin/cat /home/spatel/rrd/rrd-conf.pl`);
die $# if $#;
# set variables
my $datafile = "/home/spatel/rrd/db/load.rrd";
my $picbase = "/var/www/mrtg/rrd/load-";
# global error variable
my $ERR;
# whoami?
my $hostname = `/bin/hostname`;
chomp $hostname;
# generate database if absent
if ( ! -e $datafile ) {
# max 70000 for all values
RRDs::create($datafile,
"--step=60",
"DS:load1:GAUGE:120:0:70000",
"DS:load2:GAUGE:120:0:70000",
"DS:load3:GAUGE:120:0:70000",
"DS:procs:GAUGE:120:0:70000",
"RRA:AVERAGE:0.5:1:120",
"RRA:AVERAGE:0.5:5:600",
"RRA:AVERAGE:0.5:30:700",
"RRA:AVERAGE:0.5:120:775",
"RRA:AVERAGE:0.5:1440:797",
"RRA:MAX:0.5:1:120",
"RRA:MAX:0.5:5:600",
"RRA:MAX:0.5:6:700",
"RRA:MAX:0.5:120:775",
"RRA:MAX:0.5:1440:797",
"RRA:MIN:0.5:1:120",
"RRA:MIN:0.5:5:600",
"RRA:MIN:0.5:6:700",
"RRA:MIN:0.5:120:775",
"RRA:MIN:0.5:1440:797"
);
$ERR=RRDs::error;
die "ERROR while creating $datafile: $ERR\n" if $ERR;
print "created $datafile\n";
}
# data aquisition is done externally every minute:
my #procs = glob '/proc/[0-9]*';
my $file = '/proc/loadavg';
open my $fh, '<', $file or die "Failed to open '$file': $!";
my $load = <$fh>;
my $p = (scalar #procs);
my $l = (join ':', (split ' ', $load)[0..2]);
print "${l}:${p}";
# update rrd
RRDs::update($datafile,
"N:${l}:${p}"
);
$ERR=RRDs::error;
die "ERROR while updating $datafile: $ERR\n" if $ERR;
# draw pictures
foreach ( [3600, "hour"], [86400, "day"], [604800, "week"], [31536000, "year"] ) {
my ($time, $scale) = #{$_};
RRDs::graph($picbase . $scale . ".png",
"--start=-${time}",
'--lazy',
'--imgformat=PNG',
"--title=${hostname} system load (last $scale)",
"--width=$conf{GRAPH_WIDTH}",
"--height=$conf{GRAPH_HEIGHT}",
'--slope-mode',
'--alt-autoscale',
"DEF:load1=${datafile}:load1:AVERAGE",
"DEF:load2=${datafile}:load2:AVERAGE",
"DEF:load3=${datafile}:load3:AVERAGE",
"DEF:procsx=${datafile}:procs:AVERAGE",
"DEF:procminx=${datafile}:procs:MIN",
"DEF:procmaxx=${datafile}:procs:MAX",
'CDEF:procs=procsx,100,/',
'CDEF:procmin=procminx,100,/',
'CDEF:procrange=procmaxx,procminx,-,100,/',
'AREA:procmin',
'STACK:procrange#E0E0E0',
'AREA:load3#000099:loadavg3',
'LINE2:load2#0000FF:loadavg2',
'LINE1:load1#9999FF:loadavg1',
'COMMENT:\n',
'LINE1:procs#000000:processes/100',
);
$ERR=RRDs::error;
die "ERROR while drawing $datafile $time: $ERR\n" if $ERR;
}
Update:
Here is the output of script:
[root#spatel tmp]# pwd
/tmp
[root#spatel tmp]# /home/spatel/rrd/load.pl
0.15:0.06:0.01:664
Its most likely an environment problem.
A "cron" scripts starts of with the default system environment (Default PATH, LANG etc.) your .profile, .rc are NOT executed.
So you need to provide all the environment variables PATHs etc. your program requires in the script. This is a bit of a pain for a pure perl script so its probably better to wrap it in a shell script which sets whatever your ".profile" script sets.
Solved:
You won't believe how problem got resolved, I use perl instead of /usr/bin/perl in crontab, I though its best practice to use full PATH in crontab but it prove wrong, Still i don't know why and How?
0-59/5 * * * * perl /home/spatel/rrd/load.pl
Related
I'm trying to develop a perl script that looks through all of the user's directories for a particular file name without the user having to specify the entire pathname to the file.
For example, let's say the file of interest was data.list. It's located in /home/path/directory/project/userabc/data.list. At the command line, normally the user would have to specify the pathname to the file like in order to access it, like so:
cd /home/path/directory/project/userabc/data.list
Instead, I want the user just to have to enter script.pl ABC in the command line, then the Perl script will automatically run and retrieve the information in the data.list. which in my case, is count the number of lines and upload it using curl. the rest is done, just the part where it can automatically locate the file
Even though very feasible in Perl, this looks more appropriate in Bash:
#!/bin/bash
filename=$(find ~ -name "$1" )
wc -l "$filename"
curl .......
The main issue would of course be if you have multiple files data1, say for example /home/user/dir1/data1 and /home/user/dir2/data1. You will need a way to handle that. And how you handle it would depend on your specific situation.
In Perl that would be much more complicated:
#! /usr/bin/perl -w
eval 'exec /usr/bin/perl -S $0 ${1+"$#"}'
if 0; #$running_under_some_shell
use strict;
# Import the module File::Find, which will do all the real work
use File::Find ();
# Set the variable $File::Find::dont_use_nlink if you're using AFS,
# since AFS cheats.
# for the convenience of &wanted calls, including -eval statements:
# Here, we "import" specific variables from the File::Find module
# The purpose is to be able to just type '$name' instead of the
# complete '$File::Find::name'.
use vars qw/*name *dir *prune/;
*name = *File::Find::name;
*dir = *File::Find::dir;
*prune = *File::Find::prune;
# We declare the sub here; the content of the sub will be created later.
sub wanted;
# This is a simple way to get the first argument. There is no
# checking on validity.
our $filename=$ARGV[0];
# Traverse desired filesystem. /home is the top-directory where we
# start our seach. The sub wanted will be executed for every file
# we find
File::Find::find({wanted => \&wanted}, '/home');
exit;
sub wanted {
# Check if the file is our desired filename
if ( /^$filename\z/) {
# Open the file, read it and count its lines
my $lines=0;
open(my $F,'<',$name) or die "Cannot open $name";
while (<$F>){ $lines++; }
print("$name: $lines\n");
# Your curl command here
}
}
You will need to look at the argument-parsing, for which I simply used $ARGV[0] and I do dont know what your curl looks like.
A more simple (though not recommended) way would be to abuse Perl as a sort of shell:
#!/usr/bin/perl
#
my $fn=`find /home -name '$ARGV[0]'`;
chomp $fn;
my $wc=`wc -l '$fn'`;
print "$wc\n";
system ("your curl command");
Following code snippet demonstrates one of many ways to achieve desired result.
The code takes one parameter, a word to look for in all subdirectories inside file(s) data.list. And prints out a list of found files in a terminal.
The code utilizes subroutine lookup($dir,$filename,$search) which calls itself recursively once it come across a subdirectory.
The search starts from current working directory (in question was not specified a directory as start point).
use strict;
use warnings;
use feature 'say';
my $search = shift || die "Specify what look for";
my $fname = 'data.list';
my $found = lookup('.',$fname,$search);
if( #$found ) {
say for #$found;
} else {
say 'Not found';
}
exit 0;
sub lookup {
my $dir = shift;
my $fname = shift;
my $search = shift;
my $files;
my #items = glob("$dir/*");
for my $item (#items) {
if( -f $item && $item =~ /\b$fname\b/ ) {
my $found;
open my $fh, '<', $item or die $!;
while( my $line = <$fh> ) {
$found = 1 if $line =~ /\b$search\b/;
if( $found ) {
push #{$files}, $item;
last;
}
}
close $fh;
}
if( -d $item ) {
my $ret = lookup($item,$fname,$search);
push #{$files}, $_ for #$ret;
}
}
return $files;
}
Run as script.pl search_word
Output sample
./capacitor/data.list
./examples/data.list
./examples/test/data.list
Reference:
glob,
Perl file test operators
I have a giant crontab with many scripts on a linux machine. I need to be able to a) change the subject and/or from of cronjob result emails, because the default is unreadably long. b) Do so via a centralized solution. c) Only require minimal changes to the crontab itself.
For example this crontab line:
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3
Creates this email subject:
Cron <cronuser#myserver> /path/to/script1 | /path/to/script2 | /path/to/script3
Which in my inbox gets cut off somewhere in the path to script1. (And many lines in the crontab are significantly longer.)
Options I've tried:
Piping to mail and setting subject etc per line (The -E preserves cron's default only-send-on-output behavior):
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | mail -E -s "test subject" -S from="Cron Script2 <cronuser#myserver.com>" recipient#myserver.com
This "works", but I want to centralize my changes in one place, and minimize how much I add to each cron line for readability
Using shell : (noop) command, which shows up first in the subject (note that the space after the : is important!):
0 */3 * * * : Descriptive Words; /path/to/script1 | /path/to/script2 | /path/to/script3
Unfortunately there's still too much unnecessary that cron puts before "Descriptive Words" on the subject line, so this is still unusable.
What I want to create:
Something generic, like this:
0 */3 * * * /path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | coolmailer.pl
coolmailer.pl would build the subject line by getting the commands on that cron line, strip out the paths and arguments, and email me this (optionally only if there's a fail in any of the scripts):
SUBJECT: script1 | script2 | script3
FROM: Cron Script3<noreply#myserver.com>
(actual results of the command /path/to/script1 | /path/to/script2 | /path/to/script3)
As a bonus, I'd also love to say whether any of the previous commands (script1 or script2) failed on the subject line.
This has turned out to be... way more complicated than I expected.
Challenges:
Find a way for a pipeline member (coolmailer) to know the other
members of the pipeline.
There's an ingenious method for 1 here using lsof how-do-you-determine-the-actual-command-that-is-piping-into-you but it also sometimes finds commands started by the scripts in my pipeline (ie if script1 forks processes or does system calls, those show up too any time they take long enough to complete.) Ditto for the method using process groups at the same link.
Find a way for a pipeline member (coolmailer) to know the results of
other members of the pipeline. (I realize this may not be possible at all, but the lsof hack gives me hope.)
Any better way? Does the fact that I'm running from cron buy me anything? Part of me wants to combine the lsof strategy with grepping through crontab -l results, but that just seems too kludgy and prone to errors.
Caveats:
I can have changes made to my account, but I can't make changes that
would effect all users. I.e. if there's a way to change cron's
mailing format server-wide, that doesn't help.
I can't realistically update every script called to handle emailing its own results, even if that's probably the "right" way.
I know about the mail -s -E -S options, but would prefer to have a single place to change things. Also, I really want to find a way to get the pipeline.
Language used now for "coolmailer" is Perl, but I'll try anything
My first attempt:
(which works, except it often also shows other commands started inside my scripts, which means it doesn't work)
#!/usr/bin/perl -w
my $pgid=`ps -o pgid= -p $$`;
my $lsofout = `/usr/sbin/lsof -g $pgid`;
my #otherpids = `echo "$lsofout" | awk '\$5 == "1w" { print \$2 }'`;
my #longcmds;
my #shortcmds;
foreach my $pid (#otherpids) {
chomp($pid);
if (my $cmd = `ps -o cmd= -p $pid 2>/dev/null`) {
chomp($cmd);
push #longcmds, $cmd;
next;
}
}
my $cmdline = join (' | ',#longcmds);
foreach my $cmd (#longcmds) {
$cmd =~ s/(\/\S+\/)(\S+)/$2/g;
push #shortcmds, $cmd;
}
my $subj = join('|',#shortcmds);
print "SUBJ:$subj\n";
print "CMDLINE: $cmdline\n";
# and now do some mail stuff
And final version, based on suggestion by Jhnc
#!/usr/bin/perl -w
# cronmgr.pl -- understand cron emails for once
# usage: 0 */3 * * * cronmgr.pl cd blah\; /path/to/script1 \| /path/to/script2 \| /path/to/script3
# note that ; | & in any cronmgr.pl line must be backslashed to run!
use strict;
use IPC::Cmd qw[can_run run run_forked];
my $CMDLINE = join(' ',#ARGV);
my( $success, $error_message, $full_buf, $stdout_buf, $stderr_buf ) =
run( command => $CMDLINE, verbose => 0 ); # verbose = 0 means don't output normally, capture all output
my ($stdout, $stderr);
$stdout = join "", #$stdout_buf;
$stderr = join "", #$stderr_buf;
my $emailsubject;
if( $success ) {
if ($stdout eq '' && $stderr eq '') { # if there's no output, don't send any email!
exit;
}
} else {
print "CMD FAIL!\n$error_message\nSTDERR:\n$stderr";
$emailsubject = "FAIL:$error_message";
}
# etc etc
(Edited for clarity re goals and why options attempted so far aren't sufficient.)
determining the pipeline
If you always invoke coolmailer.pl with a unique argument then you can simply grep it from your list of cronjobs:
#!/usr/bin/perl -wT
$ENV{PATH} = '/sensible:/path';
my ($pipeline) = grep /\|\s+$0\s+$ARGV[0]/, `crontab -l`;
$pipeline ||= "oops";
# ... mung $pipeline ...
# ... do mail stuff ...
checking pipe failure
If you rewrite your cronjob entries from:
/path/to/script1 | /path/to/script2 | /path/to/script3 2>&1 | coolmailer.pl
to:
coolermailer /path/to/script1 \| /path/to/script2 \| /path/to/script3
then you could construct the pipeline manually and have control over pipe member status information. (This also gives you the pipeline directly, although you then have to construct it before it will run.)
For example, with a bash implementation, you might make use of eval and PIPESTATUS. With Perl, you might use results() from IPC::Run
The following Perl script generates an .xls file from a text file. It runs great in our linux test environment, but generates an empty spreadsheet (.xls) in our production environment when run via cron (cron works in test, as well.) Nothing jumps out at our sys admins in terms of system level settings that might account for this behavior. Towards the bottom of the script in the import_data subroutine, the correct number of lines is reported, but nothing is written to the spreadsheet and no errors are returned at either the script or system level. I ran it through the perl debugger but my skills fell short of being able to interactively watch it populate the file. The cron entry looks like this:
cd <script directory>; cvs2xls input.txt output.xls 2>&1
Any debugging tips would be appreciated, as well as potential system settings that I can forward on to our sysadmins.
#!/usr/bin/perl
use strict;
use warnings;
use lib '/apps/tu01688/perl5/lib/perl5';
use Spreadsheet::WriteExcel;
use Text::CSV::Simple;
BEGIN {
unshift #INC, "/apps/tu01688/jobs/mayo-expert";
};
my $infile = shift;
usage() unless defined $infile && -f $infile;
my $parser = Text::CSV::Simple->new;
my #data = $parser->read_file($infile);
my $headers = shift #data;
my $outfile = shift || $infile . ".xls";
my $subject = shift || 'worksheet';
sub usage {
print "csv2xls infile [outfile] [subject]\n";
exit;
}
my $workbook = Spreadsheet::WriteExcel->new($outfile);
my $bold = $workbook->add_format();
$bold->set_bold(1);
import_data($workbook, $subject, $headers, \#data);
# Add a worksheet
sub import_data {
my $workbook = shift;
my $base_name = shift;
my $colums = shift;
my $data = shift;
my $limit = shift || 50_000;
my $start_row = shift || 1;
my $worksheet = $workbook->add_worksheet($base_name);
$worksheet->add_write_handler(qr[\w], \&store_string_widths);
#$worksheet->add_write_handler(qr[\w]| \&store_string_widths);
my $w = 1;
$worksheet->write('A' . $start_row, $colums, ,$bold);
my $i = $start_row;
my $qty = 0;
for my $row (#$data) {
$qty++;
if ($i > $limit) {
$i = $start_row;
$w++;
$worksheet = $workbook->add_worksheet("$base_name - $w");
$worksheet->write('A1', $colums,$bold);
}
$worksheet->write($i++, 0, $row);
}
autofit_columns($worksheet);
warn "Converted $qty rows.";
return $worksheet;
}
###############################################################################
###############################################################################
#
# Functions used for Autofit.
#
###############################################################################
#
# Adjust the column widths to fit the longest string in the column.
#
sub autofit_columns {
my $worksheet = shift;
my $col = 0;
for my $width (#{$worksheet->{__col_widths}}) {
$worksheet->set_column($col, $col, $width) if $width;
$col++;
}
}
###############################################################################
#
# The following function is a callback that was added via add_write_handler()
# above. It modifies the write() function so that it stores the maximum
# unwrapped width of a string in a column.
#
sub store_string_widths {
my $worksheet = shift;
my $col = $_[1];
my $token = $_[2];
# Ignore some tokens that we aren't interested in.
return if not defined $token; # Ignore undefs.
return if $token eq ''; # Ignore blank cells.
return if ref $token eq 'ARRAY'; # Ignore array refs.
return if $token =~ /^=/; # Ignore formula
# Ignore numbers
#return if $token =~ /^([+-]?)(?=\d|\.\d)\d*(\.\d*)?([Ee]([+-]?\d+))?$/;
# Ignore various internal and external hyperlinks. In a real scenario
# you may wish to track the length of the optional strings used with
# urls.
return if $token =~ m{^[fh]tt?ps?://};
return if $token =~ m{^mailto:};
return if $token =~ m{^(?:in|ex)ternal:};
# We store the string width as data in the Worksheet object. We use
# a double underscore key name to avoid conflicts with future names.
#
my $old_width = $worksheet->{__col_widths}->[$col];
my $string_width = string_width($token);
if (not defined $old_width or $string_width > $old_width) {
# You may wish to set a minimum column width as follows.
#return undef if $string_width < 10;
$worksheet->{__col_widths}->[$col] = $string_width;
}
# Return control to write();
return undef;
}
###############################################################################
#
# Very simple conversion between string length and string width for Arial 10.
# See below for a more sophisticated method.
#
sub string_width {
return length $_[0];
}
Uhmm.. don't put chained commands in cron, use an external script instead. Anyway: some suggestions that may help you:
Debugging cron commands
Check the mail! By default cron will mail any output from the command to the user it is running the command as. If there is no output there will be no mail. If you want cron to send mail to a different account then you can set the MAILTO environment variable in the crontab file e.g.
MAILTO=user#somehost.tld
1 2 * * * /path/to/your/command
Capture the output yourself
1 2 * * * /path/to/your/command &>/tmp/mycommand.log
which captures stdout and stderr to /tmp/mycommand.log
Look at the logs; cron logs its actions via syslog, which (depending on your setup) often go to /var/log/cron or /var/log/syslog.
If required you can filter the cron statements with e.g.
grep CRON /var/log/syslog
Now that we've gone over the basics of cron, where the files are and how to use them let's look at some common problems.
Check that cron is running
If cron isn't running then your commands won't be scheduled ...
ps -ef | grep cron | grep -v grep
should get you something like
root 1224 1 0 Nov16 ? 00:00:03 cron
or
root 2018 1 0 Nov14 ? 00:00:06 crond
If not restart it
/sbin/service cron start
or
/sbin/service crond start
There may be other methods; use what your distro provides.
cron runs your command in a restricted environment.
What environment variables are available is likely to be very limited. Typically, you'll only get a few variables defined, such as $LOGNAME, $HOME, and $PATH.
Of particular note is the PATH is restricted to /bin:/usr/bin. The vast majority of "my cron script doesn't work" problems are caused by this restrictive path. If your command is in a different location you can solve this in a couple of ways:
Provide the full path to your command.
1 2 * * * /path/to/your/command
Provide a suitable PATH in the crontab file
PATH=/usr:/usr/bin:/path/to/something/else
1 2 * * * command
If your command requires other environment variables you can define them in the crontab file too.
cron runs your command with cwd == $HOME
Regardless of where the program you execute resides on the filesystem, the current working directory of the program when cron runs it will be the user's home directory. If you access files in your program, you'll need to take this into account if you use relative paths, or (preferably) just use fully-qualified paths everywhere, and save everyone a whole lot of confusion.
The last command in my crontab doesn't run
Cron generally requires that commands are terminated with a new line. Edit your crontab; go to the end of the line which contains the last command and insert a new line (press enter).
Check the crontab format
You can't use a user crontab formatted crontab for /etc/crontab or the fragments in /etc/cron.d and vice versa. A user formatted crontab does not include a username in the 6th position of a row, while a system formatted crontab includes the username and runs the command as that user.
I put a file in /etc/cron.{hourly,daily,weekly,monthly} and it doesn't run
Check that the filename doesn't have an extension see run-parts
Ensure the file has execute permissions.
Tell the system what to use when executing your script (eg. put #!/bin/sh at top)
Cron date related bugs
If your date is recently changed by a user or system update, timezone or other, then crontab will start behaving erratically and exhibit bizarre bugs, sometimes working, sometimes not. This is crontab's attempt to try to "do what you want" when the time changes out from underneath it. The "minute" field will become ineffective after the hour is changed. In this scenario, only asterisks would be accepted. Restart cron and try it again without connecting to the internet (so the date doesn't have a chance to reset to one of the time servers).
Percent signs, again
To emphasise the advice about percent signs, here's an example of what cron does with them:
# cron entry
* * * * * cat >$HOME/cron.out%foo%bar%baz
will create the ~/cron.out file containing the 3 lines
foo
bar
baz
This is particularly intrusive when using the date command. Be sure to escape the percent signs
* * * * * /path/to/command --day "$(date "+\%Y\%m\%d")"
Thanks so much for the extensive feedback, everyone. I'm certainly taking a lot more away from this than I put into it. In any event, I ran across the answer. In my perl5 lib folder I found that somehow the IO and OLE libraries were missing on production. Copying those over from development resulted in everything working fine. The fact that I was unable to determine/capture this through conventional debugging efforts as opposed to merely comparing directory listings out of exasperation speaks to how much more I have to learn along these lines. But I'm confident that the great feedback I received will go a long ways towards getting me there. Thanks again, everyone.
I want to download thousand of files from a URL. Each line in "FileName.txt" contains the name of file to download. I am using a Perl script to take the file name from "FileName.txt" and downloading them after a random time. I run script as "./program.pl Filename.txt"
Filename.txt
A
B
C
B
program.pl
#!/usr/bin/perl
$file1=$ARGV[0];
open(FP1, $file1);
while($s1=<FP1>)
<br>
{ chomp ($s1);
$range = 5;
$minimum = 3;
$random_number = int(rand($range)) + $minimum;
`wget --wait="$random_number" "http://URL=$s1"`;
}
I am getting the output for few initial file but not for remaining file. For remaining file $ emacs fileD.txt give
[13] 29699
Could you kindly tell me why I am getting "[13] 29699", and what is the best way to download file after random time interval. Sorry, the program at while does not show the correct handler. Thanks
You don't show where $id comes from, but presumably some URLs contain & which puts the process in the background. You should use single quotes for wget's argument or use the list form of system.
Further, wget's wait parameter is only relevant if your are using wget itself to traverse links from a given URL. In your case, you need your Perl script to sleep between invoking wget for each URL:
#!/usr/bin/env perl
use strict;
use warnings;
use constant WAIT_MINIMUM => 3;
use constant WAIT_RANGE => 5;
my ($url_list_file) = #ARGV;
defined($url_list_file)
or die "Need URL list\n";
open my $fh, '<', $url_list_file
or die "Cannot open '$url_list_file': $!";
while (my $url = <$fh>) {
$url =~ s/\R\z//;
my #cmd = (wget => 'http://$url');
print "#cmd\n";
my $error = system #cmd;
if ($error) {
warn "''#cmd' failed: $?";
}
sleep WAIT_MINIMUM + rand(WAIT_RANGE);
}
What means URL=? wget takes url as simple paramter. Seems to be you need
`wget --wait=$random_number 'http://$s1'`;
Is there a way to parse input and output from bash commands in an interactive terminal before they reach the screen ? I was thinking maybe something in .bashrc, but I'm new to using bash.
For example:
I type "ls /home/foo/bar/"
That gets passed through a script that replaces all instances of 'bar' with 'eggs'
"ls /home/foo/eggs/" gets executed
The output gets sent back to the replace script
The output of the script is sent to the screen
Yes. Here's something I wrote for my own use, to wrap old command line Fortran programs that ask for file-paths. It allows escaping back to the shell for e.g. running 'ls'. This only works one way, i.e. intercepts user-input and then passes it on to a program, but gets you most of what you want. You can adapt it to your needs.
#!/usr/bin/perl
# shwrap.pl - Wrap any process for convenient escape to the shell.
# ire_and_curses, September 2006
use strict;
use warnings;
# Check args
my $executable = shift || die "Usage: shwrap.pl executable";
my #escape_chars = ('#'); # Escape to shell with these chars
my $exit = 'exit'; # Exit string for quick termination
open my $exe_fh, "|$executable #ARGV" or die "Cannot pipe to program $executable: $!";
# Set magic buffer autoflush on...
select((select($exe_fh), $| = 1)[0]);
# Accept input until the child process terminates or is terminated...
while ( 1 ) {
chomp(my $input = <STDIN>);
# End if we receive the special exit string...
if ( $input =~ m/$exit/ ) {
close $exe_fh;
print "$0: Terminated child process...\n";
exit;
}
foreach my $char ( #escape_chars ) {
# Escape to the shell if the input starts with an escape character...
if ( my ($command) = $input =~ m/^$char(.*)/ ) {
system $command;
}
# Otherwise pass the input on to the executable...
else {
print $exe_fh "$input\n";
}
}
}