Below is my Bash Shell Script from which I am executing my two Hive SQL Queries which is working fine. And I am calculating Error Percentage in this Bash Shell Script.
#!/bin/bash
QUERY1=`hive -e "
set mapred.job.queue.name=hdmi-technology;
SELECT SUM(total_items_purchased), SUM(total_items_missingormismatch) from lip_data_quality where dt='$DATE_YEST_FORMAT2';"`
QUERY2=`hive -e "
set mapred.job.queue.name=hdmi-technology;
SELECT 100 * SUM(total_items_missingormismatch*1.0) / SUM(total_items_purchased) FROM lip_data_quality where dt='$DATE_YEST_FORMAT2';"`
mailx -s "LIP Data Quality Report for $DATE_YEST_FORMAT1" -r rj#host.com rj#host.com <<EOF
Data Successfully loaded into LIP_DATA_QUALITY table
Total Items Purchased: `echo $QUERY1 | awk '{print $1}'`
Total Items MissingorMismatch: `echo $QUERY1 | awk '{print $2}'`
Error Percentage: $QUERY2
EOF
Problem Statement:-
I will be getting Error Percentage number in $QUERY2. I need to make a Simple Pie Graph that can show Error Percentage and No Error Percentage by using the number from $QUERY2 just like below graph using Bash Shell Script.
I am running SunOS. Is this possible to do in Bash Shell script? Any thoughts will be appreciated.
Update:-
Below is the Shell Script that I am using, that I created using vi editor.
1 #! /bin/bash
2
3 TEMP=$(mktemp -t chart)
4 QUERY1=36
5 QUERY2=64
6 cat > $TEMP <<EOF
7 <html>
8 <head>
9 <!--Load the AJAX API-->
10 <script type="text/javascript" src="https://www.google.com/jsapi"></script>
11 <script type="text/javascript">
12
13 // Load the Visualization API and the piechart package.
14 google.load('visualization', '1.0', {'packages':['corechart']});
15
16 // Set a callback to run when the Google Visualization API is loaded.
17 google.setOnLoadCallback(drawChart);
18
19 // Callback that creates and populates a data table,
20 // instantiates the pie chart, passes in the data and
21 // draws it.
22 function drawChart() {
23
24 // Create the data table.
25 var data = new google.visualization.DataTable();
26 data.addColumn('string', 'Title');
27 data.addColumn('number', 'Value');
28 data.addRows([
29 ['Error Percentage', $QUERY1],
30 ['No Error Percentage', $QUERY2]
31 ]);
32
33 // Set chart options
34 var options = {'title':'Errors',
35 'width':400,
36 'height':300};
37
38 // Instantiate and draw our chart, passing in some options.
39 var chart = new google.visualization.PieChart(document.getElementById('chart_div'));
40 chart.draw(data, options);
41 }
42 </script>
43 </head>
44
45 <body>
46 <!--Div that will hold the pie chart-->
47 <div id="chart_div"></div>
48 </body>
49 </html>
50 EOF
51
52 # open browser
53 case $(uname) in
54 Darwin)
55 open -a /Applications/Google\ Chrome.app $TEMP
56 ;;
57
58 Linux|SunOS)
59 firefox $TEMP
60 ;;
61 esac
62
Error that I got after running the above shell script as sh -x chart.sh-
bash-3.00$ sh -x chart.sh
chart.sh: syntax error at line 3: `TEMP=$' unexpected
Any thoughts will be appreciated.
Another Update:-
After the below suggestions, when I tried something like this-I got another error.
bash-3.00$ bash -x chart.sh
++ mktemp -t chart
mktemp: failed to create file: /tmp/chart
+ TEMP=
+ QUERY1=36
+ QUERY2=64
+ cat
chart.sh: line 6: $TEMP: ambiguous redirect
Another Update: Made Some Progress I guess. Not sure where the output files will be going? Or it will open into a browser?
bash-3.00$ bash -x chart.sh
++ mktemp -t chart
+ TEMP=/tmp/chart
+ QUERY1=36
+ QUERY2=64
+ cat
++ uname
A very simple way of creating a Google Chart:
#! /bin/bash
TEMP=$(mktemp -t chart.XXXXX)
QUERY1=36
QUERY2=64
cat > $TEMP <<EOF
<html>
<head>
<!--Load the AJAX API-->
<script type="text/javascript" src="https://www.google.com/jsapi"></script>
<script type="text/javascript">
// Load the Visualization API and the piechart package.
google.load('visualization', '1.0', {'packages':['corechart']});
// Set a callback to run when the Google Visualization API is loaded.
google.setOnLoadCallback(drawChart);
// Callback that creates and populates a data table,
// instantiates the pie chart, passes in the data and
// draws it.
function drawChart() {
// Create the data table.
var data = new google.visualization.DataTable();
data.addColumn('string', 'Title');
data.addColumn('number', 'Value');
data.addRows([
['Error Percentage', $QUERY1],
['No Error Percentage', $QUERY2]
]);
// Set chart options
var options = {'title':'Errors',
'width':400,
'height':300};
// Instantiate and draw our chart, passing in some options.
var chart = new google.visualization.PieChart(document.getElementById('chart_div'));
chart.draw(data, options);
}
</script>
</head>
<body>
<!--Div that will hold the pie chart-->
<div id="chart_div"></div>
</body>
</html>
EOF
# open browser
case $(uname) in
Darwin)
open -a /Applications/Google\ Chrome.app $TEMP
;;
Linux|SunOS)
firefox $TEMP
;;
esac
if you save it as chart.sh then run it as
$ bash -x chart.sh
or
$ chmod +x chart.sh
$ ./chart.sh
which gives you something like
Notice that you only need bash and an Internet connection, there's nothing to install.
mktemp should be available in Solaris (http://docs.oracle.com/cd/E23824_01/html/821-1461/mktemp-1.html). If you don't have it just set TEMP to whatever file you want the HTML output to be.
A simple way to create a pie chart in only shell, is to generate an svg image.
A simple/small program to generate other image types is Ploticus.
http://ploticus.sourceforge.net/doc/welcome.html
There is probably no native way of doing it, so I would assume that your best bet is to get a third party tool installed, like-
PLOTRIX
Documentation
Related
I am working on a script where I want to iterate between the numbers 1 to 15, but want it shown as 01 02 03 ... 13 14 15. Essentially what I am trying to do is add 15 users using the newusers command and using this script as < to the command. newusers needs to be in this format:
pw_name:pw_passwd:pw_uid:pw_gid:pw_gecos:pw_dir:pw_shell
Basically, it should look like this when I run the script with arguments =
cstuser01:EzVlK9Je8JvfQump:1001:1001:CST8177 user:/home/cstuser01:/bin/bash
cstuser02:EsKOfvhgnWpiBT6c:1002:1002:CST8177 user:/home/cstuser02:/bin/bash
cstuser03:qzQuR5vRgxdzY6dq:1003:1003:CST8177 user:/home/cstuser03:/bin/bash
I got most of it working but I am getting the error below:
./15users.sh: 57: ./15users.sh: Illegal number: 08
Here is my script so far (I took out a couple sections with error checking) =
#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH
umask=022
#num=1 (this variable is needed depending on which loop I use below)
user=$prefix"user"
uid=1001
gid=$uid
home=/home/$user
shell=/bin/bash
#echo "pw_name:pw_passwd:pw_uid:pw_gid:pw_gecos:pw_dir:pw_shell"
#PASSWD=$(openssl rand -base64 12)
I originally had this but ran into a few problems:
while [ $NUM -le 15 ] ; do
if [ $NUM -lt 10 ] ; then
NUM=0$NUM
fi
echo "$USER$NUM:$(openssl rand -base64 12):$UID:$GID:$GECO:$HOME$NUM:$SHELL"
UID=$(( UID + 1 ))
GID=$(( GID + 1 ))
NUM=$(( NUM + 1 ))
done
A friend of mine suggested this, it works perfectly fine. But I am trying to future proof this thing. What if I have a 100 or 1,000 users to add.
for NUM in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 ; do
echo "$USER$NUM:$(openssl rand -base64 12):$UID:$GID:$GECO:$HOME$NUM:$SHELL"
done
This didn't work:
for num in {01..15} ; do
i=09
echo "$(( 10#$num + 1 ))"
10
done
I then tried this getting a syntax error =
./15users.sh: 50: ./15users.sh: Syntax error: Bad for loop variable
for (( num=1; num<=15; num++ )) ; do
printf "%02d\n" $num
done
I tried this as well but seq prints vertically not horizontally:
#iterate=$(seq -w 1 15)
for $iterate ; do
echo "$user$num:$(openssl rand -base64 12):$uid:$gid:$geco:$home$num:$shell"
done
To loop over 01 to 15, it is much simpler to use brace expansion:
$ for num in {01..15}; do echo "$num"; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
In bash, by default, numbers beginning with 0 are octal. Since 08 and 09 are illegal as base-8 numbers, they will cause an error. To avoid that, explicitly specify the base:
$ i=09; echo $(( 10#$i + 1 ))
10
The expression 10#$i tells bash to interpret $i as a base-10 number.
Do NOT use all caps for your script variables. The system uses all caps and you don't want to accidentally overwrite a system variable.
In the case of UID, it is a read-only bash variable. Attempts by your script to assign UID will fail. Use lower or mixed-case for your script variables.
Another example of the all caps problem is $HOME. Note that the following code works:
$ openssl rand -base64 12
1bh9+dp+Ap7xFIBB
But the following fails:
$ (HOME=/home/user; openssl rand -base64 12)
zceZeWsQGpohTPvv
unable to write 'random state'
Apparently, openssl expects to have write-access to $HOME.
Assigning HOME to a non-existent directory causes an error.
So, again, do not all all caps for your script variables.
I won't try to diagnose your error message, but you're over-complicating what you're trying to achieve.
for i in {01..15}; do echo $i; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
Bash supports C style loops as well:
$ for (( i=1; i<=15; i++ )); do printf "%02d\n" $i; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
Just use printf with the flag to print leading 0 and you have your output.
Since it hasn't been mentioned yet:
seq -w 1 15
seq -w 1 15 | while read num; do echo "n=$num"; done
I have the below piece of code in one of my scripts. Although the script works fine and produces the expected output, I get the below errors each time:
callHomeProperties.sh: line 154: 1: command not found
callHomeProperties.sh: line 160: 1: command not found
148 if [[ "$OS_VERSION_DB" = "$OS_VERSION_FILE" && "$DB_DETAILS_DB" = "$DB_DETAILS_FILE" && "$APP_SERVER_DB" = "$APP_SERVER_FILE" && "$AAOP_VERSION_DB" = "$AAOP_VERSION_FILE" && "$VMWARE_DB" = "$VMWARE_FILE" ]]; then
149 echo "same entry for the customer updating the time stamp and the data date"
150 `$ORACLE_HOME/bin/sqlplus -s /nolog <<endSQL
151 connect call_home/call_home#$ORASVC
152 update call_home_headers set LAST_UPDATED_DATE = sysdate where (CUSTOMER_NAME='$CUSTOMER_NAME_FILE' and OS_VERSION='$OS_VERSION_DB' and DB_DETAILS='$DB_DETAILS_DB' and APP_SERVER='$A PP_SERVER_DB' and AAOP_VERSION='$AAOP_VERSION_DB' and VMWARE='$VMWARE_DB');
153 exit
154 endSQL`
155 `$ORACLE_HOME/bin/sqlplus -s /nolog <<endSQL
156 connect call_home/call_home#$ORASVC
157 update call_home_headers set DATA_DATE = TO_DATE('$DATA_DATE_FILE','yyyy/MM/dd HH/MI/SS') where (CUSTOMER_NAME='$CUSTOMER_NAME_FILE' and OS_VERSION='$OS_VERSION_DB' and DB_DETAILS='$ DB_DETAILS_DB' and APP_SERVER='$APP_SERVER_DB' and AAOP_VERSION='$AAOP_VERSION_DB' and VMWARE='$VMWARE_DB');
158
159 exit
160 endSQL`
161
Can anyone please help out here. I have already tried dos2unix assuming that its a problem because of line endings, but that didn't work.
Thanks
Each of your sqlplus commands are in backquotes, so the shell will try to execute the output of the command. I'm guessing the output isn't valid shell, and that's why it's complaining.
The simple fix is to remove the backquotes:
"$ORACLE_HOME"/bin/sqlplus -s /nolog <<endSQL
connect call_home/call_home#$ORASVC
update call_home_headers set LAST_UPDATED_DATE = sysdate where (...);
endSQL
It may be a good idea to sanitise $ORASVC and other variables you use, unless they come from trusted sources (i.e. not user input) and don't contain anything meaningful to SQL, otherwise you have a pile of SQL-injection vulnerabilities waiting to be exploited.
I'm not great with Linux, my apologies if this is easy.
I need to run 7 different scripts via cron at 15 minute intervals. I have worked out that each script needs to run every 105 minutes (every 1 hour & 45 minutes) as follows :
Script 1 Script 2 Script 3 Script 4 Script 5 Script 6 Script 7
00:15 00:30 00:45 01:00 01:15 01:30 01:45
02:00 02:15 02:30 02:45 03:00 03:15 03:30
etc...
Now I know I can't just put this:
*/105 * * * * something
Because the minutes will only go up to 59.
Other than writing each one in a line separately, is there an easy way to do this?
Any assistance would be greatly appreciated.
No, cron can't do that directly.
What you can do is have a cron job that runs every 15 minutes, invoking a wrapper script. The wrapper script can query the current time and decide which of your 7 scripts to execute.
Doing the math is left as an exercise. Don't assume that your wrapper will start exactly at the top of the minute; there can be some delay between the time a cron job is scheduled to execute and the time it actually runs, depending on system load.
at is a good candidate for this problem.
Write a wrapper (or build into your script):
#!/bin/bash
at -f $0 now + 105 minutes
<program>
Then use at -f <script> <time> for each instance.
Or generate the crontab entries using a script (my first draft was a bit too long to be a oneliner):
#!/usr/bin/perl -Ws
use strict;
use DateTime;
our ($period, $offset, $command);
my ($offh, $offm) = split(/:/, $offset || '0:00');
my %crontab = ();
for (my $i = 0; $i < 1440; $i += $period) {
my ($h,$m) = split /:/, 'DateTime'->today->set('hour', $offh, 'minute', $offm)->add('minutes', $i)->hms;
$crontab{$m}{$h}++;
}
printf("%02d %-15s * * *\t%s\n", $_, join(',', sort keys %{$crontab{$_}}), $command) for sort keys %crontab;
Usage:
$ ./gencrontab -period=105 -offset=1:30 -command=myprogram
00 05,12,19 * * * myprogram
15 00,03,10,17 * * * myprogram
30 01,08,15,22 * * * myprogram
45 06,13,20 * * * myprogram
I am using a bash script to generate mobility files (setdest) in ns2 for various seeds. But I am running into this troublesome segmentation fault. Any help would be appreciated. The setdest.cc has been modified, so its not the standard ns2 file.
I will walk you through the problem.
This code in a shell script returns the segmentation fault.
#! /bin/sh
setdest="/root/ns-allinone-2.1b9a/ns-2.1b9a/indep-utils/cmu-scen-gen/setdest/setdest_mesh_seed_mod"
let nn="70" #Number of nodes in the simulation
let time="900" #Simulation time
let x="1000" #Horizontal dimensions
let y="1000" #Vertical dimensions
for speed in 5
do
for pause in 10
do
for seed in 1 5
do
echo -e "\n"
echo Seed = $seed Speed = $speed Pause Time = $pause
chmod 700 $setdest
setdest -n $nn -p $pause -s $speed -t $time -x $x -y $y -l 1 -m 50 > scen-mesh-n$nn-seed$seed-p$pause-s$speed-t$time-x$x-y$y
done
done
done
error is
scengen_mesh: line 21: 14144 Segmentation fault $setdest -n $nn -p $pause -s $speed -t $time -x $x -y $y -l 1 -m 50 >scen-mesh-n$nn-seed$seed-p$pause-s$speed-t$time-x$x-y$y
line 21 is the last line of the shell script (done)
The strange thing is If i run the same setdest command on the terminal, there is no problem! like
$setdest -n 70 -p 10 -s 5 -t 900 -x 1000 -y 1000 -l 1 -m 50
I have made out where the problem is exactly. Its with the argument -l. If i remove the argument in the shell script, there is no problem. Now i will walk you through the modified setdest.cc where this argument is coming from.
This modified setdest file uses a text file initpos to read XY coordinates of static nodes for a wireless mesh topology. the relevant lines of code are
FILE *fp_loc;
int locinit;
fp_loc = fopen("initpos","r");
while ((ch = getopt(argc, argv, "r:m:l:n:p:s:t:x:y:i:o")) != EOF) {
switch (ch) {
case 'l':
locinit = atoi(optarg);
break;
default:
usage(argv);
exit(1);
if(locinit)
fscanf(fp_loc,"%lf %lf",&position.X, &position.Y);
if (position.X == -1 && position.Y == -1){
position.X = uniform() * MAXX;
position.Y = uniform() * MAXY;
}
What i dont get is...
In Shell script..
-option -l if supplied by 0 returns no error,
-but if supplied by any other value (i used 1 mostly) returns this segmentation fault.
In Terminal..
-no segmentation fault with any value. 0 or 1
something to do with the shell script surely. I am amazed what is going wrong where!
Your help will be highly appreciated.
Cheers
We have a very simple tcp messaging script that cats some text to a server port which returns and displays a response.
The part of the script we care about looks something like this:
cat someFile | netcat somehost 1234
The response the server returns is 'complete' once we get a certain character code (specifically &001C) returned.
How can I close the connection when I receive this special character?
(Note: The server won't close the connection for me. While I currently just CTRL+C the script when I can tell it's done, I wish to be able to send many of these messages, one after the other.)
(Note: netcat -w x isn't good enough because I wish to push these messages through as fast as possible)
Create a bash script called client.sh:
#!/bin/bash
cat someFile
while read FOO; do
echo $FOO >&3
if [[ $FOO =~ `printf ".*\x00\x1c.*"` ]]; then
break
fi
done
Then invoke netcat from your main script like so:
3>&1 nc -c ./client.sh somehost 1234
(You'll need bash version 3 for the regexp matching).
This assumes that the server is sending data in lines - if not you'll have to tweak client.sh so that it reads and echoes a character at a time.
How about this?
Client side:
awk -v RS=$'\x1c' 'NR==1;{exit 0;}' < /dev/tcp/host-ip/port
Testing:
# server side test script
while true; do ascii -hd; done | { netcat -l 12345; echo closed...;}
# Generate 'some' data for testing & pipe to netcat.
# After netcat connection closes, echo will print 'closed...'
# Client side:
awk -v RS=J 'NR==1; {exit;}' < /dev/tcp/localhost/12345
# Changed end character to 'J' for testing.
# Didn't wish to write a server side script to generate 0x1C.
Client side produces:
0 NUL 16 DLE 32 48 0 64 # 80 P 96 ` 112 p
1 SOH 17 DC1 33 ! 49 1 65 A 81 Q 97 a 113 q
2 STX 18 DC2 34 " 50 2 66 B 82 R 98 b 114 r
3 ETX 19 DC3 35 # 51 3 67 C 83 S 99 c 115 s
4 EOT 20 DC4 36 $ 52 4 68 D 84 T 100 d 116 t
5 ENQ 21 NAK 37 % 53 5 69 E 85 U 101 e 117 u
6 ACK 22 SYN 38 & 54 6 70 F 86 V 102 f 118 v
7 BEL 23 ETB 39 ' 55 7 71 G 87 W 103 g 119 w
8 BS 24 CAN 40 ( 56 8 72 H 88 X 104 h 120 x
9 HT 25 EM 41 ) 57 9 73 I 89 Y 105 i 121 y
10 LF 26 SUB 42 * 58 : 74
After 'J' appears, server side closes & prints 'closed...', ensuring that the connection has indeed closed.
Try:
(cat somefile; sleep $timeout) | nc somehost 1234 | sed -e '{s/\x01.*//;T skip;q;:skip}'
This requires GNU sed.
How it works:
{
s/\x01.*//; # search for \x01, if we find it, kill it and the rest of the line
T skip; # goto label skip if the last s/// failed
q; # quit, printing current pattern buffer
:skip # label skip
}
Note that this assumes there'll be a newline after \x01 - sed won't see it otherwise, as sed operates line-by-line.
Maybe have a look at Ncat as well:
"Ncat is the culmination of many key features from various Netcat incarnations such as Netcat 1.x, Netcat6, SOcat, Cryptcat, GNU Netcat, etc. Ncat has a host of new features such as "Connection Brokering", TCP/UDP Redirection, SOCKS4 client and server supprt, ability to "Chain" Ncat processes, HTTP CONNECT proxying (and proxy chaining), SSL connect/listen support, IP address/connection filtering, plus much more."
http://nmap-ncat.sourceforge.net
This worked best for me. Just read the output with a while loop and then check for "0x1c" using an if statement.
while read i; do
if [ "$i" = "0x1c" ] ; then # Read until "0x1c". Then exit
break
fi
echo $i;
done < <(cat someFile | netcat somehost 1234)