How to capture running log using bash script? [Case Closed] - linux

Question

I'm a new player in bash scripting. There's something that I want to know about capture logfile using bash script.
Let's say there is a server which store logfile every hour with format file below.
file[20160509_130000].log
The logfile has detailed information like this.
13:00:00 INFO [file.chdev130] Event: ADIEVN_PLAY_DONE, Finished , nbytes=16360
13:00:00 INFO [file.chdev39] adiCollectDigits() success
My question is how can i read and store the running log or one hour before to get specific parameter (e.g. "Event") to new file using bash scripting?
Can someone teach me how to do it? Thanks.
Update
Here the flow that I want (for this time I want to know how the first point works):
Get the running log or one hour before.
Read the 15 minute interval (13:00:00 - 13:15:00).
grep the parameter (e.g. "Event) in that interval.
Count the parameter.
Store it to another file.
SOLVED
Here the solution in case someone need it.
List all the content based on time stamp using ls -t then pipe it
Use grep -v ^d (i still doesn't know the exact explanation for ^d), pipe again
Display first few lines with head
So the result is,
ls -t *.log | grep -v ^d | head -1 (for display the running log)
ls -t *.log | grep -v ^d | head -2 | tail -1 (for display the one log before the running log)
Hope it'll help. Thanks
== Case Closed ==

Answer 1

tail -f /path-to-logfile will allow you to read a file continuously until you cancel tail execution.
grep "^13:" /path-to-logfile will show you all strings in the file, which starts from "13:", in our case you'll get every record for 13-th hour.
grep "Event" /path-to-logfile will show you all strings with "Event" in them, i think, you got the idea about grep already. )

Answer 2

You can figure out the current or the previous log filename using date, using your logfile name convention:
fname="file[`date -v-1H '+%Y%m%d_%H0000'`].log"
will give you the previous filename. Omit -v-1H to get the current one.
For the 15-minute intervals, you may use regexps like '^\d\d:(0\d|1[0-4])' for 00:00-14:59 interval, '^\d\d:(1[5-9]|2\d)' for 15:00-29:59, etc.
For example, in the first regex, ^ matches the beginning of the line, \d\d: matches "two digits and a colon", and (0\d|1[0-4]) matches either 0 with any adjacent digit, or 1 with adjacent digit from 0 to 4. In the second regex, (1[5-9]|2\d) matches 1 with digit from 5 to 9, or 2 with any digit.
Then you grep -Po '(?<=Event: ).+(?=,)' the type of events in your log, assuming that the type of event always ends with a ,. That regexp will greedily match any symbols, as many as it can, starting from one symbol, if they are between strings Event: and , (Event: and , themselves are not matched, that's what lookbehind/lookaheads are for). Then use sort | uniq -c to count number of different events entries .
So the resulting script would look something like
fname="file[`date -v-1H '+%Y%m%d_%H0000'`].log"
grep -P '^\d\d:(0\d|1[0-4])' "$fname" | grep -Po '(?<=Event: ).+(?=,)' | sort | uniq -c > "/tmp/$fname_00-14" # entries for first 15 minutes
grep -P '^\d\d:(1[5-9]|2\d)' "$fname" | grep -Po '(?<=Event: ).+(?=,)' | sort | uniq -c > "/tmp/$fname_15-29" # entries for second 15 minutes
grep -P '^\d\d:(3\d|4[0-4])' "$fname" | grep -Po '(?<=Event: ).+(?=,)' | sort | uniq -c > "/tmp/$fname_30-44" # entries for third 15 minutes
grep -P '^\d\d:(4[5-9]|5\d)' "$fname" | grep -Po '(?<=Event: ).+(?=,)' | sort | uniq -c > "/tmp/$fname_45-59" # entries for fourth 15 minutes
for the last hour log.
Another approach is to use logtail with cron entries to get last log entries, instead of grepping. You can set up your script to be run by cron at 00, 15, 30 and 45 minutes each hour, determine the log filename inside it and logtail the file like logtail -f "$fname". The catch here would be that when running at 00 minutes, you'd need to use the -v-1H switch for date, and this approach is not as accurate as grepping out the times.