I'm running an application that on start updates the same line:
1 of 1,000,000
200 of 1,000,000
300 of 1,000,000
But the above is on a single line.
Every time the cursor is updated I'd like to write it to a log file so that I can observe the duration between updates.
This command seems to work for lines that were output to console before the number starts increment on the same line, but does not log the number updates.
command |& tee >(ts "%d-%m-%y %H_%M_%S" > play.log)
Is there a trick to log the state screen on update along with a timestamp?
You could keep track of changes on a file by using tail and if you would like to prefix the contents with a custom input you could use xargs, for example:
$ tail -F file | xargs -I# date +"%d-%m-%y %H_%M_%S --> #"
Or pass the output of your command:
$ echo "foo" | xargs -I# date +"%d-%m-%y %H_%M_%S --> #"
11-08-18 15_53_53 --> foo
Related
I have a process that prints output, but this output also includes loading bars.
I would like to both write this output to a file and display this output.
Normally I could just do:
./my_process.sh | tee -a my_log_file.txt
or
./my_process.sh >> my_log_file.txt
tail -f my_log_file.txt
This prints everything to my terminal, however it also prints EVERYTHING to the log file, including each step of the progress bar!
I would like to exclude progress bar iterations from getting printed to the log file.
For my purposes, any line with a carriage return can be excluded from the log file. How can I exclude carriage return lines from getting appended to the log file while still printing them to stdout on the terminal?
You can filter the tee before logging
for example
$ echo -e "progress_ignore\r\nlog this\nprogress_ignore\r" | tee >(awk '!/\r/' >> output.log)
progress_ignore
log this
progress_ignore
$ cat output.log
log this
I use
tail -f file | grep pattern
all the time for continuous grep.
However, is there a way I can make grep output its pattern at the same spot, say at the top of the screen? so that the screen doesn't scroll all the time?
My case is something like this: tail -f log_file | grep Status -A 2 will show the current status and what changed it to that status. The problem is the screen scrolls and it becomes annoying. I'd rather have the output stuck on the first 3 lines in the screen.
Thank you!
you could use the watch command; which will always execute the same command, but the position on the screen will stay the same. The process might eat some more CPU or memory though:
watch "tail file | grep pattern"
by default watch executes that command every 2 seconds. You can adjust up to 0.1 seconds using:
watch -n 0.1
NOTE
As noted by #etanReisner: this is not exactly the same as tail -f: tail -f will change immediately if something is added to your logfile, the watch command will only notice that when it executes, ie every 2 (or 0.1 seconds).
Assuming you are using a vt100 compatible emulator...
This command will tail a file, pipe it into grep, read it a line at a time and then display it in reverse on the top line of the screen:
TOSL=$(tput sc;tput cup 0 0;tput rev;tput el)
FROMSL=$(tput sgr0; tput rc)
tail -f file | grep --line-buffered pattern | while read line
do
echo -n "$TOSL${line}$FROMSL"
done
It assumes your output appears a line at a time. If you want more than one line, you can read more than a line, but you have to decide how you want to buffer the output. You could also use the csr terminfo command to set up an entire separate scrolling region instead of just having one line.
Here is the scrolling region version with a ten line status area at the top:
TOSL=$(tput sc; tput csr 0 10; tput cup 10 0;tput rev;tput el)
FROMSL=$(tput sgr0; tput rc;tput csr 10 50;tput rc)
tail -f file | grep --line-buffered pattern | while read line
do
echo -n "$TOSL${line}
$FROMSL"
done
Note that it is not impossible that your display will be corrupted from time-to-time as it could be that the output from your main shell and your background task get mixed up.
Simply replace the newlines with carriage returns.
tail -f file | grep --line-buffered whatever | tr '\012' '\015'
The line buffering is to avoid jumpy output; see http://mywiki.wooledge.org/BashFAQ/009
This is quick and dirty. As noted in comments, this will leave the previous contents of the line underneath, so a shorter line will not completely overlay a longer line. You could add some control codes to address that, but then you might as well use Curses for the formatting too, like in rghome's answer.
Outside screen -
[j#firedesire rb]$ diff -bu root_es-US.pres root_es-MX.pres | sed -n "s/^-\([^=]*\)=.*/'\1'/p" | tr "\\n" ","
'IMG.HEAD.LOGO.URL','LNK.COPYRIGHT.URL','LNK.FOOTSECURITY.URL','LNK.HEADHELP.URL','LNK.HEADYAHOO.URL','LNK.PRIVACY.URL','LNK.TOS.URL','STR.HEAD.LOGO.HEIGHT','STR.HEAD.LOGO.WIDTH','REG.TOS.MAIL.URL','UPGRADE.STR.AGREE.STATEMENT',[j#firedesire rb]$
Inside screen -
[j#firedesire rb]$ diff -bu root_es-US.pres root_es-MX.pres | sed -n "s/^-\([^=]*\)=.*/'\1'/p" | tr "\\n" ","
'IMG.HEAD.LOGO.URL','LNK.COPYRIGHT.URL','LNK.FOOTSECURITY.URL','LNK.HEADHELP.URL','LNK.HEADYAHOO.URL','LNK.PRIVACY.URL','LNK.TOS.URL','STR.HEAD.LOGO.HEIGHT','STR.HEAD.LOGO.WIDTH','REG.TOS.MAIL.URL','UPGR
It gets cut off at 'UPGR where line ends. What could be causing this problem, how can I fix this ?
Your output is wider than your screen
You can force the output to wrap by piping it through less -+S, you'll have to use your arrow keys to view all of the output.
There's an option in screen to toggle wrapping on or off; you may have luck with that as well: https://www.gnu.org/software/screen/manual/html_node/Wrap.html
Another method besides less is to pipe it through cat:
ps aux | cat
top | cat
That won't pause at every screenful, like less or more would do, but when programs would otherwise halt until it receives user input, when they detect that their output is being piped to another process, they just dump out the results and keep going.
I use this a lot with external diff programs attached to my git or svn flags: git diff | cat keeps feeding my diff program all the extra files after the first one, while git diff by itself doesn't.
I use screen to run a minecraft server .jar, and I would like to write a bash script to see if the most recent line has changed every five minutes or so. If it has, then the script would start from the beginning and make the check again in another five minutes. If not, it should kill the java process.
How would I go about getting the last line of text from a screen via a bash script?
If I have understand, you can redirect the output of your program in a file and work on it, with the operator >.
Try to run :
ls -l > myoutput.txt
and open the file created.
You want to use the tail command. tail -n 1 will give you the last line of the file or redirected standard output, while tail -f will keep the tail program going until you cancel it yourself.
For example:
echo -e "Jello\nPudding\nSkittles" | tail -n 1 | if grep -q Skittles ; then echo yes; fi
The first section simply prints three lines of text:
Jello
Pudding
Skittles
The tail -n 1 finds the last line of text ("Skittles") and passes that to the next section.
grep -q simply returns TRUE if your pattern was found or FALSE if not, without actually dumping or outputting anything to screen.
So the if grep -q Skittles section will check the result of that grep -q Skittles pattern and, if it found Skittles, prints 'yes' to the screen. If not, nothing gets printed (try replacing Skittles with Pudding, and even though it was in the original input, it never made it out the other end of the tail -n 1 call).
Maybe you can use that logic and output your .jar to standard output, then search that output every 5 minutes?
I am working on a Java EE application where its logs will be generated inside a Linux server .
I have used the command tail -f -n -10000 MyLog
It displayed last 1000 lines from that log file .
Now I pressed Ctrl + c in Putty to disconnect the logs updation ( as i am feared it may be updated with new requests and I will loose my data )
In the displayed result, how can I search for a particular keyword ?? (Used / String name to search but it's not working)
Pipe your output to PAGER.
tail -f -n LINE_CNT LOG_FILE | less
then you can use
/SEARCH_STRING
Two ways:
tail -n 10000 MyLog| grep -i "search phrase"
tail -f -n 10000 MyLog | less
The 2nd method will allow you to search with /. It will only search down but you can press g to go back to the top.
Edit: On testing it seems method 2 doesn't work all that well... if you hit the end of the file it will freeze till you ctrl+c the tail command.
You need to redirect the output from tail into a search utility (e.g. grep). You could do this in two steps: save the output to a file, then search in the file; or in one go: pipe the ouput to the search utility
To see what goes into the file (so you can hit Ctlr+c) you can use the tee command, which duplicates the output to the screen and to a file:
tail -f -n -10000 MyLog | tee <filename>
Then search within the file.
If you want to pipe the result into the search utility, you can use the same trick as above, but use your search program instead of tee
Controlling terminal output on the fly
While running any command in a terminal such as Putty you can use CTRL-S and CTRL-Q to stop and start output to the Putty terminal.
Excluding lines using grep
If you want to exclude lines that contain a specific pattern use grep -v the following would remove all line that contain the string INFO
tail -f logfile | grep -v INFO
Show lines that do not contain the words INFO or DEBUG
tail -f logfile | grep -v -E 'INFO|DEBUG'
Finally, the MOTHER AND FATHER of all tailing tools is xtail.pl
If you have perl on your host xtail.pl is a very nice tool to learn and in a nutshell you can use it to tail multiple files. Very handy.
You can just open it with less command
less logfile_name
when you open the file you can use this guide here
Tip: I suggest, first to use G to go to the end of the file and then to you use Backward Search