A lot of times (not always) the stdout is displayed in colors. Normally I keep every output log in a different file too. Naturally in the file, the colors are not displayed anymore.
I'd like to know if there's a way (in Linux) to write the output to a file with colors. I'm trying to use tee to write the output of vagrant to a file, this way I can still see the output (when it applies). I want to use it specifically for vagrant (it may change in the future, of course...)
Since many programs will only output color sequences if their stdout is a terminal, a general solution to this problem requires tricking them into believing that the pipe they write to is a terminal. This is possible with the script command from bsdutils:
script -q -c "vagrant up" filename.txt
This will write the output from vagrant up to filename.txt (and the terminal). If echoing is not desirable,
script -q -c "vagrant up" filename > /dev/null
will write it only to the file.
You can save the ANSI sequences that colourise your output to a file:
echo a | grep --color=always . > colour.txt
cat colour.txt
Some programs, though, tend not to use them if their output doesn't go to the terminal (that's why I had to use --color-always with grep).
In Ubuntu, you can install the package bsdutils to output to a text file with ANSI color codes:
script -q -c "ls --color=always" /tmp/t
Install kbtin to generate a clean HTML file:
ls --color=always | ansi2html > /tmp/t.html
Install aha and wkhtmltopdf to generate a nice PDF:
ls --color=always | aha | wkhtmltopdf - /tmp/t.pdf
Use any of the above with tee to display the output also on the console or to save a copy in another file. Example:
ls --color=always | tee /dev/stderr | aha | wkhtmltopdf - /tmp/test.pdf
You can also color your output with echo with different colours and save the coloured output in file. Example
echo -e '\E[37;44m'"Hello World" > my_file
Also You would have to be acquainted with the terminal colour codes
Using tee
< command line > |tee -a 'my_colour_file'
Open your file in cat
cat 'my_colour_file'
Using a named pipe can also work to redirect all output from the pipe with colors to another file
for example
Create a named pipe
mkfifo pipe.fifo
each command line redirect it to the pipe as follows
<command line> > pipe.fifo
In another terminal redirect all messages from the pipe to your file
cat pipe.fifo > 'my_log_file_with_colours'
open your file with cat and see the expected results.
I found out that using the tool called ansi2html.sh
Is the most simple way to export colorful terminal data to html file,
The commands to use it are:
ls --color=always | ansi2html.sh --palette=solarized > ~/Desktop/ls.html
All is needed is to send the output using a pipe and then output the stdout to simple html file
Solution
$ script -q /dev/null -c "your command" > log.txt
$ cat log.txt
Explanation
According to the man page of script, the --quit option only makes sure to be quiet (do not write start and done messages to standard output). Which means that the start and done messages will always be written to the file.
In order to utilize script and discard the output file at the same file, we can simply specify the null device /dev/null to it! Also, redirect the output to our desired destination and the color content will be written to the destination.
I was trying out some of the solutions listed here, and I also realized you could do it with the echo command and the -e flag.
$ echo -e "\e[1;33m This is yellow text \e[0m" > sample.txt
Next, we can view the contents of our sample.txt file.
$ cat sample.txt
Click link to see the output in yellow
Additionally, we can also use tee and pipe it with our echo command:
echo -e "\e[1;33m This is yellow text \e[0m" | tee -a sample.txt
On macOS, script is from the BSD codebase and you can use it like so:
script -q /dev/null mvn dependency:tree mvn-tree.colours.txt
It will run mvn dependency:tree and store the coloured output into mvn-tree.colours.txt
The tee utility supports colours, so you can pipe it to see the command progress:
script -q /dev/null mvn dependency:tree | tee mvn-tree.colours.txt
To get the script manual you can type man script:
SCRIPT(1) BSD General Commands Manual SCRIPT(1)
NAME
script -- make typescript of terminal session
SYNOPSIS
script [-adkpqr] [-F pipe] [-t time] [file [command ...]]
In the RedHat/Rocky/CentOS family, the ansi2html utility does not seem to be available (except for Fedora 32 and up). An equivalent utility is ansifilter from the EPEL repository. Unfortunately, it seems to have been removed from EPEL 8.
script is preinstalled from the util-linux package.
To set up:
yum install ansifilter -y
To use it:
ls --color=always | ansifilter -H > output.html
To generate a pretty PDF (not tested), have ansifilter generate LaTeX output, and then post-process it:
ls --color=always | ansifilter -L | pdflatex >output.pdf
Obviously, combine this with the script utility, or whatever else may be appropriate in your situation.
Related
this is a curiosity.
Can i start nano editor from bash, passing a piped command?
this is my situation:
I've a log file with dates formatted in tai64. For print my file i launch:
$> cat /var/log/qmail/current | tai64nlocal
that print what i want.
but i want to view this in nano or another editor in one command. for example:
$> cat /var/log/qmail/current | tai64nlocal > nano
but this doesn't work.
Any suggestion?
Thanks in advance
if you want to nano to open stdin use dash-notation (-):
echo "foo" | nano -
in your case this would translate to
cat /var/log/qmail/current | tai64nlocal | nano -
Use process substitution:
nano <(cat /var/log/qmail/current | tai64nlocal)
Also, you don't need to use cat
nano <(tai64nlocal < /var/log/qmail/current)
It didn't work because you are not using pipes. You are using a redirect which works slightly different.
| vs >
By doing
cat /var/log/qmail/current | tai64nlocal > nano
You are piping cat's stdout to tai64nlocal stdin. Then you redirect it's stdout to a filestream, in this case a file named nano in your pwd.
Based on what you wanted, it partially works because the tai command does both printing and echoing to stdout.
Older versions of nano do not support being piped to though. This was introduced in nano 2.2.
You would do
Command | nano -
The single dash tells nano to open stdin as a pager, like more or less would.
davymartu's command "nano < ( cat /var/log/maillog | tai64nlocal )" generates a syntax error because of the space between "<" and "(". If the space is removed, as it is in konsolebox's examples, the command will execute.
Is it possible to redirect the output of a command to one file, but still keep the output echoing on the terminal?
I want the output to go to a file, but I still want to see it on the terminal (just so I know something is going on)
Yes, tee does what you want:
command | tee output.log
You can use "tee" i.e.
ls -ltr | tee -a mylog.log
In terminal, sometimes I would like to display the standard output and also save it as a backup. but if I use redirection ( > &> etc), it does not display the output in the terminal anymore.
I think I can do for example ls > localbackup.txt | cat localbackup.txt. But it just doesn't feel right. Is there any shortcut to achieve this?
Thank you!
tee is the command you are looking for:
ls | tee localbackup.txt
In addition to using tee to duplicate the output (and it's worth mentioning that tee is able to append to the file instead of overwriting it, by using tee -a, so that you can run several commands in sequence and retain all of the output), you can also use tail -f to "follow" the output file from a parallel process (e.g. a separate terminal):
command1 >localbackup.txt # create output file
command2 >>localbackup.txt # append to output
and from a separate terminal, at the same time:
tail -f localbackup.txt # this will keep outputting as text is appended to the file
I have a Markdown string in JavaScript, and I'd like to display it (with bolding, etc) in a less (or, I suppose, more)-style viewer for the command line.
For example, with a string
"hello\n" +
"_____\n" +
"*world*!"
I would like to have output pop up with scrollable content that looks like
hello
world
Is this possible, and if so how?
Pandoc can convert Markdown to groff man pages.
This (thanks to nenopera's comment):
pandoc -s -f markdown -t man foo.md | man -l -
should do the trick. The -s option tells it to generate proper headers and footers.
There may be other markdown-to-*roff converters out there; Pandoc just happens to be the first one I found.
Another alternative is the markdown command (apt-get install markdown on Debian systems), which converts Markdown to HTML. For example:
markdown README.md | lynx -stdin
(assuming you have the lynx terminal-based web browser).
Or (thanks to Danny's suggestion) you can do something like this:
markdown README.md > README.html && xdg-open README.html
where xdg-open (on some systems) opens the specified file or URL in the preferred application. This will probably open README.html in your preferred GUI web browser (which isn't exactly "less-style", but it might be useful).
I tried to write this in a comment above, but I couldn't format my code block correctly. To write a 'less filter', try, for example, saving the following as ~/.lessfilter:
#!/bin/sh
case "$1" in
*.md)
extension-handler "$1"
pandoc -s -f markdown -t man "$1"|groff -T utf8 -man -
;;
*)
# We don't handle this format.
exit 1
esac
# No further processing by lesspipe necessary
exit 0
Then, you can type less FILENAME.md and it will be formatted like a manpage.
If you are into colors then maybe this is worth checking as well:
terminal_markdown_viewer
It can be used straightforward also from within other programs, or python modules.
And it has a lot of styles, like over 200 for markdown and code which can be combined.
Disclaimer
It is pretty alpha there may be still bugs
I'm the author of it, maybe some people like it ;-)
A totally different alternative is mad. It is a shell script I've just discovered. It's very easy to install and it does render markdown in a console pretty well.
I wrote a couple functions based on Keith's answer:
mdt() {
markdown "$*" | lynx -stdin
}
mdb() {
local TMPFILE=$(mktemp)
markdown "$*" > $TMPFILE && ( xdg-open $TMPFILE > /dev/null 2>&1 & )
}
If you're using zsh, just place those two functions in ~/.zshrc and then call them from your terminal like
mdt README.md
mdb README.md
"t" is for "terminal", "b" is for browser.
Using OSX I prefer to use this command
brew install pandoc
pandoc -s -f markdown -t man README.md | groff -T utf8 -man | less
Convert markupm, format document with groff, and pipe into less
credit: http://blog.metamatt.com/blog/2013/01/09/previewing-markdown-files-from-the-terminal/
This is an alias that encapsulates a function:
alias mdless='_mdless() { if [ -n "$1" ] ; then if [ -f "$1" ] ; then cat <(echo ".TH $1 7 `date --iso-8601` Dr.Beco Markdown") <(pandoc -t man $1) | groff -K utf8 -t -T utf8 -man 2>/dev/null | less ; fi ; fi ;}; _mdless '
Explanation
alias mdless='...' : creates an alias for mdless
_mdless() {...}; : creates a temporary function to be called afterwards
_mdless : at the end, call it (the function above)
Inside the function:
if [ -n "$1" ] ; then : if the first argument is not null then...
if [ -f "$1" ] ; then : also, if the file exists and is regular then...
cat arg1 arg2 | groff ... : cat sends this two arguments concatenated to groff; the arguments being:
arg1: <(echo ".TH $1 7date --iso-8601Dr.Beco Markdown") : something that starts the file and groff will understand as the header and footer notes. This substitutes the empty header from -s key on pandoc.
arg2: <(pandoc -t man $1) : the file itself, filtered by pandoc, outputing the man style of file $1
| groff -K utf8 -t -T utf8 -man 2>/dev/null : piping the resulting concatenated file to groff:
-K utf8 so groff understands the input file code
-t so it displays correctly tables in the file
-T utf8 so it output in the correct format
-man so it uses the MACRO package to outputs the file in man format
2>/dev/null to ignore errors (after all, its a raw file being transformed in man by hand, we don't care the errors as long as we can see the file in a not-so-much-ugly format).
| less : finally, shows the file paginating it with less (I've tried to avoid this pipe by using groffer instead of groff, but groffer is not as robust as less and some files hangs it or do not show at all. So, let it go through one more pipe, what the heck!
Add it to your ~/.bash_aliases (or alike)
I personally use this script:
#!/bin/bash
id=$(uuidgen | cut -c -8)
markdown $1 > /tmp/md-$id
google-chrome --app=file:///tmp/md-$id
It renders the markdown into HTML, puts it into a file in /tmp/md-... and opens that in a kiosk chrome session with no URI bar etc.. You just pass the md file as an argument or pipe it into stdin. Requires markdown and Google Chrome. Chromium should also work but you need to replace the last line with
chromium-browser --app=file:///tmp/md-$id
If you wanna get fancy about it, you can use some css to make it look nice, I edited the script and made it use Bootstrap3 (overkill) from a CDN.
#!/bin/bash
id=$(uuidgen | cut -c -8)
markdown $1 > /tmp/md-$id
sed -i "1i <html><head><style>body{padding:24px;}</style><link rel=\"stylesheet\" type=\"text/css\" href=\"http://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css\"></head><body>" /tmp/md-$id
echo "</body>" >> /tmp/md-$id
google-chrome --app=file:///tmp/md-$id > /dev/null 2>&1 &
I'll post my unix page answer here, too:
An IMHO heavily underestimated command line markdown viewer is the markdown-cli.
Installation
npm install markdown-cli --global
Usage
markdown-cli <file>
Features
Probably not noticed much, because it misses any documentation...
But as far as I could figure out by some example markdown files, some things that convinced me:
handles ill formatted files much better (similarly to atom, github, etc.; eg. when blank lines are missing before lists)
more stable with formatting in headers or lists (bold text in lists breaks sublists in some other viewers)
proper table formatting
syntax highlightning
resolves footnote links to show the link instead of the footnote number (not everyone might want this)
Screenshot
Drawbacks
I have realized the following issues
code blocks are flattened (all leading spaces disappear)
two blank lines appear before lists
In bash, calling foo would display any output from that command on the stdout.
Calling foo > output would redirect any output from that command to the file specified (in this case 'output').
Is there a way to redirect output to a file and have it display on stdout?
The command you want is named tee:
foo | tee output.file
For example, if you only care about stdout:
ls -a | tee output.file
If you want to include stderr, do:
program [arguments...] 2>&1 | tee outfile
2>&1 redirects channel 2 (stderr/standard error) into channel 1 (stdout/standard output), such that both is written as stdout. It is also directed to the given output file as of the tee command.
Furthermore, if you want to append to the log file, use tee -a as:
program [arguments...] 2>&1 | tee -a outfile
$ program [arguments...] 2>&1 | tee outfile
2>&1 dumps the stderr and stdout streams.
tee outfile takes the stream it gets and writes it to the screen and to the file "outfile".
This is probably what most people are looking for. The likely situation is some program or script is working hard for a long time and producing a lot of output. The user wants to check it periodically for progress, but also wants the output written to a file.
The problem (especially when mixing stdout and stderr streams) is that there is reliance on the streams being flushed by the program. If, for example, all the writes to stdout are not flushed, but all the writes to stderr are flushed, then they'll end up out of chronological order in the output file and on the screen.
It's also bad if the program only outputs 1 or 2 lines every few minutes to report progress. In such a case, if the output was not flushed by the program, the user wouldn't even see any output on the screen for hours, because none of it would get pushed through the pipe for hours.
Update: The program unbuffer, part of the expect package, will solve the buffering problem. This will cause stdout and stderr to write to the screen and file immediately and keep them in sync when being combined and redirected to tee. E.g.:
$ unbuffer program [arguments...] 2>&1 | tee outfile
Another way that works for me is,
<command> |& tee <outputFile>
as shown in gnu bash manual
Example:
ls |& tee files.txt
If ‘|&’ is used, command1’s standard error, in addition to its standard output, is connected to command2’s standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error to the standard output is performed after any redirections specified by the command.
For more information, refer redirection
You can primarily use Zoredache solution, but If you don't want to overwrite the output file you should write tee with -a option as follow :
ls -lR / | tee -a output.file
Something to add ...
The package unbuffer has support issues with some packages under fedora and redhat unix releases.
Setting aside the troubles
Following worked for me
bash myscript.sh 2>&1 | tee output.log
Thank you ScDF & matthew your inputs saved me lot of time..
Using tail -f output should work.
In my case I had the Java process with output logs. The simplest solution to display output logs and redirect them into the file(named logfile here) was:
my_java_process_run_script.sh |& tee logfile
Result was Java process running with output logs displaying and
putting them into the file with name logfile
You can do that for your entire script by using something like that at the beginning of your script :
#!/usr/bin/env bash
test x$1 = x$'\x00' && shift || { set -o pipefail ; ( exec 2>&1 ; $0 $'\x00' "$#" ) | tee mylogfile ; exit $? ; }
# do whaetever you want
This redirect both stderr and stdout outputs to the file called mylogfile and let everything goes to stdout at the same time.
It is used some stupid tricks :
use exec without command to setup redirections,
use tee to duplicates outputs,
restart the script with the wanted redirections,
use a special first parameter (a simple NUL character specified by the $'string' special bash notation) to specify that the script is restarted (no equivalent parameter may be used by your original work),
try to preserve the original exit status when restarting the script using the pipefail option.
Ugly but useful for me in certain situations.
Bonus answer since this use-case brought me here:
In the case where you need to do this as some other user
echo "some output" | sudo -u some_user tee /some/path/some_file
Note that the echo will happen as you and the file write will happen as "some_user" what will NOT work is if you were to run the echo as "some_user" and redirect the output with >> "some_file" because the file redirect will happen as you.
Hint: tee also supports append with the -a flag, if you need to replace a line in a file as another user you could execute sed as the desired user.
< command > |& tee filename # this will create a file "filename" with command status as a content, If a file already exists it will remove existed content and writes the command status.
< command > | tee >> filename # this will append status to the file but it doesn't print the command status on standard_output (screen).
I want to print something by using "echo" on screen and append that echoed data to a file
echo "hi there, Have to print this on screen and append to a file"
tee is perfect for this, but this will also do the job
ls -lr / > output | cat output