This question already has answers here:
How to trick an application into thinking its stdout is a terminal, not a pipe
(9 answers)
Closed 5 years ago.
Various bash commands I use -- fancy diffs, build scripts, etc, produce lots of color output.
When I redirect this output to a file, and then cat or less the file later, the colorization is gone -- presumably b/c the act of redirecting the output stripped out the color codes that tell the terminal to change colors.
Is there a way to capture colorized output, including the colorization?
One way to capture colorized output is with the script command. Running script will start a bash session where all of the raw output is captured to a file (named typescript by default).
Redirecting doesn't strip colors, but many commands will detect when they are sending output to a terminal, and will not produce colors by default if not. For example, on Linux ls --color=auto (which is aliased to plain ls in a lot of places) will not produce color codes if outputting to a pipe or file, but ls --color will. Many other tools have similar override flags to get them to save colorized output to a file, but it's all specific to the individual tool.
Even once you have the color codes in a file, to see them you need to use a tool that leaves them intact. less has a -r flag to show file data in "raw" mode; this displays color codes. edit: Slightly newer versions also have a -R flag which is specifically aware of color codes and displays them properly, with better support for things like line wrapping/trimming than raw mode because less can tell which things are control codes and which are actually characters going to the screen.
Inspired by the other answers, I started using script. I had to use -c to get it working though. All other answers, including tee, different script examples did not work for me.
Context:
Ubuntu 16.04
running behavior tests with behave and starting shell command during the test with python's subprocess.check_call()
Solution:
script --flush --quiet --return /tmp/ansible-output.txt --command "my-ansible-command"
Explanation for the switches:
--flush was needed, because otherwise the output is not well live-observable, coming in big chunks
--quiet supresses the own output of the script tool
-c, --command directly provides the command to execute, piping from my command to script did not work for me (no colors)
--return to make script propagate the exit code of my command so I know if my command has failed
I found that using script to preserve colors when piping to less doesn't really work (less is all messed up and on exit, bash is all messed up) because less is interactive. script seems to really mess up input coming from stdin even after exiting.
So instead of running:
script -q /dev/null cargo build | less -R
I redirect /dev/null to it before piping to less:
script -q /dev/null cargo build < /dev/null | less -R
So now script doesn't mess with stdin and gets me exactly what I want. It's the equivalent of command | less but it preserves colors while also continuing to read new content appended to the file (other methods I tried wouldn't do that).
some programs remove colorization when they realize the output is not a TTY (i.e. when you redirect them into another program). You can tell some of those to use color forcefully, and tell the pager to turn on colorization, for example use less -R
This question over on superuser helped me when my other answer (involving tee) didn't work. It involves using unbuffer to make the command think it's running from a shell.
I installed it using sudo apt install expect tcl rather than sudo apt-get install expect-dev.
I needed to use this method when redirecting the output of apt, ironically.
I use tee: pipe the command's output to teefilename and it'll keep the colour. And if you don't want to see the output on the screen (which is what tee is for: showing and redirecting output at the same time) then just send the output of tee to /dev/null:
command| teefilename> /dev/null
Related
As the name implies, I want to log the output of a command to a file without changing the terminal behavior. I still want to see the output and importantly, I still need to be able to do input.
The input requirement is why I specifically cannot use tee. The application I am working with doesn't handle input properly when using tee. (No, I cannot modify the application to fix this). I'm hoping a more fundamental approach with '>' redirection gets around this issue.
Theoretically, this should work exactly as I want, but, like I said, it does not.
command | tee -a foo.log
Also notice I added the -a flag. Not strictly required because I can definitely do a work around but it'd be nice if that was a feature as well.
Use script.
% script -c vi vi.log
Script started, output log file is 'vi.log'.
... inside vi command - fully interactive - save the file, takes to:
Script done.
% ls -l vi.log
-rw-r--r-- 1 risner risner 889 Sep 13 15:16 vi.log
I want to execute yum list installed > file.txt and output results to a txt file which is NOT wrapped by X characters. At least, I want to be able to control "width" of this output buffer.
I know that stty columns 250 will set column width of my console window to 250 characters but how do I accomplish this when I redirect output to a file?
This has certainly been asked before but I just could not find an answer...
Edit:
This seems to be a yum thing since ps aux > ps.txt works just fine. With yum, file is limited to only 80 characters so I'm adding yum tag. I have no idea how can yum give different output on screen and on the file while other programs work just fine (also note that I'm a beginner in bash).
I think you have two options.
Edit the yum source (http://yum.baseurl.org/download/3.4/yum-3.4.3.tar.gz). The 80 is hard coded in output.py, line 53.
It's probably doable to make yum believe it is writing to a terminal. Whether -- and if, how -- it is possible to set the number of columns for that fake terminal I am not sure of. One thing that pops up is Linux' unbuffer (cf. Piping data to Linux program which expects a TTY (terminal)). Perhaps a little self written unbuffer-like C wrapper may use a pty and have more control over it; even bash may have an esoteric feature for that.
I stumbled across a hackish way to do this using screen:
screen -L yum --color=never list installed
This will result in a file called screenlog.0 containing the output, and yum will think it is outputting to a TTY with the same width as your current screen.
I want to write a program that will print out the autocompletions of bash.
Basically I'm writting something into bash stdin with
childProc.stdin.write("./myfi")
And would like to receive autocompletion for it like "./myfile.txt"
But childProc.stdout is empty after childProc.stdin.write("\t") so there has to be some other way to trigger autocompletion.
Any ideas?
Command-completion in only enabled in interactive shells. Bash is interactive if:
Neither a script filename nor the -c option were specified when invoking bash, and
both stdin and stderr are attached to terminals (as determined by isatty()), or
bash was started with the -i flag.
In your case, stdin is clearly a pipe, which is not a terminal. So probably command completion has been disabled. But if it were enabled, you'd see the result on stderr not stdout.
So you could try supplying the -i command line option when starting your bash shell, or you could attach its stdin, stdout and stderr file descriptors to a pseudo-tty. In either case, what you will see coming back from bash will be intermingled with terminal control codes, so you'll probably want to set TERM to something basic (like dumb).
If you want to see the completions which bash might generate, you can use the compgen built-in. compgen does not know about the customized completion settings installed by the complete command, and it is not easy to get the environment set up correctly for the -F and -C function, but other than that you can probably get it to generate whatever completion lists you would like. See the Bash manual for detailed option documentation.
I've found an answer.
The thing that made it work was Pseudoterminal.
This module actually.
https://www.npmjs.com/package/pty.js-dl
A predecessor of mine installed a crappy piece of software on an old machine (running Linux) which I've inherited. Said crappy piece of software installed flotsam all over the place, and also is sufficiently bloated that I want it off ASAP -- it no longer has any functional purpose since we've moved on to better software.
Vendor provided an uninstall script. Not trusting the crappy piece of software, I opened the uninstall script in an editor (a 200+ line Bash monster), and it starts off something like this:
SWROOT=`cat /etc/vendor/path.conf`
...
rm -rf $SWROOT/bin
...
It turns out that /etc/vendor/path.conf is missing. Don't know why, don't know how, but it is. If I had run this lovely little script, it would have deleted the /bin folder, which would have had rather amusing implications. Of course this script required root to run!
I've dealt with this issue by just manually running all the install commands (guh) where sensible. This kind of sucked because I had to interpolate all the commands manually. In general, is there some sort of way I can "dry run" a script to have it dump out all the commands it would execute, without it actually executing them?
bash does not offer dry-run functionality (and neither do ksh, zsh, or any other shell I know).
It seems to me that offering such a feature in a shell would be next to impossible: state changes would have to be simulated and any command invoked - whether built in or external - would have to be aware of these simulations.
The closest thing that bash, ksh, and zsh offer is the ability to syntax-check a script without executing it, via option -n:
bash -n someScript # syntax-check a script, without executing it.
If there are no syntax errors, there will be no output, and the exit code will be 0.
If there are syntax errors, analysis will stop at the first error, an error message including the line number is written to stderr, and the exit code will be:
2 in bash
3 in ksh
1 in zsh
Separately, bash, ksh, and zsh offer debugging options:
-v to print each raw source code line[1]
to stderr before it is executed.
-x to print each expanded simple command to stderr before it is executed (env. var. PS4 allows tweaking the output format).
Combining -n with -v and/or -x offers little benefit:
With -n specified, -x has no effect at all, because nothing is being executed.
With -n specified, -v will effectively simply print the source code.
If there is a syntax error, there may be benefit in the source code getting print up to the point where the error occurs; keep in mind, though that the error message produced by
-n always includes the offending line number.
[1] Typically, it is individual lines that are printed, but the true unit is however many lines a given command - which may be a compound command such as while or a command list (such as a pipeline) - spans.
You could try running the script under Kornshell. When you execute a script with ksh -D, it reads the commands and checks them for syntax, but doesn't execute them. Combine that with set -xv, and you'll print out the commands that will be executed.
You can also use set -n for the same effect. Kornshell and BASH are fairly compatible with each other. If it's a pure Bourne shell script, both Kornshell and BASH will execute it pretty much the same.
You can also run ksh -u which will cause unset shell variables to cause the script to fail. However, that wouldn't have caught the catless cat of a nonexistent file. In that case, the shell variable was set. It was set to null.
Of course, you could run the script under a restricted shell too, but that's probably not going to uninstall the package.
That's the best you can probably do.
I quickly searched for this before posting, but could not find any similar posts. Let me know if they exist.
The commands being executed seem very simple. A directory listing is used as the input for a function.
The directory contains a bunch of files named "epi1_mcf_0###.nii.gz"
Command-line version (bash is running when this is executed):
fslmerge -t output_file `ls epi1_mcf_0*.nii.gz`
Shell script version:
#!/bin/bash
fslmerge -t output_file `ls epi1_mcf_0*.nii.gz`
The command-line version fails, but the shell script one works perfectly.
The error message is specific to the function, but it's included anyway.
** ERROR (nifti_image_read): failed to find header file for 'epi1_mcf_0000.nii.gz'
** ERROR: nifti_image_open(epi1_mcf_0000.nii.gz): bad header info
Error: failed to open file epi1_mcf_0000.nii.gz
Cannot open volume epi1_mcf_0000.nii.gz for reading!
I have been very frustrated with this problem (less so after I figured out that there was a way to get the command to work).
Any help would be appreciated.
(Or is the general consensus that the problem should be looked for in the "fslmerge" function?)
`ls epi1_mcf_0*.nii.gz` is better written as simply epi1_mcf_0*.nii.gz. As in:
fslmerge -t output_file epi1_mcf_0*.nii.gz
The `ls` doesn't add anything.
Note: Posted as an answer instead of comment. The Markdown-lite comment parser choked on my `` `ls epi1_mcf_0*.nii.gz` `` markup.
(I mentioned this in a comment first, but I'll make an answer since it helped!)
Do you have any shell aliases defined? (Type alias) Those will affect commands typed at the command line, but not scripts.
Linux often has ls defined as ls --color. This may affect the output since the colour codes are sent as escape codes through the regular output stream. If you use ls --color=auto it will auto-detect whether its output is a terminal or not. From man ls:
By default, color is not used to distinguish types of files. That is
equivalent to using --color=none. Using the --color option without the
optional WHEN argument is equivalent to using --color=always. With
--color=auto, color codes are output only if standard output is connected to a terminal (tty).