Some programs makes beautiful progressbars and stuff using ANSI escape sequences. That's nice.
What's not nice though is that if i put the output of that kind of program into a file and then try to view it it's filled with strange escape sequences.
Is there a way to strip away all the ANSI codes while logging?
I usually log the output of a script this way:
./script >> /tmp/output.log
Try:
$ TERM=dumb ./script >> /tmp/output.log
If that doesn't work, it's because the ANSI codes have been hard-coded into the script, so there is no easy way to remove them. If it does, it's because it's doing the right thing, delegating things like pretty output to libncurses or similar, so that when you change the TERM variable, the library no longer sends those codes.
Related
When using fish shell in a terminal-emulator (such as terminator) together with a command that outputs lots of text it could be useful to get some color coding on the output. I know that a script can add color code information to the output like "grep --color=auto". I guess it's possible to modify the fish terminal to scan through the output and add this in special places?
What I want to do is that the text "error" appearing in the output from any script always is marked red and that "warning" always is marked yellow. Anyone knows if this is possible by introducing function files in the ~/.config/fish/functions dir or similar?
This is basically a layering violation. Usually the output of external commands does not go back through the shell. It goes directly to the terminal.
Also, anything you do here has the potential to slow output down. (And because of fish issue #1396, this can be rather extreme).
That said, it's possible if you always pipe to a function like this
function colorstuff
while read -l input
switch $input
case "*error*"
set_color red
echo $input
case "*warning*"
set_color yellow
echo $input
case "*"
set_color normal
echo $input
end
end
set_color normal
end
Use it like somecommand | colorstuff. (And maybe add 2>&1 if you also wish to have stderr colored)
In my tests, this causes a noticeable slowdown, and even with that issue fixed it will still be slower since it has to match every single line.
Really, the real solution is for whatever tool you are using to color itself, since it knows what the output means. All this can do is look for keywords.
For general output colorization needs, I added the grc plugin to Tackle for precisely that purpose.
I'd like to have a command I can insert into a command pipeline that adds color escapes to its input according to vim's syntax highlighting capabilities.
For example:
cat somefile.js | vim - <???> | less
The resulting text would be that of somefile.js, but colorized according to how the current vim configuration would do it in-editor.
It occurs to me that this must be possible. I agree that the example up there isn't what a sane man might call exactly useful, but that doesn't mean the idea never is.
I think your idea has one basic flaw: that nobody ever thought about allowing such a thing.
Clearly vim is capable of doing syntax highlighting. But I'll bet you an ice cream cone that if you can manage to get vim to stream text through and process it, that you won't like the results.
Consider what happens when you pipe text through more (or less if you prefer). When it goes to the terminal, these programs display one screenful and wait for you to hit the space bar. But if you redirect stdout to some other place than the terminal, these programs notice this and simply copy their input to their output unchanged.
If vim doesn't notice that you are piping text through, it is likely to send cursor-movement commands that you probably don't want in your output. If vim does notice, it is likely to just pass the text, and not syntax-color it. Only if vim does do the syntax-coloring but does not inject cursor-movement stuff will your idea work.
You could try it. Here's an answer that discusses piping stuff through vim:
Execute a command within Vim from the command line
But I say why not pipe your text through a program that was designed and intended to have text piped through it? Pygments can colorize every major programming language and markup format.
http://pygments.org/
The major advantage I see for your idea: you can customize the way vim does syntax coloring, get it the way you want it, and then also use vim to process your text. But it's probably not that hard to customize Pygments, and it might even be satisfactory out of the box, in which case it would definitely be the easiest way to go. And Pygments not only has ANSI sequence output, it also has HTML output, RTF, LaTeX, etc. So if you get Pygments working the way you want it to, it should be able to output whatever output format you need; vim will only have the ANSI sequence one.
There's a Perl module called Text::VimColor that I've heard will do kinda what you're looking for.
http://search.cpan.org/dist/Text-VimColor/
But let me ask this: Why do want it to go through less? Why not use vim as a perfectly good file viewer? view - will read from standard input in read-only mode.
https://gist.github.com/echristopherson/4090959
Via https://superuser.com/a/554531/7198.
Tried on /etc/passwd and it works surprisingly well!
This might be what you're after
cat filename.sh | vim - -c 'syntax on; syn=bash'
This is ugly, but you could alias this:
alias vim.sh="vim -c 'syntax on; syn=bash'"
Then use like this:
cat filename.sh | vim.sh -
Use vimcat !
wget -O /usr/local/bin/vimcat "https://www.vim.org/scripts/download_script.php?src_id=23422"
chmod 755 /usr/local/bin/vimcat
vimcat /etc/passwd
See also: https://www.vim.org/scripts/script.php?script_id=4325
I'm using cucumber to run some tests. It colorizes its output using ANSI escapes. This is great, but currently its producing more output than I care about, and shoving things I do care about off the screen. There doesn't seem to be a way to eliminate the other lines from within cucumber, but I can pipe the output through grep to pare down to the ones I care about.
The downside of this solution, though, is that all the colors are lost. I know it's not my shell or grep's fault, because % echo "\e[35mhello\e[00m world" | grep hello works just fine, so it must be cucumber disabling its own color somehow.
How can I preserve the colored output when I pipe the output of cucumber?
Doh. It's covered in cucumber -h. Use the -c flag to force colorized output.
How do some programs edit whats being displayed on the terminal (to pick a random example, the program 'sl')? I'm thinking of the Linux terminal here, it may happen in other OS's too, I don't know. I've always thought once some text was displayed, it stayed there. How do you change it without redrawing the entire screen?
Depending on the terminal you send control seuqences. Common sequences are for example esc[;H to send the cursor to a specific position (e.g. on Ansi, Xterm, Linux, VT100). However, this will vary with the type or terminal the user has ... curses (in conjunction with the terminfo files) will wrap that information for you.
Many applications make use of the curses library, or some language binding to it.
For rewriting on a single line, such as updating progress information, the special character "carriage return", often specified by the escape sequence "\r", can return the cursor to the start of the current line allowing subsequent output to overwrite what was previously written there.
try this shellscript
#!/bin/bash
i=1
while [ true ]
do
echo -e -n "\r $i"
i=$((i+1))
done
the -n options prevents the newline ... and the \r does the carriage return ... you write again and again into the same line - no scroling or what so ever
If you terminate a line sent to the terminal with a carriage return ('\r') instead of a linefeed ('\n'), it will move the cursor to the beginning of the current line, allowing the program to print more text over top of what it printed before. I use this occasionally for progress messages for long tasks.
If you ever need to do more terminal editing than that, use ncurses or a variant thereof.
There are characters that can be sent to the terminal that move the cursor back. Then text can be overwritten.
There is a list here. Note the "move cursor something" lines.
NCurses is a cross-platform library that lets you draw user interfaces on smart terminals.
Corporal Touchy has answered how this is done at the lowest level. For easier development the curses library gives a higher level of control than simply sending characters to the terminal.
To build on #Corporal Touchy's answer, there are libraries available that will handle some of this functionality for you such as curses/ncurses
I agree with danio, ncurses is the way to go. Here's a good tutorial:
http://tldp.org/HOWTO/NCURSES-Programming-HOWTO/
If I do something like:
$ cat /bin/ls
into my terminal, I understand why I see a bunch of binary data, representing the ls executable. But afterwards, when I get my prompt back, my own keystrokes look crazy. I type "a" and I get a weird diagonal line. I type "b" and I get a degree symbol.
Why does this happen?
Because somewhere in your binary data were some control sequences that your terminal interpreted as requests to, for example, change the character set used to draw. You can restore everything to normal like so:
reset
Just do a copy-paste:
echo -e '\017'
to your bash and characters will return to normal. If you don't run bash, try the following keystrokes:
<Ctrl-V><Ctrl-O><Enter>
and hopefully your terminal's status will return to normal when it complains that it can't find either a <Ctrl-V><Ctrl-O> or a <Ctrl-O> command to run.
<Ctrl-N>, or character 14 —when sent to your terminal— orders to switch to a special graphics mode, where letters and numbers are replaced with symbols. <Ctrl-O>, or character 15, restores things back to normal.
The terminal will try to interpret the binary data thrown at it as control codes, and garble itself up in the process, so you need to sanitize your tty.
Run:
stty sane
And things should be back to normal. Even if the command looks garbled as you type it, the actual characters are being stored correctly, and when you press return the command will be invoked.
You can find more information about the stty command here.
You're getting some control characters piped into the shell that are telling the shell to alter its behavior and print things differently.
VT100 is pretty much the standard command set used for terminal windows, but there are a lot of extensions. Some control character set used, keyboard mapping, etc.
When you send a lot of binary characters to such a terminal, a lot of settings change. Some terminals have options to 'clear' the settings back to default, but in general they simply weren't made for binary data.
VT100 and its successors are what allow Linux to print in color text (such as colored ls listings) in a simple terminal program.
-Adam
If you really must dump binary data to your terminal, you'd have much better luck if you pipe it to a pager like less, which will display it in a slightly more readable format. (You may also be interested in strings and od, both can be useful if you're fiddling around with binary files.)