From time to time I have to run a command-line tool (a Python script) whose output seems to break my terminal.
After the execution is finished, the typing feedback is gone (I can't see what I'm typing), and also line breaks are not displayed. This happens if the terminal is started remotely via Putty, and also locally when using gnome-terminal.
For example, after the problem happens, if I type ENTER pwd ENTER, I would expect to see:
[userA#host006 ~]$
[userA#host006 ~]$ pwd
/home/userA
[userA#host006 ~]$
But actually the output is:
[userA#host006 ~]$ [userA#host006 ~]$ /home/userA
[userA#host006 ~]$
The only way to fix it is to close that terminal and start a new one.
Maybe be related: the script output contains some terminal-based formatting (e.g. invert foreground/background to highlight some status messages). If I dump this output to a file I can see things like [07mSome Message Here[0m.
Any ideas what I could do to prevent this?
Execute the command reset and your terminal should be restored (reference).
This issue happens generally when dumping binary data to the terminal STDOUT which when the escape codes received are processed can do anything from change the color of the text, disable echo, even change character set.
The easy way to avoid this is to ensure you do not dump unknown binary data to the terminal, and if you must then convert it to hexadecimal to ensure it doesn't change the terminal settings.
To elaborate on Joshua Briefman's answer, executing reset -c will only reset the control characters responsible for your problem:
tset, reset - terminal initialization
Usage: tset [options] [terminal]
Options:
-c set control characters
-e ch erase character
-I no initialization strings
-i ch interrupt character
-k ch kill character
-m mapping map identifier to type
-Q do not output control key settings
-r display term on stderr
-s output TERM set command
-V print curses-version
-w set window-size
Also note the following form the command's manual:
Note, you may have to type
<LF>reset<LF>
(the line-feed character is normally control-J) to get the terminal to work, as carriage-return may no longer work in the abnormal state. Also, the terminal will often not echo the command.
Related
This question already has answers here:
How to trick an application into thinking its stdout is a terminal, not a pipe
(9 answers)
Closed 5 years ago.
Various bash commands I use -- fancy diffs, build scripts, etc, produce lots of color output.
When I redirect this output to a file, and then cat or less the file later, the colorization is gone -- presumably b/c the act of redirecting the output stripped out the color codes that tell the terminal to change colors.
Is there a way to capture colorized output, including the colorization?
One way to capture colorized output is with the script command. Running script will start a bash session where all of the raw output is captured to a file (named typescript by default).
Redirecting doesn't strip colors, but many commands will detect when they are sending output to a terminal, and will not produce colors by default if not. For example, on Linux ls --color=auto (which is aliased to plain ls in a lot of places) will not produce color codes if outputting to a pipe or file, but ls --color will. Many other tools have similar override flags to get them to save colorized output to a file, but it's all specific to the individual tool.
Even once you have the color codes in a file, to see them you need to use a tool that leaves them intact. less has a -r flag to show file data in "raw" mode; this displays color codes. edit: Slightly newer versions also have a -R flag which is specifically aware of color codes and displays them properly, with better support for things like line wrapping/trimming than raw mode because less can tell which things are control codes and which are actually characters going to the screen.
Inspired by the other answers, I started using script. I had to use -c to get it working though. All other answers, including tee, different script examples did not work for me.
Context:
Ubuntu 16.04
running behavior tests with behave and starting shell command during the test with python's subprocess.check_call()
Solution:
script --flush --quiet --return /tmp/ansible-output.txt --command "my-ansible-command"
Explanation for the switches:
--flush was needed, because otherwise the output is not well live-observable, coming in big chunks
--quiet supresses the own output of the script tool
-c, --command directly provides the command to execute, piping from my command to script did not work for me (no colors)
--return to make script propagate the exit code of my command so I know if my command has failed
I found that using script to preserve colors when piping to less doesn't really work (less is all messed up and on exit, bash is all messed up) because less is interactive. script seems to really mess up input coming from stdin even after exiting.
So instead of running:
script -q /dev/null cargo build | less -R
I redirect /dev/null to it before piping to less:
script -q /dev/null cargo build < /dev/null | less -R
So now script doesn't mess with stdin and gets me exactly what I want. It's the equivalent of command | less but it preserves colors while also continuing to read new content appended to the file (other methods I tried wouldn't do that).
some programs remove colorization when they realize the output is not a TTY (i.e. when you redirect them into another program). You can tell some of those to use color forcefully, and tell the pager to turn on colorization, for example use less -R
This question over on superuser helped me when my other answer (involving tee) didn't work. It involves using unbuffer to make the command think it's running from a shell.
I installed it using sudo apt install expect tcl rather than sudo apt-get install expect-dev.
I needed to use this method when redirecting the output of apt, ironically.
I use tee: pipe the command's output to teefilename and it'll keep the colour. And if you don't want to see the output on the screen (which is what tee is for: showing and redirecting output at the same time) then just send the output of tee to /dev/null:
command| teefilename> /dev/null
When I run "cat /bin/bash",the final interactive prompt becomes
sh-5.0$ 1;2c1;2c
sh: 1: command not found
sh: 2c1: command not found
sh: 2c: command not found
sh-5.0$
And when I press "Enter", it runs something! The cat command only writes to stdout, why there are some bytes being put into stdin? Is this a vulnerability?
Some terminals and emulators have various "answerback" facilites where they send some text back to the computer upon reception of certain control codes. It seems likely that your bash binary contained some such control code.
And yes, this can be a vulnerability, especially if the response text can also be controlled by control codes (which one hopes it can't, nowadays, but one never knows...)
More here: https://unix.stackexchange.com/questions/15101/how-to-avoid-escape-sequence-attacks-in-terminals
# cat > file
Awaits input from user, type desired text and press CTRL+D (hold down Ctrl Key and type ‘d‘) to exit. The text will be written in test2 file. You can see content of file with following cat command.
in your case
# cat /bin/bash > fileoutput
So I was installing node and somehow I ended up in the /user/local/bin folder checking what was there.
I wanted to so a ls | grep node but did a ls | cat node. The node directory then started to show itself in all its glory. I chuckled, laughed, then stopped the cat but that left my terminal looking something like this:
It is easily fixable by exiting the window.
I don't recommend you try it but i've managed to recreate it reliably. Just cat the node directory leave it for a few seconds then cancel it.
My question is what on earth is going on here? AFAIK using cat shouldn't be able to make this stuff happen.
Terminals interpret certain combinations of bytes as commands. This is how command line programs can output colors, or change the cursor position. What you're seeing is the output after one of these commands activated a box drawing character set by accident. You can activate it yourself with echo or printf:
my cool prompt$ printf '\033(0'
└≤ ␌⎺⎺┌ ⎻⎼⎺└⎻├$
To get out of this mess, type reset and press Enter - it will clear the screen and set all settings back to default.
└≤ ␌⎺⎺┌ ⎻⎼⎺└⎻├$ ⎼␊⎽␊├
(..screen clears..)
my cool prompt$
You managed to output the contents of the node executable to the terminal. What happens when you output random octets to the terminal depends on what those characters are and on the terminal. It often leaves the terminal in a state that the user did not really expect.
I have a program that runs and shows a GUI window. It also prints a lot of things on the shell. I need to view the first thing printed and the last thing printed. the problem is that when the program terminates, if I scroll to the top of the window, the stuff printed when it began is removed. So stuff printed during the program is now at the top. So that means I can't view the first thing printed.
Also I tried doing > out.txt, but the problem is that the file only gets closed and readable when I manually close the GUI window. If it gets outed to a file, nothing gets printed on the screen and I have no way to know if the program finished. I can't modify any of the code too.
Is there a way I can see the whole list of text printed on the shell?
Thanks
You can just use tee command to get output/error in a file as well on terminal:
your-command |& tee out.log
Though just keep in mind that this output is line buffered by default (4k in size).
When the output of a program goes to your terminal window, the program generally flushes its output after each newline. This is why you see the output interactively.
When you redirect the output of the program to out.txt, it only flushes its output when its internal buffer is full, which is probably after every 8KiB of output. This is why you don't see anything in the file right away, and you don't see the last things printed by the program until it exits (and flushes its last, partially-full buffer).
You can trick a program into thinking it's sending its output to a terminal using the script command:
script -q -f -c myprogram out.txt
This script command runs myprogram connected to a newly-allocated “pseudo-terminal” (or pty for short). This tricks myprogram into thinking it's talking to a terminal, so it flushes its output on every newline. The script command copies myprogram's output to your terminal window and to the file out.txt.
Note that script will write a header line to out.txt. I can't find a way to disable that on my test Linux system.
In the example above, I assumed your program takes no arguments. If it does, you either need to put the program and arguments in quotes:
script -q -f -c 'myprogram arg1 arg2 arg3' out.txt
Or put the program command line in a shell script and pass that shell script to the script command.
I would like my tcsh script to launch an editor (e.g., vi, emacs):
#!/bin/tcsh
vi my_file
This starts up vi with my_file but first displays a warning "Vim: Warning: Output is not to a terminal" and my keystrokes don't appear on the screen. After I kill vi, my terminal window is messed up (no newlines), requiring a "reset". I tried "emacs -nw", "xemacs -nw", and pico with similar results. "xemacs" works but launches a separate window. I want to reuse the same terminal window.
Is there a way to launch an editor from a script so that it reuses the same terminal window?
I answered my own question! You have to redirect terminal input and output:
#!/bin/tcsh
vi my_file < `tty` > `tty`
The reason you're getting the error is that when you start a shell in your environment, it's starting in a subshell that has STDIN and STDOUT not connected to a TTY — probably because this is in something like a pipeline. When you redirect, you're opening a new connection directly to the device. So, for example, your command line turns
$ vi < `tty` > `tty`
into
$ vi < /dev/ttys000 > /dev/ttys000
So you're not really using your old STDIN/STDOUT, you're creating two new files and mapping them to your vi process's STDIN/STDOUT.
Now, tell us what you're doing with this and we'll tell you how to avoid this kludge.
I wanted to do something similar. I wanted an alias that would find the last file I was working on, and open it in vi(1) for editing. Anyway, I couldn't figure out how to do it as a readable alias (in tcsh) so I just created an ugly shell script (csh because I'm old) instead:
#!/bin/csh
set DIR = "~/www/TooMuchRock/shows/"
set file = $DIR`ls -t $DIR | head -1`
set tty = `tty`
vi $file <$tty >$tty
(1) kraftwerk:bin> which vi
vi: aliased to /usr/local/bin/vim -u ~/.exrc
Absolutely. :-)
Write your script and have it call the EDITOR environment variable, which you will have set to "emacsclient". Then start up Emacs, execute M-x server-start, switch to a shell buffer (M-x shell) and execute your script. Emacsclient will pop up the thing to be edited and C-x # will act as a "done" command and take you back to your script with edits completed or aborted, as you choose.
Enjoy.
Edit: I meant to add that these days Emacs IS my terminal program. I have dozens of shell buffers and never have to worry about losing output and can use all the power of Emacs to manipulate and analyse the terminal output. And have Emacs scripts generate input to the shells. Awesome actually. For example, watching Tomcat output scroll by in a shell buffer while editing sources or processing mail or doing most any Emacs thing is very convenient. When a Tomcat stack trace appears I can quickly respond to it.
Had the same trouble with 'pinfo' in a shell script 'while' loop. The line can be used in the script, it uses 'ps' to find the tty of the current process number, "$$", and stores that tty in $KEY_TTY:
KEY_TTY=/dev/`ps | grep $$ | tr -s '[:blank:]' | cut -d " " -f 3`
Later in the script, just call the tty-only proggie, with $KEY_TTY as input, in my case it was:
pinfo -m $s $page < $KEY_TTY
For 'vi' it'd be:
vi $a < $KEY_TTY > $KEY_TTY
The advantage is that the script as a whole can still accept STDIN input, and 'vi' (or whatever) should work fine -- without having to remember to set any environmental variables before running the script.
Set your terminal tty to a variable, and then redirect the editor i/o through that variable.
In your script:
#!/bin/sh
ls | while read a; do vi $a < $MYTTY >$MYTTY; done
And then execute the script with:
$ MYTTY=`tty` ./myscript >/tmp/log
I was able to get the desired behavior under bash+Cygwin+Terminator:
#!/bin/bash
vim foo
Run the script, vim loads, no error messages, behaves as normal. There are undoubtedly dozens of variations between our setups, however, so I can't hazard a guess as to what makes the difference. I'm curious what it is, but you got it working, which is the important part.