I encountered a problem with my terminal where when I paste text, it is prefixed by 00~ and suffixed with 01~.
For example, I will highlight text and push Command-C. I then push Command-V into the terminal and I see those weird characters pop up at the beginning and end of the text.
For example, I can highlight text and paste it into the terminal. I then see 00~text01~.
The text can be from anywhere, even from the Terminal itself. I do not have any copy/paste plugins installed, this is just the normal Copy/Paste. I am using the default Mac Terminal without any modifications.
I did some searching online, apparently the Paste wraps the text in special characters so that certain applications will see that this is pasted text and will handle it properly. However, the terminal is not handling this correctly, and is therefore not removing the weird characters. Apparently this paste mode is called the "Bracketed Paste Mode" http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#h2-Bracketed-Paste-Mode
I found another question that gave a solution on how to solve this issue on a linux machine, but after trying that solution I still have that same problem.
Can someone tell me how I can disable bracketed paste mode for the terminal? Or tell me the right way to get rid of these annoying characters?
What happens when text is pasted
Text has made it into the "system" (e.g. X, MacOS/Aqua) copy buffer from somewhere, maybe from the same terminal. The text is not altered here.
The text is pasted into the terminal; that is, "system" sees to that the terminal (e.g. xterm) receives the unaltered character sequence from the copy buffer. The terminal is aware that this is a paste, not keyboard input.
The terminal sends the char sequence in the buffer to the program running in the foreground (a shell, an editor, whatever). To the program the received data is indistinguishable from user input through the keyboard.
Discussion
This transparency (or opaqueness? whatever) is often a good thing much like the Unix paradigm of transparent pipe plumbing in general. But sometimes programs could deal with the data better if they knew it is pasted. For example an editor like vim could switch off auto indent — after all, the code is likely indented already!
Bracketed paste
Enter bracketed paste. For principal reasons the paradigm of transparent data piping cannot be altered; but the data can be decorated with sequences which would ordinarily not appear in terminal input to mark its start and end. If the terminal is so configured — for the xterm the configuration would be to send ESC [ ? 2 0 0 4 h — the pasted data is bracketed with escape sequences: ESC [ 2 0 0 ~ <buffer contents> ESC [ 2 0 1 ~.
The foreground program receives this "decorated" data, and it's up to to the program to handle it. A naive program treats all of it as user input, which is what you see.
A good discussion of bracketed paste can be found in this article.
Remedies
There are two issues in your case: The terminal ends up unexpectedly in bracketed paste mode; and the receiving program — presumably the shell — does not handle it.
One solution is user83536's: Identify the program which leaves the terminal in that state and call it through a wrapper which simply switches bracketed paste mode off again after the program has ended.
The program probably tried to switch bracketed paste mode off but failed. One reason can be that it sends the wrong escape sequence. Try setting the TERMINAL environment variable to the value best describing your terminal.
Try to switch off bracketed paste in the offending application. In vim one would say set t_BE=. That prevents vim from putting the terminal in bracketed paste mode and when it is set in a session, sends the "end bracketed paste mode" to the terminal.
Embrace bracketed paste. It seems to be a good idea. For the bash and other programs using readline one would put set enable-bracketed-paste on. For vim one could follow the suggestions here.
This may not apply directly to your problem, but I found this symptom to probably, in my case, be caused by my editor-of-choice 'mcedit' (Midnight Commander)
To alleviate the bug problem, I added the following function to my .bashrc file:
### vvv 'function mcedit' is a fix-up for the ~0/~1 paste problem
function mcedit() { command mcedit $# ; printf '\e[?2004l' ; }
Then 'source .bashrc'
Now every time I execute 'mcedit', it automatically adds the 'printf "\e[?2004l"' when I close out to reset the "Bracketed Paste Mode"
Works for me, YMMV.
To disable bracketed paste mode in your terminal, run the following command:
printf '\e[?2004l'
To disable bracketed paste globally, on Linux, add this line to ~/.inputrc :
set enable-bracketed-paste 0
To disable only in the current running Xterm (v 372) (running bash shell version 5.1.16 (probably earlier too, but I don't know)):
% bind 'set enable-bracketed-paste 0'
With either of the above two methods, you can re-enable bracketed paste (in the current Xterm) in the obvious way, namely:
% bind 'set enable-bracketed-paste 1'
This may be an odd question, but still. I use cat to display a file in bash (KDE Konsole),
cat foobar.rb
Now, I would like to use Vim to colourize that foobar.rb file according to what you would get when you start foobar.rb in Vim. Edit: But only for display purpose, on the terminal.
I am not sure this is possible, but I thought it would be neat if I could use Vim for that.
I really just want colourized keywords, and Vim has the perfect colour definitions.
So I thought combining this would be great.
Is this possible in Vim out of the box though?
One approach would be to use a library such as Pygments, which is a general purpose syntax highlighter. You could write a wrapper called ccat or something that would apply syntax highlighting to an input file and write to stdout.
If you want to page up and down in a highlighted file, you can use less with the -R switch, which passes control characters through to the terminal directly, preserving colours. So:
ccat file.rb | less -R
But at that point, you're pretty much at the capabilities of view.
I'm not sure if I understand your question correctly, but if you are only looking for a command that will give you a read-only view of the input file (like cat) but with coloured keywords, use view. view is an alternative way to start vim in read-only mode, so you have all syntax highlighting possibilities. From the vim man page:
view Start in read-only mode. You will be protected from writing
the files. Can also be done with the "-R" argument.
gvim gview
The GUI version. Starts a new window. Can also be done with
the "-g" argument.
evim eview
The GUI version in easy mode. Starts a new window. Can also
be done with the "-y" argument.
rvim rview rgvim rgview
Like the above, but with restrictions. It will not be possi-
ble to start shell commands, or suspend Vim. Can also be
done with the "-Z" argument.
I have always seen view on systems that have vim installed.
Closest is the less script that comes with vim:
cat myfile | vim -u /usr/share/vim/vim72/macros/less.vim -
Note the - argument to vim. You may need to change the vim72 to your version (and the whole path if you have it installed elsewhere)
Now, this isn't exactly what you want, because its behaviour is less-like, in that you have to press keys to make it scroll down or complete. However, they are briefer than usual vim. For example, space to scroll down; and q to quit (not :q).
You want a cat-like version; me too. But there doesn't seem to be one.
EDIT uh, there's also a vimpager project, that includes vimcat - exactly what you want. But it doesn't come with vim, and I haven't tried it yet.
vim.org: http://www.vim.org/scripts/script.php?script_id=1723
github: https://github.com/rkitover/vimpager
In bash, when I go back in history, edit some command and run it, this edited command is appended to history and the original one is left intact. But every once in a while I somehow manage to affect the original command, i.e. my edit replaces the original command back in history. I can't put my finger on how this happens.
Can someone explain? My goal is to avoid this, so any edit to a previous command always gets appended to history and never replaces the original.
I somehow manage to affect the original command, i.e. my edit replaces the original command back in history.
Right. If you go back in your history and edit the line without pressing return to execute the command but instead moving to another history entry, you've just edited the history entry. If you then list your history, you will see a * on the line indicating that you edited it. I find this "feature" immensely frustrating. Others have provided good examples of how to reproduce this.
My goal is to avoid this, so any edit to a previous command always gets appended to history and never replaces the original.
I too wanted to disable it. I found the solution via this answer over on unix.stackexchange.
To summarize, you need to enable the revert-all-at-newline readline setting which is off by default. If the setting is on then bash will revert any changes you made to your history when you execute the next command.
To enable this setting in your shell, you should add the following to your ~/.inputrc file and then restart your shell:
$include /etc/inputrc
set revert-all-at-newline on
The first line is needed because I guess that if you supply your own .inputrc file the default /etc/inputrc file is not included which is probably not what you want.
If you go back to some previous command and edit it, but then DON'T execute it (instead using history commands to go to some other command and execute it), then the edits will remain there in your history list.
Pressing Ctrl + C, after editing, counters this behaviour. It leaves the original command in tact i.e. it cancels remembering edits to the original.
Here's my own answer, please correct or provide more details if you can.
When the "vi" option is set in bash ("set -o vi" -- "Use a vi-style command line editing interface"), there are two modes of editing a command from history.
The first mode (let's call it "basic") is when you start editing immediately using Backspace, Del and character keys.
The other mode is the "vi mode", entered when you hit Esc.
If you want to keep your history intact, DO NOT use both modes in the same edit. I don't know how bash works exactly, but you can think of it this way:
Entering the "vi mode" applies any changes done in "basic mode" to the original command, and creates a copy of the command that you can edit further using vi-style commands.
The changes get applied when you hit Enter (execute), Up, Down or j,k (move to another command in history).
The changes do not get applied if you hit Ctrl-C.
Using either basic or vi-style editing ALONE does not affect the original command in history.
What do
echo $HISTCONTROL
echo $HISTIGNORE
give you?
Edit:
I was able to reproduce behavior similar to what you've seen by following these steps:
At the shell prompt, enter:
echo abcd
echo efgh
Press up arrow twice, so "echo abcd" is shown
Press 1 to add that character at the end
Press escape to enter command mode
Press left arrow twice so the cursor is on the "c"
Press x to delete the "c"
Press enter
Now as you step back through history, you'll see a new entry at the end:
echo abd1
and the entry that previously had "echo abcd" will now look like this:
echo abcd1
This is one way, I'm sure there are others.
I have this dataset in a csv file
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504, 0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644, 0.00078635, 0.000730052, 0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669, 0.00230171, 0.00217917
As you can see, the numbers are formatted differently and misaligned. Is there a way in vim to quickly align the columns properly, so that the result is this
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504,0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644,0.00078635, 0.000730052,0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669,0.00230171, 0.00217917
That would be great to copy and paste sections with ctrl-v. Any hints?
If you're on some kind of UNIX (Linux, etc), you can cheat and filter it through the column(1) command.
:%!column -t
The above will parse on delimiters inside string literals which is wrong, so you will likely need pre-processing steps and specifying the delimiter for this file for example:
%!sed 's/","/\&/' | column -t -s '&'
Sometimes we want to align just two columns. In that case, we don't need any plugins and can use pure Vim functionality like this:
Choose a separator. In OP's post this is a comma, in my example this is =.
Add spaces before/after it. I use s/=/= ...spaces... / in visual selection for this.
Locate to the longest word and place cursor after it.
Remove all the extra whitespace using dw and vertical movement.
Example of this technique demonstrated below:
I don't find myself needing to align things often enough to install another plugin, so this was my preferred way of accomplishing it - especially that it doesn't require much thinking.
As sunny256 suggested, the column command is a great way of doing this on Unix/Linux machines, but if you want to do it in pure Vim (so that it can be used in Windows as well), the easiest way is to install the Align plugin and then do:
:%Align ,
:%s/\(\s\+\),\s/,\1/g
The first line aligns the entries on the commas and the second moves the comma so that it's flush with the preceding value. You may be able to use AlignCtrl to define a custom mapping that does the whole lot in one go, but I can never remember how to use it...
Edit
If you don't mind two spaces between entries and you want to do this in one command, you can also do:
:%Align ,\zs
This is a great answer using vim macros: https://stackoverflow.com/a/8363786/59384 - basically, you start recording a macro, format the first column, stop recording then repeat the macro for all remaining lines.
Copy/pasted from that answer:
qa0f:w100i <Esc>19|dwjq4#a
Note the single space after the 100i, and the <Esc> means "press escape"--don't type "<Esc>" literally.
Translation:
qa -- record macro in hotkey a
0 -- go to beginning of line
f: -- go to first : symbol
w -- go to next non-space character after the symbol
100i <Esc> -- insert 100 spaces
19| -- go to 19th column (value 19 figured out manually)
dw -- delete spaces until : symbol
j -- go to next line
q -- stop recording macro
4#a -- run the macro 4 times (for the remaining 4 lines)
We now also have the fabulous EasyAlign plugin, written by junegunn.
Demonstration GIF from its README:
Also, Tabularize is quite good http://vimcasts.org/episodes/aligning-text-with-tabular-vim/
You could use the csv.vim plugin.
:%ArrangeColumn
However, this will not do exactly what you have asked: it will right adjust the contents of cells, whereas you have your values aligned by the decimal point or by the first digit.
The plugin has many other useful commands for working with CSV files.
also if you have very long columns it can be handy to disable default wrapping
:set nowrap
:%!column -t
(note in debian you also have a further option for column -n which if you want to split multiple adjacent delimiters)
Here’s a pure Vim script answer, no plugins, no macros:
It might be most clear to start out with my problem’s solution as an example. I selected the lines of code I wanted to affect, then used the following command (recall that entering command mode from visual mode automatically prepends the “'<,'>”, so it acts on the visual range):
:'<,'>g``normal / "value<0d>D70|P`
Except I did NOT actually type “<0d>”. You can enter unprintable characters on the command line by pressing ctrl-v, then the key you want to type. “<0d>” is what is rendered on the command line after I typed ‘ctrl-v enter’. Here, it’s parsed by the “normal” command as the exit from “/” search mode. The cursor then jumps to “ value” in the current line.
Then we simply [D]elete the rest of the line, jump to column 70 (or whatever you need in your case), and [P]ut what we just deleted. This does mean we have to determine the width of the widest line, up to our search. If you haven’t put that information in your statusline, you can see the column of the cursor by entering the normal mode command ‘g ctrl-g’. Also note that jumping to a column that doesn’t exist requires the setting 'virtualedit'!
I left the search term for the :g(lobal) command empty, since we used a visual block and wanted to affect every line, but you can leave off using a visual selection (and the “'<,'>”) and put a search term there instead. Or combine a visual selection and a search term to narrow things more finely/easily.
Here’s something I learned recently: if you mess up on a complex command mode command, undo with ‘u’ (if it affected the buffer), then press “q:” to enter a special command history buffer that acts much like a conventional buffer. Edit any line and press enter, and the changed command is entered as a new command. Indispensable if you don’t want to have to stress over formulating everything perfectly the first time.
I just wrote tablign for this purpose. Install with
pip3 install tablign --user
Then simply mark the table in vim and do
:'<,'>:!tablign
Pretty old question, but I've recently availed myself of an excellent vim plugin that enables table formatting either on the fly or after-the-fact (as your use case requires):
https://github.com/dhruvasagar/vim-table-mode
I have this in my .vimrc.
command! CSV set nowrap | %s/,/,|/g | %!column -n -t -s "|"
This aligns the columns while keeping the comma, which may be needed later for correct reading. For example, with Python Pandas read_csv(..., skipinitialspace=True), thanks Pandas guys for this smart option, otherwise in vim %s/,\s\+/,/g. It may be easier if your column has the option --output-separator I guess, my doesn't and I'm not sure why (my man page for column says 2004, on ubuntu 18.04, not sure ubuntu will get a new version). Anyway, this works for me, and comment if you have any suggestions.
I made a cli tool written in Perl.
You can find it here: https://github.com/bas080/colcise