Caret notation and control characters in Vim - vim

In Vim (or in a terminal?), control characters and their caret notation cannot be distinguished, like Enter vs. C-M. However, there is an only exception; BS vs. C-H.
Why are they only privileged?
For example, if you map Enter to NOP, also C-M will be mapped. Other many control characters have the same behavior. On the other hand, BS and C-H can be mapped to different keys respectively.
:map <Enter> <NOP> // also <C-M> will be mapped
:map <BS> <NOP> // only <BS> will be mapped, <C-H> won't

You can insert a backspace into a file by pressing controlV before the backspace. In that case, vim (and most editors) will display it as ^H. But normally backspace is treated as a command (telling vim to do something, depending on the mode).
When moving the cursor in a file containing control characters, you can readily distinguish between a control character versus caret next to another character: as you move the cursor left/right, vim will move two columns for the control character, but one column for each of the caret and other character.

I don't know all the nitty-gritty details, but basically (as I understand it) Vim uses an API to obtain the pressed keys that cannot distinguish between those different keys. For graphical applications like GVIM, all keys could be distinguished without problems; it's just a matter of switching to a more powerful (and GUI-only) API. For terminals, not all can actually distinguish between those keys (with <BS> / <C-H> the only exception). Many modern once can, though, it it again becomes a matter of switching to a more powerful API.
Some people (foremost Paul LeoNerd Evans) want to fix that (even for console Vim in terminals that support this), and have floated various proposals, cp. http://groups.google.com/group/vim_dev/browse_thread/thread/626e83fa4588b32a/bfbcb22f37a8a1f8
But as of today, no patches or volunteers have yet come forward, though many have expressed a desire to have this in a future Vim release.
More details on the problem
Due to the way that the keyboard input is handled internally, it unfortunately isn't generally possible today to distinguish certain keys, even in GVIM. Some key combinations, like Ctrl + non-alphabetic cannot be mapped, and Ctrl + letter vs. Ctrl + Shift + letter cannot be distinguished. (Unless your terminal sends a distinct termcap code for it, which most don't.) In insert or command-line mode, try typing the key combination. If nothing happens / is inserted, you cannot use that key combination. This also applies to <Tab> / <C-I>, <CR> / <C-M> / <Esc> / <C-[> etc. (Only exception is <BS> / <C-H>.) This is a known pain point, and the subject of various discussions on vim_dev and the #vim IRC channel.

This is because the XK_BackSpace keysym generally causes terminals to send ASCII DEL (0x7F), not ASCII BS (0x08, ^H). To balance things out, XK_Delete sends a sequence like \033[3~. Thus, unlike XK_Tab/^I, XK_BackSpace and ^H send completely different bytes.
Programs using the terminfo database can read the kbs and kdch1 capabilities to know what Backspace and Delete send, respectively. You can see your terminal's value with tput:
$ tput kbs | sed -n l
\177$
\177 is octal for ASCII DEL (see ascii(7)).
Another method is to check the termios(4) structure, which controls the “cooked mode” used by e.g. cat to read input (unlike bash or zsh). This is what Vim does in get_tty_info (src/os_unix.c), uh, after it checks termcap(5), compares backspace and delete, and, uh, maybe calls :fixdel? But you get the idea.

Related

In Vim, the 5-key on the numeric keypad inserts a line above the current line with "E"

While using Vim under Linux Mint 20.2 (with NumLock off), when I hit the NumPad center key (5), a line is inserted above the current line, a capital E appears, and I am left in Insert mode. I often mistakenly hit the 5 key when trying to navigate using the NumPad cursor keys, and I would love to redefine the NumPad 5 to do nothing. I've tried remapping the key in .vimrc (using noremap <Esc>[E <Esc>, for example), but it has no effect. Note, the ANSI sequence produced when I press the NumPad5 is \033[E
FYI, I have TERM=xterm-256color.
Note: this answer generally talks about the numpad in a no-numlock context. All numbers on the numpad can be mapped when numlock is on by using <k0>...<k9>. References to the mapability of keys is therefore in a no-numlock context
Note, the ANSI sequence produced when I press the NumPad5 is \033[E
Vim doesn't see it that way.
On my system (Mint 20.2 with Cinnamon under X11), no-numlock 5 produces ^[OE. That O is very important to the observed behavior. o means "go to a new line under and enter insert mode", and O means the exact same thing, but to a new line above. The next E is then interpreted as an insert mode character, because Vim.
Moreover, esc is represented as ^[, so for mapping purposes, you only need <esc>. This point is easier to demonstrate in Vim:
^[ is additionally treated as a single character. If you want to play around with this, pop into insert mode and press <C-v>. The next key(bind) you press will be printed literally to the screen
This means what you actually want is:
nnoremap <Esc>OE <nop>
Note that I'm not sure how terminal input processing applies here; this may only hold for certain terminals, or even only apply to X11.
Another trick in general here, and a way to make sure the keycode is correct for your terminal, is that you can type :nnoremap , and then press Ctrl+v+Numpad 5. This will give you the exact keycode as Vim sees it, which you then can map to whatever you have planned.
If you do this in gVim, however, you'll find that numpad 5 without numlock doesn't actually produce a keycode detectable by gVim. This presumably means it can't be mapped (unless there's some way I just don't know about - if there is, please prove me wrong). But because it doesn't spew out a terminal code, there's the added benefit of not needing to remap it for this particular purpose, because it doesn't have the problem you're trying to avoid here in the first place.
romainl mentioned :help key-notation in the comments, but there's something to keep in mind here:
gVim not producing a keycode usually means there isn't one. <C-v> followed by numpad 9 in gVim produces <kPageUp>, but followed by numpad 5, it doesn't produce anything. Secondly, if you read :h key-notation, you'll see that it only defines home, end, up, and down, as well as the various operators, enter, and the decimal point. 2, 4, 6, and 8 are all registered as normal arrow keys, meaning <Down>, <Left>, <Right>, and <Up>. There's no way to detect these on the numpad specifically, but that's not really important.
If you've been keeping track, you'll find that numpad 5 is excluded. Note: When numlock is on, <C-v> + numpad 5 doesn't output anything in Vim or gVim, because the definition is:
Insert the next non-digit literally.
See :h i_CTRL-V.
TL;DR:
<C-v> is your friend.
Use:
nnoremap <Esc>OE <nop>
Or use <C-v> to insert the keycode, that may or may not be terminal dependent.
The mapping isn't necessary in gVim, and may not be possible.
The reason why you see this behavior is that Vim uses only the termcap interface of the terminal database. If it used terminfo, it would see a wider range of information.
Typing the key on the keyboard is misleading, because Vim (like almost all terminal applications) puts the keypad into application mode, which causes the terminal to send Escape and O rather than Escape and [ as a prefix. In terminal notation, that is \EO. In XTerm Control Sequences, that is referred to as SS3 (technically a misnomer since that applies to output, while keyboard input is input — but since no standards apply to keyboard input, that's good enough).
The terminal description for xterm at one point assigned \EOE to the kbeg capability (i.e., terminfo, which termcap names #1). That was a while back, around 1996.
Terminfo names are generally 3-5 characters, while termcap names are always 2.
Standard terminfo has a limited number of special keys (such as those for the numeric keypad which you are familiar with). All of the standard termcap names correspond to these terminfo names. In the standard arrangement, there are only five keys associated with a keypad: ka1, ka3, kb2, kc1, and kc3, which correspond to termcap's K1, K2, K3, K4, and K5. (Why there are no standard k2, kb1, kb3, or kc2 is long forgotten).
Looking for ways to improve the terminal descriptions in ncurses, a few different layouts of the standard keys have been proposed. But the best solution lies in adding new keys, using extended capabilities (a feature of ncurses since 1999). Those extensions all have names, and are chosen to conform to terminfo naming conventions. So... in May 2019, the terminfo building blocks xterm+keypad and vt220+keypad were introduced. In the former (used in xterm-new and xterm-256color, that key is named kp5.
There's no kp5 in the standard names listed in the terminfo(5) manpage. It is an extended (user-definable) key. In the older scheme some other key was unavailable for applications.
An ncurses application would automatically be able to use all of the keys, because ncurses uses terminfo (and adds all of the keys in a terminal description to its special cases for timing).
Vim will not see that key's definition because it views only a fraction (about a third) of the terminal description. Since it does not see the definition, it does not add that to the special cases where it will wait (a short time) for the completion of the character-sequence sent on that key. As noted in the other answer, in that case, Vim will see the commands Escape and O to insert the third character in the sequence, "E".

vim <C-S-Key> mapping

I am working with Cygwin/Mintty/Vim.
With <C-v> I see that <C-S-c> is encoded <83>. This mean vim can read it and I can map it using the map command.
Unfortunately if I try:
:inoremap <C-S-c> foobar
it doesn't work...
How can I make it work and why vim refuses to map Unicode keystokes?
Same question for <C-S-F1>. If I execute this command:
:inoremap <C-S-F1> foobar
I will get something like this:
[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~[20;5~
Where F1 is [1;5P and F9 is [20;5~
Vim apparently does not support exotic key combinations involving Control (<C-…>). That includes F1 – though your other attempted mapping ShiftC is supported (and is equivalent to Control with just C). From a post by Tony Mechelynck on a forum thread “Mapping ctrl-; (ctrl semicolon)”:
The only printable keys which can reliably be used with Ctrl, with
predictable results, in cooked mode on any OS, are those defined by
ASCII, and that means the following AND NO OTHERS:
ASCII characters 0x40 to 0x5F (i.e. uppercase A to Z, plus the six
nonalpha characters at-sign, left-bracket, slash, right-bracket,
[caret] and underscore), where Ctrl subtracts 0x40, thus mapping them
to 0x00 to 0x1F respectively. This explains why Ctrl-[ means Esc, Ctrl-I
means Tab, Ctrl-M means Enter, etc.
Lowercase letters, whose Ctrl counterpart is the same as for the
corresponding uppercase (thus Ctrl+letter and Ctrl+Shift+letter are
always the same for a given letter)
In addition, Ctrl-? is mapped to 0x7F (DEL).
Vim may or may not see other Ctrl-combinations, but that depends on the
terminal, and in most cases it won't see them, or confuse them with
something else such as the same key without Ctrl.
As for your <C-S-c> mapping, I’m not totally sure why you can see <C-S-c> with <C-v> but you can’t map it. When I test in MacVim (a GUI), I get the character ^C when I try <C-v><C-S-c>, and :inoremap <C-S-c> foobar works. When I test in OS X’s Terminal, the Terminal swallows the keystroke and beeps, with both <C-v> and :inoremap. In both cases, <C-v>’s behavior is consistent with :inoremap’s, so I don’t know why you are seeing the discrepancy.
Due to the way that the keyboard input is handled internally, this unfortunately isn't generally possible today, even in GVIM. Some key combinations, like Ctrl + non-alphabetic cannot be mapped, and Ctrl + letter vs. Ctrl + Shift + letter cannot be distinguished. (Unless your terminal sends a distinct termcap code for it, which most don't.) In insert or command-line mode, try typing the key combination. If nothing happens / is inserted, you cannot use that key combination. This also applies to <Tab> / <C-I>, <CR> / <C-M> / <Esc> / <C-[> etc. (Only exception is <BS> / <C-H>.) This is a known pain point, and the subject of various discussions on vim_dev and the #vim IRC channel.
Some people (foremost Paul LeoNerd Evans) want to fix that (even for console Vim in terminals that support this), and have floated various proposals, cp. http://groups.google.com/group/vim_dev/browse_thread/thread/626e83fa4588b32a/bfbcb22f37a8a1f8
But as of today, no patches or volunteers have yet come forward, though many have expressed a desire to have this in a future Vim 8 major release.

How to rebind Ctrl-[ in .vimrc

In vim, Ctrl+[ acts like Esc by default. However, I would like to rebind Ctrl+[ to perform a custom action. (Specifically, to Ctrl-t because I find it more natural for [ and ] to have complementary actions for ctags.)
However, when I execute the command:
:nnoremap <C-[> <C-t>
then my arrow key navigation get messed up. I'm not sure what happens, but clearly that's not the ideal way to do it.
I've tried to unbind Ctrl+[ but vim reports that it wasn't bound, and I've tried some tricks like first binding Ctrl+[ to itself and then unbinding. Always the same result.
Side note: Interestingly, when I add it to my .vimrc (as the last command) it's even worse. Something nondeterministic happens and vim opens randomly in one of these 3 states:
The bottom status line says "E73: tag stack empty", implying it received a Ctrl+t-esque command, however the if I hit a nav key like j, it deletes the current and bottom line and then puts me into insert mode. (Happens ~70% of the time.)
Sometimes it's in replace mode. (Happens ~15% of the time.)
Sometimes it's in normal mode. (Happens ~15% of the time.)
How can I properly remap Ctrl+[ to a different function in .vimrc?
You can't rebind Ctrl[. Pressing the Escape key in a terminal sends Ctrl[, just like pressing the Tab key sends CtrlI. There's no separate Esc code.
Vim is probably acting crazily when you try this because basically every ANSI key sequence starts with Ctrl[, so your rebinding is firing on all terminal input, and whatever else is in the key sequence looks like more commands to vim.
Due to the way that the keyboard input is handled internally, this unfortunately isn't generally possible today, even in GVIM. Some key combinations, like Ctrl + non-alphabetic cannot be mapped, and Ctrl + letter vs. Ctrl + Shift + letter cannot be distinguished. (Unless your terminal sends a distinct termcap code for it, which most don't.) In insert or command-line mode, try typing the key combination. If nothing happens / is inserted, you cannot use that key combination. This also applies to <Tab> / <C-I>, <CR> / <C-M> / <Esc> / <C-[> etc. (Only exception is <BS> / <C-H>.) This is a known pain point, and the subject of various discussions on vim_dev and the #vim IRC channel.
Some people (foremost Paul LeoNerd Evans) want to fix that (even for console Vim in terminals that support this), and have floated various proposals, cp. http://groups.google.com/group/vim_dev/browse_thread/thread/626e83fa4588b32a/bfbcb22f37a8a1f8
But as of today, no patches or volunteers have yet come forward, though many have expressed a desire to have this in a future Vim 8 major release.

in vim, enter is being mapped to a ctrl key custom key map

I am encountering some strange behavior with a keymap in vim that uses the ctrl key. I would guess that this has a simple cause and solution, but I just can't see it.
During the editing of restructuredtext, I find myself typing things like
:math:`x`
often (this :math: role will cause whatever is inside the ticks to be typeset as math in e.g. the latex output).
I want to map a key like m to enter :math:`` into the text and position the cursor inside the ticks.
I have done this
map m i:math:``ha
and that seems to work fine.
However, I would like to be able to use this map in insert mode. For that, I thought that using ctrl+m would be best. I've done
imap <c-m> :math:``ha
Although that correctly inputs :math:`` and positions the cursor inside the ticks when I do ctrl+m, the trouble is that after this point, every time I press enter in insert mode, it runs the same command as if I typed ctrl+m. In other words, enter in insert mode now seems to be mapped to
:math:``ha
as well.
It seems like it is definitely something to do with using the ctrl key. If I bind e.g. the F5 key as follows
imap <F5> :math:``ha
everything is fine.
I can use the e.g. F5 key and save myself any further bother, but I would like to know what is going on for future reference.
Is there something basic about the use of the ctrl key in a key map that I am missing?
thank you,
You have to use a different control combination for your mapping, e.g. <C-g>.
Due to the way that the keyboard input is handled internally, this unfortunately isn't generally possible today without these side effects, even in GVIM. Some key combinations, like Ctrl + non-alphabetic cannot be mapped, and Ctrl + letter vs. Ctrl + Shift + letter cannot be distinguished. (Unless your terminal sends a distinct termcap code for it, which most don't.) This also applies to <Tab> / <C-I>, <CR> / <C-M> / <Esc> / <C-[> etc. (Only exception is <BS> / <C-H>.) This is a known pain point, and the subject of various discussions on vim_dev and the #vim IRC channel.
Some people (foremost Paul LeoNerd Evans) want to fix that (even for console Vim in terminals that support this), and have floated various proposals, cp. http://groups.google.com/group/vim_dev/browse_thread/thread/626e83fa4588b32a/bfbcb22f37a8a1f8
But as of today, no patches or volunteers have yet come forward, though many have expressed a desire to have this in a future Vim 8 major release.
if you do a
:h keycodes
you will see:
notation meaning equivalent decimal value(s) ~
-----------------------------------------------------------------------
....
<CR> carriage return CTRL-M 13 *carriage-return*
<Return> same as <CR> *<Return>*
<Enter> same as <CR> *<Enter>*
so it tells, <c-m> is same as <Enter> (same keycode 13)
you could test in your shell too, for example, type ls and then <c-m> instead of <Enter>

Why doesn't this .vimrc mapping work in terminal (but it does in MacVim): map <S-Enter> O<Esc>

I have two lines in my .vimrc file:
map <S-Enter> O<Esc>
map <CR> o<Esc>
The second line works as expected, but the first line does not.
The idea is that if I hit the Enter key then vim should insert a new line below the cursor position and move the cursor down to that line. If I hit Shift-Enter, vim should insert a new line above the cursor position and move the cursor up to that line. This mapping should work because the capital O key from normal mode will insert a newline above the line the cursor is on, place the cursor at the beginning of that line, and then place the user into insert mode. Capital O followed by <Esc> should simply revert vim back to command mode.
I have checked and capital O works as I expect. So why doesn't my mapping work? I have tried using map <Shift-CR> and map <Shift-Enter>, and neither of those does the trick.
EDIT Turns out it works in MacVim but not in iTerm. If anyone can explain why and suggest a workaround, I'd appreciate it.
Are you using console Vim? Some key combinations can only be mapped in GVIM. In all / most Linux terminals, Enter cannot be combined with Shift or Ctrl. Your mapping per se is correct, but you'd have to use a different LHS in the mapping.
Due to the way that the keyboard input is handled internally, some key combinations cannot be used even in GVIM. Some key combinations, like Ctrl + non-alphabetic cannot be mapped, and Ctrl + letter vs. Ctrl + Shift + letter cannot be distinguished. This is a known pain point, and the subject of various discussions on vim_dev and the #vim IRC channel.
Some people (foremost Paul LeoNerd Evans) want to fix that (even for console Vim in terminals that support this), and have floated various proposals, cp. http://groups.google.com/group/vim_dev/browse_thread/thread/626e83fa4588b32a/bfbcb22f37a8a1f8
But as of today, no patches or volunteers have yet come forward, though many have expressed a desire to have this in a future Vim 8 major release.

Resources