There seems to be some sort of asymmetry in the way Vim treats ^M when doing string replacement (:s/x/y/).
Perhaps an example is best; say we have this text file:
foo:bar:biz
I want to split this into lines. This works fine:
:s/:/^M/g
(note that ^M is produced by typing Ctrl-V, Enter)
This results in the text file:
foo
bar
baz
Now, if I undo that and try again, I notice that this does not work:
:s/:/\n/g
Here, the resulting text is:
foo^#bar^#biz
That is to say, they are joined by the ASCII NUL byte (0x00).
Question 1: Why does using \n in the replacement result in NUL bytes?
Now, I figure "okay, I guess ^M is used as the 'line separator' character in some way, for Vim; I can work with that".
So I do another experiment, starting with the one-item-per-line text file:
foo
bar
baz
and now, I want to join them with colons, so it looks like the very first incarnation, above.
So I run:
:%s/^M/:/
But this fails, with the error:
E486: Pattern not found: ^M
However, this command does work:
:%s/\n/:/
producing:
foo:bar:biz:
(I can get rid of the trailing colon myself)
So Question 2: Why does \n work in this case, where ^M does not?
And ultimately, Question 3: Why is there this asymmetry between \n and ^M depending on whether it's on the right- or left-hand side of a string replacement command?
When searching, \n is a "catch-all" atom that conveniently matches any kind of "end-of-line": CRLF, CR, and LF.
When replacing, \n is <Nul> and represented as ^#.
When replacing, \r is the legal "end-of-line" for the current fileformat.
In short, get used to this pattern and carry on:
:s/\n/\r
See :help NL-used-for-Nul and CR-used-for-NL.
Related
Specific question
Description
After recording the desired action to registrar o, I pasted the whole macro to my ~/.vimrc and assigned it as follows (directly pasting the mappings are not displayed properly)
Expected behavior
I would like to use this macro to get myself a new "comment line" that leads a new section of script, formatted such that the name of the section is centered. After populating the "section title", I would like to enter insert mode in a new line.
In the following screen-record, I have tested both #o and #p$ on the word "time". The second attempt with#p` worked as desired.
The problem (on Windows machine specifically)
As you see, the #o mapping gets me junk phrases which had been part of my definition for the macro. Does this have to do with the ^M operator? And, how can I fix the #o mapping, which uses * to populate the line?
The two mapping worked just fine on Linux system. (Don't know why, as I have recorded and pasted the macro-definition on Windows machine.) This also does not appear to be a problem on Mac with MacVim.
Generalized question
Is there a way to properly substitute the ^M operator (for <CR>, or "Enter"-key)?
Is there a way to properly substitute the ^[ operator (for <ESC>, or the "Escape"-key)?
Is there a systematic list of mappings from these weird representation of keystrokes, as recorded by the "recording" function through q.
Solution
Substitute the ^M marks in the macro-definition with \r. And, substitute ^[ to be \x1b, for the ESC key. The mappings are fixed as follows:
let #o = ":center\ri\r\x1bkV:s/ /\*/g\rJx50A\*\x1b80d|o"
let #p = ":center\ri\r\x1bkV:s/ /\"/g\rJx50A\"\x1b80d|o"
Complete list of key-codes/mappings? Approach 1: through hex code.
Thanks to Zbynek Vyskovsky, the picture is clear. For whatever key one may think of, Vim takes its ASCII value at the "face value". (The trick is to use a escape clause starting with \x, where x serves as the leader key/string/character connecting to the hex values.) Thus, the correspondence list (incomplete yet), goes as follows:
Enter --- \x0d --- \r
ESC --- \x1b --- \e
Solution native to Vim
By chance, :help expr-quote gives the following list of special characters. This shall serve as the definite answer to the original question in general form.
string *string* *String* *expr-string* *E114*
------
"string" string constant *expr-quote*
Note that double quotes are used.
A string constant accepts these special characters:
\... three-digit octal number (e.g., "\316")
\.. two-digit octal number (must be followed by non-digit)
\. one-digit octal number (must be followed by non-digit)
\x.. byte specified with two hex numbers (e.g., "\x1f")
\x. byte specified with one hex number (must be followed by non-hex char)
\X.. same as \x..
\X. same as \x.
\u.... character specified with up to 4 hex numbers, stored according to the
current value of 'encoding' (e.g., "\u02a4")
\U.... same as \u but allows up to 8 hex numbers.
\b backspace <BS>
\e escape <Esc>
\f formfeed <FF>
\n newline <NL>
\r return <CR>
\t tab <Tab>
\\ backslash
\" double quote
\<xxx> Special key named "xxx". e.g. "\<C-W>" for CTRL-W. This is for use
in mappings, the 0x80 byte is escaped.
To use the double quote character it must be escaped: "<M-\">".
Don't use <Char-xxxx> to get a utf-8 character, use \uxxxx as
mentioned above.
Note that "\xff" is stored as the byte 255, which may be invalid in some
encodings. Use "\u00ff" to store character 255 according to the current value
of 'encoding'.
Note that "\000" and "\x00" force the end of the string.
As you use assigning to register using vim expression language, it's definitely possible in platform independent way. The strings in vim expressions understand the standard escape sequences, therefore it's best to replace ^M with \r and Esc with \x1b:
let #o = ":center\riSomeInsertedString\x1b"
There is no list of of special characters to be translated as far as I know but you can simply take all control characters (ASCII below 32) and translate them to corresponding escape sequence "\xHexValue" where HexValue is the value of the character. Even \r (or ^M) can be translated to \x0d as its ASCII value is 13 (0x0d hex).
From question How to replace a character for a newline in Vim?. You have to use \r when replacing text for a newline, like this
:%s/%/\r/g
But when replacing end of lines and newlines for a character, you can do it like:
:%s/\n/%/g
What section of the manual documents these behaviors, and what's the reasoning behind them?
From http://vim.wikia.com/wiki/Search_and_replace :
When Searching
...
\n is newline, \r is CR (carriage return = Ctrl-M = ^M)
When Replacing
...
\r is newline, \n is a null byte (0x00).
From vim docs on patterns:
\r matches <CR>
\n matches an end-of-line -
When matching in a string instead of
buffer text a literal newline
character is matched.
Another aspect to this is that \0, which is traditionally NULL, is taken in
s//\0/ to mean "the whole matched pattern". (Which, by the way, is redundant with, and longer than, &).
So you can't use \0 to mean NULL, so you use \n
So you can't use \n to mean \n, so you use \r.
So you can't use \r to mean \r, but I don't know who would want to add that char on purpose.
—☈
:help NL-used-for-Nul
Technical detail:
<Nul> characters in the file are stored as <NL> in memory. In the display
they are shown as "^#". The translation is done when reading and writing
files. To match a <Nul> with a search pattern you can just enter CTRL-# or
"CTRL-V 000". This is probably just what you expect. Internally the
character is replaced with a <NL> in the search pattern. What is unusual is
that typing CTRL-V CTRL-J also inserts a <NL>, thus also searches for a <Nul>
in the file. {Vi cannot handle <Nul> characters in the file at all}
First of all, open :h :s to see the section "4.2 Substitute" of documentation on "Change". Here's what the command accepts:
:[range]s[ubstitute]/{pattern}/{string}/[flags] [count]
Notice the description about pattern and string
For the {pattern} see |pattern|.
{string} can be a literal string, or something
special; see |sub-replace-special|.
So now you know that the search pattern and replacement patterns follow different rules.
If you follow the link to |pattern|, it takes you to the section that explains the whole regexp patterns used in Vim.
Meanwhile, |sub-replace-special| takes you to the subsection of "4.2 Substitute", which contains the patterns for substitution, among which is \r for line break/split.
(The shortcut to this part of manual is :h :s%)
I'm using vim to make my programs in c/c++ and I would like to know how can I put "\n" (which represents a newline) in my code or use "%" using :%s.
For instance, sometimes I forget to put "%" in front of "d" or "f" in many lines or forget to put "\n" in some printf() calls.
printf("This is my d code.", x);
But the following command does not work, it puts a space in place of "\n"!
:%s/\<code.\>/code.\n/gc
or
:%s/\<d\>/%d/gc
How can I do what I want?
The :help s/\n has the answer:
\n insert a <NL> (<NUL> in the file)
(does NOT break the line) *s/\n*
You'll also find the solution there: to insert a literal backslash, escape it \\ by doubling; to split the line, a \r has to be used. Yes, this is inconsistent, and it works differently in similar tools like sed, but that's unfortunately how it is.
Notes
The \n doesn't insert a space, but the special <NL> character, which usually is shown as ^#.
The \<code.\> isn't right; to match a literal period, you have to escape it: \.. Else, it matches any character. Likewise, the . usually isn't a keyword character, so the \> boundary wouldn't match.
You don't need to repeat the match text in the replacement, you can use & for it. Also read up on capture groups (:help /\() and the submatch references :help s/\1. This is a better way:
:%s/\<code\./&\\n/gc
(I don't see a problem with the second substitution.)
You want to insert the two-character sequence \n, not a literal newline (the latter would create a syntax error).
A sample line to be changed is:
printf("This is my d code.", x);
One problem with your attempt:
:%s/\<code.\>/code.\n/gc
is that there is no word boundary between the . and the " following the word code. The other problem is that \ in the target is used to escape special characters (for example you can refer to a / character as \/), so the \ must itself be escaped.
This should do the job:
:%s/\<code\."/code.\\n"/gc
A more general solution might be:
:g/printf/s/"/\\n"/egc
which offers to replace " by \n" on each line that contains printf -- but that will miss any printf calls that span more than one line.
As for replacing the d by %d, the command you have in your question:
:%s/\<d\>/%d/gc
is correct.
I have the following characters being repeated at the end of every line:
^[[00m
How can I remove them from each line using the Vim editor?
When I give the command :%s/^[[00m//g, it doesn't work.
You could use :%s/.\{6}$// to literally delete 6 characters off the end of each line.
The : starts ex mode which lets you execute a command. % is a range that specifies that this command should operate on the whole file. The s stands for substitute and is followed by a pattern and replace string in the format s/pattern/replacement/. Our pattern in this case is .\{6}$ which means match any character (.) exactly 6 times (\{6}) followed by the end of the line ($) and replace it with our replacement string, which is nothing. Therefore, as I said above, this matches the last 6 characters of every line and replaces them with nothing.
I would use the global command.
Try this:
:g/$/norm $xxxxxx
or even:
:g/$/norm $5Xx
I think the key to this problem is to keep it generic and not specific to the characters you are trying to delete. That way the technique you learn will be applicable to many other situations.
Assuming this is an ANSI escape sequence, the ^[ stands for a single <Esc> character. You have to enter it by pressing Ctrl + V (or Ctrl + Q) on many Windows Vim installations), followed by Esc. Notice how this is then highlighted in a slightly different color, too.
It's easy enough to replace the last six characters of every line being agnostic to what those characters are, but it leaves considerable room for error so I wouldn't recommend it. Also, if ^[ is an escape character, you're really looking for five characters.
Escape code
Using ga on the character ^[ you can determine whether it's an escape code, in which case the status bar would display
<^[> 27, Hex 1b, Octal 033
Assuming it is, you can replace everything using
:%s/\%x1b\[00m$//gc
With \%x1b coming from the hex value above. Note also that you have to escape the bracket ([) because it's a reserved character in Vim regex. $ makes sure it occurs at the end of a line, and the /gc flags will make it global and confirm each replacement (you can press a to replace all).
Not escape code
It's a simple matter of escaping then. You can use either of the two below:
:%s/\^\[\[00m$//gc
:%s/\V^[[00m\$//gc
If they are all aligning, you can do a visual-block selection and delete it then.
Otherwise, if you have a sequence unknown how to input, you can visually select it by pressing v, then mark and yank it y (per default into register "). Then you type :%s/<C-R>"//g to delete it.
Note:
<C-R>" puts the content of register " at the cursor position.
If you yanked it into another register, say "ay (yank to register a - the piglatin yank, as I call it) and forgot where you put it, you can look at the contents of your registers with :reg.
<C-R> is Vim speak for Ctrl+R
This seems to work fine when the line is more than 5 chars long:
:perldo $_ = substr $_, 0, -5
but when the line is 5 or less chars long it does nothing.
Maybe there is a easy way in perl to delete the last 5 chars of a string, but I don't really know it:)
Use this to delete:
:%s/^[[00m//gc
From question How to replace a character for a newline in Vim?. You have to use \r when replacing text for a newline, like this
:%s/%/\r/g
But when replacing end of lines and newlines for a character, you can do it like:
:%s/\n/%/g
What section of the manual documents these behaviors, and what's the reasoning behind them?
From http://vim.wikia.com/wiki/Search_and_replace :
When Searching
...
\n is newline, \r is CR (carriage return = Ctrl-M = ^M)
When Replacing
...
\r is newline, \n is a null byte (0x00).
From vim docs on patterns:
\r matches <CR>
\n matches an end-of-line -
When matching in a string instead of
buffer text a literal newline
character is matched.
Another aspect to this is that \0, which is traditionally NULL, is taken in
s//\0/ to mean "the whole matched pattern". (Which, by the way, is redundant with, and longer than, &).
So you can't use \0 to mean NULL, so you use \n
So you can't use \n to mean \n, so you use \r.
So you can't use \r to mean \r, but I don't know who would want to add that char on purpose.
—☈
:help NL-used-for-Nul
Technical detail:
<Nul> characters in the file are stored as <NL> in memory. In the display
they are shown as "^#". The translation is done when reading and writing
files. To match a <Nul> with a search pattern you can just enter CTRL-# or
"CTRL-V 000". This is probably just what you expect. Internally the
character is replaced with a <NL> in the search pattern. What is unusual is
that typing CTRL-V CTRL-J also inserts a <NL>, thus also searches for a <Nul>
in the file. {Vi cannot handle <Nul> characters in the file at all}
First of all, open :h :s to see the section "4.2 Substitute" of documentation on "Change". Here's what the command accepts:
:[range]s[ubstitute]/{pattern}/{string}/[flags] [count]
Notice the description about pattern and string
For the {pattern} see |pattern|.
{string} can be a literal string, or something
special; see |sub-replace-special|.
So now you know that the search pattern and replacement patterns follow different rules.
If you follow the link to |pattern|, it takes you to the section that explains the whole regexp patterns used in Vim.
Meanwhile, |sub-replace-special| takes you to the subsection of "4.2 Substitute", which contains the patterns for substitution, among which is \r for line break/split.
(The shortcut to this part of manual is :h :s%)