Is there a way to make \n or \r characters visible in vim? (I want to be able to see if the csv file I am looking at uses just \n or \r and \n at the end of lines.
:setl fileformat?
dos means CR-LF (consistently; if only some lines contain both, it'll be listed as unix, and you'll see ^M characters at the end of those lines), unix means LF; mac means only CR newlines.
Related
I would like to display ^M (\r) as newlines (\n) in vim. The file has both \r and \n characters. I would like to do this without:
Editing the file
Changing the way existing \n characters are displayed
Having to do a search and replace every time I open the file
In short, the same way Notepad++ displays them.
There seems to be some sort of asymmetry in the way Vim treats ^M when doing string replacement (:s/x/y/).
Perhaps an example is best; say we have this text file:
foo:bar:biz
I want to split this into lines. This works fine:
:s/:/^M/g
(note that ^M is produced by typing Ctrl-V, Enter)
This results in the text file:
foo
bar
baz
Now, if I undo that and try again, I notice that this does not work:
:s/:/\n/g
Here, the resulting text is:
foo^#bar^#biz
That is to say, they are joined by the ASCII NUL byte (0x00).
Question 1: Why does using \n in the replacement result in NUL bytes?
Now, I figure "okay, I guess ^M is used as the 'line separator' character in some way, for Vim; I can work with that".
So I do another experiment, starting with the one-item-per-line text file:
foo
bar
baz
and now, I want to join them with colons, so it looks like the very first incarnation, above.
So I run:
:%s/^M/:/
But this fails, with the error:
E486: Pattern not found: ^M
However, this command does work:
:%s/\n/:/
producing:
foo:bar:biz:
(I can get rid of the trailing colon myself)
So Question 2: Why does \n work in this case, where ^M does not?
And ultimately, Question 3: Why is there this asymmetry between \n and ^M depending on whether it's on the right- or left-hand side of a string replacement command?
When searching, \n is a "catch-all" atom that conveniently matches any kind of "end-of-line": CRLF, CR, and LF.
When replacing, \n is <Nul> and represented as ^#.
When replacing, \r is the legal "end-of-line" for the current fileformat.
In short, get used to this pattern and carry on:
:s/\n/\r
See :help NL-used-for-Nul and CR-used-for-NL.
From question How to replace a character for a newline in Vim?. You have to use \r when replacing text for a newline, like this
:%s/%/\r/g
But when replacing end of lines and newlines for a character, you can do it like:
:%s/\n/%/g
What section of the manual documents these behaviors, and what's the reasoning behind them?
From http://vim.wikia.com/wiki/Search_and_replace :
When Searching
...
\n is newline, \r is CR (carriage return = Ctrl-M = ^M)
When Replacing
...
\r is newline, \n is a null byte (0x00).
From vim docs on patterns:
\r matches <CR>
\n matches an end-of-line -
When matching in a string instead of
buffer text a literal newline
character is matched.
Another aspect to this is that \0, which is traditionally NULL, is taken in
s//\0/ to mean "the whole matched pattern". (Which, by the way, is redundant with, and longer than, &).
So you can't use \0 to mean NULL, so you use \n
So you can't use \n to mean \n, so you use \r.
So you can't use \r to mean \r, but I don't know who would want to add that char on purpose.
—☈
:help NL-used-for-Nul
Technical detail:
<Nul> characters in the file are stored as <NL> in memory. In the display
they are shown as "^#". The translation is done when reading and writing
files. To match a <Nul> with a search pattern you can just enter CTRL-# or
"CTRL-V 000". This is probably just what you expect. Internally the
character is replaced with a <NL> in the search pattern. What is unusual is
that typing CTRL-V CTRL-J also inserts a <NL>, thus also searches for a <Nul>
in the file. {Vi cannot handle <Nul> characters in the file at all}
First of all, open :h :s to see the section "4.2 Substitute" of documentation on "Change". Here's what the command accepts:
:[range]s[ubstitute]/{pattern}/{string}/[flags] [count]
Notice the description about pattern and string
For the {pattern} see |pattern|.
{string} can be a literal string, or something
special; see |sub-replace-special|.
So now you know that the search pattern and replacement patterns follow different rules.
If you follow the link to |pattern|, it takes you to the section that explains the whole regexp patterns used in Vim.
Meanwhile, |sub-replace-special| takes you to the subsection of "4.2 Substitute", which contains the patterns for substitution, among which is \r for line break/split.
(The shortcut to this part of manual is :h :s%)
I'm on Windows and I have an odd text file containing mostly CR+LF line ending. A few lines end with only CR. Which tool to use to transform these odd lines into well formatted (e.g. CR+LF terminated) lines?
I could use either GnuWin32 tools or Python to solve this.
The main problem I have is that I cannot open the file as text file since Python (as most other text processors, such as awk) don't recognize the mixed line endings. So I believe the solution must incorporate binary processing of the file.
The again, I cannot just replace CR by CR LF, since there are also CR LF line endings existing that must not be touched.
To replace lines you can use regular expressions:
\r+ to find CR
\r\n is the text you want as replacement text.
Regular Expressions in Python:
Regular Expression
import re
txt='text where you want to replace the linebreak'
out = re.sub("\r+", '\r\n', txt)
print out
From question How to replace a character for a newline in Vim?. You have to use \r when replacing text for a newline, like this
:%s/%/\r/g
But when replacing end of lines and newlines for a character, you can do it like:
:%s/\n/%/g
What section of the manual documents these behaviors, and what's the reasoning behind them?
From http://vim.wikia.com/wiki/Search_and_replace :
When Searching
...
\n is newline, \r is CR (carriage return = Ctrl-M = ^M)
When Replacing
...
\r is newline, \n is a null byte (0x00).
From vim docs on patterns:
\r matches <CR>
\n matches an end-of-line -
When matching in a string instead of
buffer text a literal newline
character is matched.
Another aspect to this is that \0, which is traditionally NULL, is taken in
s//\0/ to mean "the whole matched pattern". (Which, by the way, is redundant with, and longer than, &).
So you can't use \0 to mean NULL, so you use \n
So you can't use \n to mean \n, so you use \r.
So you can't use \r to mean \r, but I don't know who would want to add that char on purpose.
—☈
:help NL-used-for-Nul
Technical detail:
<Nul> characters in the file are stored as <NL> in memory. In the display
they are shown as "^#". The translation is done when reading and writing
files. To match a <Nul> with a search pattern you can just enter CTRL-# or
"CTRL-V 000". This is probably just what you expect. Internally the
character is replaced with a <NL> in the search pattern. What is unusual is
that typing CTRL-V CTRL-J also inserts a <NL>, thus also searches for a <Nul>
in the file. {Vi cannot handle <Nul> characters in the file at all}
First of all, open :h :s to see the section "4.2 Substitute" of documentation on "Change". Here's what the command accepts:
:[range]s[ubstitute]/{pattern}/{string}/[flags] [count]
Notice the description about pattern and string
For the {pattern} see |pattern|.
{string} can be a literal string, or something
special; see |sub-replace-special|.
So now you know that the search pattern and replacement patterns follow different rules.
If you follow the link to |pattern|, it takes you to the section that explains the whole regexp patterns used in Vim.
Meanwhile, |sub-replace-special| takes you to the subsection of "4.2 Substitute", which contains the patterns for substitution, among which is \r for line break/split.
(The shortcut to this part of manual is :h :s%)