I'm trying to remove all CLRF characters from a projects git repo. I'm writing a command to grep through the repo recursively to find instances. However, some of the 'hits' when opened in vim show very explicitly that there is ^M and yet others do not display these characters.
However, when running
file <filename without visual ^M>
It says
blah.java ASCII Java program text, with CRLF line terminators
and
od -cx <filename without visual ^M>
returns with \r\n peppered throughout.
I'm just interested why vim sometimes shows them and sometimes not.
EDIT:
I created a test text file and manually added ^M (ie ctrl V + ctrl M) and vim displayed those characters. Then I ran:
sed -i '' -e 's/\r//g' controlm.txt
And opened the file with vim and the visual ^M were gone, but od -cx still showed \r \n, however I then ran
sed -i '' -e 's/^M//g' controlm.txt
Then it removed not only the visual ^M in vim as well as I've confirmed that od -cx displays that \r \n are now just \n.
This question would probably better be asked on Superuser.com, not here, because it's about using vim, not programming. But to answer it:
When opening a file, vim tries to detect if it's a MS-DOS/Windows or a unix file. If all lines are terminated by \r\n, it's probably a DOS file, if only some of them are, vim may assume unix as well. If the file format is set to DOS, vim ignores \r when reading the file, and shows [dos] in the status line directly after reading the file.
When writing back the file, it terminates each line with \r\n; if the file format is unix, it terminates lines with \n. You can set the mode with the command
:se fileformat=unix
or
:se fileformat=dos
Try creating a file x.txt in Windows, open it in vim. Then, :se fileformat=unix and :w y.txt; then :se fileformat=dos and :w z.txt. Test y.txt and z.txt with od cx. y.txt will have \r\n line endings, z.txt won't.
When only some, but not all, lines in the file end in \r, for example if (unix) git added some headers (without \r) to a file that was created on dos/windows, the file format detection sees the headers first, assumes unix, does not remove the \r from the rest of the file when reading, and shows those as ^M.
#Guntram Blohm is right, but he left out part of the answer: the 'fileformats' option (short form 'ffs'). If you
:set ffs=unix
:e dosfile.txt
then vim will stubbornly refuse to accept the CRLF line endings and will show all the ^M characters explicitly.
:help 'ffs'
This question already has answers here:
Indenting in VIM with all the files in Folder
(4 answers)
Closed 8 years ago.
I want to indent all my files using the command gg=G of VIM. Is there anyway to write a script to do that ?
I imagine it can be something like
find . | xargs -n 1 | vim [ with some option to indent ]
I am quite sure vim -c may help, but not sure what is gg=G equivalent..
vim has two options you could take a look: (from man vim)
-s {scriptin}
The script file {scriptin} is read. The characters in the file are interpreted as if you had typed them. The same can be done with the command
":source! {scriptin}". If the end of the file is reached before the editor exits, further characters are read from the keyboard.
and
-w {scriptout}
All the characters that you type are recorded in the file {scriptout}, until you exit Vim. This is useful if you want to create a script file to be
used with "vim -s" or ":source!". If the {scriptout} file exists, characters are appended.
that means, you could record your key sequence by vim -w script for example gg=GZZ then you could vim -s script file
I think vimgolf uses this mechanism too.
You can use bufdo to apply a command to all buffers, and normal gg=G to run the normal mode command gg=G. And wqall to save them all.
You can do the following all within vim.
:args `find .`
:argdo normal gg=G
:argdo w
This builds up the argslist and then runs the normal command gg=G on each file in the argslist. Then we save each file in the argslist. Note: this requires set hidden.
Drew Neil over at Vimcast has some nice screencasts about this subject:
Meet the arglist
Populating the arglist
Using :argdo to change multiple files
Project-wide find and replace
For more help see:
:h argslist
:h 'hidden'
Is there a text editor on Linux that allows me to see line breaks and carriage returns? Does Vim support this feature?
To disagree with the official answer:
:set list will not show ^M characters (CRs). Supplying the -b option to vi/Vim will work. Or, once Vim is loaded, type :e ++ff=unix.
Assuming your vim settings for :set listchars=... is set to visualize the characters you are attempting to see, in this case the carriage return characters (typed with CTL + V, CTRM + M) —— otherwise, as reported in many of the comments on this answer, the ^M character will not show on :set list
:set list in Vim will show whitespace. End of lines show as '$' and carriage returns usually show as '^M'.
vi shows newlines (LF character, code x0A) by showing the subsequent text on the next line.
Use the -b switch for binary mode. For example , vi -b filename or vim -b filename --.
It will then show CR characters (x0D), which are not normally used in Unix style files, as the characters ^M.
Just to clarify why :set list won't show CR's as ^M without e ++ff=unix and why :set list has nothing to do with ^M's.
Internally when Vim reads a file into its buffer, it replaces all line-ending characters with its own representation (let's call it $'s). To determine what characters should be removed, it firstly detects in what format line endings are stored in a file. If there are only CRLF '\r\n' or only CR '\r' or only LF '\n' line-ending characters, then the 'fileformat' is set to dos, mac and unix respectively.
When list option is set, Vim displays $ character when the line break occurred no matter what fileformat option has been detected. It uses its own internal representation of line-breaks and that's what it displays.
Now when you write buffer to the disc, Vim inserts line-ending characters according to what fileformat options has been detected, essentially converting all those internal $'s with appropriate characters. If the fileformat happened to be unix then it will simply write \n in place of its internal line-break.
The trick is to force Vim to read a dos encoded file as unix one. The net effect is that it will remove all \n's leaving \r's untouched and display them as ^M's in your buffer. Setting :set list will additionally show internal line-endings as $. After all, you see ^M$ in place of dos encoded line-breaks.
Also notice that :set list has nothing to do with showing ^M's. You can check it by yourself (make sure you have disabled list option first) by inserting single CR using CTRL-V followed by Enter in insert mode. After writing buffer to disc and opening it again you will see ^M despite list option being set to 0.
You can find more about file formats on http://vim.wikia.com/wiki/File_format or by typing:help 'fileformat' in Vim.
Try the following command.
:set binary
In Vim, this should do the same thing as using the "-b" command line option. If you put this in your startup (i.e., .vimrc) file, it will always be in place for you.
On many *nix systems, there is a "dos2unix" or "unix2dos" command that can process the file and correct any suspected line ending issues. If there aren't any problems with the line endings, the files will not be changed.
I suggest you to edit your .vimrc file, for running a list of commands.
Edit your .vimrc file, like this:
cat >> ~/.vimrc <<EOF
set ffs=unix
set encoding=utf-8
set fileencoding=utf-8
set listchars=eol:¶
set list
EOF
When you're executing Vim, the commands in file .vimrc are executed, and you can see this example:
My line with CRLF eol here ^M¶
By using cat and -A you can see new lines as $ and tabs as ^I:
cat -A myfile
You can view break lines using the gedit editor.
First, if you don't have it installed, for Debian/Ubuntu/Mint based distros:
sudo apt-get install gedit
For Fedora/CentOS/RedHat based distros:
sudo dnf install gedit
or
sudo yum install gedit
Now, install gedit plugins:
sudo apt-get install gedit-plugins
or
Under Gnome2, user plugins were put into ~/.gnome2/gedit/plugins/
For Gnome3: ~/.local/share/gedit/plugins/
Download the plugins from: https://help.gnome.org/users/gedit/stable/gedit-plugin-guide.html.en#gedit-additional-plugins
and select Draw Spaces plugin, enter on Preferences, and chose Draw new lines:
Using Visual Studio Code, you can install the Line endings extension.
Sublime Text 3 has a plugin called RawLineEdit that will display line endings and allow the insertion of arbitrary line-ending type
Shift + Ctrl + P and start type the name of the plugin, and toggle to show line endings.
Add the following alias to your .bashrc or .bash_aliases:
alias vidos='vi -c ":e ++ff=unix" -c "set list"'
Then you can use vidos to edit the file and see newline as $ and carriage return as ^M.
Vim shows ^M on every line ending.
How do I replace this with a normal line break in a file opened in Vim?
Command
:%s/<Ctrl-V><Ctrl-M>/\r/g
Where <Ctrl-V><Ctrl-M> means type Ctrl+V then Ctrl+M.
Explanation
:%s
substitute, % = all lines
<Ctrl-V><Ctrl-M>
^M characters (the Ctrl-V is a Vim way of writing the Ctrl ^ character and Ctrl-M writes the M after the regular expression, resulting to ^M special character)
/\r/
with new line (\r)
g
And do it globally (not just the first occurrence on the line).
On Linux and Mac OS, the following works,
:%s/^V^M/^V^M/g
where ^V^M means type Ctrl+V, then Ctrl+M.
Note: on Windows you probably want to use ^Q instead of ^V, since by default ^V is mapped to paste text.
Within vim, look at the file format — DOS or Unix:
:set filetype=unix
:set fileformat=unix
The file will be written back without carriage return (CR, ^M) characters.
This is the only thing that worked for me:
:e ++ff=dos
Found it at: http://vim.wikia.com/wiki/File_format
A file I had created with BBEdit seen in MacVim was displaying a bunch of ^M line returns instead of regular ones. The following string replace solved the issue:
:%s/\r/\r/g
It's interesting because I'm replacing line breaks with the same character, but I suppose Vim just needs to get a fresh \r to display correctly. I'd be interested to know the underlying mechanics of why this works.
First, use :set ff? to figure out the file format your file is.
I guess it could be unix, then the problem is your file was created with fileformat=dos adding "^M^J" to the line end but read with flieformat=unix only removing the "^J" from the line end, leaving the "^M" there.
Just input :e ++ff=dos in Vim command line to change your file's format from unix to dos. It should solve the problem. If not, :%s/\r//g should help you out.
in order to get the ^M character to match I had to visually select it and then use the OS copy to clipboard command to retrieve it. You can test it by doing a search for the character before trying the replace command.
/^M
should select the first bad line
:%s/^M/\r/g
will replace all the errant ^M with carriage returns.
This is as functions in MacVim, which is based on gvim 7.
EDIT:
Having this problem again on my Windows 10 machine, which has Ubuntu for Windows, and I think this is causing fileformat issues for vim. In this case changing the ff to unix, mac, or dos did nothing other than to change the ^M to ^J and back again.
The solution in this case:
:%s/\r$/ /g
:%s/ $//g
The reason I went this route is because I wanted to ensure I was being non-destructive with my file. I could have :%s/\r$//g but that would have deleted the carriage returns right out, and could have had unexpected results. Instead we convert the singular CR character, here a ^M character, into a space, and then remove all spaces at the end of lines (which for me is a desirable result regardless)
Sorry for reviving an old question that has long since been answered, but there seemed to be some confusion afoot and I thought I'd help clear some of that up since this is coming up high in google searches.
None of these worked for me, so I tried this, which worked:
type :%s/
press CTRL-VCTRL-M
type //g
press Enter
So the overall command in Vim shoud look like :%s/^M//g
What this does: :%s (find and replace) /^M/ (that symbol) / (with no chars) g (globally).
^M is retrieved by Ctrl+V and M, so do
s/^M//g
Without needing to use Ctrl:
:%s/\r$//
Simple thing that worked for me
dos2unix filename
I did this with sed:
sed -i -e 's/\r/\n/g' filename
What about just:
:%s/\r//g
That totally worked for me.
What this does is just to clean the end of line of all lines, it removes the ^M and that's it.
There are many other answers to this question, but still, the following works best for me, as I needed a command line solution:
vim -u NONE -c 'e ++ff=dos' -c 'w ++ff=unix' -c q myfile
Explanation:
Without loading any .vimrc files, open myfile
Run :e ++ff=dos to force a reload of the entire file as dos line endings.
Run :w ++ff=unix to write the file using unix line endings
Quit vim
Ctrl+M minimizes my window, but Ctrl+Enter actually inserts a ^M character. I also had to be sure not to lift off the Ctrl key between presses.
So the solution for me was:
:%s/<Ctrl-V><Ctrl-Enter>/\r/g
Where <Ctrl-V><Ctrl-Enter> means to press and hold Ctrl, press and release V, press and release Enter, and then release Ctrl.
If you are working on a Windows-generated file
The above solution will add an additional line between existing lines, because there is already an invisible \r after the ^M.
To prevent this, you want to delete the ^M characters without replacing them.
:%s/<Ctrl-V><Ctrl-Enter>//g
Where % means "in this buffer," s means "substitute," / means "(find) the following pattern," <Ctrl-V><Ctrl-Enter> refers to the keys to press to get the ^M character (see above), // means "with nothing" (or, "with the pattern between these two slashes, which is empty"), and g is a flag meaning "globally," as opposed to the first occurrence in a line.
This worked for me:
Set file format to unix (\n line ending)
save the file
So in vim:
:set ff=unix
:w
In my case,
Nothing above worked, I had a CSV file copied to Linux machine from my mac and I used all the above commands but nothing helped but the below one
tr "\015" "\n" < inputfile > outputfile
I had a file in which ^M characters were sandwitched between lines something like below
Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKT6TG,TRO_WBFB_500,Trico,CARS,Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKTG0A,TRO_WB_T500,Trico,
Alternatively, there are open-source utilities called dos2unix and unix2dos available that do this very thing. On a linux system they are probably installed by default; for a windows system you can download them from http://www.bastet.com/ amongst others.
sed s/^M//g file1.txt > file2.txt
where ^M is typed by simultaneously pressing the 3 keys, ctrl + v + m
use dos2unix utility if the file was created on windows,
use mac2unix utility if the file was created on mac. :)
Use one of these commands:
:%s/\r//g
Or
:%s/\r\(\n\)/\1/g
In command mode in VIM:
:e ++ff=dos | setl ff=unix | up
e ++ff=dos - force open file in dos format.
setl ff=unix - convert file to unix format.
up - save file only when has been modified.
To save keystrokes, you can avoid typing Ctrl+VCtrl+M by placing this in a mapping. Just open a file containing a ^M character, yank it, and paste it into a line like this in your .vimrc:
nnoremap <Leader>d :%s/^M//g<CR>
This worked for me:
:% s/\r\n/\r
To use sed on MacOS, do this:
sed -i.bak $'s/\r//' <filename>
Explanation: The $'STRING' syntax here pertains to the bash shell. Macs don't treat \r as special character. By quoting the command string in $'' you're telling the shell to replace \r with the actual \r character specified in the ANSI-C standard.
None of these suggestions were working for me having managed to get a load of ^M line breaks while working with both vim and eclipse. I suspect that I encountered an outside case but in case it helps anyone I did.
:%s/.$//g
And it sorted out my problem
:g/^M/s// /g
If you type ^M using Shift+6 Caps+M it won't accept.
You need to type ctrl+v ctrl+m.
^M gives unwanted line breaks. To handle this we can use the sed command as follows:
sed 's/\r//g'
Just removeset binary in your .vimrc!
On Solaris:
:%s/<CTRL+V><CTRL+M>//g
that is:
:%s/^M//g
That means:
% = all lines,
s = substitute,
^M = what you desire to substitute
// = replace with nothing
g = globally (not only the first occurrance)