I used to know a vim command sequence where I would remove all the non-printable extraneous characters in a file, like ^M on Windows files that ended up with col -xb, but I can't recall the keyboard sequence before I enter the col -xb command to say "do this from top-to-bottom of file." Does this ring a bell to anybody?
:%!col -xb
: enters the ex-command mode as usual,% selects all lines (you can also write other ranges, like 100,150 for lines 100 to 150) and ! pipes them through an external command.
run the following command in vim:
:set fileformat=unix
it worked for me, and also check out: Are shell scripts sensitive to encoding and line endings?
I am trying to reproduce the example in chapter 15 of Steve Losh's "Learn Vimscript the Hard Way" (http://learnvimscriptthehardway.stevelosh.com/chapters/15.html), in the section called "Changing the Start."
First, I start vim with the following command:
vim -u NONE foo
to ensure that no plugins are loaded.
Next I type
print foo(bar)
in the buffer and then enter
:onoremap in( :<c-u>normal! f(vi(<cr>
Finally, I place the cursor over the "p" in "print" and enter cin(. The status line displays:
:<c-u>normal! f(vi(<cr>
with a blinking cursor at the end. I hit enter and the status line then displays:
E488: Trailing characters
The expected outcome, according to the book, is that vim will delete the contents of the parentheses and place you in insert mode between them.
Here is the version info:
VIM - Vi IMproved 8.0 (2016 Sep 12, compiled May 2 2017 03:55:34)
MacOS X (unix) version
Included patches: 1-596
Compiled by Homebrew
TL;DR: use vim -u DEFAULTS foo instead.
That should keep Vim in 'nocompatible' mode, which is needed for <Key> codes to be recognized in mappings.
The symptoms you describe are consistent to having < included in 'cpoptions', which is documented to:
Disable the recognition of special key codes in <> form in mappings, abbreviations, and the "to" part of menu commands. For example, the command :map X <Tab> results in X being mapped to "<Tab>" (5 characters) when < is included; but "^I" (^I is a real <Tab>) when < is excluded.
That is consistent with you getting the literal result of your mapping in your command-line, without proper expansion of the <c-u> and <cr> key codes.
The defaults for 'cpoptions' are:
Vim default: "aABceFs",
Vi default: all flags
So this means < (along with all other flags) will be included by default in Vi mode, which is triggered when 'compatible' is set (or, perhaps more accurately, when 'nocompatible' is not set.)
It turns out that when you use -u NONE in your command-line starting Vim, it will start in 'compatible' mode.
You can use instead -u DEFAULTS, which is quite similar to -u NONE, but it will still load the defaults.vim script shipped with Vim runtime, which will among other things set 'nocompatible'.
From the docs:
When -u DEFAULTS (all uppercase) is passed, this has the same effect as -u NONE, but the defaults.vim script is loaded, which will also set 'nocompatible'.
Using the -u argument with another argument than DEFAULTS has the side effect that the 'compatible' option will be on by default. This can have unexpected effects.
Is there a text editor on Linux that allows me to see line breaks and carriage returns? Does Vim support this feature?
To disagree with the official answer:
:set list will not show ^M characters (CRs). Supplying the -b option to vi/Vim will work. Or, once Vim is loaded, type :e ++ff=unix.
Assuming your vim settings for :set listchars=... is set to visualize the characters you are attempting to see, in this case the carriage return characters (typed with CTL + V, CTRM + M) —— otherwise, as reported in many of the comments on this answer, the ^M character will not show on :set list
:set list in Vim will show whitespace. End of lines show as '$' and carriage returns usually show as '^M'.
vi shows newlines (LF character, code x0A) by showing the subsequent text on the next line.
Use the -b switch for binary mode. For example , vi -b filename or vim -b filename --.
It will then show CR characters (x0D), which are not normally used in Unix style files, as the characters ^M.
Just to clarify why :set list won't show CR's as ^M without e ++ff=unix and why :set list has nothing to do with ^M's.
Internally when Vim reads a file into its buffer, it replaces all line-ending characters with its own representation (let's call it $'s). To determine what characters should be removed, it firstly detects in what format line endings are stored in a file. If there are only CRLF '\r\n' or only CR '\r' or only LF '\n' line-ending characters, then the 'fileformat' is set to dos, mac and unix respectively.
When list option is set, Vim displays $ character when the line break occurred no matter what fileformat option has been detected. It uses its own internal representation of line-breaks and that's what it displays.
Now when you write buffer to the disc, Vim inserts line-ending characters according to what fileformat options has been detected, essentially converting all those internal $'s with appropriate characters. If the fileformat happened to be unix then it will simply write \n in place of its internal line-break.
The trick is to force Vim to read a dos encoded file as unix one. The net effect is that it will remove all \n's leaving \r's untouched and display them as ^M's in your buffer. Setting :set list will additionally show internal line-endings as $. After all, you see ^M$ in place of dos encoded line-breaks.
Also notice that :set list has nothing to do with showing ^M's. You can check it by yourself (make sure you have disabled list option first) by inserting single CR using CTRL-V followed by Enter in insert mode. After writing buffer to disc and opening it again you will see ^M despite list option being set to 0.
You can find more about file formats on http://vim.wikia.com/wiki/File_format or by typing:help 'fileformat' in Vim.
Try the following command.
:set binary
In Vim, this should do the same thing as using the "-b" command line option. If you put this in your startup (i.e., .vimrc) file, it will always be in place for you.
On many *nix systems, there is a "dos2unix" or "unix2dos" command that can process the file and correct any suspected line ending issues. If there aren't any problems with the line endings, the files will not be changed.
I suggest you to edit your .vimrc file, for running a list of commands.
Edit your .vimrc file, like this:
cat >> ~/.vimrc <<EOF
set ffs=unix
set encoding=utf-8
set fileencoding=utf-8
set listchars=eol:¶
set list
EOF
When you're executing Vim, the commands in file .vimrc are executed, and you can see this example:
My line with CRLF eol here ^M¶
By using cat and -A you can see new lines as $ and tabs as ^I:
cat -A myfile
You can view break lines using the gedit editor.
First, if you don't have it installed, for Debian/Ubuntu/Mint based distros:
sudo apt-get install gedit
For Fedora/CentOS/RedHat based distros:
sudo dnf install gedit
or
sudo yum install gedit
Now, install gedit plugins:
sudo apt-get install gedit-plugins
or
Under Gnome2, user plugins were put into ~/.gnome2/gedit/plugins/
For Gnome3: ~/.local/share/gedit/plugins/
Download the plugins from: https://help.gnome.org/users/gedit/stable/gedit-plugin-guide.html.en#gedit-additional-plugins
and select Draw Spaces plugin, enter on Preferences, and chose Draw new lines:
Using Visual Studio Code, you can install the Line endings extension.
Sublime Text 3 has a plugin called RawLineEdit that will display line endings and allow the insertion of arbitrary line-ending type
Shift + Ctrl + P and start type the name of the plugin, and toggle to show line endings.
Add the following alias to your .bashrc or .bash_aliases:
alias vidos='vi -c ":e ++ff=unix" -c "set list"'
Then you can use vidos to edit the file and see newline as $ and carriage return as ^M.
Vim shows ^M on every line ending.
How do I replace this with a normal line break in a file opened in Vim?
Command
:%s/<Ctrl-V><Ctrl-M>/\r/g
Where <Ctrl-V><Ctrl-M> means type Ctrl+V then Ctrl+M.
Explanation
:%s
substitute, % = all lines
<Ctrl-V><Ctrl-M>
^M characters (the Ctrl-V is a Vim way of writing the Ctrl ^ character and Ctrl-M writes the M after the regular expression, resulting to ^M special character)
/\r/
with new line (\r)
g
And do it globally (not just the first occurrence on the line).
On Linux and Mac OS, the following works,
:%s/^V^M/^V^M/g
where ^V^M means type Ctrl+V, then Ctrl+M.
Note: on Windows you probably want to use ^Q instead of ^V, since by default ^V is mapped to paste text.
Within vim, look at the file format — DOS or Unix:
:set filetype=unix
:set fileformat=unix
The file will be written back without carriage return (CR, ^M) characters.
This is the only thing that worked for me:
:e ++ff=dos
Found it at: http://vim.wikia.com/wiki/File_format
A file I had created with BBEdit seen in MacVim was displaying a bunch of ^M line returns instead of regular ones. The following string replace solved the issue:
:%s/\r/\r/g
It's interesting because I'm replacing line breaks with the same character, but I suppose Vim just needs to get a fresh \r to display correctly. I'd be interested to know the underlying mechanics of why this works.
First, use :set ff? to figure out the file format your file is.
I guess it could be unix, then the problem is your file was created with fileformat=dos adding "^M^J" to the line end but read with flieformat=unix only removing the "^J" from the line end, leaving the "^M" there.
Just input :e ++ff=dos in Vim command line to change your file's format from unix to dos. It should solve the problem. If not, :%s/\r//g should help you out.
in order to get the ^M character to match I had to visually select it and then use the OS copy to clipboard command to retrieve it. You can test it by doing a search for the character before trying the replace command.
/^M
should select the first bad line
:%s/^M/\r/g
will replace all the errant ^M with carriage returns.
This is as functions in MacVim, which is based on gvim 7.
EDIT:
Having this problem again on my Windows 10 machine, which has Ubuntu for Windows, and I think this is causing fileformat issues for vim. In this case changing the ff to unix, mac, or dos did nothing other than to change the ^M to ^J and back again.
The solution in this case:
:%s/\r$/ /g
:%s/ $//g
The reason I went this route is because I wanted to ensure I was being non-destructive with my file. I could have :%s/\r$//g but that would have deleted the carriage returns right out, and could have had unexpected results. Instead we convert the singular CR character, here a ^M character, into a space, and then remove all spaces at the end of lines (which for me is a desirable result regardless)
Sorry for reviving an old question that has long since been answered, but there seemed to be some confusion afoot and I thought I'd help clear some of that up since this is coming up high in google searches.
None of these worked for me, so I tried this, which worked:
type :%s/
press CTRL-VCTRL-M
type //g
press Enter
So the overall command in Vim shoud look like :%s/^M//g
What this does: :%s (find and replace) /^M/ (that symbol) / (with no chars) g (globally).
^M is retrieved by Ctrl+V and M, so do
s/^M//g
Without needing to use Ctrl:
:%s/\r$//
Simple thing that worked for me
dos2unix filename
I did this with sed:
sed -i -e 's/\r/\n/g' filename
What about just:
:%s/\r//g
That totally worked for me.
What this does is just to clean the end of line of all lines, it removes the ^M and that's it.
There are many other answers to this question, but still, the following works best for me, as I needed a command line solution:
vim -u NONE -c 'e ++ff=dos' -c 'w ++ff=unix' -c q myfile
Explanation:
Without loading any .vimrc files, open myfile
Run :e ++ff=dos to force a reload of the entire file as dos line endings.
Run :w ++ff=unix to write the file using unix line endings
Quit vim
Ctrl+M minimizes my window, but Ctrl+Enter actually inserts a ^M character. I also had to be sure not to lift off the Ctrl key between presses.
So the solution for me was:
:%s/<Ctrl-V><Ctrl-Enter>/\r/g
Where <Ctrl-V><Ctrl-Enter> means to press and hold Ctrl, press and release V, press and release Enter, and then release Ctrl.
If you are working on a Windows-generated file
The above solution will add an additional line between existing lines, because there is already an invisible \r after the ^M.
To prevent this, you want to delete the ^M characters without replacing them.
:%s/<Ctrl-V><Ctrl-Enter>//g
Where % means "in this buffer," s means "substitute," / means "(find) the following pattern," <Ctrl-V><Ctrl-Enter> refers to the keys to press to get the ^M character (see above), // means "with nothing" (or, "with the pattern between these two slashes, which is empty"), and g is a flag meaning "globally," as opposed to the first occurrence in a line.
This worked for me:
Set file format to unix (\n line ending)
save the file
So in vim:
:set ff=unix
:w
In my case,
Nothing above worked, I had a CSV file copied to Linux machine from my mac and I used all the above commands but nothing helped but the below one
tr "\015" "\n" < inputfile > outputfile
I had a file in which ^M characters were sandwitched between lines something like below
Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKT6TG,TRO_WBFB_500,Trico,CARS,Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKTG0A,TRO_WB_T500,Trico,
Alternatively, there are open-source utilities called dos2unix and unix2dos available that do this very thing. On a linux system they are probably installed by default; for a windows system you can download them from http://www.bastet.com/ amongst others.
sed s/^M//g file1.txt > file2.txt
where ^M is typed by simultaneously pressing the 3 keys, ctrl + v + m
use dos2unix utility if the file was created on windows,
use mac2unix utility if the file was created on mac. :)
Use one of these commands:
:%s/\r//g
Or
:%s/\r\(\n\)/\1/g
In command mode in VIM:
:e ++ff=dos | setl ff=unix | up
e ++ff=dos - force open file in dos format.
setl ff=unix - convert file to unix format.
up - save file only when has been modified.
To save keystrokes, you can avoid typing Ctrl+VCtrl+M by placing this in a mapping. Just open a file containing a ^M character, yank it, and paste it into a line like this in your .vimrc:
nnoremap <Leader>d :%s/^M//g<CR>
This worked for me:
:% s/\r\n/\r
To use sed on MacOS, do this:
sed -i.bak $'s/\r//' <filename>
Explanation: The $'STRING' syntax here pertains to the bash shell. Macs don't treat \r as special character. By quoting the command string in $'' you're telling the shell to replace \r with the actual \r character specified in the ANSI-C standard.
None of these suggestions were working for me having managed to get a load of ^M line breaks while working with both vim and eclipse. I suspect that I encountered an outside case but in case it helps anyone I did.
:%s/.$//g
And it sorted out my problem
:g/^M/s// /g
If you type ^M using Shift+6 Caps+M it won't accept.
You need to type ctrl+v ctrl+m.
^M gives unwanted line breaks. To handle this we can use the sed command as follows:
sed 's/\r//g'
Just removeset binary in your .vimrc!
On Solaris:
:%s/<CTRL+V><CTRL+M>//g
that is:
:%s/^M//g
That means:
% = all lines,
s = substitute,
^M = what you desire to substitute
// = replace with nothing
g = globally (not only the first occurrance)