I used to know a vim command sequence where I would remove all the non-printable extraneous characters in a file, like ^M on Windows files that ended up with col -xb, but I can't recall the keyboard sequence before I enter the col -xb command to say "do this from top-to-bottom of file." Does this ring a bell to anybody?
:%!col -xb
: enters the ex-command mode as usual,% selects all lines (you can also write other ranges, like 100,150 for lines 100 to 150) and ! pipes them through an external command.
run the following command in vim:
:set fileformat=unix
it worked for me, and also check out: Are shell scripts sensitive to encoding and line endings?
I have a CSV file. Because of some reasons, when I open it with vim in ubuntu, some lines appear to be like that :
This is actually ^M
a line.
I want the result to be :
This is actually a line.
So, I want to delete all ^M characters and connect those lines separated by them.
These may be multi-line cells saved from Excel. It looks as if you have carriage-return/linefeed pairs, as you have ^M (carriage return) followed by a line break.
The example you show has a space before the ^M - is that always the case?
To remove all carriage-return/linefeed pairs you could use:
:%s/\r\n//g
(\r is the escape sequence for carriage return, and \n is the escape sequence for linefeed or newline)
Or to replace them by spaces:
:%s/\r\n/ /g
The ^M is a carriage return, probably this file was created in Windows. You could simply call dos2unix filename.csv
In vim you can do :%s/^M//g, (press ^ followed by M while holding down Ctrl to get the ^M).
First You should reopen the file with the correct lineendings:
:edit ++ff=dos
To prevent this mistake in the future, put this in to Your .vimrc:
set fileformats=unix,dos
Joining of lines should work now.
If you want to join only the lines with ^M ending, then forgot the aboves and use this command:
:g/^M/ s/^M// | normal J
To type in ^M, press C-V then Enter.
My vim used ctrl-v as paste short-cut, which conflict with commands need ctrl-v as shortcut prefix.
For example, ^M need ctrl-v ctrl-m. I can do that in vim command line, and vim without intialization. But the symbol won't go to buffer from vim command line. :s/aaa/^M/g doesn't work.
Although, I can use echo -e "\r" or no-initialized vim to work around it.
But how to type this symbol in my current vim configuration?
to input ^M you can try followings:
Ctrl-V Enter
Ctrl-V Ctrl-M
use digraph Ctrl-K C R, C R means CR.
You could try using a mapping to the original function of ctrl-v:
noremap! <C-Q> <C-V>
But as explained on :h CTRL-V-alternative and noted by ZyX on the comments, your issues may be that your terminal is capturing the ctrl-q (more details here). If the mapping above doesn't works you could try mapping to something else:
noremap! <leader><C-V> <C-V>
Then hitting leader (usually \) followed by ctrl-v ctrl-m should insert ^M on both insert and command mode.
try Ctrl-Q Enter
(This solution also works when you try to enter digraphs by number in Vim)
Vim shows ^M on every line ending.
How do I replace this with a normal line break in a file opened in Vim?
Command
:%s/<Ctrl-V><Ctrl-M>/\r/g
Where <Ctrl-V><Ctrl-M> means type Ctrl+V then Ctrl+M.
Explanation
:%s
substitute, % = all lines
<Ctrl-V><Ctrl-M>
^M characters (the Ctrl-V is a Vim way of writing the Ctrl ^ character and Ctrl-M writes the M after the regular expression, resulting to ^M special character)
/\r/
with new line (\r)
g
And do it globally (not just the first occurrence on the line).
On Linux and Mac OS, the following works,
:%s/^V^M/^V^M/g
where ^V^M means type Ctrl+V, then Ctrl+M.
Note: on Windows you probably want to use ^Q instead of ^V, since by default ^V is mapped to paste text.
Within vim, look at the file format — DOS or Unix:
:set filetype=unix
:set fileformat=unix
The file will be written back without carriage return (CR, ^M) characters.
This is the only thing that worked for me:
:e ++ff=dos
Found it at: http://vim.wikia.com/wiki/File_format
A file I had created with BBEdit seen in MacVim was displaying a bunch of ^M line returns instead of regular ones. The following string replace solved the issue:
:%s/\r/\r/g
It's interesting because I'm replacing line breaks with the same character, but I suppose Vim just needs to get a fresh \r to display correctly. I'd be interested to know the underlying mechanics of why this works.
First, use :set ff? to figure out the file format your file is.
I guess it could be unix, then the problem is your file was created with fileformat=dos adding "^M^J" to the line end but read with flieformat=unix only removing the "^J" from the line end, leaving the "^M" there.
Just input :e ++ff=dos in Vim command line to change your file's format from unix to dos. It should solve the problem. If not, :%s/\r//g should help you out.
in order to get the ^M character to match I had to visually select it and then use the OS copy to clipboard command to retrieve it. You can test it by doing a search for the character before trying the replace command.
/^M
should select the first bad line
:%s/^M/\r/g
will replace all the errant ^M with carriage returns.
This is as functions in MacVim, which is based on gvim 7.
EDIT:
Having this problem again on my Windows 10 machine, which has Ubuntu for Windows, and I think this is causing fileformat issues for vim. In this case changing the ff to unix, mac, or dos did nothing other than to change the ^M to ^J and back again.
The solution in this case:
:%s/\r$/ /g
:%s/ $//g
The reason I went this route is because I wanted to ensure I was being non-destructive with my file. I could have :%s/\r$//g but that would have deleted the carriage returns right out, and could have had unexpected results. Instead we convert the singular CR character, here a ^M character, into a space, and then remove all spaces at the end of lines (which for me is a desirable result regardless)
Sorry for reviving an old question that has long since been answered, but there seemed to be some confusion afoot and I thought I'd help clear some of that up since this is coming up high in google searches.
None of these worked for me, so I tried this, which worked:
type :%s/
press CTRL-VCTRL-M
type //g
press Enter
So the overall command in Vim shoud look like :%s/^M//g
What this does: :%s (find and replace) /^M/ (that symbol) / (with no chars) g (globally).
^M is retrieved by Ctrl+V and M, so do
s/^M//g
Without needing to use Ctrl:
:%s/\r$//
Simple thing that worked for me
dos2unix filename
I did this with sed:
sed -i -e 's/\r/\n/g' filename
What about just:
:%s/\r//g
That totally worked for me.
What this does is just to clean the end of line of all lines, it removes the ^M and that's it.
There are many other answers to this question, but still, the following works best for me, as I needed a command line solution:
vim -u NONE -c 'e ++ff=dos' -c 'w ++ff=unix' -c q myfile
Explanation:
Without loading any .vimrc files, open myfile
Run :e ++ff=dos to force a reload of the entire file as dos line endings.
Run :w ++ff=unix to write the file using unix line endings
Quit vim
Ctrl+M minimizes my window, but Ctrl+Enter actually inserts a ^M character. I also had to be sure not to lift off the Ctrl key between presses.
So the solution for me was:
:%s/<Ctrl-V><Ctrl-Enter>/\r/g
Where <Ctrl-V><Ctrl-Enter> means to press and hold Ctrl, press and release V, press and release Enter, and then release Ctrl.
If you are working on a Windows-generated file
The above solution will add an additional line between existing lines, because there is already an invisible \r after the ^M.
To prevent this, you want to delete the ^M characters without replacing them.
:%s/<Ctrl-V><Ctrl-Enter>//g
Where % means "in this buffer," s means "substitute," / means "(find) the following pattern," <Ctrl-V><Ctrl-Enter> refers to the keys to press to get the ^M character (see above), // means "with nothing" (or, "with the pattern between these two slashes, which is empty"), and g is a flag meaning "globally," as opposed to the first occurrence in a line.
This worked for me:
Set file format to unix (\n line ending)
save the file
So in vim:
:set ff=unix
:w
In my case,
Nothing above worked, I had a CSV file copied to Linux machine from my mac and I used all the above commands but nothing helped but the below one
tr "\015" "\n" < inputfile > outputfile
I had a file in which ^M characters were sandwitched between lines something like below
Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKT6TG,TRO_WBFB_500,Trico,CARS,Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKTG0A,TRO_WB_T500,Trico,
Alternatively, there are open-source utilities called dos2unix and unix2dos available that do this very thing. On a linux system they are probably installed by default; for a windows system you can download them from http://www.bastet.com/ amongst others.
sed s/^M//g file1.txt > file2.txt
where ^M is typed by simultaneously pressing the 3 keys, ctrl + v + m
use dos2unix utility if the file was created on windows,
use mac2unix utility if the file was created on mac. :)
Use one of these commands:
:%s/\r//g
Or
:%s/\r\(\n\)/\1/g
In command mode in VIM:
:e ++ff=dos | setl ff=unix | up
e ++ff=dos - force open file in dos format.
setl ff=unix - convert file to unix format.
up - save file only when has been modified.
To save keystrokes, you can avoid typing Ctrl+VCtrl+M by placing this in a mapping. Just open a file containing a ^M character, yank it, and paste it into a line like this in your .vimrc:
nnoremap <Leader>d :%s/^M//g<CR>
This worked for me:
:% s/\r\n/\r
To use sed on MacOS, do this:
sed -i.bak $'s/\r//' <filename>
Explanation: The $'STRING' syntax here pertains to the bash shell. Macs don't treat \r as special character. By quoting the command string in $'' you're telling the shell to replace \r with the actual \r character specified in the ANSI-C standard.
None of these suggestions were working for me having managed to get a load of ^M line breaks while working with both vim and eclipse. I suspect that I encountered an outside case but in case it helps anyone I did.
:%s/.$//g
And it sorted out my problem
:g/^M/s// /g
If you type ^M using Shift+6 Caps+M it won't accept.
You need to type ctrl+v ctrl+m.
^M gives unwanted line breaks. To handle this we can use the sed command as follows:
sed 's/\r//g'
Just removeset binary in your .vimrc!
On Solaris:
:%s/<CTRL+V><CTRL+M>//g
that is:
:%s/^M//g
That means:
% = all lines,
s = substitute,
^M = what you desire to substitute
// = replace with nothing
g = globally (not only the first occurrance)