I have a CSV file. Because of some reasons, when I open it with vim in ubuntu, some lines appear to be like that :
This is actually ^M
a line.
I want the result to be :
This is actually a line.
So, I want to delete all ^M characters and connect those lines separated by them.
These may be multi-line cells saved from Excel. It looks as if you have carriage-return/linefeed pairs, as you have ^M (carriage return) followed by a line break.
The example you show has a space before the ^M - is that always the case?
To remove all carriage-return/linefeed pairs you could use:
:%s/\r\n//g
(\r is the escape sequence for carriage return, and \n is the escape sequence for linefeed or newline)
Or to replace them by spaces:
:%s/\r\n/ /g
The ^M is a carriage return, probably this file was created in Windows. You could simply call dos2unix filename.csv
In vim you can do :%s/^M//g, (press ^ followed by M while holding down Ctrl to get the ^M).
First You should reopen the file with the correct lineendings:
:edit ++ff=dos
To prevent this mistake in the future, put this in to Your .vimrc:
set fileformats=unix,dos
Joining of lines should work now.
If you want to join only the lines with ^M ending, then forgot the aboves and use this command:
:g/^M/ s/^M// | normal J
To type in ^M, press C-V then Enter.
When I try to execute my script I got ^M is an invalid character but in Vim, I see $ upon entering :set list
I tried :%s/^V^M//g but it says ^M pattern is not found
I guessed this occurred because I used some .vimrc I found here which converts the end of line characters to $
Without figuring this out, my only option would be retyping my script.
It looks like your script consistently has ^M line endings, and therefore got detected as fileformat=dos. :setlocal fileformat? will tell you.
To convert this file to Unix (LF) line endings, just :setlocal fileformat=unix and :write, or combine this in :w ++ff=unix.
If you never want Vim to detect such files (and show the ^M instead), put :set fileformats-=dos into your ~/.vimrc (or edit an existing config).
se nolist and the dollar signs will disappear.
No matter the OS, you always have line endings in your text.
Line endings are whitespace and always present, but usually just not shown.
Are you using Windows or *nix?
in windows, you can replace ^M as ctlr-q ctrl-m to input the ^M. In *nix, you can just use dos2unix to translate your script file to unix format.
If after opening your file you don’t see ^M at the end of line, but when you try sourcing it vim does show complains about ^M in various places the only thing you need to do is w ++ff=unix and reopen this file.
When you open a file vim detects line ending format. Thus trying to substitute ^M will not work: all detected line endings are converted into internal string end. E.g. when file format is dos like in your case it looks like
set nocompatible\r\nset ignorecase\r\n...
(where \r is carriage return, sometimes represented as ^M, and \n is line feed character, and \r\n sequence is dos line ending). When file format is unix it looks like
set nocompatible\nset ignorecase\n...
. For mac it looks like
set nocompatible\rset ignorecase\r...
. But if vim correctly detected line ending all these files transform into
"set nocompatible"
"set ignorecase"
"..."
C strings in internal structure representing buffer, each string represents one line. No \r and no \n are there.
When you do :w files are converted back into a sequence of bytes. :w ++ff=unix forces line endings. Reopening is needed because fileformat setting is not changed in this case thus next w without ++ff will save with dos line endings again. When you reopen line endings are redetected and fileformat setting is reassigned. You can do :set fileformat=unix manually after :w ++ff=unix, but :e is much faster to type.
I noticed that sometimes Vim shows ^M at the end of every line, or ^[ in front of an opening bracket [.
What do these characters mean and how can I get rid of them?
I'm running Vim 7.3 on Debian.
^M is dos-style line endings. You can get rid of them by using the dos2unix program:
dos2unix (yourfile)
These are control characters. The ^M represents the carriage return, used in windows as the other answer already explain.
The ^[ is the escape character. When followed by a opening square bracket ("[") it probably means an ANSI escape sequence. See this article to know more:
http://en.wikipedia.org/wiki/ANSI_escape_code
And give it a try. For example, in your terminal:
echo ^[[7mHello World!^[[m
Where each ^[ can be inserted with controlVcontrol[. So the sequence of typing is actually:
... controlVcontrol[[7m ...
These are control characters. Here is a link on how to remove them in vi.
This article on Vim wiki should help you: File format.
Though the article title may seem to be different it does talk about line endings and unix/dos/macos file formats.
Vim shows ^M on every line ending.
How do I replace this with a normal line break in a file opened in Vim?
Command
:%s/<Ctrl-V><Ctrl-M>/\r/g
Where <Ctrl-V><Ctrl-M> means type Ctrl+V then Ctrl+M.
Explanation
:%s
substitute, % = all lines
<Ctrl-V><Ctrl-M>
^M characters (the Ctrl-V is a Vim way of writing the Ctrl ^ character and Ctrl-M writes the M after the regular expression, resulting to ^M special character)
/\r/
with new line (\r)
g
And do it globally (not just the first occurrence on the line).
On Linux and Mac OS, the following works,
:%s/^V^M/^V^M/g
where ^V^M means type Ctrl+V, then Ctrl+M.
Note: on Windows you probably want to use ^Q instead of ^V, since by default ^V is mapped to paste text.
Within vim, look at the file format — DOS or Unix:
:set filetype=unix
:set fileformat=unix
The file will be written back without carriage return (CR, ^M) characters.
This is the only thing that worked for me:
:e ++ff=dos
Found it at: http://vim.wikia.com/wiki/File_format
A file I had created with BBEdit seen in MacVim was displaying a bunch of ^M line returns instead of regular ones. The following string replace solved the issue:
:%s/\r/\r/g
It's interesting because I'm replacing line breaks with the same character, but I suppose Vim just needs to get a fresh \r to display correctly. I'd be interested to know the underlying mechanics of why this works.
First, use :set ff? to figure out the file format your file is.
I guess it could be unix, then the problem is your file was created with fileformat=dos adding "^M^J" to the line end but read with flieformat=unix only removing the "^J" from the line end, leaving the "^M" there.
Just input :e ++ff=dos in Vim command line to change your file's format from unix to dos. It should solve the problem. If not, :%s/\r//g should help you out.
in order to get the ^M character to match I had to visually select it and then use the OS copy to clipboard command to retrieve it. You can test it by doing a search for the character before trying the replace command.
/^M
should select the first bad line
:%s/^M/\r/g
will replace all the errant ^M with carriage returns.
This is as functions in MacVim, which is based on gvim 7.
EDIT:
Having this problem again on my Windows 10 machine, which has Ubuntu for Windows, and I think this is causing fileformat issues for vim. In this case changing the ff to unix, mac, or dos did nothing other than to change the ^M to ^J and back again.
The solution in this case:
:%s/\r$/ /g
:%s/ $//g
The reason I went this route is because I wanted to ensure I was being non-destructive with my file. I could have :%s/\r$//g but that would have deleted the carriage returns right out, and could have had unexpected results. Instead we convert the singular CR character, here a ^M character, into a space, and then remove all spaces at the end of lines (which for me is a desirable result regardless)
Sorry for reviving an old question that has long since been answered, but there seemed to be some confusion afoot and I thought I'd help clear some of that up since this is coming up high in google searches.
None of these worked for me, so I tried this, which worked:
type :%s/
press CTRL-VCTRL-M
type //g
press Enter
So the overall command in Vim shoud look like :%s/^M//g
What this does: :%s (find and replace) /^M/ (that symbol) / (with no chars) g (globally).
^M is retrieved by Ctrl+V and M, so do
s/^M//g
Without needing to use Ctrl:
:%s/\r$//
Simple thing that worked for me
dos2unix filename
I did this with sed:
sed -i -e 's/\r/\n/g' filename
What about just:
:%s/\r//g
That totally worked for me.
What this does is just to clean the end of line of all lines, it removes the ^M and that's it.
There are many other answers to this question, but still, the following works best for me, as I needed a command line solution:
vim -u NONE -c 'e ++ff=dos' -c 'w ++ff=unix' -c q myfile
Explanation:
Without loading any .vimrc files, open myfile
Run :e ++ff=dos to force a reload of the entire file as dos line endings.
Run :w ++ff=unix to write the file using unix line endings
Quit vim
Ctrl+M minimizes my window, but Ctrl+Enter actually inserts a ^M character. I also had to be sure not to lift off the Ctrl key between presses.
So the solution for me was:
:%s/<Ctrl-V><Ctrl-Enter>/\r/g
Where <Ctrl-V><Ctrl-Enter> means to press and hold Ctrl, press and release V, press and release Enter, and then release Ctrl.
If you are working on a Windows-generated file
The above solution will add an additional line between existing lines, because there is already an invisible \r after the ^M.
To prevent this, you want to delete the ^M characters without replacing them.
:%s/<Ctrl-V><Ctrl-Enter>//g
Where % means "in this buffer," s means "substitute," / means "(find) the following pattern," <Ctrl-V><Ctrl-Enter> refers to the keys to press to get the ^M character (see above), // means "with nothing" (or, "with the pattern between these two slashes, which is empty"), and g is a flag meaning "globally," as opposed to the first occurrence in a line.
This worked for me:
Set file format to unix (\n line ending)
save the file
So in vim:
:set ff=unix
:w
In my case,
Nothing above worked, I had a CSV file copied to Linux machine from my mac and I used all the above commands but nothing helped but the below one
tr "\015" "\n" < inputfile > outputfile
I had a file in which ^M characters were sandwitched between lines something like below
Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKT6TG,TRO_WBFB_500,Trico,CARS,Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKTG0A,TRO_WB_T500,Trico,
Alternatively, there are open-source utilities called dos2unix and unix2dos available that do this very thing. On a linux system they are probably installed by default; for a windows system you can download them from http://www.bastet.com/ amongst others.
sed s/^M//g file1.txt > file2.txt
where ^M is typed by simultaneously pressing the 3 keys, ctrl + v + m
use dos2unix utility if the file was created on windows,
use mac2unix utility if the file was created on mac. :)
Use one of these commands:
:%s/\r//g
Or
:%s/\r\(\n\)/\1/g
In command mode in VIM:
:e ++ff=dos | setl ff=unix | up
e ++ff=dos - force open file in dos format.
setl ff=unix - convert file to unix format.
up - save file only when has been modified.
To save keystrokes, you can avoid typing Ctrl+VCtrl+M by placing this in a mapping. Just open a file containing a ^M character, yank it, and paste it into a line like this in your .vimrc:
nnoremap <Leader>d :%s/^M//g<CR>
This worked for me:
:% s/\r\n/\r
To use sed on MacOS, do this:
sed -i.bak $'s/\r//' <filename>
Explanation: The $'STRING' syntax here pertains to the bash shell. Macs don't treat \r as special character. By quoting the command string in $'' you're telling the shell to replace \r with the actual \r character specified in the ANSI-C standard.
None of these suggestions were working for me having managed to get a load of ^M line breaks while working with both vim and eclipse. I suspect that I encountered an outside case but in case it helps anyone I did.
:%s/.$//g
And it sorted out my problem
:g/^M/s// /g
If you type ^M using Shift+6 Caps+M it won't accept.
You need to type ctrl+v ctrl+m.
^M gives unwanted line breaks. To handle this we can use the sed command as follows:
sed 's/\r//g'
Just removeset binary in your .vimrc!
On Solaris:
:%s/<CTRL+V><CTRL+M>//g
that is:
:%s/^M//g
That means:
% = all lines,
s = substitute,
^M = what you desire to substitute
// = replace with nothing
g = globally (not only the first occurrance)