^M at the end of every line in Vim - vim

When I am editing source files using Vim and other editors, sometimes I get these ^M characters at the end of each line.
I think that it has something to do with editing a file on Windows and then on Linux.
How can I remove all of these automatically?

As a command, type
:%s/^M$//
(To get ^M, press ^V ^M, where ^ is CTRL on most keyboards)

One easy way to strip out the DOS line endings is to use the ff option:
:set ff=unix
:wq
Now your file is back to the good-old-Unix-way.
If you want to add the DOS line-endings (to keep a printer happy, or transfer files with Windows friends who don't have nice tools) you can go the opposite direction easily:
:set ff=dos
:wq

You can do this:
:set fileformats=dos
It will hide the ^M's, without touching the file.

There's a program called dos2unix that should strip those for you. Windows uses different line-ending characters which is why that happens.

This worked for me in a file that had everything on one line:
First find all matches
:%s/^M//
(To get ^M, press ^V ^M, where ^ is Ctrl on most keyboards)
Then replace with newlines
:%s//\r/g
Combined command would be:
:%s/^M/\r/g

I tend to run afflicted files through fromdos before reopening them. fromdos is part of the tofrodos package.

The origin of the problem may have been through an FTP transfer. When you FTP these files from one box to another, make sure to use ASCII transfers. Use the command "ASC."

" put this in your ~/.vimrc file and :source ~/.vimrc
" then you can do: Dos2Unix
" dos2unix ^M
fun! Dos2unixFunction()
let _s=#/
let l = line(".")
let c = col(".")
try
set ff=unix
w!
"%s/\%x0d$//e
catch /E32:/
echo "Sorry, first save the file."
endtry
let #/=_s
call cursor(l, c)
endfun
com! Dos2Unix keepjumps call Dos2unixFunction()

mcedit: shift+f2, set unix format (LF), ok

Related

Vim execute :e ++ff=dos from vimrc

Occasionally I have to work with mixed Unix/DOS files in Vim. That file will show ^M on the end on most of the lines. I can't convert the whole file to just Unix format.
Fix for that is :e ++ff=dos
I have tried to incorporate that in my vimrc at least as a shorcut (if not autocmd) but without success.
ga for ^M will show:
<^M> 13, Hex 0d, Octal 15
This won't work from vimrc:
function! Fix_dos()
execute "normal :e ++ff=dos<cr>"
endfunction
I have tried something with conceal feature, but that won't conceal all ^M chars.
:set conceallevel=2
:syntax match Todo /\r/ conceal
Also, is there a way to detect if file will show ^M chars?
Thanks for your help.
If you do :e +ff=dos, you convert the file to DOS format; i.e. after writing, all line endings will be converted to CR-LF. You've mentioned that you cannot convert to the whole file to Unix format, but if converting to DOS is okay, then this is the solution.
For your function, you don't need to go through :normal, as the :e command is an Ex command, and those can be directly used in a function. You only need :normal for normal-mode commands like dd. Therefore, you can rewrite:
function! Fix_dos()
" This would be the correct syntax, but no need for :normal:
"execute "normal :e ++ff=dos\<cr>"
" Also, no need for execute:
"execute "e ++ff=dos"
edit ++ff=dos
endfunction
In fact, I would define:
:command! FixDos edit ++ff=dos
If you cannot covert the entire file, you'll have to live with the ^M; best you can do is trying to hide them, as you've tried with concealing. This variant might catch more instances of ^M by also appying inside other syntax groups:
:syntax match Todo /\r/ conceal containedin=ALL
Opening a buffer in DOS fileformat does not convert the file to UNIX format. It is only sufficient if you wish to edit the buffer as DOS, and write it as DOS too. Hiding ^M also doesn't convert - your questions gives me the impression you think it does.
If you want to convert to LF-line-endings-only,
Remove all ^M characters: :%s/<C-V><cr>//ge<cr>. You can incorporate that in a function or autocommand.
Mark the buffer to use only LF line endings: :setlocal ff=unix
If you want to avoid having to press a key before your file opens:
"Press ENTER or type command to continue"
Run the command silently, like this:
com! SetFileFormatDos edit ++ff=dos
:silent SetFileFormatDos
Also, note the uppercase "S" from SetFileFormatDos.

How can I execute the current line as Vim EX commands?

Say I'm editing my _vimrc file and I've just added a couple of lines, for instance a new key mapping. I don't want to reload the whole file (:so %) since that will reset a lot of temporary stuff I'm experimenting with. I just want to run the two lines that I'm currently working on.
I'm having no luck trying to copy/paste the lines into the command buffer, since I can't use the put command in there. Is there any way I could run the current line (or current selection) as EX commands?
Summary:
After Anton Kovalenko's answer and Peter Rincker's comment I ended up with these key maps, which either executes the current line, or the current selected lines if in visual mode:
" Execute current line or current selection as Vim EX commands.
nnoremap <F2> :exe getline(".")<CR>
vnoremap <F2> :<C-w>exe join(getline("'<","'>"),'<Bar>')<CR>
To execute the current line as an ex command, you may also use:
yy:#"
This will yank the current line to the "-register and execute it. I don't think it is too much typing.
Executing the line under cursor as an Ex command:
:execute getline(".")
Convenient enough for 2 lines. (I'd figure out something for doing it with regions, but I'm not a vim user). And for currently selected region, the following seems to do the job:
:execute getreg("*")
As commented by Peter Rincker, this mapping can be used for executing the currently selected lines:
:vnoremap <f2> :<c-u>exe join(getline("'<","'>"),'<bar>')<cr>
For that purpose, I have defined the following commands and mappings:
":[range]Execute Execute text lines as ex commands.
" Handles |line-continuation|.
" The same can be achieved via "zyy#z (or yy#" through the unnamed register);
" but there, the ex command must be preceded by a colon (i.e. :ex)
command! -bar -range Execute silent <line1>,<line2>yank z | let #z = substitute(#z, '\n\s*\\', '', 'g') | #z
" [count]<Leader>e Execute current [count] line(s) as ex commands, then
" {Visual}<Leader>e jump to the following line (to allow speedy sequential
" execution of multiple lines).
nnoremap <silent> <Leader>e :Execute<Bar>execute 'normal! ' . v:count1 . 'j'<CR>
xnoremap <silent> <Leader>e :Execute<Bar>execute 'normal! ' . v:count1 . 'j'<CR>
Just after posting this, I found a work-around. I can copy text into the clipboard using "*y, then put that text into the command buffer by using the middle mouse button. This works for me, but is hardly a convenient solution for people without clipboard support, mouse support or just an aversion to removing their hands from the Vim position.
The accepted answer doesn't handle continuation sections. Also, surprisingly, the bar isn't needed, newlines are fine. This will work, first yanking the text into register x:
vno <c-x> "xy:exe substitute(#x,"\n\\",'','g')<cr>
As someone has already mentioned, the only exception are commands that "eat up" newlines. Eg, executing the above mapping on:
:sign define piet text=>> texthl=Search
:exe ":sign place 2 line=23 name=piet file=" . expand("%:p")
will cause vim to to think that the user is trying to define textl as "Search\n:exe ":sign place... etc.
You could also try
:<C-R><C-L><CR>
Per the vim docs, the combination will plop the current line into the command line. From there, hitting enter should do the trick. I realize that this does not handle multiline cases, however it doesn't require a .vimrc and therefore works out of the box.
If you're doing a lot of experimenting (trying things out that you might want to add to your vimrc, I assume?) it might help to do so in a scratch file like experimental.vim so you aren't just relying on your history to know what you're trying out. Now that you have these great mappings, it will be easy to rerun things from experimental or vimrc without sourcing the whole file.
Also (sorry, I can't comment on answers yet, it seems), I tried this mapping of Peter's:
vnoremap <Leader>es :<c-u>exec join(getline("'<","'>"),'<BAR>')<CR>
This works in most cases, but it fails specifically on function definitions.
function! TestMe()
echo "Yay!"
endfunction
This mapping joins the lines into a single string, separated by <BAR> and then execs them.
I'm not entirely sure why, but if I try to do that with a function definition in normal mode:
:exec 'function! TestMe()| echo "Yay!"|endfunction'
-> E488: Trailing characters
After some testing, I've found that it will work with newline separators instead:
:exec "function! TestMe()\n echo 'Yay!'\nendfunction"
:call TestMe()
-> Yay!
So, I've changed my mapping to this:
vnoremap <Leader>es :<c-u>exec join(getline("'<","'>"),"\n")<CR>
I suppose there is a vim or ex reason why the <BAR> method doesn't work on functions (maybe even some setting I have on?), and I'm curious to hear what it is if someone knows.
I don't want to reload the whole file (:so %) since that will reset a lot of temporary stuff I'm experimenting. I just want to run the two lines that I'm currently working on.
If you want to execute a command because you want to refine it before committing it to _.vimrc, then you should launch a Command Line Window for Ex-mode commands with q:.
At launch the Command Line Window is buffered with the contents of the command line history. It is a normal Vim window the contents of which can be edited as any text buffer with the exception of pressing on any line which executes the command on that line. It is very useful when you want to slightly change a long, complex command you wrote earlier and re-run it.
To launch a 'Command Line Window' for search strings press q/.
!! (shorthand for :.!) executes the current line as input to a command, per POSIX ex & vi. You may need to append sh if it is a system command.
Executing !! on a blank line (and omitting sh) is a shortcut for reading a shell command straight into the buffer. By it's nature :.! overwrites the current line while :.r! inserts on the line below.
ls -A | head -n +4
~
~
!sh
Results:
.sh_history
.sh_logout
.kshrc
corelist.txt
~
~
4 lines added; 1 line deleted`
This means there is no need to redirect pipelines to a file and then examine the data to see if the contents are valid. Just execute commands in vi directly and undo if you make a mistake.
Alternately, yanking a line as a named buffer allows you to execute it as an ex command, almost like a macro. You can still edit and undo the line to get it correct instead of trying to edit the : line in command mode.
The functions recommended here are all POSIX and have been supported for over 40 years, so no special vim or other enhanced features are required.
:%s/meep/pEEp/ | g/foo/ s//BAR
foo
grok
meep
~
~
Yank the ex command (line 1, :%s...) into a named buffer / macro.
I just use the label m for "macro".
"myy
or
:1y m
Now execute the named buffer / macro, in command mode, using #:
#m
Results:
:%s/pEEp/pEEp/ | g/BAR / s//BAR
BAR
grok
pEEp
~
~
4 lines changed
But remember that "multiple undo" is not POSIX. undo is only a toggle between undo and redo in a "historically accurate & compliant" ex / vi implementation.
The work-around is to save to a temporary (or valid) file name before executing a questionable edit:
:w $$.tmp
Then just :e! to "reset and reload" if needed.
You can also use :pre (preserve) to make a special temporary backup file prior to making multiple changes.
Then use :reco! % (recover this!) to restore back to that point.
Realize that :preserve creates a snapshot-like file which is deleted as soon as it is rolled back to. It does not matter if you save the edit(s) or not.
Therefore writing your own file (:w ...) and restoring with :e! may still have value because the system will not automatically delete it.
:pre is perfect when you should have ran sudo vi ... or otherwise do not have the necessary permissions - but you only realized the mistake after making several changes. i.e. vi /etc/sudoers instead of sudo vi /etc/sudoers.
^^ NEVER DO THIS! ONLY AN EXAMPLE! USE sudo visudo INSTEAD!
You can get a list of existing recovery files with vi -r and recover one directly with vi -r filename as needed, optionally with something like sudo vi -r filename.
The distinction here is that even though the ":preserved file" has it's own "special" name and path internally, it will :write to the original, intended location when ":recovered ==> /etc/sudoers
Just be sure to use :wq! and not something like ZZ when done with your "recovery" or you will still lose the edits which you tried to save.
By the way, ^R is expected to redraw or repaint the display per POSIX; it is not "undo" in any compliant vi implementation.

Remove specific character from code [duplicate]

When I am editing source files using Vim and other editors, sometimes I get these ^M characters at the end of each line.
I think that it has something to do with editing a file on Windows and then on Linux.
How can I remove all of these automatically?
As a command, type
:%s/^M$//
(To get ^M, press ^V ^M, where ^ is CTRL on most keyboards)
One easy way to strip out the DOS line endings is to use the ff option:
:set ff=unix
:wq
Now your file is back to the good-old-Unix-way.
If you want to add the DOS line-endings (to keep a printer happy, or transfer files with Windows friends who don't have nice tools) you can go the opposite direction easily:
:set ff=dos
:wq
You can do this:
:set fileformats=dos
It will hide the ^M's, without touching the file.
There's a program called dos2unix that should strip those for you. Windows uses different line-ending characters which is why that happens.
This worked for me in a file that had everything on one line:
First find all matches
:%s/^M//
(To get ^M, press ^V ^M, where ^ is Ctrl on most keyboards)
Then replace with newlines
:%s//\r/g
Combined command would be:
:%s/^M/\r/g
I tend to run afflicted files through fromdos before reopening them. fromdos is part of the tofrodos package.
The origin of the problem may have been through an FTP transfer. When you FTP these files from one box to another, make sure to use ASCII transfers. Use the command "ASC."
" put this in your ~/.vimrc file and :source ~/.vimrc
" then you can do: Dos2Unix
" dos2unix ^M
fun! Dos2unixFunction()
let _s=#/
let l = line(".")
let c = col(".")
try
set ff=unix
w!
"%s/\%x0d$//e
catch /E32:/
echo "Sorry, first save the file."
endtry
let #/=_s
call cursor(l, c)
endfun
com! Dos2Unix keepjumps call Dos2unixFunction()
mcedit: shift+f2, set unix format (LF), ok

How can I setup vim to automatically convert the line endings of any text file I open to unix if it already isn't unix?

I'm looking for some sort of plugin/vi-code that detects if there are any ^M ( carriage returns? ) in any files or any sort of indication that the line endings are not unix, and if so either remove them or run dos2unix on the file.
I need this to work automatically before/during the file opening.
Also, for the people who are going to suggest ways of converting the line endings manually, one answer points out do :set ff=unix but this doesn't kill ^M in a unix file, I think.
And isn't there a trick to using :%s/^M//g because using a literal ^ wont match the char?
The function below is interesting because
it keeps the cursor to its original position
put it in your. vimrc
" dos2unix ^M
fun! Dos2unixFunction()
let _s=#/
let l = line(".")
let c = col(".")
try
set ff=unix
w!
"%s/\%x0d$//e
catch /E32:/
echo "Sorry, the file is not saved."
endtry
let #/=_s
call cursor(l, c)
endfun
com! Dos2Unix keepjumps call Dos2unixFunction()
au BufReadPost * keepjumps call Dos2unixFunction()
Since you flagged this git - you can have git convert line ending s on checkin/out automatically (http://www.kernel.org/pub/software/scm/git/docs/git-config.html)
ps. The trick to doing ^M in a replace is
:%s/{Ctrl+V}{Ctrl+M}//{Enter}
If you want to replace any ^M in any file you open, an autocmd like the following in you vimrc could help:
au BufReadPost * %s/^M//g
Inserting the ^M as has been indicated in the previous answer.

Convert ^M (Windows) line breaks to normal line breaks

Vim shows ^M on every line ending.
How do I replace this with a normal line break in a file opened in Vim?
Command
:%s/<Ctrl-V><Ctrl-M>/\r/g
Where <Ctrl-V><Ctrl-M> means type Ctrl+V then Ctrl+M.
Explanation
:%s
substitute, % = all lines
<Ctrl-V><Ctrl-M>
^M characters (the Ctrl-V is a Vim way of writing the Ctrl ^ character and Ctrl-M writes the M after the regular expression, resulting to ^M special character)
/\r/
with new line (\r)
g
And do it globally (not just the first occurrence on the line).
On Linux and Mac OS, the following works,
:%s/^V^M/^V^M/g
where ^V^M means type Ctrl+V, then Ctrl+M.
Note: on Windows you probably want to use ^Q instead of ^V, since by default ^V is mapped to paste text.
Within vim, look at the file format — DOS or Unix:
:set filetype=unix
:set fileformat=unix
The file will be written back without carriage return (CR, ^M) characters.
This is the only thing that worked for me:
:e ++ff=dos
Found it at: http://vim.wikia.com/wiki/File_format
A file I had created with BBEdit seen in MacVim was displaying a bunch of ^M line returns instead of regular ones. The following string replace solved the issue:
:%s/\r/\r/g
It's interesting because I'm replacing line breaks with the same character, but I suppose Vim just needs to get a fresh \r to display correctly. I'd be interested to know the underlying mechanics of why this works.
First, use :set ff? to figure out the file format your file is.
I guess it could be unix, then the problem is your file was created with fileformat=dos adding "^M^J" to the line end but read with flieformat=unix only removing the "^J" from the line end, leaving the "^M" there.
Just input :e ++ff=dos in Vim command line to change your file's format from unix to dos. It should solve the problem. If not, :%s/\r//g should help you out.
in order to get the ^M character to match I had to visually select it and then use the OS copy to clipboard command to retrieve it. You can test it by doing a search for the character before trying the replace command.
/^M
should select the first bad line
:%s/^M/\r/g
will replace all the errant ^M with carriage returns.
This is as functions in MacVim, which is based on gvim 7.
EDIT:
Having this problem again on my Windows 10 machine, which has Ubuntu for Windows, and I think this is causing fileformat issues for vim. In this case changing the ff to unix, mac, or dos did nothing other than to change the ^M to ^J and back again.
The solution in this case:
:%s/\r$/ /g
:%s/ $//g
The reason I went this route is because I wanted to ensure I was being non-destructive with my file. I could have :%s/\r$//g but that would have deleted the carriage returns right out, and could have had unexpected results. Instead we convert the singular CR character, here a ^M character, into a space, and then remove all spaces at the end of lines (which for me is a desirable result regardless)
Sorry for reviving an old question that has long since been answered, but there seemed to be some confusion afoot and I thought I'd help clear some of that up since this is coming up high in google searches.
None of these worked for me, so I tried this, which worked:
type :%s/
press CTRL-VCTRL-M
type //g
press Enter
So the overall command in Vim shoud look like :%s/^M//g
What this does: :%s (find and replace) /^M/ (that symbol) / (with no chars) g (globally).
^M is retrieved by Ctrl+V and M, so do
s/^M//g
Without needing to use Ctrl:
:%s/\r$//
Simple thing that worked for me
dos2unix filename
I did this with sed:
sed -i -e 's/\r/\n/g' filename
What about just:
:%s/\r//g
That totally worked for me.
What this does is just to clean the end of line of all lines, it removes the ^M and that's it.
There are many other answers to this question, but still, the following works best for me, as I needed a command line solution:
vim -u NONE -c 'e ++ff=dos' -c 'w ++ff=unix' -c q myfile
Explanation:
Without loading any .vimrc files, open myfile
Run :e ++ff=dos to force a reload of the entire file as dos line endings.
Run :w ++ff=unix to write the file using unix line endings
Quit vim
Ctrl+M minimizes my window, but Ctrl+Enter actually inserts a ^M character. I also had to be sure not to lift off the Ctrl key between presses.
So the solution for me was:
:%s/<Ctrl-V><Ctrl-Enter>/\r/g
Where <Ctrl-V><Ctrl-Enter> means to press and hold Ctrl, press and release V, press and release Enter, and then release Ctrl.
If you are working on a Windows-generated file
The above solution will add an additional line between existing lines, because there is already an invisible \r after the ^M.
To prevent this, you want to delete the ^M characters without replacing them.
:%s/<Ctrl-V><Ctrl-Enter>//g
Where % means "in this buffer," s means "substitute," / means "(find) the following pattern," <Ctrl-V><Ctrl-Enter> refers to the keys to press to get the ^M character (see above), // means "with nothing" (or, "with the pattern between these two slashes, which is empty"), and g is a flag meaning "globally," as opposed to the first occurrence in a line.
This worked for me:
Set file format to unix (\n line ending)
save the file
So in vim:
:set ff=unix
:w
In my case,
Nothing above worked, I had a CSV file copied to Linux machine from my mac and I used all the above commands but nothing helped but the below one
tr "\015" "\n" < inputfile > outputfile
I had a file in which ^M characters were sandwitched between lines something like below
Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKT6TG,TRO_WBFB_500,Trico,CARS,Audi,A4,35 TFSi Premium,,CAAUA4TP^MB01BNKTG0A,TRO_WB_T500,Trico,
Alternatively, there are open-source utilities called dos2unix and unix2dos available that do this very thing. On a linux system they are probably installed by default; for a windows system you can download them from http://www.bastet.com/ amongst others.
sed s/^M//g file1.txt > file2.txt
where ^M is typed by simultaneously pressing the 3 keys, ctrl + v + m
use dos2unix utility if the file was created on windows,
use mac2unix utility if the file was created on mac. :)
Use one of these commands:
:%s/\r//g
Or
:%s/\r\(\n\)/\1/g
In command mode in VIM:
:e ++ff=dos | setl ff=unix | up
e ++ff=dos - force open file in dos format.
setl ff=unix - convert file to unix format.
up - save file only when has been modified.
To save keystrokes, you can avoid typing Ctrl+VCtrl+M by placing this in a mapping. Just open a file containing a ^M character, yank it, and paste it into a line like this in your .vimrc:
nnoremap <Leader>d :%s/^M//g<CR>
This worked for me:
:% s/\r\n/\r
To use sed on MacOS, do this:
sed -i.bak $'s/\r//' <filename>
Explanation: The $'STRING' syntax here pertains to the bash shell. Macs don't treat \r as special character. By quoting the command string in $'' you're telling the shell to replace \r with the actual \r character specified in the ANSI-C standard.
None of these suggestions were working for me having managed to get a load of ^M line breaks while working with both vim and eclipse. I suspect that I encountered an outside case but in case it helps anyone I did.
:%s/.$//g
And it sorted out my problem
:g/^M/s// /g
If you type ^M using Shift+6 Caps+M it won't accept.
You need to type ctrl+v ctrl+m.
^M gives unwanted line breaks. To handle this we can use the sed command as follows:
sed 's/\r//g'
Just removeset binary in your .vimrc!
On Solaris:
:%s/<CTRL+V><CTRL+M>//g
that is:
:%s/^M//g
That means:
% = all lines,
s = substitute,
^M = what you desire to substitute
// = replace with nothing
g = globally (not only the first occurrance)

Resources