How to replace a character by a newline in Vim - vim
I'm trying to replace each , in the current file by a new line:
:%s/,/\n/g
But it inserts what looks like a ^# instead of an actual newline. The file is not in DOS mode or anything.
What should I do?
If you are curious, like me, check the question Why is \r a newline for Vim? as well.
Use \r instead of \n.
Substituting by \n inserts a null character into the text. To get a newline, use \r. When searching for a newline, you’d still use \n, however. This asymmetry is due to the fact that \n and \r do slightly different things:
\n matches an end of line (newline), whereas \r matches a carriage return. On the other hand, in substitutions \n inserts a null character whereas \r inserts a newline (more precisely, it’s treated as the input CR). Here’s a small, non-interactive example to illustrate this, using the Vim command line feature (in other words, you can copy and paste the following into a terminal to run it). xxd shows a hexdump of the resulting file.
echo bar > test
(echo 'Before:'; xxd test) > output.txt
vim test '+s/b/\n/' '+s/a/\r/' +wq
(echo 'After:'; xxd test) >> output.txt
more output.txt
Before:
0000000: 6261 720a bar.
After:
0000000: 000a 720a ..r.
In other words, \n has inserted the byte 0x00 into the text; \r has inserted the byte 0x0a.
Here's the trick:
First, set your Vi(m) session to allow pattern matching with special characters (i.e.: newline). It's probably worth putting this line in your .vimrc or .exrc file:
:set magic
Next, do:
:s/,/,^M/g
To get the ^M character, type Ctrl + V and hit Enter. Under Windows, do Ctrl + Q, Enter. The only way I can remember these is by remembering how little sense they make:
A: What would be the worst control-character to use to represent a newline?
B: Either q (because it usually means "Quit") or v because it would be so easy to type Ctrl + C by mistake and kill the editor.
A: Make it so.
In the syntax s/foo/bar, \r and \n have different meanings, depending on context.
Short:
For foo:
\r == "carriage return" (CR / ^M)
\n == matches "line feed" (LF) on Linux/Mac, and CRLF on Windows
For bar:
\r == produces LF on Linux/Mac, CRLF on Windows
\n == "null byte" (NUL / ^#)
When editing files in linux (i.e. on a webserver) that were initially created in a windows environment and uploaded (i.e. FTP/SFTP) - all the ^M's you see in vim, are the CR's which linux does not translate as it uses only LF's to depict a line break.
Longer (with ASCII numbers):
NUL == 0x00 == 0 == Ctrl + # == ^# shown in vim
LF == 0x0A == 10 == Ctrl + J
CR == 0x0D == 13 == Ctrl + M == ^M shown in vim
Here is a list of the ASCII control characters. Insert them in Vim via Ctrl + V,Ctrl + ---key---.
In Bash or the other Unix/Linux shells, just type Ctrl + ---key---.
Try Ctrl + M in Bash. It's the same as hitting Enter, as the shell realizes what is meant, even though Linux systems use line feeds for line delimiting.
To insert literal's in bash, prepending them with Ctrl + V will also work.
Try in Bash:
echo ^[[33;1mcolored.^[[0mnot colored.
This uses ANSI escape sequences. Insert the two ^['s via Ctrl + V, Esc.
You might also try Ctrl + V,Ctrl + M, Enter, which will give you this:
bash: $'\r': command not found
Remember the \r from above? :>
This ASCII control characters list is different from a complete ASCII symbol table, in that the control characters, which are inserted into a console/pseudoterminal/Vim via the Ctrl key (haha), can be found there.
Whereas in C and most other languages, you usually use the octal codes to represent these 'characters'.
If you really want to know where all this comes from: The TTY demystified. This is the best link you will come across about this topic, but beware: There be dragons.
TL;DR
Usually foo = \n, and bar = \r.
You need to use:
:%s/,/^M/g
To get the ^M character, press Ctrl + v followed by Enter.
\r can do the work here for you.
With Vim on Windows, use Ctrl + Q in place of Ctrl + V.
This is the best answer for the way I think, but it would have been nicer in a table:
Why is \r a newline for Vim?
So, rewording:
You need to use \r to use a line feed (ASCII 0x0A, the Unix newline) in a regex replacement, but that is peculiar to the replacement - you should normally continue to expect to use \n for line feed and \r for carriage return.
This is because Vim used \n in a replacement to mean the NIL character (ASCII 0x00). You might have expected NIL to have been \0 instead, freeing \n for its usual use for line feed, but \0 already has a meaning in regex replacements, so it was shifted to \n. Hence then going further to also shift the newline from \n to \r (which in a regex pattern is the carriage return character, ASCII 0x0D).
Character | ASCII code | C representation | Regex match | Regex replacement
-------------------------+------------+------------------+-------------+------------------------
nil | 0x00 | \0 | \0 | \n
line feed (Unix newline) | 0x0a | \n | \n | \r
carriage return | 0x0d | \r | \r | <unknown>
NB: ^M (Ctrl + V Ctrl + M on Linux) inserts a newline when used in a regex replacement rather than a carriage return as others have advised (I just tried it).
Also note that Vim will translate the line feed character when it saves to file based on its file format settings and that might confuse matters.
From Eclipse, the ^M characters can be embedded in a line, and you want to convert them to newlines.
:s/\r/\r/g
But if one has to substitute, then the following thing works:
:%s/\n/\r\|\-\r/g
In the above, every next line is substituted with next line, and then |- and again a new line. This is used in wiki tables.
If the text is as follows:
line1
line2
line3
It is changed to
line1
|-
line2
|-
line3
Here's the answer that worked for me. From this guy:
----quoting Use the vi editor to insert a newline char in replace
Something else I have to do and cannot remember and then have to look up.
In vi, to insert a newline character in a search and replace, do the following:
:%s/look_for/replace_with^M/g
The command above would replace all instances of “look_for” with “replace_with\n” (with \n meaning newline).
To get the “^M”, enter the key combination Ctrl + V, and then after that (release all keys) press the Enter key.
If you need to do it for a whole file, it was also suggested to me that you could try from the command line:
sed 's/\\n/\n/g' file > newfile
in vim editor the following command successfully replaced \n with new line
:%s/\\n/\r/g
Related
how to visualise and delete trailing newline at the end of file in vim\nvim
Sometimes I need to edit files which should not end with a newline. However vim\nvim by default do not visualise in any way the newline character at the end of file. Therefore I am not able to: visually confirm if the file has a newline character at the end or not remove that character Are there any setting which would allow me to see the tailing newline character and edit it in the same way as any other characters? For example, after create 2 files as follows: echo test > file-with-newline echo -n test > file-without-newline opening first one with nvim file-with-newline shows: test ~ ~ file-with-newline opening second one with nvim file-without-newline shows: test ~ ~ file-without-newline Navigating with the cursor to the end of line in either case yields the same result (the cursor stops after last visible character: t). There is no way to tell if the newline is there or not, let alone remove it using familiar commands used to remove ordinary characters (or newlines within the file).
You can enable the option :help 'list': :set list to show that "newline character" as a $ at the end of the line (among other things): Note, however, that the option doesn't make the character "editable" in any way.
if the file has a newline character at the end or not :set eol? endofline remove that character :set noeol nofixeol :update
Surround parentheses with its content by extra parentheses
I have lines that I want to convert from (variable=value) to ((variable=value)) How should I go about doing that from the vim command line?
May be you can use following substitute command :%s/(.*)/(&)/g where .* - all strings of characters of any length and & - the whole matched pattern
This is commonly done with the surround.vim plugin: First select the single-parentheses block with va(, then surround with another set of parens via S(.
One possible solution (if you have only this pattern in the line): :.s/.*/(&) .s ................... current line substitute .* ................... everything ( .................... open paren & .................... all pattern searched ) .................... close paren Or :norm! I(^[A) OBS: The simbol ^[ should be typed with Ctrl-vCtrl-[. Don't forget you can repat the last command : in the current line by typing #: and in the subsequent lines ##. In normal mode With no plugins you can do this (if the pattern does not repeat a lot): ca( ................ start changing the pattern (text goes to default register ") ( .................. start typing open parenthesis Ctrl-r" ............ insert default register ) .................. close parenthesis OBS: This action is repeatable by typing .
perl output messed up in fedora, ubuntu
I wrote a perl script for mapping two data sets. When I run the program using the Linux terminal, the output is messed up. It seems like the output is overlapping. I am using Fedora 25. I have tried the code on Windows and it works fine.
Same problem is there on Ubuntu as well.
DESIRED:
ADAM 123 JOHN 321
TOM 473 BENTLY 564
and so on....
OUTPUT that i am getting:
ADAM 123N 321
TOM 473TLY 564
and so on......
I have tested the code on Windows and it works perfectly fine. Though the same problem remains on Ubuntu 16.04 lts.
please help.
code:
use warnings;
open F, "friendship_network_wo_weights1.txt", or die;
open G, "username_gender_1.txt", or die;
while (<G>){
chomp $_;
my #a = split /\t/, $_;
$list{$a[0]} = $a[1];
}
close G;
while (<F>){
chomp $_;
my #b = split /\t/, $_;
if ((exists $list{$b[0]}) && (exists $list{$b[1]})){
$get = "$b[0]\t${list{$b[0]}}\t$b[1]\t${list{$b[1]}}\n";
$get =~ s/\r//g;
print "$get";
}
}
close F;
The problem is on Windows the newline is \r\n. On everything else it's \n. Assuming these files were created on Windows, when you read them on Unix each line will still have a trailing \r after the chomp.
\r is the "carriage return" character. It's like on an old typewriter how you had to move the whole typehead back to the left side at the end of a line, computer displays used to be fancy typewriters called Teleprinters. When you print it, the cursor moves back to the beginning of the line. Anything you print after that gets overwritten. Here's a simple example.
print "foo\rbar\r\n";
What you'll see is bar. This is because it prints...
foo
\r sends the cursor back to the start of the line
bar overwrites foo
\r sends the cursor back to the start of the line
\n goes to the start of the next line (doesn't matter where the cursor is)
chomp will only remove whatever is in $/ off the end of the string. On Unix that's \n. On Windows it's \r\n.
There's a number of ways to solve this. One of the safest is to manually remove newlines of both types with a regex.
# \015 is octal character 015 which is carriage return.
# \012 is octal character 012 which is newline
$line =~ s{\015?\012$}{};
That says to remove maybe a \r and definitely a \n at the end of the line.
vi keep only first 10 characters of a column
how do i do this in vi?
awk -F"," awk '{print substr($1,1,10)}'
I only want to keep the first 10 characters of my date column (example 2014-01-01) and not include the timestamp.
I tried to do it in awk but i got this error:
sed: RE error: illegal byte sequence
I believe it is a bash_profile setting error.
This is what i have in my bash_profile:
#export LANG=en_US.UTF-8
#export LOCALE=UTF-8
export LC_CTYPE=C
export LANG=C
in vim, do: :%norm! 11|D this will affect all lines in your buffer. If you like, :s could do this job too.
:%s/.\{,10}\zs.*//
:%s/: apply the substitution to all the lines
.\{,10}: match anything up to 10 times (greedy)
\zs: indicates the beginning of the match
.*: match the rest of the line
/: end of the first part of :s
/: end of the second part of s (since there's nothing between the two /, replace with nothing, ie delete)
For editing blocks of text there is a -- VISUAL BLOCK -- mode accessible via CTRL-V (on Windows ussually CTRL-Q). Then you can press d to delete your selection. Or with a simple substitute command :%s/\%>10c.*// \%>10c - matches after tenth column . - matches any single character but not an end-of-line * - matches 0 or more of the preceding atom, as many as possible Or you can use range :1,3s/\%>10c.*// This would substitute for the first three lines.
Why can't vim insert newlines with s? [duplicate]
I'm trying to replace each , in the current file by a new line: :%s/,/\n/g But it inserts what looks like a ^# instead of an actual newline. The file is not in DOS mode or anything. What should I do? If you are curious, like me, check the question Why is \r a newline for Vim? as well.
Use \r instead of \n. Substituting by \n inserts a null character into the text. To get a newline, use \r. When searching for a newline, you’d still use \n, however. This asymmetry is due to the fact that \n and \r do slightly different things: \n matches an end of line (newline), whereas \r matches a carriage return. On the other hand, in substitutions \n inserts a null character whereas \r inserts a newline (more precisely, it’s treated as the input CR). Here’s a small, non-interactive example to illustrate this, using the Vim command line feature (in other words, you can copy and paste the following into a terminal to run it). xxd shows a hexdump of the resulting file. echo bar > test (echo 'Before:'; xxd test) > output.txt vim test '+s/b/\n/' '+s/a/\r/' +wq (echo 'After:'; xxd test) >> output.txt more output.txt Before: 0000000: 6261 720a bar. After: 0000000: 000a 720a ..r. In other words, \n has inserted the byte 0x00 into the text; \r has inserted the byte 0x0a.
Here's the trick: First, set your Vi(m) session to allow pattern matching with special characters (i.e.: newline). It's probably worth putting this line in your .vimrc or .exrc file: :set magic Next, do: :s/,/,^M/g To get the ^M character, type Ctrl + V and hit Enter. Under Windows, do Ctrl + Q, Enter. The only way I can remember these is by remembering how little sense they make: A: What would be the worst control-character to use to represent a newline? B: Either q (because it usually means "Quit") or v because it would be so easy to type Ctrl + C by mistake and kill the editor. A: Make it so.
In the syntax s/foo/bar, \r and \n have different meanings, depending on context. Short: For foo: \r == "carriage return" (CR / ^M) \n == matches "line feed" (LF) on Linux/Mac, and CRLF on Windows For bar: \r == produces LF on Linux/Mac, CRLF on Windows \n == "null byte" (NUL / ^#) When editing files in linux (i.e. on a webserver) that were initially created in a windows environment and uploaded (i.e. FTP/SFTP) - all the ^M's you see in vim, are the CR's which linux does not translate as it uses only LF's to depict a line break. Longer (with ASCII numbers): NUL == 0x00 == 0 == Ctrl + # == ^# shown in vim LF == 0x0A == 10 == Ctrl + J CR == 0x0D == 13 == Ctrl + M == ^M shown in vim Here is a list of the ASCII control characters. Insert them in Vim via Ctrl + V,Ctrl + ---key---. In Bash or the other Unix/Linux shells, just type Ctrl + ---key---. Try Ctrl + M in Bash. It's the same as hitting Enter, as the shell realizes what is meant, even though Linux systems use line feeds for line delimiting. To insert literal's in bash, prepending them with Ctrl + V will also work. Try in Bash: echo ^[[33;1mcolored.^[[0mnot colored. This uses ANSI escape sequences. Insert the two ^['s via Ctrl + V, Esc. You might also try Ctrl + V,Ctrl + M, Enter, which will give you this: bash: $'\r': command not found Remember the \r from above? :> This ASCII control characters list is different from a complete ASCII symbol table, in that the control characters, which are inserted into a console/pseudoterminal/Vim via the Ctrl key (haha), can be found there. Whereas in C and most other languages, you usually use the octal codes to represent these 'characters'. If you really want to know where all this comes from: The TTY demystified. This is the best link you will come across about this topic, but beware: There be dragons. TL;DR Usually foo = \n, and bar = \r.
You need to use: :%s/,/^M/g To get the ^M character, press Ctrl + v followed by Enter.
\r can do the work here for you.
With Vim on Windows, use Ctrl + Q in place of Ctrl + V.
This is the best answer for the way I think, but it would have been nicer in a table: Why is \r a newline for Vim? So, rewording: You need to use \r to use a line feed (ASCII 0x0A, the Unix newline) in a regex replacement, but that is peculiar to the replacement - you should normally continue to expect to use \n for line feed and \r for carriage return. This is because Vim used \n in a replacement to mean the NIL character (ASCII 0x00). You might have expected NIL to have been \0 instead, freeing \n for its usual use for line feed, but \0 already has a meaning in regex replacements, so it was shifted to \n. Hence then going further to also shift the newline from \n to \r (which in a regex pattern is the carriage return character, ASCII 0x0D). Character | ASCII code | C representation | Regex match | Regex replacement -------------------------+------------+------------------+-------------+------------------------ nil | 0x00 | \0 | \0 | \n line feed (Unix newline) | 0x0a | \n | \n | \r carriage return | 0x0d | \r | \r | <unknown> NB: ^M (Ctrl + V Ctrl + M on Linux) inserts a newline when used in a regex replacement rather than a carriage return as others have advised (I just tried it). Also note that Vim will translate the line feed character when it saves to file based on its file format settings and that might confuse matters.
From Eclipse, the ^M characters can be embedded in a line, and you want to convert them to newlines. :s/\r/\r/g
But if one has to substitute, then the following thing works: :%s/\n/\r\|\-\r/g In the above, every next line is substituted with next line, and then |- and again a new line. This is used in wiki tables. If the text is as follows: line1 line2 line3 It is changed to line1 |- line2 |- line3
Here's the answer that worked for me. From this guy: ----quoting Use the vi editor to insert a newline char in replace Something else I have to do and cannot remember and then have to look up. In vi, to insert a newline character in a search and replace, do the following: :%s/look_for/replace_with^M/g The command above would replace all instances of “look_for” with “replace_with\n” (with \n meaning newline). To get the “^M”, enter the key combination Ctrl + V, and then after that (release all keys) press the Enter key.
If you need to do it for a whole file, it was also suggested to me that you could try from the command line: sed 's/\\n/\n/g' file > newfile
in vim editor the following command successfully replaced \n with new line :%s/\\n/\r/g