In order to add a licence to all php files, I would like to change <?php by:
<?php
/*
* Copyright (c) 2016-2017 My Project
* This file is part of My Project 1.2, more information at https://websitemyproject.com
* Licence another line
* Last line of licence
*/
So I tested with sed on one file, for the first line of the licence:
sed -i -- 's/<?php/<?php\n\/*\n * Copyright(c) 2016-2017 My Project \n *\/ /g' index.php
It works but it adds some ^M at the end of each line of the file index.php and I don't understand why.
<?php
/*
* Copyright(c) 2016-2017 My Project
*/ ^M
^M
^M
$page = 'home';^M
^M
Thank you for your help
you can insert s/\r//; to delete a carriage return chars with your sed command
sed -i -- 's/\r//;s/<?php/<?php\n\/*\n * Copyright(c) 2016-2017 My Project \n *\/ /g' index.php
Your file has Windows line endings (^M^J), your sed command is outputting Unix line endings (^J), and the result is a mixture of both.
In other words, it might look like sed is adding ^M to each line, but actually those ^M characters were already there. Your editor probably auto-detects the line-endings, and the mixed file is producing mixed results.
You can convert the line endings using a tool like dos2unix, or you can tell your sed command to output Windows style line endings by using \r\n instead of just \n.
Related
I am new to linux (not my own server) and I want to split some windows txt files by calling a bash script from a third party application:
So far I have it working in two ways up to a point:
split -l 5000 LargeFile.txt SmallFile
for file in LargeFile.*
do
mv "$file" "$file.txt"
done
awk '{filename = "wrd." int((NR-1)/5000) ".txt"; print >> filename}' LargeFile.txt
But both give me txt files with the result:
line1line2line3line4
I found some topics about putting LargeFile.txt like this $ (LargeFile.txt) but it is not working for me. (Also I found a swich to let the split command produce txt files directly, but this is also not working)
I hope some one can help me out on this one.
Explanation: Line terminators
As explained by various answers to this question, the standard line terminators differ between OS's:
Linux uses LF (line feed, 0x0a)
Windows uses CRLF (carriage return and line feed 0x0d 0x0a)
Mac, pre OS X used CR (carriage return CR)
To solve your problem, it would be important to figure out what line terminators your LargeFile.txt uses. The simplest way would be the file command:
file LargeFile.txt
The output will indicate if line terminators are CR or CRLF and otherwise just state that it is an ASCII file.
Since LF and CRLF line terminators will be recognized properly in Linux and lines should not appear merged together (no matter which way you use to view the file) unless you configure an editor specifically so that they do, I will assume that your file has CR line terminators.
Example solution to your problem (assuming CR line terminators)
If you want to split the file in the shell and with shell commands, you will potentially face the problem that the likes of cat, split, awk, etc will not recognize line endings in the first place. If your file is very large, this may additionally lead to memory issues (?).
Therefore, the best way to handle this may be to translate the line terminators first (using the tr command) so that they are understood in Linux (i.e. to LF) and then apply your split or awk code before translating the line terminators back (if you believe you need to do this).
cat LargeFile.txt | tr "\r" "\n" > temporary_file.txt
split -l 5000 temporary_file.txt SmallFile
rm temporary_file.txt
for file in `ls SmallFile*`; do filex=$file.txt; cat $file | tr "\n" "\r" > $filex; rm $file; done
Note that the last line is actually a for loop:
for file in `ls SmallFile*`
do
filex=$file.txt
cat $file | tr "\n" "\r" > $filex
rm $file
done
This loop will again use tr to restore the CR line terminators and additionally give the resulting files a txt filename ending.
Some Remarks
Of course, if you would like to keep the LF line terminators you should not execute this line.
And finally, if you find that you have a different type of line terminators, you may need to adapt the tr command in the first line.
Both tr and split (and also cat and rm) are part of GNU coreutils and should be installed on your system unless you are in a very untypical environment (a rescue shell of an initial RAM disk perhaps). The same (should typically be available) goes for the file command, this one.
I'm trying to find a line in a file and replace the next line with a specific value. I tried sed, but it seems to not like the \n. How else can this be done?
The file looks like this:
<key>ConnectionString</key>
<string>anything_could_be_here</string>
And I'd like to change it to this
<key>ConnectionString</key>
<string>changed_value</string>
Here's what I tried:
sed -i '' "s/<key>ConnectionString<\/key>\n<string><\/string>/<key>ConnectionString<\/key>\n<string>replaced_text<\/string>/g" /path/to/file
One way:
Sample file
$ cat file
Cygwin
Unix
Linux
Solaris
AIX
Using sed, replacing the next line after the pattern 'Unix' with 'hi':
$ sed '/Unix/{n;s/.*/hi/}' file
Cygwin
Unix
hi
Solaris
AIX
For your specific question:
$ sed '/<key>ConnectionString<\/key>/{n;s/<string>.*<\/string>/<string>NEW STRING<\/string>/}' your_file
<key>ConnectionString</key>
<string>NEW STRING</string>
This might work for you (GNU sed):
sed '/<key>ConnectionString<\/key>/!b;n;c<string>changed_value</string>' file
!b negates the previous address (regexp) and breaks out of any processing, ending the sed commands, n prints the current line and then reads the next into the pattern space, c changes the current line to the string following the command.
It works. Additionaly is interested to mention that if you write,
sed '/<key>ConnectionString<\/key>/!b;n;n;c<string>changed_value</string>' file
Note the two n's, it replaces after two lines and so forth.
This question already has answers here:
Appending a line to a file only if it does not already exist
(25 answers)
Closed 1 year ago.
I need to edit several lines in a file such that
if a line begins with (av or avpgw) then replace these with new text,
else just insert the new text in beginning.
How can I do this using sed ?
You can do it this way:
sed -e 's/^avpgw/new text/' -e t -e 's/^av/new text/' -e t -e 's/^/new text/' file
This replaces the pattern with new text (s///) and jumps to the end (t). Otherwise it tries the next pattern. See also sed commands summary.
You can also separate the commands with ;:
sed 's/^avpgw/new text/; t; s/^av/new text/; t; s/^/new text/' file
or put the commands in a sed command file:
s/^avpgw/new text/
t
s/^av/new text/
t
s/^/new text/
and call it this way:
sed -f commandfile file
If you want to ignore case, append an i at the end of the substitute command as in s/^av/new text/i. Another way is to spell it out with character sets s/^[aA][vV]/new text/. But that is not sed specific, for this you can search for regular expressions in general.
This might work for you (GNU sed):
sed -r 's/^av(pgw)?.*/replacement/;t;s/^/replacement /' file
My answer is but a minor clarification to Olaf's excellent solution above.
If you are using OSX (which for me, using Catalina, uses a 2005 non-GNU version of sed), and you try to use his more-concise semicolon-based solution (repeated below, tiny modification with the 1/2/3's at the end):
sed 's/^avpgw/new text 1/; t; s/^av/new text 2/; t; s/^/new text 3/' file
... you may get an error similar to the below, due to the use of the t command in combination with the semicolons [;] (which are themselves separated from the rest of the command by regular spaces):
sed: 1: "s/^avpgw/new text 1/; t ...": undefined label '; s/^av/new text 2/; t; s/^/new text 3/'
This problem, and a solution, is described in: sed “undefined label” on MacOS
Below is a version of Olaf's "all-in-one command" solution that works on OSX. It's debatable whether the \n characters are really any better than just using extra -es. lolz aside: I'm mainly posting because I wanted to share the "root cause" of this problem so that people using OS X won't drive themselves crazy thinking that their sed version doesn't properly support the t command robustly -- it's just an issue of semicolons versus newlines, that's all!
Test file:
bash-3.2$ cat file
+ cat file
something
avpgw something else
av another thing
text
Original command, executed on OS X:
bash-3.2$ sed 's/^avpgw/new text 1/; t; s/^av/new text 2/; t; s/^/new text 3/' file
sed: 1: "s/^avpgw/new text 1/; t ...": undefined label '; s/^av/new text 2/; t; s/^/new text 3/'
With newlines instead of semicolons:
bash-3.2$ sed $'s/^avpgw/new text 1/\nt\ns/^av/new text 2/\nt\ns/^/new text 3/' file
new text 3something
new text 1 something else
new text 2 another thing
new text 3text
I have a csv file into which has crept some ^M dos line ends, and I want to get rid of them, as well as 16 spaces and 3 tabs which follow. Like, I have to merge that line with the next one down. Heres an offending record and a good one as a sample of what I mean:
"Mary had a ^M
little lamb", "Nursery Rhyme", 1878
"Mary, Mary quite contrary", "Nursery Rhyme", 1838
I can remove the ^M using sed as you can see, but I cannot work out how to rm the nix line end to join the lines back up.
sed -e "s/^M$ //g" rhymes.csv > rhymes.csv
UPDATE
Then I read "However, the Microsoft CSV format allows embedded newlines within a double-quoted field. If embedded newlines within fields are a possibility for your data, you should consider using something other than sed to work with the data file." from:
http://sed.sourceforge.net/sedfaq4.html
So editing my question to ask Which tool I should be using?
With help from How can I replace a newline (\n) using sed?, I made this one:
sed -e ':a;N;$!ba;s/\r\n \t\t\t/=/' -i rhymes.csv
<CR> <LF> <16 spaces> <3 tabs>
If you just want to delete the CR, you could use:
<yourfile tr -d "\r" | tee yourfile
(or if the two input and output file are different: <yourfile tr -d "\r" > output)
dos2unix file_name
to convert file, or
dos2unix old_file new_file
to create new file.
Appending can be done using tee command.
cat file | tee -a >> *
Is there a way to do a prepend/insertion?
Thanks.
Using sed might help
example:
sed -i.bak '3 r tmp1.txt' settings.xml
will add the contents of tmp1.txt after line 3 in settings.xml (and create a backup file with the .bak extension)
Just a brief example: say, comment out specific/particular/arbitrary C lines:
$ echo -e "1\n2\n3\n4\n5\n6\n" | sed "3s,^,/* ,;5s,$, */,"
1
2
/* 3
4
5 */
6
Note:
single sed command follows a format "${linenum}s/${search}/${replace}/"
then two commands can be delimited by a semicolon ';'
using a comma ',' as delimiter, for easier reading of s///
Caret '^' matches start of line; dollar '$' matches end of line; the s/// will replace only those (meta?)"characters"
Of course, then this should be modified with the -i switch to sed, to eventually replace file contents..
Cheers!
EDIT: Refs:
Can we give multiple patterns to a sed command??? - The UNIX and Linux Forums
Cute ‘sed’ Tricks to Modify Specific Lines Within File | The Linux Daily