I copied a directory structure from a windows box to a Linux box and I would like to use sed to replace c:\IBM\WebSphere with /opt/IBM/WebSphere in all files under this directory.
Any thoughts?
I think sed is a little inconvenient for that purpose, if you want to change the actual files you can use perl one-liner
perl -pi -e 's/c:\\IBM\\/\/opt\/IBM\//g' *
Add or adjust the paths according to what you need (add WebSphere if you want the replacement to change only these dirs)
Since sed can take any character as a delimiter, use
sed -e 's_\\_/_g'
to replace the \ to /.
sed -e 's_[Cc]:_/opt_g'
to replace the c: with /opt
You can string those together:
echo "C:\\IBM\\WebSphere" | sed -e 's_\\_/_g' -e 's_[Cc]:_/opt_g'
Output:
/opt/IBM/WebSphere
I don't see an awk solution, just add one:
awk -F'\' -v OFS='/' '$1=/^c:/?"/opt":$1'
test:
kent$ awk -F'\' -v OFS='/' '$1=/^c:/?"/opt":$1' <<<'c:\IBM\WebSphere'
/opt/IBM/WebSphere
echo "C:\Users\San.Tan\My Folder\project1" | sed -e 's/C:\\/mnt\/c\//;s/\\/\//g'
replaces
C:\Users\San.Tan\My Folder\project1
to
mnt/c/Users/San.Tan/My Folder/project1
in case someone needs to replace windows paths to Windows Subsystem for Linux(WSL) paths
Related
I've 95 files that looks like :
2019-10-29-18-00/dev/xx;512.00;0.4;/var/x/xx/xxx
2019-10-29-18-00/dev/xx;512.00;0.68;/xx
2019-10-29-18-00/dev/xx;512.00;1.84;/xx/xx/xx
2019-10-29-18-00/dev/xx;512.00;80.08;/opt/xx/x
2019-10-29-18-00/dev/xx;20480.00;83.44;/var/x/x
2019-10-29-18-00/dev/xx;3584.00;840.43;/var/xx/x
2019-10-30-00-00/dev/xx;2048.00;411.59;/
2019-10-30-00-00/dev/xx;7168.00;6168.09;/usr
2019-10-30-00-00/dev/xx;3072.00;1036.1;/var
2019-10-30-00-00/dev/xx;5120.00;348.72;/tmp
2019-10-30-00-00/dev/xx;20480.00;2033.19;/home
2019-10-30-12-00;/dev/xx;5120.00;348.72;/tmp
2019-10-30-12-00;/dev/hd1;20480.00;2037.62;/home
2019-10-30-12-00;/dev/xx;512.00;0.43;/xx
2019-10-30-12-00;/dev/xx;3584.00;794.39;/xx
2019-10-30-12-00;/dev/xx;512.00;0.4;/var/xx/xx/xx
2019-10-30-12-00;/dev/xx;512.00;0.68;/xx
2019-10-30-12-00;/dev/xx;512.00;1.84;/var/xx/xx
2019-10-30-12-00;/dev/xx;512.00;80.08;/opt/xx/x
2019-10-30-12-00;/dev/xx;20480.00;83.44;/var/xx/xx
2019-10-30-12-00;/dev/x;3584.00;840.43;/var/xx/xx
For some lines I've 2019-10-29-18-00/dev and for some other lines, I've 2019-10-30-12-00;/dev/
I want to add the ; before the /dev/ where it is missing, so for that I use this sed command :
sed 's/\/dev/\;\/dev/'
But How I can apply this command for each lines where the ; is missing ? I try this :
for i in $(cat /home/xxx/xxx/xxx/*.txt | grep -e "00/dev/")
do
sed 's/\/dev/\;\/dev/' $i > $i
done
But it doesn't work... Can you help me ?
Could you please try following with GNU awkif you are ok with it.
awk -i inplace '/00\/dev\//{gsub(/00\/dev\//,"/00;/dev/")} 1' *.txt
sed solution: Tested with GNU sed for few files and it worked fine.
sed -i.bak '/00\/dev/s/00\/dev/00\;\/dev/g' *.txt
This might work for you (GNU sed & parallel):
parallel -q sed -i 's#;*/dev#;/dev#' ::: *.txt
or if you prefer:
sed -i 's#;*/dev#;/dev#' *.txt
Ignore lines with ;/dev.
sed '/;\/dev/{p;d}; s^/dev^;/dev^'
The /;\/dev/ check if the line has ;/dev. If it has ;/dev do: p - print the current line and d - start from the beginning.
You can use any character with s command in sed. Also, there is no need in escaping \;, just ;.
How I can apply this command for each lines where the ; is missing ? I try this
Don't edit the same file redirecting to the same file $i > $i. Think about it. How can you re-write and read from the same file at the same time? You can't, the resulting file will be in most cases empty, as the > $i will "execute" first making the file empty, then sed $i will start running and it will read an empty file. Use a temporary file sed ... "$i" > temp.txt; mv temp.txt "$i" or use gnu extension -i sed option to edit in place.
What you want to do really is:
grep -l '00/dev/' /home/xxx/xxx/xxx/*.txt |
xargs -n1 sed -i '/;\/dev/{p;d}; s^/dev^;/dev^'
grep -l prints list of files that match the pattern, then xargs for each single one -n1 of the files executes sed which -i edits files in place.
grep for filtering can be eliminated in your case, we can accomplish the task with a single sed command:
for f in $(cat /home/xxx/xxx/xxx/*.txt)
do
[[ -f "$f" ]] && sed -Ei '/00\/dev/ s/([^;])(\/dev)/\1;\2/' "$f"
done
The easiest way would be to adjust your regex so that it's looking a bit wider than '/dev/', e.g.
sed -i -E 's|([0-9])/dev|\1;/dev|'
(note that I'm taking advantage of sed's flexible approach to delimiters on substitute. Also, -E changes the group syntax)
Alternatively, sed lets you filter which lines it handles:
sed -i '/[0-9]\/dev/ s/\/dev/;/dev/'
This uses the same substitution you already have but only applied on lines that match the filter regex
Is there an invocation of sed todo in-place editing without backups that works both on Linux and Mac? While the BSD sed shipped with OS X seems to need sed -i '' …, the GNU sed Linux distributions usually come with interprets the quotes as empty input file name (instead of the backup extension), and needs sed -i … instead.
Is there any command line syntax which works with both flavors, so I can use the same script on both systems?
If you really want to just use sed -i the 'easy' way, the following DOES work on both GNU and BSD/Mac sed:
sed -i.bak 's/foo/bar/' filename
Note the lack of space and the dot.
Proof:
# GNU sed
% sed --version | head -1
GNU sed version 4.2.1
% echo 'foo' > file
% sed -i.bak 's/foo/bar/' ./file
% ls
file file.bak
% cat ./file
bar
# BSD sed
% sed --version 2>&1 | head -1
sed: illegal option -- -
% echo 'foo' > file
% sed -i.bak 's/foo/bar/' ./file
% ls
file file.bak
% cat ./file
bar
Obviously you could then just delete the .bak files.
This works with GNU sed, but not on OS X:
sed -i -e 's/foo/bar/' target.file
sed -i'' -e 's/foo/bar/' target.file
This works on OS X, but not with GNU sed:
sed -i '' -e 's/foo/bar/' target.file
On OS X you
can't use sed -i -e since the extension of the backup file would be set to -e
can't use sed -i'' -e for the same reasons—it needs a space between -i and ''.
When on OSX, I always install GNU sed version via Homebrew, to avoid problems in scripts, because most scripts were written for GNU sed versions.
brew install gnu-sed --with-default-names
Then your BSD sed will be replaced by GNU sed.
Alternatively, you can install without default-names, but then:
Change your PATH as instructed after installing gnu-sed
Do check in your scripts to chose between gsed or sed depending on your system
As Noufal Ibrahim asks, why can't you use Perl? Any Mac will have Perl, and there are very few Linux or BSD distributions that don't include some version of Perl in the base system. One of the only environments that might actually lack Perl would be BusyBox (which works like GNU/Linux for -i, except that no backup extension can be specified).
As ismail recommends,
Since perl is available everywhere I just do perl -pi -e s,foo,bar,g target.file
and this seems like a better solution in almost any case than scripts, aliases, or other workarounds to deal with the fundamental incompatibility of sed -i between GNU/Linux and BSD/Mac.
Answer: No.
The originally accepted answer actually doesn't do what is requested (as noted in the comments). (I found this answer when looking for the reason a file-e was appearing "randomly" in my directories.)
There is apparently no way of getting sed -i to work consistently on both MacOS and Linuces.
My recommendation, for what it is worth, is not to update-in-place with sed (which has complex failure modes), but to generate new files and rename them afterwards. In other words: avoid -i.
There is no way to have it working.
One way is to use a temporary file like:
TMP_FILE=`mktemp /tmp/config.XXXXXXXXXX`
sed -e "s/abc/def/" some/file > $TMP_FILE
mv $TMP_FILE some/file
This works on both
Here's another version that works on Linux and macOS without using eval and without having to delete backup files. It uses Bash arrays for storing the sed parameters, which is cleaner than using eval:
# Default case for Linux sed, just use "-i"
sedi=(-i)
case "$(uname)" in
# For macOS, use two parameters
Darwin*) sedi=(-i "")
esac
# Expand the parameters in the actual call to "sed"
sed "${sedi[#]}" -e 's/foo/bar/' target.file
This does not create a backup file, neither a file with appended quotes.
The -i option is not part of POSIX Sed. A more portable method would be
to use Vim in Ex mode:
ex -sc '%s/alfa/bravo/|x' file
% select all lines
s replace
x save and close
Steve Powell's answer is quite correct, consulting the MAN page for sed on OSX and Linux (Ubuntu 12.04) highlights the in-compatibility within 'in-place' sed usage across the two operating systems.
JFYI, there should be no space between the -i and any quotes (which denote an empty file extension) using the Linux version of sed, thus
sed Linux Man Page
#Linux
sed -i""
and
sed OSX Man page
#OSX (notice the space after the '-i' argument)
sed -i ""
I got round this in a script by using an alias'd command and the OS-name output of 'uname' within a bash 'if'. Trying to store OS-dependant command strings in variables was hit and miss when interpreting the quotes. The use of 'shopt -s expand_aliases' is necessary in order to expand/use the aliases defined within your script. shopt's usage is dealt with here.
Portable script for both GNU systems and OSX:
if [[ $(uname) == "Darwin" ]]; then
SP=" " # Needed for portability with sed
fi
sed -i${SP}'' -e "s/foo/bar/g" -e "s/ping/pong/g" foobar.txt
I ran into this problem. The only quick solution was to replace the sed in mac to the gnu version:
brew install gnu-sed
If you need to do sed in-place in a bash script, and you do NOT want the in-place to result with .bkp files, and you have a way to detect the os (say, using ostype.sh), -- then the following hack with the bash shell built-in eval should work:
OSTYPE="$(bash ostype.sh)"
cat > myfile.txt <<"EOF"
1111
2222
EOF
if [ "$OSTYPE" == "osx" ]; then
ISED='-i ""'
else # $OSTYPE == linux64
ISED='-i""'
fi
eval sed $ISED 's/2222/bbbb/g' myfile.txt
ls
# GNU and OSX: still only myfile.txt there
cat myfile.txt
# GNU and OSX: both print:
# 1111
# bbbb
# NOTE:
# if you just use `sed $ISED 's/2222/bbbb/g' myfile.txt` without `eval`,
# then you will get a backup file with quotations in the file name,
# - that is, `myfile.txt""`
The problem is that sed is a stream editor, therefore in-place editing is a non-POSIX extension and everybody may implement it differently. That means for in-place editing you should use ed for best portability. E.g.
ed -s foobar.txt <<<$',s/foo/bar/g\nw'
Also see https://wiki.bash-hackers.org/howto/edit-ed.
You can use sponge. Sponge is an old unix program, found in moreutils package (both in ubuntu and probably debian, and in homebrew in mac).
It will buffer all the content from the pipe, wait until the pipe is close (probably meaning that the input file is already close) and then overwrite:
From the man page:
Synopsis
sed '...' file | grep '...' | sponge file
The following works for me on Linux and OS X:
sed -i' ' <expr> <file>
e.g. for a file f containing aaabbaaba
sed -i' ' 's/b/c/g' f
yields aaaccaaca on both Linux and Mac. Note there is a quoted string containing a space, with no space between the -i and the string. Single or double quotes both work.
On Linux I am using bash version 4.3.11 under Ubuntu 14.04.4 and on the Mac version 3.2.57 under OS X 10.11.4 El Capitan (Darwin 15.4.0).
So example, I have the file my.txt as below
This is my xzy.
My color is white.
I want to replace "my" to our, and I also want to replace "xyz" to "abc".
How can I do this using a one line command?
Use sed with multiple replacement:
sed -i 's/my/our/g; s/xyz/abc/g' text.txt
With sed
For GNU sed
sed -i -e 's/my/our/g' -e 's/xyz/abc/g' my.txt
In MacOS, -e option may not be available. -i option in MacOS needs an extension. Use something like this:
sed -i.bak 's/my/our/g;s/xyz/abc/g' my.txt
I need to replace first 4 header lines of only selected 250 erlang files (with extension .erl), but there are 400 erlang files in total in the directory+subdirectories, I need to avoid modifying the files which doesn't need the change.
I've the list of file names that are to be modified, but don't know how to make my linux command to make use of them.
sed -i '1s#.*#%% This Source Code Form is subject to the terms of the Mozilla Public#' *.erl
sed -i '2s#.*#%% License, v. 2.0. If a copy of the MPL was not distributed with this file,#' *.erl
sed -i '3s#.*#%% You can obtain one at http://mozilla.org/MPL/2.0/.#' *.erl
sed -i '4s#.*##' *.erl
in the above commands instead of passing *.erl I want to pass those list of file names which I need to modify, doing that one by one will take me more than 3 days to complete it.
Is there any way to do this?
Iterate over the shortlisted file names using awk and use xargs to execute the sed. You can execute multiple sed commands to a file using -e option.
awk '{print $1}' your_shortlisted_file_lists | xargs sed -i -e first_sed -e second_sed $1
xargs gets the file name from awk in a $1 variable.
Try this:
< file_list.txt xargs -1 sed -i -e 'first_cmd' -e 'second_cmd' ...
Not answering your question but a suggestion for improvement. Four sed commands for replacing header is inefficient. I would instead write the new header into a file and do the following
sed -i -e '1,3d' -e '4{r header' -e 'd}' file
will replace the first four lines of the file with header.
Another concern with your current s### approach is you have to watch for special chars \, & and your delimiter # in the text you are replacing.
You can apply the sed c (for change) command to each file of your list :
while read file; do
sed -i '1,4 c\
%% This Source Code Form is subject to the terms of the Mozilla Public\
%% License, v. 2.0. If a copy of the MPL was not distributed with this file,\
%% You can obtain one at http://mozilla.org/MPL/2.0/.\
' "$file"
done < filelist
Let's say you have a file called file_list.txt with all file names as content:
file1.txt
file2.txt
file3.txt
file4.txt
You can simply read all lines into a variable (here: files) and then iterate through each one:
files=`cat file_list.txt`
for file in $files; do
echo "do something with $file"
done
Is it possible to search in a file using shell and then replace a value? When I install a service I would like to be able to search out a variable in a config file and then replace/insert my own settings in that value.
Sure, you can do this using sed or awk. sed example:
sed -i 's/Andrew/James/g' /home/oleksandr/names.txt
You can use sed to perform search/replace. I usually do this from a bash shell script, and move the original file containing values to be substituted to a new name, and run sed writing the output to my original file name like this:
#!/bin/bash
mv myfile.txt myfile.txt.in
sed -e 's/PatternToBeReplaced/Replacement/g' myfile.txt.in > myfile.txt.
If you don't specify an output, the replacement will go to stdout.
sed -i 's/variable/replacement/g' *.conf
You can use sed to do this:
sed -i 's/toreplace/yoursetting/' configfile
sed is probably available on every unix like system out there. If you want to replace more than one occurence you can add a g to the s-command:
sed -i 's/toreplace/yoursetting/g' configfile
Be careful since this can completely destroy your configfile if you don't specify your toreplace-value correctly. sed also supports regular expressions in searching and replacing.
Look at the UNIX power tools awk, sed, grep and in-place edit of files with Perl.
filepath="/var/start/system/dir1"
searchstring="test"
replacestring="test01"
i=0;
for file in $(grep -l -R $searchstring $filepath)
do
cp $file $file.bak
sed -e "s/$searchstring/$replacestring/ig" $file > tempfile.tmp
mv tempfile.tmp $file
let i++;
echo "Modified: " $file
done
Generally a tool like awk or sed are used for this.
$ sed -i 's/ugly/beautiful/g' /home/bruno/old-friends/sue.txt