I need to change a directive in a config file and got it working in Linux but in Solaris, it says command garbled.
Here is the directive
enable-cache passwd yes
I need to simply change the yes to no. How can I do with with sed that will work for Solaris, HPUX and Linux?
Here is the sed command that worked in Linux. Solaris doesn't like the -r
sed -r 's/^([[:space:]]*check-files[[:space:]]+passwd[[:space:]]+)yes([[:space:]]*)$/\1no\2/' inputfile
The end goal is to put this command in a script and run it across the enterprise.
Thanks
Greg
I also posted something similar yesterday which worked for Linux but not for the others.
Solaris has /usr/bin/sed and /usr/xpg4/bin/sed. None of these support an -r option, which option for Linux is to use an extended regex. sed in Solaris does not have any option to set the regex like that. You can use other tools, specifically awk, if you want simpler portability. Or you will have to use two flavors of regex, one with -r and an extended regex, one without -r and a different regex. And you probably want to specify /usr/xpg4/bin/sed on Solaris boxes only:
#!/bin/bash
sun=`expr index Solaris $(uname -a)`
if [ $sun -ne 0 ] ; then
/usr/xpg4/bin/sed [longer regex here ]
else
/usr/bin/sed -r [ extended regex here ]
fi
This is not strictly equivalent as :space: match more characters but I assume only space and tab are to be expected in your file. I'm only using standard shell and sed commands so this should work on Solaris, Linux and HP-UX:
space=$(printf " \t")
sed 's/^\(['"$space"']*check-files['"$space"']+passwd['"$space"']+\)yes\(['"$space"']*\)$/\1no\2/' inputfile
Note that your script doesn't match your sample directive as it expects check-files but is given enable-cache.
The GNU [[:space:]] is usually equivalent to just [ \t]. And you need to escape the parentheses. And + is not supported. So with these replacements, your working sed command becomes:
sed 's/^\([ \t]*check-files[ \t][ \t]*passwd[ \t][ \t]*\)yes\([ \t]*\)$/\1no\2/' inputfile
Further note: The older sed's don't have a -i option for doing in-place changes, so you might first have to copy your target to a temporary file, and apply sed to it, redirecting the output to the target.
Related
What's the simplest way to do a find and replace for a given input string, say abc, and replace with another string, say XYZ in file /tmp/file.txt?
I am writting an app and using IronPython to execute commands through SSH — but I don't know Unix that well and don't know what to look for.
I have heard that Bash, apart from being a command line interface, can be a very powerful scripting language. So, if this is true, I assume you can perform actions like these.
Can I do it with bash, and what's the simplest (one line) script to achieve my goal?
The easiest way is to use sed (or perl):
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while IFS='' read -r a; do
echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)
For Mac users:
sed -i '' 's/abc/XYZ/g' /tmp/file.txt
(See the comment below why)
File manipulation isn't normally done by Bash, but by programs invoked by Bash, e.g.:
perl -pi -e 's/abc/XYZ/g' /tmp/file.txt
The -i flag tells it to do an in-place replacement.
See man perlrun for more details, including how to take a backup of the original file.
I was surprised when I stumbled over this...
There is a replace command which ships with the "mysql-server" package, so if you have installed it try it out:
# replace string abc to XYZ in files
replace "abc" "XYZ" -- file.txt file2.txt file3.txt
# or pipe an echo to replace
echo "abcdef" |replace "abc" "XYZ"
See man replace for more on this.
This is an old post but for anyone wanting to use variables as #centurian said the single quotes mean nothing will be expanded.
A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work:
sed -i -e "s/$var1/$var2/g" /tmp/file.txt
Bash, like other shells, is just a tool for coordinating other commands. Typically you would try to use standard UNIX commands, but you can of course use Bash to invoke anything, including your own compiled programs, other shell scripts, Python and Perl scripts etc.
In this case, there are a couple of ways to do it.
If you want to read a file, and write it to another file, doing search/replace as you go, use sed:
sed 's/abc/XYZ/g' <infile >outfile
If you want to edit the file in place (as if opening the file in an editor, editing it, then saving it) supply instructions to the line editor 'ex'
echo "%s/abc/XYZ/g
w
q
" | ex file
Example is like vi without the fullscreen mode. You can give it the same commands you would at vi's : prompt.
I found this thread among others and I agree it contains the most complete answers so I'm adding mine too:
sed and ed are so useful...by hand.
Look at this code from #Johnny:
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
When my restriction is to use it in a shell script, no variable can be used inside in place of "abc" or "XYZ". The BashFAQ seems to agree with what I understand at least. So, I can't use:
x='abc'
y='XYZ'
sed -i -e 's/$x/$y/g' /tmp/file.txt
#or,
sed -i -e "s/$x/$y/g" /tmp/file.txt
but, what can we do? As, #Johnny said use a while read... but, unfortunately that's not the end of the story. The following worked well with me:
#edit user's virtual domain
result=
#if nullglob is set then, unset it temporarily
is_nullglob=$( shopt -s | egrep -i '*nullglob' )
if [[ is_nullglob ]]; then
shopt -u nullglob
fi
while IFS= read -r line; do
line="${line//'<servername>'/$server}"
line="${line//'<serveralias>'/$alias}"
line="${line//'<user>'/$user}"
line="${line//'<group>'/$group}"
result="$result""$line"'\n'
done < $tmp
echo -e $result > $tmp
#if nullglob was set then, re-enable it
if [[ is_nullglob ]]; then
shopt -s nullglob
fi
#move user's virtual domain to Apache 2 domain directory
......
As one can see if nullglob is set then, it behaves strangely when there is a string containing a * as in:
<VirtualHost *:80>
ServerName www.example.com
which becomes
<VirtualHost ServerName www.example.com
there is no ending angle bracket and Apache2 can't even load.
This kind of parsing should be slower than one-hit search and replace but, as you already saw, there are four variables for four different search patterns working out of one parse cycle.
The most suitable solution I can think of with the given assumptions of the problem.
You can use sed:
sed -i 's/abc/XYZ/gi' /tmp/file.txt
You can use find and sed if you don't know your filename:
find ./ -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Find and replace in all Python files:
find ./ -iname "*.py" -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Be careful if you replace URLs with "/" character.
An example of how to do it:
sed -i "s%http://domain.com%http://www.domain.com/folder/%g" "test.txt"
Extracted from: http://www.sysadmit.com/2015/07/linux-reemplazar-texto-en-archivos-con-sed.html
If the file you are working on is not so big, and temporarily storing it in a variable is no problem, then you can use Bash string substitution on the whole file at once - there's no need to go over it line by line:
file_contents=$(</tmp/file.txt)
echo "${file_contents//abc/XYZ}" > /tmp/file.txt
The whole file contents will be treated as one long string, including linebreaks.
XYZ can be a variable eg $replacement, and one advantage of not using sed here is that you need not be concerned that the search or replace string might contain the sed pattern delimiter character (usually, but not necessarily, /). A disadvantage is not being able to use regular expressions or any of sed's more sophisticated operations.
You may also use the ed command to do in-file search and replace:
# delete all lines matching foobar
ed -s test.txt <<< $'g/foobar/d\nw'
See more in "Editing files via scripts with ed".
To edit text in the file non-interactively, you need in-place text editor such as vim.
Here is simple example how to use it from the command line:
vim -esnc '%s/foo/bar/g|:wq' file.txt
This is equivalent to #slim answer of ex editor which is basically the same thing.
Here are few ex practical examples.
Replacing text foo with bar in the file:
ex -s +%s/foo/bar/ge -cwq file.txt
Removing trailing whitespaces for multiple files:
ex +'bufdo!%s/\s\+$//e' -cxa *.txt
Troubleshooting (when terminal is stuck):
Add -V1 param to show verbose messages.
Force quit by: -cwq!.
See also:
How to edit files non-interactively (e.g. in pipeline)? at Vi SE
Try the following shell command:
find ./ -type f -name "file*.txt" | xargs sed -i -e 's/abc/xyz/g'
You can use python within the bash script too. I didn't have much success with some of the top answers here, and found this to work without the need for loops:
#!/bin/bash
python
filetosearch = '/home/ubuntu/ip_table.txt'
texttoreplace = 'tcp443'
texttoinsert = 'udp1194'
s = open(filetosearch).read()
s = s.replace(texttoreplace, texttoinsert)
f = open(filetosearch, 'w')
f.write(s)
f.close()
quit()
Simplest way to replace multiple text in a file using sed command
Command -
sed -i 's#a/b/c#D/E#g;s#/x/y/z#D:/X#g;' filename
In the above command s#a/b/c#D/E#g where I am replacing a/b/c with D/E and then after the ; we again doing the same thing
You can use rpl command. For example you want to change domain name in whole php project.
rpl -ivRpd -x'.php' 'old.domain.name' 'new.domain.name' ./path_to_your_project_folder/
This is not clear bash of cause, but it's a very quick and usefull. :)
For MAC users in case you don't read the comments :)
As mentioned by #Austin, if you get the Invalid command code error
For the in-place replacements BSD sed requires a file extension after the -i flag to save to a backup file with given extension.
sed -i '.bak' 's/find/replace' /file.txt
You can use '' empty string if you want to skip backup.
sed -i '' 's/find/replace' /file.txt
All merit to #Austin
Open file using vim editor. In command mode
:%s/abc/xyz/g
This is the simplest
In case of doing changes in multiple files together we can do in a single line as:-
user_name='whoami'
for file in file1.txt file2.txt file3.txt; do sed -i -e 's/default_user/${user_name}/g' $file; done
Added if in case could be useful.
I am using a bison parser in my project. When I run the following command:
sed -i y.tab.c -e "s/ __attribute__ ((__unused__))$/# ifndef __cplusplus\n __attribute__ ((__unused__));\n# endif/"
I get this error
sed: -i may not be used with stdin
The command works fine in linux machines. I am using Mac OS X 10.9. It throws an error only on mac os x. I am not sure why. Can anyone help?
Thanks
The problem is that Mac OS X uses the BSD version of sed, which treats the -i option slightly differently. The GNU version used in Linux takes an optional argument with -i: if present, sed makes a backup file whose name consists of the input file plus the argument. Without an argument, sed simply modifies the input file without saving a backup of the original.
In BSD sed, the argument to -i is required. To avoid making a backup, you need to provide a zero-length argument, e.g. sed -i '' y.tab.c ....
Your command, which simply edits y.tab.c with no backup in Linux, would attempt to save a backup file using 'y.tab.c' as an extension. But now, with no other file in the command line, sed thinks you want to edit standard input in-place, something that is not allowed.
From the sed manpage:
-i extension
Edit files in-place, saving backups with the specified extension.
If a zero-length extension is given, no backup will be saved. It
is not recommended to give a zero-length extension when in-place
editing files, as you risk corruption or partial content in situ-
ations where disk space is exhausted, etc.
The solution is to send a zero-length extension like this:
sed -i '' 's/apples/oranges/' file.txt
You need to put the input file as the last parameter.
sed -i -e "s/ __attribute__ ((__unused__))$/# ifndef __cplusplus\n __attribute__ ((__unused__));\n# endif/" y.tab.c
Piggy-backing off of #chepner's explanation for a quick-and-dirty solution:
Install the version of sed that'll get the job done with brew install gnu-sed, then replace usages of sed in your script with gsed.
(The homebrew community is fairly cognizant of issues that can arise of OS X built-ins are overridden unexpectedly and has worked to not do that for most alternate-distro commands.)
Hi using following command on an x86 machine (using /bin/sh) returns: <port>3<port>
test="port 3"
echo $test | sed -r 's/\s*port\s*([0-9]+)\s*/<port>\1<\/port>/'
but running same command on sh shell of an ARM based network switch returns the string port 3.
How can I get same result on switch as I got on my x86 machine? To me it seems like digit is not being captured by [0-9].
\s is a GNU sed extension to the standard sed behavior. GNU sed is the implementation on desktop/server Linux systems. Most embedded Linux systems run BusyBox, a suite of utilities with a markedly smaller footprint and fewer features.
A standard way of specifying “any space character” is the [:space:] character class. It is supported by BusyBox (at least, by most BusyBox installations; most BusyBox features can be stripped off for an even lower footprint).
BusyBox also doesn't support the -r option, you need to use a basic regular expression. In a BRE, \(…\) marks groups, and there is no + operator, only *.
echo "$test" | sed 's/[[:space:]]*port[[:space:]]*\([0-9][0-9]*\)[[:space:]]*/<port>\1<\/port>/'
Note that since you didn't put any quotes around $test, the shell performed word splitting and wildcard expansion on the value of the variable. That is, the value of the variable was treated as a whitespace-separated list of file names which were then joined by a single space. So if you leave out the quotes, you don't have to worry about different kinds of whitespace, you can write echo $test | sed 's/ *port *([0-9][0-9]*) */<port>\1<\/port>/'. However, if $test had been port *, the result would have depended on what files exist in the current directory.
Not all seds support reg-expression short-hand like \s. A more portable version is
test="port 3"
echo "$test" | sed -r 's/[ ]*port[ ]*([0-9]+)[ ]*/<port>\1<\/port>/'
If you really need to check for tab chars as well, just add them to the char class (in all 3 places) that, in my example just contain space chars, i.e. the [ ] bit.
output
<port>3</port>
I hope this helps.
I want to do some simple string replace in Bash with sed. I am Ubuntu 10.10.
Just see the following code, it is self-explanatory:
name="A%20Google.."
echo $name|sed 's/\%20/_/'|sed 's/\.+/_/'
I want to get A_Google_ but I get A_Google..
The sed 's/\.+/_/' part is obviously wrong.
BTW, sed 's/\%20/_/' and sed 's/%20/_/' both work. Which is better?
sed speaks POSIX basic regular expressions, which don't include + as a metacharacter. Portably, rewrite to use *:
sed 's/\.\.*/_/'
or if all you will ever care about is Linux, you can use various GNU-isms:
sed -r 's/\.\.*/_/' # turn on POSIX EREs (use -E instead of -r on OS X)
sed 's/\.\+/_/' # GNU regexes invert behavior when backslash added/removed
That last example answers your other question: a character which is literal when used as is may take on a special meaning when backslashed, and even though at the moment % doesn't have a special meaning when backslashed, future-proofing means not assuming that \% is safe.
Additional note: you don't need two separate sed commands in the pipeline there.
echo $name | sed -e 's/\%20/_/' -e 's/\.+/_/'
(Also, do you only need to do that once per line, or for all occurrences? You may want the /g modifier.)
The sed command doesn't understand + so you'll have to expand it by hand:
sed 's/\.\.*/_/'
Or tell sed that you want to use extended regexes:
sed -r 's/\.+/_/' # GNU
sed -E 's/\.+/_/' # OSX
Which switch, -r or -E, depends on your sed and it might not even support extended regexes so the portable solution is to use \.\.* in place of \.+. But, since you're on Linux, you should have GNU sed so sed -r should do the trick.
This works fine on Linux (Debian):
sed -e 's,^[ \t]*psd\(.*\)\;,,'
On mac, I believe I have to use the -E flag, instead of -e:
sed -E 's,^[ \t]*psd\(.*\)\;,,'
but the regexp does not match, and hence does not remove the lines I want.
Any tips on how to solve this?
Sample input:
apa
bepa
psd(cepa);
depa psd(epa);
psd(fepa gepa hepa);
For that input, the expected output is:
apa
bepa
depa psd(epa);
The -E flag means to use extended regular expressions. You should just use -e, as on Linux. The sed in Mac OS X is based on BSD sed, so doesn't have the GNU extensions.
After copying your sample input:
[~ 507] pbpaste | sed -e 's,^[[:space:]]*psd\(.*\);,,'
apa
bepa
depa psd(epa);
Alternatively you can use the GNU version of sed instead of the implementation provided by Mac OSX.
Mac port provides a port for it sudo port install gsed. After installing it you can use gsed instead of sed.
The '\t' is not standard in 'sed', it is a GNU extension.
To match a 'tab', you need to put a real 'tab' in your script. This is easy in a file, harder in shell.
The same problem can happen in AIX, Solaris and HP-UX or other UNIXes.
In addition to the answers above, you can exploit a useful (but shell-dependent) trick. In bash, use $'\t' to introduce a literal tab character. The following works on my Mac:
sed -e 's,^[ '$'\t''*psd\(.*\);,,'
Note how the whole sed expression consists now of three concatenated strings.
This trick might be useful in case you need the tab character specifically, without matching other whitespace (i.e., when [[:blank:]] would be too inclusive). For the above, the -e flag is not essential.
I've check this sample input on my machine and faced the problem when in third line was tab character from the beginning of line and regexp ^[ \t]*psd\(.*\)\; didn't match it. This can be passed by sed character class [[:blank:]] that equal combination of space and tab character. So you can try the following:
sed -E 's,^[[:blank:]]*psd\(.*\)\;,,' demo.txt
this produce the following output:
apa
bepa
depa psd(epa);
but it keeps the empty lines in result.
To get the exact output as you expected I used the following:
sed -n '/^[[:blank:]]*psd\(.*\)\;/!p' demo.txt
result:
apa
bepa
depa psd(epa);
this is just inverse output of matching pattern (!p).
EDIT: To match tab characters in regexp in sed (macosx) you can also try recommendation from How can I insert a tab character with sed on OS X?