I remember doing magic with vi by "programming" it with input commands but I don't remember exactly how.
My sepcial request are:
launch vi in a script with command to be executed.
do an insert in one file.
search for a string in the file.
use $VARIABLE in the vi command line to replace something in the command.
finish with :wq.
I my memory, I sent the command exactly like in vi and the ESC key was emulate by '[' or something near.
I used this command into script to edit and change files.
I'm going to see the -c option but for now I cannot use $VARIABLE and insert was impossible (with 'i' or 'o').
#!/bin/sh
#
cp ../data/* .
# Retrieve filename.
MODNAME=$(pwd | awk -F'-' '{ print $2 }')
mv minimod.c $MODNAME.c
# Edit and change filename into the file (from mimimod to the content of $VARIABLE).
vi $MODENAME.c -c ":1,$s/minimod/$MODNAME/" -c ':wq!'
This example is not functionning (it seems the $VARIABLE is not good in -c command).
Could you help me remember memory ;) ?
Thanks a lot.
Joe.
You should not use vi for non-interactive editing. There's already another tool for that, it's called sed or stream editor.
What you want to do is
sed -i 's/minimod/'$MODNAME'/g' $MODNAME.c
to replace your vi command.
Maybe you are looking for the ex command, that can be run non-interatively:
ex $MODENAME.c <<< "1,\$s/minimod/$MODNAME/
wq!
"
Or if you use an old shell that does not implement here strings:
ex $MODENAME.c << EOF
1,\$s/minimod/$MODNAME/
wq!
EOF
Or if you do not want to use here documents either:
echo "1,\$s/minimod/$MODNAME/" > cmds.txt
echo "wq!" >> cmds.txt
ex $MODENAME.c < cmds.txt
rm cmds.txt
One command per line in the standard input. Just remember not to write the leading :. Take a look at this page for a quick review of ex commands.
Granted, it would be better to use the proper tool for the job, that would be sed, as #IsaA answer suggests, or awk for more complex commands.
Related
I want to save a command to a file (for example I want to save the string "cat /etc/passwd" to a file) but I can't use the echo command.
How can I create and save string to a file directly without using echo command?
You can redirect cat to a file, type the text, and press Control-D when you're done, like this:
cat > file.txt
some text
some more text
^D
By ^D I mean to press Control-D at the end. The line must be empty.
It will not be part of the file, it is just to terminate the input.
Are you avoiding ECHO for security purposes (e.g. you're using a shared terminal and you don't want to leave trace in the shell history of what you've written inside your files) or you're just curious for an alternative method?
Simple alternative to echo:
As someone said, redirecting cat is probably the simpler way to go.
I'd suggest you to manually type your end-of-file, like this:
cat <<EOF > outputfile
> type here
> your
> text
> and finish it with
> EOF
Here's the string you're asking for, as an example:
cat <<EOF > myscript.sh
cat /etc/passwd
EOF
You probably don't want everyone to know you've peeked into that file, but if that's your purpose please notice that wrapping it inside an executable file won't make it more private, as that lines will be logged anyway...
Security - Avoiding history logs etc..
In modern shell, just try adding a space at the beginning of every command and use freely whatever you want.
BTW, my best hint is to avoid using that terminal at all, if you can. If you got two shells (another machine or even just another secure user in the same machine), I'd recommend you using netcat. See here: http://www.thegeekstuff.com/2012/04/nc-command-examples/?utm_source=feedburner
{ { command ls $(dirname $(which cat)) |
grep ^ca't$'; ls /etc/passwd; } |
tr \\n ' '; printf '\n'; } > output-file
But it's probably a lot simpler to just do : printf 'cat /etc/passwd\n'
To be clear, this is a tongue-in-cheek solution. The initial command is an extraordinarily convoluted way to get what you want, and this is intended to be a humorous answer. Perhaps instructive to understand.
I am not sure I understood you correctly but
cat /etc/passwd > target.file
use the > operator to write it to file without echoing
If you need to use it, inside a program :
cat <<EOF >file.txt
some text
some more text
EOF
I would imagine that you are probably trying to print the content of a string to a file, hence you mentioned echo.
You are avoiding this:
echo "cat /etc/passwd" > target.file
You can use a here string combined with cat.
cat > target.file <<< "cat /etc/passwd"
Now the file target.file will contain a string cat /etc/passwd.
$ cat target.file
cat /etc/passwd
$
To create string:
var1=your command
to save a file or variable in a file without echo use:
cat $FILE/VAR1 > /new/file/path
I am looking for a shell parameter to represent the redirection operand.
e.g: data/temp.txt in this command:
cat file.txt > data/temp.txt
Is there any such bash special Parameters which will allow me to open the file that I am redirecting to in interactive use, after the command will exit ?
$ cat file.txt > data/temp.txt
$ vim data/temp.txt
I don't think so. But since the redirection operand usually is the last, you can use Bash's history expansion:
echo hello > testfile
cat !$ # cat testfile
Or for interactive session, alt+. or esc+_ can be used to insert the last word of the previous line, as a comment pointed out.
Bash documentation for history expansion.
Another possibility here is bash's process substitution -- if you're most interested in opening an editor on the output of cat, but you're not as interested in writing the output to a file, you can use
vim <(cat file.txt)
This writes stdout to a named pipe and opens the named pipe (probably named something like /dev/fd/NN) in a text editor. You make your edits, and you can then save your edits using <esc>:w data/temp.txt if you wish.
What's the simplest way to do a find and replace for a given input string, say abc, and replace with another string, say XYZ in file /tmp/file.txt?
I am writting an app and using IronPython to execute commands through SSH — but I don't know Unix that well and don't know what to look for.
I have heard that Bash, apart from being a command line interface, can be a very powerful scripting language. So, if this is true, I assume you can perform actions like these.
Can I do it with bash, and what's the simplest (one line) script to achieve my goal?
The easiest way is to use sed (or perl):
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while IFS='' read -r a; do
echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)
For Mac users:
sed -i '' 's/abc/XYZ/g' /tmp/file.txt
(See the comment below why)
File manipulation isn't normally done by Bash, but by programs invoked by Bash, e.g.:
perl -pi -e 's/abc/XYZ/g' /tmp/file.txt
The -i flag tells it to do an in-place replacement.
See man perlrun for more details, including how to take a backup of the original file.
I was surprised when I stumbled over this...
There is a replace command which ships with the "mysql-server" package, so if you have installed it try it out:
# replace string abc to XYZ in files
replace "abc" "XYZ" -- file.txt file2.txt file3.txt
# or pipe an echo to replace
echo "abcdef" |replace "abc" "XYZ"
See man replace for more on this.
This is an old post but for anyone wanting to use variables as #centurian said the single quotes mean nothing will be expanded.
A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work:
sed -i -e "s/$var1/$var2/g" /tmp/file.txt
Bash, like other shells, is just a tool for coordinating other commands. Typically you would try to use standard UNIX commands, but you can of course use Bash to invoke anything, including your own compiled programs, other shell scripts, Python and Perl scripts etc.
In this case, there are a couple of ways to do it.
If you want to read a file, and write it to another file, doing search/replace as you go, use sed:
sed 's/abc/XYZ/g' <infile >outfile
If you want to edit the file in place (as if opening the file in an editor, editing it, then saving it) supply instructions to the line editor 'ex'
echo "%s/abc/XYZ/g
w
q
" | ex file
Example is like vi without the fullscreen mode. You can give it the same commands you would at vi's : prompt.
I found this thread among others and I agree it contains the most complete answers so I'm adding mine too:
sed and ed are so useful...by hand.
Look at this code from #Johnny:
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
When my restriction is to use it in a shell script, no variable can be used inside in place of "abc" or "XYZ". The BashFAQ seems to agree with what I understand at least. So, I can't use:
x='abc'
y='XYZ'
sed -i -e 's/$x/$y/g' /tmp/file.txt
#or,
sed -i -e "s/$x/$y/g" /tmp/file.txt
but, what can we do? As, #Johnny said use a while read... but, unfortunately that's not the end of the story. The following worked well with me:
#edit user's virtual domain
result=
#if nullglob is set then, unset it temporarily
is_nullglob=$( shopt -s | egrep -i '*nullglob' )
if [[ is_nullglob ]]; then
shopt -u nullglob
fi
while IFS= read -r line; do
line="${line//'<servername>'/$server}"
line="${line//'<serveralias>'/$alias}"
line="${line//'<user>'/$user}"
line="${line//'<group>'/$group}"
result="$result""$line"'\n'
done < $tmp
echo -e $result > $tmp
#if nullglob was set then, re-enable it
if [[ is_nullglob ]]; then
shopt -s nullglob
fi
#move user's virtual domain to Apache 2 domain directory
......
As one can see if nullglob is set then, it behaves strangely when there is a string containing a * as in:
<VirtualHost *:80>
ServerName www.example.com
which becomes
<VirtualHost ServerName www.example.com
there is no ending angle bracket and Apache2 can't even load.
This kind of parsing should be slower than one-hit search and replace but, as you already saw, there are four variables for four different search patterns working out of one parse cycle.
The most suitable solution I can think of with the given assumptions of the problem.
You can use sed:
sed -i 's/abc/XYZ/gi' /tmp/file.txt
You can use find and sed if you don't know your filename:
find ./ -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Find and replace in all Python files:
find ./ -iname "*.py" -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Be careful if you replace URLs with "/" character.
An example of how to do it:
sed -i "s%http://domain.com%http://www.domain.com/folder/%g" "test.txt"
Extracted from: http://www.sysadmit.com/2015/07/linux-reemplazar-texto-en-archivos-con-sed.html
If the file you are working on is not so big, and temporarily storing it in a variable is no problem, then you can use Bash string substitution on the whole file at once - there's no need to go over it line by line:
file_contents=$(</tmp/file.txt)
echo "${file_contents//abc/XYZ}" > /tmp/file.txt
The whole file contents will be treated as one long string, including linebreaks.
XYZ can be a variable eg $replacement, and one advantage of not using sed here is that you need not be concerned that the search or replace string might contain the sed pattern delimiter character (usually, but not necessarily, /). A disadvantage is not being able to use regular expressions or any of sed's more sophisticated operations.
You may also use the ed command to do in-file search and replace:
# delete all lines matching foobar
ed -s test.txt <<< $'g/foobar/d\nw'
See more in "Editing files via scripts with ed".
To edit text in the file non-interactively, you need in-place text editor such as vim.
Here is simple example how to use it from the command line:
vim -esnc '%s/foo/bar/g|:wq' file.txt
This is equivalent to #slim answer of ex editor which is basically the same thing.
Here are few ex practical examples.
Replacing text foo with bar in the file:
ex -s +%s/foo/bar/ge -cwq file.txt
Removing trailing whitespaces for multiple files:
ex +'bufdo!%s/\s\+$//e' -cxa *.txt
Troubleshooting (when terminal is stuck):
Add -V1 param to show verbose messages.
Force quit by: -cwq!.
See also:
How to edit files non-interactively (e.g. in pipeline)? at Vi SE
Try the following shell command:
find ./ -type f -name "file*.txt" | xargs sed -i -e 's/abc/xyz/g'
You can use python within the bash script too. I didn't have much success with some of the top answers here, and found this to work without the need for loops:
#!/bin/bash
python
filetosearch = '/home/ubuntu/ip_table.txt'
texttoreplace = 'tcp443'
texttoinsert = 'udp1194'
s = open(filetosearch).read()
s = s.replace(texttoreplace, texttoinsert)
f = open(filetosearch, 'w')
f.write(s)
f.close()
quit()
Simplest way to replace multiple text in a file using sed command
Command -
sed -i 's#a/b/c#D/E#g;s#/x/y/z#D:/X#g;' filename
In the above command s#a/b/c#D/E#g where I am replacing a/b/c with D/E and then after the ; we again doing the same thing
You can use rpl command. For example you want to change domain name in whole php project.
rpl -ivRpd -x'.php' 'old.domain.name' 'new.domain.name' ./path_to_your_project_folder/
This is not clear bash of cause, but it's a very quick and usefull. :)
For MAC users in case you don't read the comments :)
As mentioned by #Austin, if you get the Invalid command code error
For the in-place replacements BSD sed requires a file extension after the -i flag to save to a backup file with given extension.
sed -i '.bak' 's/find/replace' /file.txt
You can use '' empty string if you want to skip backup.
sed -i '' 's/find/replace' /file.txt
All merit to #Austin
Open file using vim editor. In command mode
:%s/abc/xyz/g
This is the simplest
In case of doing changes in multiple files together we can do in a single line as:-
user_name='whoami'
for file in file1.txt file2.txt file3.txt; do sed -i -e 's/default_user/${user_name}/g' $file; done
Added if in case could be useful.
I want to change my PS1 in my .bashrc file.
I've found a script using printf with %q directive to escape characters :
#!/bin/bash
STR=$(printf "%q" "PS1=\u#\h:\w\$ ")
sed -i '/PS1/c\'"$STR" ~/.bashrc
The problem is that I get this error :
script.sh: 2: printf: %q: invalid directive
Any idea ? Maybe an other way to escape the characters ?
The printf command is built into bash. It's also an external command, typically installed in /usr/bin/printf. On most Linux systems, /usr/bin/printf is the GNU coreutils implementation.
Older releases of the GNU coreutils printf command do not support the %q format specifier; it was introduced in version 8.25, released 2016-10-20. bash's built-in printf command does -- and has as long as bash has had a built-in printf command.
The error message implies that you're running script.sh using something other than bash.
Since the #!/bin/bash line appears to be correct, you're probably doing one of the following:
sh script.sh
. script.sh
source script.sh
Instead, just execute it directly (after making sure it has execute permission, using chmod +x if needed):
./script.sh
Or you could just edit your .bashrc file manually. The script, if executed correctly, will add this line to your .bashrc:
PS1=\\u#\\h:\\w\$\
(The space at the end of that line is significant.) Or you can do it more simply like this:
PS1='\u#\h:\w\$ '
One problem with the script is that it will replace every line that mentions PS1. If you just set it once and otherwise don't refer to it, that's fine, but if you have something like:
if [ ... ] ; then
PS1=this
else
PS1=that
fi
then the script will thoroughly mess that up. It's just a bit too clever.
Keith Thompson has given good advice in his answer. But FWIW, you can force bash to use a builtin command by preceding the command name with builtin eg
builtin printf "%q" "PS1=\u#\h:\w\$ "
Conversely,
command printf "%s\n" some stuff
forces bash to use the external command (if it can find one).
command can be used to invoke commands on disk when a function with the same name exists. However, command does not invoke a command on disk in lieu of a Bash built-in with the same name, it only works to suppress invocation of a shell function. (Thanks to Rockallite for bringing this error to my attention).
It's possible to enable or disable specific bash builtins (maybe your .bashrc is doing that to printf). See help enable for details. And I guess I should mention that you can use
type printf
to find out what kind of entity (shell function, builtin, or external command) bash will run when you give it a naked printf. You can get a list of all commands with a given name by passing type the -a option, eg
type -a printf
You can use grep to see the lines in your .bashrc file that contain PS1:
grep 'PS1' ~/.bashrc
or
grep -n0 --color=auto 'PS1=' ~/.bashrc
which gives you line numbers and fancy coloured output. And then you can use the line number to force sed to just modify the line you want changed.
Eg, if grep tells you that the line you want to change is line 7, you can do
sed -i '7c\'"$STR" ~/.bashrc
to edit it. Or even better,
sed -i~ '7c\'"$STR" ~/.bashrc
which backs up the original version of the file in case you make a mistake.
When using sed -i I generally do a test run first without the -i so that the output goes to the shell, to let me see what the modifications do before I write them to the file.
There is a command I don't understand:
custom_command << EOF!!
I want to ask what EOF!! is in the bash script. I did find EOF with google, but google will ignore the "!!" automatically, so I cannot find EOF!!.
I know the end of the file token, but I don't exactly know what it means with the "!!" in the script. Is this a mark to force something to do something like in vim's wq! ?
Plus, why and when should we use EOF!! instead of EOF?
On the command line, !! would be expanded to the last command executed. Bash will print the line for you:
$ ls
a.txt b.txt
$ cat <<EOF!!
cat <<EOFls
>
In a script, though, history expansion is disabled by default, so the exclamation marks are part of the word.
#! /bin/bash
ls
cat <<EOF!!
echo 1
EOFls
echo 2
Produces:
a.txt b.txt
script.sh: line 7: warning: here-document at line 3 delimited by end-of-file (wanted `EOF!!')
echo 1
EOFls
echo 2
To enable history and history expansion in a script, add the following lines:
set -o history
set -H
You can use whatever string as here document terminator.
EOF!! is just what the person writing the script decided to use.
It's probably just a weird heredoc.
Example:
cat << EOF!!
blabla
EOF!!
Note: this only works in script files. The command line parser interprets !!.
As others already wrote, this is a here-document.
The token used for that should be chosen carefully; as the probability that the here-document contains EOF!! is lower than for EOF itself, they chose that.
I suppose they checked it does not harm before using it; !! in a script does NOT refer to the history, but it stays as it is.
The bash manual lists this under "Event designators", saying:
!!
Refer to the previous command. This is a synonym for !-1`.
I simply searched for "bash manual double exclamation".