Bash has this feature called history expansion where you can use shortcuts that expand to things you've typed previously into bash.
Example: !! - expands into previous command
$> rm -f /var/log/access.log
rm: /var/log/access.log: Permission Denied
$> sudo !!
sudo rm -f /var/log/access.log
$> echo "i am teh hax"
Another: !$ - expands into last arg of previous command
$> echo "no one was here" > access.log
$> cp !$ /var/log/
cp access.log /var/log/
Does bash, or some other shell, have the ability to use substitution shortcuts within the command itself?
Something like
$> cp httpd.conf !$.bak
cp httpd.conf httpd.conf.bak
$> echo "SABOTEUR!!!" > httpd.conf
I need to up my 1980's cyberpunk skills. Please Help.
by within the command itself, do you mean you want to refer to httpd.conf?
Then this is your solution in superuser
Using bash history expansion:
mv path/to/oldfile !#:1:h/newfile
where !#:1:h means: from the line you're currently typing (!#), take the first word (:1), then take only the path component (:h -- the head) from it.
The answer by justhalf is what you want.
But for your requirement, there is one more hack/misuse available.
sed -i.bak '' /path/to/file
e.g.
sed -i.bak '' httpd.conf
It will copy your file to another file with .bak appended.
Advantage: /path/to/file can contain wildcards/globs, or you can directly give a white-space separated list of files.
Related
I implemented the following command to create a file called * (asterisk):
echo > '*'
Now I'm supposed to remove this file without using any quotations.
I know how to remove this by using quotations, but not sure how without using quotations.
I tried the following commands which I was sure that they won't work because of command line expansion:
rm ./*
rm /*
If someone can help me with this, I would greatly appreciate it.
I think you're supposed to work this out yourself :-)
The simplest solution not involving quoting is to use the pattern [*]. Bracket expressions in a shell work much like character classes in regular expressions so that will match a file whose name is the single character *. Thus, you can delete your file with
rm [*]
Note that you cannot use that pattern to create a file named * because the shell substitutes words containing patterns with the name(s) of the files which match the pattern; if no such file exists, then the pattern is not matched and no substitution is performed. So if there is no file named *, then touch [*] will create a file named [*].
You could use history expansion. If the rm command directly follows the echo command, you can use !$:
echo > '*'
rm !$
!$ is shorthand for !!:$: repeat the last word ($) of the last command (!!).
If there are commands between the echo and the rm command, you can find the history number using fc -l:
$ echo > '*'
$ cmd1
$ cmd2
$ cmd3
$ fc -l
[...]
27628 echo > '*'
27629 cmd1
27630 cmd2
27631 cmd3
$ rm !27628:$
!27628 expands to the command with that number in the history, and $ is again the last word of that command.
If you have to run this in a script, you can't really look up the command number and insert it, but you can count the number of commands between the echo and the rm and use a relative event designator:
echo > '*'
cmd1
rm !-2:$
where !-2 refers to the command two lines back. Notice that history expansion is by default disabled in non-interactive shells; use
shopt -o history
to enable it.
You could use rm -i * if the number of files is not too big. This will ask for confirmation for every single file. Confirm deletion only for the file * and reject it for all others.
I'm trying to copy a file from: ~/.ssh/
but everytime I run the script it keeps saying
pi#raspberrypi:/etc/greenwich $ ./copybash.sh
cat: ~/.ssh/testfilegen2.log: No such file or directory
copybash.sh
!/bin/bash
sourceFile="~/.ssh/testfilegen2.log"
targetFile="/etc/network/interfaces2"
sudo cat "$sourceFile" > "$targetFile"
sudo service networking restart
Any Suggestions?
Thank you
Unquote the tilde in the assignment to sourceFile so that it expands properly. Tilde expansion does not occur on parameter expansion.
sourceFile=~/".ssh/testfilegen2.log"
(In this case, no quotes would be necessary at all, but just to demonstrate that the ~ and the following / are the only things that need to remain unquoted for tilde expansion to occur.)
Take a look to this snippet code:
#!/bin/bash
v1=~/'file1.txt'
v2=~/'file2.txt'
echo 'Hi!' > $v1
cat $v1 > $v2
cat $v2
$ script.sh
Hi!
The documentation is in the section "Tilde Expansion"
of the "General Commands Manual BASH".
What's the simplest way to do a find and replace for a given input string, say abc, and replace with another string, say XYZ in file /tmp/file.txt?
I am writting an app and using IronPython to execute commands through SSH — but I don't know Unix that well and don't know what to look for.
I have heard that Bash, apart from being a command line interface, can be a very powerful scripting language. So, if this is true, I assume you can perform actions like these.
Can I do it with bash, and what's the simplest (one line) script to achieve my goal?
The easiest way is to use sed (or perl):
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while IFS='' read -r a; do
echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)
For Mac users:
sed -i '' 's/abc/XYZ/g' /tmp/file.txt
(See the comment below why)
File manipulation isn't normally done by Bash, but by programs invoked by Bash, e.g.:
perl -pi -e 's/abc/XYZ/g' /tmp/file.txt
The -i flag tells it to do an in-place replacement.
See man perlrun for more details, including how to take a backup of the original file.
I was surprised when I stumbled over this...
There is a replace command which ships with the "mysql-server" package, so if you have installed it try it out:
# replace string abc to XYZ in files
replace "abc" "XYZ" -- file.txt file2.txt file3.txt
# or pipe an echo to replace
echo "abcdef" |replace "abc" "XYZ"
See man replace for more on this.
This is an old post but for anyone wanting to use variables as #centurian said the single quotes mean nothing will be expanded.
A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work:
sed -i -e "s/$var1/$var2/g" /tmp/file.txt
Bash, like other shells, is just a tool for coordinating other commands. Typically you would try to use standard UNIX commands, but you can of course use Bash to invoke anything, including your own compiled programs, other shell scripts, Python and Perl scripts etc.
In this case, there are a couple of ways to do it.
If you want to read a file, and write it to another file, doing search/replace as you go, use sed:
sed 's/abc/XYZ/g' <infile >outfile
If you want to edit the file in place (as if opening the file in an editor, editing it, then saving it) supply instructions to the line editor 'ex'
echo "%s/abc/XYZ/g
w
q
" | ex file
Example is like vi without the fullscreen mode. You can give it the same commands you would at vi's : prompt.
I found this thread among others and I agree it contains the most complete answers so I'm adding mine too:
sed and ed are so useful...by hand.
Look at this code from #Johnny:
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
When my restriction is to use it in a shell script, no variable can be used inside in place of "abc" or "XYZ". The BashFAQ seems to agree with what I understand at least. So, I can't use:
x='abc'
y='XYZ'
sed -i -e 's/$x/$y/g' /tmp/file.txt
#or,
sed -i -e "s/$x/$y/g" /tmp/file.txt
but, what can we do? As, #Johnny said use a while read... but, unfortunately that's not the end of the story. The following worked well with me:
#edit user's virtual domain
result=
#if nullglob is set then, unset it temporarily
is_nullglob=$( shopt -s | egrep -i '*nullglob' )
if [[ is_nullglob ]]; then
shopt -u nullglob
fi
while IFS= read -r line; do
line="${line//'<servername>'/$server}"
line="${line//'<serveralias>'/$alias}"
line="${line//'<user>'/$user}"
line="${line//'<group>'/$group}"
result="$result""$line"'\n'
done < $tmp
echo -e $result > $tmp
#if nullglob was set then, re-enable it
if [[ is_nullglob ]]; then
shopt -s nullglob
fi
#move user's virtual domain to Apache 2 domain directory
......
As one can see if nullglob is set then, it behaves strangely when there is a string containing a * as in:
<VirtualHost *:80>
ServerName www.example.com
which becomes
<VirtualHost ServerName www.example.com
there is no ending angle bracket and Apache2 can't even load.
This kind of parsing should be slower than one-hit search and replace but, as you already saw, there are four variables for four different search patterns working out of one parse cycle.
The most suitable solution I can think of with the given assumptions of the problem.
You can use sed:
sed -i 's/abc/XYZ/gi' /tmp/file.txt
You can use find and sed if you don't know your filename:
find ./ -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Find and replace in all Python files:
find ./ -iname "*.py" -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Be careful if you replace URLs with "/" character.
An example of how to do it:
sed -i "s%http://domain.com%http://www.domain.com/folder/%g" "test.txt"
Extracted from: http://www.sysadmit.com/2015/07/linux-reemplazar-texto-en-archivos-con-sed.html
If the file you are working on is not so big, and temporarily storing it in a variable is no problem, then you can use Bash string substitution on the whole file at once - there's no need to go over it line by line:
file_contents=$(</tmp/file.txt)
echo "${file_contents//abc/XYZ}" > /tmp/file.txt
The whole file contents will be treated as one long string, including linebreaks.
XYZ can be a variable eg $replacement, and one advantage of not using sed here is that you need not be concerned that the search or replace string might contain the sed pattern delimiter character (usually, but not necessarily, /). A disadvantage is not being able to use regular expressions or any of sed's more sophisticated operations.
You may also use the ed command to do in-file search and replace:
# delete all lines matching foobar
ed -s test.txt <<< $'g/foobar/d\nw'
See more in "Editing files via scripts with ed".
To edit text in the file non-interactively, you need in-place text editor such as vim.
Here is simple example how to use it from the command line:
vim -esnc '%s/foo/bar/g|:wq' file.txt
This is equivalent to #slim answer of ex editor which is basically the same thing.
Here are few ex practical examples.
Replacing text foo with bar in the file:
ex -s +%s/foo/bar/ge -cwq file.txt
Removing trailing whitespaces for multiple files:
ex +'bufdo!%s/\s\+$//e' -cxa *.txt
Troubleshooting (when terminal is stuck):
Add -V1 param to show verbose messages.
Force quit by: -cwq!.
See also:
How to edit files non-interactively (e.g. in pipeline)? at Vi SE
Try the following shell command:
find ./ -type f -name "file*.txt" | xargs sed -i -e 's/abc/xyz/g'
You can use python within the bash script too. I didn't have much success with some of the top answers here, and found this to work without the need for loops:
#!/bin/bash
python
filetosearch = '/home/ubuntu/ip_table.txt'
texttoreplace = 'tcp443'
texttoinsert = 'udp1194'
s = open(filetosearch).read()
s = s.replace(texttoreplace, texttoinsert)
f = open(filetosearch, 'w')
f.write(s)
f.close()
quit()
Simplest way to replace multiple text in a file using sed command
Command -
sed -i 's#a/b/c#D/E#g;s#/x/y/z#D:/X#g;' filename
In the above command s#a/b/c#D/E#g where I am replacing a/b/c with D/E and then after the ; we again doing the same thing
You can use rpl command. For example you want to change domain name in whole php project.
rpl -ivRpd -x'.php' 'old.domain.name' 'new.domain.name' ./path_to_your_project_folder/
This is not clear bash of cause, but it's a very quick and usefull. :)
For MAC users in case you don't read the comments :)
As mentioned by #Austin, if you get the Invalid command code error
For the in-place replacements BSD sed requires a file extension after the -i flag to save to a backup file with given extension.
sed -i '.bak' 's/find/replace' /file.txt
You can use '' empty string if you want to skip backup.
sed -i '' 's/find/replace' /file.txt
All merit to #Austin
Open file using vim editor. In command mode
:%s/abc/xyz/g
This is the simplest
In case of doing changes in multiple files together we can do in a single line as:-
user_name='whoami'
for file in file1.txt file2.txt file3.txt; do sed -i -e 's/default_user/${user_name}/g' $file; done
Added if in case could be useful.
I have this list of files on a Linux server:
abc.log.2012-03-14
abc.log.2012-03-27
abc.log.2012-03-28
abc.log.2012-03-29
abc.log.2012-03-30
abc.log.2012-04-02
abc.log.2012-04-04
abc.log.2012-04-05
abc.log.2012-04-09
abc.log.2012-04-10
I've been deleting selected log files one by one, using the command rm -rf see below:
rm -rf abc.log.2012-03-14
rm -rf abc.log.2012-03-27
rm -rf abc.log.2012-03-28
Is there another way, so that I can delete the selected files at once?
Bash supports all sorts of wildcards and expansions.
Your exact case would be handled by brace expansion, like so:
$ rm -rf abc.log.2012-03-{14,27,28}
The above would expand to a single command with all three arguments, and be equivalent to typing:
$ rm -rf abc.log.2012-03-14 abc.log.2012-03-27 abc.log.2012-03-28
It's important to note that this expansion is done by the shell, before rm is even loaded.
Use a wildcard (*) to match multiple files.
For example, the command below will delete all files with names beginning with abc.log.2012-03-.
rm -f abc.log.2012-03-*
I'd recommend running ls abc.log.2012-03-* to list the files so that you can see what you are going to delete before running the rm command.
For more details see the Bash man page on filename expansion.
If you want to delete all files whose names match a particular form, a wildcard (glob pattern) is the most straightforward solution. Some examples:
$ rm -f abc.log.* # Remove them all
$ rm -f abc.log.2012* # Remove all logs from 2012
$ rm -f abc.log.2012-0[123]* # Remove all files from the first quarter of 2012
Regular expressions are more powerful than wildcards; you can feed the output of grep to rm -f. For example, if some of the file names start with "abc.log" and some with "ABC.log", grep lets you do a case-insensitive match:
$ rm -f $(ls | grep -i '^abc\.log\.')
This will cause problems if any of the file names contain funny characters, including spaces. Be careful.
When I do this, I run the ls | grep ... command first and check that it produces the output I want -- especially if I'm using rm -f:
$ ls | grep -i '^abc\.log\.'
(check that the list is correct)
$ rm -f $(!!)
where !! expands to the previous command. Or I can type up-arrow or Ctrl-P and edit the previous line to add the rm -f command.
This assumes you're using the bash shell. Some other shells, particularly csh and tcsh and some older sh-derived shells, may not support the $(...) syntax. You can use the equivalent backtick syntax:
$ rm -f `ls | grep -i '^abc\.log\.'`
The $(...) syntax is easier to read, and if you're really ambitious it can be nested.
Finally, if the subset of files you want to delete can't be easily expressed with a regular expression, a trick I often use is to list the files to a temporary text file, then edit it:
$ ls > list
$ vi list # Use your favorite text editor
I can then edit the list file manually, leaving only the files I want to remove, and then:
$ rm -f $(<list)
or
$ rm -f `cat list`
(Again, this assumes none of the file names contain funny characters, particularly spaces.)
Or, when editing the list file, I can add rm -f to the beginning of each line and then:
$ . ./list
or
$ source ./list
Editing the file is also an opportunity to add quotes where necessary, for example changing rm -f foo bar to rm -f 'foo bar' .
Just use multiline selection in sublime to combine all of the files into a single line and add a space between each file name and then add rm at the beginning of the list. This is mostly useful when there isn't a pattern in the filenames you want to delete.
[$]> rm abc.log.2012-03-14 abc.log.2012-03-27 abc.log.2012-03-28 abc.log.2012-03-29 abc.log.2012-03-30 abc.log.2012-04-02 abc.log.2012-04-04 abc.log.2012-04-05 abc.log.2012-04-09 abc.log.2012-04-10
A wild card would work nicely for this, although to be safe it would be best to make the use of the wild card as minimal as possible, so something along the lines of this:
rm -rf abc.log.2012-*
Although from the looks of it, are those just single files? The recursive option should not be necessary if none of those items are directories, so best to not use that, just for safety.
I am not a linux guru, but I believe you want to pipe your list of output files to xargs rm -rf. I have used something like this in the past with good results. Test on a sample directory first!
EDIT - I might have misunderstood, based on the other answers that are appearing. If you can use wildcards, great. I assumed that your original list that you displayed was generated by a program to give you your "selection", so I thought piping to xargs would be the way to go.
if you want to delete all files that belong to a directory at once.
For example:
your Directory name is "log" and "log" directory include abc.log.2012-03-14, abc.log.2012-03-15,... etc files. You have to be above the log directory and:
rm -rf /log/*
I know my topic is a little confusing, but here is what I want to do.
I have a file which I would like to create a link to in my home directory ~/bin, but when I execute the file that is symbolically linked, the file requires another file in its directory. Therefore, it fails to run because it cannot find the other file. What can I do?
Well, you have two simple solutions.
edit the shell script to point to the absolute path of the file, not just the the basename.
./path/to/file.sh
VS
file.sh
so something like this should do what your after. sed -i 's|file.sh|./path/to/file.sh|g' ~/bin/script.sh it searches your symlinked file, script.sh in this case, and replaces the call to file.sh to ./path/to/file.sh. note you often see sed use /'s. but it can use just about anything as a delimiter, if you wish to use /'s here you will need to escape them. /. you may want to consider escaping the . (period) as well, but in this case its not necessary. If you are new to sed realize that the -i flag means it will edit the file in place. Lastly, realize its a simple search and replace operation and you may chose to do it by hand.
The second way is to create a ln -s to the file as you did with the other file so there exists a symbolic link between both files.
ln -s /far/off/script.sh ~/bin/script.sh
and
ln -s /far/off/file.sh ~/bin/file.sh
more on symlinking
I would rather create a script file in ~/bin/` that calls your executable from the appropriate directory.
Here is an example using /sbin/ifconfig:
$ cat > ~/bin/file
#!/bin/bash
file=/sbin/ifconfig
cd `dirname $file`
`basename $file`
(ctr+d)
$ chmod +x ~/bin/file
$ file
Here you should see the output of ifconfig but the point is: its get executed from the /sbin directory. So if ifconfig had dependencies it would work properly. Just replace /sbin/ifconfig with your absolute path.
Alternatively, you can modify your script as
pushd ~/bin
##### your script here
popd
Combination of readlink and dirname will get the actual directory of the script:
my_dir=$(dirname "$(readlink -f "$0")")
source "$my_dir/other_file"