I want to mass-edit a ton of files that are returned in a grep. (I know, I should get better at sed).
So if I do:
grep -rnI 'xg_icon-*'
How do I pipe all of those files into vi?
The easiest way is to have grep return just the filenames (-l instead of -n) that match the pattern. Run that in a subshell and feed the results to Vim.
vim $(grep -rIl 'xg_icon-*' *)
A nice general solution to this is to use xargs to convert a stdout from a process like grep to an argument list.
A la:
grep -rIl 'xg_icon-*' | xargs vi
if you use vim and the -p option, it will open each file in a tab, and you can switch between them using gt or gT, or even the mouse if you have mouse support in the terminal
You can do it without any processing of the grep output! This will even enable you to go the the right line (using :help quickfix commands, eg. :cn or :cw). So, if you are using bash or zsh:
vim -q <(grep foo *.c)
if what you want to edit is similar across all files, then no point using vi to do it manually. (although vi can be scripted as well), hypothetically, it looks something like this, since you never mention what you want to edit
grep -rnI 'xg_icon-*' | while read FILE
do
sed -i.bak 's/old/new/g' $FILE # (or other editing commands, eg awk... )
done
vi `grep -l -i findthisword *`
Related
I remember doing magic with vi by "programming" it with input commands but I don't remember exactly how.
My sepcial request are:
launch vi in a script with command to be executed.
do an insert in one file.
search for a string in the file.
use $VARIABLE in the vi command line to replace something in the command.
finish with :wq.
I my memory, I sent the command exactly like in vi and the ESC key was emulate by '[' or something near.
I used this command into script to edit and change files.
I'm going to see the -c option but for now I cannot use $VARIABLE and insert was impossible (with 'i' or 'o').
#!/bin/sh
#
cp ../data/* .
# Retrieve filename.
MODNAME=$(pwd | awk -F'-' '{ print $2 }')
mv minimod.c $MODNAME.c
# Edit and change filename into the file (from mimimod to the content of $VARIABLE).
vi $MODENAME.c -c ":1,$s/minimod/$MODNAME/" -c ':wq!'
This example is not functionning (it seems the $VARIABLE is not good in -c command).
Could you help me remember memory ;) ?
Thanks a lot.
Joe.
You should not use vi for non-interactive editing. There's already another tool for that, it's called sed or stream editor.
What you want to do is
sed -i 's/minimod/'$MODNAME'/g' $MODNAME.c
to replace your vi command.
Maybe you are looking for the ex command, that can be run non-interatively:
ex $MODENAME.c <<< "1,\$s/minimod/$MODNAME/
wq!
"
Or if you use an old shell that does not implement here strings:
ex $MODENAME.c << EOF
1,\$s/minimod/$MODNAME/
wq!
EOF
Or if you do not want to use here documents either:
echo "1,\$s/minimod/$MODNAME/" > cmds.txt
echo "wq!" >> cmds.txt
ex $MODENAME.c < cmds.txt
rm cmds.txt
One command per line in the standard input. Just remember not to write the leading :. Take a look at this page for a quick review of ex commands.
Granted, it would be better to use the proper tool for the job, that would be sed, as #IsaA answer suggests, or awk for more complex commands.
What's the simplest way to do a find and replace for a given input string, say abc, and replace with another string, say XYZ in file /tmp/file.txt?
I am writting an app and using IronPython to execute commands through SSH — but I don't know Unix that well and don't know what to look for.
I have heard that Bash, apart from being a command line interface, can be a very powerful scripting language. So, if this is true, I assume you can perform actions like these.
Can I do it with bash, and what's the simplest (one line) script to achieve my goal?
The easiest way is to use sed (or perl):
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while IFS='' read -r a; do
echo "${a//abc/XYZ}"
done < /tmp/file.txt > /tmp/file.txt.t
mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name. (For robustness and security, the temporary file name should not be static or predictable, but let's not go there.)
For Mac users:
sed -i '' 's/abc/XYZ/g' /tmp/file.txt
(See the comment below why)
File manipulation isn't normally done by Bash, but by programs invoked by Bash, e.g.:
perl -pi -e 's/abc/XYZ/g' /tmp/file.txt
The -i flag tells it to do an in-place replacement.
See man perlrun for more details, including how to take a backup of the original file.
I was surprised when I stumbled over this...
There is a replace command which ships with the "mysql-server" package, so if you have installed it try it out:
# replace string abc to XYZ in files
replace "abc" "XYZ" -- file.txt file2.txt file3.txt
# or pipe an echo to replace
echo "abcdef" |replace "abc" "XYZ"
See man replace for more on this.
This is an old post but for anyone wanting to use variables as #centurian said the single quotes mean nothing will be expanded.
A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work:
sed -i -e "s/$var1/$var2/g" /tmp/file.txt
Bash, like other shells, is just a tool for coordinating other commands. Typically you would try to use standard UNIX commands, but you can of course use Bash to invoke anything, including your own compiled programs, other shell scripts, Python and Perl scripts etc.
In this case, there are a couple of ways to do it.
If you want to read a file, and write it to another file, doing search/replace as you go, use sed:
sed 's/abc/XYZ/g' <infile >outfile
If you want to edit the file in place (as if opening the file in an editor, editing it, then saving it) supply instructions to the line editor 'ex'
echo "%s/abc/XYZ/g
w
q
" | ex file
Example is like vi without the fullscreen mode. You can give it the same commands you would at vi's : prompt.
I found this thread among others and I agree it contains the most complete answers so I'm adding mine too:
sed and ed are so useful...by hand.
Look at this code from #Johnny:
sed -i -e 's/abc/XYZ/g' /tmp/file.txt
When my restriction is to use it in a shell script, no variable can be used inside in place of "abc" or "XYZ". The BashFAQ seems to agree with what I understand at least. So, I can't use:
x='abc'
y='XYZ'
sed -i -e 's/$x/$y/g' /tmp/file.txt
#or,
sed -i -e "s/$x/$y/g" /tmp/file.txt
but, what can we do? As, #Johnny said use a while read... but, unfortunately that's not the end of the story. The following worked well with me:
#edit user's virtual domain
result=
#if nullglob is set then, unset it temporarily
is_nullglob=$( shopt -s | egrep -i '*nullglob' )
if [[ is_nullglob ]]; then
shopt -u nullglob
fi
while IFS= read -r line; do
line="${line//'<servername>'/$server}"
line="${line//'<serveralias>'/$alias}"
line="${line//'<user>'/$user}"
line="${line//'<group>'/$group}"
result="$result""$line"'\n'
done < $tmp
echo -e $result > $tmp
#if nullglob was set then, re-enable it
if [[ is_nullglob ]]; then
shopt -s nullglob
fi
#move user's virtual domain to Apache 2 domain directory
......
As one can see if nullglob is set then, it behaves strangely when there is a string containing a * as in:
<VirtualHost *:80>
ServerName www.example.com
which becomes
<VirtualHost ServerName www.example.com
there is no ending angle bracket and Apache2 can't even load.
This kind of parsing should be slower than one-hit search and replace but, as you already saw, there are four variables for four different search patterns working out of one parse cycle.
The most suitable solution I can think of with the given assumptions of the problem.
You can use sed:
sed -i 's/abc/XYZ/gi' /tmp/file.txt
You can use find and sed if you don't know your filename:
find ./ -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Find and replace in all Python files:
find ./ -iname "*.py" -type f -exec sed -i 's/abc/XYZ/gi' {} \;
Be careful if you replace URLs with "/" character.
An example of how to do it:
sed -i "s%http://domain.com%http://www.domain.com/folder/%g" "test.txt"
Extracted from: http://www.sysadmit.com/2015/07/linux-reemplazar-texto-en-archivos-con-sed.html
If the file you are working on is not so big, and temporarily storing it in a variable is no problem, then you can use Bash string substitution on the whole file at once - there's no need to go over it line by line:
file_contents=$(</tmp/file.txt)
echo "${file_contents//abc/XYZ}" > /tmp/file.txt
The whole file contents will be treated as one long string, including linebreaks.
XYZ can be a variable eg $replacement, and one advantage of not using sed here is that you need not be concerned that the search or replace string might contain the sed pattern delimiter character (usually, but not necessarily, /). A disadvantage is not being able to use regular expressions or any of sed's more sophisticated operations.
You may also use the ed command to do in-file search and replace:
# delete all lines matching foobar
ed -s test.txt <<< $'g/foobar/d\nw'
See more in "Editing files via scripts with ed".
To edit text in the file non-interactively, you need in-place text editor such as vim.
Here is simple example how to use it from the command line:
vim -esnc '%s/foo/bar/g|:wq' file.txt
This is equivalent to #slim answer of ex editor which is basically the same thing.
Here are few ex practical examples.
Replacing text foo with bar in the file:
ex -s +%s/foo/bar/ge -cwq file.txt
Removing trailing whitespaces for multiple files:
ex +'bufdo!%s/\s\+$//e' -cxa *.txt
Troubleshooting (when terminal is stuck):
Add -V1 param to show verbose messages.
Force quit by: -cwq!.
See also:
How to edit files non-interactively (e.g. in pipeline)? at Vi SE
Try the following shell command:
find ./ -type f -name "file*.txt" | xargs sed -i -e 's/abc/xyz/g'
You can use python within the bash script too. I didn't have much success with some of the top answers here, and found this to work without the need for loops:
#!/bin/bash
python
filetosearch = '/home/ubuntu/ip_table.txt'
texttoreplace = 'tcp443'
texttoinsert = 'udp1194'
s = open(filetosearch).read()
s = s.replace(texttoreplace, texttoinsert)
f = open(filetosearch, 'w')
f.write(s)
f.close()
quit()
Simplest way to replace multiple text in a file using sed command
Command -
sed -i 's#a/b/c#D/E#g;s#/x/y/z#D:/X#g;' filename
In the above command s#a/b/c#D/E#g where I am replacing a/b/c with D/E and then after the ; we again doing the same thing
You can use rpl command. For example you want to change domain name in whole php project.
rpl -ivRpd -x'.php' 'old.domain.name' 'new.domain.name' ./path_to_your_project_folder/
This is not clear bash of cause, but it's a very quick and usefull. :)
For MAC users in case you don't read the comments :)
As mentioned by #Austin, if you get the Invalid command code error
For the in-place replacements BSD sed requires a file extension after the -i flag to save to a backup file with given extension.
sed -i '.bak' 's/find/replace' /file.txt
You can use '' empty string if you want to skip backup.
sed -i '' 's/find/replace' /file.txt
All merit to #Austin
Open file using vim editor. In command mode
:%s/abc/xyz/g
This is the simplest
In case of doing changes in multiple files together we can do in a single line as:-
user_name='whoami'
for file in file1.txt file2.txt file3.txt; do sed -i -e 's/default_user/${user_name}/g' $file; done
Added if in case could be useful.
I am trying to open all the files listed in file a.lst:
symptom1.log
symptom2.log
symptom3.log
symptom4.log
But trying the following command:
cat a.lst | tr "\n" " " | vim -
opens only the stdin output
symptom1.log symptom2.log symptom3.log symptom4.log
It doesn't open symptom1.log, symptom2.log, symptom3.log & symptom4.log in vim.
How to open all the files listed in a.lst using vim?
You could use xargs to line upp the arguments to vi:
vim $(cat 1.t | xargs)
or
cat a.lst | xargs vim
If you want them open in split view, use -o (horizontal) or -O (vertical):
cat a.lst | xargs vim -o
cat a.lst | xargs vim -O
while read f ; do cat $f ; done < a.lst | vim -
I like a variation on Qiau's xargs option:
xargs vim < a.lst
This works because the input redirection is applied to the xargs command rather than vim.
If your shell is bash, another option is this:
vim $(<a.lst)
This works because within the $(...), input redirection without a command simply prints the results of the input, hence expanding the file into a list of files for vim to open.
UPDATE:
You mentioned in comments that you are using csh as your shell. So another option for you might be:
vim `cat a.lst`
This should work in POSIX shells as well, but I should point out that backquotes are deprecated in some other shells (notably bash) in favour of the $(...) alternative.
Note that redirection can happen in multiple places on your command line. This should also work in both csh and bash:
< a.lst xargs vim
vim may complain that its input is not coming from a terminal, but it appears to work for me anyway.
I want to get automatically to the positions of the results in Vim after grepping, on command line. Is there such feature?
Files to open in Vim on the lines given by grep:
% grep --colour -n checkWordInFile *
SearchToUser.java:170: public boolean checkWordInFile(String word, File file) {
SearchToUser.java~:17: public boolean checkWordInFile(String word, File file) {
SearchToUser.java~:41: if(checkWordInFile(word, f))
If you pipe the output from grep into vim
% grep -n checkWordInFile * | vim -
you can put the cursor on the filename and hit gF to jump to the line in that file that's referenced by that line of grep output. ^WF will open it in a new window.
From within vim you can do the same thing with
:tabedit
:r !grep -n checkWordInFile *
which is equivalent to but less convenient than
:lgrep checkWordInFile *
:lopen
which brings up the superfantastic quickfix window so you can conveniently browse through search results.
You can alternatively get slower but in-some-ways-more-flexible results by using vim's native grep:
:lvimgrep checkWordInFile *
:lopen
This one uses vim REs and paths (eg allowing **). It can take 2-4 times longer to run (maybe more), but you get to use fancy \(\)\#<=s and birds of a feather.
Have a look at "Grep search tools integration with Vim" and "Find in files within Vim". Basically vim provides these commands for searching files:
:grep
:lgrep
:vimgrep
:lvimgrep
The articles feature more information regarding their usage.
You could do this:
% vim "+/checkWordInFile" $(grep -l checkWordInFile *)
This will put in the vim command line a list of all the files that match the regex. The "+/..." option will tell vim to search from the start of each file until it finds the first line that matches the regex.
Correction:
The +/... option will only search the first file for the regex. To search in every file you need this:
% vim "-c bufdo /checkWordInFile" $(grep -l checkWordInFile *)
If this is something you need to do often you could write a bash function so that you only need to specify the regex once (assuming that the regex is valid for both grep and vim).
I think this is what you are looking for:
http://www.vim.org/scripts/script.php?script_id=2184
When you open a file:line, for instance when coping and pasting from an error from your compiler (or grep output) vim tries to open a file with a colon in its name. With this little script in your plugins folder if the stuff after the colon is a number and a file exists with the name especified before the colon vim will open this file and take you to the line you wished in the first place.
It's definitely what I was looking for.
I highly recommend ack.vim over grep for this functionality.
http://github.com/mileszs/ack.vim
http://betterthangrep.com/
You probably want to make functions for these. :)
Sequential vim calls (console)
grep -rn "implements" app | # Or any (with "-n") you like
awk '{
split($0,a,":"); # split on ":"
print "</dev/tty vim", a[1], "+" a[2] # results in lines with "</dev/tty vim <foundfile> +<linenumber>
}' |
parallel --halt-on-error 1 -j1 --tty bash -ec # halt on error and "-e" important to make it possible to quit in the middle
Use :cq from vim to stop editing.
Concurrent opening in tabs (gvim)
Start the server:
gvim --servername GVIM
Open the tabs:
grep -rn "implements" app | # again, any grep you like (with "-n")
awk "{ # double quotes because of $PWD
split(\$0,a,\":\"); # split on ":"
print \":tabedit $PWD/\" a[1] \"<CR>\" a[2] \"G\" # Vim commands. Open file, then jump to line
}" |
parallel gvim --servername GVIM --remote-send # of course the servername needs to match
If you use git, results are often more meaningful when you search only in the files tracked by git. To open files at the given line which is a search result of git grep in vim you will need the fugitive plugin, then
:copen
:Ggrep pattern
Will give you the list in a buffer and you can choose to open files from your git grep results.
In this particular example:
vim SearchToUser.java +170
I was wondering if there's a way to see the output of any command,
straight inside vim, rather than first redirecting it into a file and
then opening that file.
E.x. I need something like
$ gvim < diff -r dir1/ dir2/
This gives ambiguous redirect error message
I just want to see the diffs between dir1 and dir2 straight inside
gvim.
Can any one provide a nice hack?
diff file1 file2 | vim -R -
The -R makes it read-only so you don't accidentally modify the input (which may or may not be your desired behavior). The single dash tells vim to reads its input over standard input. Works for other commands, too.
Also, when already in Vim:
:r! diff file1 file2
vim -d file1 file2
Although I would also suggest vimdiff or vim -d for the case of looking at a diff, I just have to share this (more general) approach for using vim usage in pipes: vipe (from the moreutils package in Ubuntu).
For example:
find -name '*.png' | vipe | xargs rm
would allow you to first edit (in vim) the list of .png files found before passing it to xargs rm.
jst use gvimdiff instead
or vimdiff
to paste the output of a command straight into vim, for example ls, try
:%r!ls
BTW, there is a DirDiff plugin.
You can do this with
diff -r dir1/ dir2/ | gvim -
the '-' option to vim (or gvim) tells vim to open STDIN
I often use vimdiff -g <file1> <file2>
One of the most simple and convenient ways is to do it like this:
vimdiff -R <file1> <file2>
Again the '-R' flag is to open it for read-only mode to avoid any accidental changes.
What you are looking for is called process substitution:
vim <(diff -r dir1/ dir2/)
But the DirDiff plugin mentioned by Luc is much more useful for comparing directories.