I have a requirement to batch edit a bunch of files using vim based on their content. The simplest example is that I'd like to perform a series of let's say substitutions on files but only if the first line of the file matches a certain pattern.
I'm trying to do this kind of thing:
vim -e -s $file < changes.vim
I should add that I have no access to tools like sed and awk and would like to perform the entire operation in vim.
I recommend that you find the list of files you need, and pass that list into the command you want. For this, a combination of awk and xargs would seem useful. There are probably clever shorter things you can do…
awk 'FNR>1 {nextfile} /pattern/ { print FILENAME ; nextfile }' filePattern | xargs -I{} vim -e -s {} < changes.vim
In the above, filePattern gives all the files you want (maybe *.c), /pattern/ is the regex of the match you are looking for. xargs will take "one output at a time" and substitute it into the following command at the place where I put the {}.
I want to give a tip of the hat to this link where I found the inspiration for this answer.
vim only solution
EDIT - after I posted this you said you need a "vim only" solution. Here it is…
Step 1: create a conditionalEdits.vim file with the following lines at the start:
let line_num = search('searchExpression') " any regex
if line_num == 1 " first line matched
center " put your editing commands here...
update " save changes
endif
quit
Of course, instead of just centering the first line, you will want to put all your editing commands inside the if statement.
Now, you execute this command with
vim -c '/path/to/my/conditionalEdits.vim' -s filePattern
where filePattern matches all the files you might be interested in (but you will know for sure after you have looked at line 1 inside…)
Obviously you can navigate through the file in the usual way and look for matches / patterns etc to your heart's content - but this is the basic idea.
Helpful links: http://www.ibm.com/developerworks/library/l-vim-script-1/
and http://learnvimscriptthehardway.stevelosh.com
I highly recommend that you do this in a separate directory, using copies of a handful of files first, to make sure this actually does what you think it does. I would hate to be responsible for a bunch of files being overwritten (you do back up, right?)
You can loop over all files, if you find the pattern, open vim. Once it is modified to your needs and closed, the next one will open.
#!/usr/bin/env bash
for file in *; do
if [[ "$(sed '1q' ${file})" == "pattern" ]]; then
vim ${file}
fi
done
Within Vim, you can determine the matching files via :vimgrep; to check for a match in the first line, the \%l atom is handy:
:vimgrep /\%1lcertain pattern/ {file-glob}
Then, you can iterate through all matches with :cfnext, or use the :QFDo command from here.
You can pass those commands either via vim -c {cmd} -c {cmd} ..., or in a separate script, as you outline in your question.
Related
I remember doing magic with vi by "programming" it with input commands but I don't remember exactly how.
My sepcial request are:
launch vi in a script with command to be executed.
do an insert in one file.
search for a string in the file.
use $VARIABLE in the vi command line to replace something in the command.
finish with :wq.
I my memory, I sent the command exactly like in vi and the ESC key was emulate by '[' or something near.
I used this command into script to edit and change files.
I'm going to see the -c option but for now I cannot use $VARIABLE and insert was impossible (with 'i' or 'o').
#!/bin/sh
#
cp ../data/* .
# Retrieve filename.
MODNAME=$(pwd | awk -F'-' '{ print $2 }')
mv minimod.c $MODNAME.c
# Edit and change filename into the file (from mimimod to the content of $VARIABLE).
vi $MODENAME.c -c ":1,$s/minimod/$MODNAME/" -c ':wq!'
This example is not functionning (it seems the $VARIABLE is not good in -c command).
Could you help me remember memory ;) ?
Thanks a lot.
Joe.
You should not use vi for non-interactive editing. There's already another tool for that, it's called sed or stream editor.
What you want to do is
sed -i 's/minimod/'$MODNAME'/g' $MODNAME.c
to replace your vi command.
Maybe you are looking for the ex command, that can be run non-interatively:
ex $MODENAME.c <<< "1,\$s/minimod/$MODNAME/
wq!
"
Or if you use an old shell that does not implement here strings:
ex $MODENAME.c << EOF
1,\$s/minimod/$MODNAME/
wq!
EOF
Or if you do not want to use here documents either:
echo "1,\$s/minimod/$MODNAME/" > cmds.txt
echo "wq!" >> cmds.txt
ex $MODENAME.c < cmds.txt
rm cmds.txt
One command per line in the standard input. Just remember not to write the leading :. Take a look at this page for a quick review of ex commands.
Granted, it would be better to use the proper tool for the job, that would be sed, as #IsaA answer suggests, or awk for more complex commands.
I need to find a string in a textfile, delete the line containing it, and save the file. The string is found (read from) another textfile, containing hundreds of different strings, one per row. The process is to go on from the first to the last string in the file.
Any (hopefully easy to use) text editors (on Windows OS) recommended ? To achive the task.
I am not into serious day-to-day editing. So I'd be ever so happy if the task could be accomplished with a easy-to-use but still reliable editor.
Thanks a bunch,
Frank
You can try notepad++ since it has a lot of plugins, also a great search algorithm. I did a similar task where I had to do a lot of search/replace stuff, and used a plugin I dug up from the internet, can't remember the name exactly (try google-ing I think it's replaacc for notepad++ or something similar).
On unix/linux/cygwin:
grep -v -f pattern_file unmodified_file > new_file
Remove all lines containing the patterns in pattern_file from unmodified_file, write to new_file.
grep -v outputs lines not matching any pattern. -f reads patterns from a file.
On windows this appears equivalent to running this at the command prompt:
FINDSTR /V /G:pattern_file unmodified_file > new_file
That's it. If you already have the two source files, it's a one-liner.
pattern_file is going to be whitespace and case sensitive unless you delve into other options, which are described with FINDSTR /?
Using sed:
sed -n '/PATTERN/n;p' FILE > FILE.new # then copy FILE.new to FILE
Tells sed to not output anything by default (-n), find the pattern (/PATTERN/) and skip this line if found (n), otherwise print the line (;p). If you have GNU sed you do can just call
sed -i -n '/PATTERN/n;p' FILE`
which automatically updates the file due to (-i /--in-place).
For example, i have a task to search all file under current directory where contains "foo" and then replace to "bar".
Now here is my current solution:
:vim /bar/ **/*
Use this to search all appearances of foo, and then replace it one by one to "bar"
:s/foo/bar/gc
Obviously it is not a good solution when replaces becomes large. So if there is a better solution to combine these two operations into one. But there is a precondition : Must give a hint before replacement just like what the c does in the second command. This is prevent replace some word that doesn't need to replace.
Open all files in vim:
$ vim *
Replace foo with bar
:argdo %s/foo/bar/ | update
Tutorial: http://vim.wikia.com/wiki/Search_and_replace_in_multiple_buffers
the easiest would be:
vim $(egrep -l '/foo/' **/*) -c 'bufdo %s/foo/bar/g'
but this will fail if there's whitespace on the filenames, a more robust approach would be:
files=()
while IFS= read -rd '' filename; do
files+=("$filename")
done < <(egrep -Zl '/foo/' **/*)
vim "${files[#]}" -c 'bufdo %s/foo/bar/g'
I have gotten comfortable using expressions in Vim to do replaces across large files and would like to know what utility program to learn that will allow me to do such a thing across folders of files in a similar fashion.
What command line program for search and replace is most like vims and works across folders?
sed -i.bak 's/foo/bar/g' *
will replace every foo with bar on every line in every file in the current directory in place and creates a backup file with .bak extension.
So I'd look into sed. But... Vim is very capable:
vim -c "bufdo!%s/foo/bar/g" -c "wqa" *
is almost the same.
bufdo! executes the following command (%s/foo/bar/g in this case => replace every foo with bar on every line (% is a special line address for every line)). wqa means: write all then quit. You can specify at most 10 -c switches to Vim, which will be executed in the specified order. So, basically this is Vim automation:
open every file in the current dir => every file has its own buffer
execute the substitution on every buffer
then write the files and quit
The only difference is the lack of backup file creation (which can be achieved easily with some .vimrc settings). But sed is faster.
probably, you should use vim not external command line program. see :help :find or :help :grep
for example, if you want to grep words across folders, pass **/*.txt.
you could do this by combining perl and bash
for example:
find . -type f -name "*.cpp" -print | xargs perl -i -pe 's/pattern/replace/g'
this will find all '.cpp' files starting at '.' and pass (pipe) each of them (the path) to perl command
I want to get automatically to the positions of the results in Vim after grepping, on command line. Is there such feature?
Files to open in Vim on the lines given by grep:
% grep --colour -n checkWordInFile *
SearchToUser.java:170: public boolean checkWordInFile(String word, File file) {
SearchToUser.java~:17: public boolean checkWordInFile(String word, File file) {
SearchToUser.java~:41: if(checkWordInFile(word, f))
If you pipe the output from grep into vim
% grep -n checkWordInFile * | vim -
you can put the cursor on the filename and hit gF to jump to the line in that file that's referenced by that line of grep output. ^WF will open it in a new window.
From within vim you can do the same thing with
:tabedit
:r !grep -n checkWordInFile *
which is equivalent to but less convenient than
:lgrep checkWordInFile *
:lopen
which brings up the superfantastic quickfix window so you can conveniently browse through search results.
You can alternatively get slower but in-some-ways-more-flexible results by using vim's native grep:
:lvimgrep checkWordInFile *
:lopen
This one uses vim REs and paths (eg allowing **). It can take 2-4 times longer to run (maybe more), but you get to use fancy \(\)\#<=s and birds of a feather.
Have a look at "Grep search tools integration with Vim" and "Find in files within Vim". Basically vim provides these commands for searching files:
:grep
:lgrep
:vimgrep
:lvimgrep
The articles feature more information regarding their usage.
You could do this:
% vim "+/checkWordInFile" $(grep -l checkWordInFile *)
This will put in the vim command line a list of all the files that match the regex. The "+/..." option will tell vim to search from the start of each file until it finds the first line that matches the regex.
Correction:
The +/... option will only search the first file for the regex. To search in every file you need this:
% vim "-c bufdo /checkWordInFile" $(grep -l checkWordInFile *)
If this is something you need to do often you could write a bash function so that you only need to specify the regex once (assuming that the regex is valid for both grep and vim).
I think this is what you are looking for:
http://www.vim.org/scripts/script.php?script_id=2184
When you open a file:line, for instance when coping and pasting from an error from your compiler (or grep output) vim tries to open a file with a colon in its name. With this little script in your plugins folder if the stuff after the colon is a number and a file exists with the name especified before the colon vim will open this file and take you to the line you wished in the first place.
It's definitely what I was looking for.
I highly recommend ack.vim over grep for this functionality.
http://github.com/mileszs/ack.vim
http://betterthangrep.com/
You probably want to make functions for these. :)
Sequential vim calls (console)
grep -rn "implements" app | # Or any (with "-n") you like
awk '{
split($0,a,":"); # split on ":"
print "</dev/tty vim", a[1], "+" a[2] # results in lines with "</dev/tty vim <foundfile> +<linenumber>
}' |
parallel --halt-on-error 1 -j1 --tty bash -ec # halt on error and "-e" important to make it possible to quit in the middle
Use :cq from vim to stop editing.
Concurrent opening in tabs (gvim)
Start the server:
gvim --servername GVIM
Open the tabs:
grep -rn "implements" app | # again, any grep you like (with "-n")
awk "{ # double quotes because of $PWD
split(\$0,a,\":\"); # split on ":"
print \":tabedit $PWD/\" a[1] \"<CR>\" a[2] \"G\" # Vim commands. Open file, then jump to line
}" |
parallel gvim --servername GVIM --remote-send # of course the servername needs to match
If you use git, results are often more meaningful when you search only in the files tracked by git. To open files at the given line which is a search result of git grep in vim you will need the fugitive plugin, then
:copen
:Ggrep pattern
Will give you the list in a buffer and you can choose to open files from your git grep results.
In this particular example:
vim SearchToUser.java +170