I have a text file with simple text lines. I want to create a function for vim (and gvim) text editor which
can be sent a variable number of patterns and
it should find lines will all patterns (in any order)
and keep only these lines
while deleting the rest.
I searched the net and found some useful links but none that could do all above:
Following will find and delete all lines not containing the pattern:
:v/pattern/d
Multiple searching and highlighting can be done with scripts as MultipleSearch http://www.vim.org/scripts/script.php?script_id=479 . Boolean logic searching can be done with LogiPat script https://vim.sourceforge.io/scripts/script.php?script_id=1290 . A filtering package is also available but I could not make it work: http://www.vim.org/scripts/script.php?script_id=2759
To search and keep lines with multiple patterns with AND :
":v/.*pattern1\&.*pattern2/d"
but I have enter the code everytime.
How can I create a function that recursively runs :v/pattern/d to find only lines that contain all the patterns? I expect the function can be run as:
:Myfn pattern1 pattern2 pattern3
Edit: I have tried to write following code for this function, trying to use Linux grep command from within vim:
:function Myfn (Title, ...)
: let outstr=""
: for s in a:000
: outstr=!echo outstr | grep s
: endfor
: return outstr
: endfunction
But I get following error:
Not an editor command: : outstr=!echo outstr | grep s
with OR
:v/pattern1\|pattern2\|pattern3/d
with AND
:v/pattern1\&.*pattern2\&.*pattern/d
any order
:v/.*pattern1\&.*pattern2\&.*pattern3/d
(sorry looks like you already had this in of your solutions)
You can create a command Keepword:
com! -nargs=* Keepword v/\v<args>/d
and call it like:
Keepword word1|word2|word3
After using the command all lines (except those with the words provided) will be deleted.
Related
I want pick the specific format of file among the list of files in a directory. Please find the below example.
I have a below list of files (6 files).
Set-1
1) MAG_L_NT_AA_SUM_2017_01_20.dat
2) MAG_L_NT_AA_2017_01_20.dat
Set-2
1) MAG_L_NT_BB_SUM_2017_01_20.dat
2) MAG_L_NT_BB_2017_01_20.dat
Set-3
1) MAG_L_NT_CC_SUM_2017_01_20.dat
2) MAG_L_NT_CC_2017_01_20.dat
From the above three sets I need only 3 files.
1) MAG_L_NT_AA_2017_01_20.dat
2) MAG_L_NT_BB_2017_01_20.dat
3) MAG_L_NT_CC_2017_01_20.dat
Note: There can be multiple lines of commands because i have create the script for above req. Thanks
Probably easiest and least complex solution to your problem is combining find (a tool for searching for files in a directory hierarchy) and grep (tool for printing lines that match a pattern). You also can read those tools manuals by typing man find and man grep.
Before going straight to solution we need to understand, how we will approach your problem. To find pattern in a name of file we search we will use find command with option -name:
-name pattern
Base of file name (the path with the leading directories removed) matches shell pattern pattern. The metacharacters ('*', '?', and '[]')
match a '.' at the start of the base name (this is a change in
findutils-4.2.2; see section STANDARDS CONFORMANCE below). To ignore a
directory and the files under it, use -prune; see an example in the
description of -path. Braces are not recognised as being special,
despite the fact that some shells including Bash imbue braces with a
special meaning in shell patterns. The filename matching is performed
with the use of the fnmatch(3) library function. Don't forget to
enclose the pattern in quotes in order to protect it from expansion by
the shell.
For instance, if we want to search for a file containing string 'abc' in directory called 'words_directory', we will enter following:
$ find words_directory -name "*abc*"
And if we want to search all directories in directory:
$ find words_directory/* -name "*abc*"
So first, we will need to find all files, which begin with string "MAG_L_NT_" and end with ".dat", therefore to find all matching names in /your/specified/path/ which contains many subdirectories, which could contain files that match this pattern:
$ find /your/specified/path/* -name "MAG_L_NT_*.dat"
However this prints all found filenames, but we still get names containing "SUM" string, there comes in grep. To exclude names containing unwanted string we will use option -v:
-v, --invert-match
Invert the sense of matching, to select non-matching lines. (-v is
specified by POSIX .)
To use grep to filter out first commands output we will use pipe () |:
The standard shell syntax for pipelines is to list multiple commands,
separated by vertical bars ("pipes" in common Unix verbiage). For
example, to list files in the current directory (ls), retain only the
lines of ls output containing the string "key" (grep), and view the
result in a scrolling page (less), a user types the following into the
command line of a terminal:
ls -l | grep key | less
"ls -l" produces a process, the output (stdout) of which is piped to
the input (stdin) of the process for "grep key"; and likewise for the
process for "less". Each process takes input from the previous process
and produces output for the next process via standard streams. Each
"|" tells the shell to connect the standard output of the command on
the left to the standard input of the command on the right by an
inter-process communication mechanism called an (anonymous) pipe,
implemented in the operating system. Pipes are unidirectional; data
flows through the pipeline from left to right.
process1 | process2 | process3
After you got acquainted to mentioned commands and options which will be used to achieve your goal, you are ready for solution:
$ find /your/specified/path/* -name "MAG_L_NT_*.dat" | grep -v "SUM"
This command will produce output of all names which begin "MAG_L_NT_" and end with ".dat". grep -v will use first command output as input and remove all lines containing "SUM" string.
I want to create a new file using vi editor from command line and add a string to it multiple times say 100. Using vi -S command.script file.txt is supposed to do the trick where a new file file.txt will be created and the commands given in command.script file can write to this file. My command.script contains
:%100a hello world
:wq
But its's not working, what I am doing wrong?
If you interactively execute :%100a hello world in a Vim session, you'll get E488: Trailing characters. Looking up :help :a:
:{range}a[ppend][!] Insert several lines of text below the specified
line. If the {range} is missing, the text will be
inserted after the current line. [...]
These two commands will keep on asking for lines, until you type a line
containing only a ".".
tells you that the text has to be put in following lines (and concluded by a line with only a . character).
Or did you mean to use the normal mode a command? (That one takes a [count] to multiply; your %100 range is wrong, too!)
You can also use the low-level function append(), repeating the string with repeat().
summary
$append
hello world
[...]
hello world
.
execute "$normal! 100ahello world\<CR>"
" Easier with o instead of a:
$normal! 100ohello world
call append('$', repeat(['hello world'], 100))
non-Vim alternatives
But honestly, if that is your real use case (and not just a simplified toy example), you don't need Vim at all for this. Here's one example for the Bash shell:
$ for i in $(seq 100); do echo "hello world" >> file.txt; done
I have a big python file as follows:
#login_required
#user_passes_test(lambda u: u.is_superuser)
def foo():
//function body
#login_required
#user_passes_test(lambda u: u.is_superuser)
def foobar():
//function body
.
.
.
Like this there are many functions in the file. I want to comment all the lines which contains the pattern login_required or user_passes_test. How to comment those lines?
I use tComment plugin. So I can toggle line comment using gcc key-mapping. Can it be used?
There are also other files in the project which contains similar functions. So how can I comment these lines in all files in the project?
And again if I need to uncomment those lines how do I?
The :global/{pat}/{cmd} command will run a command, {cmd}, on every line matching pattern, {pat}. You can execute your tComment command via the :normal command. All together it looks like this:
:g/#login_required/norm gcc
For more help see:
:h :g
:h :norm
If you want to comment certain lines, then uncomment those same lines later, I'd use some kind of "marker" in the comment to make the job easier.
So to comment, for example:
1,$s/^\(.*#login_require\)/#FOO \1/
Then to uncomment:
1,$s/^#FOO //
You would choose #FOO so as not to be using it anywhere else for another purpose. You can even pick something simpler like ##... really anything that starts with # that you're not already using.
This will not be with VIM, but i think easier way to comment out in all project,multiple files:
sed -i 's/#login_required/#login_required/g' *
or for files in directories:
find ./ -type f -exec sed -i 's/#login_required/#login_required/g' {} \;
Comment lines containing string:
:%s/\(.*string\)/# \1/c
%s - global substitute
\(.*string\) - pattern to match
# \1 - replacement, \1 is for the matched pattern
c - confirm before substitution
Similarly, to uncomment:
:%s/# \(.*string\)/\1/c
So far, I have been manually refactoring code by using the find-and-replace operation
%s:/stringiwanttoreplace/newstring/g
in vim.
But this is a slow and laborious process if I have stringiwanttoreplace in many files inside a specific directory.
My current/typical slow and laborious process involves a grep:-
grep -rn "stringiwanttoreplace" .
in my terminal to reveal all the locations/filenames where stringiwanttoreplace are; and now that I know which files contain stringiwanttoreplace, I will open each file one-by-one to perform the find-and-replace operation in each file.
Is there a more efficient workflow (in vim) to get this done?
CLARIFICATION: I would prefer a vim-based solution instead of a bash script/one-liner.
Here's the full sequence of commands that I would use:
/stringiwanttoreplace
:vimgrep /<c-r>// **
:Qargs
:argdo %s//newstring/g
:argdo update
In the first line, we search for the target pattern. That populates the last search pattern register (:help quote/), which means that we won't have to type it out in full again.
The :vimgrep command searches the entire project for the specified pattern. Type <c-r>/ as ctlr+r followed by / - this inserts the contents of the last search pattern register onto the command line. The first and last / symbols are delimiters for the search field. The trailing ** tells Vim to look inside every file and directory below the current directory.
At this point, the quickfix list will be populated with search matches from all matching files. :Qargs is a custom command, which populates the argument list with all of the files listed in the quickfix list. Here's the implementation:
command! -nargs=0 -bar Qargs execute 'args ' . QuickfixFilenames()
function! QuickfixFilenames()
" Building a hash ensures we get each buffer only once
let buffer_numbers = {}
for quickfix_item in getqflist()
let buffer_numbers[quickfix_item['bufnr']] = bufname(quickfix_item['bufnr'])
endfor
return join(values(buffer_numbers))
endfunction
Add that to your vimrc file.
Having run :Qargs, our argument list should now contain all of the files that include our target string. So we can run the substitution command with :argdo, to execute the command in each file. We can leave the search field of the substitution command blank, and it will automatically use the most recent search pattern. If you want, you could include the c flag when you run the substitution command, then you'll be prompted for confirmation.
Finally, the :argdo update command saves each file that was changed.
As #Peter Rincker pointed out, you should ensure that Vim's 'hidden' option is enabled, otherwise it will raise an error when you try to switch to another buffer before writing any changes to the active buffer.
Also, note that the last 3 commands can be executed in a single command line, by separating them with a pipe character.
:Qargs | argdo %s//replacement/gc | update
The :Qargs command is pinched from this answer (by me), which in turn was inspired by this answer by DrAl. A very similar solution was posted by #ib, which suggests to me that Vim should really implement something like :quickfixdo natively.
If you really want to do it in Vim you can follow the suggestions here.
You can call this from within Vim (:!find ...) but you don't need to:
find . -type f | xargs sed -i 's/stringiwanttoreplace/newstring/g'
Fine-tune the file selection with the dozens of parameters described in
man find
(e.g., replace only in HTML files: -name \*.html)
This solution will try to attempt the replacement in all files. You can filter that through grep before, but that is just doing twice the work for no gain.
By the way: sed uses almost the same syntax for regular expressions as Vim (stemming from the same history).
You could open all the files and type
:bufdo :s/stringiwanttoreplace/newstring/g
It performs the search/replace in all your buffers.
You don't need vim to do this, you can use command line tools. Using sed in a loop on the list of files to do this for you automatically. Something like this:
for each in `grep -l "stringiwanttoreplace" *` ;
do
cat $each | sed -e "s/stringiwanttoreplace/newstring/g" > $each
; done
vim7 has recursive grep built-in
:vimgrep /pattern/[j][g] file file1 file2 ... fileN
the result will be shown in a quickfix-window (:help quickfix)
to do the search recursively use the **-wildcard like
**/*.c to search through the current folder and recursively through all subdirectories.
I am using the Ack plugin in Vim, which helps me to quickly search for strings in my project. However, sometimes I want to replace all or some occurrences of the found strings. You can do some kind of global search and replace using the Vim arglist like this (source)
:
:args app/views/*/*
:argdo %s/, :expire.*)/)/ge | update
But instead of using args, I would prefer to do a search via Ack and then do the replace in all files that have been found. Is there a way to do it similar to the argdo command?
I've decided to use ack and perl to solve this problem outside of Vim so I could use the more powerful Perl regular expressions instead of the GNU subset. You could map this to a key stroke in your .vimrc.
ack -l 'pattern' | xargs perl -pi -E 's/pattern/replacement/g'
Explanation
ack
ack is an awesome command line tool that is a mix of grep, find, and full Perl regular expressions (not just the GNU subset). It's written in pure Perl, it's fast, it has match highlighting, it works on Windows and it's friendlier to programmers than the traditional command line tools. Install it on Ubuntu with sudo apt-get install ack-grep.
xargs
xargs is an old Unix command line tool. It reads items from standard input and executes the command specified followed by the items read for standard input. So basically the list of files generated by ack are being appended to the end of the perl -pi -E 's/pattern/replacement/g' command.
perl -pi -E
Perl is a programming language.
The -p option causes Perl to create a loop around your program which iterates over filename arguments.
The -i option causes Perl to edit the file in place. You can modify this to create backups.
The -E option causes Perl to execute the one line of code specified as the program. In our case the program is just a Perl regex substitution.
For more information on Perl command line options, see perldoc perlrun. For more information on Perl, see http://www.perl.org/.
Now, Vim has this new command cdo that will run the given command to each line of the quickfix list.
So you can use
:Ack pattern
:cdo s/pattern/newpattern/g
I don't believe there's a built in way of doing this, but it should be easy to make one.
What you need to do is create a command that calls a custom function. The function should then use the getqflist() function to get all of the entries in the quickfix list and exe to do the dirty work. Be careful what you pass as an argument!
" Define a command to make it easier to use
command! -nargs=+ QFDo call QFDo(<q-args>)
" Function that does the work
function! QFDo(command)
" Create a dictionary so that we can
" get the list of buffers rather than the
" list of lines in buffers (easy way
" to get unique entries)
let buffer_numbers = {}
" For each entry, use the buffer number as
" a dictionary key (won't get repeats)
for fixlist_entry in getqflist()
let buffer_numbers[fixlist_entry['bufnr']] = 1
endfor
" Make it into a list as it seems cleaner
let buffer_number_list = keys(buffer_numbers)
" For each buffer
for num in buffer_number_list
" Select the buffer
exe 'buffer' num
" Run the command that's passed as an argument
exe a:command
" Save if necessary
update
endfor
endfunction
You could using ack by this way
:args `ack -l User app/`
:argdo %s/, :expire.*)/)/ge | update
Or use ag
:args `ag -l User app/`
:argdo %s/, :expire.*)/)/gec | w
I use MacVim (activated with mvim in a shell). I pipe the results of ack to mvim:
mvim -f $(ack -l $#)
Then in MacVim, I search/replace using bufdo:
:bufdo %s/SEARCH/REPLACE/gce | update
Omit the c option if confirmation is not needed.