I'm trying to clean up a bunch of yaml files in vim. The yaml files are fixtures for tests, and they contain literal strings of XML. They look like:
-
- "<xml>blah blah blah …1000 characters later</xml>"
- "<more>…</more>"
I want them to look like:
-
- >
<xml>
<nicely>formatted</nicely>
…
</xml>
- >
<more>
…
</more>
I strip the quotes, put the cursor at the beginning of the unformatted xml and hit >, enter, <tab>, I get
-
- >
<xml>…
But then I try to format the xml the only way I know how: shift-v to select the line. :'<,'>!xmllint --nocdata --format - | awk 'NR>1' to format the xml, and I get
-
- >
<xml>
<nicely>formatted</nicely>
…
</xml>
and then I have to reselect the xml and use 2>> to reindent.
Is there a better way to maintain the indentation when executing a ! command in Vim?
Your question doesn't state this outright, but it somewhat implies that vim is the reason the indentation is stripped, so just to be clear, the indentation is being taken out by xmllint.
You could try to save the indentation first, but that could be confusing if the indentation is different amongst the different lines. In this case it probably makes more sense from looking at the commands you are running to just add the indentation later - you can do it automatically by adding it to your command filter, such as:
:'<,'>!xmllint --nocdata --format - | awk 'NR>1' | sed -e 's/^/\t/'
Related
want to search and replace in vim, the /find finds the pattern but :s%//g will not?
have a script that monitors software raid (if interested check it out https://dwaves.org/2019/09/06/linux-server-monitor-software-raid-mail-notification-on-failure/)
echo "=== smart status of all drives ==="| tee -a /scripts/monitor/raid_status_mail.log
# want to search and replace the /path/to/file.sh with $LOGFILE
# searching for the pattern works like charm
/\/scripts\/monitor\/raid_status_mail.log
# but replacing it won't
:s%/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g
# what does one do wrong?
should replace /scripts/monitor/raid_status_mail.log with $LOGFILE
The substitution operation needs to be prefixed with %s and not the other way around as s%. So doing
%s/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g
should work as expected. Or just the Vim's equivalent ex in command line mode as
printf '%s\n' "%s/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g" w q | ex -s file
You inverted the beginning s%. Use %s instead.
Also, you use / as separation for the different fields, it works but makes the command less readable. You can replace the separation character by anything else. You could use : for example:
%s:/scripts/monitor/raid_status_mail.log:$LOGFILE:g
One last tip: install vim-over
This will highlight your searches in live while replacing something in vim.
In bash I am trying to parse following file:
Input:
</a></td></tr><tr><td>stuff.txt (15.18 KB)</td><td>12/01/2015</td><td>Large things</td><td>158520312</td><td><a class="btn-down" download href="https://resource.com/stones">
</a></td></tr><tr><td>flowers.pdf (83.03 MB)</td><td>23/03/2011</td><td>Large flowers</td><td>872448000</td><td><a class="btn-down" download href="https://resource.com/flosers with stuff">
</a></td></tr><tr><td>apples.pdf (281.16 MB)</td><td>21/04/2012</td><td>Large things like apples</td><td>299009564</td><td><a class="btn-down" download href="https://resource.com/apples">
</a></td></tr><tr><td>stones.pdf (634.99 MB)</td><td>11/07/2011</td><td>Large stones from mountains</td><td>67100270</td><td><a class="btn-down" download href="https://stuff.com/findstones">
Wanted output:
12/01/2015 158520312 "https://resource.com/stones"
23/03/2011 872448000 "https://resource.com/flosers with stuff"
21/04/2012 299009564 "https://resource.com/apples~withstuff"
11/07/2011 67100270 "https://stuff.com/findstones"
I got to the point that I have:
# less input.txt | sed -e "s/><tr><td//" -e "s/\///" -e "s/a>//" -e "s/<\/td><\/tr>//g" -e "s/<\/td><td>//g" -e "s/>$//g" -e "s/<a class=\"btn-down\" download href=//g"
<stuff.txt (15.18 KB)12/01/2015Large things158520312"https://resource.com/stones"
<flowers.pdf (83.03 MB)23/03/2011Large flowers872448000"https://resource.com/flosers with stuff"
<apples.pdf (281.16 MB)21/04/2012Large things like apples299009564"https://resource.com/apples"
<stones.pdf (634.99 MB)11/07/2011Large stones from mountains67100270"https://stuff.com/findstones"
Is there a easier way to parse it? I feel that it can be done much simpler and I am not even in the middle of parsing.
Could you please try following and let us know if this helps you.
awk -F"[><]" '{sub(/.*=/,"",$28);print $15,$23,$28}' Input_file
I'm sure the best way to solve your problem is to use an HTML parser. Solution for shown sample of file:
sed -r 's/.*(..\/..\/....).*>([0-9]*)<\/.*href=([^>]*)>/\1 \2 \3/I' input.txt
Personally, I'd use perl, but that's not what you asked, so...
A pedantic stepwise approach, so that you can edit bits of the logic when needed.
Assuming the input is a file named x:
</a></td></tr><tr><td>stuff.txt (15.18 KB)</td><td>12/01/2015</td><td>Large things</td><td>158520312</td><td><a class="btn-down" download href="https://resource.com/stones">
</a></td></tr><tr><td>stuff.txt (15.18 KB)</td><td>12/01/2015</td><td>Large things</td><td>158520312</td><td><a class="btn-down" download href="https://resource.com/stones">
</a></td></tr><tr><td>flowers.pdf (83.03 MB)</td><td>23/03/2011</td><td>Large flowers</td><td>872448000</td><td><a class="btn-down" download href="https://resource.com/flosers with stuff">
</a></td></tr><tr><td>apples.pdf (281.16 MB)</td><td>21/04/2012</td><td>Large things like apples</td><td>299009564</td><td><a class="btn-down" download href="https://resource.com/apples">
</a></td></tr><tr><td>stones.pdf (634.99 MB)</td><td>11/07/2011</td><td>Large stones from mountains</td><td>67100270</td><td><a class="btn-down" download href="https://stuff.com/findstones">
Try this:
sed -E '
s/>$//;
s/href=/>/;
s/(<[^>]+>)+/~/g;
s/~[^~]+~//;
s/~[^~]+~/ /;
s/~/ /;
' x
Output:
12/01/2015 158520312 "https://resource.com/stones"
23/03/2011 872448000 "https://resource.com/flosers with stuff"
21/04/2012 299009564 "https://resource.com/apples"
11/07/2011 67100270 "https://stuff.com/findstones"
Explained:
sed -E '
This uses extended regexes, and opens a script of sed code so that I can list each pattern individually. Each will be executed in order on each line, so it's not super efficient, but it's "readable" as regex code goes, and reasonably maintainable once you understand it, and so easy to edit when something needs tweaking.
s/>$//;
Strip the closing > off the end, to preserve the URL before squashing out all the other tags.
s/href=/>/;
use the href= as a hook to insert the > back so we can squash out all the tags in one pass.
s/(<[^>]+>)+/~/g;
Convert ALL the strings of tags and everything still in them to a simple delimiter each.
s/~[^~]+~//;
Eliminate the leading and second delimiter and the first unneeded field between them.
s/~[^~]+~/ /;
Eliminate the third and fourth delimiters and the unneeded third field between them, replacing them with the space you wanted in the output.
Those two are very similar, and could certainly be combined with minimal shenannigans, but I left them nigh-redundant for easier explication.
s/~/ /;
Convert the remaining delimiter to the other space you wanted between the remaining fields.
' x
Close the script and give it the filename to read.
Obviously, this leaves a LOT of room for improvement, and is in many ways stylistically repulsive, but hopefully it is a simple explanation of tricks you can hack into a maintainably useful solution to your issue.
Good luck.
Is there a way to limit :Ag output so it always takes one line and doesn't blow up the quickfix window?
At the moment it looks like this and it's awful. I can't see filenames, everything is super slow and just sucks:
Update For the record, I scrolled Quickfix window a bit to illustrate the point better. And while it is usable via :cn :cp, I would like to be able to quickly glance over the results with j k.
Looking over the man page, there does not seem to be any way to limit the output built into Ag itself.
Is there another way of limiting the line length? Actually, you do have the built in "cut" command in Linux, e.g. using it on the shell:
ag --column foo | cut -c 1-80
Limit all lines to 80.
Now we have to make ag.vim execute our specially crafted command, for which the g:agprg exists. So the first thing I thought of is this:
let g:agprg='ag --column \| cut -c 1-80' " doesn't work
The problem with this is that the ag.vim plugin just appends extra arguments to the end, and thus you end up executing something like ag --column | cut -c 1-80 something-i-searched-for. Is there a way to directly "insert" the arguments before the |?
One trick is to use a temporary shell function, like this:
f() { ag --column "$#" | cut -c 1-80 }; f something-i-search-for
Unfortunately, we still can't use this. ag.vim checks whether or not the first word is an actual command. So it complains that no executable by the name of "f()" exists. So my final solution:
let g:agprg='true ; f(){ ag --column "$#" \| cut -c 1-80 }; f'
As true always exists and doesn't do anything except return true, this works!
To your actual screenwidth instead of 80, you could use:
let g:agprg='true ; f(){ ag --column "$#" \| cut -c 1-'.(&columns - 6).' }; f'
I added the magic - 6 here to account for the extra characters Vim itself adds.
ag now supports a --width switch. rg has a similar --max-columns switch.
Assuming you are using this plugin. You should add this to your ~/.vimrc as specified by :h g:ag_qhandler
let g:ag_qhandler = 'copen 1'
However you can probably just do let :g:ag_qhandler = 'cc'. This will print the results at the in the bottom. When you move through the quickfix list via :cnext or :cprev it will print the current result as well.
For more help see:
:h g:ag_qhandler
:h :cope
Changing the geometry of the quickfix window won't help you fix your problem: the window is unusable not because of its size but because your search results are polluted by superfluous matches in minimized files.
Minimized JavaScript or CSS is the frontend development's equivalent of a binary and that kind of file should be ignored by search tools, indexing tools, file navigation tools and even version control tools, sometimes, because they are generally irrelevant.
Adding these lines to your ~/.agignore will make Ag search only in actual source files:
*.min*
*-min*
*_min*
*.min.*
bundle
min
vendor
tags
cscope.*
Adjust that list to your liking.
For some reason I have errors in tags file generated by ctags.
Have tried to solve the errors by changing ctags options, did no good.
Now I would like to try another approach - make vim ignore errors in the tags file.
Is it possible?
The tags file contains lines that are not tags, just sentences taken from somewhere in the code, I guess.
The tag file is quite large:
$ wc tags
591708 3711802 65594557 tags
e.g.:
$grep -vn -C 1 "\/" tags
510630-packet ISS/code/opensource/ssl/ssl.h /^ unsigned char *packet;$/;" m struct:ssl_st
510631:packet that is present on the network wire. Even if an inbound packet is denied
510632-packetSrcAdr ISS/code/future/ospf/inc/osfssnmp.h /^ tIPADDR packetSrcAdr;$/;" m struct:_IfConfErrTrapInfo
found out that there is a faq.tex file that many 'sentences' come from it. I'm trying to exclude it from ctags (--exclude)
If the corrupted lines in the tags file match some pattern (regex), you could fix-up the tags file. e.g. place this in your Makefile:
tags:
ctags -f $#.tmp ...
grep -v 'pattern of bogus tag lines' $#.tmp > $#
rm $#.tmp
The best solution is to look for the origin of the lines with the error and try to eliminate those lines / files, causing the errors in the tags file.
So far, I have been manually refactoring code by using the find-and-replace operation
%s:/stringiwanttoreplace/newstring/g
in vim.
But this is a slow and laborious process if I have stringiwanttoreplace in many files inside a specific directory.
My current/typical slow and laborious process involves a grep:-
grep -rn "stringiwanttoreplace" .
in my terminal to reveal all the locations/filenames where stringiwanttoreplace are; and now that I know which files contain stringiwanttoreplace, I will open each file one-by-one to perform the find-and-replace operation in each file.
Is there a more efficient workflow (in vim) to get this done?
CLARIFICATION: I would prefer a vim-based solution instead of a bash script/one-liner.
Here's the full sequence of commands that I would use:
/stringiwanttoreplace
:vimgrep /<c-r>// **
:Qargs
:argdo %s//newstring/g
:argdo update
In the first line, we search for the target pattern. That populates the last search pattern register (:help quote/), which means that we won't have to type it out in full again.
The :vimgrep command searches the entire project for the specified pattern. Type <c-r>/ as ctlr+r followed by / - this inserts the contents of the last search pattern register onto the command line. The first and last / symbols are delimiters for the search field. The trailing ** tells Vim to look inside every file and directory below the current directory.
At this point, the quickfix list will be populated with search matches from all matching files. :Qargs is a custom command, which populates the argument list with all of the files listed in the quickfix list. Here's the implementation:
command! -nargs=0 -bar Qargs execute 'args ' . QuickfixFilenames()
function! QuickfixFilenames()
" Building a hash ensures we get each buffer only once
let buffer_numbers = {}
for quickfix_item in getqflist()
let buffer_numbers[quickfix_item['bufnr']] = bufname(quickfix_item['bufnr'])
endfor
return join(values(buffer_numbers))
endfunction
Add that to your vimrc file.
Having run :Qargs, our argument list should now contain all of the files that include our target string. So we can run the substitution command with :argdo, to execute the command in each file. We can leave the search field of the substitution command blank, and it will automatically use the most recent search pattern. If you want, you could include the c flag when you run the substitution command, then you'll be prompted for confirmation.
Finally, the :argdo update command saves each file that was changed.
As #Peter Rincker pointed out, you should ensure that Vim's 'hidden' option is enabled, otherwise it will raise an error when you try to switch to another buffer before writing any changes to the active buffer.
Also, note that the last 3 commands can be executed in a single command line, by separating them with a pipe character.
:Qargs | argdo %s//replacement/gc | update
The :Qargs command is pinched from this answer (by me), which in turn was inspired by this answer by DrAl. A very similar solution was posted by #ib, which suggests to me that Vim should really implement something like :quickfixdo natively.
If you really want to do it in Vim you can follow the suggestions here.
You can call this from within Vim (:!find ...) but you don't need to:
find . -type f | xargs sed -i 's/stringiwanttoreplace/newstring/g'
Fine-tune the file selection with the dozens of parameters described in
man find
(e.g., replace only in HTML files: -name \*.html)
This solution will try to attempt the replacement in all files. You can filter that through grep before, but that is just doing twice the work for no gain.
By the way: sed uses almost the same syntax for regular expressions as Vim (stemming from the same history).
You could open all the files and type
:bufdo :s/stringiwanttoreplace/newstring/g
It performs the search/replace in all your buffers.
You don't need vim to do this, you can use command line tools. Using sed in a loop on the list of files to do this for you automatically. Something like this:
for each in `grep -l "stringiwanttoreplace" *` ;
do
cat $each | sed -e "s/stringiwanttoreplace/newstring/g" > $each
; done
vim7 has recursive grep built-in
:vimgrep /pattern/[j][g] file file1 file2 ... fileN
the result will be shown in a quickfix-window (:help quickfix)
to do the search recursively use the **-wildcard like
**/*.c to search through the current folder and recursively through all subdirectories.