Replace visual selection with command output - vim

I would like to replace parts of one line with the result of the selection being piped into a command.
For example:
echo "hello $(echo "world" | base64)" | vim -
This will open a buffer with hello d29ybGQK in it. Now press wvw to visually select d29ybGQK.
Then I attempted :!base64 -d and I expected the buffer to contain hello world, which did not happen. Indeed, the whole line was piped into the command, and the whole line was replaced.
Is it possible to replace only the visual selection, and have only that selection piped into the command?
I also attempted c<c-r>=system('base64 -d') but that did not send the visual selection to the command's stdin.

Filtering with ! is always line-wise. Your solution with c and the
expression register is an excellent way to solve this. You only forgot
to pass the input to system(), which is its second optional argument.
Since you just changed the text selected, it went into the " register
automatically. All you need to do is to grab it back and pass it to
system with getreg():
c<C-R>=system('base64 -D', getreg('"'))
Note that base64 may echo a newline at the end. If you want to remove
it, either wrap the whole thing in trim(), a new function in Vim 8, or
use [:-2]:
c<C-R>=trim(system('base64 -D', getreg('"')))
c<C-R>=system('base64 -D', getreg('"'))[:-2]
This is a shorthand for [0:-2], meaning grab everything from character
0 to second-last in the resulting string.
Consider creating a visual map if you use it often:
vnoremap <leader>d c<C-R>=system('base64 -D', getreg('"'))[:-2]<CR>

For historical reasons, the Ex commands are inherently line-based; venerable vi also didn't have visual mode yet. That limitation includes filtering through an external command with :range!; it will always filter complete lines.
manual solution
For simple input like in your example, it's probably easiest to temporarily split the line, filter, and then reassemble. For example:
:substitute/ /\r/ | execute '.!base64 -d' | -1join
plugin solution
For a universal solution, you need to use or implement a plugin that grabs the selected text, filters it (probably through system()), and then replaces the selection with the result.
My SubstituteExpression plugin has a {Visual}g= mapping that can filter through Vimscript expressions, Vim functions and commands, and external commands.
express.vim by Tom McDonald offers an almost identical implementation. It also allows on-the-fly creation of operators via :MapExpress and :MapSubpress, something for which I would use my TextTransform plugin, which you need to install as a dependency, anyway. My plugin offers more advanced (cross-mode) repeats, and the :Ex-command expression variant, but has two large dependencies you also need to install.

Related

vim: Run multiple commands based off of one :global command

Apologies if this has been posted already, for I cannot find an answer, even on the vim wiki.
Is there a way I can run multiple commands in vim command-line mode off of a single :g search?
For example,
:%g/foo/ s/bar/\=#a/g | exe "norm /cat\<enter>\"ayiw"
Which (for what I intend it to do) should, on every line matching foo, replace bar with the contents of register a, and then find the next iteration of cat (even if it is many lines ahead), and put the surrounding word into register a.
Instead, this specific syntax completes the subsitution command using the current contents of the initial a register, and then executes the normal mode command on a single line (after the substitution has been completed).
This example is not my specific use-case but shows one instance where this functionality is useful. I realize I could put it all into a single exe, i.e., %g/foo/exe "norm :s/bar/\\=#a/g\<enter>/cat\<enter>\"ayiw", but I would like to do it the first way, as I feel it is more flexible.
I would prefer to do this using vanilla vim, but if a plugin exists for this, that is an okay alternative. Does anybody know if there is syntax to do such a thing?
Okay a "little bit" dirty, but does this work for you?
:let list = split(execute('g/cat/p'), '\n') | g/foo/ s/bar/\=matchstr(remove(list, 0), '\s\d\+\s\zs.*')/g
It first reads all occurences of cat save them in a list.
Then replace the first bar with the first cat... and so on.
The dirty part ist the matchstr command. the g//p also returns a number for the result so the list looks like this:
1 cat
2 cat
3 cat
...
that's why we have to remove a bit from the front. I would love to hear if someone knows a clean solution for that (I am also interested in a clean vimscript solution, does not have to be a oneliner).
You can do this (at least for multiple :s commands applied to a single :g). Example:
" SHORT STORY TITLES to single word of CapitalizedWords within <h3>s
.,$g/^\L\+$/s/[^A-Z0-9 ]\+//ge|s/\u\+/\L&/ge|s/\<\l\+\>/\u&/ge|s/ \+//ge|s/.*/<h3>&<\/h3>/

Pipe partial line selection through external command in Vim

I know I can use :! with a visual selection to pipe selected lines through an external command, but is there a way to do the same for a single word on a line? I need to base64 encode tokens in a config file and I'm having trouble and the entire line is sent to base64. If I move the word to its own line, I finish up with a trailing \n character encoded in the base64 string. I know there's a plugin specifically for this, but in general is it possible to pipe units of the buffer smaller than entire lines via an external program?
The Ex commands (:! is one of them) are all line-based, because the ex editor on which this mode is based was line-based.
If you need filtering of parts of lines often, I would indeed recommend to use one of the plugins. #romainl's answer outlines the (tedious) steps if you want to do this manually - plugins can greatly simplify that:
With the venerable vis.vim, you can use :B !base64
The unimpaired.vim plugin formerly had ]Y / [Y mappings to encode / decode Base64 directly (implemented in Vimscript)
express.vim defines a g= operator, and will then query for an expression to apply to it. You can use !base64 here.
Ex commands (everything that starts with :) work on lines and there's nothing you can do about that.
Filtering "non-lines" is more involved. You need to:
yank the selection,
escape it if necessary,
run your filter with that selection in a subshell,
clean up the output if necessary,
replace the selection with the output of the filter.
In a nutshell:
c<C-r>=system('echo "<C-r>"" | base64 | tr -d "\n"')<CR>
which is obviously a lot more work than for filtering lines. Map it to something easier.

How to repeat the same search and replace command over disjunct line ranges in Vim?

I had a situation where I wanted to replace FOO with BAR through out a file. However, I only want to do it in certain places, say, between lines 68–104, 500–537, and 1044–1195. In practice, I dropped markers at the lines of interest (via ma, mb, mc, etc.) and ran the following:
:'a,'b s/FOO/BAR/g | 'c,'d s/FOO/BAR/g | 'e,'f s/FOO/BAR/g
I had to repeat this dozens of times with different search and replace terms s/CAT/DOG, etc., and it became a pain to have to rewrite the command line each time. I was lucky in that I had only three places that I needed to confine my search to (imagine how messy the command line would get if there were 30 or 40).
Short of writing a function, is there any neater way of doing this?
On a related note. I copied FOO to the s (search) register, and BAR to the r (replace) and tried running
:'a,'b s/\=#s/\=#r/ | 'c,'d s/\=#s/\=#r/ | 'e,'f s/\=#s/\=#r/
This would have saved me having to rewrite the command line each time, but, alas, it didn’t work. The replace bit \=#r was fine, but the \=#s bit in the search pattern gave me an error.
Any tips would be appreciated.
If you need to perform a set of line-wise operations (like substitutions) on a bunch of different ranges of lines, one trick you can use is to make those lines look different by first adding a prefix (that isn't shared by any of the other lines).
The way I usually do this is to indent the entire file with something like >G performed on the first line, and then use either :s/^ /X/ commands or block-visual to replace the leading spaces with X on the lines I want.
Then use :g in conjunction with :s. eg:
:%g/^X/s/FOO/BAR/g
:%g/^X/s/BAZ/QUUX/g
Finally, remove the temporary prefixes.
In order to get rid of the necessity to retype the same search
pattern, substitution string and flags, one can simply use the
:& command with the & flag:
:'a,'bs/pat/str/g | 'c,'d&& | 'e,'f&&
(See :help :& for details.)
Instead of using marker use this one :
:68,104s/FOO/BAR/g << substitue from line 68 to 104
This should make your job a little bit easier and clearer.
inspired by #Vdt's answer:
I am not sure but you could write all the substitutions down in a file and source that file i think.
substitutions.vim:
68,104s/FOO/BAR/g
168,204s/FOO/BAR/g
618,644s/FOO/BAR/g
681,1014s/FOO/BAR/g
.
.
.
68,104s/BAZ/BOOO/g
168,204s/BAZ/BOOO/g
and then :so substitutions.vim maybe you can also use this for multiple files of same structure. you can add an e to add an ignore error message, if it is not clear that the substitutions are found on the corresponding line blocks.
With q:, you can recall previous command lines and edit them as a normal Vim buffer, so you can quickly replace FOO and BAR with something else, then re-execute the line with Enter.
The s/\=#s/\=#r/ doesn't work; as you said, this only works in the replacement part. But for the pattern, you can use Ctrl + R Ctrl + R s to insert the contents of register s, instead of \=#s. Preferably use the default register, then it's a simple s//, but you probably know that already.
When performed over a closed fold, substitutions are limited to that fold.
fold each region
put the cursor on one closed fold
perform the substitution: :s/foo/bar<CR>
move to the next closed fold with zj or zk
use the command-line history: :<C-p><CR> or :<Up><CR> to perform the same substitution
repeat…
You can also add the c flag at the end of your substitution so that Vim asks you for a confirmation before actually performing it. This can be tedious if you have lot of matches.
Here's the simplest way to do it
:5,10s/old/new/g
5,10 : startlinenum,endlinenum

Run bash command on Vim and copy result to clipboard

How can I create a Vim command and copy it's results to clipboard?
I want to convert Markdown to HTML and copy the result to the clipboard. So far I got:
nmap md :%!/bin/markdown/Markdown.pl --html4tags
But this will substitute my opened file on Vim to the result of Markdown.
You didn't say which system you're using, but generally saving it in the +
register should work. You can call system():
:let #+=system("markdown --html4tags", join(getline(1,line("$")), "\n"))
The system() function takes the second parameter (optional) as input to the
command, and here I'm using a chain of other functions to retrieve the contents
of the current buffer. Not sure, but there should be a better way to do it (if
someone knows, please let me know).
Alternatively, you can pass markdown your file name as input directly:
:let #+=system("markdown --html4tags " . shellescape(expand("%:p")))
But keep in mind that you'll need to write the file before calling this.
Two important notes:
I didn't type your full path to markdown. Use it.
I didn't use maps here, the final result would be something like:
nnoremap md :let #+=system(...)
get the xsel package
and pipe stdout to xsel --clipboard
For instance:
cat /etc/passwd | xsel --clipboard
Is that what you're looking for?
Filling in a missing piece (2+ years late). With the clarification that the user was on a Mac and since the asker's "why doesn't it work for me?" question was not answered.
To redirect the output of a command to the system clipboard from within MacVim (GUI version) you need to set the '*' to be the "clipboard register" you need to change the clipboard setting to 'unnamed':
set clipboard 'unnamed' # 'cb' can be substituted for 'clipboard'
Then sidyll's answer should work except specify the '*' register and not the '+' register:
:let #*=system(...)
The clipboard feature is likely not compiled into the "terminal version" of MacVim and when it is available option setting is different from 'unnamed'. To see more details regarding what works where and how, see the documentation in MacVim using the Vim help command:
:help 'clipboard' (include the single quotes since it's a set option!)
(I'll skip the command mapping issue since it always takes me several tries and I still have to look it up; finding the help for the mapping commands should be easier than finding it for the * register.)

reformat in vim for a nice column layout

I have this dataset in a csv file
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504, 0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644, 0.00078635, 0.000730052, 0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669, 0.00230171, 0.00217917
As you can see, the numbers are formatted differently and misaligned. Is there a way in vim to quickly align the columns properly, so that the result is this
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504,0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644,0.00078635, 0.000730052,0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669,0.00230171, 0.00217917
That would be great to copy and paste sections with ctrl-v. Any hints?
If you're on some kind of UNIX (Linux, etc), you can cheat and filter it through the column(1) command.
:%!column -t
The above will parse on delimiters inside string literals which is wrong, so you will likely need pre-processing steps and specifying the delimiter for this file for example:
%!sed 's/","/\&/' | column -t -s '&'
Sometimes we want to align just two columns. In that case, we don't need any plugins and can use pure Vim functionality like this:
Choose a separator. In OP's post this is a comma, in my example this is =.
Add spaces before/after it. I use s/=/= ...spaces... / in visual selection for this.
Locate to the longest word and place cursor after it.
Remove all the extra whitespace using dw and vertical movement.
Example of this technique demonstrated below:
I don't find myself needing to align things often enough to install another plugin, so this was my preferred way of accomplishing it - especially that it doesn't require much thinking.
As sunny256 suggested, the column command is a great way of doing this on Unix/Linux machines, but if you want to do it in pure Vim (so that it can be used in Windows as well), the easiest way is to install the Align plugin and then do:
:%Align ,
:%s/\(\s\+\),\s/,\1/g
The first line aligns the entries on the commas and the second moves the comma so that it's flush with the preceding value. You may be able to use AlignCtrl to define a custom mapping that does the whole lot in one go, but I can never remember how to use it...
Edit
If you don't mind two spaces between entries and you want to do this in one command, you can also do:
:%Align ,\zs
This is a great answer using vim macros: https://stackoverflow.com/a/8363786/59384 - basically, you start recording a macro, format the first column, stop recording then repeat the macro for all remaining lines.
Copy/pasted from that answer:
qa0f:w100i <Esc>19|dwjq4#a
Note the single space after the 100i, and the <Esc> means "press escape"--don't type "<Esc>" literally.
Translation:
qa -- record macro in hotkey a
0 -- go to beginning of line
f: -- go to first : symbol
w -- go to next non-space character after the symbol
100i <Esc> -- insert 100 spaces
19| -- go to 19th column (value 19 figured out manually)
dw -- delete spaces until : symbol
j -- go to next line
q -- stop recording macro
4#a -- run the macro 4 times (for the remaining 4 lines)
We now also have the fabulous EasyAlign plugin, written by junegunn.
Demonstration GIF from its README:
Also, Tabularize is quite good http://vimcasts.org/episodes/aligning-text-with-tabular-vim/
You could use the csv.vim plugin.
:%ArrangeColumn
However, this will not do exactly what you have asked: it will right adjust the contents of cells, whereas you have your values aligned by the decimal point or by the first digit.
The plugin has many other useful commands for working with CSV files.
also if you have very long columns it can be handy to disable default wrapping
:set nowrap
:%!column -t
(note in debian you also have a further option for column -n which if you want to split multiple adjacent delimiters)
Here’s a pure Vim script answer, no plugins, no macros:
It might be most clear to start out with my problem’s solution as an example. I selected the lines of code I wanted to affect, then used the following command (recall that entering command mode from visual mode automatically prepends the “'<,'>”, so it acts on the visual range):
:'<,'>g``normal / "value<0d>D70|P`
Except I did NOT actually type “<0d>”. You can enter unprintable characters on the command line by pressing ctrl-v, then the key you want to type. “<0d>” is what is rendered on the command line after I typed ‘ctrl-v enter’. Here, it’s parsed by the “normal” command as the exit from “/” search mode. The cursor then jumps to “ value” in the current line.
Then we simply [D]elete the rest of the line, jump to column 70 (or whatever you need in your case), and [P]ut what we just deleted. This does mean we have to determine the width of the widest line, up to our search. If you haven’t put that information in your statusline, you can see the column of the cursor by entering the normal mode command ‘g ctrl-g’. Also note that jumping to a column that doesn’t exist requires the setting 'virtualedit'!
I left the search term for the :g(lobal) command empty, since we used a visual block and wanted to affect every line, but you can leave off using a visual selection (and the “'<,'>”) and put a search term there instead. Or combine a visual selection and a search term to narrow things more finely/easily.
Here’s something I learned recently: if you mess up on a complex command mode command, undo with ‘u’ (if it affected the buffer), then press “q:” to enter a special command history buffer that acts much like a conventional buffer. Edit any line and press enter, and the changed command is entered as a new command. Indispensable if you don’t want to have to stress over formulating everything perfectly the first time.
I just wrote tablign for this purpose. Install with
pip3 install tablign --user
Then simply mark the table in vim and do
:'<,'>:!tablign
Pretty old question, but I've recently availed myself of an excellent vim plugin that enables table formatting either on the fly or after-the-fact (as your use case requires):
https://github.com/dhruvasagar/vim-table-mode
I have this in my .vimrc.
command! CSV set nowrap | %s/,/,|/g | %!column -n -t -s "|"
This aligns the columns while keeping the comma, which may be needed later for correct reading. For example, with Python Pandas read_csv(..., skipinitialspace=True), thanks Pandas guys for this smart option, otherwise in vim %s/,\s\+/,/g. It may be easier if your column has the option --output-separator I guess, my doesn't and I'm not sure why (my man page for column says 2004, on ubuntu 18.04, not sure ubuntu will get a new version). Anyway, this works for me, and comment if you have any suggestions.
I made a cli tool written in Perl.
You can find it here: https://github.com/bas080/colcise

Resources