Vimscript remove control characters - vim

I have a vimscript function shown below that performs a search and replace on the currently yanked/copied text buffer and pastes them into a file.
function Repaste(s, ...)
for i in a:000
let sub = substitute(getreg('"'), a:s, i, 'ge')
let sane = substitute(sub, '[^[:print:]]', '\n', 'ge')
call append(line('.'), sane)
endfor
endfunction
command -nargs=* RP call Repaste(<f-args>)
When I call this function I get ^# characters in place of new lines.
Here is an example of the yanked/copied text
set cindent
Here is an example of the command executed
:RP c d e f
Here is the output
set findent ^#
set eindent ^#
set dindent ^#
How do i remove these characters and why do they appear? Thanks.

The append() function is a low-level one. :help append() shows that the {expr} as a String type is inserted as one text line, regardless of newlines in its content. The ^# is the representation of \n inside a line; cp :help <Nul>.
If you really want to keep using append(), you have to provide a List type; to obtain this, you can split() your String:
call append(line('.'), split(sane, '\n'))
However, I think you're better off by using a higher-level function to insert the created lines, using :put with the expression register to insert the contents of your variable:
put =sane
This will:
automatically handle embedded newlines
set the change marks '[ and '] to the inserted text
print a message 4 more lines (if the inserted number exceeds the 'report' threshold).

Replacing the call to append with put=sub produces the required result.

Related

Indent a block of code on the basis of a single character

The question's title might sound a little vague, so I'll explain the situation here more clearly.
I have these lines of code in a file which I want to align with respect to the character =.
const service = require('./service');
const baseService = require('./baseService');
const config = require('../config');
const Promise = require('bluebird');
const errors = require('../errors');
I want the above lines to somehow look like this
const service = require('./service');
const baseService = require('./baseService');
const config = require('../config');
const Promise = require('bluebird');
const errors = require('../errors');
I want all the = characters to lie in the same column and shift the after-coming code accordingly. What can I do to achieve this task?
A plugin capable of doing this would be nice, but it'd be great if I could do this without the aid of any plugin. That way I'd also learn something.
Using GNU tools
:%!column -t -s= -o=
% ............. current file
! ............. use external command
-t ............. use tabs
-s ............. input separator
-o ............. output separator
Instead of whole file it can be a range of lines 1,5 or you can select a paragraph with vip
I ended up creating a function called AlignText that uses column command to solve this issue:
" Align text by a chosen char
" https://stackoverflow.com/questions/57093175/
" https://vi.stackexchange.com/a/2412/7339
if !exists('*AlignText')
function! AlignText(param) range
execute a:firstline . ',' . a:lastline . '!column -t -s' . a:param . ' -o' . a:param
endfunction
endif
command! -range=% -nargs=1 Align <line1>,<line2>call AlignText(<q-args>)
" :Align =
" :8,$ Align =
If you wnat to test this function before put it into your vimrc, copy the code to your clipboard and then try this:
:#+
Now you can use something like this:
:15,22Align /
:Align ,
I by any change you want to use the function call instead of the command one, don't forget to pass the range and put the argument between quotes.
A slitly different version that accepts nor arguments and uses just "column -t"
" https://stackoverflow.com/questions/57093175/
" https://vi.stackexchange.com/a/2412/7339
function! AlignText(...) range
if a:0 < 1
execute a:firstline . ',' . a:lastline . '!column -t'
else
execute a:firstline . ',' . a:lastline . '!column -t -s' . a:1 . ' -o' . a:1
endif
endfunction
command! -range=% -nargs=? Align <line1>,<line2>call AlignText(<f-args>)
there are plugins can do this kind of alignment. E.g.
Align or easy-align. I have been using Align for a long time, for your requirement, I just select those lines and do <leader>t=
https://github.com/vim-scripts/Align
https://github.com/junegunn/vim-easy-align
You can of course code by yourself, you find out the max length before the =, then you know how many spaces you should insert between partBefore and =. (the diff) on each line.
(NOTE: Originally answered at the Vi and Vim Stack Exchange.)
If you're in a pinch and want to get the expressions aligned, without having to install and learn any plug-ins, here is a quick way to do it.
Select the lines on a visual selection. For example, if this is your whole buffer, you could use ggVG, but if these lines are in the middle of a file, just select appropriately. Perhaps V4j?
Insert enough whitespace before the =, which you can do with :normal f=9i . (Note the "space" at the end.) This will add 9 spaces before each = in the lines of the visual selection. If 9 is not enough, add more (like 99 or 999, as much as you want.) Note that when you type : with the visual selection, Vim will automatically insert the range, so the actual command is :'<,'>normal f=9i , but you don't need to type those characters.
Move to the column where you want the =s to be flushed to. In this case, line 2 has the longest variable name, so move to two spaces after the end of that name, which is where the =s should be at the end. If this is the whole buffer, you could use 2G2e2l to get there.
Staying on that same column, move to the first line of the block. In this case, you're moving from line 2 to line 1, so k is enough.
Start visual-block selection, pressing Ctrl-V.
Move to the last line of the block. If this is the whole buffer, you could use G, if this is the middle of a file, you could use 4j to go four lines down, etc.
(Optional) Use $ to select until the end of the line, on every line of the visual block selection. UPDATE: This step is actually not necessary here, since < will work correctly even if you don't select lines to the end. Using $ is important when using e.g. A to append to the end of all lines though.
Now you can use the < command to shift the lines left, but until they hit the left of the visual block. Each < will shift them by one 'shiftwidth' only, so you're likely to need more than one. So, to be sure, use 9< (or 99<, or 999<, if you added tons of spaces in step 2.)
VoilĂ !
This is a pretty cool technique and it can be helpful when you need more flexibility than plug-ins can afford you. It's a good one to learn and keep on your Vim toolbox.

Decode Vim find/replace string that calls a function

I am using the following Vim command line that inserts line numbers to the beginning of lines:
:let i = 1 | %s/^/\='LINE_' . Inc()/g
Inc() is a function that increments the i variable.
This is all working fine. My questions:
1) What does the dot do in the replacement part?:
:let i = 1 | %s/^/\='LINE_' . Inc()/g
^
2) What does the pipe character do? Is there actual piping going on, or is it just syntax?
3) What does the \= do? I think it is used to call the function, but Vim help only shows information for \= as being a quantifier in regex.
4) I have not been able to insert a space after the line number and the first character of the actual line. How can I do this? Anything I place after Inc() in the replacement part is either being ignored or causing an E15 invalid expression error.
I am using Vim 7.3 on Windows 7.
Some explanation:
. expression will concatenate two strings. See :h expr-.
| will separate to ex-commands. See :h :bar
A replacement starting with \= in :s command means the rest of the replacement is to be treated as an vim expression. See :h :s\=
Concatenate a string with a space after the Inc() function call. :let i = 1 | %s/^/\='LINE_' . Inc() . ' '/g

How do I append a list to a file in vim?

I want to append a list of strings to a file in VimL
Here is my workaround code:
let lines = ["line1\n", "line2\n", "line3\n"]
call writefile(lines, "/tmp/tmpfile")
call system("cat /tmp/tmpfile >> file_to_append_to")
Any way to append to the file directly in vim?
There should be, but I can't find anything
Try using readfile()+writefile().
If you are using Vim 7.3.150+, (or if you are absolutely sure that file in question ends with \n):
function AppendToFile(file, lines)
call writefile(readfile(a:file)+a:lines, a:file)
endfunction
For Vim older than 7.3.150:
" lines must be a list without trailing newlines.
function AppendToFile(file, lines)
call writefile(readfile(a:file, 'b')+a:lines, a:file, 'b')
endfunction
" Version working with file *possibly* containing trailing newline
function AppendToFile(file, lines)
let fcontents=readfile(a:file, 'b')
if !empty(fcontents) && empty(fcontents[-1])
call remove(fcontents, -1)
endif
call writefile(fcontents+a:lines, a:file, 'b')
endfunction
The write command can be used to append the entire current buffer to a file:
:write >> append_file.txt
You can limit it to range of lines in current buffer if you want. E.g., this would append lines 1 through 8 to end of append_file.txt:
:1,8write >> append_file.txt
Vim 7.4.503 added support for appending to a file with writefile using the "a" flag :
:call writefile(["foo"], "event.log", "a")
From :h writefile:
writefile({list}, {fname} [, {flags}])
Write |List| {list} to file {fname}. Each list item is
separated with a NL. Each list item must be a String or
Number.
When {flags} contains "a" then append mode is used, lines are
appended to the file:
:call writefile(["foo"], "event.log", "a")
:call writefile(["bar"], "event.log", "a")
This may be useful, but it appends content to current file.
Create an array removing \n from each field.
:let lines = ["line1", "line2", "line3"]
And append at the end to current file:
:call append( line('$'), lines )

Escape characters during paste in vim

I copy stuff from output buffers into C++ code I'm working on in vim.
Often this output gets stuck into strings. And it'd be nice to be able to escape all the control characters automatically rather than going back and hand editing the pasted fragment.
As an example I might copy something like this:
error in file "foo.dat"
And need to put it into something like this
std::string expected_error = "error in file \"foo.dat\""
I'm thinking it might be possible to apply a replace function to the body of the last paste using the start and end marks of the last paste, but I'm not sure how to make it fly.
UPDATE:
Joey Mazzarelli sugested using
`[v`]h:%s/\%V"/\\"/g
after a paste.
Since no explaination was given for what that was going and I initially found it a bit terse, but hard to explain in the comments I thought I'd put an explaination of what I think that does here:
`[ : Move to start of last paste
v : Start visual mode
`] : Move to end of last paste
h : adjust cursor one position left
:% : apply on the lines of the selection
s/ : replace
\%V : within the visual area
" : "
/ : with
\\" : \"
/g : all occurrences
This seems like a good approach, but only handles the one character, ", I'd like it to handle newlines, tabs, and other things that might be expected to fall in text. (Probably not general unicode though) I understand that may not have been clear in the problem definition.
Here are a couple of vimscript functions that should do what you want.
EscapeText() transforms arbitrary text to the C-escaped equivalent. It converts newline to \n, tab to \t, Control+G to \a, etc., and generates octal escapes (like \o032) for special characters that don't have a friendly name.
PasteEscapedRegister() escapes the contents of the register named by v:register, then inserts it into the current buffer. (The register is restored when the function returns, so the function can be called repeatedly without escaping the register contents multiple times.)
There are also a couple of key mappings included to make PasteEscapedRegister() easy to use interactively. <Leader>P pastes escaped register contents before the cursor position, and <Leader>p pastes after. Both can be prefixed with a register specification, like "a\P to paste the escaped contents of register a.
Here's the code:
function! EscapeText(text)
let l:escaped_text = a:text
" Map characters to named C backslash escapes. Normally, single-quoted
" strings don't require double-backslashing, but these are necessary
" to make the substitute() call below work properly.
"
let l:charmap = {
\ '"' : '\\"',
\ "'" : '\\''',
\ "\n" : '\\n',
\ "\r" : '\\r',
\ "\b" : '\\b',
\ "\t" : '\\t',
\ "\x07" : '\\a',
\ "\x0B" : '\\v',
\ "\f" : '\\f',
\ }
" Escape any existing backslashes in the text first, before
" generating new ones. (Vim dictionaries iterate in arbitrary order,
" so this step can't be combined with the items() loop below.)
"
let l:escaped_text = substitute(l:escaped_text, "\\", '\\\', 'g')
" Replace actual returns, newlines, tabs, etc., with their escaped
" representations.
"
for [original, escaped] in items(charmap)
let l:escaped_text = substitute(l:escaped_text, original, escaped, 'g')
endfor
" Replace any other character that isn't a letter, number,
" punctuation, or space with a 3-digit octal escape sequence. (Octal
" is used instead of hex, since octal escapes terminate after 3
" digits. C allows hex escapes of any length, so it's possible for
" them to run up against subsequent characters that might be valid
" hex digits.)
"
let l:escaped_text = substitute(l:escaped_text,
\ '\([^[:alnum:][:punct:] ]\)',
\ '\="\\o" . printf("%03o",char2nr(submatch(1)))',
\ 'g')
return l:escaped_text
endfunction
function! PasteEscapedRegister(where)
" Remember current register name, contents, and type,
" so they can be restored once we're done.
"
let l:save_reg_name = v:register
let l:save_reg_contents = getreg(l:save_reg_name, 1)
let l:save_reg_type = getregtype(l:save_reg_name)
echo "register: [" . l:save_reg_name . "] type: [" . l:save_reg_type . "]"
" Replace the contents of the register with the escaped text, and set the
" type to characterwise (so pasting into an existing double-quoted string,
" for example, will work as expected).
"
call setreg(l:save_reg_name, EscapeText(getreg(l:save_reg_name)), "c")
" Build the appropriate normal-mode paste command.
"
let l:cmd = 'normal "' . l:save_reg_name . (a:where == "before" ? "P" : "p")
" Insert the escaped register contents.
"
exec l:cmd
" Restore the register to its original value and type.
"
call setreg(l:save_reg_name, l:save_reg_contents, l:save_reg_type)
endfunction
" Define keymaps to paste escaped text before or after the cursor.
"
nmap <Leader>P :call PasteEscapedRegister("before")<cr>
nmap <Leader>p :call PasteEscapedRegister("after")<cr>
This might at least get you started...
After pasting it in:
`[v`]h:%s/\%V"/\\"/g
You can obviously map that to something easier to type.
While Joeys solution looks like it might be extensible to cover all the cases that I need, I thought I'd share my partial solution using vims python integration (Since I'm more familiar at python than vim script)
# FILE : tocstring.py
import vim
def setRegister(reg, value):
vim.command( "let #%s='%s'" % (reg, value.replace("'","''") ) )
def getRegister(reg):
return vim.eval("#%s" % reg )
def transformChar( map, c):
if c in map:
return map[c]
return c
def transformText( map, text ):
return ''.join( [ transformChar(map,c) for c in text ] )
cmap={}
cmap["\\"]="\\\\"
cmap["\n"]="\\n"
cmap["\t"]=r"\\t"
cmap['"']="\\\""
def convertToCString( inregister, outregister ):
setRegister(outregister, transformText( cmap, getRegister(inregister) ) )
Then in my .vimrc or other conf file I can put
# FILE cpp.vim
python import tocstring
# C-Escape and paste the currently yanked content
nmap <Leader>P :python tocstring.convertToCString("#","z")<CR>"zP
# C-Escape and paste the current visual selection
vmap <Leader>P "zd:python tocstring.convertToCString("z","z")<CR>"zP
It would be nice if I could the first function to work so that "a\P pasted the transformed contents of the "a" register, and I assume this is doable using v:register somehow, but it escapes me.
A version of this that works in the same way as Joeys solution could be crafted as
nmap <Leader>P `[v`]"zd:python tocstring.convertToCString("z","z")<CR>"zP
Acknowledgement : This uses code from Can you access registers from python functions in vim for interacting with registers from vims python
for Java/JavaScript type of escaping one can use json_encode
nmap <leader>jp :call setreg('e', json_encode(#+))\| normal "ep<CR>
json_encode(#+) - json encode content of register + (mapped to clipboard)
setreg('e',...) - write it to register e
normal "ep - paste content of register e

How to "apply" backspace characters within a text file (ideally in vim)

I have a log file with backspace characters in it (^H). I'm looking through the file in Vim and it can be quite hard to see what's going on.
Ideally I'd like to be able to "apply" all the ^H on a given line/range so that I can see the final result.
I'd much rather do this within Vim on a line-by-line basis, but a solution which converts the whole file is better than nothing.
Turn on the 'paste' option (using :set paste), and then press dd i <CTRL-R> 1 <ESC> on each line that you want to apply the backspaces to. This also works if you delete multiple lines, or even the whole file.
The key here is that you are using <CTRL-R> 1 in insert mode to 'type out' the contents of register 1 (where your deleted lines just got put), and 'paste' option prevents Vim from using any mappings or abbreviations.
I googled this while trying to remember the command I had used before to `apply' backspaces, and then I remembered it: col -b - here is the manpage. (It does a little more and comes from BSD or more exactly AT&T UNIX as the manpage says, so if you are on Linux you may need to install an additional package, on debian its in bsdmainutils.)
Simplistic answer:
:%s/[^^H]^H//g
where ^^H is:
Literal ^ character
Ctrl-V Ctrl-H
and repeat it couple of times (until vim will tell you that no substitutions have been made
If you want without repetition, and you don't mind using %!perl:
%!perl -0pe 's{([^\x08]+)(\x08+)}{substr$1,0,-length$2}eg'
All characters are literal - i.e. you don't have to do ctrl-v ... anywhere in above line.
Should work in most cases.
All right, here is a bare-metal solution.
Copy this code into a file named crush.c:
#include <stdio.h>
// crush out x^H sequences
// there was a program that did this, once
// cja, 16 nov 09
main()
{
int c, lc = 0;
while ((c = getchar()) != EOF) {
if (c == '\x08')
lc = '\0';
else {
if (lc)
putchar(lc);
lc = c;
}
}
if (lc)
putchar(lc);
}
Compile this code with your favorite compiler:
gcc crush.c -o crush
Then use it like this to crush out those bothersome sequences:
./crush <infilename >outfilename
Or use it in a pipeline ("say" is a speech-to-text app on the Mac)
man date | ./crush | say
You can copy crush to your favorite executable directory (/usr/local/bin, or some such) and then reference it as follows
man date | crush | say
Just delete all occurrences of .^H (where . is the regex interpretation of .):
:s/.^H//g
(insert ^H literally by entering Ctrl-V Ctrl-H)
That will apply to the current line. Use whatever range you want if you want to apply it to other lines.
Once you done one :s... command, you can repeat on another line by just typing :sg (you need to g on the end to re-apply to all occurrences on the current line).
How about the following function? I've used \%x08 instead of ^H as it's easier to copy and paste the resulting code. You could type it in and use Ctrl-V Ctrl-H if you prefer, but I thought \%x08 might be easier. This also attempts to handle backspaces at the start of the line (it just deletes them).
" Define a command to make it easier to use (default range is whole file)
command! -range=% ApplyBackspaces <line1>,<line2>call ApplyBackspaces()
" Function that does the work
function! ApplyBackspaces() range
" For each line in the selected lines
for index in range(a:firstline, a:lastline)
" Get the line as a string
let thisline = getline(index)
" Remove backspaces at the start of the line
let thisline = substitute(thisline, '^\%x08*', '', '')
" Repeatedly apply backspaces until there are none left
while thisline =~ '.\%x08'
" Substitute any character followed by backspace with nothing
let thisline = substitute(thisline, '.\%x08', '', 'g')
endwhile
" Remove any backspaces left at the start of the line
let thisline = substitute(thisline, '^\%x08*', '', '')
" Write the line back
call setline(index, thisline)
endfor
endfunction
Use with:
" Whole file:
:ApplyBackspaces
" Whole file (explicitly requested):
:%ApplyBackspaces
" Visual range:
:'<,'>ApplyBackspaces
For more information, see:
:help command
:help command-range
:help function
:help function-range-example
:help substitute()
:help =~
:help \%x
Edit
Note that if you want to work on a single line, you could do something like this:
" Define the command to default to the current line rather than the whole file
command! -range ApplyBackspaces <line1>,<line2>call ApplyBackspaces()
" Create a mapping so that pressing ,b in normal mode deals with the current line
nmap ,b :ApplyBackspaces<CR>
or you could just do:
nmap ,b :.ApplyBackspaces<CR>
Here's a much faster Awk filter that does the same:
#!/usr/bin/awk -f
function crushify(data) {
while (data ~ /[^^H]^H/) {
gsub(/[^^H]^H/, "", data)
}
print data
}
crushify($0)
Note that where ^^H appears, the first caret in ^^H is a caret (shift-6) and the second caret with H is entered (into vim) by typing CTRL-v CTRL-H
Here's a Bash-based filter you can use to process the whole file:
#!/bin/bash
while read LINE; do
while [[ "$LINE" =~ '^H' ]]; do
LINE="${LINE/[^^H]^H/}"
done
echo "$LINE"
done
Note that where ^H appears, it is entered into vim using CTRL-v CTRL-h, and the ^^H is entered as SHIFT-6 CTRL-v CTRL-h.

Resources