Implementation of regex matches highlighting in text editor - text-editor

I can't understand how text editor can update highlighting of regex while user typing - editing any char of text can change match - it's requeres to rescan all text on every user key pressing. But they do.
Incremental perl-based regex? - they are not in nature
Try to search after previous match? - previous and next matches can be missing or too far from screen that need to rescan biggest part of document, also regex with lookaround will be broken
Another thread? - seems to be only true solution, but you receive annoying highlighting lag
may be possible to move old match highlighting manualy on user input (but we still have problem with new match highlighting/old highlighting removing lag)
may be search in screen buffer (most user regex consume short strings) and then run thread for highlighting to find other long matches with lag (better show highlight late than never)
I would like to know more about existent realizations.
For example, we have: 5,9 MB file (start with ABC, ends with abc) and 2 regex: ABC(.|\n)*abc and using(.*?);
Sublime Text(regex1): regex out of stack space
Sublime Text(regex2): lags ~300 ms, don't update screen untill key up
EditPad Lite 7(regex1): ~500 ms lag on text input, if remove last 'c' - 5000 ms
EditPad Lite 7(ABC.*abc - they use multiline dot): no lags! if remove last 'c' - lags 400ms.
EditPad Lite 7(ABC.*abc, file 578,3 Mb): no lags. How?? Remove last 'c' - 50 seconds on input char
EditPad Lite 7(regex2): no lags!
Vim(regex1): maxmempattern, but if use \(.\|\n\)*abc - highlights if file scrolled to end, no lags, highlighting works until scroll up by 300 lines of 17445, then it's missing and don't went back. How it works?
Vim(regex2): no lags, no bugs
Whole file scanning on every user entry:
.NET Regex(regex1): ~300 ms lag on text input (if remove last 'c' from text - freezes forever)
.NET Regex(regex2): ~300 ms
.NET x86 PCRE clumsy 2-functions wrapper(regex1): fail with error
.NET x86 PCRE clumsy 2-functions wrapper(regex2): ~300 ms

If anyone is interested - I've used buffer with size = screen chars + stock (just don't highlight big matches)

Related

How to repeat a navigation command in Vim

The . key can be used to repeat the last insert command. However, we might do some navigation that is not part of the insert, but we want it repeated.
Imagine commenting out lines like so:
// line of text
// line of text
line of text
line of text
The insert command is to put the two forward slashes and a space. That can be repeated using the . key. The navigation would be to navigate down one line and then left some number of characters. That part is not captured by the . key command.
How can we achieve this functionality? I read that it was not available in Vi some years ago, but I'm wondering if it exists now in the latest version of Vim.
Press qX, where X is any of the writable registers (typically: pick any lowercase letter).
Do whatever actions you want to record.
Press q again to stop recording.
Press #X (where X is the same register) to play it back (count times, if used with a count).
Press ## to replay the most recently used macro (count times).
I read that it was not available in Vi some years ago, but I'm wondering if it exists now in the latest version of Vim.
If the Vim docs are to be believed, Vi did not support recording (steps 1-3), but did support #. Then you would have to manually yank the characters into the target register with "Xy<motion> or some other register-writing command. That also works under Vim, but I can't recommend it because it is much more error prone.
Another approach would be "block select then edit" approach:
ctrl + v - block select
then go down j or down-arrow
shift + i will put you in insert mode. Make the change here where you want it to be reflected on all the other lines you've selected.
esc twice will show/repeat that change you made on the line one.
If you have a big range of similar lines and want to put // at the beginning of it, you can do something like:
:15,25norm! I//<space>
You can also use visual area (vip selects an entire paragraph)
:'<,'>norm! I//<space>
using a pattern
:g/TODO/norm! I//<space>

How do I prevent randomly injected repeat characters in vim

I sometimes encounter the following behaviour when editing large files in vim (it only appears to occur with > 500 lines).
When creating a sentence in insert mode occasionally a character will be repeated on screen (the v in visual):
Changing to normal mode and moving to the line below removes the errant letter (the v is removed):
The closest issue I could find while searching for a solution was described here - in my case setting the TERM variable to xterm-color (as opposed to the current setting of xterm-256color) was not effective.
Any suggestions for how to debug this would be appreciated.
As suggested by Holloway, it was related to a plugin. In my case, when following the install instructions for GitGutter, I had reduced the updatetime to be 250ms. I was able to resolve my issue by increasing this (set updatetime=2500). The plugin is a little less responsive, but it's still extremely useful and I no longer get the repeat errors.

Vim: 100+ character length search query causes crash

In the gVim Search window ("q/"), my screen can fit 100 characters before wrapping the line. (This is apart from the text-wrapping setting in the main editing window.) In building a complex search query (that I would later plug into a command statement) it locks up gVim when I try to run it - which only occurs if the query line wraps within this window. There appears to be nothing in the line I wrote that was out of the ordinary. My first sign that something might be wrong was when I saw gVim's automatically placed pre "/" (before the search query sentence within the Search Window) was added to the wrapped portion of the query line - but, obviously, I am not sure that's the issue.
Has anyone else experienced difficulty in developing long search queries in gVim? I'm running 7.3 (w/patches 1-35) on a Ubuntu Natty system.
I realize I can break this line apart into smaller functions, but I was almost done doing it this way and would be interested in discovering a solution - if there is one.
As a workaround, you can use a regular vim script/buffer to assign search patterns
One 'simple' approach:
open a (new) buffer
type the search pattern as you would in the search window, on a single long line
do the following command to set the searchpattern as into the search register:
:let #/=getline('.')
This has the same effect as pressing enter inside the Search Window, except for the fact that
the cursor won't jump to the first match
the pattern won't be recorded in the search history (effectively avoiding your crash)
There are a number of variation on this basic theme. The essence of which is: assign your search pattern directly into #/

What are the most-used vim commands/keypresses?

I'm a Ruby programming trying to switch from Textmate to MacVim, and I'm having trouble wading through the gargantuan lists of things you can do in VIM and all of the keypresses for them. I'm tired of hearing "You can use 'I' for inserting text, or 'a' for appending text after the character, or 'A' for appending text at the end of the line, or…" I can't imagine everyone uses all 20 different keypresses to navigate text, 10 or so keys to start adding text, and 18 ways to visually select an inner block. Or do you!?
My ideal cheat sheet would be the 30-40 most-used keypresses or commands that everyone uses for writing code on a daily basis, along with the absolute essential plugins that rubyists use daily and the 10 most-used commands for them. In theory, once I have that and start becoming as proficient in VIM as I am in Textmate, then I can start learning the thousands of other VIM commands that will make me more efficient.
Or, am I learning VIM the wrong way altogether?
Here's a tip sheet I wrote up once, with the commands I actually use regularly:
References
vim documentation online
advanced vim tips
more useful tips and graphical cheat sheet
General
Nearly all commands can be preceded by a number for a repeat count. eg. 5dd delete 5 lines
<Esc> gets you out of any mode and back to command mode
Commands preceded by : are executed on the command line at the bottom of the screen
:help help with any command
Navigation
Cursor movement: ←h ↓j ↑k l→
By words:
w next word (by punctuation); W next word (by spaces)
b back word (by punctuation); B back word (by spaces)
e end word (by punctuation); E end word (by spaces)
By line:
0 start of line; ^ first non-whitespace
$ end of line
By paragraph:
{ previous blank line; } next blank line
By file:
gg start of file; G end of file
123G go to specific line number
By marker:
mx set mark x; 'x go to mark x
'. go to position of last edit
' ' go back to last point before jump
Scrolling:
^F forward full screen; ^B backward full screen
^D down half screen; ^U up half screen
^E scroll one line up; ^Y scroll one line down
zz centre cursor line
Editing
u undo; ^R redo
. repeat last editing command
Inserting
All insertion commands are terminated with <Esc> to return to command mode.
i insert text at cursor; I insert text at start of line
a append text after cursor; A append text after end of line
o open new line below; O open new line above
Changing
r replace single character; R replace multiple characters
s change single character
cw change word; C change to end of line; cc change whole line
c<motion> changes text in the direction of the motion
ci( change inside parentheses (see text object selection for more examples)
Deleting
x delete char
dw delete word; D delete to end of line; dd delete whole line
d<motion> deletes in the direction of the motion
Cut and paste
yy copy line into paste buffer; dd cut line into paste buffer
p paste buffer below cursor line; P paste buffer above cursor line
xp swap two characters (x to delete one character, then p to put it back after the cursor position)
Blocks
v visual block stream; V visual block line; ^V visual block column
most motion commands extend the block to the new cursor position
o moves the cursor to the other end of the block
d or x cut block into paste buffer
y copy block into paste buffer
> indent block; < unindent block
gv reselect last visual block
Global
:%s/foo/bar/g substitute all occurrences of "foo" to "bar"
% is a range that indicates every line in the file
/g is a flag that changes all occurrences on a line instead of just the first one
Searching
/ search forward; ? search backward
* search forward for word under cursor; # search backward for word under cursor
n next match in same direction; N next match in opposite direction
fx forward to next character x; Fx backward to previous character x
; move again to same character in same direction; , move again to same character in opposite direction
Files
:w write file to disk
:w name write file to disk as name
ZZ write file to disk and quit
:n edit a new file; :n! edit a new file without saving current changes
:q quit editing a file; :q! quit editing without saving changes
:e edit same file again (if changed outside vim)
:e . directory explorer
Windows
^Wn new window
^Wj down to next window; ^Wk up to previous window
^W_ maximise current window; ^W= make all windows equal size
^W+ increase window size; ^W- decrease window size
Source Navigation
% jump to matching parenthesis/bracket/brace, or language block if language module loaded
gd go to definition of local symbol under cursor; ^O return to previous position
^] jump to definition of global symbol (requires tags file); ^T return to previous position (arbitrary stack of positions maintained)
^N (in insert mode) automatic word completion
Show local changes
Vim has some features that make it easy to highlight lines that have been changed from a base version in source control. I have created a small vim script that makes this easy: http://github.com/ghewgill/vim-scmdiff
http://www.viemu.com/a_vi_vim_graphical_cheat_sheet_tutorial.html
This is the greatest thing ever for learning VIM.
Here is a great cheat sheet for vim:
Have you run through Vim's built-in tutorial? If not, drop to the command-line and type vimtutor. It's a great way to learn the initial commands.
Vim has an incredible amount of flexibility and power and, if you're like most vim users, you'll learn a lot of new commands and forget old ones, then relearn them. The built-in help is good and worthy of periodic browsing to learn new stuff.
There are several good FAQs and cheatsheets for vim on the internet. I'd recommend searching for vim + faq and vim + cheatsheet. Cheat-Sheets.org#vim is a good source, as is Vim Tips wiki.
What most people do is start out with the bare basics, like maybe i, yw, yy, and p. You can continue to use arrow keys to move around, selecting text with the mouse, using the menus, etc. Then when something is slowing you down, you look up the faster way to do it, and gradually add more and more commands. You might learn one new command per day for a while, then it will trickle to one per week. You'll feel fairly productive in a month. After a year you will have a pretty solid repertoire, and after 2-3 years you won't even consciously think what your fingers are typing, and it will look weird if you have to spell it out for someone. I learned vi in 1993 and still pick up 2 or 3 new commands a year.
#Greg Hewgill's cheatsheet is very good. I started my switch from TextMate a few months ago. Now I'm as productive as I was with TM and constantly amazed by Vim's power.
Here is how I switched. Maybe it can be useful to you.
Grosso modo, I don't think it's a good idea to do a radical switch. Vim is very different and it's best to go progressively.
And to answer your subquestion, yes, I use all of iaIAoO everyday to enter insert mode. It certainly seems weird at first but you don't really think about it after a while.
Some commands incredibly useful for any programming related tasks:
r and R to replace characters
<C-a> and <C-x>to increase and decrease numbers
cit to change the content of an HTML tag, and its variants (cat, dit, dat, ci(, etc.)
<C-x><C-o> (mapped to ,,) for omnicompletion
visual block selection with <C-v>
and so on…
Once you are accustomed to the Vim way it becomes really hard to not hit o or x all the time when editing text in some other editor or textfield.
I can't imagine everyone uses all 20 different keypresses to navigate text, 10 or so keys to start adding text, and 18 ways to visually select an inner block. Or do you!?
I do.
In theory, once I have that and start becoming as proficient in VIM as I am in Textmate, then I can start learning the thousands of other VIM commands that will make me more efficient.
That's the right way to do it. Start with basic commands and then pick up ones that improve your productivity. I like following this blog for tips on how to improve my productivity with vim.
tuxfiles.org holds a pretty good cheat sheet. I think there are a couple of points to learning the commands:
Read the cheat sheet regularly. Don't worry about using all of them, or remembering all the keys, just know that the command exists. Look up the command and use it when you find yourself doing something repetitive.
If you find yourself doing something regularly (like deleting an entire line after a particular character d$), go a quick google search to see if you can find a command for it.
Write down commands you think you'll find useful and keep that list where you can see it while you're writing your code. I would argue against printing something out and instead encourage you to use post it notes for just a few commands at a time.
If possible, watch other programmers use vim, and ask them what commands they are using as you see them do something interesting.
Besides these tips, there are some basic concepts you should understand.
vim will use the same character to represent the same function. For example, to delete a line after a character use d$. To highlight a line after a particular character use v$. So notice that $ indicates you will be doing something to the end of the line from where your cursor currently is.
u is undo, and ctrl+r is redo.
putting a number in front of a command will execute it repeatedly. 3dd will delete the line your cursor is on and the two lines that follow, similarly 3yy will copy the line your cursor is on and the two lines that follow.
understand how to navigate through the buffers use :ls to list the buffers, and :bn, :bp to cycle through them.
read through the tutorial found in :help This is probably the best way to 'learn the ropes', and the rest of the commands you will learn through usage.
Go to Efficient Editing with vim and learn what you need to get started. Not everything on that page is essential starting off, so cherry pick what you want.
From there, use vim for everything. "hjkl", "y", and "p" will get you a long way, even if it's not the most efficient way. When you come up against a task for which you don't know the magic key to do it efficiently (or at all), and you find yourself doing it more than a few times, go look it up. Little by little it will become second nature.
I found vim daunting many moons ago (back when it didn't have the "m" on the end), but it only took about a week of steady use to get efficient. I still find it the quickest editor in which to get stuff done.
Put this in your .bashrc to open vim with last edited file at last edited line
alias vil="vim +\"'\"0"

reformat in vim for a nice column layout

I have this dataset in a csv file
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504, 0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644, 0.00078635, 0.000730052, 0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669, 0.00230171, 0.00217917
As you can see, the numbers are formatted differently and misaligned. Is there a way in vim to quickly align the columns properly, so that the result is this
1.33570301776, 3.61194e-06, 7.24503e-06, -9.91572e-06, 1.25098e-05, 0.0102828, 0.010352, 0.0102677, 0.0103789, 0.00161604, 0.00167978, 0.00159998, 0.00182596, 0.0019804, 0.0133687, 0.010329, 0.00163437, 0.00191202, 0.0134425
1.34538754675, 3.3689e-06, 9.86066e-06, -9.12075e-06, 1.18058e-05, 0.00334344, 0.00342207, 0.00332897, 0.00345504,0.00165532, 0.00170412, 0.00164234, 0.00441903, 0.00459294, 0.00449357, 0.00339737, 0.00166596, 0.00451926, 0.00455153
1.34808186291, -1.99011e-06, 6.53026e-06, -1.18909e-05, 9.52337e-06, 0.00158065, 0.00166529, 0.0015657, 0.0017022, 0.000740644,0.00078635, 0.000730052,0.00219736, 0.00238191, 0.00212762, 0.00163783, 0.000750669,0.00230171, 0.00217917
That would be great to copy and paste sections with ctrl-v. Any hints?
If you're on some kind of UNIX (Linux, etc), you can cheat and filter it through the column(1) command.
:%!column -t
The above will parse on delimiters inside string literals which is wrong, so you will likely need pre-processing steps and specifying the delimiter for this file for example:
%!sed 's/","/\&/' | column -t -s '&'
Sometimes we want to align just two columns. In that case, we don't need any plugins and can use pure Vim functionality like this:
Choose a separator. In OP's post this is a comma, in my example this is =.
Add spaces before/after it. I use s/=/= ...spaces... / in visual selection for this.
Locate to the longest word and place cursor after it.
Remove all the extra whitespace using dw and vertical movement.
Example of this technique demonstrated below:
I don't find myself needing to align things often enough to install another plugin, so this was my preferred way of accomplishing it - especially that it doesn't require much thinking.
As sunny256 suggested, the column command is a great way of doing this on Unix/Linux machines, but if you want to do it in pure Vim (so that it can be used in Windows as well), the easiest way is to install the Align plugin and then do:
:%Align ,
:%s/\(\s\+\),\s/,\1/g
The first line aligns the entries on the commas and the second moves the comma so that it's flush with the preceding value. You may be able to use AlignCtrl to define a custom mapping that does the whole lot in one go, but I can never remember how to use it...
Edit
If you don't mind two spaces between entries and you want to do this in one command, you can also do:
:%Align ,\zs
This is a great answer using vim macros: https://stackoverflow.com/a/8363786/59384 - basically, you start recording a macro, format the first column, stop recording then repeat the macro for all remaining lines.
Copy/pasted from that answer:
qa0f:w100i <Esc>19|dwjq4#a
Note the single space after the 100i, and the <Esc> means "press escape"--don't type "<Esc>" literally.
Translation:
qa -- record macro in hotkey a
0 -- go to beginning of line
f: -- go to first : symbol
w -- go to next non-space character after the symbol
100i <Esc> -- insert 100 spaces
19| -- go to 19th column (value 19 figured out manually)
dw -- delete spaces until : symbol
j -- go to next line
q -- stop recording macro
4#a -- run the macro 4 times (for the remaining 4 lines)
We now also have the fabulous EasyAlign plugin, written by junegunn.
Demonstration GIF from its README:
Also, Tabularize is quite good http://vimcasts.org/episodes/aligning-text-with-tabular-vim/
You could use the csv.vim plugin.
:%ArrangeColumn
However, this will not do exactly what you have asked: it will right adjust the contents of cells, whereas you have your values aligned by the decimal point or by the first digit.
The plugin has many other useful commands for working with CSV files.
also if you have very long columns it can be handy to disable default wrapping
:set nowrap
:%!column -t
(note in debian you also have a further option for column -n which if you want to split multiple adjacent delimiters)
Here’s a pure Vim script answer, no plugins, no macros:
It might be most clear to start out with my problem’s solution as an example. I selected the lines of code I wanted to affect, then used the following command (recall that entering command mode from visual mode automatically prepends the “'<,'>”, so it acts on the visual range):
:'<,'>g``normal / "value<0d>D70|P`
Except I did NOT actually type “<0d>”. You can enter unprintable characters on the command line by pressing ctrl-v, then the key you want to type. “<0d>” is what is rendered on the command line after I typed ‘ctrl-v enter’. Here, it’s parsed by the “normal” command as the exit from “/” search mode. The cursor then jumps to “ value” in the current line.
Then we simply [D]elete the rest of the line, jump to column 70 (or whatever you need in your case), and [P]ut what we just deleted. This does mean we have to determine the width of the widest line, up to our search. If you haven’t put that information in your statusline, you can see the column of the cursor by entering the normal mode command ‘g ctrl-g’. Also note that jumping to a column that doesn’t exist requires the setting 'virtualedit'!
I left the search term for the :g(lobal) command empty, since we used a visual block and wanted to affect every line, but you can leave off using a visual selection (and the “'<,'>”) and put a search term there instead. Or combine a visual selection and a search term to narrow things more finely/easily.
Here’s something I learned recently: if you mess up on a complex command mode command, undo with ‘u’ (if it affected the buffer), then press “q:” to enter a special command history buffer that acts much like a conventional buffer. Edit any line and press enter, and the changed command is entered as a new command. Indispensable if you don’t want to have to stress over formulating everything perfectly the first time.
I just wrote tablign for this purpose. Install with
pip3 install tablign --user
Then simply mark the table in vim and do
:'<,'>:!tablign
Pretty old question, but I've recently availed myself of an excellent vim plugin that enables table formatting either on the fly or after-the-fact (as your use case requires):
https://github.com/dhruvasagar/vim-table-mode
I have this in my .vimrc.
command! CSV set nowrap | %s/,/,|/g | %!column -n -t -s "|"
This aligns the columns while keeping the comma, which may be needed later for correct reading. For example, with Python Pandas read_csv(..., skipinitialspace=True), thanks Pandas guys for this smart option, otherwise in vim %s/,\s\+/,/g. It may be easier if your column has the option --output-separator I guess, my doesn't and I'm not sure why (my man page for column says 2004, on ubuntu 18.04, not sure ubuntu will get a new version). Anyway, this works for me, and comment if you have any suggestions.
I made a cli tool written in Perl.
You can find it here: https://github.com/bas080/colcise

Resources