Why does this regular expression not work in Vim syntax highlighting? - python-3.x

I am attempting to create my own Python syntax highlighting file for Vim.
I'm trying to highlight the class inheritance object and the regex I've created
works in various regex testers, but doesn;t work in Vim. I've read that Vim's regex is close to Perl style
so that is what I've been using.
I'm trying to highlight the word 'Subscribers' in the following text:
class Divisions(Subscribers):
The regex I've composed is:
(?!:class\s\w+)(?<=\()\w+(?=\):)
I'll be honest here, I stumbled into this while I was struggling to make a negative lookbehind work with quantifiers,
which I now understand isn't possible. I was experimenting with the non-capturing group (?:class\s\w+) and accidentally
inserted the exclamation mark which 'magically' solved the problem. At least in the multiple regex testers I was using.
Just for clarity, then follows a look behind (?<=() to caputre but not include the '('
and then a look ahead after the word (?=):) to capture but not include the closing '):'
I've added it to my Vim syntax file as:
syn match pythonClassInherit "(?!:class\s\w+)(?<=\()\w+(?=\):)"
Is this a valid regular expression in Vim? If not, can anybody offer a working solution for Vim?

EDIT: I realized that I've had overcomplicated the issue; you just need a right Vim regex. Try
syntax match pythonClassInherit "\%(class\s\+\h\w*\s*(\s*\)\#<=\h\w*\%(\s*):\)\#="
highlight link pythonClassInherit pythonImport
Then you'll see the result:
You may replace pythonImport by another predefined highlight group.
For highlighting, I found https://learnvimscriptthehardway.stevelosh.com/ (highlighting starts from Chap. 45) helpful. As for regex, romainl's suggestion is awesome.
My first attempt was to use regex \zs. This, unlike \#<=, requires that the syntax group cover the part that comes before \zs, which cannot be satisfied because some other groups have already occupied that part. This led me a complicated solution, by overriding the default highlight groups to include a new group (for Subscribers). But it turns out we really don't have to.
If interested, see Vim syntax file not matching with \zs.

Online regular expression playgrounds don't support Vim's syntax so using them is pointless, here.
See vimregex.com for an overview, :help usr_27 for a gentle tutorial, :help pattern for the definitive reference, and :help perl-patterns for the differences between the Vim syntax and the Perl syntax.

Related

What is the syntax called that's used for syntax highlighting?

My apologies if this is such a trivial question for most, but it is not readily obvious to me and I can't find an answer. I am trying to understand how to code syntax highlighting for vi, but the syntax used [for the syntax highlighting] eludes me. For example, I know that
syn match myNumber '\d\+'
hi myNumber ctermfg=blue
will highlight positive integers blue. What confuses me is the '\d\+' part. Playing around it seems that \d means digit and \+ means several? But I have no idea what this syntax is called and thus can't find any documentation that could help me. I have only found links of people using it without explanation. I can probably decipher how
'[-+]\=\d[[:digit:]]*\.\d*[eE][\-+]\=\d\+'
means 'positive and negative numbers with decimals and exponents', but if I am to make more complex highlighting I don't know where to begin. Does anybody know of any documentation that I could use to learn this?
That's called a "regular expression" or regex (or regexp) for short. See :help pattern and :help usr_27.txt. Also see vimregex.com.

Substitute in vim, with special characters

I am trying to use substitute command in vim to enclose all occurences of a particular pattern
\cite{author1}
\cite{author2}
with
(\cite{author1})
(\cite{author2})
Based on other answers in stack exchangeI used the following vim command
%s/\\cite{(\w\+)}/(\\cite{\1})/g
But, no luck. It says "no matches found". I put two back slashes, one of which is supposed to be the escape character. Kindly help.
I know I could use some other editor and finish the job, but I want to know my mistake. Thank you.
please escape ()
%s/\\cite{\(\w\+\)}/(\\cite{\1})/g
You do not need a capture group to get the entire match. You can use \0 or & for the whole match in the replacement portion of your substitution.
:%s/\\cite{\w\+}/(&)/g
If you do want to use a capture group, then you need to escape the capture group/parenthesis.
:%s/\(foo\)/(\1)/g
For more help see:
:h :s%
:h /magic
As mentioned, normally you need escape the parentheses.
You can use very magic mode (\v) to make the regex simpler; removing the need to escape lots of the regex parts, including the parentheses for capturing:
%s/\v(\\cite\{\w+\})/(\1)/g
Knowing what every sign in a regular expression does in vim is sometimes very
difficult, especially for all the modes and configuration variables that this
depends on. For that reason, it is very important to have visual feedback about
what text are we really matching in any moment while crafting regular
expressions.
If you work with neovim, there is a special option called inccommand that you
can enable to live preview the matches and substitution. That way you can figure
out you errors more quickly. This feature is very likely to be included also in
vim in the future, but independently of that, you can also use simple vim to
give you visual feedback if you enable the option incsearch.
With incsearch set, you can watch the matches of / and ? while you write them
just to be sure that the pattern is correct. Later you can exploit a nice
feature from the substitution: if you leave the first section empty, the last
search will be used.
So you could first make the search:
/\\cite{\w\+}/(&)/g
In the meantime, vim will be showing you the matched text visually. Once you
are sure that the pattern is correct press enter, and finally type:
:%s//(\1)<Enter>
Also, in case you want something more powerful than this simple hack, you can go
for my plugin Extend.vim, that can
do that and much more with great visual feedback.

Vim recipe for a minor syntax highlighting extension

I want to leave my system's Python syntax highlighting mostly intact, but I have a specific pattern I'd like to highlight for an idiom I use a lot. How can I add additional highlighting instructions on top of the existing highlighting done by vim?
(Apologies if this has already been asked. All the vim syntax highlighting questions I found seemed to involve writing a new syntax highlighting from scratch.)
Put your additional :syntax commands into ~/.vim/after/syntax/python.vim, and they will be automatically executed after the original syntax script.
It's easy to highlight stuff that so far isn't parsed at all.
For elements already parsed / hightlighted, you need to find out by which syntax group (e.g. pythonFunction), and add a containedin=pythonFunction clause to your :syntax commands. Without that, the original matching will obscure yours. To find out which syntax group causes the highlighting. :syn list shows all active groups, but it's easier when you install the SyntaxAttr.vim - Show syntax highlighting attributes of character under cursor plugin.
Introducing highlighting across (larger) elements that have multiple existing syntax groups is difficult, as your match will obscure the original ones, and that may break the entire parsing. You need to carefully examine the existing nested element structure, and try to fit in yours, again via contains= and containedin= clauses. Depending on the actual situation, that can be difficult.
For the actual syntax definitions, see the help starting at :h :syn-keyword. Basically, there are simple keyword definitions, regular expression matches, and regions defined by start and end patterns.

Making a Vim theme that disables highlighting except for some special keywords

Inspired by several posts, like Your syntax highlighter is wrong, Coding in color and A case against syntax highlighting and some others, I decided making a Vim theme that applied some of these concepts would be a good idea.
The thing is I'm not exactly sure how.
From what I can tell, in order to make a Vim theme you need to basically link a color with a syntax identifier or name. And repeat this hundreds or dozens of time in order to have in your lap a theme.
Like for example linking the color #ff0000 (red) and the syntax identifier, or key, Error. As an example. Not sure if that's actually the syntax key.
This would work fine, except that, every syntax that I don't consider important I have to define as just a default foreground value.
And let's say I wanted to add a new syntax keyword, I'd have to do it with ftsyntax and stuff (I believe) and that would be filetype specific etc.
So the first question is:
What would be the best way to give everything a default foreground color and only pick the exceptions to have some colors?
And the second, perhaps more important question is:
How do I syntax highlight a specific piece of text without having to add a syntax rule? For example have a regex that finds any = and highlights them green, without having to add a syntax rule specific for that.
Any help is appreciated. Of course if the approach I'm taking to this is not ideal or sucks I am open to suggestions to alternatives. Thank you. :)
See the example syntax file below:
syn keyword myKeywords We Are Important Keywords
syn match myEquals '='
hi link myKeywords Special
hi link myEquals Operator
This will put We, Are, Important and Keywords into the myKeywords syntax group and = into the myEquals syntax group.
Then we specify how we want to highlight them, by linking it to the Special and Operator highlight groups.
See: :help group-name for a list of the highlight groups and what the colors look like with your color-scheme.
In my color-scheme, Special is Red and Operator is green.
By default, everything else is set to the default foreground color.
I saved this to ~/.vim/syntax/greduan.vim and tested with :set syntax=greduan
Your question touches two domains:
syntax definition
syntax highlighting
Syntax definition, as in Caek's answer, is simple for the first 10 minutes but grows very quickly into a major PITA because it is a core aspect of Vim's architecture with ramifications far beyond syntax highlighting.
Syntax highlighting has its pitfalls but it is a lot simpler than syntax definition.
I think that you can tackle the problem described in those blog posts with syntax highlighting first and, if needed, graduate to syntax definition.
Grab a simple colorscheme like Busybee.
Link all the highlight groups you don't need to Normal while leaving the ones you want to keep:
hi link Foo Normal
What would be the best way to give everything a default foreground
color and only pick the exceptions to have some colors?
What is best depends ... for me the best, because quickest way was clearing unwanted highlighting in ~/.vimrc:
sy on
hi c Constant|hi c Error|hi c PreProc|hi c Special|hi c Statement|hi c Type
hi c Identifier
How do I syntax highlight a specific piece of text without having to
add a syntax rule?
If by syntax rule you mean syntax item, I'd say you cannot have syntax highlighting without defining syntax items.

Vim syntax highlighting: overlapping regex

I try to add some syntax highlighting for javascript to vim, but I keep running into one problem: when characters are already highlighted, they seem to be completely ignored by all other regular expressions.
For example, I tried to add syntax highlighting for the argument list of a function. While creating the right rexex I disabled the syntax highlighting for the function keyword, such that it was easier to see what my regex did. I ended up with the following (working) regex:
syn match javaScriptArguments "[(=\:\s,]function.\{-}(\zs.\{-}\ze)"
However, as soon as I enabled the highlighting for the function keyword again, this line doesn't work anymore. It seems that vim simply excludes everything which is already highlighted, and thus it won't find any matches for the regex above, even though it won't result in characters being highlighted twice.
How can I solve/work around this problem?
Syntax definitions must be contained for them to match inside other syntax items. Find all the gruesome details at :help syn-contains.
In your case, you're relying on a look-before of the "function" keyword via \zs. In my experience, that's bound to cause problems, but may turn out to be unneccessary once you use contained. In general, it is difficult to extend an existing syntax definition without modifying the original script (which I suppose is what you're intending to do). Have a look at :help :syn-containedin and :help :syn-nextgroup.

Resources