[using MacVim 7.3 on OS X Lion]
I have a vimscript function which runs an external command. It's executing correctly, but the output displays (unwanted) control characters, such as [0m, [33m, [36m, and -1H. The relevant line in the vimscript function is:
exec ":!bundle exec rspec --color " . a:filename
Which produces:
:!bundle exec rspec --color spec/acceptance/user_logs_in.feature
[33m*[0m
Pending:
[33m User logs in [0m
[36m # the step 'the user "foo#test.host" exists' is not implemented[0m
[36m # [0m
Finished in 0.07121 seconds
[33m1 example, 0 failures, 1 pending[0m
Here's what the same command and output look like from the terminal, which is how I want it to display in vim:
$ bundle exec rspec --color spec/acceptance/user_logs_in.feature
*
Pending:
User logs in
# the step 'the user "foo#test.host" exists' is not implemented
#
Finished in 0.1161 seconds
1 example, 0 failures, 1 pending
Also, any time I execute and external command, vim displays -1H immediately after it. For example, if I type:
:ls<return>
I see:
:ls-1H
<rest of the output is as expected>
Any ideas on hiding those control characters and -1H.
(disclaimer: I'm very new to vim so please don't assume too much background knowledge on my part.)
Thanks.
Update 3/31/2012 # 17:32 PM
Sam Goldman's correct: MacVim doesn't know how to display colors, so it outputs the color codes.
I've switched to terminal vim (which supports colors, at least with iTerm), but using the version of vim that comes with MacVim, which is more up-to-date and is compiled with ruby support (among other things). The easiest way to do that is:
brew install macvim --override-system-vim
MacVim doesn't know how to display colors. Terminal vim will display the colors correctly, or you can add --no-color to your rspec command (or a .rspec file). I'm not sure about the -1H thing. Maybe some customization you have for your terminal?
Related
I am trying to open a 200MB text file in gvim on Windows, and it's choking. The vim window opens and just remains blank and unresponsive. My system has 8GB of RAM and the file opens fine in about 2-5 seconds using notepad.
I found this SO post on editing very large text files in vim, and I tried opening the file with plugins turned off:
vim -u "NONE" my200MBfile.text
But that didn't help. Is there anything else I can do to make it work? It seems strange that vim would choke on my machine on what is not really such a large file.
Thanks!
Most likely it's the syntax plugin that makes vim halt. I never have any issues with files of Mbs-Gbs. Unless I accidentally open them with some - not-so-well-behaved - filetypes.
Press ^C while loading/hanging (interrupting plugins)
:syntax off
Does the trick. There are other options with syntax that makes it easier to work with a file without fully 'synchronized' syntax highlighting
I played around with this some more, and I found indeed some of my plugins would make operating on (very) large files very slow. I did this, for fun:
On a large input file (~600Mb):
$ wc input.txt.full
2674568 2674568 608825278 input.txt.full
I launched vim with the following tweaks:
vim -u NONE -n +'se nonu nowrap ul=-1 | syn off' input.txt.full
Note this
-u NONE prevents execution of initialization (user) scripts and plugins
-n disables the writing of swapfiles (extremely important on slow/small disks)
se nonu nowrap ul=-1 disables
line numbering
line wrapping
undo history (setting undolevels to a negative value)
Basically, everything that could take a lot of CPU or memory
syn off to disables syntax highlighing (should only make a difference if syntax highlighting was in effect for the file type)
Now, I wanted to duplicate each line on the line itself (globally: copy line, join with previous):
:g/^/t.|-j
Unfortunately, the file would become too large for available memory (~3Gb)1 so I opted to act on the first 20% (~535,000 lines):
:exec "norm 20%"|1,.g/^/t.|-j
This works in a "jiffy" and without problems. Manual navigation (jumps, scrolling, mode switching, searching etc.) seems to be responsive.
Commandline Benchmark:
$ xxd -c 44 /dev/urandom | head -n 3800000 > input.txt.full
$ wc input.txt.full
3800000 91820644 627000000 input.txt.full
$ time vim -u NONE -n +'se nonu nowrap ul=-1 | syn off' input.txt.full +'exec "norm 10%"|1,.g/^/t.|-j' +wq
real 0m7.778s
user 0m6.680s
sys 0m0.952s
$ wc input.txt.full
3800000 101002757 689940307 input.txt.full
(Note that 689940307 ÷ 627000000 = 110.03 %, so that is exactly right).
This isn't slow in my book. For comparison, the wc invocation itself takes the same amount of time (7.7s).
1 All test performed on tmpfs to avoid cache differences.
The problem is more likely that vim doesn't like lines that long very much.
Try using a tool like json_reformat on the file first...
If you absolutely have to edit the file as-is, try :syntax off and possibly :set nowrap before you :edit the file in question.
Although Vim doesn't have a problem opening files of that size (or bigger), it is mentioned that it has a problem of opening a file with, as is your case, only one line (of that size).
You could try various workarounds, but I would suggest going with the "right tool for the right job" approach, and try EmEditor, a text editor made specifically for editing large files (amongst other things).
Note that this is not Vim bashing, but Vim is an editor for editing source code mostly. An excellent one at that, but like every other tool it has its limitations.
When building my application using the :make command in Vim, the output is not colorized. I have configured the makefile to use clang as the C compiler, and when running make outside of Vim or when running :!make, clang's output is colorized. :set makeprg returns makeprg=make, just for reference.
I have the same issue with grep: when running :grep, the output is not colorized; when running :!grep, it is. I have tried using the --color option with :grep, to no avail. :set grepprg returns grepprg=grep -n $* /dev/null.
I've read through VIM Unix commands printed in color and also How to color my vimgrep result patterns. The former seems to have the opposite problem (i.e. :!command output not colorized); the latter doesn't have any alternative to dropping down to the shell, which I don't feel is a "correct" fix for the issue.
The problem is that when Vim runs other commands via :make or :grep, those commands don't get a terminal for their standard output -- in the sense that for them isatty(STDOUT_FILENO) is false -- because Vim is capturing the output on its way to being displayed on the terminal. On the other hand, when you use :!make or :!grep, standard output is just going to the terminal.
Clang by default and grep --color=auto (which is probably how you have it aliased) use the terminalness of stdout to decide whether to colourise their output. This is convenient in that you get colourful output on your terminal but capture just the text when you redirect output to a file -- all without needing to add extra command line options.
So what you want to do is override these commands' usual smarts so that they always colourise their output.
For grep, you can use --color=always when it is run via :grep within Vim:
:set grepprg=grep\ --color=always\ -n\ $*\ /dev/null
and depending on your colour settings and version of grep this will work well enough.
For clang, you can change your Makefile to use clang -fcolor-diagnostics so as to force colourisation or more flexibly add an extra variable to $(CC) that will be overridden when run via :make within Vim:
:set makeprg=make\ EXTRA_CFLAGS=-fcolor-diagnostic
However (at least with clang 3.0 and vim 7.3) you will find that clang's style of colourisation prevents Vim from picking out filenames and line numbers from the diagnostics, so doing this wrecks the advantage of using :make rather than :!make.
You may be able to teach Vim to pick out the filenames etc from the surrounding ANSI escape sequences that do the colourisation by adding more entries to Vim's errorformat option to match the colourised clang-style diagnostics. (And similarly with grepformat if your grep colourisation colours the filenames or linenumbers.)
When you run :grep or :make (as opposed to :!grep or :!make),
the output is not only shown in the terminal,
but also sent to the quick-fix window, from which it is processed.
You can access the quick fix windo using the vim-command :copen.
The quick fix window is essentially a text file that is opened in read-only mode.
Like in any other text file, colors are not supported in the quick fix file.
Instead, they are represented with escape characters like [01;34m.
Therefore, producing colorized output from make (or grep) will
mess up the output as it is shown in the quick-fix window, even if you can get vim to process it,
and send the cursor to the selected error/warning/find message.
The question whether the output is colorized now becomes a little subtle: I suggest that the
terminal output should remain uncolored, but that the quick-fix output should be colorized.
The color scheme in the quick fix window is not defined by any color-indications in the file itself, but in the syntax highlighting
of the quick fix window, defined in the file qf.vim (/usr/share/vim/vim81/syntax/qf.vim on my computer).
The color scheme defined in qf.vim does not add much color to the quick fix window, but the syntax hightlighting
scheme may be extended by creating the file ~/.vim/syntax/after/qf.vim. I use cmake in combination with the
gnu and/or the intel compilers, and get nice-looking results with the following contents for ~/.vim/syntax/after/qf.vim:
syn match qBuilt "Built target *" nextgroup=qTarget
syn match qTarget ".*$" contained
syn match qEnteringLeaving ": \(Entering\|Leaving\) directory *" nextgroup=qdSeparator
syn match qdSeparator "'" nextgroup=qdName contained
syn match qdName "[^']*" contained
syn match qbProgress "\[ *[0-9]*%\]"
syn match qBuild "Building .* object"
syn match qWarn "warning\( *#[0-9]*\|\):"
syn match qError "error\( *#[0-9]*\|\):"
syn match qRemark "remark\( *#[0-9]*\|\):"
hi def link qTarget Constant
hi def link qError Error
hi def link qWarn Error
hi def link qRemark WarningMsg
hi def link qEnteringLeaving Keyword
hi def link qBuild Keyword
hi def link qBuilt Keyword
hi def link qdName Include
hi def link qbProgress Special
Are there packages to color-highlight jags amd bugs model files? I have ESS installed, but it doesn't seem to recognize .bug files or jags/bugs syntax out of the box.
Syntax highlighting
I'm using ESS 5.14 (from ELPA) and syntax highlighting or smart underscore works fine for me with GNU Emacs 24.1.1. If you want to highlight a given file, you can try M-x ess-jags-mode or add a hook to highlight JAGS file each time, e.g.
(add-to-list 'auto-mode-alist '("\\.jag\\'" . jags-mode))
However, that is not really needed since you can simply
(require 'ess-jags-d)
in your .emacs. There's a corresponding mode for BUGS file. This file was already included in earlier release (at least 5.13), and it comes with the corresponding auto-mode-alist (for "\\.[jJ][aA][gG]\\'" extension).
(Please note that there seems to exist subtle issue with using both JAGS and BUGS, but I can't tell more because I only use JAGS.)
Running command file
If you want to stick with Emacs for running JAGS (i.e., instead of rjags or other R interfaces to JAGS/BUGS), there's only one command to know:
As described in the ESS manual, when working on a command file, C-c C-c should create a .jmd file, and then C-c C-c'ing again should submit this command file to Emacs *shell* (in a new buffer), and call jags in batch mode. Internally, this command is binded to a 'Next Action' instruction (ess-*-next-action). For example, using the mice data that comes with JAGS sample files, you should get a mice.jmd that looks like that:
model in "mice.jag"
data in "mice.jdt"
compile, nchains(1)
parameters in "mice.in1", chain(1)
initialize
update 10000
update 10000
#
parameters to "mice.to1", chain(1)
coda \*, stem("mice")
system rm -f mice.ind
system ln -s miceindex.txt mice.ind
system rm -f mice1.out
system ln -s micechain1.txt mice1.out
exit
Local Variables:
ess-jags-chains:1
ess-jags-command:"jags"
End:
Be careful with default filenames! Here, data are assumed to be in file mice.jdt and initial values for parameters in mice.in1. You can change this in the Emacs buffer if you want, as well as modify the number of chains to use.
I'm using VIM 7.1 on Debian. I have 9 plugins that I load via pathogen.vim. It takes around 8 sec's to load which is quite slow since this is in non-GUI/xterm mode. I ran vim -V and it shows that each module is being searched for in multiple directories.
Initially, ftoff.vim, debian.vim and other "system" related .vim files are searched for in ~/.vim/ and then in /usr/share/vim/vim71/ - I fixed this by moving my .vimrc to .vim/vimrc and: export VIM=/root/.vim, within .vimrc i did a set runtimepath=/usr/share/vim/vim71
But now, when the modules load, they alter this runtimepath and when pathogen loads it's even worse. Is there a way to specify a hash of module-name to dirPath so that this error prone lookup is avoided? Or a way to manually specify runtimepath on a per module basis within vimrc?
Here is an example of my runtimepath after pathogen loads my modules. Obviously, any further loading of a module invovles searching all those pathnames before locating the right path.
runtimepath=~/.vim,~/.vim/bundle/Align294,~/.vim/bundle/minibufexpl.vim_-_Elegant_buffer_explorer,~/.vim/bu
ndle/The_NERD_Commenter,~/.vim/bundle/The_NERD_tree,~/.vim/bundle/pathogen,~/.vim/bundle/vim-addon-mw-utils,
/.vim/bundle/tlib,~/.vim/bundle/snipMate,~/.vim/bundle/SuperTab,~/.vim/bundle/surround,~/.vim/bundle/taglist
~/.vim/bundle/Align294,~/.vim/bundle/minibufexpl.vim_-_Elegant_buffer_explorer,~/.vim/bundle/pathogen,~/.vim
bundle/snipMate,~/.vim/bundle/SuperTab,~/.vim/bundle/surround,~/.vim/bundle/taglist,~/.vim/bundle/The_NERD_C
mmenter,~/.vim/bundle/The_NERD_tree,~/.vim/bundle/tlib,~/.vim/bundle/vim-addon-manager,~/.vim/bundle/vim-add
n-manager-known-repositories,~/.vim/bundle/vim-addon-mw-utils,/var/lib/vim/addons,/usr/share/vim/vimfiles,/u
r/share/vim/vim71,/usr/share/vim/vimfiles/after,/var/lib/vim/addons/after,~/.vim/bundle/snipMate/after,~/.vi
/after,~/.vim/bundle/snipMate/after
I use vim-addon-manager and have 33 paths in rtp, but it takes around 0.7-0.8 seconds to start and immideately close vim (with vim -c 'qa!'), so the problem is either one of the plugins or your system. To check how long it takes to load each plugin, try the following script:
vim --cmd 'profile start profile.log' \
--cmd 'profile func *' \
--cmd 'profile file *' \
-c 'profdel func *' \
-c 'profdel file *' \
-c 'qa!'
You will get all timings in the profile.log. Table with function timings will be
present at the end of the file, to get per-script timings, use the following
script:
" Open profile.log file in vim first
let timings=[]
g/^SCRIPT/call add(timings, [getline('.')[len('SCRIPT '):], matchstr(getline(line('.')+1), '^Sourced \zs\d\+')]+map(getline(line('.')+2, line('.')+3), 'matchstr(v:val, ''\d\+\.\d\+$'')'))
enew
call setline('.', ['count total (s) self (s) script']+map(copy(timings), 'printf("%5u %9s %8s %s", v:val[1], v:val[2], v:val[3], v:val[0])'))
This will open a new file containing just the same table as at the end of
profile.log, but 1) for scripts, not for functions, 2) unsorted.
If problem is your system, you may try the following:
When computer starts create a ram disk and mount it to ~/.vim, then copy all plugins there.
Try merging plugins into a single file, see :h scriptmanager2#MergePluginFiles() (vim-addon-manager must be activated)
Upgrade your computer
Try creating a hardlinks to all plugins in ~/.vim:
cd ~/.vim/bundle;for d in *;do cd "$d";for f in **/*.vim;do t="$HOME/.vim/$(dirname "$f")";test -d "$t"||mkdir -p "$t";ln "$f" "$t";done;cd ..;done
it might not be related, but for me the variable DISPLAY makes a big difference in the time it takes to start vim (even when I have vim compiled without gui).
Try with
DISPLAY= vim
and
DISPLAY=:0 vim
and see if you notice a difference.
http://pastebin.com/R6E4czN7
I've pasted the output of vim -V to pastebin (should be valid for 1 month). It's self explanatory. There are a gazillion searches(414 search lines - most of them are useless). I need to reduce the number of incorrect searches.
1297651453.71068: Searching for "/root/.vim/bundle/pathogen/autoload/scriptmanager.vim"[J
1297651453.71456: Searching for "/root/.vim/bundle/snipMate/autoload/scriptmanager.vim"[J
1297651453.71846: Searching for "/root/.vim/bundle/SuperTab/autoload/scriptmanager.vim"[J
1297651453.78737: Searching for "/root/.vim/bundle/surround/autoload/scriptmanager.vim"[J
1297651453.79179: Searching for "/root/.vim/bundle/taglist/autoload/scriptmanager.vim"[J
1297651453.79684: Searching for "/root/.vim/bundle/The_NERD_Commenter/autoload/scriptmanager.vim"[J
1297651453.80756: Searching for "/root/.vim/bundle/The_NERD_tree/autoload/scriptmanager.vim"[J
1297651453.83: Searching for "/root/.vim/bundle/tlib/autoload/scriptmanager.vim"[J
1297651453.86193: Searching for "/root/.vim/bundle/vim-addon-manager/autoload/scriptmanager.vim"[J
1297651453.8662: line 3: sourcing "/root/.vim/bundle/vim-addon-manager/autoload/scriptmanager.vim"[J
1297651453.88259: finished sourcing /root/.vim/bundle/vim-addon-manager/autoload/scriptmanager.vim[J
the -w and -W options of vim have theoretically the following effect:
-w {scriptout} All the characters that you type are recorded in the file
"scriptout", until you exit Vim.
This is useful if you want to create
a script file to be used with "vim -s"
or ":source!". When the "scriptout"
file already exists, new characters
are appended. See also
|complex-repeat|. {scriptout} cannot
start with a digit. {not in Vi}
-W {scriptout} Like -w, but do not append, overwrite an existing file.
{not in Vi}
But when I do this, the {scriptout} file will always begin with a hexadecimal sequence like 80 fd 60 (sometimes it is 80 fd 62).
I am using gvimportable.exe 7.3 from portableapps.com. With the -u NONE switch, it does the same.
What is this “magic number” for? Under Windows with gvim.exe I cannot replay my scriptout until I have removed those three leading bytes…
It seems that this feature, which could be very useful, is poorly documented.
Thank you for your answers.
(This answer is probably fragmented significantly, it took me a while playing around - I wanted to find a solution too because it intrigued me - not just the bounty of 200 :P. It more or less shows my train of thought and experimentation.)
I can now reproduce it with gvim on Linux, which is /usr/bin/vim.gnome -g; running as vim -g does just the same.
Delving into the code: (futile in this case, but fun to do and to learn how to do)
I've looked through the source code and I can now explain it somewhat (but not usefully!); it gets the outfile FILE (src/globals.h:1004) set (src/main.h:2275); this is then written to in src/getchar.h:1501, in the updatescript method which is used by gotchars (line 1215) which is used by vgetorpeek, which is used by vgetc and vpeekc... (no, I don't know where this is going!) then these are used in a number of places.
Anyway, I suppose the key is somewhere in src/gui.c, but I don't know where at the moment! It's also possible that some key sequence is being "sent" (physically or virtually, I don't know), but seeing as the issue is the same across platforms it would seem more likely to be a Vim issue than otherwise.
Interesting situations leading to a probable explanation:
It's also worth while noting that if you automatically quit, gvim -u NONE -w scriptout -c quit (:quit after loading) or gvim -u NONE -w scriptout -c quit (instant :quit, never shows GUI), the file scriptout is left empty.
Additionally, if you open gvim and then close it using the X button, pressing no keys:
0000000: 80fd 6280 fd63 80fd 62 ..b..c..b
If you open gvim, click away, click back and use :q:
0000000: 80fd 6280 fd63 80fd 6280 fd2c 80fd 2e3a ..b..c..b..,...:
0000010: 710d q.
So I think it's the window events are internally translated into something else. 80 fd 62 is the open sequence and 80 fd 63 80 fd 62 is the close sequence.
I've found another way of triggering 80fd as well, which leads me to thing it's some sort of "user has access to the window"; by default with GNOME in Ubuntu, Ctrl+Alt+S does something with the window (can't remember what it's called; slides it all up into the title bar, app inside loses keyboard control etc.). gvim ... (you know the arguments!), i<Ctrl+Alt+S (contracted) Ctrl+Alt+S (expanded) >Esc Z Q produces this for me:
0000000: 80fd 6269 3c80 fd63 80fd 623e 1b5a 51 ..bi<..c..b>.ZQ
Summary: so there we have what I believe is the solution; gVim catches the window messages in some form and - whether it should or shouldn't - puts them in its scriptout. If you think it shouldn't (or would like to know why they're left in or if they're even meant to be or whether you should care at all), ask on the Vim list, I think.
My best guess is that this is a bug in the GUI code of gVim.
Using gVim 7.3, if I run gvim -u NONE -W scriptout then I see the problem, but if I run vim -u NONE -W scriptout then the unwanted bytes are not present.
I also tested Vim 7.2 from the shell in Linux, the version of Vim included in Snow Leopard (7.2), and the GUI and terminal versions of MacVim 7.2 (with mvim -W and /Applications/MacVim/Contents/MacOS/Vim -W, respectively) and they all worked correctly.
Someone has done the hard work for us in the vimgolf project, in particular this well-commented file: https://github.com/igrigorik/vimgolf/blob/master/lib/vimgolf/lib/vimgolf/keylog.rb
0x80 in escape sequence for special two-byte codes. In this case they represent gvim focus events. See here:
# If you use gvim, you'll get an entry in your keylog every time the
# window gains or loses focus. These "keystrokes" should not show and
# should not be counted.
"\xfd\x60" => nil, # 7.2 Focus Gained compat
"\xfd\x61" => nil, # Focus Gained (GVIM) (>7.4.1433)
"\xfd\x62" => nil, # Focus Gained (GVIM)
"\xfd\x63" => nil, # Focus Lost (GVIM)