vim doesn't display cp1252 characters - vim

I'm running gvim, under Windows. I pasted some text from a web page but vim does not display the hyphen and smart quotes.
When I check the encoding that vim is using (:set enc) vim reports that it is using cp1252.
When I check the hex value of the codes under the cursor (ga) vim reports the correct cp1252 code values (0x96, 0x93, and 0x94).
And yet it does display the smart single quotes (0x91 and 0x92)
Can anyone explain what is happening?
Thanks,
Steve

The "DejaVu_Sans_Mono" and "Lucinda_Console" fonts both displayed the characters with codes between 0xA0 and 0x9F correctly. There are probably others that do so as well. I'm using gvim, so I selected them from the menubar (Edit/Select Font). Subsequently I added the line "set guifont=DejaVu_Sans_Mono:h12" to my _vimrc file. The "h12" specifies font size. – justerman

Related

Missing glyph character and garbage replace leading 5 characters of file name when written to disk

FreeBSD-12.1p5
vim-8.2
I have a zfs clone of an iocage thick jail that I am using for testing. When I open a file with vim in character mode and I use :wq to save my changes then I see these characters in place of those expected (/usr/:)
[>4;mlocal/etc/smb4.conf" 24L, 672C written
There is a missing glyph character (a single square box one character in width and height with hex decimal characters [0..F] inside) that is positioned immediately before the [ character in the sample given above. This character disappears when this message is displayed.
This does not happen if I use :w by itself, only when I use :wq. It appears that vim actually displays the correct string when I use :wq but that the first few characters are immediately overwritten.
This is not a problem with the file name or path, only with what vim displays when saving.
I have not encountered this before. Why are these characters messed up when vim displays the name of the file it is saving?
I have verified all packages are intact and all dependencies installed. I have also reinstalled vim. The problem remains.
Should be resolved by adding these lines to your .vimrc:
set t_TI=
set t_TE=

Editing in binary

I've been tinkering with multiple hex editors but nothing really has worked.
What I'm looking for is a way to change a binary in actual binary (not in hex). This is purely for educational purposes and I know it's trivial to convert between both, but I wanted to be able to change the ones and zeroes just like I would do hex.
I've tried using vim with the %!xxd -b but then it won't work with %!xxd -r. I know how to convert the file into binary, but I'm looking for a way to dynamically change it in this format and being able to save it.
Better yet would be if I could find a way to actually create a binary by coding purely in actual binary.
Any help would be appreciated :D
vim or gvim should work for you directly, without the xxd filter.
Open the file in (g)vim. Place your cursor on a character and type ga to see its character code in the status line. To insert character NNN, place your cursor where you want it, go in insert mode and type Ctrl-v and then the three digit decimal code value. Use Ctrl-v x HH to enter the character by its hexadecimal code.
Make sure your terminal is not set to use UTF8, because in UTF8, typing Ctrl-v 128 will in fact insert c280, the utf-8 encoding of character 128, instead of 80.
LC_ALL=C vim binary-file
is the easiest way to make sure you're doing binary character based editing in vim, but that might do weird things if your terminal is utf-8.
LC_ALL=C gvim binary-file
should open a stand-alone window with proper display.
FYI, if you did want to work in utf-8, Ctrl-v u HHHH is how to enter the Unicode character with Hex code point HHHH.
windows
open cmd.exe or notepad++ or whatever editor
enable numlock key
On laptops you need to use the function key or the blue / grey silver numbers above alphabet keys (using the numbers on the top line will not work as they map to different scan code.
press alt key + 255 will correspond to 0xff
press alt key + 254 will correspond to 0xfe
see below for a demo
C:\>copy con rawbin.bin
 ■²ⁿ√·∙⌂~}─^Z
^Z
1 file(s) copied.
C:\>xxd rawbin.bin
0000000: fffe fdfc fbfa f97f 7e7d c41a 0d0a ........~}....
C:\>

How to display UTF-8 characters in Vim correctly

I want/need to edit files with UTF-8 characters in it and I want to use Vim for it.
Before I get accused of asking something that was asked before, I've read the Vim documentation on encoding, fileencoding[s], termencoding and more, googled the subject, and read this question among other texts.
Here is a sentence with a UTF-8 character in it that I use as a test case.
From Japanese 勝 (katsu) meaning "victory"
If I open the (UTF-8) file with Notepad it is displayed correct.
When I open it with Vim, the best thing I get is a black square where the Japanese character for katsu should be.
Changing any of the settings for fileencoding or encoding does not make a difference.
Why is Vim giving me a black square where Notepad is displaying it without problems? If I copy the text from Vim with copy/paste to Notepad it is displayed correctly, indicating that the text is not corrupted but displayed wrong. But what setting(s) have influence on that?
Here is the relevant part of my _vimrc:
if has("multi_byte")
set encoding=utf-8
if &termencoding == ""
let &termencoding = &encoding
endif
setglobal fileencoding=utf-8
set fileencodings=ucs-bom,utf-8,latin1
endif
The actual settings when I open the file are:
encoding=utf-8
fileencoding=utf-8
termencoding=utf-8
My PC is running Windows 10, language is English (United States).
This is what the content of the file looks like after loading it in Vim and converting it to hex:
0000000: efbb bf46 726f 6d20 4a61 7061 6e65 7365 ...From Japanese
0000010: 20e5 8b9d 2028 6b61 7473 7529 206d 6561 ... (katsu) mea
0000020: 6e69 6e67 2022 7669 6374 6f72 7922 0d0a ning "victory"..
The first to bytes is the Microsoft BOM magic, the rest is just like ASCII except for the second, third and fourth byte on the second line, which must represent the non-ASCII character somehow.
There are two steps to make Vim successfully display a UTF-8 character:
File encoding. You've correctly identified that this is controlled by the 'encoding' and 'fileencodings' options. Once you've properly set this up (which you can verify via :setlocal filenencoding?, or the ga command on a known character, or at least by checking that each character is represented by a single cell, not its constituent byte values), there's:
Character display. That is, you need to use a font that contains the UTF-8 glyphs. UTF-8 is large; most fonts don't contain all glyphs. In my experience, that's less of a problem on Linux, which seems to have some automatic fallbacks built in. But on Windows, you need to have a proper font installed and configured (gVim: in guifont).
For example, to properly display Japanese Kanji characters, you need to install the far eastern language support in Windows, and then
:set guifont=MS_Gothic:h12:cSHIFTJIS

how to get the digraph key or unicode from a special character

I am configuring tmux & powerline. I would like to change the default separators in tmux status line. I managed to figure out where to do it, however I could not get the special character that I want.
I want to type, in Vim, a left/right pointing triangle that spans the whole line-height, but the only thing I could find is a small triangle (unicode : 25B6,25BA,25C0,25C4...)
There is a big right pointing triangle already in a powerline configuration file, which I could copy and paste, but I want to know its unicode and want a left one. Is there a way to get the unicode from the symbol in Vim or anywhere else?
You can get the codepoint value of a character in Vim by positioning the cursor on the character in question and typing either ga or :ascii.
You can either use ga in command mode or :ascii on the command line (even though it says ascii, it shows information for general encodings)
You may add a permanent preview of current character in your status line (see :h statusline), eg.:
:let &statusline = &statusline . "\ [%03b\ 0x%B]"

Vim UTF-8 encoding error on Windows

I have a text file with Polish characters. As long as I do not set :set encoding=utf-8 the characters are not displayed correctly. As soon as I set it to Unicode the characters are displayed but umlauts in error messages in Vim on the other hand are not displayed anymore.
Example:
E37: Kein Schreibvorgang seit der letzten <c4>nderung (erzwinge mit !)
Instead of the <c4> there should be the character Ä displayed. Can anybody explain me why this happens?
I'm experiencing similar issues (you can view some of the questions in my account info, or search for "central european characters" or "croatian characters").
Changing the encoding value changes the way Vim displays the characters - so, the way some of the characters are displayed is changed - that's why you're getting characters. You could probably solve your problem of Polish characters by choosing some other encoding value (one of the cpXXXX for example instead of utf8), but then you would lose the ability to display utf8 characters which can make Vim rather pretty. At least this works for my case (Croatian).
So, either use while writing polish texts one of the cpXXXX encoding values, or stick to utf8 completely. I recommend the first one. But do not change them.
Still working on that here.

Resources