How to enter NUL character through keyboard. Example, like for for ESC character we are using ctrl+Esc. For your info: I'm using win 2008 OS.
Related
This question already has answers here:
What does the ^M character mean in Vim?
(15 answers)
Closed 2 years ago.
I'm new to Linux sorry if my question sounds dumb.
We know that Linux and Mac OS X use \n (0xa), which is the ASCII line feed (LF) character. MS Windows and Internet protocols such as HTTP use the sequence \r\n (0xd 0xa). If you create a file foo.txt in Windows and then view it in a Linux text editor, you’ll see an annoying ^M at the end of each line, which is how Linux tools display the CR character.
Bu why Linux tools display the CR character as ^M? as my understanding is, \r (carriage return) is to move the cursor in the beginning of the current line, so the sensible approach to display it is like, when you open the file, you see the cursor is in the beginning of the line(that contains \r), so ^M shouldn't be displayed?
PS: some people post answers that how to remove ^M, but I wnat to know why eventually^M is displayed rather than moving the cursor in the beginning, which is the definition of carriage return.
The ASCII control characters like TAB, CR, NL and others are intended to control the printing position of a teletypewriter-like display device.
A text editor isn't such a device. It is not appropriate for a text editor to treat a CR character literally as meaning "go to the first column"; it would make a confusing gibberish out of the editing experience.
A text editor works by parsing a text file's representation, to create an internal representation which is presented to the user. On Unix-like operating systems, a file is represented by zero or more lines, which are terminated by the ASCII NL character. Any CR characters that occur just look like part of the data, and not part of the line separation.
Not all editors behave the same way. For instance, the Vim editor will detect that a file uses CR-LF line endings, and load it properly using that representation. A flag is set for that buffer which indicates that it's a "DOS" file, so that when you save it, the same representation is reproduced.
That said, there is a feature in the actual Linux kernel for representing control characters like CR using the ^M notation. The TTY line discipline for any given TTY device can be configured to print characters in this notation, but only when echoing back the characters received.
Demo:
$ stty echoctl # turn on notational echo of control characters
$ cat # run some non-interactive program with rudimentary line input
^F^F^F^F^F^F
^C
$
Above, the Ctrl-F that I entered was echoed back as ^F. So, in fact there is a "Linux editor" which uses this notation: the rudimentary line editor of the "canonical input mode" line discipline.
I've been tinkering with multiple hex editors but nothing really has worked.
What I'm looking for is a way to change a binary in actual binary (not in hex). This is purely for educational purposes and I know it's trivial to convert between both, but I wanted to be able to change the ones and zeroes just like I would do hex.
I've tried using vim with the %!xxd -b but then it won't work with %!xxd -r. I know how to convert the file into binary, but I'm looking for a way to dynamically change it in this format and being able to save it.
Better yet would be if I could find a way to actually create a binary by coding purely in actual binary.
Any help would be appreciated :D
vim or gvim should work for you directly, without the xxd filter.
Open the file in (g)vim. Place your cursor on a character and type ga to see its character code in the status line. To insert character NNN, place your cursor where you want it, go in insert mode and type Ctrl-v and then the three digit decimal code value. Use Ctrl-v x HH to enter the character by its hexadecimal code.
Make sure your terminal is not set to use UTF8, because in UTF8, typing Ctrl-v 128 will in fact insert c280, the utf-8 encoding of character 128, instead of 80.
LC_ALL=C vim binary-file
is the easiest way to make sure you're doing binary character based editing in vim, but that might do weird things if your terminal is utf-8.
LC_ALL=C gvim binary-file
should open a stand-alone window with proper display.
FYI, if you did want to work in utf-8, Ctrl-v u HHHH is how to enter the Unicode character with Hex code point HHHH.
windows
open cmd.exe or notepad++ or whatever editor
enable numlock key
On laptops you need to use the function key or the blue / grey silver numbers above alphabet keys (using the numbers on the top line will not work as they map to different scan code.
press alt key + 255 will correspond to 0xff
press alt key + 254 will correspond to 0xfe
see below for a demo
C:\>copy con rawbin.bin
■²ⁿ√·∙⌂~}─^Z
^Z
1 file(s) copied.
C:\>xxd rawbin.bin
0000000: fffe fdfc fbfa f97f 7e7d c41a 0d0a ........~}....
C:\>
Let's to create target file to operate with.
python3
>>> mfile = open("f:/test.txt","wb")
>>> mfile.write(b'\xe3\x80\x80')
3
>>> mfile.close()
Now to open f:/test.txt with xxd,you will see three bytes \xe3\x80\x80 in it,our target file encoding with utf-8 contains three bytes \xe3\x80\x80.
python3
b'\xe3\x80\x80'.decode('utf-8')
'\u3000'
It means that the unicode of three bytes in test.txt encoding with utf-8 is 3000.
:s/\%u3000/ /g
s/\%u3000/ /g can replace bytes \xe3\x80\x80 with byte \x20 in vim.
Issue remains still here.
:s/\%u3000/\%u20/g
:s/\%u3000/\%x20/g
:s/\%u3000/\x20/g
All the three formats above here can't work,why \xe3\x80\x80 can be expressed by \%u3000 in vim, (white blank) can't be expressed by \%u20 or \%x20 or \x20 ?
can express \x20, white blank is printable character,what's more, i want to replace the three bytes \xe3\x80\x80 with latin-1's nbsp?
The nbsp in latin-1 encoding means Non-breaking space which is NON PRINTABLE CHARACTERS,how to write the expression in vim?
:s/\%u3000/\%ua0/g
:s/\%u3000/\%xa0/g
:s/\%u3000/\xa0/g
None of them can work for the case.
You can type the \xe3\x80\x80 or u3000 character by pressing ctrl+v then u and then the 4 Unicode characters, in your case 3000 (check :help i_CTRL-V_digit ), since is a black character you will see nothing but just a space, you could type :set list to see all the places where you have that character or in any case add this to your .vimrc
set listchars=tab:▸\ ,eol:¬,trail:·,extends:#,nbsp:.
Now in the same way you enter the character, you could try to replace it within the command line, but in this case to be available to enter the ctrl+v you could try using the command-line window (:help cedit).
Go to command mode and after having the : press ctrl+f it will open the command-line window in where you could go into insert mode and type: %s/ctrl+vu3000/ /g and when done press enter to apply command.
Give a try first before entering the command-line window, since when using ctrl+v it may work, not like when using ctrl+k (http://vim.wikia.com/wiki/Entering_special_characters)
In the image instead of replacing with a white space / /, Is replacing with ---- just to visually see the changes.
1.How to input non printable characters when to edit a file in vim?
In the insert mode:
1.ctrl+v (ctrl+q if ctrl+v call paste from regitor)
2.input u
3.input the unicode value of non printable characters
4.input enter key
2.How to input non printable characters in substitute command of vim's ex mode?
For example, to replace all bytes \xe3\x80\x80 with \xa0,all byte's encoding is utf-8.
1. get the byte's unicode value
`\xe3\x80\x80`'s unicode value is `3000`,
`\xa0`'s unicode value is `a0`.
2.press `:` into ex mode.
3.:s/\%u3000/
4:ctrl+v ua0
do not input enter as above process
5.go on to input `/g`.
6.press enter.
In this particular script they use \n.
#!/usr/bin/expect
set password [lindex $argv 0]
spawn asadmin enable-secure-admin
expect "admin"
send "admin\n"
expect "password"
send "$password\n"
expect eof
exit
Question
Could \r just as well have been used? If not, what are the differences?
\n is linefeed, Ctrl-J or character 012. \r is carriage return, Ctrl-M or character 015.
In an interactive unix context, when you're typing them (or simulating typing them, as with expect), they are interchangeable. Linefeed is the formal line-terminator character, but tty devices normally translate carriage return to linefeed on input. On your keyboard, the BKWA (big key with arrow which might be labelled "enter" or "return"), sends a Ctrl-M, which the tty device will translate to Ctrl-J. If your BKWA is broken, you can actually type Ctrl-M or Ctrl-J and it'll work just as well.
On output they're not interchangeable. As I said, linefeed is the formal line terminator character, so programs (and text files) will indicate end-of-line with a linefeed. When a linefeed is output to a tty, the tty will normally translate it into a carriage-return-linefeed pair.
When the characters actually reach the display device, carriage return moves the cursor to the beginning of the row, while linefeed moves it down a row.
Thats is how this special character is displayed in vim:
Ive tryed with /\x20(\x0e|\x0f)/ and /\xe2\x80[\x8e\x8f]/ without results.
First, if you want to replace byte 0x20 (it is space, if I am not mistaking), you need to use \%x20, not \x20 because \x designates a hex digit (unless used inside a collection, there \x20 means what expected). But if you want to replace given unicode character, you should use \%u200E (\u200E inside a collection).
Second, both \%x20 and [\x20] will match character with unicode code 0x20, not byte with code 0x20. It does not matter for the space, but makes difference for code points >0x7F.
Try to replace \u200e :)
You can test this works by inserting that character into your buffer, and seeing that it appears as <200e>, if you type this in while in insert mode: <C-R>="\u200e"<CR> (that's CTRL+R and <CR> means ENTER)
I would put the cursor on the blue <200e>, then type yl to yank (copy) the character.
Then, type :%s/<C-R>"/replacement/g
(where <C-R> is Control+R, of course).
Use your terminal's mechanism for entering characters by Unicode codepoint. In the case of gnome-terminal, that's CtrlShiftU followed by the hex code (e.g. 200e) and then Enter.