I'm investigating a mainly UTF-8 file with lot of long lines. However, the file is not entirely text file, there is some garbage. To find my point of interest I'm using hd and grep.
So at some point I know it'm interested in e.g. 0000301a, so I want to quickly open the file in Vim and jump to that position.
Example (actually a tiny file, here the position is 0000001c):
me#here:~$ hd file | grep -C 10 \ 00\
00000000 6c 69 6e 65 31 0a 6c 69 6e 65 32 0a 6c 69 6e 65 |line1.line2.line|
00000010 33 0a 6c 69 6e 65 34 0a 6c 69 6e 65 00 35 0a 6c |3.line4.line.5.l|
00000020 69 6e 65 36 0a 6c 69 6e 65 37 0a 6c 69 6e 65 38 |ine6.line7.line8|
00000030 0a 6c 69 6e 65 39 0a 6c 69 6e 65 31 30 0a |.line9.line10.|
0000003e
me#here:~$
Is there a trick in Vim to jump to a byte position? Or as close as possible?
Yes, there is. It's the normal mode command go.
From :h go:
[count]go Go to {count} byte in the buffer.
For example, 42go jumps to byte 42.
In order to jump to a hexadecimal or octal address you would need to construct a command with :normal! go or the equivalent :goto, and the str2nr() or printf() function.
:exe 'normal! ' . str2nr('0x2a', 16) . 'go'
:exe 'goto' str2nr('0x2a', 16)
While this question has merit of its own, may I suggest that you're doing this whole thing in a rather roundabout fashion, using all of hd, grep, Vim, and the shell?
Why not use Vim and only Vim for the whole process?
The key is at :h 23.4, "Binary files": Vim can be your hex editor.
After starting Vim with your file, vim -b myfile, the workflow becomes:
:%!xxd -g1<Enter>
/00 <Enter>
...
...
:%!xxd -r<Enter>
Be sure to read :h hex-editing, too, for more details.
Yes, using :help starting.txt you can find the help about all vim command line arguments.
Using the above goto command given by several others you should be able to give vim a command from the command line to go to the byte offset such as
vim "+exe 'goto' str2nr('0x2a', 16)" file
There is a compile time flag described at :help +byte_offset you can check for using the :ve command. My Ubuntu version has it compiled in by default.
Related
I am working on python escape characters. I found that \b is
not working for the last indexed character of the string in
Python 3.8.Why is it so?
Input :
print("abc#\bde#\bf#\bghi#\bjklmn#\bop\b#\b")
Output :
abcdefghijklmno#
This is got to do with how the terminal interprets escape characters.
It had been reported for Python 2 a few years back.
Quoting:
Python itself doesn’t treat backspace specially. What you are probably seeing is the terminal interpreting the backspace specially by moving the cursor left (without erasing anything).
This means that, if you have a character after \b, the terminal will move the cursor left and "overwrite" the original character with the new one.
If there are no other characters, nothing will overwrite the old character.
Trying piping the output to hexdump using your example shows:
$ python3.8 -c
'print("abc#\bde#\bf#\bghi#\bjklmn#\bop\b")' | hexdump -C
00000000 61 62 63 23 08 64 65 23 08 66 23 08 67 68 69 23 |abc#.de#.f#.ghi#|
00000010 08 6a 6b 6c 6d 6e 23 08 6f 70 08 0a |.jklmn#.op..|
0000001c
You can notice the 08 character there (\b).
I have a file that I am trying to find some items in.
Here is an example extract:
68 65 6C 6C 6F 20 77 6F 72 6C 64 20 68 6F 77 20 61 72 65 20 79 6F 75 20 68 65 6C 6C 6F 20 77 6F 72 6C 64
In my editor it shows up as:
hello world how are you hello world
I used Ctrl + f to bring up Find.
I entered what I was looking for (hello) in Search for and pressed Search. The first result shows up, but my file contains multiple results as you can see above. How do I move to the next result?
I.e. from the first hello, to the second.
For the version of HxD I am using (1.7.7.0), this function is bound to F3. To search in the reverse direction, press Shift+F3.
Alternatively, both options are selectable under 'Search' located on the menu bar.
This is an example of my case function:
function SendToScreen(){
echo -e "$*"
}
So I call it by:
SendToScreen "Hello"
And, if I want to add color codes:
VioletForeGroundColor="\033[38;5;99m"
NormalColor="\033[0m"
SendToScreen "Hello"$VioletForeGroundColor" violet "$NormalColor" word."
That gives me a correct:
But the problem comes if I want to send some DOS-type path (including \ slash):
VioletForeGroundColor="\033[38;5;99m"
NormalColor="\033[0m"
MyDOSPath="d:\vivisector"
SendToScreen "Hello"$VioletForeGroundColor" violet "$NormalColor" word. The path is $MyDOSPath"
Because \v is some sort of ANSI code, so this time I obtain:
I need my function to output color text (bold, cursive, underline... etc), so I must use echo -e.
How could I solve the problem with such nagging control codes colliding characters like this \v (I suppose there will be another ones)?
I would like to repair the isssue by modifying the function, but I am not sure this is the proper method.
Thanks.
EDIT-1: We will choose \033 also known as \e as the only ANSI code that needs to remain.
New answer:
function SendToScreen() {
echo -e $(echo "${*//\\/\\\\}" | sed 's/\\\\033\[/\\033\[/g');
}
This one escapes everything, then un-escapes anything that looks like a color sequence (\033[). The possibility of sending filenames as color sequences is greatly reduced. You can reduce it even further by white-listing only those color sequences that you want to allow, and changing the sed command to a sequence of sed commands that un-escapes those exact sequences.
Old answer:
Let's say you want to escape \v and \n, you can do this:
function SendToScreen(){
a="${*//\\v/\\\\v}"
a="${a//\\n/\\\\n}"
echo -e "$a"
}
You can extend this with whatever other escapes you don't want to process.
The echo -e simply interprets sequences starting with backslash, so you simply need to ensure that the $MyDOSPath argument has all backslashes doubled up. That could be:
SendToScreen "Hello ${VioletForeGroundColor}violet${NormalColor} word." \
"The path is ${MyDOSPath//\\/\\\\}"
which uses a 'substitute' parameter expansion. The // means 'change every backslash to double backslash'.
As discussed in various comments, maybe the design of SendToScreen is sub-optimal. One possible alternative design uses:
SendToScreen [-e "string-to-expand"][-p "plain-string"] [-- "plain strings"]
Arguments that need to be expanded are, and those that should not be expanded are not. By default, they're not. So, example usage:
$ VioletForeGroundColor="\033[38;5;99m"
$ NormalColor="\033[0m"
$ MyDOSPath="C:\new\table\value\alert\form\033.txt"
$ echo "$MyDOSPath"
C:\new\table\value\alert\form\033.txt
$ bash SendToScreen.sh -e "${VioletForeGroundColor}violet${NormalColor}" -e "The path is ${MyDOSPath//\\/\\\\}" -p "Or $MyDOSPath" "Plain $MyDOSPath"
violet The path is C:\new\table\value\alert\form\033.txt Or C:\new\table\value\alert\form\033.txt Plain C:\new\table\value\alert\form\033.txt
$ bash SendToScreen.sh -e "${VioletForeGroundColor}violet${NormalColor}" -e "The path is ${MyDOSPath//\\/\\\\}" -p "Or $MyDOSPath" -e "Oops! $MyDOSPath" "Plain $MyDOSPath"
violet The path is C:\new\table\value\alert\form\033.txt Or C:\new\table\value\alert\form\033.txt Oops! C: ew able
aluelert
orm.txt Plain C:\new\table\value\alert\form\033.txt
$
A hex dump of the last lot of output was:
0x0000: 1B 5B 33 38 3B 35 3B 39 39 6D 76 69 6F 6C 65 74 .[38;5;99mviolet
0x0010: 1B 5B 30 6D 20 54 68 65 20 70 61 74 68 20 69 73 .[0m The path is
0x0020: 20 43 3A 5C 6E 65 77 5C 74 61 62 6C 65 5C 76 61 C:\new\table\va
0x0030: 6C 75 65 5C 61 6C 65 72 74 5C 66 6F 72 6D 5C 30 lue\alert\form\0
0x0040: 33 33 2E 74 78 74 20 4F 72 20 43 3A 5C 6E 65 77 33.txt Or C:\new
0x0050: 5C 74 61 62 6C 65 5C 76 61 6C 75 65 5C 61 6C 65 \table\value\ale
0x0060: 72 74 5C 66 6F 72 6D 5C 30 33 33 2E 74 78 74 20 rt\form\033.txt
0x0070: 4F 6F 70 73 21 20 43 3A 20 65 77 20 61 62 6C 65 Oops! C: ew able
0x0080: 0B 61 6C 75 65 07 6C 65 72 74 0C 6F 72 6D 1B 2E .alue.lert.orm..
0x0090: 74 78 74 20 50 6C 61 69 6E 20 43 3A 5C 6E 65 77 txt Plain C:\new
0x00A0: 5C 74 61 62 6C 65 5C 76 61 6C 75 65 5C 61 6C 65 \table\value\ale
0x00B0: 72 74 5C 66 6F 72 6D 5C 30 33 33 2E 74 78 74 0A rt\form\033.txt.
0x00C0:
You'll have to take my word for it that violet appeared in violet.
Clearly, the user (caller) of SendToScreen has to know which arguments should be expanded and which should not. However, it makes it very explicit.
Here's the code I used as a script. Repackaging as a function is left as an exercise for the reader. Extending it to add -c colour (or maybe -f foreground and -b background) is an exercise for the reader.
#!/bin/bash
output=()
while getopts "p:e:" opt
do
case "$opt" in
(e) output+=( $(echo -e "$OPTARG") );;
(p) output+=( "$OPTARG" );;
esac
done
shift $(($OPTIND - 1))
echo "${output[#]}" "$#"
Have fun!
I have a bash file with the content
cd /var/www/path/to/folder
git pull
When I run it I get
: No such file or directorywww/path/to/folder
' is not a git command. See 'git --help'.
Did you mean this?
pull
Any idea why bash gets a truncated version of commands?
You have carriage returns (Windows text file line endings) in your bash script. Remove them.
The bash file should look like this under hexdump -C:
00000000 63 64 20 2f 76 61 72 2f 77 77 77 2f 70 61 74 68 |cd /var/www/path|
00000010 2f 74 6f 2f 66 6f 6c 64 65 72 0a 67 69 74 20 70 |/to/folder.git p|
00000020 75 6c 6c 0a |ull.|
00000024
But yours looks like this instead:
00000000 63 64 20 2f 76 61 72 2f 77 77 77 2f 70 61 74 68 |cd /var/www/path|
00000010 2f 74 6f 2f 66 6f 6c 64 65 72 0d 0a 67 69 74 20 |/to/folder..git |
00000020 70 75 6c 6c 0d 0a |pull..|
Note the extra 0d's (hex 0D = decimal 13 = ASCII carriage return, ANSI \r) in front of the 0as (hex 0A = decimal 10 = ASCII linefeed, ANSI \n, which is what bash treats as the end of a line).
A carriage return is not whitespace in bash, so it is treated as part of the last argument on the command line. You're getting errors because the folder /var/www/path/to/folder.git\r doesn't exist and pull\r isn't a valid git subcommand.
When printed, a carriage return moves the cursor to the start of the line, which is why your error messages look wrong. Bash and git are printing something like foo.bash: line 1: cd: /www/path/to/folder\r: No such file or directory and git: 'pull\r' is not a git command. See 'git --help', but after the \r moves the cursor to the start of the line, the tail end of each message overwrites its beginning.
There's a program called dos2unix that converts a text file from DOS to Unix:
dos2unix filename >newfilename
But that conversion really consists of nothing but deleting the carriage returns, which you could also do explicitly with tr:
tr -d '\r' <filename >newfilename
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 2 years ago.
#!/bin/bash
cd ~/workspace/trunk;
svn up;
When I run ./build.sh form command line, it says:
: command not found
And nothing happens. How can I solve this ?
I solved adding execute permissions:
sudo chmod +x file.sh
I Have resolved my error from this command.
sudo chmod +x build.sh
My guess is that you have unprintable control characters in the file, or it has \r\n (CRLF) line endings (dos/windows mode).
Try checking it with these commands:
$ hexdump -C build.sh
00000000 23 21 2f 62 69 6e 2f 62 61 73 68 0a 63 64 20 7e |#!/bin/bash.cd ~|
00000010 2f 77 6f 72 6b 73 70 61 63 65 2f 74 72 75 6e 6b |/workspace/trunk|
00000020 3b 0a 73 76 6e 20 75 70 3b 0a |;.svn up;.|
0000002a
$ file build.sh
build.sh: Bourne-Again shell script, ASCII text executable
$ unix2dos build.sh
unix2dos: converting file build.sh to DOS format ...
$ hexdump -C build.sh
00000000 23 21 2f 62 69 6e 2f 62 61 73 68 0d 0a 63 64 20 |#!/bin/bash..cd |
00000010 7e 2f 77 6f 72 6b 73 70 61 63 65 2f 74 72 75 6e |~/workspace/trun|
00000020 6b 3b 0d 0a 73 76 6e 20 75 70 3b 0d 0a |k;..svn up;..|
0000002d
$ file build.sh
build.sh: Bourne-Again shell script, ASCII text executable, with CRLF line terminators
None of the above worked for me except
sudo ./build.sh
Remove ; from the end of your script lines.
This doesn't happen in my bash, so I'm not sure what exactly is wrong, but my guess is this:
; is a separator of commands. Since your last command ends in ;, your bash probably expects another command after. Since the script finishes, though, it reads an empty command, which it can't execute.