I have a bash file with the content
cd /var/www/path/to/folder
git pull
When I run it I get
: No such file or directorywww/path/to/folder
' is not a git command. See 'git --help'.
Did you mean this?
pull
Any idea why bash gets a truncated version of commands?
You have carriage returns (Windows text file line endings) in your bash script. Remove them.
The bash file should look like this under hexdump -C:
00000000 63 64 20 2f 76 61 72 2f 77 77 77 2f 70 61 74 68 |cd /var/www/path|
00000010 2f 74 6f 2f 66 6f 6c 64 65 72 0a 67 69 74 20 70 |/to/folder.git p|
00000020 75 6c 6c 0a |ull.|
00000024
But yours looks like this instead:
00000000 63 64 20 2f 76 61 72 2f 77 77 77 2f 70 61 74 68 |cd /var/www/path|
00000010 2f 74 6f 2f 66 6f 6c 64 65 72 0d 0a 67 69 74 20 |/to/folder..git |
00000020 70 75 6c 6c 0d 0a |pull..|
Note the extra 0d's (hex 0D = decimal 13 = ASCII carriage return, ANSI \r) in front of the 0as (hex 0A = decimal 10 = ASCII linefeed, ANSI \n, which is what bash treats as the end of a line).
A carriage return is not whitespace in bash, so it is treated as part of the last argument on the command line. You're getting errors because the folder /var/www/path/to/folder.git\r doesn't exist and pull\r isn't a valid git subcommand.
When printed, a carriage return moves the cursor to the start of the line, which is why your error messages look wrong. Bash and git are printing something like foo.bash: line 1: cd: /www/path/to/folder\r: No such file or directory and git: 'pull\r' is not a git command. See 'git --help', but after the \r moves the cursor to the start of the line, the tail end of each message overwrites its beginning.
There's a program called dos2unix that converts a text file from DOS to Unix:
dos2unix filename >newfilename
But that conversion really consists of nothing but deleting the carriage returns, which you could also do explicitly with tr:
tr -d '\r' <filename >newfilename
Related
I'm trying to render the output of a linux shell command in HTML. For example, systemctl status mysql looks like this in my terminal:
As I understand from Floz'z Misc I was expecting that the underlying character stream would contain control codes. But looking at it in say hexyl (systemctl status mysql | hexyl) I can't see any codes:
Looking near the bottom on lines 080 and 090 where the text "Active: failed" is displayed, I was hoping to find some control sequences to change the color to red. While not necessarily ascii, I used some ascii tables to help me:
looking at the second lot of 8 characters on line 090 where the letters ive: fa are displayed, I find:
69 = i
76 = v
65 = e
3a = :
20 = space
66 = f
61 = a
69 = i
There are no bytes for control sequences.
I wondered if hexyl is choosing not to display them so I wrote a Java program which outputs the raw bytes after executing the process as a bash script and the results are the same - no control sequences.
The Java is roughly:
p = Runtime.getRuntime().exec(new String[]{"/bin/sh", "-c", "systemctl status mysql"}); // runs in the shell
p.waitFor();
byte[] bytes = p.getInputStream().readAllBytes();
for(byte b : bytes) {
System.out.println(b + "\t" + ((char)b));
}
That outputs:
...
32
32
32
32
32
65 A
99 c
116 t
105 i
118 v
101 e
58 :
32
102 f
97 a
105 i
108 l
101 e
100 d
...
So the question is: How does bash know that it has to display the word "failed" red?
systemctl detects that the output is not a terminal, and it removes colors codes from the output.
Related: Detect if stdin is a terminal or pipe? , https://unix.stackexchange.com/questions/249723/how-to-trick-a-command-into-thinking-its-output-is-going-to-a-terminal , https://superuser.com/questions/1042175/how-do-i-get-systemctl-to-print-in-color-when-being-interacted-with-from-a-non-t
Tools sometimes (sometimes not) come with options to enable color codes always, like ls --color=always, grep --color=always on in case of systemd with SYSTEMD_COLORS environment variable.
What tool can I use to see them?
You can use hexyl to see them.
how does bash know that it has to mark the word "failed" red?
Bash is the shell, it is completely unrelated.
Your terminal, the graphical window that you are viewing the output with, knows to mark it red because of ANSI escape sequences in the output. There is no interaction with Bash.
$ SYSTEMD_COLORS=1 systemctl status dbus.service | grep runn | hexdump -C
00000000 20 20 20 20 20 41 63 74 69 76 65 3a 20 1b 5b 30 | Active: .[0|
00000010 3b 31 3b 33 32 6d 61 63 74 69 76 65 20 28 72 75 |;1;32mactive (ru|
00000020 6e 6e 69 6e 67 29 1b 5b 30 6d 20 73 69 6e 63 65 |nning).[0m since|
00000030 20 53 61 74 20 32 30 32 32 2d 30 31 2d 30 38 20 | Sat 2022-01-08 |
00000040 31 39 3a 35 37 3a 32 35 20 43 45 54 3b 20 35 20 |19:57:25 CET; 5 |
00000050 64 61 79 73 20 61 67 6f 0a |days ago.|
00000059
I have the following bash script
#!/bin/bash
for i in 4 5 6; do
echo $i
done
When I run the script it works fine:
$ ./testbash
4
5
6
When I source the same script I get the following syntax error
$ source testbash
bash: /home/nick/bin/testbash: line 5: syntax error: unexpected end of file
Are certain commands disallowed when sourcing a bash file? I'm using GNU bash, version 4.1.2
Here is the hexdump of the script
$ hexdump -C testbash
00000000 23 21 2f 62 69 6e 2f 62 61 73 68 0a 66 6f 72 20 |#!/bin/bash.for |
00000010 69 20 69 6e 20 34 20 35 20 36 3b 20 64 6f 0a 20 |i in 4 5 6; do. |
00000020 20 65 63 68 6f 20 24 69 0a 64 6f 6e 65 0a | echo $i.done.|
0000002e
There are many reasons why sourcing and executing would give different results, but far fewer why you would get a new bash syntax error.
Run env -i bash --norc to get a clean shell so you can check whether it's related to your current shell.
If sourcing works fine in this clean shell, compare the outputs of the following commands with the shell where it fails:
alias (in case you replace or introduce keywords like done)
shopt (in case you modify syntax changing options like extglob)
echo $BASH_VERSION (in case your interactive rc files end up exec'ing a different shell)
This is an example of my case function:
function SendToScreen(){
echo -e "$*"
}
So I call it by:
SendToScreen "Hello"
And, if I want to add color codes:
VioletForeGroundColor="\033[38;5;99m"
NormalColor="\033[0m"
SendToScreen "Hello"$VioletForeGroundColor" violet "$NormalColor" word."
That gives me a correct:
But the problem comes if I want to send some DOS-type path (including \ slash):
VioletForeGroundColor="\033[38;5;99m"
NormalColor="\033[0m"
MyDOSPath="d:\vivisector"
SendToScreen "Hello"$VioletForeGroundColor" violet "$NormalColor" word. The path is $MyDOSPath"
Because \v is some sort of ANSI code, so this time I obtain:
I need my function to output color text (bold, cursive, underline... etc), so I must use echo -e.
How could I solve the problem with such nagging control codes colliding characters like this \v (I suppose there will be another ones)?
I would like to repair the isssue by modifying the function, but I am not sure this is the proper method.
Thanks.
EDIT-1: We will choose \033 also known as \e as the only ANSI code that needs to remain.
New answer:
function SendToScreen() {
echo -e $(echo "${*//\\/\\\\}" | sed 's/\\\\033\[/\\033\[/g');
}
This one escapes everything, then un-escapes anything that looks like a color sequence (\033[). The possibility of sending filenames as color sequences is greatly reduced. You can reduce it even further by white-listing only those color sequences that you want to allow, and changing the sed command to a sequence of sed commands that un-escapes those exact sequences.
Old answer:
Let's say you want to escape \v and \n, you can do this:
function SendToScreen(){
a="${*//\\v/\\\\v}"
a="${a//\\n/\\\\n}"
echo -e "$a"
}
You can extend this with whatever other escapes you don't want to process.
The echo -e simply interprets sequences starting with backslash, so you simply need to ensure that the $MyDOSPath argument has all backslashes doubled up. That could be:
SendToScreen "Hello ${VioletForeGroundColor}violet${NormalColor} word." \
"The path is ${MyDOSPath//\\/\\\\}"
which uses a 'substitute' parameter expansion. The // means 'change every backslash to double backslash'.
As discussed in various comments, maybe the design of SendToScreen is sub-optimal. One possible alternative design uses:
SendToScreen [-e "string-to-expand"][-p "plain-string"] [-- "plain strings"]
Arguments that need to be expanded are, and those that should not be expanded are not. By default, they're not. So, example usage:
$ VioletForeGroundColor="\033[38;5;99m"
$ NormalColor="\033[0m"
$ MyDOSPath="C:\new\table\value\alert\form\033.txt"
$ echo "$MyDOSPath"
C:\new\table\value\alert\form\033.txt
$ bash SendToScreen.sh -e "${VioletForeGroundColor}violet${NormalColor}" -e "The path is ${MyDOSPath//\\/\\\\}" -p "Or $MyDOSPath" "Plain $MyDOSPath"
violet The path is C:\new\table\value\alert\form\033.txt Or C:\new\table\value\alert\form\033.txt Plain C:\new\table\value\alert\form\033.txt
$ bash SendToScreen.sh -e "${VioletForeGroundColor}violet${NormalColor}" -e "The path is ${MyDOSPath//\\/\\\\}" -p "Or $MyDOSPath" -e "Oops! $MyDOSPath" "Plain $MyDOSPath"
violet The path is C:\new\table\value\alert\form\033.txt Or C:\new\table\value\alert\form\033.txt Oops! C: ew able
aluelert
orm.txt Plain C:\new\table\value\alert\form\033.txt
$
A hex dump of the last lot of output was:
0x0000: 1B 5B 33 38 3B 35 3B 39 39 6D 76 69 6F 6C 65 74 .[38;5;99mviolet
0x0010: 1B 5B 30 6D 20 54 68 65 20 70 61 74 68 20 69 73 .[0m The path is
0x0020: 20 43 3A 5C 6E 65 77 5C 74 61 62 6C 65 5C 76 61 C:\new\table\va
0x0030: 6C 75 65 5C 61 6C 65 72 74 5C 66 6F 72 6D 5C 30 lue\alert\form\0
0x0040: 33 33 2E 74 78 74 20 4F 72 20 43 3A 5C 6E 65 77 33.txt Or C:\new
0x0050: 5C 74 61 62 6C 65 5C 76 61 6C 75 65 5C 61 6C 65 \table\value\ale
0x0060: 72 74 5C 66 6F 72 6D 5C 30 33 33 2E 74 78 74 20 rt\form\033.txt
0x0070: 4F 6F 70 73 21 20 43 3A 20 65 77 20 61 62 6C 65 Oops! C: ew able
0x0080: 0B 61 6C 75 65 07 6C 65 72 74 0C 6F 72 6D 1B 2E .alue.lert.orm..
0x0090: 74 78 74 20 50 6C 61 69 6E 20 43 3A 5C 6E 65 77 txt Plain C:\new
0x00A0: 5C 74 61 62 6C 65 5C 76 61 6C 75 65 5C 61 6C 65 \table\value\ale
0x00B0: 72 74 5C 66 6F 72 6D 5C 30 33 33 2E 74 78 74 0A rt\form\033.txt.
0x00C0:
You'll have to take my word for it that violet appeared in violet.
Clearly, the user (caller) of SendToScreen has to know which arguments should be expanded and which should not. However, it makes it very explicit.
Here's the code I used as a script. Repackaging as a function is left as an exercise for the reader. Extending it to add -c colour (or maybe -f foreground and -b background) is an exercise for the reader.
#!/bin/bash
output=()
while getopts "p:e:" opt
do
case "$opt" in
(e) output+=( $(echo -e "$OPTARG") );;
(p) output+=( "$OPTARG" );;
esac
done
shift $(($OPTIND - 1))
echo "${output[#]}" "$#"
Have fun!
I'm investigating a mainly UTF-8 file with lot of long lines. However, the file is not entirely text file, there is some garbage. To find my point of interest I'm using hd and grep.
So at some point I know it'm interested in e.g. 0000301a, so I want to quickly open the file in Vim and jump to that position.
Example (actually a tiny file, here the position is 0000001c):
me#here:~$ hd file | grep -C 10 \ 00\
00000000 6c 69 6e 65 31 0a 6c 69 6e 65 32 0a 6c 69 6e 65 |line1.line2.line|
00000010 33 0a 6c 69 6e 65 34 0a 6c 69 6e 65 00 35 0a 6c |3.line4.line.5.l|
00000020 69 6e 65 36 0a 6c 69 6e 65 37 0a 6c 69 6e 65 38 |ine6.line7.line8|
00000030 0a 6c 69 6e 65 39 0a 6c 69 6e 65 31 30 0a |.line9.line10.|
0000003e
me#here:~$
Is there a trick in Vim to jump to a byte position? Or as close as possible?
Yes, there is. It's the normal mode command go.
From :h go:
[count]go Go to {count} byte in the buffer.
For example, 42go jumps to byte 42.
In order to jump to a hexadecimal or octal address you would need to construct a command with :normal! go or the equivalent :goto, and the str2nr() or printf() function.
:exe 'normal! ' . str2nr('0x2a', 16) . 'go'
:exe 'goto' str2nr('0x2a', 16)
While this question has merit of its own, may I suggest that you're doing this whole thing in a rather roundabout fashion, using all of hd, grep, Vim, and the shell?
Why not use Vim and only Vim for the whole process?
The key is at :h 23.4, "Binary files": Vim can be your hex editor.
After starting Vim with your file, vim -b myfile, the workflow becomes:
:%!xxd -g1<Enter>
/00 <Enter>
...
...
:%!xxd -r<Enter>
Be sure to read :h hex-editing, too, for more details.
Yes, using :help starting.txt you can find the help about all vim command line arguments.
Using the above goto command given by several others you should be able to give vim a command from the command line to go to the byte offset such as
vim "+exe 'goto' str2nr('0x2a', 16)" file
There is a compile time flag described at :help +byte_offset you can check for using the :ve command. My Ubuntu version has it compiled in by default.
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 2 years ago.
#!/bin/bash
cd ~/workspace/trunk;
svn up;
When I run ./build.sh form command line, it says:
: command not found
And nothing happens. How can I solve this ?
I solved adding execute permissions:
sudo chmod +x file.sh
I Have resolved my error from this command.
sudo chmod +x build.sh
My guess is that you have unprintable control characters in the file, or it has \r\n (CRLF) line endings (dos/windows mode).
Try checking it with these commands:
$ hexdump -C build.sh
00000000 23 21 2f 62 69 6e 2f 62 61 73 68 0a 63 64 20 7e |#!/bin/bash.cd ~|
00000010 2f 77 6f 72 6b 73 70 61 63 65 2f 74 72 75 6e 6b |/workspace/trunk|
00000020 3b 0a 73 76 6e 20 75 70 3b 0a |;.svn up;.|
0000002a
$ file build.sh
build.sh: Bourne-Again shell script, ASCII text executable
$ unix2dos build.sh
unix2dos: converting file build.sh to DOS format ...
$ hexdump -C build.sh
00000000 23 21 2f 62 69 6e 2f 62 61 73 68 0d 0a 63 64 20 |#!/bin/bash..cd |
00000010 7e 2f 77 6f 72 6b 73 70 61 63 65 2f 74 72 75 6e |~/workspace/trun|
00000020 6b 3b 0d 0a 73 76 6e 20 75 70 3b 0d 0a |k;..svn up;..|
0000002d
$ file build.sh
build.sh: Bourne-Again shell script, ASCII text executable, with CRLF line terminators
None of the above worked for me except
sudo ./build.sh
Remove ; from the end of your script lines.
This doesn't happen in my bash, so I'm not sure what exactly is wrong, but my guess is this:
; is a separator of commands. Since your last command ends in ;, your bash probably expects another command after. Since the script finishes, though, it reads an empty command, which it can't execute.