I wrote simple program that prints out argv[0] and compiled by using gcc in terminal. Then i executed a.out file by writing ./a.out
I expected the outcome to be a.out but it was “./a.out”. Isnt argv[0] is a program name? Or program name includes the path??
Thanks a lot
The first argument (argv[0]) will include the path. You can see this by moving to another directory and executing your simple program from that location. You will need to do your own parsing to get just the program name.
Executables under Linux use special paths like /usr/bin/ to search for executables.
As security measure the current directory (.) is not sought through for executing.
But if you explicitly write ./my_own_batch_file.sh then that is executed.
Thus if you are in some directory with thousands of files, you still can type ls (Windows dir) without surprises.
Related
In a script you must include a #! on the first line followed by the path to the program that will execute the script (e.g.: sh, perl).
As far as I know, the # character denotes the start of a comment and that line is supposed to be ignored by the program executing the script. It would seem, that this first line is at some point read by something in order for the script to be executed by the proper program.
Could somebody please shed more light on the workings of the #!?
I'm really curious about this, so the more in-depth the answer the better.
Recommended reading:
The UNIX FAQ: Why do some scripts start with #! ... ?
The #! magic, details about the shebang/hash-bang mechanism on various Unix flavours
Wikipedia: Shebang
The unix kernel's program loader is responsible for doing this. When exec() is called, it asks the kernel to load the program from the file at its argument. It will then check the first 16 bits of the file to see what executable format it has. If it finds that these bits are #! it will use the rest of the first line of the file to find which program it should launch, and it provides the name of the file it was trying to launch (the script) as the last argument to the interpreter program.
The interpreter then runs as normal, and treats the #! as a comment line.
The Linux kernel exec system call uses the initial bytes #! to identify file type
When you do on bash:
./something
on Linux, this calls the exec system call with the path ./something.
This line gets called in the kernel on the file passed to exec: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_script.c#L25
if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))
It reads the very first bytes of the file, and compares them to #!.
If the comparison is true, then the rest of the line is parsed by the Linux kernel, which makes another exec call with path /usr/bin/env python and current file as the first argument:
/usr/bin/env python /path/to/script.py
and this works for any scripting language that uses # as a comment character.
And yes, you can make an infinite loop with:
printf '#!/a\n' | sudo tee /a
sudo chmod +x /a
/a
Bash recognizes the error:
-bash: /a: /a: bad interpreter: Too many levels of symbolic links
#! is human readable, but that is not necessary.
If the file started with different bytes, then the exec system call would use a different handler. The other most important built-in handler is for ELF executable files: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_elf.c#L1305 which checks for bytes 7f 45 4c 46 (which also happens to be human readable for .ELF). Let's confirm that by reading the 4 first bytes of /bin/ls, which is an ELF executable:
head -c 4 "$(which ls)" | hd
output:
00000000 7f 45 4c 46 |.ELF|
00000004
So when the kernel sees those bytes, it takes the ELF file, puts it into memory correctly, and starts a new process with it. See also: How does kernel get an executable binary file running under linux?
Finally, you can add your own shebang handlers with the binfmt_misc mechanism. For example, you can add a custom handler for .jar files. This mechanism even supports handlers by file extension. Another application is to transparently run executables of a different architecture with QEMU.
I don't think POSIX specifies shebangs however: https://unix.stackexchange.com/a/346214/32558 , although it does mention in on rationale sections, and in the form "if executable scripts are supported by the system something may happen". macOS and FreeBSD also seem to implement it however.
Short story: The shebang (#!) line is read by the shell (e.g. sh, bash, etc.) the operating system's program loader. While it formally looks like a comment, the fact that it's the very first two bytes of a file marks the whole file as a text file and as a script. The script will be passed to the executable mentioned on the first line after the shebang. Voilà!
Slightly longer story: Imagine you have your script, foo.sh, with the executable bit (x) set. This file contains e.g. the following:
#!/bin/sh
# some script commands follow...:
# *snip*
Now, on your shell, you type:
> ./foo.sh
Edit: Please also read the comments below after or before you read the following! As it turns out, I was mistaken. It's apparently not the shell that passes the script to the target interpreter, but the operating system (kernel) itself.
Remember that you type this inside the shell process (let's assume this is the program /bin/sh). Therefore, that input will have to be processed by that program. It interprets this line as a command, since it discovers that the very first thing entered on the line is the name of a file that actually exists and which has the executable bit(s) set.
/bin/sh then starts reading the file's contents and discovers the shebang (#!) right at the very beginning of the file. To the shell, this is a token ("magic number") by which it knows that the file contains a script.
Now, how does it know which programming language the script is written it? After all, you can execute Bash scripts, Perl scripts, Python scripts, ... All the shell knows so far is that it is looking at a script file (which is not a binary file, but a text file). Thus it reads the next input up to the first line break (which will result in /bin/sh, compare with the above). This is the interpreter to which the script will be passed for execution. (In this particular case, the target interpreter is the shell itself, so it doesn't have to invoke a new shell for the script; it simply processes the rest of the script file itself.)
If the script was destined for e.g. /bin/perl, all that the Perl interpreter would (optionally) have to do is look whether the shebang line really mentions the Perl interpreter. If not, the Perl interpreter would know that it cannot execute this script. If indeed the Perl interpreter is mentioned in the shebang line, it reads the rest of the script file and executes it.
I had written a simple shell script (called test.sh) to compile a test C++ file using two different compilers (g++ and clang++) and put some echo statements in to compare the output. On the command line, I accidentally typed make test, even though there was no Makefile in that directory. Instead of complaining about no Makefile or no target defined, it executed the following commands (my system is running the 64-bit Debian stretch OS with GNU Make 4.1 ):
user#hostname test_dir$ make test
cat test.sh >test
chmod a+x test
user#hostname test_dir$
Curious about that, I made another shell script (other.sh) and did the same thing.
Here is my other.sh file:
#!/bin/bash
echo ""
echo "This is another test script!"
echo ""
Command line:
user#hostname test_dir$ make other
cat other.sh >other
chmod a+x other
user#hostname test_dir$
My question is why does make automatically create an executable script (without the .sh extension) when running the make command in the terminal? Is this normal/expected/standard behavior? Can I rely on this behavior on all Linux machines?
Side question/note: Is there a list of supported "implicit suffixes" for which make will automatically create an executable?
This is one of a number of "implicit rules" which are built into Gnu make. (Every make implementation will have some implicit rules, but there is no guarantee that they are the same.)
Why does make automatically create an executable script without the .sh extension?
There is an old source repository system called Source Code Control System (SCCS). Although it no longer has much use, it was once the most common way of maintaining source code repositories. It had the quirk that it did not preserve file permissions, so if you kept an (executable) shell script in SCCS and later checked it out, it would no longer be executable. Gnu make could automatically extract files from an SCCS repository; to compensate for the disappearing executable permission issue, it was common to use the .sh extension with shell scripts; make could then do a two-step extraction, where it first extracted foo.sh from the repository and then copied it to foo, adding the executable permission.
Is this normal/expected/standard behavior? Can I rely on this behavior on all Linux machines?
Linux systems with a development toolset installed tend to use Gnu make, so you should be able to count on this behaviour on Linux systems used for development.
BSD make also comes with a default rule for .sh, but it only copies the file; it doesn't change permissions (at least on the bsdmake distribution on my machine). So the behaviour is not universal.
Is there a list of supported "implicit suffixes" for which make will automatically create an executable?
Yes, there is. You'll find it in the make manual:
The default suffix list is: .out, .a, .ln, .o, .c, .cc, .C, .cpp, .p, .f, .F, .m, .r, .y, .l, .ym, .lm, .s,
.S, .mod, .sym, .def, .h, .info, .dvi, .tex, .texinfo, .texi, .txinfo, .w, .ch,
.web, .sh, .elc, .el.
For a more accurate list of implicit rules, you can use the command
make -p -f/dev/null
# or, if you like typing, make --print-data-base -f /dev/null
as described in the make options summary.
From the make man page:
The purpose of the make utility is to determine automatically
which
pieces of a large program need to be recompiled, and issue the commands
to recompile them. The manual describes the GNU implementation of
make, which was written by Richard Stallman and Roland McGrath, and is
currently maintained by Paul Smith. Our examples show C programs,
since they are most common, but you can use make with any programming
language whose compiler can be run with a shell command. In fact, make
is not limited to programs. You can use it to describe any task where
some files must be updated automatically from others whenever the others change.
make really is more than most people make it out to be...
I've got this EFL file which I need to debug/step-through. It's a reverse engineering competition. All I need to do is to find out the value of a register at a particular point in time and in a particular place. I used Hopper Disassembler to find out the address of interest.
Here's the problem. I don't know how to debug an ELF file. It's my first time debugging in a Linux environment. Learning how to execute the ELF file itself took me a while. I execute by using
ld-linux.so.2 ./[EFLFILE] [arguments]
Is there a way I can atleast attach a debugger onto the proess? I can't even find it with the ps command. Also, I've heard that it's possible to have remote debugger; to have a debugger running on a windows machine and have the binary to be examined running on a linux.
Could anyone help me achieve just any of this?
Usually an ELF file can be executed as follows:
$ /path/to/elffile [arguments]
To debug it using GDB you can do:
$ gdb /path/to/elffile
Or passing arguments:
$ gdb --args /path/to/elffile arguments...
In your case:
$ gdb --args ./[EFLFILE] [arguments]
Then type run or simly r and press < Enter >.
Type help to get help on the gdb commands.
Note: if your program needs some external libs, before running it, you should define LD_LIBRARY_PATH pointing on the folder containing those libs (export LD_LIBRARY_PATH=/the/path/to/libs)
They handle executable elfs, scripts and symbolic links from PATH, however what the algorithm of this doing? I'm afraid of I cannot find a source code of this part of a shell.
UDP: Oh, I'm stupid. It looks for EACH executable file in PATH, either directory or ordinary file.
Well, the actual search is performed by find_user_command_in_path() in findcmd.c:553.
The algorithm to search for a command ${foo} is basically:
check if ${foo} is absolute: if it is return this path and stop searching
iterate over all elements in PATH: for p in ${PATH}
construct a path ${p}/${foo} and see if it exists
if it exists and is executable return this path and stop searching
I'm no expert in this area, but I'm almost perfectly sure that on Linux the executable bit in file permissions is all that matters. No sophisticated algorithm needed.
Let's say that we have a file called hello in the current directory, and that the file contains just one line: echo "hello"
If you ran chmod 755 on the file and and you subsequently execute the file, then the bash shell will look through every path that you have listed in the PATH variable of say .bashrc, starting with the first path, until it locates the first path that contains your hello executable. Think of PATH as a linked list and think of the bash shell as going through the linked list of paths, path by path. If the bash shell is not running the hello executable that you want it to run, you have one option: put your hello executable in any one of the preceeding paths.
I am lazy. I don't bother to turn hello into an executable i.e. I am not running the chmod command and I just run
bash hello
where the bash shell is going to look for the hello file in the current directory, fork a bash process and the forked bash process is going to run the hello file before the forked bash process dies.
I am using the bash shell as an example but any other shell will behave the same way.
I am developing an application on Beaglebone board with Angstrom distrubition fo Linux.I faced an interesting problem.
When I execute :
sh /home/root/Desktop/BBTCP/out/vehicleDetect 192.168.10.29
in terminal it says
/home/root/Desktop/BBTCP/out/vehicleDetect: /home/root/Desktop/BBTCP/out/vehicleDetect: cannot execute binary file
But when i execute
cd /home/root/Desktop/BBTCP/
and
sh out/vehicleDetect 192.168.10.29
it starts working??
What is the reason and why I can't run tha application with first configuration?
I think it is about the difference between ./ and sh. What are the differences?
My first guess would be that one of the folders in the path /home/root/Desktop/BBTCP is a link. If vehicleDetect is a script and it invokes itself recursively, then this link might be confusing it.
If that's not the case, try sh -x /home/root/Desktop/BBTCP/out/vehicleDetect and see what that prints.
Lastly, check what's in the folder /home/root/Desktop/BBTCP. There might be an executable sh in there. If your path contains ., a different shell might be executed.
Seems like /home/root/Desktop/BBTCP/out/vehicleDetect is invoking a binary file (an executable) that was built on a different architecture.
The main difference between sh and ./ is that ./ will attempt to execute the file itself as an executable, whereas sh will do that for you. It could be that there is a weird magic number at the start of the file, but you would expect sh to complain about that.
Best guess though it is a #! line at the start of the file which either contains invalid characters or refers to a strange file. Did you, for example, bring this script file from another operating system (like Windows) that has different line endings? I have seen similar effects when text file scripts have not been converted but just copied. Maybe you downloaded it in a strange format?
Check with od -xc /home/root/Desktop/BBTCP/out/vehicleDetect and look at the first line.