When I do a vi filename from the command prompt, what fuse functions are called if I am using the fusexmp example ? I could guess mknod, open are called.
When I do a write ie when i do :wq write is called. is that right.
There's no fantastically easy way to see which FUSE functions are called for any given file operation, but running strace(1) will record the system calls, which is quite close to the FUSE functions:
$ strace -o /tmp/vim.all vim /etc/motd
A lot of those system calls aren't related to the one file specifically, but to the process of loading vim, its dynamically linked libraries, your local configuration, and all its supporting files.
Here's some selected lines that refer to the /etc/motd that I opened:
stat("/etc/motd", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
stat("/etc/motd", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
stat("/etc/motd", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
stat("/etc/motd", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
access("/etc/motd", W_OK) = -1 EACCES (Permission denied)
open("/etc/motd", O_RDONLY) = 7
close(7) = 0
open("/etc/motd", O_RDONLY) = 7
read(7, "Welcome to Ubuntu 11.04 (GNU/Lin"..., 8192) = 183
read(7, "", 65536) = 0
close(7) = 0
stat("/etc/motd", {st_mode=S_IFREG|0644, st_size=183, ...}) = 0
The intervening lines make the repeated stat(2) calls a little less silly looking.
Related
I've got very interested in Linux internals recently, and currently trying to understand how things work.
I knew that when I type ls
opendir() - function is called;
readdir() - function called for each directory entry in directory data store;
stat() - function can be called to get additional information on files, if required.
Please correct me if I'm missing something or if it's wrong.
The part which is mystery to me is filename expansion(globbing).
I've compared the output of strace ls
open(".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=270336, ...}) = 0
getdents(3, /* 14 entries */, 32768) = 440
getdents(3, /* 0 entries */, 32768) = 0
close(3) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
write(1, "2q.c ds.c fglob fnoglob\n", 272q.c ds.c fglob fnoglob
and strace ls *.c,
stat("2q.c", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
lstat("2q.c", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
stat("ds.c", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
lstat("ds.c", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
write(1, "2q.c ds.c\n", 112q.c ds.c
) = 11
and from my limited knowledge can tell that in the first case, it indeed behaves as I expected open, stat followed by getdents.
But the later one, with shell globbing isn't clear to me, because it there's already a list of files, which match the pattern. Where does this list came from?
Thanks!
Shell globbing patterns on the command line are expanded by the shell before the utility is invoked.
You can see this by enabling tracing in the shell with set -x:
$ set -x
$ ls -l f*
+ ls -l file1 file2 file3
-rw-r--r-- 1 kk wheel 0 May 11 16:49 file1
-rw-r--r-- 1 kk wheel 0 May 11 16:49 file2
-rw-r--r-- 1 kk wheel 0 May 11 16:49 file3
As you can see, the shell tells you what command it invokes (at the + prompt), and at that point it has already expanded the pattern on the command line.
The ls command does not do filename globbing. In fact, if you single quote the globbing pattern to protect it from the shell, ls is bound to be confused:
$ ls -l 'f*'
+ ls -l f*
ls: f*: No such file or directory
(unless there's actually something in the current directory called f* of course).
I've been studying UNIX and system calls and I came across a low-level and tricky questions. The question asks what system calls are called for this command:
grep word1 word2 > file.txt
I did some research and I was unable to find a huge number of resources on the underlying UNIX calls. However, it seems to me that the answer would be open (to open and the file descriptor for the file file.txt), then dup2 (to change the STDOUT of grep to the file descriptor of open), then write to write the STDOUT of grep (which is now the file descriptor of file.txt), and finally close(), to close the file descriptor of file.txt... However, I have no idea if I am right or on the correct path, can anyone with experience in UNIX enlighten me on this topic?
You are on correct direction in your research. This command is very helpful to trace system calls in any program:
strace
On my PC it shows output (without stream redirection):
$ strace grep abc ss.txt
execve("/bin/grep", ["grep", "abc", "ss.txt"], [/* 237 vars */]) = 0
brk(0) = 0x13de000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1785694000
close(3) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
stat("ss.txt", {st_mode=S_IFREG|0644, st_size=13, ...}) = 0
open("ss.txt", O_RDONLY) = 3
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fffa0e4f370) = -1 ENOTTY (Inappropriate ioctl for device)
read(3, "abc\n123\n321\n\n", 32768) = 13
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f178568c000
write(1, "abc\n", 4abc
) = 4
read(3, "", 32768) = 0
close(3) = 0
close(1) = 0
munmap(0x7f178568c000, 4096) = 0
close(2) = 0
exit_group(0) = ?
I installed git-flow, which copied some bash scripts to /usr/lib/git-core/ which are now available in bash complete.
I know that bash reads the following directories looking for autocomplete scripts:
/etc/bash_completion.d/
/usr/share/bash-completion/
What makes bash aware to those new scripts?
I installed git-flow from the debian repositories, and I am trying to understand what makes this package ticks.
I did not succeed in finding where bash determines it needs to read this directory. Looking through /usr/share/bash-completion/completions/git did not help either.
update
Looking around, it seems that /usr/git-core/git and /usr/bin/git are the same binary file:
~ $ file /usr/bin/git
/usr/bin/git: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=4f634dadca99fb29ed72568d5604df087d6e6502, stripped
~ $ file /usr/lib/git-core/git
/usr/lib/git-core/git: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=4f634dadca99fb29ed72568d5604df087d6e6502, stripped
OK, given the hint from Etan, it seems, that what seems to be a bash completion is actually a git completion. As far why /usr/lib/git-core is search, strace shows that it's hard coded when configuring and building the package:
execve("/usr/bin/git", ["git", "help", "-a"], [/* 73 vars */]) = 0
brk(0) = 0xab4000
...
openat(AT_FDCWD, "/usr/lib/git-core", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
getdents(3, /* 169 entries */, 32768) = 6192
stat("/usr/lib/git-core/git-fetch-pack", {st_mode=S_IFREG|0755, st_size=1577256, ...}) = 0
stat("/usr/lib/git-core/git-web--browse", {st_mode=S_IFREG|0755, st_size=4398, ...}) = 0
stat("/usr/lib/git-core/git-upload-archive", {st_mode=S_IFREG|0755, st_size=1577256, ...}) = 0
stat("/usr/lib/git-core/git-credential-store", {st_mode=S_IFREG|0755, st_size=765192, ...}) = 0
...
When compiling the package from source with configure --prefix=/usr/local/bin/git the strace log is:
execve("/usr/local/git/bin/git", ["/usr/local/git/bin/git", "help", "-a"], [/* 73 vars */]) = 0
brk(0) = 0x2395000
...
openat(AT_FDCWD, "/usr/local/git/libexec/git-core", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
getdents(3, /* 159 entries */, 32768) = 5792
stat("/usr/local/git/libexec/git-core/git-fetch-pack", {st_mode=S_IFREG|0755, st_size=8448439, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-web--browse", {st_mode=S_IFREG|0755, st_size=4398, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-upload-archive", {st_mode=S_IFREG|0755, st_size=8448439, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-credential-store", {st_mode=S_IFREG|0755, st_size=4146627, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-p4", {st_mode=S_IFREG|0755, st_size=122122, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-merge-tree", {st_mode=S_IFREG|0755, st_size=8448439, ...}) = 0
stat("/usr/local/git/libexec/git-core/git-mergetool", {st_mode=S_IFREG|0755, st_size=8377, ...}) = 0
The need to read /etc/bash_completion.d is built into bash which sources all scripts contained within this directory making any defined functionality available.
Each time you start a new terminal, this directory is re-read as part of the bash warmup procedure making all defined completions available to the shell.
Different vendors may have different directories read alongside /etc/bash_completion.d including /usr/share/bash-completion but this one is the minimum defined.
If you want to define your own completions, have a look at https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion.html.
I have cross compiled libraries and a linux loader.
I have placed a custom ld-.so.conf under /etc , The conf file has a path that contains all the cross compiled library and the loader.
But when i run ldconfig,
ldconfig -C /etc/ld-.so.cache -f /etc/ld-.so.conf
All the system libraries and their paths are present in the cache file.
I need the cache file generated to contain only my cross compiled libraries.
Strace of ldconfig operation is as below:
strace /opt/me/ldconfig -C /etc/ld-me.so.cache -f /etc/ld-me.so.conf
execve("/opt/me/ldconfig", ["/opt/me/ldc"..., "-C",
"/etc/ld-me.so.cache", "-f", "/etc/ld-me.so.conf"], [/* 38 vars */]) =
0
uname({sys="Linux", node="ip-172-31-32-236", ...}) = 0 brk(0)
= 0x10c1000 brk(0x10c2180) = 0x10c2180 arch_prctl(ARCH_SET_FS, 0x10c1860) = 0 brk(0x10e3180)
= 0x10e3180 brk(0x10e4000) = 0x10e4000 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3 fstat(3,
{st_mode=S_IFREG|0644, st_size=99154480, ...}) = 0 mmap(NULL,
99154480, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f46155a4000 close(3)
= 0 open("/etc/ld-me.so.conf", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0640, st_size=25, ...}) = 0 mmap(NULL, 4096,
PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f46155a3000 read(3, "/opt/me/lib\n", 4096) = 25 stat("/opt/me/lib",
{st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0 read(3, "", 4096)
= 0 close(3) = 0 munmap(0x7f46155a3000, 4096) = 0 stat("/lib", {st_mode=S_IFDIR|0555, st_size=4096,
...}) = 0 stat("/lib64", {st_mode=S_IFDIR|0555, st_size=12288, ...}) =
0 stat("/usr/lib", {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
stat("/usr/lib64", {st_mode=S_IFDIR|0555, st_size=12288, ...}) = 0
open("/opt/me/lib", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
Can anybody tell me why system libraries are getting added ?
Because that is the defined behavior of ldconfig:
ldconfig creates the necessary links and cache to the most
recent shared libraries found in the directories specified on the
command line, in the file /etc/ld.so.conf, and in the trusted
directories (/lib and /usr/lib). The cache is used by the run-time
linker, ld.so or ld-linux.so. ldconfig checks the header and
filenames of the libraries it encounters when
determining which versions should have their links updated.
The trusted directory list was probably updated with lib64 dirs sometime after the man page I cut'n'pasted from was written.
You could create a directory structure based on your conf file with symlinks or bind mounts pointing to the real directories and then use
-r directory to make ldconfig build based on empty versions of the system directories.
As per the man page /proc/pid/exe is a symlink containing the actual path of the executed command..
when I run valgrind on my program, I see that /proc/pid/exe points to /usr/lib64/valgrind/amd64-linux/memcheck
lnx-host> which valgrind
/usr/bin/valgrind
Any idea why /proc/pid/exe points to usr/lib64/valgrind/amd64-linux/memcheck when I am invoking it as valgrind ?
In my code I am trying to get the executable name from the pid, and in this case expecting to see valgrind.
memcheck is the default tool used by Valgrind, unless you tell it to use another of the tools, such as callgrind.
Use --tool=<name> to specify the tool you want to invoke.
Side-note: is your /usr/bin/valgrind also a script just like it is by default? Why not play with that to do what you want to achieve? On my system that invokes first of all /usr/bin/valgrind.bin and then the respective (backend) tool (/usr/lib/valgrind/memcheck-amd64-linux).
Relevant output from strace:
execve("/usr/bin/valgrind", ["valgrind", "./myprog"], [/* 35 vars */]) = 0
stat("/home/user/HEAD/myprog", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
execve("/usr/bin/valgrind.bin", ["/usr/bin/valgrind.bin", "./myprog"], [/* 39 vars */]) = 0
open("./myprog", O_RDONLY) = 3
execve("/usr/lib/valgrind/memcheck-amd64-linux", ["/usr/bin/valgrind.bin", "./myprog"], [/* 40 vars */]) = 0
getcwd("/home/user/HEAD/myprog", 4095) = 25
open("./myprog", O_RDONLY) = 3
stat("./myprog", {st_mode=S_IFREG|0755, st_size=1886240, ...}) = 0
readlink("/proc/self/fd/3", "/home/user/HEAD/myprog/myprog", 4096) = 31
readlink("/proc/self/fd/3", "/home/user/HEAD/myprog/myprog", 4096) = 31
open("./myprog", O_RDONLY) = 3
write(1015, "./myprog", 8) = 8
write(1016, "==23547== Command: ./myprog\n", 28==23547== Command: ./myprog
stat("/home/user/HEAD/myprog/myprog", {st_mode=S_IFREG|0755, st_size=1886240, ...}) = 0
open("/home/user/HEAD/myprog/myprog", O_RDONLY) = 3
stat("/home/user/HEAD/myprog/myprog", {st_mode=S_IFREG|0755, st_size=1886240, ...}) = 0
open("/home/user/HEAD/myprog/myprog", O_RDONLY) = 3
open("/home/user/HEAD/myprog/myprog", O_RDONLY) = 3
readlink("/proc/self/fd/3", "/home/user/HEAD/myprog/myprog", 4096) = 31
getcwd("/home/user/HEAD/myprog", 4096) = 25
lstat("/home/user/HEAD/myprog/myprog", {st_mode=S_IFREG|0755, st_size=1886240, ...}) = 0
open("/home/user/HEAD/myprog/datafile", O_RDONLY) = 3
access("/home/user/HEAD/myprog/datafile", F_OK) = 0
open("/home/user/HEAD/myprog/datafile", O_RDONLY) = 3
open("/home/user/HEAD/myprog/datafile", O_RDONLY) = 4
You'll notice that all execve calls are not referring to ./myprog but instead to the Valgrind wrapper script, the binary and then the backend tool:
execve("/usr/bin/valgrind", ["valgrind", "./myprog"], [/* 35 vars */]) = 0
execve("/usr/bin/valgrind.bin", ["/usr/bin/valgrind.bin", "./myprog"], [/* 39 vars */]) = 0
execve("/usr/lib/valgrind/memcheck-amd64-linux", ["/usr/bin/valgrind.bin", "./myprog"], [/* 40 vars */]) = 0