I have to take some text my professor has given me, put it into two files, and I have to use the "diff" function to compare those two files. But, some lines in those files are missing an entry in the 6th (last) column. I need to filter the diff command to exclude those lines.
I tried to use the diff command with a regex, but I was unable to make a regex that was able to filter just that last column.
The file I am trying to filter looks something like:
b72cc000-b72ce000 rw-p 00147000 fc:00 3671434 /usr/lib/i386-linux-gnu/libX11.so.6.3.0
b72ce000-b72d0000 rw-p 00000000 00:00 0
b72d0000-b72d7000 r-xp 00000000 fc:00 7606808 /lib/i386-linux-gnu/librt-2.23.so
b72d7000-b72d8000 r--p 00006000 fc:00 7606808 /lib/i386-linux-gnu/librt-2.23.so
b72d8000-b72d9000 rw-p 00007000 fc:00 7606808 /lib/i386-linux-gnu/librt-2.23.so
b72d9000-b72f2000 r-xp 00000000 fc:00 7602217 /lib/i386-linux-gnu/libpthread-2.23.so
b72f2000-b72f3000 r--p 00018000 fc:00 7602217 /lib/i386-linux-gnu/libpthread-2.23.so
b72f3000-b72f4000 rw-p 00019000 fc:00 7602217 /lib/i386-linux-gnu/libpthread-2.23.so
b72f4000-b72f6000 rw-p 00000000 00:00 0
b72f6000-b72f9000 r-xp 00000000 fc:00 7602261 /lib/i386-linux-gnu/libdl-2.23.so
b72f9000-b72fa000 r--p 00002000 fc:00 7602261 /lib/i386-linux-gnu/libdl-2.23.so
I need to run the diff command on two files similar to this in a linux virtual machine. The diff should filter out the 2nd and 9th rows because there is nothing in the last column. I do not have install privileges on the machine, so I can't install anything new. There is a vim editor and a C compiler pre-installed; however, so I can use those.
Use awk and filter by the number of fields.
awk 'NF > 5' filename
will only print the lines that have at least 6 fields.
You can use this with diff with process substitution:
diff <(awk 'NF > 5' file1.txt) <(awk 'NF > 5' file2.txt)
Related
In Linux if we look at the /proc/self/maps:
00400000-004ef000 r-xp 00000000 08:01 12845058 /bin/bash
006ef000-006f0000 r--p 000ef000 08:01 12845058 /bin/bash
006f0000-006f9000 rw-p 000f0000 08:01 12845058 /bin/bash
006f9000-006ff000 rw-p 00000000 00:00 0
00d5a000-010a2000 rw-p 00000000 00:00 0 [heap]
7f6fe582a000-7f6fe5835000 r-xp 00000000 08:01 1048595 /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f6fe5835000-7f6fe5a34000 ---p 0000b000 08:01 1048595 /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f6fe5a34000-7f6fe5a35000 r--p 0000a000 08:01 1048595 /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f6fe5a35000-7f6fe5a36000 rw-p 0000b000 08:01 1048595 /lib/x86_64-linux-gnu/libnss_files-2.19.so
I want information like pathname and starting address for Qnx. How can I get that information?
My analysis says that there is not /proc/self/maps file in Qnx.
Try to use this:
pidin -p<process> mapinfo
We have self file in QNX too !!
The address space is represented by a binary file called "as" in every folder in /proc. /proc/self/as contains the address space of the current process.
Problem is that its not human readable directly. You need to use devctl() to read it. Below is the link for all the devctl commands regarding address space decoding.
http://www.qnx.com/developers/docs/6.5.0_sp1/index.jsp?topic=%2Fcom.qnx.doc.neutrino_cookbook%2Fs3_procfs.html
Example -
devctl("/proc/self/as",DCMD_PROC_TIDSTATUS,&debug_data, sizeof(debug_data)
This command will give you debug data of the thread in a struct _debug_thread_info structure.
Similarly DCMD_PROC_PAGEDATA and DCMD_PROC_MAPINFO can be used to get the segment mapping ( in linux maps file does this) information of a process's address space.
Hope this helps.
I am using ubuntu-13.10 with ASLR enabled
root#ubuntu:/home/meltdown# cat /proc/sys/kernel/randomize_va_space
2
I have compiled a simple hello world program with gcc -pie option.
if I run this program stand alone, the base address of PIE executable is random. like this.
root#ubuntu:/home/meltdown# cat /proc/8872/maps
b758b000-b758c000 rw-p 00000000 00:00 0
b758c000-b773a000 r-xp 00000000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b773a000-b773c000 r--p 001ae000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b773c000-b773d000 rw-p 001b0000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b773d000-b7740000 rw-p 00000000 00:00 0
b7752000-b7756000 rw-p 00000000 00:00 0
b7756000-b7757000 r-xp 00000000 00:00 0 [vdso]
b7757000-b7777000 r-xp 00000000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
b7777000-b7778000 r--p 0001f000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
b7778000-b7779000 rw-p 00020000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
b7779000-b777a000 r-xp 00000000 08:01 14942231 /tmp/a
b777a000-b777b000 r--p 00000000 08:01 14942231 /tmp/a
b777b000-b777c000 rw-p 00001000 08:01 14942231 /tmp/a
bf9f4000-bfa15000 rw-p 00000000 00:00 0 [stack]
however, if I debug this program with gdb, the PIE base address is always same(80000000).
root#ubuntu:/home/meltdown# cat /proc/8840/maps
80000000-80001000 r-xp 00000000 08:01 14942231 /tmp/a
80001000-80002000 r--p 00000000 08:01 14942231 /tmp/a
80002000-80003000 rw-p 00001000 08:01 14942231 /tmp/a
b7e12000-b7e13000 rw-p 00000000 00:00 0
b7e13000-b7fc1000 r-xp 00000000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b7fc1000-b7fc3000 r--p 001ae000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b7fc3000-b7fc4000 rw-p 001b0000 08:01 10749216 /lib/i386-linux-gnu/libc-2.17.so
b7fc4000-b7fc7000 rw-p 00000000 00:00 0
b7fdb000-b7fdd000 rw-p 00000000 00:00 0
b7fdd000-b7fde000 r-xp 00000000 00:00 0 [vdso]
b7fde000-b7ffe000 r-xp 00000000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
b7ffe000-b7fff000 r--p 0001f000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
b7fff000-b8000000 rw-p 00020000 08:01 10749212 /lib/i386-linux-gnu/ld-2.17.so
bffdf000-c0000000 rw-p 00000000 00:00 0 [stack]
can someone explain why??
The gdb debugger, by default, turns off address space layout randomisation. This is partially to ensure you're always debugging the same environment. From the gdb documentation (search for disable-randomization):
This option is useful for multiple debugging sessions to make the execution better reproducible and memory addresses reusable across debugging sessions.
It's the same reason why I use srand(42) at the start of my code (only when debugging) rather than srand(time(NULL)) - it provides an absolutely consistent environment from run to run, to make debugging easier.
If you had a very subtle bug that was caused by where your code was located in the address space, it may come and go on different runs within the debugger, unless ASLR was disabled.
You can use:
set disable-randomization off
before starting your program from within gdb to re-enable ASLR, as per the gdb documentation.
I suspect you can also use gdb to attach to an already-running process (which is under the ASLR effect) rather than having gdb start your program from scratch though I tend to prefer the use of ~/.gdbinit to force ASLR to be activated.
Is there any system call or function that can get page attribute like readable, writable, executable by page address?
I know we can use mprotect to set pages attribute, but I have no idea if we can get pages attribute. Any comment is appreciated.
There isn't. You need to use the proc interface, which contains lots of information about each process in various files. The information you're looking for is in plaintext, in /proc/<pid>/maps. You can take a look at it for your current process by running:
$ cat /proc/$$/maps
You can find some more information in man 5 proc.
The example given at the above link is:
address perms offset dev inode pathname
08048000-08056000 r-xp 00000000 03:0c 64593 /usr/sbin/gpm
08056000-08058000 rw-p 0000d000 03:0c 64593 /usr/sbin/gpm
08058000-0805b000 rwxp 00000000 00:00 0
40000000-40013000 r-xp 00000000 03:0c 4165 /lib/ld-2.2.4.so
40013000-40015000 rw-p 00012000 03:0c 4165 /lib/ld-2.2.4.so
4001f000-40135000 r-xp 00000000 03:0c 45494 /lib/libc-2.2.4.so
40135000-4013e000 rw-p 00115000 03:0c 45494 /lib/libc-2.2.4.so
4013e000-40142000 rw-p 00000000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
As you can see, the permissions are the second (space-delimited) field there. So from a program (like in C, since you mentioned mprotect(), you could open up /proc/$$/maps with fopen(), then use fgets() or scanf() to pull the data out. You're just looking for that perms field of the range where your page lies.
The mmap() function shall establish a mapping between a process virtual address space and a device file or physical memory region.
A process virtual memory layout has the following sections:
Which region of Process Virtual Address Space does mmap() use for mapping?
Mmap uses "unallocated memory".
Please note that the picture you drew is unlikely to be used on any UNIX system that is younger than about 30 years. UNIX used do have that memory layout in the early 70s, but the picture is much more complicated nowdays, especially when using shared libraries.
To get an insight of what is happenning today, try (on Linux) the following command
cat /proc/self/maps
on my machine, it gives now
00400000-0040c000 r-xp 00000000 08:01 1850896 /bin/cat
0060c000-0060d000 rw-p 0000c000 08:01 1850896 /bin/cat
00adc000-00afd000 rw-p 00000000 00:00 0 [heap]
7ffe843ef000-7ffe84569000 r-xp 00000000 08:01 787567 /lib/x86_64-linux-gnu/libc-2.13.so
7ffe84569000-7ffe84769000 ---p 0017a000 08:01 787567 /lib/x86_64-linux-gnu/libc-2.13.so
7ffe84769000-7ffe8476d000 r--p 0017a000 08:01 787567 /lib/x86_64-linux-gnu/libc-2.13.so
7ffe8476d000-7ffe8476e000 rw-p 0017e000 08:01 787567 /lib/x86_64-linux-gnu/libc-2.13.so
7ffe8476e000-7ffe84773000 rw-p 00000000 00:00 0
7ffe84773000-7ffe84792000 r-xp 00000000 08:01 790578 /lib/x86_64-linux-gnu/ld-2.13.so
7ffe8495e000-7ffe84961000 rw-p 00000000 00:00 0
7ffe84990000-7ffe84992000 rw-p 00000000 00:00 0
7ffe84992000-7ffe84993000 r--p 0001f000 08:01 790578 /lib/x86_64-linux-gnu/ld-2.13.so
7ffe84993000-7ffe84994000 rw-p 00020000 08:01 790578 /lib/x86_64-linux-gnu/ld-2.13.so
7ffe84994000-7ffe84995000 rw-p 00000000 00:00 0
7fffdbaac000-7fffdbacd000 rw-p 00000000 00:00 0 [stack]
7fffdbb66000-7fffdbb67000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
It shows the memory map of the process executing the cat command.
I understand the meaning of rwxps bits. r-xp is for .text. rw-p is for .data/.bss/heap/stack. What is the use of just ---p pages?
For example see this output of cat /proc/self/maps
00400000-0040b000 r-xp 00000000 08:03 827490 /bin/cat
0060b000-0060c000 rw-p 0000b000 08:03 827490 /bin/cat
0060c000-0062d000 rw-p 00000000 00:00 0 [heap]
3819a00000-3819a1e000 r-xp 00000000 08:03 532487 /lib64 ld-2.11.2.so
3819c1d000-3819c1e000 r--p 0001d000 08:03 532487 /lib64/ld-2.11.2.so
3819c1e000-3819c1f000 rw-p 0001e000 08:03 532487 /lib64/ld-2.11.2.so
3819c1f000-3819c20000 rw-p 00000000 00:00 0
3819e00000-3819f70000 r-xp 00000000 08:03 532490 /lib64/libc-2.11.2.so
3819f70000-381a16f000 ---p 00170000 08:03 532490 /lib64/libc-2.11.2.so
381a16f000-381a173000 r--p 0016f000 08:03 532490 /lib64/libc-2.11.2.so
381a173000-381a174000 rw-p 00173000 08:03 532490 /lib64/libc-2.11.2.so
381a174000-381a179000 rw-p 00000000 00:00 0
7fb859c49000-7fb85fa7a000 r--p 00000000 08:03 192261 /usr/lib/locale/locale-archive
7fb85fa7a000-7fb85fa7d000 rw-p 00000000 00:00 0
7fb85fa95000-7fb85fa96000 rw-p 00000000 00:00 0
7fff64894000-7fff648a9000 rw-p 00000000 00:00 0 [stack]
7fff649ff000-7fff64a00000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
According to the man page, it means private (copy on write). No idea what the usefulness of such a mapping is without being able to read/write/execute anything in it, though.
Possibly it is private to libc, allowing it to modify the permissions to access it without a user program accidentally mucking it up.
This is something I've wondered about the specifics of too. It didn't appear until sometime in the last few years, but I'm unsure whether GNU binutils or the glibc dynamic linker (ld-linux.so.2) is responsible for the change.
At first I thought it was a sort of guard region created by the dynamic linker to protect against out of bounds access to a library's data segment, but it makes no sense for it to be so large. It's possible that it's a complete map of the while library file so that the dynamic linker can make it readable again at some time in the future (perhaps during dlopen or dlsym calls) to access ELF metadata that doesn't normally need to be mapped.
In any case, it's nasty bloat, especially on 32-bit machines where virtual address space is a precious resource. It also bloats the kernel page tables, increasing the kernelspace resources used by a process.
P.S. Sorry this isn't really an answer. I know it's just random bits and pieces that might help lead to an answer, but it was way too long for a comment.
Private mapping (MAP_PRIVATE): Modifications to the contents of the mapping are not visible to other processes.
For file mapping they are not carried through to the underlying file. Changes to the contents of the mapping are nevertheless private to each process.
The kernel accomplishes this by using the copy-on-write technique. This means that whenever a process attempts to modify the contents of a page, the kernel first creates a new, separate copy of that page for the process (and adjusts the process’s page tables).
For this reason, a MAP_PRIVATE mapping is sometimes referred to as a private, copy-on-write mapping. (Source: The Linux Programming Interface book)