I have a pintool that has calls to PIN_Backtrace(). It seems that libunwind does not support some registers in x86_64 and this leads to the following error message and an abnormal exit:
libunwind: uint64_t libunwind::Registers_x86_64::getRegister(int) const Source/bionic/libunwind/src/Registers.hpp:342 - unsupported x86_64 register
Aborted
I want the program to continue even with these errors. What is the most efficient way to handle this situation?
I debugged my pintool using gdb. Here is the backtrace when abort() is received by the debugger:
(gdb) bt
#0 0x00007ffff726c22b in ?? ()
#1 0x00007fffeca88730 in ?? ()
#2 0x00007ffff726c363 in ?? ()
#3 0x0000000000005386 in ?? ()
#4 0x0000000000000001 in ?? ()
#5 0x0000000000000000 in ?? ()
info proc mapping shows the address ranges as follows:
...
0x7fffeea3a000 0x7ffff71fb000 0x87c1000 0x0
0x7ffff71fb000 0x7ffff7283000 0x88000 0x0 /home/ahmad/Programs/pin-3.7-97619-g0d0c92f4f-gcc-linux/intel64/runtime/pincrt/libc-dynamic.so
0x7ffff7283000 0x7ffff7284000 0x1000 0x0
0x7ffff7284000 0x7ffff7286000 0x2000 0x88000 /home/ahmad/Programs/pin-3.7-97619-g0d0c92f4f-gcc-linux/intel64/runtime/pincrt/libc-dynamic.so
...
and
...
0x7fffe1687000 0x7fffec287000 0xac00000 0x0
0x7fffec287000 0x7fffed28e000 0x1007000 0x0
0x7fffed28e000 0x7fffedb6a000 0x8dc000 0x0
...
It seems that the libunwind library used by Pin is a special one exlusive to PinCRT. This can be seen in the info /proc/<PID>/maps output for the pintool:
...
7ffff7e33000-7ffff7e45000 r-xp 00000000 08:13 466277 /home/ahmad/Programs/pin-3.7-97619-g0d0c92f4f-gcc-linux/intel64/runtime/pincrt/libunwind-dynamic.so
7ffff7e45000-7ffff7e46000 r--p 00011000 08:13 466277 /home/ahmad/Programs/pin-3.7-97619-g0d0c92f4f-gcc-linux/intel64/runtime/pincrt/libunwind-dynamic.so
7ffff7e46000-7ffff7e47000 rw-p 00012000 08:13 466277 /home/ahmad/Programs/pin-3.7-97619-g0d0c92f4f-gcc-linux/intel64/runtime/pincrt/libunwind-dynamic.so
...
Related
I am new to Rust and I try to figure out where things are allocated.
I use Rust 1.64.0 on Ubuntu 22.04.0 (jammy):
$ rustc --version
rustc 1.64.0 (a55dd71d5 2022-09-19)
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
I try to explore the executable produce by compiling this very basic Rust program:
fn tester() {
let borrowed_string: &str = "world";
println!("{}", borrowed_string);
}
fn main() { tester(); }
Compilation (OK, this is overkill... but I want to be sure that I have all I need for using GDB):
$ cargo build --config "profile.dev.debug=true" --config "profile.dev.opt-level=0" --profile=dev
Compiling variables v0.1.0 (/home/denis/Documents/github/rust-playground/variables)
Finished dev [unoptimized + debuginfo] target(s) in 0.71s
Run rust-gdb, set the brakpoint and run the program:
$ rust-gdb target/debug/variables
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
...
Reading symbols from target/debug/variables...
(gdb) b 3
Breakpoint 1 at 0x7959: file src/main.rs, line 3.
(gdb) r
Starting program: /home/denis/Documents/github/rust-playground/variables/target/debug/variables
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, variables::tester () at src/main.rs:3
3 println!("{}", borrowed_string);
(gdb)
s
Print the memory mapping;
(gdb) info proc map
process 6985
Mapped address spaces:
Start Addr End Addr Size Offset Perms objfile
0x555555554000 0x55555555a000 0x6000 0x0 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x55555555a000 0x555555591000 0x37000 0x6000 r-xp /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x555555591000 0x55555559f000 0xe000 0x3d000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a0000 0x5555555a3000 0x3000 0x4b000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a3000 0x5555555a4000 0x1000 0x4e000 rw-p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a4000 0x5555555c5000 0x21000 0x0 rw-p [heap]
0x7ffff7d5c000 0x7ffff7d5f000 0x3000 0x0 rw-p
0x7ffff7d5f000 0x7ffff7d87000 0x28000 0x0 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7d87000 0x7ffff7f1c000 0x195000 0x28000 r-xp /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f1c000 0x7ffff7f74000 0x58000 0x1bd000 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f74000 0x7ffff7f78000 0x4000 0x214000 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f78000 0x7ffff7f7a000 0x2000 0x218000 rw-p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f7a000 0x7ffff7f87000 0xd000 0x0 rw-p
0x7ffff7f87000 0x7ffff7f8a000 0x3000 0x0 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7f8a000 0x7ffff7fa1000 0x17000 0x3000 r-xp /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa1000 0x7ffff7fa5000 0x4000 0x1a000 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa5000 0x7ffff7fa6000 0x1000 0x1d000 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa6000 0x7ffff7fa7000 0x1000 0x1e000 rw-p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fb8000 0x7ffff7fb9000 0x1000 0x0 ---p
0x7ffff7fb9000 0x7ffff7fbd000 0x4000 0x0 rw-p
0x7ffff7fbd000 0x7ffff7fc1000 0x4000 0x0 r--p [vvar]
0x7ffff7fc1000 0x7ffff7fc3000 0x2000 0x0 r-xp [vdso]
0x7ffff7fc3000 0x7ffff7fc5000 0x2000 0x0 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7fc5000 0x7ffff7fef000 0x2a000 0x2000 r-xp /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7fef000 0x7ffff7ffa000 0xb000 0x2c000 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7ffb000 0x7ffff7ffd000 0x2000 0x37000 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7ffd000 0x7ffff7fff000 0x2000 0x39000 rw-p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffffffde000 0x7ffffffff000 0x21000 0x0 rw-p [stack]
0xffffffffff600000 0xffffffffff601000 0x1000 0x0 --xp [vsyscall]
(gdb)
We have:
HEAP: [0x5555555A4000, 0x5555555C4FFF]
STACK: [0x7FFFFFFDE000, 0x7FFFFFFFEFFF]
I assume that the data "world" is allocated in the heap. Thus, let's find out... but look at what I obtain!!!
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4ba0
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4be0
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4c20
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4c60
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4ca0
1 pattern found.
(gdb)
Please note that:
0x5555555A4BA0 -> 0x5555555A4BE0 => 64
0x5555555A4BE0 -> 0x5555555A4C20 => 64
0x5555555A4C20 -> 0x5555555A4C60 => 64
Question: why does the position of the data change from one scan to the next?
(This question may not be related specifically to Rust.)
(gdb) p borrowed_string
$1 = "world"
(gdb) ptype borrowed_string
type = struct &str {
data_ptr: *mut u8,
length: usize,
}
(gdb) p borrowed_string.data_ptr
$2 = (*mut u8) 0x555555591000
(gdb) x/5c 0x555555591000
0x555555591000: 119 'w' 111 'o' 114 'r' 108 'l' 100 'd'
We have:
0x5555555A4000: start of the heap (included)
0x5555555C5000: end of the heap (excluded), and start of the "uncharted" memory territory
0x555555591000: address of the data "world"
0x7FFFF7D5F000: end of the "uncharted" memory territory
Question: what is this "uncharted" memory territory between 0x5555555C5000 (included) and 0x7FFFF7D5F000 (excluded)?
EDIT:
As mentioned in the comments below, I made a mistake. The data "world" is "before" the heap... The comments make sense. Thank you.
Question: what is this "uncharted" memory territory between 0x5555555C5000 (included) and 0x7FFFF7D5F000 (excluded)?
It's nothing. It's memory which is not mapped.
Where are "str" values allocated? Is it in the heap?
Your memory map answers the question:
0x555555591000 0x55555559f000 0xe000 0x3d000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
This is one of the areas where the binary itself is mapped. Most likely the "rodata" segment of the executable. String literals are generally "allocated" in the binary itself, then a static reference is loaded.
More generally str values are not allocated, they're a reference to data living somewhere else which could be on the heap (if the reference was obtained from a String or Box<str> or some pointers), in the binary (literals, static includes), or even on the stack (e.g. you can create an array on the stack then pass it through std::str::from_utf8 and get an &str out of it)
We have: HEAP: [0x5555555A4000, 0x5555555C4FFF]
That's... not really helpful or useful or true, frankly: modern systems have largely moved away from linear heaps and to memory map heaps. While Linux software still makes some limited use of (s)brk, BSDs qualify them as "historical curiosities". If you assume the heap is contained within these bounds you're going to face lots of surprises.
Question: why does the position of the data change from one scan to the next?
Because the debugger is allocating stuff in context of the program, I assume? So you might be finding the data gdb itself is storing.
I'm trying to calculate the base address of the library of a binary file.
I have the address of printf, puts ecc and then I subtract it's offset to get the base address of the library.
I was doing this for printf, puts and signal, but every time I got a different base address.
I also tried to do the things in this post, but I couldn't get the right result either.
ASLR is disabled.
this is where I take the address of the library function:
gdb-peda$ x/20wx 0x804b018
0x804b018 <signal#got.plt>: 0xf7e05720 0xf7e97010 0x080484e6 0x080484f6
0x804b028 <puts#got.plt>: 0xf7e3fb40 0x08048516 0x08048526 0xf7df0d90
0x804b038 <memset#got.plt>: 0xf7f18730 0x08048556 0x08048566 0x00000000
then I have:
gdb-peda$ info proc mapping
process 114562
Mapped address spaces:
Start Addr End Addr Size Offset objfile
0x8048000 0x804a000 0x2000 0x0 /home/ofey/CTF/Pwnable.tw/applestore/applestore
0x804a000 0x804b000 0x1000 0x1000 /home/ofey/CTF/Pwnable.tw/applestore/applestore
0x804b000 0x804c000 0x1000 0x2000 /home/ofey/CTF/Pwnable.tw/applestore/applestore
0x804c000 0x806e000 0x22000 0x0 [heap]
0xf7dd8000 0xf7fad000 0x1d5000 0x0 /lib/i386-linux-gnu/libc-2.27.so
0xf7fad000 0xf7fae000 0x1000 0x1d5000 /lib/i386-linux-gnu/libc-2.27.so
0xf7fae000 0xf7fb0000 0x2000 0x1d5000 /lib/i386-linux-gnu/libc-2.27.so
0xf7fb0000 0xf7fb1000 0x1000 0x1d7000 /lib/i386-linux-gnu/libc-2.27.so
0xf7fb1000 0xf7fb4000 0x3000 0x0
0xf7fd0000 0xf7fd2000 0x2000 0x0
0xf7fd2000 0xf7fd5000 0x3000 0x0 [vvar]
0xf7fd5000 0xf7fd6000 0x1000 0x0 [vdso]
0xf7fd6000 0xf7ffc000 0x26000 0x0 /lib/i386-linux-gnu/ld-2.27.so
0xf7ffc000 0xf7ffd000 0x1000 0x25000 /lib/i386-linux-gnu/ld-2.27.so
0xf7ffd000 0xf7ffe000 0x1000 0x26000 /lib/i386-linux-gnu/ld-2.27.so
0xfffdd000 0xffffe000 0x21000 0x0 [stack]
and :
gdb-peda$ info sharedlibrary
From To Syms Read Shared Object Library
0xf7fd6ab0 0xf7ff17fb Yes /lib/ld-linux.so.2
0xf7df0610 0xf7f3d386 Yes /lib/i386-linux-gnu/libc.so.6
I then found the offset of signal and puts to calculate the base libc address.
base_with_signal_offset = 0xf7e05720 - 0x3eda0 = 0xf7dc6980
base_with_puts_offset = 0xf7e3fb40 - 0x809c0 = 0xf7dbf180
I was expecting base_with_signal_offset = base_with_puts_offset = 0xf7dd8000, but that's not the case.
What I'm doing wrong?
EDIT(To let you understand where I got those offset):
readelf -s /lib/x86_64-linux-gnu/libc-2.27.so | grep puts
I get :
191: 00000000000809c0 512 FUNC GLOBAL DEFAULT 13 _IO_puts##GLIBC_2.2.5
422: 00000000000809c0 512 FUNC WEAK DEFAULT 13 puts##GLIBC_2.2.5
496: 00000000001266c0 1240 FUNC GLOBAL DEFAULT 13 putspent##GLIBC_2.2.5
678: 00000000001285d0 750 FUNC GLOBAL DEFAULT 13 putsgent##GLIBC_2.10
1141: 000000000007f1f0 396 FUNC WEAK DEFAULT 13 fputs##GLIBC_2.2.5
1677: 000000000007f1f0 396 FUNC GLOBAL DEFAULT 13 _IO_fputs##GLIBC_2.2.5
2310: 000000000008a640 143 FUNC WEAK DEFAULT 13 fputs_unlocked##GLIBC_2.2.5
I was expecting base_with_signal_offset = base_with_puts_offset = 0xf7dd8000
There are 3 numbers in your calculation:
&puts_at_runtime - symbol_value_from_readelf == &first_executable_pt_load_segment_libc.
The readelf output shows that you got one of these almost correct: the value of puts in 64-bit /lib/x86_64-linux-gnu/libc-2.27.so is indeed 0x809c0, but that is not the library you are actually using. You need to repeat the same on the actually used 32-bit library: /lib/i386-linux-gnu/libc-2.27.so.
For the first number -- &puts_at_runtime, you are using value from the puts#got.plt import stub. That value is only guaranteed to have been resolved (point to actual puts in libc.so) IFF you have LD_BIND_NOW=1 set in the environment, or you linked your executable with -z now linker flag, or you actually called puts already.
It may be better to print &puts in GDB.
The last number -- &first_executable_pt_load_segment_libc is correct (because info shared shows that libc.so.6 .text section starts at 0xf7df0610, which is between 0xf7dd8000 and 0xf7fad000.
So putting it all together, the only error was that you used the wrong version of libc.so to extract the symbol_value_from_readelf.
On my system:
#include <signal.h>
#include <stdio.h>
int main() {
puts("Hello");
signal(SIGINT, SIG_IGN);
return 0;
}
gcc -m32 t.c -fno-pie -no-pie
gdb -q a.out
... set breakpoint on exit from main
Breakpoint 1, 0x080491ae in main ()
(gdb) p &puts
$1 = (<text variable, no debug info> *) 0xf7e31300 <puts>
(gdb) p &signal
$2 = (<text variable, no debug info> *) 0xf7df7d20 <ssignal>
(gdb) info proc map
process 114065
Mapped address spaces:
Start Addr End Addr Size Offset objfile
0x8048000 0x8049000 0x1000 0x0 /tmp/a.out
...
0x804d000 0x806f000 0x22000 0x0 [heap]
0xf7dc5000 0xf7de2000 0x1d000 0x0 /lib/i386-linux-gnu/libc-2.29.so
...
(gdb) info shared
From To Syms Read Shared Object Library
0xf7fd5090 0xf7ff0553 Yes (*) /lib/ld-linux.so.2
0xf7de20e0 0xf7f2b8d6 Yes (*) /lib/i386-linux-gnu/libc.so.6
Given above, we expect readelf -s to give us 0xf7e31300 - 0xf7dc5000 ==
0x6c300 for puts and 0xf7df7d20 - 0xf7dc5000 == 0x32d20 for signal respectively.
readelf -Ws /lib/i386-linux-gnu/libc-2.29.so | egrep ' (puts|signal)\W'
452: 00032d20 68 FUNC WEAK DEFAULT 14 signal##GLIBC_2.0
458: 0006c300 400 FUNC WEAK DEFAULT 14 puts##GLIBC_2.0
QED.
I was trying to debug a deadlock issue in my code using gdb. When the program locked up, I attached to it with gdb to see what the threads were doing. For some reason, I cannot obtain backtraces for threads other than the current one:
(gdb) bt
#0 0xb774dcb0 in ?? ()
#1 0x8420e178 in TableCollect
#2 0x84200c12 in main
(gdb) thread apply all bt
Thread 3 (Thread 0xb6becb40 (LWP 18937)):
#0 0xb774dcb0 in ?? ()
#1 0x000d6ee6 in ?? ()
Thread 2 (Thread 0xb1f5eb40 (LWP 18939)):
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x5:
#0 0xb774dcb0 in ?? ()
Cannot access memory at address 0x5
(gdb) info threads
Id Target Id Frame
3 Thread 0xb6becb40 (LWP 18937) "loader.elf" 0xb774dcb0 in ?? ()
2 Thread 0xb1f5eb40 (LWP 18939) "Tuner" 0xb774dcb0 in ?? ()
* 1 Thread 0xb6bed700 (LWP 18936) "loader.elf" 0xb774dcb0 in ?? ()
(gdb) thread 2
[Switching to thread 2 (Thread 0xb1f5eb40 (LWP 18939))]
#0 0xb774dcb0 in ?? ()
(gdb) bt
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x5:
#0 0xb774dcb0 in ?? ()
Cannot access memory at address 0x5
(gdb) thread 3
[Switching to thread 3 (Thread 0xb6becb40 (LWP 18937))]
#0 0xb774dcb0 in ?? ()
(gdb) bt
#0 0xb774dcb0 in ?? ()
#1 0x000d6ee6 in ?? ()
ninja#vm:build$ cat /proc/`pidof loader.elf`/maps | grep b774d
b774b000-b774d000 r--p 00000000 00:00 0 [vvar]
b774d000-b774f000 r-xp 00000000 00:00 0 [vdso]
So it looks like all threads are inside a system call? That's likely since I'm expecting them all to be stuck on ptread_mutex_lock, but why don't I have valid stack frames below that? Why the 0x5 address for thread 2? And why does thread 3 have 0xd6ee6 on the stack, which isn't even mentioned in the memory map?
I am trying to debug our embedded linux system under very low temperatures (<40C). The problem is that it does not always boot correctly and I am trying to figure out why. After some analysis I saw that the kernel goes into panic during the start-up with the following output:
can: controller area network core (rev 20090105 abi 8)
NET: Registered protocol family 29
can: raw protocol (rev 20090105)
/opt/elinos-5.1/linux/linux-ppc-2.6.34/drivers/rtc/hctosys.c: unable to open rtc
device (rtc0)
ADDRCONF(NETDEV_UP): eth0: link is not ready
IP-Config: Complete:
device=eth0, addr=192.168.100.100, mask=255.255.255.0, gw=255.255.255.255,
host=192.168.100.100, domain=, nis-domain=(none),
bootserver=192.168.100.20, rootserver=192.168.100.20, rootpath=
Freeing unused kernel memory: 156k init
init started: BusyBox v1.6.1 (2013-06-03 11:53:03 CEST) multi-call binary
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
udevd-work[84]: '/sbin/modprobe -bv of:NioctlT<NULL>Cfsl,mpc5125-ioctl' unexpect
ed exit with status 0x000b
------------[ cut here ]------------
Badness at /opt/elinos-5.1/linux/linux-ppc-2.6.34/kernel/sched.c:3574
NIP: c001acfc LR: c001ace4 CTR: c01c5fa4
REGS: c790f7c0 TRAP: 0700 Not tainted (2.6.34.7-ELinOS-146-ipipe)
MSR: 00021032 <ME,CE,IR,DR> CR: 28000482 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790f870 c3ba6cb0 00000001 c790f8b8 00000008 00000000 00000000
GPR08: 00000000 c0370000 00000001 c0370000 5d0fabd2 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 00000000 c649e840 00000000 00000901 00000000 00000000
NIP [c001acfc] add_preempt_count+0x48/0xac
LR [c001ace4] add_preempt_count+0x30/0xac
Call Trace:
Instruction dump:
409e0038 54290024 8009000c 2f800000 40bc0028 48133285 2f830000 419e0068
3d20c037 8009d298 2f800000 409e0058 <0fe00000> 48000050 3d20c037 8009d130
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: 00000000 LR: 00000000 CTR: c01c7778
REGS: c790f990 TRAP: 0400 Tainted: G W (2.6.34.7-ELinOS-146-ipipe)
MSR: 20009032 <EE,ME,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790fa40 c3ba6cb0 00000008 00000008 00000008 c7912804 c0348ba4
GPR08: 00000047 c78a5414 00000000 00000004 28000482 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 c790f618 00200200 00000000 c790fa34 00200200 00000000
NIP [00000000] (null)
LR [00000000] (null)
Call Trace:
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..
INFO: RCU detected CPU 0 stall (t=2500 jiffies)
INFO: RCU detected CPU 0 stall (t=10000 jiffies)
INFO: RCU detected CPU 0 stall (t=17500 jiffies)
INFO: RCU detected CPU 0 stall (t=25000 jiffies)
INFO: RCU detected CPU 0 stall (t=32500 jiffies)
INFO: RCU detected CPU 0 stall (t=40000 jiffies)
System Halted, OK to turn off power
------------[ cut here ]------------
kernel BUG at /opt/elinos-5.1/linux/linux-ppc-2.6.34/mm/vmalloc.c:1228!
Oops: Exception in kernel mode, sig: 5 [#2]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: c009b0cc LR: c0013518 CTR: 00000000
REGS: c790f7c0 TRAP: 0700 Tainted: G D W (2.6.34.7-ELinOS-146-ipipe)
MSR: 00029032 <EE,ME,CE,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 078fe000 c790f870 c3ba6cb0 00001000 00000001 00000001 c9000000 fddff000
GPR08: ffffffff 000000d0 c001018c c790e000 48000488 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: c001018c 000000d0 00001000 ffffffff c9000000 fddff000 00000001 00000001
NIP [c009b0cc] __get_vm_area_node+0x68/0x204
LR [c0013518] __ioremap_caller+0x90/0x134
Call Trace:
Instruction dump:
7c9e2378 93e1003c 7cbf2b78 90010044 9261000c 92810010 92a10014 92c10018
92e1001c 93410028 800b000c 5400016e <0f000000> 70a00001 41820030 7c7e0034
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..:
Could anybody give me some hints how to approach the problem? What I want is to understand which component of the system (i.e. memory chip, etc.) causes this failure. I will be very happy to hear any ideas. Thanks.
All the OOPS/panic information show the exception happened in udevd context, I think it may be triggered by "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl".
To verify this, you can remove the "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl" entry in your root file system to see whether the system can boot up successfully.
I guess your platform CPU is PowerPC architecture. If so, the exception vector is 0x700, it means the instruction fetch exception. CPU tried to fetch one instruction from invalid address. The instruction flow is incremented if there are no jump/branch instructions. If option 1 is verified that it is related to "/sbin/modprobe", please check the kernel module to analysis the instruction fetch exception.
Good luck!
I have created a asp.net application with framework 4.0.
It has a sort of interview screen with so many questions on it with the paging.
It may have 1000 pages with 10 questions in each page.
when users are using this application screen randomly gets frozen and asks for window credentials.
We also keep getting error for the same application in error log as below:
A process serving application pool 'appicationpoolname' suffered a
fatal communication error with the Windows Process Activation Service.
The process id was '4768'. The data field contains the error number.
Not sure if both above error are related or not.
I have created a rule in debugdiag tool for application pool crash and that has generated a dump file. The full call stack for the dump file is as below
Function Arg 1 Arg 2 Arg 3 Arg 4 Source
KERNELBASE!RaiseException+58 04242420 00000000 00000003 010ce504
clr!Debugger::SendRawEvent+9b 010ce530 0000002f 0000015c 000012a0
clr!Debugger::RaiseStartupNotification+48 6fa92811 00000001 01000000 00000000
clr!Debugger::Startup+73 6fa92bf9 00000001 01000000 00000000
clr!DoAdditionalPEChecks+e5 6fa92ac5 00000001 00000000 00000000
clr!EEStartupHelper+5f4 00000000 6fa92a9d 00000001 00000000
clr!EEStartup+52 00000000 6fa92a41 00000001 00000000
clr!EnsureEEStarted+c4 00000000 6fa92f9d 014bb2c0 00000000
clr!ClrCreateManagedInstance+41 708d4bb8 708d4ca8 010ceeb4 00000000
webengine4!LegacyActivationShim::ClrCreateManagedInstance+61 708d4bb8 708d4ca8 010ceeb4 00c6e268
webengine4!GetIsapiProcessHost+48 00c6e268 01c44e8c 6d8e0000 01c44e68
webengine!GetIsapiProcessHost+d0 00c6e268 01c44e8c 00b907d8 00000000
wbhst_pm+2e2a 01c4fed8 01c44e68 00000000 010cf56c
wbhst_pm+335f 00b907d8 01c4fed8 01c4fdf0 010cf5e4
wbhst_pm!GetProtocolManager+35 00b907d8 01c4fed8 00000000 00b90898
w3wphost!AppHostInitialize+235e 7534415b 00000000 00000000 00000000
w3wphost!AppHostInitialize+2698 010cf69c 708e199c 010cf618 00000001
w3wphost!IPM_MESSAGE_PIPE::operator=+1c03 010cf69c 708e199c 00000001 708f1274
iiscore!W3_SERVER::GetProtocolManagerCustomInterface+36 010cf69c 708e199c 00000001 708f1274
webengine4!InitClrHost+186 708f1274 00000000 014c8004 00000400
webengine4!CMgdEngGlobalModule::OnGlobalApplicationResolveModules+31 01c4efac 01c40330 010cf77c 7210a67a
iiscore!VIRTUAL_MODULE::GlobalDoWork+152 00000400 01c4efac 01c4efa8 01c4efa8
iiscore!W3_SERVER::GlobalNotify+98 00000400 01c4efac 00000000 72122f3a
iiscore!W3_APPLICATION::ResolveModules+22 014cc280 00000000 014cc284 00000001
iiscore!W3_APPLICATION::SetupNotificationContext+95 00000000 00000001 014cc284 014cc2e0
iiscore!W3_CONTEXT::SetupStateMachinePhase2+2ab 014cc280 014cc284 00000000 72103358
iiscore!W3_CONTEXT::SetupStateMachine+241 014cc284 014cc280 75341400 010cf894
iiscore!W3_MAIN_CONTEXT::StartNotificationLoop+3f 014cc284 00000000 00c712a8 014cb828
iiscore!W3_MAIN_CONTEXT::OnNewRequest+47 014cb828 014cb828 72a914e6 753413f0
w3dt!UL_NATIVE_REQUEST::DoStateProcess+26 753413f0 010cf8c0 72a9154c 00000444
w3dt!UL_NATIVE_REQUEST::DoWork+60 00000444 00000000 014cb82c 010cf8f8
w3dt!OverlappedCompletionRoutine+1a 00000000 00000444 014cb82c 00c712b0
w3tp!THREAD_POOL_DATA::ThreadPoolThread+89 00000000 00be7fd8 72cf0000 010cf924
w3tp!THREAD_POOL_DATA::ThreadPoolThread+24 00c712a8 00000000 00000000 00be7fd8
w3tp!THREAD_MANAGER::ThreadManagerThread+39 00be7fd8 010cf970 770f9ed2 00be7fd8
kernel32!BaseThreadInitThunk+12 00be7fd8 590a45de 00000000 00000000
ntdll!__RtlUserThreadStart+70 72cf1e5c 00be7fd8 ffffffff 771872ff
ntdll!_RtlUserThreadStart+1b 72cf1e5c 00be7fd8 00000000 00000000
Exception Information
In W3WP__~1.DMP the assembly instruction at KERNELBASE!RaiseException+58 in C:\Windows\SysWOW64\KERNELBASE.dll from Microsoft Corporation has caused an unknown exception (0x04242420) on thread 5
I am not sure how to use this to fix the issue. I tried using Winddbg with .loadby sos mscorwks
command but it says invalid module mscorwks
Any help would be appriciated.
Thanks,
MMC
Is this a first chance exception dump or a second chance exception dump ? When you created the rule in Debug Diagnostic tool, did you change something in the Unconfigured first chance exceptions section ?
Based on the stack it seems this is a first chance exception dump. Make sure that you change the Action time for un-configured first chance exceptions to NONE and then let the tool generate a second chance exception dump and that would be worth to look at.
If it generates one, analyze it using debugdiag and paste the stack and we can see why it is crashing