I am using an open source JAVA based crawler which juggles millions of URI objects between the memory and disk. Basically the program is memory intensive.
The crash occurs once in a blue moon, I haven't been able to find the reason.
Pasting a part of the error-log, any pointers what possibly is causing this?
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b97ee1a07df, pid=1445, tid=1102055744
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode linux-amd64 )
# Problematic frame:
# V [libjvm.so+0x6227df]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x0000000059db5000): GCTaskThread [stack: 0x0000000041a00000,0x0000000041b01000] [id=1460]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000000000d5
Registers:
RAX=0x0000000000000008, RBX=0x0000000059df95f8, RCX=0x0000000000000005, RDX=0x00002aabee55cec0
RSP=0x0000000041affb60, RBP=0x0000000041affbd0, RSI=0x0000000059df95f0, RDI=0x00002aaab5c758b8
R8 =0x000000000000004b, R9 =0x0000000000000001, R10=0x00002aaaafbdfb01, R11=0x00002aabe4c0df38
R12=0x0000000000000001, R13=0x00002aabee55cec0, R14=0x00002aabe4c0e068, R15=0x0000000059df95f0
RIP=0x00002b97ee1a07df, EFL=0x0000000000010297, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
TRAPNO=0x000000000000000e
Top of Stack: (sp=0x0000000041affb60)
0x0000000041affb60: 0000000000000400 0000000059d9bfa0
0x0000000041affb70: 0000000000000005 01002aaab6b05998
0x0000000041affb80: 0000000041affbb0 0000000000000009
.
0x0000000041affd40: 0000000059db5810 0000000059db5820
0x0000000041affd50: 0000000059db5bf8 0000000041affd80
Instructions: (pc=0x00002b97ee1a07df)
0x00002b97ee1a07cf: 41 5e 41 5f c9 c3 48 8b 4a 10 4c 89 fe 4c 89 ea
0x00002b97ee1a07df: ff 91 d0 00 00 00 eb dc 49 8b 55 08 eb c9 4c 89
Stack: [0x0000000041a00000,0x0000000041b01000], sp=0x0000000041affb60, free space=3fe0000000000000018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [libjvm.so+0x6227df]
V [libjvm.so+0x6230b0]
V [libjvm.so+0x6222d4]
V [libjvm.so+0x23d872]
V [libjvm.so+0x62583b]
V [libjvm.so+0x365dda]
V [libjvm.so+0x5da2af]
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00002aac4c093800 JavaThread "JEEvictor" daemon [_thread_blocked, id=24783, stack(0x0000000046504000,0x0000000046605000)]
0x000000005a3a2000 JavaThread "JEEvictor" daemon [_thread_blocked, id=24782, stack(0x0000000046605000,0x0000000046706000)]
0x00002aac4c093000 JavaThread "JEEvictor" daemon [_thread_blocked, id=24781, stack(0x0000000046b0a000,0x0000000046c0b000)]
Other Threads:
0x0000000059dfb000 VMThread [stack: 0x0000000041c02000,0x0000000041d03000] [id=1462]
0x0000000059e32800 WatcherThread [stack: 0x00000000423c3000,0x00000000424c4000] [id=1469]
=>0x0000000059db5000 (exited) GCTaskThread [stack: 0x0000000041a00000,0x0000000041b01000] [id=1460]
VM state:at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0000000059d8b860] Threads_lock - owner thread: 0x0000000059dfb000
[0x0000000059d8bd60] Heap_lock - owner thread: 0x00002aac3c2dd000
Heap
PSYoungGen total 835840K, used 825021K [0x00002aabb3600000, 0x00002aabeffc0000, 0x00002aac33600000)
eden space 809024K, 100% used [0x00002aabb3600000,0x00002aabe4c10000,0x00002aabe4c10000)
from space 26816K, 59% used [0x00002aabe4c10000,0x00002aabe5baf538,0x00002aabe6640000)
to space 27136K, 6% used [0x00002aabee540000,0x00002aabee6ec138,0x00002aabeffc0000)
PSOldGen total 4096448K, used 3687089K [0x00002aaab3600000, 0x00002aabad670000, 0x00002aabb3600000)
object space 4096448K, 90% used [0x00002aaab3600000,0x00002aab946ac6b0,0x00002aabad670000)
PSPermGen total 44608K, used 44199K [0x00002aaaae200000, 0x00002aaab0d90000, 0x00002aaab3600000)
object space 44608K, 99% used [0x00002aaaae200000,0x00002aaab0d29ec0,0x00002aaab0d90000)
Dynamic libraries:
40000000-40009000 r-xp 00000000 fd:00 1116699 /usr/java/jdk1.6.0_18/bin/java
40108000-4010a000 rwxp 00008000 fd:00 1116699 /usr/java/jdk1.6.0_18/bin/java
40384000-40387000 ---p 40384000 00:00 0
40387000-40485000 rwxp 40387000 00:00 0
40647000-4064a000 ---p 40647000 00:00 0
4064a000-40748000 rwxp 4064a000 00:00 0
2aac4cb1b000-2aac50000000 ---p 2aac4cb1b000 00:00 0
2b97eda60000-2b97eda61000 rwxp 2b97eda60000 00:00 0
2b97eda71000-2b97eda72000 rwxp 2b97eda71000 00:00 0
2b97eda72000-2b97eda79000 r-xp 00000000 fd:00 1119007 /usr/java/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b97eda79000-2b97edb7a000 ---p 00007000 fd:00 1119007 /usr/java/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b97edb7a000-2b97edb7c000 rwxp 00008000 fd:00 1119007 /usr/java/jdk1.6.0_18/jre/lib/amd64/jli/libjli.so
2b97edb7c000-2b97edb7e000 rwxp 2b97edb7c000 00:00 0
2b97edb7e000-2b97ee332000 r-xp 00000000 fd:00 1119054 /usr/java/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b97ee332000-2b97ee432000 ---p 007b4000 fd:00 1119054 /usr/java/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b97ee432000-2b97ee5bc000 rwxp 007b4000 fd:00 1119054 /usr/java/jdk1.6.0_18/jre/lib/amd64/server/libjvm.so
2b97ee5bc000-2b97ee5f5000 rwxp 2b97ee5bc000 00:00 0
7fffc4df8000-7fffc4e0e000 rwxp 7ffffffe9000 00:00 0 [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
VM Arguments:
jvm_args: -Dheritrix.home=/heritrix/heritrix-3.1.0 -Djava.protocol.handler.pkgs=org.archive.net -Dheritrix.out=/heritrix/heritrix-3.1.0/heritrix_out.log -Xmx6144M
java_command: org.archive.crawler.Heritrix -p 8544 -b / -a user:pass
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/usr/java/default
PATH=/apps/pig-0.9.0/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/lpp/mmfs/bin:/home/ppra/bin:/usr/lpp/mmfs/bin:/usr/lpp/mmfs/bin
LD_LIBRARY_PATH=/usr/java/jdk1.6.0_18/jre/lib/amd64/server:/usr/java/jdk1.6.0_18/jre/lib/amd64:/usr/java/jdk1.6.0_18/jre/../lib/amd64
SHELL=/bin/bash
--------------- S Y S T E M ---------------
OS:Red Hat Enterprise Linux Server release 5.4 (Tikanga)
uname:Linux 2.6.18-164.11.1.el5 #1 SMP Wed Jan 6 13:26:04 EST 2010 x86_64
libc:glibc 2.5 NPTL 2.5
rlimit: STACK 10240k, CORE 0k, NPROC 268287, NOFILE 1024, AS infinity
load average:1.31 0.80 0.67
CPU:total 16 (8 cores per cpu, 1 threads per core) family 6 model 29 stepping 1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1
Memory: 4k page, physical 32960544k(162620k free), swap 6291448k(6290568k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (16.0-b13) for linux-amd64 JRE (1.6.0_18-b07), built on Dec 17 2009 13:42:22 by "java_re" with gcc 3.2.2 (SuSE Linux)
time: Sat Aug 4 07:27:56 2012
elapsed time: 242358 seconds
Well, SIGSEGV denotes a segfault. Likely you are either trying to read something that wasn't ever initialized, or the garbage collector is screwing you.
Related
I am new to Rust and I try to figure out where things are allocated.
I use Rust 1.64.0 on Ubuntu 22.04.0 (jammy):
$ rustc --version
rustc 1.64.0 (a55dd71d5 2022-09-19)
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
I try to explore the executable produce by compiling this very basic Rust program:
fn tester() {
let borrowed_string: &str = "world";
println!("{}", borrowed_string);
}
fn main() { tester(); }
Compilation (OK, this is overkill... but I want to be sure that I have all I need for using GDB):
$ cargo build --config "profile.dev.debug=true" --config "profile.dev.opt-level=0" --profile=dev
Compiling variables v0.1.0 (/home/denis/Documents/github/rust-playground/variables)
Finished dev [unoptimized + debuginfo] target(s) in 0.71s
Run rust-gdb, set the brakpoint and run the program:
$ rust-gdb target/debug/variables
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
...
Reading symbols from target/debug/variables...
(gdb) b 3
Breakpoint 1 at 0x7959: file src/main.rs, line 3.
(gdb) r
Starting program: /home/denis/Documents/github/rust-playground/variables/target/debug/variables
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, variables::tester () at src/main.rs:3
3 println!("{}", borrowed_string);
(gdb)
s
Print the memory mapping;
(gdb) info proc map
process 6985
Mapped address spaces:
Start Addr End Addr Size Offset Perms objfile
0x555555554000 0x55555555a000 0x6000 0x0 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x55555555a000 0x555555591000 0x37000 0x6000 r-xp /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x555555591000 0x55555559f000 0xe000 0x3d000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a0000 0x5555555a3000 0x3000 0x4b000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a3000 0x5555555a4000 0x1000 0x4e000 rw-p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
0x5555555a4000 0x5555555c5000 0x21000 0x0 rw-p [heap]
0x7ffff7d5c000 0x7ffff7d5f000 0x3000 0x0 rw-p
0x7ffff7d5f000 0x7ffff7d87000 0x28000 0x0 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7d87000 0x7ffff7f1c000 0x195000 0x28000 r-xp /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f1c000 0x7ffff7f74000 0x58000 0x1bd000 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f74000 0x7ffff7f78000 0x4000 0x214000 r--p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f78000 0x7ffff7f7a000 0x2000 0x218000 rw-p /usr/lib/x86_64-linux-gnu/libc.so.6
0x7ffff7f7a000 0x7ffff7f87000 0xd000 0x0 rw-p
0x7ffff7f87000 0x7ffff7f8a000 0x3000 0x0 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7f8a000 0x7ffff7fa1000 0x17000 0x3000 r-xp /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa1000 0x7ffff7fa5000 0x4000 0x1a000 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa5000 0x7ffff7fa6000 0x1000 0x1d000 r--p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fa6000 0x7ffff7fa7000 0x1000 0x1e000 rw-p /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
0x7ffff7fb8000 0x7ffff7fb9000 0x1000 0x0 ---p
0x7ffff7fb9000 0x7ffff7fbd000 0x4000 0x0 rw-p
0x7ffff7fbd000 0x7ffff7fc1000 0x4000 0x0 r--p [vvar]
0x7ffff7fc1000 0x7ffff7fc3000 0x2000 0x0 r-xp [vdso]
0x7ffff7fc3000 0x7ffff7fc5000 0x2000 0x0 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7fc5000 0x7ffff7fef000 0x2a000 0x2000 r-xp /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7fef000 0x7ffff7ffa000 0xb000 0x2c000 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7ffb000 0x7ffff7ffd000 0x2000 0x37000 r--p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffff7ffd000 0x7ffff7fff000 0x2000 0x39000 rw-p /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
0x7ffffffde000 0x7ffffffff000 0x21000 0x0 rw-p [stack]
0xffffffffff600000 0xffffffffff601000 0x1000 0x0 --xp [vsyscall]
(gdb)
We have:
HEAP: [0x5555555A4000, 0x5555555C4FFF]
STACK: [0x7FFFFFFDE000, 0x7FFFFFFFEFFF]
I assume that the data "world" is allocated in the heap. Thus, let's find out... but look at what I obtain!!!
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4ba0
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4be0
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4c20
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4c60
1 pattern found.
(gdb) find 0x5555555A4000, 0x5555555C4FFF, "world"
0x5555555a4ca0
1 pattern found.
(gdb)
Please note that:
0x5555555A4BA0 -> 0x5555555A4BE0 => 64
0x5555555A4BE0 -> 0x5555555A4C20 => 64
0x5555555A4C20 -> 0x5555555A4C60 => 64
Question: why does the position of the data change from one scan to the next?
(This question may not be related specifically to Rust.)
(gdb) p borrowed_string
$1 = "world"
(gdb) ptype borrowed_string
type = struct &str {
data_ptr: *mut u8,
length: usize,
}
(gdb) p borrowed_string.data_ptr
$2 = (*mut u8) 0x555555591000
(gdb) x/5c 0x555555591000
0x555555591000: 119 'w' 111 'o' 114 'r' 108 'l' 100 'd'
We have:
0x5555555A4000: start of the heap (included)
0x5555555C5000: end of the heap (excluded), and start of the "uncharted" memory territory
0x555555591000: address of the data "world"
0x7FFFF7D5F000: end of the "uncharted" memory territory
Question: what is this "uncharted" memory territory between 0x5555555C5000 (included) and 0x7FFFF7D5F000 (excluded)?
EDIT:
As mentioned in the comments below, I made a mistake. The data "world" is "before" the heap... The comments make sense. Thank you.
Question: what is this "uncharted" memory territory between 0x5555555C5000 (included) and 0x7FFFF7D5F000 (excluded)?
It's nothing. It's memory which is not mapped.
Where are "str" values allocated? Is it in the heap?
Your memory map answers the question:
0x555555591000 0x55555559f000 0xe000 0x3d000 r--p /home/denis/Documents/github/rust-playground/variables/target/debug/variables
This is one of the areas where the binary itself is mapped. Most likely the "rodata" segment of the executable. String literals are generally "allocated" in the binary itself, then a static reference is loaded.
More generally str values are not allocated, they're a reference to data living somewhere else which could be on the heap (if the reference was obtained from a String or Box<str> or some pointers), in the binary (literals, static includes), or even on the stack (e.g. you can create an array on the stack then pass it through std::str::from_utf8 and get an &str out of it)
We have: HEAP: [0x5555555A4000, 0x5555555C4FFF]
That's... not really helpful or useful or true, frankly: modern systems have largely moved away from linear heaps and to memory map heaps. While Linux software still makes some limited use of (s)brk, BSDs qualify them as "historical curiosities". If you assume the heap is contained within these bounds you're going to face lots of surprises.
Question: why does the position of the data change from one scan to the next?
Because the debugger is allocating stuff in context of the program, I assume? So you might be finding the data gdb itself is storing.
I was running Gapcloser (a tool released from Soapdenovo), and it kept comes up a message and failed in the middle of the process:
*** Error in `GapCloser': free(): invalid pointer: 0x0000000001eb4a80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f7658b0a7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f7658b1337a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f7658b1753c]
GapCloser[0x405d41]
GapCloser[0x40983b]
GapCloser[0x405ad3]
GapCloser[0x403982]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f7658ab3830]
GapCloser[0x402b59]
======= Memory map: ========
00400000-0042a000 r-xp 00000000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
00629000-0062a000 r--p 00029000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
0062a000-0062b000 rw-p 0002a000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
01e9b000-01ed7000 rw-p 00000000 00:00 0 [heap]
7f6689a9b000-7f6aeab13000 rw-p 00000000 00:00 0
7f6f2ac77000-7f6f3c4b9000 rw-p 00000000 00:00 0
7f6f44000000-7f6f44021000 rw-p 00000000 00:00 0
7f6f44021000-7f6f48000000 ---p 00000000 00:00 0
7f6f4b9f4000-7f7658a93000 rw-p 00000000 00:00 0
7f7658a93000-7f7658c53000 r-xp 00000000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658c53000-7f7658e53000 ---p 001c0000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e53000-7f7658e57000 r--p 001c0000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e57000-7f7658e59000 rw-p 001c4000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e59000-7f7658e5d000 rw-p 00000000 00:00 0
7f7658e5d000-7f7658e74000 r-xp 00000000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7658e74000-7f7659073000 ---p 00017000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659073000-7f7659074000 r--p 00016000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659074000-7f7659075000 rw-p 00017000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659075000-7f765917d000 r-xp 00000000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765917d000-7f765937c000 ---p 00108000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937c000-7f765937d000 r--p 00107000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937d000-7f765937e000 rw-p 00108000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937e000-7f76594fa000 r-xp 00000000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f76594fa000-7f76596fa000 ---p 0017c000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f76596fa000-7f7659704000 r--p 0017c000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f7659704000-7f7659706000 rw-p 00186000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f7659706000-7f765970a000 rw-p 00000000 00:00 0
7f765970a000-7f7659722000 r-xp 00000000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659722000-7f7659921000 ---p 00018000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659921000-7f7659922000 r--p 00017000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659922000-7f7659923000 rw-p 00018000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659923000-7f7659927000 rw-p 00000000 00:00 0
7f7659927000-7f765994d000 r-xp 00000000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659afc000-7f7659b3a000 rw-p 00000000 00:00 0
7f7659b4b000-7f7659b4c000 rw-p 00000000 00:00 0
7f7659b4c000-7f7659b4d000 r--p 00025000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659b4d000-7f7659b4e000 rw-p 00026000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659b4e000-7f7659b4f000 rw-p 00000000 00:00 0
7ffc5a9e7000-7ffc5aa08000 rw-p 00000000 00:00 0 [stack]
7ffc5abaa000-7ffc5abad000 r--p 00000000 00:00 0 [vvar]
7ffc5abad000-7ffc5abaf000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
And this is my log file:
cat gapcloser.log
[WARNING] Maximum supported read length is 155 bp! Program will use 155 instead of 251.
Program: GapCloser
Version: 1.12
Parameters:
-a (scaffold file): sunbird_platanus_raw_kraken_scaff_scaffold.fa
-b (config file): sunbird_raw.config
-o (output file): sunbird_platanus_raw_kraken_scaff_gapclosed
-l (max read len): 155
-p (overlap para): 25
-t (thread num): 40
>>>>>>>>>>hash initializing<<<<<<<<<<
counting reads
in lib:
total reads sum: 289413008
Paired-end reads sum: 289413008
in lib:
total reads sum: 21371568
Paired-end reads sum: 21371568
in lib:
total reads sum: 21944342
Paired-end reads sum: 21944342
in lib:
total reads sum: 20671814
Paired-end reads sum: 20671814
in lib:
total reads sum: 20949232
Paired-end reads sum: 20949232
in lib:
total reads sum: 22977390
Paired-end reads sum: 22977390
in lib:
total reads sum: 19285920
Paired-end reads sum: 19285920
in lib:
total reads sum: 23065494
Paired-end reads sum: 23065494
in lib:
total reads sum: 21131136
Paired-end reads sum: 21131136
in lib:
total reads sum: 20350118
Paired-end reads sum: 20350118
in lib:
total reads sum: 20579926
Paired-end reads sum: 20579926
in lib:
total reads sum: 18998014
Paired-end reads sum: 18998014
in lib:
total reads sum: 20039196
Paired-end reads sum: 20039196
reads sum: 540777158
spent time: 1658s
reads counted
allocating memorys
spent time: 19s
memorys allocated
initializing reads
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
spent time: 1984s
reads initialized
sorting reads in reversed direction
spent time: 1357s
reads sorted
initializing hash on reversed reads
kmers sum: 413463339
spent time: 951s
hash initialized
reversing reads
spent time: 633s
reads reversed
sorting reads in forward direction
spent time: 1363s
reads sorted
initializing hash on forward reads
kmers sum: 783012674
spent time: 994s
hash initialized
marking unique reads
spent time: 41s
unique reads marked
sorting reads by serial number
spent time: 1011s
reads sorted
spent total time: 10011s
>>>>>>>>>>hash initialization finished<<<<<<<<<<
In fact, I have run this programme successfully with the same set of data before, however our server has been re-installed therefore Gapcloser was re-installed again, but running the same data no longer works due to this issue...
I thought it was memory's problem so I cleaned the memory page cache by this command, but it did not help at all, this is my current memory, which should definitely be enough for Gapcloser.
free -h
total used free shared buff/cache available
Mem: 755G 2.7G 539G 14M 214G 751G
Swap: 810G 106M 810G
By the way I am using Gapcloser source code version, and my ubuntu is 16.04 version.
This is my Soapdenovo/Gapcloser Config file which I think it's not a problem because it seems to have read all the libraries successfully?
#FOR MATEPAIR LIBRARIES
#maximal read length
max_rd_len=251
#average insert size for 1kb
[LIB]
#average insert size for gel free
avg_ins=1805
#reverse-forward, generated from circularizing libraries with typical insert size greater than 2Kb
reverse_seq=1
#In which parts the reads are used. 2 - only scaffold assembly
asm_flags=4
#assembler will cut the reads from the current library to this length
rd_len_cutof=150
#In which order the reads are used while scaffolding,1-170/200/250bp,2-350/500bp,3-800bp,4-2KB,5-5Kb,6-10Kb,7-20Kb
rank=4
#fastq files for 1kb1
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K1_R2.fastq
[LIB]
#average insert size for 3kb1
avg_ins=3918
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 3kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K1_R2.fastq
[LIB]
#average insert size for 5kb1
avg_ins=6243
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 5kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K1_R2.fastq
[LIB]
#average insert size for 8kb1
avg_ins=8930
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 8kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K1_R2.fastq
[LIB]
#average insert size for 12kb1
avg_ins=12507
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 12kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K1_R2.fastq
[LIB]
#average insert size for 20kb1
avg_ins=17407
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=7
#fastq files for 20kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K1_R2.fastq
[LIB]
#average insert size for 1kb2
avg_ins=9793
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=4
#fastq files for 1kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K2_R2.fastq
[LIB]
#average insert size for 3kb2
avg_ins=3944
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 3kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K2_R2.fastq
[LIB]
#average insert size for 5kb2
avg_ins=6225
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 5kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K2_R2.fastq
[LIB]
#average insert size for 8kb2
avg_ins=8913
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 8kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K2_R2.fastq
[LIB]
#average insert size for 12kb2
avg_ins=12491
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 12kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K2_R2.fastq
[LIB]
#average insert size for 20kb2
avg_ins=17370
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=7
#fastq files for 20kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K2_R2.fastq
### 2x250 run data
[LIB]
avg_ins=434
reverse_seq=0
#only contigs assembly
asm_flags=4
rank=2
#fastq files from LGC16_JM06_591bp
q1=/home/cch/sunbird/sunbird_kraken/Pure_LGC16_JM09_594bp_CTTGTA_L002_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_LGC16_JM09_594bp_CTTGTA_L002_R2.fastq
I have run the Gapcloser through Valgrind and got 2 errors, but I do not know how to fix it, does anyone have any ideas?
valgrind --leak-check=yes GapCloser
==77240== Memcheck, a memory error detector
==77240== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==77240== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==77240== Command: GapCloser
==77240==
==77240== Syscall param open(filename) points to unaddressable byte(s)
==77240== at 0x59FB040: __open_nocancel (syscall-template.S:84)
==77240== by 0x597DACD: _IO_file_open (fileops.c:221)
==77240== by 0x597DD34: _IO_file_fopen##GLIBC_2.2.5 (fileops.c:328)
==77240== by 0x5971D33: __fopen_internal (iofopen.c:86)
==77240== by 0x510AD2F: std::__basic_file<char>::open(char const*, std::_Ios_Openmode, int) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x514B259: std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x514B42F: std::basic_ifstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x403450: main (Main.cpp:121)
==77240== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==77240==
[Error] Can not open input file.
Version:
1.12
Contact:
soap#genomics.org.cn
Usage:
GapCloser [options]
-a <string> input scaffold file name, required.
-b <string> input library info file name, required.
-o <string> output file name, required.
-l <int> maximum read length (<=155), default=100.
-p <int> overlap param(<=31), default=25.
-t <int> thread number, default=1.
-h -? output help information.
==77240==
==77240== Process terminating with default action of signal 27 (SIGPROF)
==77240== at 0x5A0CE0F: write_gmon (gmon.c:354)
==77240== by 0x5A0D589: _mcleanup (gmon.c:422)
==77240== by 0x593DFF7: __run_exit_handlers (exit.c:82)
==77240== by 0x593E044: exit (exit.c:104)
==77240== by 0x40315F: usage() (Main.cpp:60)
==77240== by 0x403497: main (Main.cpp:124)
==77240==
==77240== HEAP SUMMARY:
==77240== in use at exit: 299,124 bytes in 2 blocks
==77240== total heap usage: 4 allocs, 2 frees, 300,700 bytes allocated
==77240==
==77240== LEAK SUMMARY:
==77240== definitely lost: 0 bytes in 0 blocks
==77240== indirectly lost: 0 bytes in 0 blocks
==77240== possibly lost: 0 bytes in 0 blocks
==77240== still reachable: 299,124 bytes in 2 blocks
==77240== suppressed: 0 bytes in 0 blocks
==77240== Reachable blocks (those to which a pointer was found) are not shown.
==77240== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==77240==
==77240== For counts of detected and suppressed errors, rerun with: -v
==77240== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 0 from 0)
Profiling timer expired
I am learning about the Linux user space memory layout on x86_64 systems and wanted to print some addresses from some of the sections. I used this Rust code:
fn main() {
let x = 3; // should be stored on stack
let s = "hello"; // should be in the .data section
println!("stack ≈ {:p}", &x);
println!(".text ≈ {:p}", main as *const ());
println!(".data ≈ {:p}", s);
use std::io;
let mut f = std::fs::File::open("/proc/self/maps").unwrap();
let out = io::stdout();
io::copy(&mut f, &mut out.lock()).unwrap();
}
This code also prints the file /proc/self/maps to stdout. I compiled this file mem.rs simply with rustc mem.rs. It printed:
stack ≈ 0x7ffffbf82f2c
.text ≈ 0x7f45b7c0a2b0
.data ≈ 0x7f45b7c4d35b
7f45b6800000-7f45b6c00000 rw-- 00000000 00:00 0
7f45b6de0000-7f45b6f9a000 r-x- 00000000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
7f45b6f9a000-7f45b6fa2000 ---- 001ba000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
[ ... more .so files]
7f45b7a22000-7f45b7a23000 r--- 00022000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f45b7a23000-7f45b7a24000 rw-- 00023000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f45b7a24000-7f45b7a25000 rw-- 00000000 00:00 0
7f45b7aa0000-7f45b7aa2000 rw-- 00000000 00:00 0
7f45b7ab0000-7f45b7ab2000 rw-- 00000000 00:00 0
7f45b7ac0000-7f45b7ac1000 rw-- 00000000 00:00 0
7f45b7ad0000-7f45b7ad1000 rw-- 00000000 00:00 0
7f45b7ae0000-7f45b7ae2000 rw-- 00000000 00:00 0
7f45b7c00000-7f45b7c5f000 r-x- 00000000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e5e000-7f45b7e62000 r--- 0005e000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e62000-7f45b7e63000 rw-- 00062000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e63000-7f45b7e64000 rw-- 00000000 00:00 0
7ffffb784000-7ffffb785000 ---- 00000000 00:00 0 [stack]
7ffffb785000-7ffffbf84000 rw-- 00000000 00:00 0
7ffffc263000-7ffffc264000 r-x- 00000000 00:00 0 [vdso]
At least the addresses I printed on my own seem to match what maps says. But when I execute cat /proc/self/maps in the terminal, I get this output:
00400000-0040b000 r-x- 00000000 00:00 107117 /bin/cat
0060a000-0060b000 r--- 0000a000 00:00 107117 /bin/cat
0060b000-0060c000 rw-- 0000b000 00:00 107117 /bin/cat
0071c000-0073d000 rw-- 00000000 00:00 0 [heap]
7f7deb933000-7f7debc30000 r--- 00000000 00:00 758714 /usr/lib/locale/locale-archive
7f7debc30000-7f7debdea000 r-x- 00000000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
7f7debdea000-7f7debdf2000 ---- 001ba000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
[ ... more .so files ...]
7f7dec222000-7f7dec223000 r--- 00022000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f7dec223000-7f7dec224000 rw-- 00023000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f7dec224000-7f7dec225000 rw-- 00000000 00:00 0
7f7dec250000-7f7dec252000 rw-- 00000000 00:00 0
7f7dec260000-7f7dec261000 rw-- 00000000 00:00 0
7f7dec270000-7f7dec272000 rw-- 00000000 00:00 0
7ffff09e8000-7ffff11e8000 rw-- 00000000 00:00 0 [stack]
7ffff1689000-7ffff168a000 r-x- 00000000 00:00 0 [vdso]
The latter result matches everything I read about this topic: the sections from the executable are mapped in the lower end of the virtual address space (beginning around 0x400000).
I executed and compiled everything in the Linux Subsystem for Windows (Ubuntu 14.04 basically). I know, it's new and stuff, but I'm fairly sure this is not an issue with the subsystem (please tell me if it is, though!). Rust 1.14 is that matters (I doubt it),
I also tried the same with a C program (excuse my probably bad C):
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char **argv) {
FILE *test_file;
char buf[4096];
if ((test_file = fopen ("/proc/self/maps", "r")) != NULL) {
while (!feof (test_file)) {
fgets (buf, sizeof (buf), test_file);
puts (buf);
}
}
return 0;
}
It outputs something similar to cat:
00400000-00401000 r-x- 00000000 00:00 1325490 /home/lukas/tmp/a.out
00600000-00601000 r--- 00000000 00:00 1325490 /home/lukas/tmp/a.out
00601000-00602000 rw-- 00001000 00:00 1325490 /home/lukas/tmp/a.out
Why is the Rust executable mapped to large addresses near the stack?
Using rustc -Z print-link-args addr.rs, you can see what linker invocation the Rust compiler will use. Since the current linker happens to be cc, we can directly reuse these options for the C program. Ignoring unimportant arguments and removing others one-by-one, I was left with this compiler invocation:
gcc -fPIC -pie addr.c -o addr-c
Compiling the C code like this produces similar addresses as the Rust-compiled executable, indicating that one or both of those options is a likely culprit. This changes the question to "why does -fPIC and/or -pie map to such high addresses?"
I found another question and answer that seems to shed light on that:
The PIE binary is linked just as a shared library, and so its default load address (the .p_vaddr of the first LOAD segment) is zero. The expectation is that something will relocate this binary away from zero page, and load it at some random address.
Using readelf -e on the Rust executable, we can see that the first LOAD segment does have a virtual address of zero:
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x000000000005e6b4 0x000000000005e6b4 R E 200000
LOAD 0x000000000005ead0 0x000000000025ead0 0x000000000025ead0
0x00000000000039d1 0x00000000000049e8 RW 200000
I guess that this then changes the question to "why are these random addresses chosen", but I'm not sure of that answer. ^_^ A hunch tells me that ASLR comes into play. This other answer seems to bear that out:
PIE is to support ASLR in executable files.
ASLR is a security technique to help harden programs against certain types of attacks, so it makes sense that Rust, with its safety-minded approach, would attempt to enable something like this by default. Indeed, the addresses change a small bit each invocation:
root#97bcff9a925c:/# ./addr | grep 'r-xp' | grep 'addr'
5587cea9d000-5587ceafc000 r-xp 00000000 00:21 206 /addr
561d8aae2000-561d8ab41000 r-xp 00000000 00:21 206 /addr
555c30ffd000-555c3105c000 r-xp 00000000 00:21 206 /addr
55db249d5000-55db24a34000 r-xp 00000000 00:21 206 /addr
55e988572000-55e9885d1000 r-xp 00000000 00:21 206 /addr
560400e1b000-560400e7a000 r-xp 00000000 00:21 206 /addr
I am trying to debug our embedded linux system under very low temperatures (<40C). The problem is that it does not always boot correctly and I am trying to figure out why. After some analysis I saw that the kernel goes into panic during the start-up with the following output:
can: controller area network core (rev 20090105 abi 8)
NET: Registered protocol family 29
can: raw protocol (rev 20090105)
/opt/elinos-5.1/linux/linux-ppc-2.6.34/drivers/rtc/hctosys.c: unable to open rtc
device (rtc0)
ADDRCONF(NETDEV_UP): eth0: link is not ready
IP-Config: Complete:
device=eth0, addr=192.168.100.100, mask=255.255.255.0, gw=255.255.255.255,
host=192.168.100.100, domain=, nis-domain=(none),
bootserver=192.168.100.20, rootserver=192.168.100.20, rootpath=
Freeing unused kernel memory: 156k init
init started: BusyBox v1.6.1 (2013-06-03 11:53:03 CEST) multi-call binary
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
udevd-work[84]: '/sbin/modprobe -bv of:NioctlT<NULL>Cfsl,mpc5125-ioctl' unexpect
ed exit with status 0x000b
------------[ cut here ]------------
Badness at /opt/elinos-5.1/linux/linux-ppc-2.6.34/kernel/sched.c:3574
NIP: c001acfc LR: c001ace4 CTR: c01c5fa4
REGS: c790f7c0 TRAP: 0700 Not tainted (2.6.34.7-ELinOS-146-ipipe)
MSR: 00021032 <ME,CE,IR,DR> CR: 28000482 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790f870 c3ba6cb0 00000001 c790f8b8 00000008 00000000 00000000
GPR08: 00000000 c0370000 00000001 c0370000 5d0fabd2 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 00000000 c649e840 00000000 00000901 00000000 00000000
NIP [c001acfc] add_preempt_count+0x48/0xac
LR [c001ace4] add_preempt_count+0x30/0xac
Call Trace:
Instruction dump:
409e0038 54290024 8009000c 2f800000 40bc0028 48133285 2f830000 419e0068
3d20c037 8009d298 2f800000 409e0058 <0fe00000> 48000050 3d20c037 8009d130
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: 00000000 LR: 00000000 CTR: c01c7778
REGS: c790f990 TRAP: 0400 Tainted: G W (2.6.34.7-ELinOS-146-ipipe)
MSR: 20009032 <EE,ME,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790fa40 c3ba6cb0 00000008 00000008 00000008 c7912804 c0348ba4
GPR08: 00000047 c78a5414 00000000 00000004 28000482 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 c790f618 00200200 00000000 c790fa34 00200200 00000000
NIP [00000000] (null)
LR [00000000] (null)
Call Trace:
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..
INFO: RCU detected CPU 0 stall (t=2500 jiffies)
INFO: RCU detected CPU 0 stall (t=10000 jiffies)
INFO: RCU detected CPU 0 stall (t=17500 jiffies)
INFO: RCU detected CPU 0 stall (t=25000 jiffies)
INFO: RCU detected CPU 0 stall (t=32500 jiffies)
INFO: RCU detected CPU 0 stall (t=40000 jiffies)
System Halted, OK to turn off power
------------[ cut here ]------------
kernel BUG at /opt/elinos-5.1/linux/linux-ppc-2.6.34/mm/vmalloc.c:1228!
Oops: Exception in kernel mode, sig: 5 [#2]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: c009b0cc LR: c0013518 CTR: 00000000
REGS: c790f7c0 TRAP: 0700 Tainted: G D W (2.6.34.7-ELinOS-146-ipipe)
MSR: 00029032 <EE,ME,CE,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 078fe000 c790f870 c3ba6cb0 00001000 00000001 00000001 c9000000 fddff000
GPR08: ffffffff 000000d0 c001018c c790e000 48000488 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: c001018c 000000d0 00001000 ffffffff c9000000 fddff000 00000001 00000001
NIP [c009b0cc] __get_vm_area_node+0x68/0x204
LR [c0013518] __ioremap_caller+0x90/0x134
Call Trace:
Instruction dump:
7c9e2378 93e1003c 7cbf2b78 90010044 9261000c 92810010 92a10014 92c10018
92e1001c 93410028 800b000c 5400016e <0f000000> 70a00001 41820030 7c7e0034
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..:
Could anybody give me some hints how to approach the problem? What I want is to understand which component of the system (i.e. memory chip, etc.) causes this failure. I will be very happy to hear any ideas. Thanks.
All the OOPS/panic information show the exception happened in udevd context, I think it may be triggered by "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl".
To verify this, you can remove the "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl" entry in your root file system to see whether the system can boot up successfully.
I guess your platform CPU is PowerPC architecture. If so, the exception vector is 0x700, it means the instruction fetch exception. CPU tried to fetch one instruction from invalid address. The instruction flow is incremented if there are no jump/branch instructions. If option 1 is verified that it is related to "/sbin/modprobe", please check the kernel module to analysis the instruction fetch exception.
Good luck!
I have created a asp.net application with framework 4.0.
It has a sort of interview screen with so many questions on it with the paging.
It may have 1000 pages with 10 questions in each page.
when users are using this application screen randomly gets frozen and asks for window credentials.
We also keep getting error for the same application in error log as below:
A process serving application pool 'appicationpoolname' suffered a
fatal communication error with the Windows Process Activation Service.
The process id was '4768'. The data field contains the error number.
Not sure if both above error are related or not.
I have created a rule in debugdiag tool for application pool crash and that has generated a dump file. The full call stack for the dump file is as below
Function Arg 1 Arg 2 Arg 3 Arg 4 Source
KERNELBASE!RaiseException+58 04242420 00000000 00000003 010ce504
clr!Debugger::SendRawEvent+9b 010ce530 0000002f 0000015c 000012a0
clr!Debugger::RaiseStartupNotification+48 6fa92811 00000001 01000000 00000000
clr!Debugger::Startup+73 6fa92bf9 00000001 01000000 00000000
clr!DoAdditionalPEChecks+e5 6fa92ac5 00000001 00000000 00000000
clr!EEStartupHelper+5f4 00000000 6fa92a9d 00000001 00000000
clr!EEStartup+52 00000000 6fa92a41 00000001 00000000
clr!EnsureEEStarted+c4 00000000 6fa92f9d 014bb2c0 00000000
clr!ClrCreateManagedInstance+41 708d4bb8 708d4ca8 010ceeb4 00000000
webengine4!LegacyActivationShim::ClrCreateManagedInstance+61 708d4bb8 708d4ca8 010ceeb4 00c6e268
webengine4!GetIsapiProcessHost+48 00c6e268 01c44e8c 6d8e0000 01c44e68
webengine!GetIsapiProcessHost+d0 00c6e268 01c44e8c 00b907d8 00000000
wbhst_pm+2e2a 01c4fed8 01c44e68 00000000 010cf56c
wbhst_pm+335f 00b907d8 01c4fed8 01c4fdf0 010cf5e4
wbhst_pm!GetProtocolManager+35 00b907d8 01c4fed8 00000000 00b90898
w3wphost!AppHostInitialize+235e 7534415b 00000000 00000000 00000000
w3wphost!AppHostInitialize+2698 010cf69c 708e199c 010cf618 00000001
w3wphost!IPM_MESSAGE_PIPE::operator=+1c03 010cf69c 708e199c 00000001 708f1274
iiscore!W3_SERVER::GetProtocolManagerCustomInterface+36 010cf69c 708e199c 00000001 708f1274
webengine4!InitClrHost+186 708f1274 00000000 014c8004 00000400
webengine4!CMgdEngGlobalModule::OnGlobalApplicationResolveModules+31 01c4efac 01c40330 010cf77c 7210a67a
iiscore!VIRTUAL_MODULE::GlobalDoWork+152 00000400 01c4efac 01c4efa8 01c4efa8
iiscore!W3_SERVER::GlobalNotify+98 00000400 01c4efac 00000000 72122f3a
iiscore!W3_APPLICATION::ResolveModules+22 014cc280 00000000 014cc284 00000001
iiscore!W3_APPLICATION::SetupNotificationContext+95 00000000 00000001 014cc284 014cc2e0
iiscore!W3_CONTEXT::SetupStateMachinePhase2+2ab 014cc280 014cc284 00000000 72103358
iiscore!W3_CONTEXT::SetupStateMachine+241 014cc284 014cc280 75341400 010cf894
iiscore!W3_MAIN_CONTEXT::StartNotificationLoop+3f 014cc284 00000000 00c712a8 014cb828
iiscore!W3_MAIN_CONTEXT::OnNewRequest+47 014cb828 014cb828 72a914e6 753413f0
w3dt!UL_NATIVE_REQUEST::DoStateProcess+26 753413f0 010cf8c0 72a9154c 00000444
w3dt!UL_NATIVE_REQUEST::DoWork+60 00000444 00000000 014cb82c 010cf8f8
w3dt!OverlappedCompletionRoutine+1a 00000000 00000444 014cb82c 00c712b0
w3tp!THREAD_POOL_DATA::ThreadPoolThread+89 00000000 00be7fd8 72cf0000 010cf924
w3tp!THREAD_POOL_DATA::ThreadPoolThread+24 00c712a8 00000000 00000000 00be7fd8
w3tp!THREAD_MANAGER::ThreadManagerThread+39 00be7fd8 010cf970 770f9ed2 00be7fd8
kernel32!BaseThreadInitThunk+12 00be7fd8 590a45de 00000000 00000000
ntdll!__RtlUserThreadStart+70 72cf1e5c 00be7fd8 ffffffff 771872ff
ntdll!_RtlUserThreadStart+1b 72cf1e5c 00be7fd8 00000000 00000000
Exception Information
In W3WP__~1.DMP the assembly instruction at KERNELBASE!RaiseException+58 in C:\Windows\SysWOW64\KERNELBASE.dll from Microsoft Corporation has caused an unknown exception (0x04242420) on thread 5
I am not sure how to use this to fix the issue. I tried using Winddbg with .loadby sos mscorwks
command but it says invalid module mscorwks
Any help would be appriciated.
Thanks,
MMC
Is this a first chance exception dump or a second chance exception dump ? When you created the rule in Debug Diagnostic tool, did you change something in the Unconfigured first chance exceptions section ?
Based on the stack it seems this is a first chance exception dump. Make sure that you change the Action time for un-configured first chance exceptions to NONE and then let the tool generate a second chance exception dump and that would be worth to look at.
If it generates one, analyze it using debugdiag and paste the stack and we can see why it is crashing