I have created a asp.net application with framework 4.0.
It has a sort of interview screen with so many questions on it with the paging.
It may have 1000 pages with 10 questions in each page.
when users are using this application screen randomly gets frozen and asks for window credentials.
We also keep getting error for the same application in error log as below:
A process serving application pool 'appicationpoolname' suffered a
fatal communication error with the Windows Process Activation Service.
The process id was '4768'. The data field contains the error number.
Not sure if both above error are related or not.
I have created a rule in debugdiag tool for application pool crash and that has generated a dump file. The full call stack for the dump file is as below
Function Arg 1 Arg 2 Arg 3 Arg 4 Source
KERNELBASE!RaiseException+58 04242420 00000000 00000003 010ce504
clr!Debugger::SendRawEvent+9b 010ce530 0000002f 0000015c 000012a0
clr!Debugger::RaiseStartupNotification+48 6fa92811 00000001 01000000 00000000
clr!Debugger::Startup+73 6fa92bf9 00000001 01000000 00000000
clr!DoAdditionalPEChecks+e5 6fa92ac5 00000001 00000000 00000000
clr!EEStartupHelper+5f4 00000000 6fa92a9d 00000001 00000000
clr!EEStartup+52 00000000 6fa92a41 00000001 00000000
clr!EnsureEEStarted+c4 00000000 6fa92f9d 014bb2c0 00000000
clr!ClrCreateManagedInstance+41 708d4bb8 708d4ca8 010ceeb4 00000000
webengine4!LegacyActivationShim::ClrCreateManagedInstance+61 708d4bb8 708d4ca8 010ceeb4 00c6e268
webengine4!GetIsapiProcessHost+48 00c6e268 01c44e8c 6d8e0000 01c44e68
webengine!GetIsapiProcessHost+d0 00c6e268 01c44e8c 00b907d8 00000000
wbhst_pm+2e2a 01c4fed8 01c44e68 00000000 010cf56c
wbhst_pm+335f 00b907d8 01c4fed8 01c4fdf0 010cf5e4
wbhst_pm!GetProtocolManager+35 00b907d8 01c4fed8 00000000 00b90898
w3wphost!AppHostInitialize+235e 7534415b 00000000 00000000 00000000
w3wphost!AppHostInitialize+2698 010cf69c 708e199c 010cf618 00000001
w3wphost!IPM_MESSAGE_PIPE::operator=+1c03 010cf69c 708e199c 00000001 708f1274
iiscore!W3_SERVER::GetProtocolManagerCustomInterface+36 010cf69c 708e199c 00000001 708f1274
webengine4!InitClrHost+186 708f1274 00000000 014c8004 00000400
webengine4!CMgdEngGlobalModule::OnGlobalApplicationResolveModules+31 01c4efac 01c40330 010cf77c 7210a67a
iiscore!VIRTUAL_MODULE::GlobalDoWork+152 00000400 01c4efac 01c4efa8 01c4efa8
iiscore!W3_SERVER::GlobalNotify+98 00000400 01c4efac 00000000 72122f3a
iiscore!W3_APPLICATION::ResolveModules+22 014cc280 00000000 014cc284 00000001
iiscore!W3_APPLICATION::SetupNotificationContext+95 00000000 00000001 014cc284 014cc2e0
iiscore!W3_CONTEXT::SetupStateMachinePhase2+2ab 014cc280 014cc284 00000000 72103358
iiscore!W3_CONTEXT::SetupStateMachine+241 014cc284 014cc280 75341400 010cf894
iiscore!W3_MAIN_CONTEXT::StartNotificationLoop+3f 014cc284 00000000 00c712a8 014cb828
iiscore!W3_MAIN_CONTEXT::OnNewRequest+47 014cb828 014cb828 72a914e6 753413f0
w3dt!UL_NATIVE_REQUEST::DoStateProcess+26 753413f0 010cf8c0 72a9154c 00000444
w3dt!UL_NATIVE_REQUEST::DoWork+60 00000444 00000000 014cb82c 010cf8f8
w3dt!OverlappedCompletionRoutine+1a 00000000 00000444 014cb82c 00c712b0
w3tp!THREAD_POOL_DATA::ThreadPoolThread+89 00000000 00be7fd8 72cf0000 010cf924
w3tp!THREAD_POOL_DATA::ThreadPoolThread+24 00c712a8 00000000 00000000 00be7fd8
w3tp!THREAD_MANAGER::ThreadManagerThread+39 00be7fd8 010cf970 770f9ed2 00be7fd8
kernel32!BaseThreadInitThunk+12 00be7fd8 590a45de 00000000 00000000
ntdll!__RtlUserThreadStart+70 72cf1e5c 00be7fd8 ffffffff 771872ff
ntdll!_RtlUserThreadStart+1b 72cf1e5c 00be7fd8 00000000 00000000
Exception Information
In W3WP__~1.DMP the assembly instruction at KERNELBASE!RaiseException+58 in C:\Windows\SysWOW64\KERNELBASE.dll from Microsoft Corporation has caused an unknown exception (0x04242420) on thread 5
I am not sure how to use this to fix the issue. I tried using Winddbg with .loadby sos mscorwks
command but it says invalid module mscorwks
Any help would be appriciated.
Thanks,
MMC
Is this a first chance exception dump or a second chance exception dump ? When you created the rule in Debug Diagnostic tool, did you change something in the Unconfigured first chance exceptions section ?
Based on the stack it seems this is a first chance exception dump. Make sure that you change the Action time for un-configured first chance exceptions to NONE and then let the tool generate a second chance exception dump and that would be worth to look at.
If it generates one, analyze it using debugdiag and paste the stack and we can see why it is crashing
Related
printk is one of the core debug technique we use in Linux driver development. I have recently experience a weird problem. I read somewhere saying printk can be used in any context. However, when I use printk inside driver "read" function implementation, I see kernel error after it runs for a while. The error message is as following:
[ 113.017140] cdns-i2c e0005000.i2c: timeout waiting on completion
[ 113.023177] edt_ft5426 1-0038: edt_ft5426_ts_read: read error, addr=0x2 len=29.
[ 113.030585] Unable to handle kernel NULL pointer dereference at virtual address 00000043
[ 113.038688] pgd = c0004000
[ 113.041392] [00000043] *pgd=00000000
[ 113.044964] Internal error: Oops - BUG: 17 [#1] PREEMPT SMP ARM
[ 113.050872] Modules linked in: bfcore(O)
[ 113.054803] CPU: 1 PID: 690 Comm: irq/60-edt-ft54 Tainted: G O 4.14.0-xilinx #1
[ 113.058533] dma_desc_p[MIC_DMA].ready = 0
[ 113.058539] dma_desc_p[SMAP_DMA].ready = 0
[ 113.071478] Hardware name: Xilinx Zynq Platform
[ 113.076004] task: df7e86c0 task.stack: df6f6000
[ 113.080535] PC is at irq_finalize_oneshot+0x0/0xf0
[ 113.085318] LR is at irq_thread_fn+0x2c/0x34
[ 113.089577] pc : [<c0153b00>] lr : [<c0153c1c>] psr: 600f0113
[ 113.095835] sp : df6f7f48 ip : 00000000 fp : df77751c
[ 113.101051] r10: 00000000 r9 : 00000001 r8 : ded48624
[ 113.106268] r7 : c0ffffff r6 : ffffffff r5 : 00000001 r4 : ffffffff
[ 113.112787] r3 : 00000000 r2 : 00000000 r1 : ffffffff r0 : ffffffff
[ 113.119307] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 113.126433] Control: 18c5387d Table: 1ec0c04a DAC: 00000051
[ 113.132171] Process irq/60-edt-ft54 (pid: 690, stack limit = 0xdf6f6210)
[ 113.138862] Stack: (0xdf6f7f48 to 0xdf6f8000)
[ 113.143215] 7f40: df50b600 ffffe000 ded48600 c0153edc 00000000 c0153d04
[ 113.151383] 7f60: 00000000 df777500 df7e4980 df6f6000 00000000 ded48600 c0153d88 df43db70
[ 113.159553] 7f80: df77751c c01332c4 df6f6000 df7e4980 c0133194 00000000 00000000 00000000
[ 113.167727] 7fa0: 00000000 00000000 00000000 c0106fb0 00000000 00000000 00000000 00000000
[ 113.175894] 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 113.184062] 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 113.192243] [<c0153b00>] (irq_finalize_oneshot) from [<c0153d04>] (irq_thread_dtor+0x0/0x7c)
[ 113.196794] dma_desc_p[MIC_DMA].ready = 0
[ 113.196799] dma_desc_p[SMAP_DMA].ready = 0
[ 113.208925] [<c0153d04>] (irq_thread_dtor) from [<df6f6000>] (0xdf6f6000)
[ 113.215713] Code: 13a00001 e12fff1e e3a00000 e12fff1e (e5903044)
[ 113.221937] ---[ end trace 97dcf8cac1432a55 ]---
[ 113.226572] genirq: exiting task "irq/60-edt-ft54" (690) is an active IRQ thread (irq 60)
The setup is on a Xilinx Zynq 7020 device. I run a UI program which need touch screen as input device. So I need I2C for that. My driver responsible for reading data from FPGA. After it runs for a while, I got the error. In the log, I have:
[ 113.196794] dma_desc_p[MIC_DMA].ready = 0
[ 113.196799] dma_desc_p[SMAP_DMA].ready = 0
These are generated by my printk statements from the "read" function. If I comment away all the printk statements, the program runs ok. Any idea why there is a conflict between printk and I2C driver?
I have to create a magic file that can detect a result of 42 on the 42nd byte.
I've created the following line to then compile
40 search 42 this is a 42 file
but when I run file -m <file_name> with this content
00000000 00000000 00000000 00000000 00042
I get the message
Warning: type `00000000 00000000 00000000 00042' invalid
file: could not find any valid magic files! (No such file or directory)
Your magic should be like this:
0 search/42 42 File containing "42"
!:mime text/x-42
Here's my test:
x#ubuntu:~$ cat testfile.txt
00000000 00000000 00000000 00000000 00042
x#ubuntu:~$ file -m magicfile.mgc testfile.txt
testfile.txt: File containing "42", ASCII text
x#ubuntu:~$
I am learning about the Linux user space memory layout on x86_64 systems and wanted to print some addresses from some of the sections. I used this Rust code:
fn main() {
let x = 3; // should be stored on stack
let s = "hello"; // should be in the .data section
println!("stack ≈ {:p}", &x);
println!(".text ≈ {:p}", main as *const ());
println!(".data ≈ {:p}", s);
use std::io;
let mut f = std::fs::File::open("/proc/self/maps").unwrap();
let out = io::stdout();
io::copy(&mut f, &mut out.lock()).unwrap();
}
This code also prints the file /proc/self/maps to stdout. I compiled this file mem.rs simply with rustc mem.rs. It printed:
stack ≈ 0x7ffffbf82f2c
.text ≈ 0x7f45b7c0a2b0
.data ≈ 0x7f45b7c4d35b
7f45b6800000-7f45b6c00000 rw-- 00000000 00:00 0
7f45b6de0000-7f45b6f9a000 r-x- 00000000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
7f45b6f9a000-7f45b6fa2000 ---- 001ba000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
[ ... more .so files]
7f45b7a22000-7f45b7a23000 r--- 00022000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f45b7a23000-7f45b7a24000 rw-- 00023000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f45b7a24000-7f45b7a25000 rw-- 00000000 00:00 0
7f45b7aa0000-7f45b7aa2000 rw-- 00000000 00:00 0
7f45b7ab0000-7f45b7ab2000 rw-- 00000000 00:00 0
7f45b7ac0000-7f45b7ac1000 rw-- 00000000 00:00 0
7f45b7ad0000-7f45b7ad1000 rw-- 00000000 00:00 0
7f45b7ae0000-7f45b7ae2000 rw-- 00000000 00:00 0
7f45b7c00000-7f45b7c5f000 r-x- 00000000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e5e000-7f45b7e62000 r--- 0005e000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e62000-7f45b7e63000 rw-- 00062000 00:00 1134580 /home/lukas/tmp/mem
7f45b7e63000-7f45b7e64000 rw-- 00000000 00:00 0
7ffffb784000-7ffffb785000 ---- 00000000 00:00 0 [stack]
7ffffb785000-7ffffbf84000 rw-- 00000000 00:00 0
7ffffc263000-7ffffc264000 r-x- 00000000 00:00 0 [vdso]
At least the addresses I printed on my own seem to match what maps says. But when I execute cat /proc/self/maps in the terminal, I get this output:
00400000-0040b000 r-x- 00000000 00:00 107117 /bin/cat
0060a000-0060b000 r--- 0000a000 00:00 107117 /bin/cat
0060b000-0060c000 rw-- 0000b000 00:00 107117 /bin/cat
0071c000-0073d000 rw-- 00000000 00:00 0 [heap]
7f7deb933000-7f7debc30000 r--- 00000000 00:00 758714 /usr/lib/locale/locale-archive
7f7debc30000-7f7debdea000 r-x- 00000000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
7f7debdea000-7f7debdf2000 ---- 001ba000 00:00 664435 /lib/x86_64-linux-gnu/libc-2.19.so
[ ... more .so files ...]
7f7dec222000-7f7dec223000 r--- 00022000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f7dec223000-7f7dec224000 rw-- 00023000 00:00 663920 /lib/x86_64-linux-gnu/ld-2.19.so
7f7dec224000-7f7dec225000 rw-- 00000000 00:00 0
7f7dec250000-7f7dec252000 rw-- 00000000 00:00 0
7f7dec260000-7f7dec261000 rw-- 00000000 00:00 0
7f7dec270000-7f7dec272000 rw-- 00000000 00:00 0
7ffff09e8000-7ffff11e8000 rw-- 00000000 00:00 0 [stack]
7ffff1689000-7ffff168a000 r-x- 00000000 00:00 0 [vdso]
The latter result matches everything I read about this topic: the sections from the executable are mapped in the lower end of the virtual address space (beginning around 0x400000).
I executed and compiled everything in the Linux Subsystem for Windows (Ubuntu 14.04 basically). I know, it's new and stuff, but I'm fairly sure this is not an issue with the subsystem (please tell me if it is, though!). Rust 1.14 is that matters (I doubt it),
I also tried the same with a C program (excuse my probably bad C):
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
int main(int argc, char **argv) {
FILE *test_file;
char buf[4096];
if ((test_file = fopen ("/proc/self/maps", "r")) != NULL) {
while (!feof (test_file)) {
fgets (buf, sizeof (buf), test_file);
puts (buf);
}
}
return 0;
}
It outputs something similar to cat:
00400000-00401000 r-x- 00000000 00:00 1325490 /home/lukas/tmp/a.out
00600000-00601000 r--- 00000000 00:00 1325490 /home/lukas/tmp/a.out
00601000-00602000 rw-- 00001000 00:00 1325490 /home/lukas/tmp/a.out
Why is the Rust executable mapped to large addresses near the stack?
Using rustc -Z print-link-args addr.rs, you can see what linker invocation the Rust compiler will use. Since the current linker happens to be cc, we can directly reuse these options for the C program. Ignoring unimportant arguments and removing others one-by-one, I was left with this compiler invocation:
gcc -fPIC -pie addr.c -o addr-c
Compiling the C code like this produces similar addresses as the Rust-compiled executable, indicating that one or both of those options is a likely culprit. This changes the question to "why does -fPIC and/or -pie map to such high addresses?"
I found another question and answer that seems to shed light on that:
The PIE binary is linked just as a shared library, and so its default load address (the .p_vaddr of the first LOAD segment) is zero. The expectation is that something will relocate this binary away from zero page, and load it at some random address.
Using readelf -e on the Rust executable, we can see that the first LOAD segment does have a virtual address of zero:
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x000000000005e6b4 0x000000000005e6b4 R E 200000
LOAD 0x000000000005ead0 0x000000000025ead0 0x000000000025ead0
0x00000000000039d1 0x00000000000049e8 RW 200000
I guess that this then changes the question to "why are these random addresses chosen", but I'm not sure of that answer. ^_^ A hunch tells me that ASLR comes into play. This other answer seems to bear that out:
PIE is to support ASLR in executable files.
ASLR is a security technique to help harden programs against certain types of attacks, so it makes sense that Rust, with its safety-minded approach, would attempt to enable something like this by default. Indeed, the addresses change a small bit each invocation:
root#97bcff9a925c:/# ./addr | grep 'r-xp' | grep 'addr'
5587cea9d000-5587ceafc000 r-xp 00000000 00:21 206 /addr
561d8aae2000-561d8ab41000 r-xp 00000000 00:21 206 /addr
555c30ffd000-555c3105c000 r-xp 00000000 00:21 206 /addr
55db249d5000-55db24a34000 r-xp 00000000 00:21 206 /addr
55e988572000-55e9885d1000 r-xp 00000000 00:21 206 /addr
560400e1b000-560400e7a000 r-xp 00000000 00:21 206 /addr
Having a hard time booting the tinycore linux kernel for an ARM A10 here, on boot the device crashes.
3.0.42 config found here: http://distro.ibiblio.org/tinycorelinux/5.x/armv7/Allwinner-A10/a10Core-kernel-3.0.42.config
U-Boot SPL 2013.01 (Feb 11 2013 - 15:19:28)
Board: mk802ii
DRAM: 1024MB
SUNXI SD/MMC: 0
U-Boot 2013.01 (Feb 11 2013 - 15:19:28) Allwinner Technology
CPU: SUNXI Family
Board: mk802ii
I2C: ready
DRAM: 1 GiB
MMC: SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment
In: serial
Out: serial
Err: serial
Net: wemac
Hit any key to stop autoboot: 0
reading uEnv.txt
117 bytes read in 2 ms (56.6 KiB/s)
Loaded environment from uEnv.txt
reading boot.scr
304 bytes read in 3 ms (98.6 KiB/s)
Jumping to boot.scr
## Executing script at 44000000
reading script.bin
42132 bytes read in 6 ms (6.7 MiB/s)
reading uImage
4109016 bytes read in 213 ms (18.4 MiB/s)
reading uCore
2951575 bytes read in 154 ms (18.3 MiB/s)
## Booting kernel from Legacy Image at 48000000 ...
Image Name: Linux-3.0.42
Created: 2015-02-16 20:40:40 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 4108952 Bytes = 3.9 MiB
Load Address: 40008000
Entry Point: 40008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 43100000 ...
Image Name: uCore for Allwinner A10
Created: 2014-12-26 21:12:42 UTC
Image Type: ARM Linux RAMDisk Image (uncompressed)
Data Size: 2951511 Bytes = 2.8 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...
<6>Initializing cgroup subsys cpuset
<5>Linux version 3.0.42 (root#localhost.localdomain) (gcc version 4.9.1 20140717 (Red Hat Cross 4.9.1-1) (GCC) ) #3 PREEMPT Mon Feb 16 15:40:29 EST 2015
CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: sun4i
<6>Memory Reserved:
<6> SYS : 0x43000000 - 0x4300ffff ( 64 kB)
<6> VE : 0x44000000 - 0x48ffffff ( 80 MB)
<6> G2D : 0x58000000 - 0x58ffffff ( 16 MB)
<6> LCD : 0x5a000000 - 0x5bffffff ( 32 MB)
Memory policy: ECC disabled, Data cache writeback
<6>BROM Ver: 1100 1100 1623
<6>chip-id: A10 (AW1623 revision C)
<7>On node 0 totalpages: 262144
<7>free_area_init_node: node 0, pgdat c07e60e4, node_mem_map c08ab000
<7> Normal zone: 1280 pages used for memmap
<7> Normal zone: 0 pages reserved
<7> Normal zone: 162560 pages, LIFO batch:31
<7> HighMem zone: 768 pages used for memmap
<7> HighMem zone: 97536 pages, LIFO batch:31
<7>pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
<7>pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 260096
<5>Kernel command line: console=tty0 init=/init rootwait panic=10 loglevel=3 dis p.screen0_output_mode=EDID:1280x720p60 hdmi.audio=EDID:0 nozswap nortc
<6>PID hash table entries: 4096 (order: 2, 16384 bytes)
<6>Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
<6>Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
<6>Memory: 1024MB = 1024MB total
<5>Memory: 896644k/896644k available, 151932k reserved, 393216K highmem
<5>Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
DMA : 0xffc00000 - 0xffe00000 ( 2 MB)
vmalloc : 0xe8800000 - 0xf0000000 ( 120 MB)
lowmem : 0xc0000000 - 0xe8000000 ( 640 MB)
pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
.init : 0xc0008000 - 0xc0035000 ( 180 kB)
.text : 0xc0035000 - 0xc07a3000 (7608 kB)
.data : 0xc07a4000 - 0xc07ef9f0 ( 303 kB)
.bss : 0xc07ef9f0 - 0xc08aa768 ( 748 kB)
<6>SLUB: Genslabs=11, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
<6>NR_IRQS:96 nr_irqs:96 96
<6>timer0: Periodic Mode
<6>Console: colour dummy device 80x30
<6>console [tty0] enabled
<6>Calibrating delay loop... <c>1001.88 BogoMIPS (lpj=5009408)
<6>pid_max: default: 32768 minimum: 301
<6>Mount-cache hash table entries: 512
<6>Initializing cgroup subsys cpuacct
<6>Initializing cgroup subsys devices
<6>Initializing cgroup subsys freezer
<6>Initializing cgroup subsys blkio
<6>CPU: Testing write buffer coherency: ok
<6>hw perfevents: enabled with ARMv7 Cortex-A8 PMU driver, 5 counters available
<6>devtmpfs: initialized
<6>print_constraints: dummy:
<6>NET: Registered protocol family 16
<6>hw-breakpoint: debug architecture 0x4 unsupported.
[ccmu] try to set apb1 parent to sata_pll failed!
SOFTWINNER DMA Driver, (c) 2003-2004,2006 Simtec Electronics
<6>Initialize DMAC OK
<6>bio: create slab <bio-0> at 0
<5>SCSI subsystem initialized
<7>libata version 3.00 loaded.
<6>usbcore: registered new interface driver usbfs
<6>usbcore: registered new interface driver hub
<6>usbcore: registered new device driver usb
<6>Advanced Linux Sound Architecture Driver Version 1.0.24.
<6>Bluetooth: Core ver 2.16
<6>NET: Registered protocol family 31
<6>Bluetooth: HCI device and connection manager initialized
<6>Bluetooth: HCI socket layer initialized
<6>Bluetooth: L2CAP socket layer initialized
<6>Bluetooth: SCO socket layer initialized
Init eGon pin module V2.0
<6>Switching to clocksource aw 64bits couter
<6>cfg80211: Calling CRDA to update world regulatory domain
<5>FS-Cache: Loaded
<6>CacheFiles: Loaded
<6>Switched to NOHz mode on CPU #0
<1>Unable to handle kernel NULL pointer dereference at virtual address 00000001
<1>pgd = c0004000
<1>[00000001] *pgd=00000000
<0>Internal error: Oops: 5 [#1] PREEMPT
<d>Modules linked in:
CPU: 0 Not tainted (3.0.42 #3)
PC is at kmem_cache_alloc+0x78/0xd0
LR is at con_insert_unipair+0xc0/0x10c
pc : [<c00f6e7c>] lr : [<c031508c>] psr: 60000093
sp : e783be88 ip : 0000025b fp : e783bea4
r10: c07c37d0 r9 : 00000001 r8 : 00000003
r7 : 00000003 r6 : 000000d0 r5 : e7802200 r4 : 00000001
r3 : 20000013 r2 : 00000000 r1 : c10b0040 r0 : 00000001
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c5387d Table: 40004019 DAC: 00000015
PC: 0xc00f6dfc:
6dfc eafffff2 c0596580 e1a0c00d e92dd878 e24cb004 e1a05000 e1a06001 e5953000
6e1c e593c004 e5934000 e3540000 0a000018 e10f3000 f10c0080 e5951000 e5910000
6e3c e1540000 0a000007 e3a00000 e121f003 e3500000 0afffff0 e3160902 1a000017
6e5c e1a00004 e89da878 e5912004 e15c0002 1afffff4 e5952014 e28cc001 e3a00001
6e7c e7942002 e5812000 e5952000 e582c004 eaffffed e1a01006 e1a0200e e1a00005
6e9c e7e067d6 ebffff2a e3500000 03a06000 12066001 e1a04000 e3560000 0affffe7
6ebc e5951010 e3510000 0affffe4 e1a00004 eb06e873 eaffffe1 e1a0c00d e92dd878
6edc e24cb004 e3074dc0 e1a06000 e34c4088 e1a05001 e5943054 e3530003 0a00000f
LR: 0xc031500c:
500c e0844a07 e3a00000 e1a03083 e18c70b3 e5963084 e0834004 e5864084 e89da8f0
502c e3073dd8 e34c3088 e593001c e3500000 0a00001c e3a010d0 ebf7876e e3500000
504c e7860105 0a00001e e2403004 e280e07c e3a0c000 e5a3c004 e153000e 1afffffc
506c eaffffe0 e3073dd8 e34c3088 e593001c e3500000 0a00000f e3a010d0 ebf7875d
508c e3500000 e1a0c000 e5850000 0a00000c e1a0000c e3a010ff e3a02080 ebfe6fcc
50ac e1a0c000 eaffffd4 e3a01010 e7861105 e1a00001 eaffffe3 e3a0c010 e585c000
50cc eafffff2 e3e0000b e89da8f0 e1a0c00d e92ddbf0 e24cb004 e2506000 089dabf0
50ec e5964098 e3540000 0a000024 e1a00004 e3a01b01 ebfe6fe6 e2466004 e3a07000
SP: 0xe783be08:
be08 e783c980 ffd23940 ffffffff 0a21fe80 e783be5c 0000040f 00000005 000000d0
be28 00000003 00000003 e783bea4 e783be40 c003b750 c0035214 00000001 c10b0040
be48 00000000 20000013 00000001 e7802200 000000d0 00000003 00000003 00000001
be68 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c 60000093 ffffffff
be88 c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8 c031508c c00f6e10
bea8 00000000 00000001 e786d780 00000003 e783bf04 e783bec8 c0315d70 c0314fd8
bec8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001 00000014 c08954d8
bee8 00000002 00000004 00000000 00000000 e783bf24 e783bf08 c001e220 c0315c88
FP: 0xe783be24:
be24 000000d0 00000003 00000003 e783bea4 e783be40 c003b750 c0035214 00000001
be44 c10b0040 00000000 20000013 00000001 e7802200 000000d0 00000003 00000003
be64 00000001 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c 60000093
be84 ffffffff c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8 c031508c
bea4 c00f6e10 00000000 00000001 e786d780 00000003 e783bf04 e783bec8 c0315d70
bec4 c0314fd8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001 00000014
bee4 c08954d8 00000002 00000004 00000000 00000000 e783bf24 e783bf08 c001e220
bf04 c0315c88 00000000 00000000 c08953b4 00000001 e783bf54 e783bf28 c001e734
R1: 0xc10affc0:
ffc0 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
ffe0 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0000 e78015e0 0000002e c0d9b020 00000000 e7802580 0000000b c0d9b040 00000000
0020 e786d840 00000299 c0d9bda0 00000000 e78bd880 000009d2 c0d9c7a0 00000000
0040 00000001 0000025a c0d9c780 00000000 e785c700 0000000d c0d9bb80 00000000
0060 e78bea00 000002d2 c0d9c7c0 00000000 e78a0400 0000000c c0d9c400 00000000
0080 e785a000 00000004 c0d9bb00 00000000 e7812000 0000025e c0d9b200 00000000
00a0 e780a000 00000001 c0d9b100 00000000 e7823130 00000291 c0d9b460 00000000
R5: 0xe7802180:
2180 c10b0030 00000000 00000006 00000040 00000040 00000000 00000040 00000040
21a0 00000040 00000000 00000004 00000000 00000040 00000040 00000000 e7800040
21c0 e7802140 e7802240 00000000 00000000 00000000 00000000 00000000 00000000
21e0 00000000 00000000 00000000 e7801060 ffffffff ffffffff ffffffff ffffffff
2200 c10b0040 00000000 00000007 00000080 00000080 00000000 00000020 00000020
2220 00000020 00000000 00000008 00000000 00000080 00000040 00000000 e7800080
2240 e78021c0 e78022c0 00000000 00000000 00000000 00000000 00000000 00000000
2260 00000000 00000000 00000000 e7801080 ffffffff ffffffff ffffffff ffffffff
R10: 0xc07c3750:
3750 f0c5f0c4 f0c7f0c6 f0c9f0c8 f0cbf0ca f0cdf0cc f0cff0ce f0d1f0d0 f0d3f0d2
3770 f0d5f0d4 f0d7f0d6 f0d9f0d8 f0dbf0da f0ddf0dc f0dff0de f0e1f0e0 f0e3f0e2
3790 f0e5f0e4 f0e7f0e6 f0e9f0e8 f0ebf0ea f0edf0ec f0eff0ee f0f1f0f0 f0f3f0f2
37b0 f0f5f0f4 f0f7f0f6 f0f9f0f8 f0fbf0fa f0fdf0fc f0fff0fe 263a0000 2665263b
37d0 25c62666 26602663 25d82022 25d925cb 26402642 266b266a 00a4263c 25ba25b6
37f0 25c425c0 203c2195 00a700b6 21a825ac 21932191 21902192 2194221f 25bc25b2
3810 00210020 00a80022 00240023 00260025 00b40027 00290028 002b002a 00b8002c
3830 00ad002d 002f002e 00310030 00330032 00350034 00370036 00390038 003b003a
<0>Process swapper (pid: 1, stack limit = 0xe783a2e8)
<0>Stack: (0xe783be88 to 0xe783c000)
<0>be80: c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8
<0>bea0: c031508c c00f6e10 00000000 00000001 e786d780 00000003 e783bf04 e783bec8
<0>bec0: c0315d70 c0314fd8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001
<0>bee0: 00000014 c08954d8 00000002 00000004 00000000 00000000 e783bf24 e783bf08
<0>bf00: c001e220 c0315c88 00000000 00000000 c08953b4 00000001 e783bf54 e783bf28
<0>bf20: c001e734 c001e1e0 c0740fc8 e783bf38 c015d534 c0894ef8 00000000 c0627944
<0>bf40: 00000013 00000000 e783bf74 e783bf58 c001dcd8 c001e5ac c06f9560 e783bf7c
<0>bf60: 0000000c c0896500 e783bf9c e783bf78 c001f2f8 c001dbb0 c070e9d4 00000001
<0>bf80: 00000013 c07efa00 e783a020 00000001 e783bfe4 e783bfa0 c003549c c001f234
<0>bfa0: e783bfbc e783bfb0 00000000 c001f228 e783bfcc e783bfc0 c00ae8f4 c002d83c
<0>bfc0: c002de00 00000001 00000013 00000000 00000000 00000000 e783bff4 e783bfe8
<0>bfe0: c0008b44 c0035380 00000000 e783bff8 c006c9fc c0008a44 ffffffff ffffffff
Backtrace:
[<c00f6e04>] (kmem_cache_alloc+0x0/0xd0) from [<c031508c>] (con_insert_unipair+0 xc0/0x10c)
r6:e786d780 r5:e78bc5e4 r4:00002665 r3:c0887dd8
[<c0314fcc>] (con_insert_unipair+0x0/0x10c) from [<c0315d70>] (con_set_default_u nimap+0xf4/0x18c)
r7:00000003 r6:e786d780 r5:00000001 r4:00000000
[<c0315c7c>] (con_set_default_unimap+0x0/0x18c) from [<c001e220>] (console_map_i nit+0x4c/0x58)
[<c001e1d4>] (console_map_init+0x0/0x58) from [<c001e734>] (vty_init+0x194/0x1a4 )
r6:00000001 r5:c08953b4 r4:00000000 r3:00000000
[<c001e5a0>] (vty_init+0x0/0x1a4) from [<c001dcd8>] (tty_init+0x134/0x14c)
r8:00000000 r7:00000013 r6:c0627944 r5:00000000 r4:c0894ef8
[<c001dba4>] (tty_init+0x0/0x14c) from [<c001f2f8>] (chr_dev_init+0xd0/0xdc)
r5:c0896500 r4:0000000c
[<c001f228>] (chr_dev_init+0x0/0xdc) from [<c003549c>] (do_one_initcall+0x128/0x 180)
r6:00000001 r5:e783a020 r4:c07efa00
[<c0035374>] (do_one_initcall+0x0/0x180) from [<c0008b44>] (kernel_init+0x10c/0x 190)
[<c0008a38>] (kernel_init+0x0/0x190) from [<c006c9fc>] (do_exit+0x0/0x754)
<0>Code: 1afffff4 e5952014 e28cc001 e3a00001 (e7942002)
<4>---[ end trace 1871642cfdaefb45 ]---
<0>Kernel panic - not syncing: Attempted to kill init!
Backtrace:
[<c003f900>] (dump_backtrace+0x0/0x10c) from [<c058702c>] (dump_stack+0x18/0x1c)
r6:e783c000 r5:c07afc64 r4:c07f2bb0 r3:00000000
[<c0587014>] (dump_stack+0x0/0x1c) from [<c0587124>] (panic+0x78/0x188)
[<c05870ac>] (panic+0x0/0x188) from [<c006d150>] (complete_and_exit+0x0/0x24)
r3:00000000 r2:e783bbd0 r1:e783bbc8 r0:c0703720
r7:00000001
[<c006c9fc>] (do_exit+0x0/0x754) from [<c003fd8c>] (die+0x298/0x300)
r7:00000001
[<c003faf4>] (die+0x0/0x300) from [<c058709c>] (__do_kernel_fault.part.5+0x6c/0x 7c)
[<c0587030>] (__do_kernel_fault.part.5+0x0/0x7c) from [<c004465c>] (do_page_faul t+0x12c/0x3a8)
r7:00000001 r3:e783be40
[<c0044530>] (do_page_fault+0x0/0x3a8) from [<c0044a20>] (do_translation_fault+0 xa4/0xa8)
[<c004497c>] (do_translation_fault+0x0/0xa8) from [<c0035248>] (do_DataAbort+0x4 0/0xa0)
r6:c004497c r5:00000005 r4:c07aaa68 r3:60000093
[<c0035208>] (do_DataAbort+0x0/0xa0) from [<c003b750>] (__dabt_svc+0x70/0xa0)
Exception stack(0xe783be40 to 0xe783be88)
be40: 00000001 c10b0040 00000000 20000013 00000001 e7802200 000000d0 00000003
be60: 00000003 00000001 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c
be80: 60000093 ffffffff
r8:00000003 r7:00000003 r6:000000d0 r5:00000005 r4:0000040f
[<c00f6e04>] (kmem_cache_alloc+0x0/0xd0) from [<c031508c>] (con_insert_unipair+0 xc0/0x10c)
r6:e786d780 r5:e78bc5e4 r4:00002665 r3:c0887dd8
[<c0314fcc>] (con_insert_unipair+0x0/0x10c) from [<c0315d70>] (con_set_default_u nimap+0xf4/0x18c)
r7:00000003 r6:e786d780 r5:00000001 r4:00000000
[<c0315c7c>] (con_set_default_unimap+0x0/0x18c) from [<c001e220>] (console_map_i nit+0x4c/0x58)
[<c001e1d4>] (console_map_init+0x0/0x58) from [<c001e734>] (vty_init+0x194/0x1a4 )
r6:00000001 r5:c08953b4 r4:00000000 r3:00000000
[<c001e5a0>] (vty_init+0x0/0x1a4) from [<c001dcd8>] (tty_init+0x134/0x14c)
r8:00000000 r7:00000013 r6:c0627944 r5:00000000 r4:c0894ef8
[<c001dba4>] (tty_init+0x0/0x14c) from [<c001f2f8>] (chr_dev_init+0xd0/0xdc)
r5:c0896500 r4:0000000c
[<c001f228>] (chr_dev_init+0x0/0xdc) from [<c003549c>] (do_one_initcall+0x128/0x 180)
r6:00000001 r5:e783a020 r4:c07efa00
[<c0035374>] (do_one_initcall+0x0/0x180) from [<c0008b44>] (kernel_init+0x10c/0x 190)
[<c0008a38>] (kernel_init+0x0/0x190) from [<c006c9fc>] (do_exit+0x0/0x754)
<0>Rebooting in 10 seconds..
Looks like infamous ARM memset() bug. Check this bugreport for solution: https://bugs.linaro.org/show_bug.cgi?id=928#c7
This is a kernel bug, which was fixed in year 2013 by following commits in mainline Linux repository:
455bd4c430b0c0a361f38e8658a0d6cb469942b5 ARM: 7668/1: fix memset-related crashes caused by recent GCC (4.7.2) optimizations
418df63adac56841ef6b0f1fcf435bc64d4ed177 ARM: 7670/1: fix the memset fix
The problem should be fixed by backporting those patches (in most cases patches could be applied without any modifications), or can be worked around with -fno-builtin-memset
I got the same kind of problem with yocto. By disabling the power management in the kernel, I was able to boot correctly. It's a workaround... I did not find the root cause yet.
I am trying to debug our embedded linux system under very low temperatures (<40C). The problem is that it does not always boot correctly and I am trying to figure out why. After some analysis I saw that the kernel goes into panic during the start-up with the following output:
can: controller area network core (rev 20090105 abi 8)
NET: Registered protocol family 29
can: raw protocol (rev 20090105)
/opt/elinos-5.1/linux/linux-ppc-2.6.34/drivers/rtc/hctosys.c: unable to open rtc
device (rtc0)
ADDRCONF(NETDEV_UP): eth0: link is not ready
IP-Config: Complete:
device=eth0, addr=192.168.100.100, mask=255.255.255.0, gw=255.255.255.255,
host=192.168.100.100, domain=, nis-domain=(none),
bootserver=192.168.100.20, rootserver=192.168.100.20, rootpath=
Freeing unused kernel memory: 156k init
init started: BusyBox v1.6.1 (2013-06-03 11:53:03 CEST) multi-call binary
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
udevd-work[84]: '/sbin/modprobe -bv of:NioctlT<NULL>Cfsl,mpc5125-ioctl' unexpect
ed exit with status 0x000b
------------[ cut here ]------------
Badness at /opt/elinos-5.1/linux/linux-ppc-2.6.34/kernel/sched.c:3574
NIP: c001acfc LR: c001ace4 CTR: c01c5fa4
REGS: c790f7c0 TRAP: 0700 Not tainted (2.6.34.7-ELinOS-146-ipipe)
MSR: 00021032 <ME,CE,IR,DR> CR: 28000482 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790f870 c3ba6cb0 00000001 c790f8b8 00000008 00000000 00000000
GPR08: 00000000 c0370000 00000001 c0370000 5d0fabd2 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 00000000 c649e840 00000000 00000901 00000000 00000000
NIP [c001acfc] add_preempt_count+0x48/0xac
LR [c001ace4] add_preempt_count+0x30/0xac
Call Trace:
Instruction dump:
409e0038 54290024 8009000c 2f800000 40bc0028 48133285 2f830000 419e0068
3d20c037 8009d298 2f800000 409e0058 <0fe00000> 48000050 3d20c037 8009d130
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: 00000000 LR: 00000000 CTR: c01c7778
REGS: c790f990 TRAP: 0400 Tainted: G W (2.6.34.7-ELinOS-146-ipipe)
MSR: 20009032 <EE,ME,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790fa40 c3ba6cb0 00000008 00000008 00000008 c7912804 c0348ba4
GPR08: 00000047 c78a5414 00000000 00000004 28000482 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 c790f618 00200200 00000000 c790fa34 00200200 00000000
NIP [00000000] (null)
LR [00000000] (null)
Call Trace:
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..
INFO: RCU detected CPU 0 stall (t=2500 jiffies)
INFO: RCU detected CPU 0 stall (t=10000 jiffies)
INFO: RCU detected CPU 0 stall (t=17500 jiffies)
INFO: RCU detected CPU 0 stall (t=25000 jiffies)
INFO: RCU detected CPU 0 stall (t=32500 jiffies)
INFO: RCU detected CPU 0 stall (t=40000 jiffies)
System Halted, OK to turn off power
------------[ cut here ]------------
kernel BUG at /opt/elinos-5.1/linux/linux-ppc-2.6.34/mm/vmalloc.c:1228!
Oops: Exception in kernel mode, sig: 5 [#2]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: c009b0cc LR: c0013518 CTR: 00000000
REGS: c790f7c0 TRAP: 0700 Tainted: G D W (2.6.34.7-ELinOS-146-ipipe)
MSR: 00029032 <EE,ME,CE,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 078fe000 c790f870 c3ba6cb0 00001000 00000001 00000001 c9000000 fddff000
GPR08: ffffffff 000000d0 c001018c c790e000 48000488 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: c001018c 000000d0 00001000 ffffffff c9000000 fddff000 00000001 00000001
NIP [c009b0cc] __get_vm_area_node+0x68/0x204
LR [c0013518] __ioremap_caller+0x90/0x134
Call Trace:
Instruction dump:
7c9e2378 93e1003c 7cbf2b78 90010044 9261000c 92810010 92a10014 92c10018
92e1001c 93410028 800b000c 5400016e <0f000000> 70a00001 41820030 7c7e0034
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..:
Could anybody give me some hints how to approach the problem? What I want is to understand which component of the system (i.e. memory chip, etc.) causes this failure. I will be very happy to hear any ideas. Thanks.
All the OOPS/panic information show the exception happened in udevd context, I think it may be triggered by "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl".
To verify this, you can remove the "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl" entry in your root file system to see whether the system can boot up successfully.
I guess your platform CPU is PowerPC architecture. If so, the exception vector is 0x700, it means the instruction fetch exception. CPU tried to fetch one instruction from invalid address. The instruction flow is incremented if there are no jump/branch instructions. If option 1 is verified that it is related to "/sbin/modprobe", please check the kernel module to analysis the instruction fetch exception.
Good luck!