My kernel oopsed at kmem_cache_alloc with Oops number 5. I googled what that denotes, and found that it means 'Page protection fault during read access in user mode'. I am not able to find what page protection fault is. Is it the same as general protection fault?
[ 402.554964] Unable to handle kernel NULL pointer dereference at virtual address 00000001
[ 402.562995] pgd = c84ac000
[ 402.566132] [00000001] *pgd=00000000
[ 402.573958] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
[ 402.579209] Modules linked in: bcmdhd
[ 402.582923] CPU: 0 PID: 2507 Comm: MediaScannerSer Not tainted 3.10.10+ #1
[ 402.589703] task: c7eab480 ti: c23b0000 task.ti: c23b0000
[ 402.595036] PC is at kmem_cache_alloc+0x78/0x1c4
[ 402.599603] LR is at fat_parse_long+0x2e4/0x314
[ 402.604095] pc : [<c01031bc>] lr : [<c01f3f2c>] psr: 20000013
[ 402.604095] sp : c23b1bf8 ip : c23b1c38 fp : c23b1c34
[ 402.615400] r10: 007d9000 r9 : 00000000 r8 : c01f3f2c
[ 402.620579] r7 : 000000d0 r6 : ef001b80 r5 : c23b0000 r4 : 00000001
[ 402.637275] r3 : 00000000 r2 : c23b1c9c r1 : 000000d0 r0 : ef001b80
[ 402.643702] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 402.650719] Control: 10c5387d Table: 584ac06a DAC: 00000015
Thanks in advance.
[ 402.554964] Unable to handle kernel NULL pointer dereference at virtual address 00000001
Probably, you have a NULL pointer somewhere in your code. You have something like this:
my_variable->my_sub_structure->my_field
where my_sub_structure is NULL, so the code tries to reach my_field from an invalid memory address.
Related
printk is one of the core debug technique we use in Linux driver development. I have recently experience a weird problem. I read somewhere saying printk can be used in any context. However, when I use printk inside driver "read" function implementation, I see kernel error after it runs for a while. The error message is as following:
[ 113.017140] cdns-i2c e0005000.i2c: timeout waiting on completion
[ 113.023177] edt_ft5426 1-0038: edt_ft5426_ts_read: read error, addr=0x2 len=29.
[ 113.030585] Unable to handle kernel NULL pointer dereference at virtual address 00000043
[ 113.038688] pgd = c0004000
[ 113.041392] [00000043] *pgd=00000000
[ 113.044964] Internal error: Oops - BUG: 17 [#1] PREEMPT SMP ARM
[ 113.050872] Modules linked in: bfcore(O)
[ 113.054803] CPU: 1 PID: 690 Comm: irq/60-edt-ft54 Tainted: G O 4.14.0-xilinx #1
[ 113.058533] dma_desc_p[MIC_DMA].ready = 0
[ 113.058539] dma_desc_p[SMAP_DMA].ready = 0
[ 113.071478] Hardware name: Xilinx Zynq Platform
[ 113.076004] task: df7e86c0 task.stack: df6f6000
[ 113.080535] PC is at irq_finalize_oneshot+0x0/0xf0
[ 113.085318] LR is at irq_thread_fn+0x2c/0x34
[ 113.089577] pc : [<c0153b00>] lr : [<c0153c1c>] psr: 600f0113
[ 113.095835] sp : df6f7f48 ip : 00000000 fp : df77751c
[ 113.101051] r10: 00000000 r9 : 00000001 r8 : ded48624
[ 113.106268] r7 : c0ffffff r6 : ffffffff r5 : 00000001 r4 : ffffffff
[ 113.112787] r3 : 00000000 r2 : 00000000 r1 : ffffffff r0 : ffffffff
[ 113.119307] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 113.126433] Control: 18c5387d Table: 1ec0c04a DAC: 00000051
[ 113.132171] Process irq/60-edt-ft54 (pid: 690, stack limit = 0xdf6f6210)
[ 113.138862] Stack: (0xdf6f7f48 to 0xdf6f8000)
[ 113.143215] 7f40: df50b600 ffffe000 ded48600 c0153edc 00000000 c0153d04
[ 113.151383] 7f60: 00000000 df777500 df7e4980 df6f6000 00000000 ded48600 c0153d88 df43db70
[ 113.159553] 7f80: df77751c c01332c4 df6f6000 df7e4980 c0133194 00000000 00000000 00000000
[ 113.167727] 7fa0: 00000000 00000000 00000000 c0106fb0 00000000 00000000 00000000 00000000
[ 113.175894] 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 113.184062] 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 113.192243] [<c0153b00>] (irq_finalize_oneshot) from [<c0153d04>] (irq_thread_dtor+0x0/0x7c)
[ 113.196794] dma_desc_p[MIC_DMA].ready = 0
[ 113.196799] dma_desc_p[SMAP_DMA].ready = 0
[ 113.208925] [<c0153d04>] (irq_thread_dtor) from [<df6f6000>] (0xdf6f6000)
[ 113.215713] Code: 13a00001 e12fff1e e3a00000 e12fff1e (e5903044)
[ 113.221937] ---[ end trace 97dcf8cac1432a55 ]---
[ 113.226572] genirq: exiting task "irq/60-edt-ft54" (690) is an active IRQ thread (irq 60)
The setup is on a Xilinx Zynq 7020 device. I run a UI program which need touch screen as input device. So I need I2C for that. My driver responsible for reading data from FPGA. After it runs for a while, I got the error. In the log, I have:
[ 113.196794] dma_desc_p[MIC_DMA].ready = 0
[ 113.196799] dma_desc_p[SMAP_DMA].ready = 0
These are generated by my printk statements from the "read" function. If I comment away all the printk statements, the program runs ok. Any idea why there is a conflict between printk and I2C driver?
I'm trying to transform virtual address to physical address and map this physical address to virtual address with android linux kernel environment.
I can modify kernel code. So I tried next flow.
malloc() in android user space native binary not app
Transform va from malloc() to pa using the guide
Is there any API for determining the physical address from virtual address in Linux?
Pass pa to a system call function I made.
Re-map received pa to va in linux kernel space using ioremap()
Read value using readl() or ioread32()
But it's not working now.
The va to pa logic is in above link; in my native binary, below is the pseudo-code.
int main(){
char *va=malloc(100);
memset(va, "ttttt", ...)
uintptr_t paddr;
vir_to_phys_user(&paddr, getpid(), va);
syscall(sys_readpa, (unsigned long)paddr);
}
system call function
void sys_readpa(unsigned long pa){
void __iomem* mapped_add = ioremap(pa);
printk("%c", readl(mapped_add));
printk("%c", ioread32(mapped_add));
}
My code has similar logic:
I define va in user space and calculate pa from va.
I set va to "ttttt".
Pass pa to linux kernel space using syscall.
Remap this pa to va in kernel space.
Read va in kernel space and expect the value to be "ttttt"
I don't know the va to pa logic is correct. But it returns an address not failure.
But when syscall is called, kernel panic occur - e.g. "dereference for 0000000 address", and other kinds of errors. I checked pa in syscall is same with the one in user space.
My purpose of this try is study. I just wonder this implementation is possible if I can modify kernel code too but I met with an obstacle.
Please let me know what is problem or it's impossible? If needed, I'll update more detail code or specific error message.
I add detail errors and my debug log.
My user space log
: vitrual address : 0xf079c000
: 0xf079c000 -> 0xa4a8a000
I pass 0xa4a8a000 to syscall.
dmesg
[ 96.794448] accepted pa : 00000000a4a8a000
[ 96.794473] ------------[ cut here ]------------
[ 96.794500] WARNING: CPU: 6 PID: 11644 at arch/arm64/mm/ioremap.c:58 __ioremap_caller+0xc0/0xcc
[ 96.794519] Modules linked in:
[ 96.794552] CPU: 6 PID: 11644 Comm: mt Not tainted 4.14.113 #1
[ 96.794590] Call trace:
[ 96.794611] [<0000000000000000>] dump_backtrace+0x0/0x2b8
[ 96.794632] [<0000000000000000>] show_stack+0x18/0x24
[ 96.794655] [<0000000000000000>] dump_stack+0xa0/0xdc
[ 96.794676] [<0000000000000000>] __warn+0xbc/0x164
[ 96.794695] [<0000000000000000>] report_bug+0xac/0xdc
[ 96.794713] [<0000000000000000>] bug_handler+0x30/0x8c
[ 96.794732] [<0000000000000000>] brk_handler+0x94/0x150
[ 96.794751] [<0000000000000000>] do_debug_exception+0xd4/0x170
[ 96.794769] Exception stack(0xffffff8010fdbc10 to 0xffffff8010fdbd50)
[ 96.794787] bc00: 0000000000000000 0000000000000004
[ 96.794805] bc20: 00e8000000000f07 ffffff8008358714 000000000000000c 0000000000002d7c
[ 96.794822] bc40: ffffffc0119630e7 5b20205d38343434 0000000000000000 0000000000000001
[ 96.794839] bc60: 0000000000000001 00000000bab00000 0000000000000000 0000000080000000
[ 96.794856] bc80: ffffff800b18d000 0000000000000082 00000000000564c8 0000000000000074
[ 96.794873] bca0: 0000000000000074 00e8000000000f07 00000000a4a8a000 0000000000001000
[ 96.794890] bcc0: ffffff8008358714 0000000000000000 0000000000000011 000000000000018f
[ 96.794908] bce0: 000000000000018e ffffff8009316000 ffffffc8767edf80 ffffff8010fdbe80
[ 96.794926] bd00: ffffff80081fe124 ffffff8010fdbe50 ffffff80081fe188 0000000020400145
[ 96.794943] bd20: 0000000000000034 7cebe7b2cf849500 0000007fffffffff ffffff8009316000
[ 96.794961] bd40: ffffff8010fdbe80 ffffff80081fe188
[ 96.794978] [<0000000000000000>] el1_dbg+0x18/0x74
[ 96.794995] [<0000000000000000>] __ioremap_caller+0xc0/0xcc
[ 96.795014] [<0000000000000000>] __ioremap+0x10/0x1c
[ 96.795035] [<0000000000000000>] sys_readpa+0x78/0xfc
[ 96.795055] Exception stack(0xffffff8010fdbec0 to 0xffffff8010fdc000)
[ 96.795072] bec0: 00000000a4a8a000 0000000028bf4d08 0000000000000003 00000000f079c000
[ 96.795090] bee0: 0000000000000000 00000000a4a8a000 0000000000000000 000000000000018e
[ 96.795107] bf00: 00000000f09afd94 00000000f09d2b99 00000000ae6c9e84 00000000ae6a261e
[ 96.795124] bf20: 00000000ff921bf0 00000000ff921be0 00000000ae5f7b27 0000000000000000
[ 96.795142] bf40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 96.795159] bf60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 96.795176] bf80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 96.795195] bfa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 96.795212] bfc0: 00000000f091ce20 0000000060000010 00000000a4a8a000 000000000000018e
[ 96.795229] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 96.795247] [<0000000000000000>] __sys_trace_return+0x0/0x4
[ 96.795265] ---[ end trace 91e76f3be7c0b9bd ]---
[ 96.795418] ioremap return null
I found fix.
ioremap have a check logic for validation of address.
This function is for reserved address but it tring to map address that's already mapped to a process.
So, I modify the check logic in ioreamp and it works well.
I have taken rpi as a reference and trying to boot kernel to our custom board, modified the "dts" file by removing all peripherals except UART and Interrupt controller by changing the base address of the rpi board to our board specific address.
Seeting the env variables as follows,
$ setenv initrd 0xc3000000;setenv initrd_high 0xc4000000;setenv fdt_high 0xc1001000;setenv fdt_addr_r 0xc1000000
$ setenv bootargs earlyprintk console=ttyAMA0 mem=128M noinitrd root=/dev/mtdblock3 rw rootfstype=jffs2 rw init=/sbin/init
This is the error log we got....
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] Unable to handle kernel paging request at virtual address 48000fe0
[ 0.000000] pgd = c0004000
[ 0.000000] [48000fe0] *pgd=00000000
[ 0.000000] Internal error: Oops: 5 [#1] ARM
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.9.22+ #81
[ 0.000000] Hardware name: BCM2835
[ 0.000000] task: c0497f58 task.stack: c0494000
[ 0.000000] PC is at __vic_init+0x3c/0x178
[ 0.000000] LR is at 0x48000fe0
[ 0.000000] pc : [<c0473b74>] lr : [<48000fe0>] psr: a00000d3
[ 0.000000] sp : c0495f50 ip : 00000000 fp : c0495f7c
[ 0.000000] r10: 00000001 r9 : 410fb767 r8 : 48000000
[ 0.000000] r7 : 00000000 r6 : 00000000 r5 : ffffffff r4 : 00000000
[ 0.000000] r3 : 48000fe0 r2 : 00000000 r1 : 00000000 r0 : 00000000
[ 0.000000] Flags: NzCv IRQs off FIQs off Mode SVC_32 ISA ARM
Segment user
[ 0.000000] Control: 00c5387d Table: c0004008 DAC: 00000055
[ 0.000000] Process swapper (pid: 0, stack limit = 0xc0494188)
[ 0.000000] Stack: (0xc0495f50 to 0xc0496000)
Please help me,Thanks in advance.
Seems like the problem occured in interrupt context in procedure __vic_init. This procedure tried to access virtual address 48000fe0.
By the way - are you sure that this is complete calltrace taken from dmesg?
If this is only a snippet taken from dmesg - put the whole calltrace here.
If you have sources for this procedure __vic_init you can debug it - I mean find the number of the line in source file and do the further debug.
Having a hard time booting the tinycore linux kernel for an ARM A10 here, on boot the device crashes.
3.0.42 config found here: http://distro.ibiblio.org/tinycorelinux/5.x/armv7/Allwinner-A10/a10Core-kernel-3.0.42.config
U-Boot SPL 2013.01 (Feb 11 2013 - 15:19:28)
Board: mk802ii
DRAM: 1024MB
SUNXI SD/MMC: 0
U-Boot 2013.01 (Feb 11 2013 - 15:19:28) Allwinner Technology
CPU: SUNXI Family
Board: mk802ii
I2C: ready
DRAM: 1 GiB
MMC: SUNXI SD/MMC: 0
*** Warning - bad CRC, using default environment
In: serial
Out: serial
Err: serial
Net: wemac
Hit any key to stop autoboot: 0
reading uEnv.txt
117 bytes read in 2 ms (56.6 KiB/s)
Loaded environment from uEnv.txt
reading boot.scr
304 bytes read in 3 ms (98.6 KiB/s)
Jumping to boot.scr
## Executing script at 44000000
reading script.bin
42132 bytes read in 6 ms (6.7 MiB/s)
reading uImage
4109016 bytes read in 213 ms (18.4 MiB/s)
reading uCore
2951575 bytes read in 154 ms (18.3 MiB/s)
## Booting kernel from Legacy Image at 48000000 ...
Image Name: Linux-3.0.42
Created: 2015-02-16 20:40:40 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 4108952 Bytes = 3.9 MiB
Load Address: 40008000
Entry Point: 40008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 43100000 ...
Image Name: uCore for Allwinner A10
Created: 2014-12-26 21:12:42 UTC
Image Type: ARM Linux RAMDisk Image (uncompressed)
Data Size: 2951511 Bytes = 2.8 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...
<6>Initializing cgroup subsys cpuset
<5>Linux version 3.0.42 (root#localhost.localdomain) (gcc version 4.9.1 20140717 (Red Hat Cross 4.9.1-1) (GCC) ) #3 PREEMPT Mon Feb 16 15:40:29 EST 2015
CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: sun4i
<6>Memory Reserved:
<6> SYS : 0x43000000 - 0x4300ffff ( 64 kB)
<6> VE : 0x44000000 - 0x48ffffff ( 80 MB)
<6> G2D : 0x58000000 - 0x58ffffff ( 16 MB)
<6> LCD : 0x5a000000 - 0x5bffffff ( 32 MB)
Memory policy: ECC disabled, Data cache writeback
<6>BROM Ver: 1100 1100 1623
<6>chip-id: A10 (AW1623 revision C)
<7>On node 0 totalpages: 262144
<7>free_area_init_node: node 0, pgdat c07e60e4, node_mem_map c08ab000
<7> Normal zone: 1280 pages used for memmap
<7> Normal zone: 0 pages reserved
<7> Normal zone: 162560 pages, LIFO batch:31
<7> HighMem zone: 768 pages used for memmap
<7> HighMem zone: 97536 pages, LIFO batch:31
<7>pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
<7>pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 260096
<5>Kernel command line: console=tty0 init=/init rootwait panic=10 loglevel=3 dis p.screen0_output_mode=EDID:1280x720p60 hdmi.audio=EDID:0 nozswap nortc
<6>PID hash table entries: 4096 (order: 2, 16384 bytes)
<6>Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
<6>Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
<6>Memory: 1024MB = 1024MB total
<5>Memory: 896644k/896644k available, 151932k reserved, 393216K highmem
<5>Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
DMA : 0xffc00000 - 0xffe00000 ( 2 MB)
vmalloc : 0xe8800000 - 0xf0000000 ( 120 MB)
lowmem : 0xc0000000 - 0xe8000000 ( 640 MB)
pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
.init : 0xc0008000 - 0xc0035000 ( 180 kB)
.text : 0xc0035000 - 0xc07a3000 (7608 kB)
.data : 0xc07a4000 - 0xc07ef9f0 ( 303 kB)
.bss : 0xc07ef9f0 - 0xc08aa768 ( 748 kB)
<6>SLUB: Genslabs=11, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
<6>NR_IRQS:96 nr_irqs:96 96
<6>timer0: Periodic Mode
<6>Console: colour dummy device 80x30
<6>console [tty0] enabled
<6>Calibrating delay loop... <c>1001.88 BogoMIPS (lpj=5009408)
<6>pid_max: default: 32768 minimum: 301
<6>Mount-cache hash table entries: 512
<6>Initializing cgroup subsys cpuacct
<6>Initializing cgroup subsys devices
<6>Initializing cgroup subsys freezer
<6>Initializing cgroup subsys blkio
<6>CPU: Testing write buffer coherency: ok
<6>hw perfevents: enabled with ARMv7 Cortex-A8 PMU driver, 5 counters available
<6>devtmpfs: initialized
<6>print_constraints: dummy:
<6>NET: Registered protocol family 16
<6>hw-breakpoint: debug architecture 0x4 unsupported.
[ccmu] try to set apb1 parent to sata_pll failed!
SOFTWINNER DMA Driver, (c) 2003-2004,2006 Simtec Electronics
<6>Initialize DMAC OK
<6>bio: create slab <bio-0> at 0
<5>SCSI subsystem initialized
<7>libata version 3.00 loaded.
<6>usbcore: registered new interface driver usbfs
<6>usbcore: registered new interface driver hub
<6>usbcore: registered new device driver usb
<6>Advanced Linux Sound Architecture Driver Version 1.0.24.
<6>Bluetooth: Core ver 2.16
<6>NET: Registered protocol family 31
<6>Bluetooth: HCI device and connection manager initialized
<6>Bluetooth: HCI socket layer initialized
<6>Bluetooth: L2CAP socket layer initialized
<6>Bluetooth: SCO socket layer initialized
Init eGon pin module V2.0
<6>Switching to clocksource aw 64bits couter
<6>cfg80211: Calling CRDA to update world regulatory domain
<5>FS-Cache: Loaded
<6>CacheFiles: Loaded
<6>Switched to NOHz mode on CPU #0
<1>Unable to handle kernel NULL pointer dereference at virtual address 00000001
<1>pgd = c0004000
<1>[00000001] *pgd=00000000
<0>Internal error: Oops: 5 [#1] PREEMPT
<d>Modules linked in:
CPU: 0 Not tainted (3.0.42 #3)
PC is at kmem_cache_alloc+0x78/0xd0
LR is at con_insert_unipair+0xc0/0x10c
pc : [<c00f6e7c>] lr : [<c031508c>] psr: 60000093
sp : e783be88 ip : 0000025b fp : e783bea4
r10: c07c37d0 r9 : 00000001 r8 : 00000003
r7 : 00000003 r6 : 000000d0 r5 : e7802200 r4 : 00000001
r3 : 20000013 r2 : 00000000 r1 : c10b0040 r0 : 00000001
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c5387d Table: 40004019 DAC: 00000015
PC: 0xc00f6dfc:
6dfc eafffff2 c0596580 e1a0c00d e92dd878 e24cb004 e1a05000 e1a06001 e5953000
6e1c e593c004 e5934000 e3540000 0a000018 e10f3000 f10c0080 e5951000 e5910000
6e3c e1540000 0a000007 e3a00000 e121f003 e3500000 0afffff0 e3160902 1a000017
6e5c e1a00004 e89da878 e5912004 e15c0002 1afffff4 e5952014 e28cc001 e3a00001
6e7c e7942002 e5812000 e5952000 e582c004 eaffffed e1a01006 e1a0200e e1a00005
6e9c e7e067d6 ebffff2a e3500000 03a06000 12066001 e1a04000 e3560000 0affffe7
6ebc e5951010 e3510000 0affffe4 e1a00004 eb06e873 eaffffe1 e1a0c00d e92dd878
6edc e24cb004 e3074dc0 e1a06000 e34c4088 e1a05001 e5943054 e3530003 0a00000f
LR: 0xc031500c:
500c e0844a07 e3a00000 e1a03083 e18c70b3 e5963084 e0834004 e5864084 e89da8f0
502c e3073dd8 e34c3088 e593001c e3500000 0a00001c e3a010d0 ebf7876e e3500000
504c e7860105 0a00001e e2403004 e280e07c e3a0c000 e5a3c004 e153000e 1afffffc
506c eaffffe0 e3073dd8 e34c3088 e593001c e3500000 0a00000f e3a010d0 ebf7875d
508c e3500000 e1a0c000 e5850000 0a00000c e1a0000c e3a010ff e3a02080 ebfe6fcc
50ac e1a0c000 eaffffd4 e3a01010 e7861105 e1a00001 eaffffe3 e3a0c010 e585c000
50cc eafffff2 e3e0000b e89da8f0 e1a0c00d e92ddbf0 e24cb004 e2506000 089dabf0
50ec e5964098 e3540000 0a000024 e1a00004 e3a01b01 ebfe6fe6 e2466004 e3a07000
SP: 0xe783be08:
be08 e783c980 ffd23940 ffffffff 0a21fe80 e783be5c 0000040f 00000005 000000d0
be28 00000003 00000003 e783bea4 e783be40 c003b750 c0035214 00000001 c10b0040
be48 00000000 20000013 00000001 e7802200 000000d0 00000003 00000003 00000001
be68 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c 60000093 ffffffff
be88 c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8 c031508c c00f6e10
bea8 00000000 00000001 e786d780 00000003 e783bf04 e783bec8 c0315d70 c0314fd8
bec8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001 00000014 c08954d8
bee8 00000002 00000004 00000000 00000000 e783bf24 e783bf08 c001e220 c0315c88
FP: 0xe783be24:
be24 000000d0 00000003 00000003 e783bea4 e783be40 c003b750 c0035214 00000001
be44 c10b0040 00000000 20000013 00000001 e7802200 000000d0 00000003 00000003
be64 00000001 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c 60000093
be84 ffffffff c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8 c031508c
bea4 c00f6e10 00000000 00000001 e786d780 00000003 e783bf04 e783bec8 c0315d70
bec4 c0314fd8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001 00000014
bee4 c08954d8 00000002 00000004 00000000 00000000 e783bf24 e783bf08 c001e220
bf04 c0315c88 00000000 00000000 c08953b4 00000001 e783bf54 e783bf28 c001e734
R1: 0xc10affc0:
ffc0 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
ffe0 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0000 e78015e0 0000002e c0d9b020 00000000 e7802580 0000000b c0d9b040 00000000
0020 e786d840 00000299 c0d9bda0 00000000 e78bd880 000009d2 c0d9c7a0 00000000
0040 00000001 0000025a c0d9c780 00000000 e785c700 0000000d c0d9bb80 00000000
0060 e78bea00 000002d2 c0d9c7c0 00000000 e78a0400 0000000c c0d9c400 00000000
0080 e785a000 00000004 c0d9bb00 00000000 e7812000 0000025e c0d9b200 00000000
00a0 e780a000 00000001 c0d9b100 00000000 e7823130 00000291 c0d9b460 00000000
R5: 0xe7802180:
2180 c10b0030 00000000 00000006 00000040 00000040 00000000 00000040 00000040
21a0 00000040 00000000 00000004 00000000 00000040 00000040 00000000 e7800040
21c0 e7802140 e7802240 00000000 00000000 00000000 00000000 00000000 00000000
21e0 00000000 00000000 00000000 e7801060 ffffffff ffffffff ffffffff ffffffff
2200 c10b0040 00000000 00000007 00000080 00000080 00000000 00000020 00000020
2220 00000020 00000000 00000008 00000000 00000080 00000040 00000000 e7800080
2240 e78021c0 e78022c0 00000000 00000000 00000000 00000000 00000000 00000000
2260 00000000 00000000 00000000 e7801080 ffffffff ffffffff ffffffff ffffffff
R10: 0xc07c3750:
3750 f0c5f0c4 f0c7f0c6 f0c9f0c8 f0cbf0ca f0cdf0cc f0cff0ce f0d1f0d0 f0d3f0d2
3770 f0d5f0d4 f0d7f0d6 f0d9f0d8 f0dbf0da f0ddf0dc f0dff0de f0e1f0e0 f0e3f0e2
3790 f0e5f0e4 f0e7f0e6 f0e9f0e8 f0ebf0ea f0edf0ec f0eff0ee f0f1f0f0 f0f3f0f2
37b0 f0f5f0f4 f0f7f0f6 f0f9f0f8 f0fbf0fa f0fdf0fc f0fff0fe 263a0000 2665263b
37d0 25c62666 26602663 25d82022 25d925cb 26402642 266b266a 00a4263c 25ba25b6
37f0 25c425c0 203c2195 00a700b6 21a825ac 21932191 21902192 2194221f 25bc25b2
3810 00210020 00a80022 00240023 00260025 00b40027 00290028 002b002a 00b8002c
3830 00ad002d 002f002e 00310030 00330032 00350034 00370036 00390038 003b003a
<0>Process swapper (pid: 1, stack limit = 0xe783a2e8)
<0>Stack: (0xe783be88 to 0xe783c000)
<0>be80: c0887dd8 00002665 e78bc5e4 e786d780 e783bec4 e783bea8
<0>bea0: c031508c c00f6e10 00000000 00000001 e786d780 00000003 e783bf04 e783bec8
<0>bec0: c0315d70 c0314fd8 c08952b4 e7803200 c07c3a28 c07c37ce e783befc 00000001
<0>bee0: 00000014 c08954d8 00000002 00000004 00000000 00000000 e783bf24 e783bf08
<0>bf00: c001e220 c0315c88 00000000 00000000 c08953b4 00000001 e783bf54 e783bf28
<0>bf20: c001e734 c001e1e0 c0740fc8 e783bf38 c015d534 c0894ef8 00000000 c0627944
<0>bf40: 00000013 00000000 e783bf74 e783bf58 c001dcd8 c001e5ac c06f9560 e783bf7c
<0>bf60: 0000000c c0896500 e783bf9c e783bf78 c001f2f8 c001dbb0 c070e9d4 00000001
<0>bf80: 00000013 c07efa00 e783a020 00000001 e783bfe4 e783bfa0 c003549c c001f234
<0>bfa0: e783bfbc e783bfb0 00000000 c001f228 e783bfcc e783bfc0 c00ae8f4 c002d83c
<0>bfc0: c002de00 00000001 00000013 00000000 00000000 00000000 e783bff4 e783bfe8
<0>bfe0: c0008b44 c0035380 00000000 e783bff8 c006c9fc c0008a44 ffffffff ffffffff
Backtrace:
[<c00f6e04>] (kmem_cache_alloc+0x0/0xd0) from [<c031508c>] (con_insert_unipair+0 xc0/0x10c)
r6:e786d780 r5:e78bc5e4 r4:00002665 r3:c0887dd8
[<c0314fcc>] (con_insert_unipair+0x0/0x10c) from [<c0315d70>] (con_set_default_u nimap+0xf4/0x18c)
r7:00000003 r6:e786d780 r5:00000001 r4:00000000
[<c0315c7c>] (con_set_default_unimap+0x0/0x18c) from [<c001e220>] (console_map_i nit+0x4c/0x58)
[<c001e1d4>] (console_map_init+0x0/0x58) from [<c001e734>] (vty_init+0x194/0x1a4 )
r6:00000001 r5:c08953b4 r4:00000000 r3:00000000
[<c001e5a0>] (vty_init+0x0/0x1a4) from [<c001dcd8>] (tty_init+0x134/0x14c)
r8:00000000 r7:00000013 r6:c0627944 r5:00000000 r4:c0894ef8
[<c001dba4>] (tty_init+0x0/0x14c) from [<c001f2f8>] (chr_dev_init+0xd0/0xdc)
r5:c0896500 r4:0000000c
[<c001f228>] (chr_dev_init+0x0/0xdc) from [<c003549c>] (do_one_initcall+0x128/0x 180)
r6:00000001 r5:e783a020 r4:c07efa00
[<c0035374>] (do_one_initcall+0x0/0x180) from [<c0008b44>] (kernel_init+0x10c/0x 190)
[<c0008a38>] (kernel_init+0x0/0x190) from [<c006c9fc>] (do_exit+0x0/0x754)
<0>Code: 1afffff4 e5952014 e28cc001 e3a00001 (e7942002)
<4>---[ end trace 1871642cfdaefb45 ]---
<0>Kernel panic - not syncing: Attempted to kill init!
Backtrace:
[<c003f900>] (dump_backtrace+0x0/0x10c) from [<c058702c>] (dump_stack+0x18/0x1c)
r6:e783c000 r5:c07afc64 r4:c07f2bb0 r3:00000000
[<c0587014>] (dump_stack+0x0/0x1c) from [<c0587124>] (panic+0x78/0x188)
[<c05870ac>] (panic+0x0/0x188) from [<c006d150>] (complete_and_exit+0x0/0x24)
r3:00000000 r2:e783bbd0 r1:e783bbc8 r0:c0703720
r7:00000001
[<c006c9fc>] (do_exit+0x0/0x754) from [<c003fd8c>] (die+0x298/0x300)
r7:00000001
[<c003faf4>] (die+0x0/0x300) from [<c058709c>] (__do_kernel_fault.part.5+0x6c/0x 7c)
[<c0587030>] (__do_kernel_fault.part.5+0x0/0x7c) from [<c004465c>] (do_page_faul t+0x12c/0x3a8)
r7:00000001 r3:e783be40
[<c0044530>] (do_page_fault+0x0/0x3a8) from [<c0044a20>] (do_translation_fault+0 xa4/0xa8)
[<c004497c>] (do_translation_fault+0x0/0xa8) from [<c0035248>] (do_DataAbort+0x4 0/0xa0)
r6:c004497c r5:00000005 r4:c07aaa68 r3:60000093
[<c0035208>] (do_DataAbort+0x0/0xa0) from [<c003b750>] (__dabt_svc+0x70/0xa0)
Exception stack(0xe783be40 to 0xe783be88)
be40: 00000001 c10b0040 00000000 20000013 00000001 e7802200 000000d0 00000003
be60: 00000003 00000001 c07c37d0 e783bea4 0000025b e783be88 c031508c c00f6e7c
be80: 60000093 ffffffff
r8:00000003 r7:00000003 r6:000000d0 r5:00000005 r4:0000040f
[<c00f6e04>] (kmem_cache_alloc+0x0/0xd0) from [<c031508c>] (con_insert_unipair+0 xc0/0x10c)
r6:e786d780 r5:e78bc5e4 r4:00002665 r3:c0887dd8
[<c0314fcc>] (con_insert_unipair+0x0/0x10c) from [<c0315d70>] (con_set_default_u nimap+0xf4/0x18c)
r7:00000003 r6:e786d780 r5:00000001 r4:00000000
[<c0315c7c>] (con_set_default_unimap+0x0/0x18c) from [<c001e220>] (console_map_i nit+0x4c/0x58)
[<c001e1d4>] (console_map_init+0x0/0x58) from [<c001e734>] (vty_init+0x194/0x1a4 )
r6:00000001 r5:c08953b4 r4:00000000 r3:00000000
[<c001e5a0>] (vty_init+0x0/0x1a4) from [<c001dcd8>] (tty_init+0x134/0x14c)
r8:00000000 r7:00000013 r6:c0627944 r5:00000000 r4:c0894ef8
[<c001dba4>] (tty_init+0x0/0x14c) from [<c001f2f8>] (chr_dev_init+0xd0/0xdc)
r5:c0896500 r4:0000000c
[<c001f228>] (chr_dev_init+0x0/0xdc) from [<c003549c>] (do_one_initcall+0x128/0x 180)
r6:00000001 r5:e783a020 r4:c07efa00
[<c0035374>] (do_one_initcall+0x0/0x180) from [<c0008b44>] (kernel_init+0x10c/0x 190)
[<c0008a38>] (kernel_init+0x0/0x190) from [<c006c9fc>] (do_exit+0x0/0x754)
<0>Rebooting in 10 seconds..
Looks like infamous ARM memset() bug. Check this bugreport for solution: https://bugs.linaro.org/show_bug.cgi?id=928#c7
This is a kernel bug, which was fixed in year 2013 by following commits in mainline Linux repository:
455bd4c430b0c0a361f38e8658a0d6cb469942b5 ARM: 7668/1: fix memset-related crashes caused by recent GCC (4.7.2) optimizations
418df63adac56841ef6b0f1fcf435bc64d4ed177 ARM: 7670/1: fix the memset fix
The problem should be fixed by backporting those patches (in most cases patches could be applied without any modifications), or can be worked around with -fno-builtin-memset
I got the same kind of problem with yocto. By disabling the power management in the kernel, I was able to boot correctly. It's a workaround... I did not find the root cause yet.
I am trying to debug our embedded linux system under very low temperatures (<40C). The problem is that it does not always boot correctly and I am trying to figure out why. After some analysis I saw that the kernel goes into panic during the start-up with the following output:
can: controller area network core (rev 20090105 abi 8)
NET: Registered protocol family 29
can: raw protocol (rev 20090105)
/opt/elinos-5.1/linux/linux-ppc-2.6.34/drivers/rtc/hctosys.c: unable to open rtc
device (rtc0)
ADDRCONF(NETDEV_UP): eth0: link is not ready
IP-Config: Complete:
device=eth0, addr=192.168.100.100, mask=255.255.255.0, gw=255.255.255.255,
host=192.168.100.100, domain=, nis-domain=(none),
bootserver=192.168.100.20, rootserver=192.168.100.20, rootpath=
Freeing unused kernel memory: 156k init
init started: BusyBox v1.6.1 (2013-06-03 11:53:03 CEST) multi-call binary
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
udevd-work[84]: '/sbin/modprobe -bv of:NioctlT<NULL>Cfsl,mpc5125-ioctl' unexpect
ed exit with status 0x000b
------------[ cut here ]------------
Badness at /opt/elinos-5.1/linux/linux-ppc-2.6.34/kernel/sched.c:3574
NIP: c001acfc LR: c001ace4 CTR: c01c5fa4
REGS: c790f7c0 TRAP: 0700 Not tainted (2.6.34.7-ELinOS-146-ipipe)
MSR: 00021032 <ME,CE,IR,DR> CR: 28000482 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790f870 c3ba6cb0 00000001 c790f8b8 00000008 00000000 00000000
GPR08: 00000000 c0370000 00000001 c0370000 5d0fabd2 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 00000000 c649e840 00000000 00000901 00000000 00000000
NIP [c001acfc] add_preempt_count+0x48/0xac
LR [c001ace4] add_preempt_count+0x30/0xac
Call Trace:
Instruction dump:
409e0038 54290024 8009000c 2f800000 40bc0028 48133285 2f830000 419e0068
3d20c037 8009d298 2f800000 409e0058 <0fe00000> 48000050 3d20c037 8009d130
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: 00000000 LR: 00000000 CTR: c01c7778
REGS: c790f990 TRAP: 0400 Tainted: G W (2.6.34.7-ELinOS-146-ipipe)
MSR: 20009032 <EE,ME,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 00000000 c790fa40 c3ba6cb0 00000008 00000008 00000008 c7912804 c0348ba4
GPR08: 00000047 c78a5414 00000000 00000004 28000482 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: 100321f0 00000000 c790f618 00200200 00000000 c790fa34 00200200 00000000
NIP [00000000] (null)
LR [00000000] (null)
Call Trace:
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..
INFO: RCU detected CPU 0 stall (t=2500 jiffies)
INFO: RCU detected CPU 0 stall (t=10000 jiffies)
INFO: RCU detected CPU 0 stall (t=17500 jiffies)
INFO: RCU detected CPU 0 stall (t=25000 jiffies)
INFO: RCU detected CPU 0 stall (t=32500 jiffies)
INFO: RCU detected CPU 0 stall (t=40000 jiffies)
System Halted, OK to turn off power
------------[ cut here ]------------
kernel BUG at /opt/elinos-5.1/linux/linux-ppc-2.6.34/mm/vmalloc.c:1228!
Oops: Exception in kernel mode, sig: 5 [#2]
PREEMPT LTT NESTING LEVEL : 0
ORION
last sysfs file: /sys/devices/platform/gpio-keys-polled/input/input0/uevent
NIP: c009b0cc LR: c0013518 CTR: 00000000
REGS: c790f7c0 TRAP: 0700 Tainted: G D W (2.6.34.7-ELinOS-146-ipipe)
MSR: 00029032 <EE,ME,CE,IR,DR> CR: 28000484 XER: 20000000
TASK = c3ba6cb0[71] 'udevd' THREAD: c78e0000
GPR00: 078fe000 c790f870 c3ba6cb0 00001000 00000001 00000001 c9000000 fddff000
GPR08: ffffffff 000000d0 c001018c c790e000 48000488 10033420 10019a6c 00000000
GPR16: 10019328 100194d4 100194c0 1002bad0 10019328 10019474 bfa35428 100192fc
GPR24: c001018c 000000d0 00001000 ffffffff c9000000 fddff000 00000001 00000001
NIP [c009b0cc] __get_vm_area_node+0x68/0x204
LR [c0013518] __ioremap_caller+0x90/0x134
Call Trace:
Instruction dump:
7c9e2378 93e1003c 7cbf2b78 90010044 9261000c 92810010 92a10014 92c10018
92e1001c 93410028 800b000c 5400016e <0f000000> 70a00001 41820030 7c7e0034
Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
Rebooting in 180 seconds..:
Could anybody give me some hints how to approach the problem? What I want is to understand which component of the system (i.e. memory chip, etc.) causes this failure. I will be very happy to hear any ideas. Thanks.
All the OOPS/panic information show the exception happened in udevd context, I think it may be triggered by "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl".
To verify this, you can remove the "/sbin/modprobe -bv of:NioctlTCfsl,mpc5125-ioctl" entry in your root file system to see whether the system can boot up successfully.
I guess your platform CPU is PowerPC architecture. If so, the exception vector is 0x700, it means the instruction fetch exception. CPU tried to fetch one instruction from invalid address. The instruction flow is incremented if there are no jump/branch instructions. If option 1 is verified that it is related to "/sbin/modprobe", please check the kernel module to analysis the instruction fetch exception.
Good luck!