Compiling driver for ARMv7 vs ARMv5 - linux

I've managed to compile a driver for an ARM based device, but the driver crashed when I try to load it.
here is the output from cpuinfo:
Processor : ARMv7 Processor rev 2 (v7l)
BogoMIPS : 999.42
Features : swp half thumb fastmult vfp edsp neon vfpv3
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x3
CPU part : 0xc08
CPU revision : 2
Here is the uname -r output
2.6.37
modinfo driver.ko
filename: cp210x.ko
description: Silicon Labs CP210x RS232 serial adaptor driver
license: GPL
vermagic: 2.6.37 mod_unload ARMv7
vermagic: 2.6.37 mod_unload modversions ARMv5
parm: debug:Enable verbose debugging messages
As you can I've added an extra vermagic (2.6.37 mod_unload ARMv7) so it will match the target system.
So if I understand this correct, I've compiled this module for an ARMv5 cpu, while the target is v7. Could this be the cause of the device driver crashing?
The device has this driver, but its embedded into an other driver package from the hw producer. This package also load some drivers that we cannot use. This driver package is not load, but I guess this indicate that this driver should work on this hardware some how.
here is the crash log
modprobe cp210x.ko
Unable to handle kernel NULL pointer dereference at virtual address 0000000a
pgd = ca1fc000
[0000000a] *pgd=870dd031, *pte=00000000, *ppte=00000000
Internal error: Oops: 17 [#1]
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in: dahdi_dummy dahdi cmemk syslink ipt_MASQUERADE nf_nat iptable_filter ip_tables ipt_LOG xt_state nf_conntrack_ftp nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 xt_recent xt_mac xt_limit work_led reset_button ipv6
CPU: 0 Not tainted (2.6.37 #1)
PC is at sys_init_module+0xfe0/0x1460
LR is at sys_init_module+0xe7c/0x1460
pc : [<c00836e8>] lr : [<c0083584>] psr: 20000013
sp : cc5e9ed0 ip : bf3828dc fp : cc5e8000
r10: bf385ca8 r9 : cf3bcb4e r8 : 000000c5
r7 : 00000027 r6 : bf382544 r5 : bf38266c r4 : bf385ca8
r3 : 00000000 r2 : c7c9f000 r1 : 0000000a r0 : 0000000a
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 8a1fc019 DAC: 00000015
Process modprobe (pid: 2676, stack limit = 0xcc5e82e8)
Stack: (0xcc5e9ed0 to 0xcc5ea000)
9ec0: bf382544 00000001 000ac048 bf382550
9ee0: 000000c5 cf3bd5a4 cf3b8000 000055f4 cf3bd20c cf3bd128 cf3bc2a0 c7c9f000
9f00: 0000266c 000028dc 00000000 00000000 00000017 00000018 00000010 0000000d
9f20: 00000009 00000000 6e72656b 00006c65 00000000 00000000 00000000 00000000
9f40: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
9f60: 00000000 00000000 c9a19540 00000000 ca2403c0 00000006 c9a19540 00000000
9f80: ca2403c0 000055f4 00000000 00000006 00000080 c0037c28 cc5e8000 00000000
9fa0: 00000001 c0037a80 000055f4 00000000 000ac998 000055f4 000ac048 000ac978
9fc0: 000055f4 00000000 00000006 00000080 000ac008 000ac028 000ac998 00000001
9fe0: bebaf968 bebaf958 00017764 40214740 60000010 000ac998 c1e38bcc 03de8ad9
[<c00836e8>] (sys_init_module+0xfe0/0x1460) from [<c0037a80>] (ret_fast_syscall+0x0/0x30)
Code: e7923103 e1a03133 e3130001 15963128 (17d33000)
---[ end trace 6e8943127db36208 ]---
Segmentation fault

I hade to change the cp210x.c file and comment out where there was any use of mutex. this was the only place:
static void cp210x_close(struct usb_serial_port *port)
{
dbg("%s - port %d", __func__, port->number);
usb_serial_generic_close(port);
/* mutex_lock(&port->serial->disc_mutex);*/
if (!port->serial->disconnected)
cp210x_set_config_single(port, CP210X_IFC_ENABLE, UART_DISABLE);
/* mutex_unlock(&port->serial->disc_mutex);*/
}

Are you trying to load a kernel module that was compiled for one kernel into another kernel? Linux modules (what you call drivers) are only supposed to be loaded into the kernel that they were compiled for. Even the same version of the kernel with different configuration or compiler settings could render the module incompatible. So playing with version magic is very dangerous.
The reason your driver is crashing is because it is trying to access kernel data structures using incorrect layout, so it is not actually reading the attributes it thinks it should be reading.
Changing architecture from ARMv7 to ARMv5 is very drastic configuration change that will completely change the memory layout of kernel data structures.
Unlike some other operating systems like Windows, Linux does not have an abstraction layer or fixed memory layouts that let you load the same loadable module into different versions of the kernel.

Related

Load binary into stm32f103c8t6 with OpenOCD and arm-none-eabi-gdb

I tried to load binary that is compiled from rust code, but it doesn't work.
First, I downloaded Rust code from https://github.com/rust-embedded/discovery.
Then, I built it.
# I am in the `src/05-led-roulette` directory
rustup target add thumbv7m-none-eabi
cargo build --target thumbv7m-none-eabi
It was successfully compiled.
After that, I successfully connected with stm32f103c8t6 using OpenOCD.
Then, I run this command.
arm-none-eabi-gdb -q target/thumbv7m-none-eabi/debug/led-roulette
But it seemed like it didn't finish reading.
Reading symbols from target/thumbv7m-none-eabi/debug/led-roulette...
(gdb)
(not done?!)
After that, I tried loadcommand, but it returned following sentences.
Start address 0x0, load size 0
Transfer rate: 0 bits in <1 sec.
I have no idea about why it doesn't work.
Please help me.
First see if your binary is good, then try telnet, then gdb. Rust also multiplies the odds of failure, so start with something simple:
so.s
.thumb
.globl _start
_start:
.word 0x20001000
.word reset
.thumb_func
reset:
ldr r0,some_addr
ldr r1,[r0]
add r1,r1,#1
str r1,[r0]
b .
.align
some_addr: .word 0x20000000
build it
arm-none-eabi-as so.s -o so.o
arm-none-eabi-ld -Ttext=0x08000000 so.o -o so.elf
arm-none-eabi-objdump -D so.elf
arm-none-eabi-objdump -D so.elf
so.elf: file format elf32-littlearm
Disassembly of section .text:
08000000 <_start>:
8000000: 20001000 andcs r1, r0, r0
8000004: 08000009 stmdaeq r0, {r0, r3}
08000008 :
8000008: 4802 ldr r0, [pc, #8] ; (8000014 <some_addr>)
800000a: 6801 ldr r1, [r0, #0]
800000c: 3101 adds r1, #1
800000e: 6001 str r1, [r0, #0]
8000010: e7fe b.n 8000010 <reset+0x8>
8000012: 46c0 nop ; (mov r8, r8)
08000014 <some_addr>:
8000014: 20000000 andcs r0, r0, r0
for small programs (Read the st documentation) this can be based at address 0x08000000 or 0x00000000 for this part. 0x08000000 is preferred. The vector table must be first in this case ignore the disassembly just look at the values
8000000: 20001000 andcs r1, r0, r0
8000004: 08000009 stmdaeq r0, {r0, r3}
The 0x08000009 is the reset address ORRed with one. so 0x08000008 | 1 is 0x08000009. So that will at least boot and try to fetch code without a fault.
This code simply reads the word at address 0x20000000 and increments it, sram is not affected by a reset so we can keep resetting and seeing that value increment.
using whatever configs you have and interface, I combine the openocd one for the st part into a single file and carry that with the project along with ones for the various interfaces (stlinks of different versions and jlink).
openocd -f jlink.cfg -f target.cfg
Open On-Chip Debugger 0.9.0 (2019-04-28-23:34)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
Info : JLink SWD mode enabled
swd
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
none separate
cortex_m reset_config sysresetreq
Info : J-Link ARM-OB STM32 compiled Jun 30 2009 11:14:15
Info : J-Link caps 0x88ea5833
Info : J-Link hw version 70000
Info : J-Link hw type J-Link
Info : J-Link max mem block 15344
Info : J-Link configuration
Info : USB-Address: 0x0
Info : Kickstart power on JTAG-pin 19: 0x0
Info : Vref = 3.300 TCK = 1 TDI = 1 TDO = 1 TMS = 1 SRST = 1 TRST = 1
Info : J-Link JTAG Interface ready
Info : clock speed 1000 kHz
Info : SWD IDCODE 0x1ba01477
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints
If you don't see the watchpoints line if it returns to the console, it didn't work.
In another window
telnet localhost 4444
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
>
Now let's stop the chip and write our program. The psr, pc, etc values may be different depending than mine depending on what you had running.
> reset halt
target state: halted
target halted due to debug-request, current mode: Thread
xPSR: 0x01000000 pc: 0x08000010 msp: 0x20001000
> flash write_image erase so.elf
auto erase enabled
device id = 0x20036410
flash size = 64kbytes
wrote 1024 bytes from file so.elf in 0.437883s (2.284 KiB/s)
Let's read it and see that it is there, should match the words from the disassembly
> mdw 0x08000000 20
0x08000000: 20001000 08000009 5000f04f 31016801 e7fe6001 ffffffff ffffffff ffffffff
0x08000020: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0x08000040: ffffffff ffffffff ffffffff ffffffff
Assume this is random garbage and that is fine so long as we see it increment.
> mdw 0x20000000
0x20000000: 2e006816
> reset
> halt
target state: halted
target halted due to debug-request, current mode: Thread
xPSR: 0x01000000 pc: 0x08000012 msp: 0x20001000
> mdw 0x20000000
0x20000000: 2e006817
So the value incremented if you do a reset, then do a halt (not a reset halt in one command) then dump that memory location it should keep incrementing every time.
Now you can choose to take the gdb path (I don't have a use for gdb so don't have one installed) with this binary or examine your rust binary by first examining the vector table to see it is correct, without at least the reset vector being correct then you will fault and not run any code on the processor. Can flash it using telnet or you can try gdb.
If gdb is having a problem with the file then perhaps you are using the wrong file. or the file is incorrectly built. did you try a simple program in that repository? can you make a minimal program from that repository, an empty entry function or an infinite loop or a counter that counts forever?
Is this truly a gdb problem? Is this an openocd problem? Is this a Rust tools problem? Is this a Rust binary problem? Is this a bug in the docs and you are pointing gdb at the wrong file problem? If the above works then openocd works, binutils at least works, the debugger/hardware works, it eliminates those and then becomes is this a rust thing, a gdb thing, using the wrong file thing, or something else?
After connecting openocd with the board don't forget to connect the debugger
arm-none-eabi-gdb with openocd.
> arm-none-eabi-gdb -se target/thumbv7em-none-eabi/release/your_binary
(gdb) target remote localhost:3333
If all is OK in the terminal console where openocd is running you will see the message:
accepting 'gdb' connection on tcp/3333`
and you should be able to start debugging.
To optimize connection setup you may create/update the .gdbinit file with the content:
target remote localhost:3333

ec2 Instance Status Checks failed

I am trying to backup 20GB mongoDB data from a running EC2 instance. I create a snapshot of EBS volumes. Create an AMI base on the snapshot and launch the AMI instance.
But the instance launch fails due to the status checks. It does pass the "System Status Checks" but not the "Instance Status Checks". Does anyone know how to fix it? Thanks very much.
**
Here is the system log:
**
Linux version 2.6.16-xenU (builder#xenbat.amazonsa) (gcc version 4.0.1 20050727 (Red Hat 4.0.1-5)) #14 SMP Mon May 28 03:41:48 SAST 2007
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000023530000 (usable)
0MB HIGHMEM available.
565MB LOWMEM available.
NX (Execute Disable) protection: active
Built 1 zonelists
Kernel command line: root=/dev/sda1 ro console=hvc0 4
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Xen reported: 1799.999 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB disabled
vmalloc area: e4000000-ff5fe000, maxmem 379fe000
Memory: 559744k/578752k available (1974k kernel code, 10400k reserved, 628k data, 156k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 3613.44 BogoMIPS (lpj=18067215)
Mount-cache hash table entries: 512
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 20480K
Checking 'hlt' instruction... OK.
Brought up 1 CPUs
migration_cost=0
Grant table initialized
NET: Registered protocol family 16
Brought up 1 CPUs
xen_mem: Initialising balloon driver.
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
i8042.c: No controller found.
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Xen virtual console successfully installed as tty1
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
Registering block device major 8
netfront: device eth0 has copying receive path.
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Using IPI No-Shortcut mode
XENBUS: Device with no driver: device/console/0
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
end_request: I/O error, dev sda1, sector 2
EXT3-fs: unable to read superblock
Unable to handle kernel NULL pointer dereference at virtual address 00000024
printing eip:
c0256ebd
0047a000 -> *pde = 00000000:24480067
00480000 -> *pme = 00000000:00000000
Oops: 0000 [#1]
SMP
Modules linked in:
CPU: 0
EIP: 0061:[<c0256ebd>] Not tainted VLI
EFLAGS: 00010097 (2.6.16-xenU #14)
EIP is at blkif_int+0x11d/0x270
eax: 00000000 ebx: 00000001 ecx: e2c5f0ac edx: 00000000
esi: 08010100 edi: 00000000 ebp: c038deb0 esp: c038de78
ds: 007b es: 007b ss: e021
Process swapper (pid: 0, threadinfo=c038c000 task=c0339400)
Stack: <0>c0352360 00000001 00000000 e2ae407c 00000000 e2c5f0ac 00000001 00000002
00000000 c34adc7c 00000001 e2aa75c0 00000000 00000000 c038ded8 c013f64f
00000105 e2ae4000 c038df40 c038df40 00000105 00008280 c0381680 00000105
Call Trace:
[<c010592d>] show_stack_log_lvl+0xcd/0x120
[<c0105b2b>] show_registers+0x1ab/0x240
[<c0105e31>] die+0x111/0x240
[<c0113157>] do_page_fault+0x707/0xc32
[<c0105307>] error_code+0x2b/0x30
[<c013f64f>] handle_IRQ_event+0x3f/0xd0
[<c013f76e>] __do_IRQ+0x8e/0xf0
[<c0106f0d>] do_IRQ+0x1d/0x30
[<c024c111>] evtchn_do_upcall+0xa1/0xe0
[<c0105349>] hypervisor_callback+0x3d/0x48
[<c01039ef>] xen_idle+0x2f/0x60
[<c0103a92>] cpu_idle+0x72/0xc0
[<c0102035>] rest_init+0x35/0x40
[<c038e56a>] start_kernel+0x2ea/0x3a0
[<c010006f>] 0xc010006f
Code: 89 82 84 00 00 00 8b 55 0c 89 b2 fc 13 00 00 8b 4d dc 0f bf 51 0a 0f b6 41 08 66 85 d2 0f 94 c3 3c 01 0f 87 38 ff ff ff 8b 55 d8 <8b> 42 24 89 5c 24 04 89 14 24 89 44 24 08 e8 60 f3 fb ff 85 c0
<0>Kernel panic - not syncing: Fatal exception in interrupt
Creating a snapshot from an EBS volume is possible, but as John Hanley pointed out, not guaranteed to succeed. Amazon has a doc with some specifics that you might want to go over: Creating an Amazon EBS-Backed Linux AMI.
The important part for you, under Creating a Linux AMI from a Snapshot, is this snippet:
If you have a snapshot of the root device volume of an instance, you can create an AMI from this snapshot using the AWS Management Console or the command line.
Important
[ ... ]
Similarly, although you can create a Windows AMI from a snapshot, you can't successfully launch an instance from the AMI.
In general, AWS advises against manually creating AMIs from snapshots.
If you want to take an exact replica of the machine you're on, the preferred way is to take an AMI of the instance:
Create an AMI from an Amazon EC2 Instance
You need to attach another private IP then stop and start instance follow this tutorial which fixed my Instance Status Checks failed issue: https://www.reluscloud.com/blog/troubleshooting-failed-ec2-status-checks

unexpected crash of embedded-linux on Zynq-Device (Zybo)

We have created, following the instruction on wiki-xilinx-linux web page, a system Zybo + embedded linux kernel (I want to remark that we didn't use PetaLinux but we have built all the file with the Xilinx tools).
When we turn on the system everything works fine and this is the output at the boot time:
U-Boot 2016.07-03720-g95e11f6-dirty (Oct 17 2016 - 02:56:48 -0700)
Model: Zynq ZYBO Development Board
Board: Xilinx Zynq
I2C: ready
DRAM: ECC disabled 512 MiB
MMC: sdhci#e0100000: 0
SF: Detected S25FL128S_64K with page size 256 Bytes, erase size 64 KiB, total 16 MiB
*** Warning - bad CRC, using default environment
In: serial#e0001000
Out: serial#e0001000
Err: serial#e0001000
Model: Zynq ZYBO Development Board
Board: Xilinx Zynq
Net: ZYNQ GEM: e000b000, phyaddr 0, interface rgmii-id
I2C EEPROM MAC address read failed
eth0: ethernet#e000b000
Hit any key to stop autoboot: 0
Device: sdhci#e0100000
Manufacturer ID: 3
OEM: 8368
Name: NCard
Tran Speed: 50000000
Rd Block Len: 512
SD version 3.0
High Capacity: Yes
Capacity: 3.7 GiB
Bus Width: 4-bit
Erase Group Size: 512 Bytes
reading uEnv.txt
** Unable to read file uEnv.txt **
Copying Linux from SD to RAM...
reading uImage
3843104 bytes read in 367 ms (10 MiB/s)
reading devicetree.dtb
10331 bytes read in 17 ms (592.8 KiB/s)
reading uramdisk.image.gz
5310018 bytes read in 504 ms (10 MiB/s)
## Booting kernel from Legacy Image at 02080000 ...
Image Name: Linux-4.6.0-xilinx-22285-g0e4e40
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3843040 Bytes = 3.7 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 04000000 ...
Image Name:
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 5309954 Bytes = 5.1 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
## Flattened Device Tree blob at 02000000
Booting using the fdt blob at 0x2000000
Loading Kernel Image ... OK
Loading Ramdisk to 1e601000, end 1eb11602 ... OK
Loading Device Tree to 1e5fb000, end 1e60085a ... OK
Starting kernel ...
Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0x0
Linux version 4.6.0-xilinx-22285-g0e4e407-dirty (leonardo#ubuntu) (gcc version 4.9.2 (Sourcery CodeBench Lite 2015.05-17) ) #1 SMP PREEMPT Mon Oct 17 03:54:44 PDT 2016
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine model: xlnx,zynq-7000
bootconsole [earlycon0] enabled
cma: Reserved 16 MiB at 0x1f000000
Memory policy: Data cache writealloc
percpu: Embedded 12 pages/cpu #debcc000 s19776 r8192 d21184 u49152
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048
Kernel command line: console=ttyPS0,115200 root=/dev/mmcblk0p2 rw earlyprintk rootfstype=ext4 rootwait devtmpfs.mount=1
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 489332K/524288K available (5304K kernel code, 230K rwdata, 1844K rodata, 1024K init, 222K bss, 18572K reserved, 16384K cma-reserved, 0K highmem)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xe0800000 - 0xff800000 ( 496 MB)
lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
.text : 0xc0008000 - 0xc07fb004 (8141 kB)
.init : 0xc0800000 - 0xc0900000 (1024 kB)
.data : 0xc0900000 - 0xc0939b60 ( 231 kB)
.bss : 0xc0939b60 - 0xc09714e8 ( 223 kB)
Preemptible hierarchical RCU implementation.
Build-time adjustment of leaf fanout to 32.
RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=2
NR_IRQS:16 nr_irqs:16 16
efuse mapped to e0800000
slcr mapped to e0802000
L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
L2C-310 erratum 769419 enabled
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
zynq_clock_init: clkc starts at e0802100
Zynq clock init
sched_clock: 64 bits at 325MHz, resolution 3ns, wraps every 4398046511103ns
clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x4af477f6aa, max_idle_ns: 440795207830 ns
Switching to timer-based delay loop, resolution 3ns
clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff, max_idle_ns: 551318127 ns
timer #0 at e080a000, irq=17
Console: colour dummy device 80x30
Calibrating delay loop (skipped), value calculated using timer frequency.. 650.00 BogoMIPS (lpj=3250000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x100000 - 0x100058
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
Brought up 2 CPUs
SMP: Total of 2 processors activated (1300.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor menu
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
zynq-ocm f800c000.ocmc: ZYNQ OCM pool: 256 KiB # 0xe0880000
zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
media: Linux media interface: v0.10
Linux video capture interface: v2.00
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti#linux.it>
PTP clock support registered
EDAC MC: Ver: 3.0.0
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm_global_timer
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (no cpio magic); looks like an initrd
Freeing initrd memory: 5188K (de601000 - deb12000)
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
futex hash table entries: 512 (order: 3, 32768 bytes)
workingset: timestamp_bits=28 max_order=17 bucket_order=0
jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
dma-pl330 f8003000.dmac: Loaded driver for PL330 DMAC-241330
dma-pl330 f8003000.dmac: DBUFF-128x8bytes Num_Chans-8 Num_Peri-4 Num_Events-16
e0001000.serial: ttyPS0 at MMIO 0xe0001000 (irq = 143, base_baud = 6250000) is a xuartps
�console [ttyPS0] enabled
console [ttyPS0] enabled
bootconsole [earlycon0] disabled
bootconsole [earlycon0] disabled
xdevcfg f8007000.devcfg: ioremap 0xf8007000 to e086c000
[drm] Initialized drm 1.1.0 20060810
brd: module loaded
loop: module loaded
CAN device driver interface
libphy: MACB_mii_bus: probed
macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 145 (00:0a:35:00:01:22)
RTL8211E Gigabit Ethernet e000b000.etherne:00: attached PHY driver [RTL8211E Gigabit Ethernet] (mii_bus:phy_addr=e000b000.etherne:00, irq=-1)
e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
usbcore: registered new interface driver usb-storage
mousedev: PS/2 mouse device common for all mice
i2c /dev entries driver
EDAC MC: ECC not enabled
Xilinx Zynq CpuIdle Driver started
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
mmc0: SDHCI controller on e0100000.sdhci [e0100000.sdhci] using DMA
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
NET: Registered protocol family 10
sit: IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
can: controller area network core (rev 20120528 abi 9)
NET: Registered protocol family 29
can: raw protocol (rev 20120528)
can: broadcast manager protocol (rev 20120528 t)
can: netlink gateway (rev 20130117) max_hops=1
Registering SWP/SWPB emulation handler
hctosys: unable to open rtc device (rtc0)
ALSA device list:
mmc0: new high speed SDHC card at address b368
No soundcards found.
mmcblk0: mmc0:b368 NCard 3.72 GiB
mmcblk0: p1 p2
RAMDISK: gzip image found at block 0
EXT4-fs warning (device ram0): ext4_update_dynamic_rev:746: updating to rev 1 because of new feature flag, running e2fsck is recommended
EXT4-fs (ram0): mounted filesystem without journal. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 1:0.
Starting rcS...
++ Mounting filesystem
mount: mounting /dev/mmcblk0p1 on /mnt failed: No such file or directory
mount: mounting /dev/mmcblk0 on /mnt failed: No such file or directory
++ Setting up mdev
++ Starting telnet daemon
++ Starting http daemon
++ Starting ftp daemon
++ Starting ssh daemon
random: sshd urandom read with 1 bits of entropy available
rcS Complete
zynq>
zynq>
zynq>
The problem is that, after some instructions, the system crash with this output:
zynq> EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 179:2.
Trying to move old root to /initrd ... /initrd does not exist. Ignored.
Unmounting old root
Trying to free ramdisk memory ... failed
devtmpfs: error mounting -2
Unable to handle kernel NULL pointer dereference at virtual address 00000020
pgd = c0004000
[00000020] *pgd=00000000
Internal error: Oops - BUG: 17 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0 PID: 10 Comm: migration/0 Not tainted 4.6.0-xilinx-22285-g0e4e407-dirty #1
Hardware name: Xilinx Zynq Platform
task: de060040 ti: de06a000 task.ti: de06a000
PC is at set_section_perms+0x8c/0x110
LR is at set_section_perms+0x18/0x110
pc : [<c0113404>] lr : [<c0113390>] psr: 80000093
sp : de06bec0 ip : dc4f3018 fp : 00000600
r10: 00000000 r9 : 00000000 r8 : 00000001
r7 : 00000003 r6 : c0902468 r5 : 00000000 r4 : c0908508
r3 : 00000000 r2 : c0000000 r1 : 00000003 r0 : 00000009
Flags: Nzcv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none
Control: 18c5387d Table: 1c4f804a DAC: 00000051
Process migration/0 (pid: 10, stack limit = 0xde06a210)
Stack: (0xde06bec0 to 0xde06c000)
bec0: c0908508 00000003 00000001 de0db780 de0db780 de0db998 00000003 c0908508
bee0: c09065c0 00000000 00000000 c01134f4 de04df70 00000001 a0000013 de04df84
bf00: 00000003 00000000 00000000 c011355c c011354c c017c420 de04df30 debccf64
bf20: de06a000 debccf68 00000000 c017c4f8 c017c478 de06a000 de02db40 c090cd94
bf40: 00000001 c0135454 de060040 de02d9c0 00000000 de02db40 c0135218 00000000
bf60: 00000000 c0132a40 4dff1176 00000001 00000000 de02db40 00000000 00030003
bf80: de06bf80 de06bf80 00000000 00000000 de06bf90 de06bf90 de06bfac de02d9c0
bfa0: c0132968 00000000 00000000 c0106ef8 00000000 00000000 00000000 00000000
bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 dfefe0f2 77e519c8
[<c0113404>] (set_section_perms) from [<c01134f4>] (update_sections_early+0x6c/0xc4)
[<c01134f4>] (update_sections_early) from [<c011355c>] (__fix_kernmem_perms+0x10/0x1c)
[<c011355c>] (__fix_kernmem_perms) from [<c017c420>] (multi_cpu_stop+0xb0/0x108)
[<c017c420>] (multi_cpu_stop) from [<c017c4f8>] (cpu_stopper_thread+0x80/0xb0)
[<c017c4f8>] (cpu_stopper_thread) from [<c0135454>] (smpboot_thread_fn+0x23c/0x254)
[<c0135454>] (smpboot_thread_fn) from [<c0132a40>] (kthread+0xd8/0xec)
[<c0132a40>] (kthread) from [<c0106ef8>] (ret_from_fork+0x14/0x3c)
Code: e1520003 2a00001b e5943014 e1a0baa2 (e599a020)
---[ end trace 07a0838423ee7708 ]---
note: migration/0[10] exited with preempt_count 2
It is clear that there is an error with the "mounted filesystem":
zynq> EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 179:2.
Trying to move old root to /initrd ... /initrd does not exist. Ignored.
Unmounting old root
Trying to free ramdisk memory ... failed
devtmpfs: error mounting -2
Why does this happen? Why before I can do everything with the system and, suddenly, I have a crash? Yes, the system crashes when I am doing nothing on it and not when I am working with it.
Are there some advice to solve this? to not to have anymore these crashes?
From the Linux Kernel Command line I can see "root=/dev/mmcblk0p2" and also the target is started with initial ramdisk image "uramdisk.image.gz". So the init process or scripts inside ramdisk image should be responsible to mount the real file system from SD i.e. from /dev/mmcblk0p2 and remove the used memory of ramdisk. To narrow down the problem, you need to analyze the steps taken in ramdisk image to mount the real filesystem and make sure you have the proper RFS flashed inside SD in /dev/mmcblk0p2. I can also seen some missing directories to mount the /dev/mmcblk0p1 and /dev/mmcblk0.
++ Mounting filesystem
mount: mounting /dev/mmcblk0p1 on /mnt failed: No such file or directory
mount: mounting /dev/mmcblk0 on /mnt failed: No such file or directory
In simple words, check the init scripts inside the ramdisk image and make sure the RFS is mounted properly.

Confusion regarding kernel version, device tree, and buildroot

I was provided with a 3.0.35 linux kernel for an arm i.mx6 board that uses the older "board file" format, as well as an older (2013.02) buildroot repo that has been used to generate the proper images and rootfs.
My goal is to use the latest buildroot public repo commit so that in the future it will be easier to keep packages up to date. Currently, I have to manually edit package files (OpenSSL, lighttpd, etc) to make the latest version available via buildroot.
I tried replicating my board configuration within the latest buildroot (post-2016.05). Everything remained more or less the same, except the toolchain had to change to support the newer systemd (Linaro 2013.01 -> CodeBench ARM 2014.05). Results:
Buildroot: build succeeds
U-boot: image works
Kernel: image causes a kernel panic
My questions really are:
Can I use my older kernel with the newer buildroot/toolchain, or is this an incompatibility? Do certain toolchains not support older kernels?
If it is an incompatiblity, is there an easier way to keep my older buildroot's packages up to date?
If both above are not options, is there any sort of tool to help migrate from a board file to device tree? Is it easy to convert to dts, or is this a long project?
EDIT:
Changed to Linaro 2014.09 toolchain. I was previously using systemd 202 as my init system. Since the newer buildroot requires a toolchain built against kernel headers >= 3.10 for systemd, I had to switch my init system to systemv. Here is the crash I'm receiving:
Starting kernel ...
Linux version 3.0.35 (eschumacher#ubuntu) (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) ) #1 SMP PREEMPT Tue Jun 21 11:01:40 PDT 2016
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: i.MX 6Quad/DualLite/Solo SMARC CPU Board
Memory policy: ECC disabled, Data cache writealloc
CPU identified as i.MX6DL/SOLO, silicon rev 1.1
PERCPU: Embedded 7 pages/cpu #8bc06000 s5440 r8192 d15040 u32768
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 84992
Kernel command line: console=ttymxc3,115200 root=/dev/mmcblk0p2 rootwait rw
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 336MB = 336MB total
Memory: 327580k/327580k available, 196708k reserved, 0K highmem
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
DMA : 0xf4600000 - 0xffe00000 ( 184 MB)
vmalloc : 0xa0800000 - 0xf2000000 (1304 MB)
lowmem : 0x80000000 - 0xa0000000 ( 512 MB)
pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
.init : 0x80008000 - 0x80039000 ( 196 kB)
.text : 0x80039000 - 0x80af9538 (11010 kB)
.data : 0x80afa000 - 0x80b53be0 ( 359 kB)
.bss : 0x80b53c04 - 0x80ba775c ( 335 kB)
SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
NR_IRQS:624
MXC GPIO hardware
sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 1431655ms
Set periph_clk's parent to pll2_pfd_400M!
arm_max_freq=800MHz
MXC_Early serial console at MMIO 0x21f0000 (options '115200')
bootconsole [ttymxc3] enabled
Console: colour dummy device 80x30
Calibrating delay loop... 1581.05 BogoMIPS (lpj=7905280)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 7 counters available
Brought up 1 CPUs
SMP: Total of 1 processors activated (1581.05 BogoMIPS).
devtmpfs: initialized
print_constraints: dummy:
NET: Registered protocol family 16
print_constraints: vddpu: 725 <--> 1300 mV at 700 mV fast normal
print_constraints: vddcore: 725 <--> 1300 mV at 1150 mV fast normal
print_constraints: vddsoc: 725 <--> 1300 mV at 1200 mV fast normal
print_constraints: vdd2p5: 2000 <--> 2775 mV at 2400 mV fast normal
print_constraints: vdd1p1: 800 <--> 1400 mV at 1100 mV fast normal
print_constraints: vdd3p0: 2625 <--> 3400 mV at 3000 mV fast normal
hw-breakpoint: found 6 breakpoint and 1 watchpoint registers.
hw-breakpoint: 1 breakpoint(s) reserved for watchpoint single-step.
hw-breakpoint: maximum watchpoint size is 4 bytes.
L310 cache controller enabled
l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x02050000, Cache size: 524288 B
bio: create slab <bio-0> at 0
mxs-dma mxs-dma-apbh: initialized
print_constraints: vmmc: 3300 mV
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Freescale USB OTG Driver loaded, $Revision: 1.55 $
imx-ipuv3 imx-ipuv3.0: IPU DMFC NORMAL mode: 1(0~1), 5B(4,5), 5F(6,7)
MIPI CSI2 driver module loaded
Advanced Linux Sound Architecture Driver Version 1.0.24.
Bluetooth: Core ver 2.16
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP socket layer initialized
Bluetooth: SCO socket layer initialized
i2c-core: driver [max17135] using legacy suspend method
i2c-core: driver [max17135] using legacy resume method
Switching to clocksource mxc_timer1
cfg80211: Calling CRDA to update world regulatory domain
Unable to handle kernel NULL pointer dereference at virtual address 00000040
pgd = 80004000
[00000040] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 Not tainted (3.0.35 #1)
PC is at kmem_cache_alloc+0xa4/0x108
LR is at con_insert_unipair+0xb8/0x104
pc : [<800f40c0>] lr : [<80299574>] psr: 60000093
sp : 9202fef0 ip : 9202e000 fp : 00000001
r10: 00000000 r9 : 0bbcf000 r8 : 20000013
r7 : 000000d0 r6 : 92002300 r5 : 00000040 r4 : 80b0fcc8
r3 : 8003a5a0 r2 : 80b0fcc8 r1 : 000024ac r0 : 00000001
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c53c7d Table: 1000404a DAC: 00000015
Process swapper (pid: 1, stack limit = 0x9202e2f0)
Stack: (0x9202fef0 to 0x92030000)
fee0: 00002665 921bcee4 920f4900 00000003
ff00: 80b2fee8 80b2fee6 00000003 80299574 00000000 00000001 920f4900 00000003
ff20: 80b2fee8 8029a1a4 00000000 80b30143 92003000 80b729d8 00000136 00000001
ff40: 00000014 80b72bfc 00000001 00000002 00000004 00000000 8001fce4 8001f214
ff60: 00000000 80b72ad8 00000001 8001f714 80a695d4 00000000 80b725d4 00000000
ff80: 8003fb04 00000013 00000000 00000000 00000000 8001eb68 80a37670 00000000
ffa0: 80b53c20 80b53c20 9202e030 800395c0 000001df 00000013 00000000 00000000
ffc0: 00000039 00000000 00000270 8002fb14 8002ffa0 8003fb04 00000013 00000000
ffe0: 00000000 00000000 00000000 8000898c 00000000 8003fb04 c38ac389 c38dc38b
[<800f40c0>] (kmem_cache_alloc+0xa4/0x108) from [<80299574>] (con_insert_unipair+0xb8/0x104)
[<80299574>] (con_insert_unipair+0xb8/0x104) from [<8029a1a4>] (con_set_default_unimap+0xe8/0x178)
[<8029a1a4>] (con_set_default_unimap+0xe8/0x178) from [<8001f214>] (console_map_init+0x44/0x50)
[<8001f214>] (console_map_init+0x44/0x50) from [<8001f714>] (vty_init+0x18c/0x19c)
[<8001f714>] (vty_init+0x18c/0x19c) from [<8001eb68>] (tty_init+0x12c/0x144)
[<8001eb68>] (tty_init+0x12c/0x144) from [<800395c0>] (do_one_initcall+0x114/0x16c)
[<800395c0>] (do_one_initcall+0x114/0x16c) from [<8000898c>] (kernel_init+0xc0/0x144)
[<8000898c>] (kernel_init+0xc0/0x144) from [<8003fb04>] (kernel_thread_exit+0x0/0x8)
Code: 1afffff3 e596a014 e2811004 e3a00001 (e795a00a)
---[ end trace 1b75b31a2719ed1c ]---
Kernel panic - not syncing: Attempted to kill init!
[<8004631c>] (unwind_backtrace+0x0/0xec) from [<80569100>] (panic+0x80/0x19c)
[<80569100>] (panic+0x80/0x19c) from [<80075934>] (complete_and_exit+0x0/0x1c)
[<80075934>] (complete_and_exit+0x0/0x1c) from [<00000001>] (0x1)
EDIT 2:
I changed buildroot config to build my own toolchain using my kernel headers. I still receive a similar kernel panic. Output:
Starting kernel ...
Linux version 3.0.35 (eschumacher#ubuntu) (gcc version 4.9.3 (Buildroot 2016.08- git-00508-geba9c44-dirty) ) #1 SMP PREEMPT Tue Jun 21 16:41:35 PDT 2016
CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c53c7d
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: i.MX 6Quad/DualLite/Solo SMARC CPU Board
Memory policy: ECC disabled, Data cache writealloc
CPU identified as i.MX6DL/SOLO, silicon rev 1.1
PERCPU: Embedded 7 pages/cpu #8bc06000 s5440 r8192 d15040 u32768
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 84992
Kernel command line: console=ttymxc3,115200 root=/dev/mmcblk0p2 rootwait rw
PID hash table entries: 2048 (order: 1, 8192 bytes)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 336MB = 336MB total
Memory: 327564k/327564k available, 196724k reserved, 0K highmem
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
DMA : 0xf4600000 - 0xffe00000 ( 184 MB)
vmalloc : 0xa0800000 - 0xf2000000 (1304 MB)
lowmem : 0x80000000 - 0xa0000000 ( 512 MB)
pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
.init : 0x80008000 - 0x80039000 ( 196 kB)
.text : 0x80039000 - 0x80afd458 (11026 kB)
.data : 0x80afe000 - 0x80b57bc0 ( 359 kB)
.bss : 0x80b57be4 - 0x80bab73c ( 335 kB)
SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
NR_IRQS:624
MXC GPIO hardware
sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 1431655ms
Set periph_clk's parent to pll2_pfd_400M!
arm_max_freq=800MHz
MXC_Early serial console at MMIO 0x21f0000 (options '115200')
bootconsole [ttymxc3] enabled
Console: colour dummy device 80x30
Calibrating delay loop... 1581.05 BogoMIPS (lpj=7905280)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 7 counters available
Brought up 1 CPUs
SMP: Total of 1 processors activated (1581.05 BogoMIPS).
devtmpfs: initialized
print_constraints: dummy:
NET: Registered protocol family 16
print_constraints: vddpu: 725 <--> 1300 mV at 700 mV fast normal
print_constraints: vddcore: 725 <--> 1300 mV at 1150 mV fast normal
print_constraints: vddsoc: 725 <--> 1300 mV at 1200 mV fast normal
print_constraints: vdd2p5: 2000 <--> 2775 mV at 2400 mV fast normal
print_constraints: vdd1p1: 800 <--> 1400 mV at 1100 mV fast normal
print_constraints: vdd3p0: 2625 <--> 3400 mV at 3000 mV fast normal
hw-breakpoint: found 6 breakpoint and 1 watchpoint registers.
hw-breakpoint: 1 breakpoint(s) reserved for watchpoint single-step.
hw-breakpoint: maximum watchpoint size is 4 bytes.
L310 cache controller enabled
l2x0: 16 ways, CACHE_ID 0x410000c8, AUX_CTRL 0x02050000, Cache size: 524288 B
bio: create slab <bio-0> at 0
mxs-dma mxs-dma-apbh: initialized
print_constraints: vmmc: 3300 mV
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Freescale USB OTG Driver loaded, $Revision: 1.55 $
imx-ipuv3 imx-ipuv3.0: IPU DMFC NORMAL mode: 1(0~1), 5B(4,5), 5F(6,7)
MIPI CSI2 driver module loaded
Advanced Linux Sound Architecture Driver Version 1.0.24.
Bluetooth: Core ver 2.16
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP socket layer initialized
Bluetooth: SCO socket layer initialized
i2c-core: driver [max17135] using legacy suspend method
i2c-core: driver [max17135] using legacy resume method
Switching to clocksource mxc_timer1
cfg80211: Calling CRDA to update world regulatory domain
Unable to handle kernel NULL pointer dereference at virtual address 00000040
pgd = 80004000
[00000040] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 Not tainted (3.0.35 #1)
PC is at kmem_cache_alloc+0xa4/0x108
LR is at con_insert_unipair+0xb8/0x104
pc : [<800f3bbc>] lr : [<80298a88>] psr: 60000093
sp : 9202fef8 ip : 9202e000 fp : 00000001
r10: 00000000 r9 : 0bbcf000 r8 : 20000013
r7 : 000000d0 r6 : 92002300 r5 : 00000040 r4 : 80b13cc8
r3 : 8003a5a0 r2 : 80b13cc8 r1 : 000024ac r0 : 00000001
Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c53c7d Table: 1000404a DAC: 00000015
Process swapper (pid: 1, stack limit = 0x9202e2f0)
Stack: (0x9202fef8 to 0x92030000)
fee0: 00002665 921bcee4
ff00: 920f4900 00000003 80b33ec8 80b33ec6 00000003 80298a88 80b34123 00000000
ff20: 00000001 920f4900 00000003 802996b8 00000001 80b34123 92003000 80b769b8
ff40: 8001fc40 00000001 00000014 80b76bdc 00000002 00000004 00000000 00000000
ff60: 8001fc40 8001f168 00000000 80b76ab8 00000001 8001f668 80a684a4 8014768c
ff80: 80b765b4 00000000 8003fac4 00000013 00000000 8001eac4 80a36540 00000000
ffa0: 80b57c00 80b57c00 9202e030 800395b8 000001df 00000013 00000000 00000000
ffc0: 00000039 00000000 00000270 8002fa74 8002ff00 8003fac4 00000013 00000000
ffe0: 00000000 00000000 00000000 8000898c 00000000 8003fac4 e345c1eb e0c32c90
[<800f3bbc>] (kmem_cache_alloc+0xa4/0x108) from [<80298a88>] (con_insert_unipair +0xb8/0x104)
[<80298a88>] (con_insert_unipair+0xb8/0x104) from [<802996b8>] (con_set_default_ unimap+0xe8/0x178)
[<802996b8>] (con_set_default_unimap+0xe8/0x178) from [<8001f168>] (console_map_ init+0x44/0x50)
[<8001f168>] (console_map_init+0x44/0x50) from [<8001f668>] (vty_init+0x18c/0x19 c)
[<8001f668>] (vty_init+0x18c/0x19c) from [<8001eac4>] (tty_init+0x12c/0x144)
[<8001eac4>] (tty_init+0x12c/0x144) from [<800395b8>] (do_one_initcall+0x114/0x1 6c)
[<800395b8>] (do_one_initcall+0x114/0x16c) from [<8000898c>] (kernel_init+0xc0/0 x144)
[<8000898c>] (kernel_init+0xc0/0x144) from [<8003fac4>] (kernel_thread_exit+0x0/ 0x8)
Code: 1afffff3 e596a014 e2811004 e3a00001 (e795a00a)
---[ end trace 1b75b31a2719ed1c ]---
Kernel panic - not syncing: Attempted to kill init!
[<800462c8>] (unwind_backtrace+0x0/0xec) from [<80567fcc>] (panic+0x80/0x19c)
[<80567fcc>] (panic+0x80/0x19c) from [<800757e0>] (complete_and_exit+0x0/0x1c)
[<800757e0>] (complete_and_exit+0x0/0x1c) from [<00000001>] (0x1)
As I stated in a comment, the crash looks very similar to the one in Freescale 3.0.35 kernel crash. If so, the crash happens in memset.S. The top two commits in memset.S in upstream kernel whose SHA1 begins with c2459d3 and 1bd4678, respectively, should solve that issue.

How to tell the difference between soft lockup in kernel and softlockup in userspace?

I have the below lockup. Is this indicating that slic_test pid 5207 is the offender, or is it only showing that because the current macro in the kernel is still pointing to the userspace process which made the system call into my driver? Furthermore - irq event stamp 0 ... what exactly does the irq event stamp represent? It can't be number of interrupts since boot... there were definitely more than 0 after 88798 seconds...
The system is uniprocessor with CONFIG_SMP disabled.
[88798.449628] BUG: soft lockup - CPU#0 stuck for 61s! [slic_test:5207]
[88798.449628] Modules linked in: slic_xxxx leds_xxxx vortex86_spi dm_mirror dm_log dm_multipath dm_mod ohci_hcd ehci_hcd r6040 vortex86_wdt vortex86_gpio [last un]
[88798.449628] irq event stamp: 0
[88798.449628] hardirqs last enabled at (0): [<00000000>] 0x0
[88798.449628] hardirqs last disabled at (0): [<c0115563>] copy_process+0x233/0x1090
[88798.449628] softirqs last enabled at (0): [<c0115563>] copy_process+0x233/0x1090
[88798.449628] softirqs last disabled at (0): [<00000000>] 0x0
[88798.449628]
[88798.449628] Pid: 5207, comm: slic_test Not tainted (2.6.27.62 #11)
[88798.449628] EIP: 0060:[<c011b6b5>] EFLAGS: 00000246 CPU: 0
[88798.449628] EIP is at __do_softirq+0x45/0xb0
[88798.449628] EAX: 00000000 EBX: 00000082 ECX: 00000001 EDX: dfac5080
[88798.449628] ESI: c0696120 EDI: 0000000a EBP: df3bdf8c ESP: df3bdf80
[88798.449628] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[88798.449628] CR0: 8005003b CR2: b7622780 CR3: 1f3c8000 CR4: 00000000
[88798.449628] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[88798.449628] DR6: ffff0ff0 DR7: 00000400
[88798.449628] [<c011b766>] do_softirq+0x46/0x50
[88798.449628] [<c011bad5>] irq_exit+0x45/0x50
[88798.449628] [<c01057ba>] do_IRQ+0x4a/0x90
[88798.449628] [<c0103e68>] common_interrupt+0x28/0x30
[88798.449628] =======================
The presence/absence of the call trace indicates the origin of the soft-lockup.
[88798.449628] [<c011b766>] do_softirq+0x46/0x50
[88798.449628] [<c011bad5>] irq_exit+0x45/0x50
[88798.449628] [<c01057ba>] do_IRQ+0x4a/0x90
[88798.449628] [<c0103e68>] common_interrupt+0x28/0x30
the Linux kernel caused the above soft-lockup as described by the above call-trace.
If an user-space process had caused the soft-lockup, a line identifying the process by its pid would logged, followed by the contents of various CPU-registers without a call-trace of any sorts.
[88798.449628] Pid: 5207, comm: slic_test Not tainted (2.6.27.62 #11)
Not tainted means issue happened at kernel.
"Tainted" flags are the kernel's way of saying it's not a kernel fault (the kernel source is open and "pure". "Taint" comes from the non-GPL modules, and others.
http://www.opensourceforu.com/2011/01/understanding-a-kernel-oops/

Resources