Kernel Panic - Not Syncing. Segfault at Init - linux

I recently tried to install php5-gd package on my debian vmware server and it failed at libc6 - i386.
Afterwards every command other than CD caused a Segmentation fault and the server would no long boot, showing the following error
[ 4.808086] init[1]: segfault at 0 ip (null) sp bff4645c error 14 in init[8048000+8000]
[ 4.808372] Kernel panic - not syncing: Attempted to kill init!
[ 4.808442] Pid: 1, comm: init Not tainted 3.2.0-4-686-pae #1 Debian 3.2.65-1
[ 4.808512] Call Trace:
(Trace continued in this image
)
I am at a complete loss on what to do at this moment. Any help or direction would be appreciated
Edit: I've since uploaded debian-live-8.3.0-i386-standard to the vmware store and booted the broken vm with the live cd.
Now I am in the live cd terminal but not sure what to do next. I did a lsblk and noted that the broken vm's boot partition is sda > sda2 and that's all I have done so far. Do I need to mount this somewhere now?
Edit2: I've now mounted the broken partition into the live cd, however when I tried to chroot, I get Segmentation Fault:
# mkdir -p /mnt/tcs1/boot
# mount /dev/tcs1/root /mnt/tcs1
# mount /dev/sda1 /mnt/tcs1/boot
# mount -t proc none /mnt/tcs1/proc
# mount -o bind /dev /mnt/tcs1/dev
# mount -o bind /run /mnt/tcs1/run
# mount -o bind /sys /mnt/tcs1/sys
# chroot /mnt/tcs1 /bin/bash
# Segmentation fault

Solved:
I relinked ld-linux.so.2 to /lib/i386-linux-gnu/ld-2.19.so from a rescue CD and managed to chroot in

Related

How to check if /init starts /etc/inittab

I have an embedded ARM system with processor AT91SAM9G45.
System consists of two components:
Linux kernel (4.14.79)
Busybox 1.29.3 as initramfs image.
I connect to the device using putty and connecting to serial port.
When kernel starts, everything goes fine. Kernel unpacks initramfs image, all files are found and listed (I see it by debug messages). But when it starts /init, log messages are:
Freeing unused kernel memory: 384K
This architecture does not have kernel memory protection.
run_init_process BEFORE /init
run_init_process AFTER /init, result = 0
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
/init is symlink to /bin/busybox. I tried to replace /init with /sbin/init, /bin/busybox, /linuxrc, but results are the same.
/etc/inittab file:
# Begin /etc/inittab
id::initdefault:
si::sysinit:/etc/init.d/rc S
#l0::wait:/etc/rc.d/init.d/rc 0
#l1::wait:/etc/rc.d/init.d/rc 1
#l2::wait:/etc/rc.d/init.d/rc 2
#l3::wait:/etc/rc.d/init.d/rc 3
#l4::wait:/etc/rc.d/init.d/rc 4
#l5::wait:/etc/rc.d/init.d/rc 5
#l6::wait:/etc/rc.d/init.d/rc 6
ca::ctrlaltdel:/sbin/shutdown -t1 -a -r now
su::once:/sbin/sulogin
1::respawn:/sbin/getty ttyS1 115200
2::respawn:/sbin/getty ttyS2 115200
3::respawn:/sbin/getty ttyS3 115200
4::respawn:/sbin/getty ttyS4 115200
5::respawn:/sbin/getty ttyS5 115200
6::respawn:/sbin/getty ttyS6 115200
# End /etc/inittab
/etc/init.d/rcS file (this file is allowed to execute):
#!/bin/busybox sh
echo "Hello world!"
I don't know if even /init process starts parsing /etc/inittab or it falls before getting /etc/inittab by some reasons I cannot find out. Maybe there are
some mistakes in my /etc/inittab and /etc/init.d/rcS files. Maybe there are some errors with terminal (/etc/init.d/rcS cannot write to stdout cause it's blocked, suspended, being used by another process and so on).
How to definitely get sured, that /etc/inittab is started?
Welcome to StackOverflow.
I see there is space between rc and S
si::sysinit:/etc/init.d/rc S
change it to
si::sysinit:/etc/init.d/rcS
let me know if it works.
/init is symlink to /bin/busybox.
The typical /init file in an initramfs built by Buildroot that incorporates Busybox is a script of seven lines:
#!/bin/sh
# devtmpfs does not get automounted for initramfs
/bin/mount -t devtmpfs devtmpfs /dev
exec 0</dev/console
exec 1>/dev/console
exec 2>/dev/console
exec /sbin/init $*
Note the comment ("devtmpfs does not get automounted for initramfs") and the mount command for /dev.
It's /sbin/init (rather than /init) that is linked to /bin/busybox.
IOW without the proper setup of the /dev directory, userland has no I/O capabilty.
Only after devtmpfs has been mounted should the init program in Busybox be executed, which will then access /etc/inittab.
See Is there a way to get Linux to treat an initramfs as the final root filesystem?
and
Make CONFIG_DEVTMPFS_MOUNT apply to initramfs/initmpfs

mount already mounted or busy

I have an Amazon EC2 instance (Ubuntu 12.04) to which I have attached two 250 GB volumes. Inadvertently, the volumes got unmounted. When I tried mounting them again, with the following command,
sudo mount /dev/xvdg /data
this is the error I get :
mount: /dev/xvdg already mounted or /data busy
Then, I tried un-mounting it as follows :
umount /dev/xvdg but it tells me that the volume is not mounted.
umount: /dev/xvdg is not mounted (according to mtab)
I tried lsof to check for any locks but there weren't any.
The lsblk output is as below :
Any help will be appreciated. What do I need to do to mount the volumes back without losing the data on them?
Ok, figured it out. Thanks #Petesh and #mootmoot for pushing me in the right direction. I was trying to mount single volumes instead of a RAID 0 array. The /dev/md127 device was running so I stopped it first with the following command :
sudo mdadm --stop /dev/md127
Then I assembled the RAID 0 array :
sudo mdadm --assemble --uuid <RAID array UUID here> /dev/md0
Once the /dev/md0 array became active, I mounted it on /data.
Try umount /dev/xvdg* and umount /data and then
mount /dev/xvdg1 /data

missing superblock on encrypted filesystem

I have a hard drive with ubuntu 14 installed. The whole disk is encrypted. My default users home directory is encrypted as well. Lately, after a system crash, I am presented with a busybox (initramfs) on startup. When I chose to start in recovery mode, I can grasp several error messages like " ... Failed to read block at offset xyz ...".
I searched and found this Q&A: Boot drops to a (initramfs) prompts/busybox
I booted from a CD and followed the instructions. However I am only able to do ...
sudo dumpe2fs /dev/sda1
... and then continue to check and repair superblocks on /dev/sda1 .
If I try ...
sudo dumpe2fs /dev/sda2
... i get the following error message:
dumpe2fs: Attempted to read block from filesystem resulted
in short read while trying to open /dev/sda2
Couldn't find valid filesystem superblock.
gparted shows the partitioning and file systems of the drive as follows:
partition file system size used unused flags
-------------------------------------------------------------
/dev/sda1 ext2 243M 210M 32M boot
/dev/sda2 extended 465G - - -
/dev/sda5!!crypt-luks 465G - - -
unallocated unallocated 1M - - -
The warning (!!) at sda5 says "Linux Unified Key Setup encryption is not yet supported".
If I try ...
sudo dumpe2fs /dev/sda5
... it returns this error message:
dumpe2fs: Bad magic number in super-block while trying to open /dev/sda5
Couldn't find valid filesystem superblock.
Mounting and rw-accessing sda1 works without error.
Any clues what is the cause and how i can repair, mount and decrypt the filesystem to boot normaly or at least to recover the data?
The given solution has missed some commands that you need to decrypt the file system and access it. Here's the full solution
Boot from Ubuntu USB
cryptsetup luksOpen /dev/rawdevice somename
sck /dev/mapper/somename
Get backup superblock:
sudo dumpe2fs /dev/mapper/ubuntu--vg-root | grep superblock
Fix:
sudo fsck -b 32768 /dev/mapper/ubuntu--vg-root -y
Verify:
mkdir /a
sudo mount /dev/mapper/ubuntu--vg-root /a
This worked for me:
Boot from Ubuntu USB
get backup superblock:
sudo dumpe2fs /dev/mapper/ubuntu--vg-root | grep superblock
fix:
sudo fsck -b 32768 /dev/mapper/ubuntu--vg-root -y
verify
mkdir /a
sudo mount /dev/mapper/ubuntu--vg-root /a
I used following links as source:
https://askubuntu.com/questions/137655/boot-drops-to-a-initramfs-prompts-busybox
https://serverfault.com/questions/375090/using-fsck-to-check-and-repair-luks-encrypted-disk

Busybox SUID on NFS rootfs

I am building a Linux system from the bottom for a Beagle Bone board. I have compiled the vanilla kernel and built a basic root file system with busybox. The system is booted with U-boot, while the rootfs is located on a Linux PC and exported through NFS:
/path/to/rootfs 10.42.0.17(rw,wdelay,no_root_squash,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
The U-boot bootargs are:
bootargs console=ttyO0,115200n8 root=/dev/nfs rw nfsroot=${serverip}:/path/to/rootfs,v3,tcp ip=dhcp
I've encountered a problem when trying to get su working for non-root users. In order to work around the problem people over internet are suggesting to set the suid bit for the busybox binary.
After doing so:
$ sudo chmod u+s busybox
and verifying:
$ ls -la
...
-rwsr-xr-x 1 myuser myuser 1882976 Jan 13 21:47 busybox
...
$ stat -c "%a %n" busybox
4755 busybox
Something went wrong. The kernel is booting and all of the usual messages are displayed, but it is getting stuck at the end, and no login line is displayed. Here are last few lines of the booting sequence:
[ 3.776185] IP-Config: Complete:
[ 3.779656] device=eth0, hwaddr=c8:a0:30:c5:80:e9, ipaddr=10.42.0.17, mask=255.255.255.0, gw=10.42.0.1
[ 3.789877] host=10.42.0.17, domain=, nis-domain=(none)
[ 3.795822] bootserver=10.42.0.1, rootserver=10.42.0.1, rootpath=
[ 3.802492] nameserver0=10.42.0.1
[ 3.871575] VFS: Mounted root (nfs filesystem) on device 0:15.
[ 3.879903] devtmpfs: mounted
[ 3.883713] Freeing unused kernel memory: 380K (c07ef000 - c084e000)
If removing the flag, the things are returning to normal:
....
[ 3.862291] Freeing unused kernel memory: 380K (c07ef000 - c084e000)
10.42.0.17 login:
If setting the flag from within the running shell on the Beagle Bone board itself, the shell is stopping responding right after the chmod is performed.
I suspect it is something to do with the way the NFS is exporting the rootfs, but it's only a guess, so qualified explanation and possible solution would be helpful.
After some research I will answer my question myself. The answer is very simple. In order the above to work, the busybox binary should be owned by root:root. The simplest solution is just to change the ownership.

Mount Squashfs as Root in initramfs Raspbian

I'm currently trying to make my raspbian boot from a squashFS image through initramfs.
The squashfs image is on file system /dev/mmcblk0p1, so I mount this prior to mounting the Image.
Unfortunately I'm stuck at mounting the SquashFS Image file.
I tried mounting through
mount -o loop
but I always get
mounting /dev/loop0 on /mnt/root failed: No such device
So I tried to mount it through loop manually. The problem is, that mounting the /dev/loop0 fails with the same Error as above.
This is my init so far:
#!/bin/busybox sh
# Mount the /proc and /sys filesystems.
mount -t proc none /proc
mount -t sysfs none /sys
#create loop devices
mknod /dev/loop0 b 7 0
mkdir /mnt/root
# Mount the boot partition
mount -o ro /dev/mmcblk0p1 /mnt/boot
sleep 5
# Mount the root FS with losetup
echo "Mounting with losetup"
losetup /dev/loop0 /mnt/boot/SYSTEM
mount /dev/loop0 /mnt/root -t squashfs
echo "Done"
#mount -o ro,loop -t squashfs /mnt/boot/SYSTEM /mnt/root
# Use Fallback mounting failed
if [ ! -e /mnt/root/home ]
then
echo "MOUNTING SDA1"
mount /dev/sda1 /mnt/root -t ext4
fi
# Clean up.
umount /proc
umount /sys
umount /mnt/boot
# Boot the real thing.
exec switch_root /mnt/root /sbin/init
Checking
losetup /dev/loop0
tells me, that the squashfs Image has been attached to loop0.
Does somebody know where to start?
I came across your question because I ran into the same thing today while building my own initramfs. I did some more digging and found this:
http://www.silas.net.br/doc.notes/unix/linux/busybox-troubleshooting.html
Then it dawned on me that I hadn't built squashfs into the kernel or loaded the module. After I did that, everything worked!
In case you haven't, make sure your kernel supports squashfs.

Resources