Workaround for FSCK "UNEXPECTED INCONSISTANCY; Run fsck manually - linux

I have a Linux System based LFS (Linux from Scrach). Linux kernel version 2.6.29.6 #1 SMP PREEMPT. This system uses Extlinux bootloader and boots from SSD (Micron USB Solid State Device). There is also a seconday harddrive in this system but not meant for booting. We changed the booting from HDD to SSD as we found SSD is fast and reliable over HDD.
Whenever there is a power outage, the unit reboots, the power Outage causes the SSD corruption. After reboot, the Fsck command is run by the script checkfs. The system halts with the Error message ""UNEXPECTED INCONSISTANCY; Run fsck manually error and unit halts and fails to reboot until we manually reboot.
I checked the checkfs script and found that, in this condition, the fsck -a -A -C -T is run and returns error value > 3 and < 16 for which the action is to halt the system and reboot using CD and run fsck manually and fix the issue.
I tried changing the checkfs script and used fsck -y which fixed all the errors and the unit booted normally but while fixing the issues, many files were deleted. Secondly if i ignore the fsck error and instead of system halt, if i go ahead with normal boot, it works but since it doesnt fix the filesystem issues, the unit may not work properly.
At this point i would like to know if there are any work arounds to resolve this issue and still boot the system normally and fix the filesystem issues? can i do something like if fsck fails then umount root file system from SSD and mount it from HDD and boot normally, then after boot recover SSD filesystem? if yes any pointers to do this? Please suggest.

You can append 'fastboot' as a kernel argument (in grub) to skip fsck.

Related

WSL2 "read-only" file system while building chromium

I'm attempting to build chromium on WSL2 according to this guide.
When I get to the fetch --nohooks chromium everything is loading and then I get the following error:
OSError: [Errno 30] Read-only file system: '/home/ghadar/chromium/src/third_party/libprotobuf-mutator/_gclient_src_0ve3yqhz'
I've looked everywhere and couldn't find any explanation to this error.
I'm running WSL2 on Windows 11 with Ubuntu 20.04 as the Linux distribution.
A few possibilities that I can think of:
Filesystem corruption (it happens)
Out of disk space on the host Windows drive
For the first one, see #6220 on the WSL Github. Recommended solution is (and it might be a good idea to backup any critical files first):
# Identify the correct drive:
mount | grep ext4
# Take the drive returned (e.g. /dev/sdd) and:
sudo e2fsck /dev/sdd -p
It could also be a disk-space issue. The Chromium source is pretty large at around 57GB. Is it possible that you are out of disk space on the Windows drive? If so, then WSL still thinks it has space remaining, because it is on a sparse virtual disk that can grow up to (250GB/1TB depending on the WSL release), but once space on the host drive is gone, WSL probably set the device read-only.
If that's the case, and you have a larger drive (SSD/NVMe recommended for performance, of course), you can "move" the virtual disk if you'd like -- See my Super User answer on the topic.
Or you might try fetching without the full repo history, as suggested in the docs, with the --no-history flag.

Bcache directory not showing up after make-bcache

I'm setting up bcache on an ubuntu virtual machine on azure. I'm following the instructions from http://blog.rralcala.com/2014/08/using-bcache-in-ec2.html.
After running make-bcache -B /dev/sdc1, the /dev/bcache0 device is not yet available.
When running make-bcache -B /dev/sdc1 a second time, /dev/bcache suddenly exists, as does /sys/fs/bcache and /sys/fs/block/bcache0 etc.
A weird side-note is that running make-bcache -B for the second time in a script (even with sleeps in between) does not fix the issue, but running it manually does.
Once bcache has initiated properly it stays stable, also after reboots and VM-relocations. You can read the configuration scripts at https://github.com/okke-formsma/azure-bcache.
Does anyone have a clue how to enable bcache on my ubuntu 16.04 machine without having to resort to a double manual make-bcache -B xxx?
[edit] According to the archlinux wiki not having a /sys/fs/bcache means that "The kernel you booted is not bcache enabled", however that is not the case.
I figured it out!
The module was not loaded by the kernel. Doing a sudo modprobe bcache made all bcache functionality available without weird workarounds.

Centos 7 Emergency Mode infinite error

I cant boot my linux machine due to /etc/fstab having bad lines of code. Currently my machine will attempt to boot and will infinitely loop the message "Welcome to Emergency Mode!..." . I can't get to the console or login.
Hitting escape at the start up screen and entering the rescue mode does not work either. Using grub2 i can see my fstab file via : cat (lvm/centos-root)/etc/fstab.
How can i edit this file and remove the bad lines of code?
Boot from a live DVD, or boot in rescue mode from the installation disk. If necessary (it probably won't be), manually mount the main system's root partition somewhere (e.g. /mnt/rescue). Edit /mnt/rescue/etc/fstab appropriately. Perform a clean reboot.

How to rescue the Linux system from erroneous insmod in rc.local in Fedora Core?

I have placed a faulty kernel object in the rc.local. Because of this faulty kernel object,the system crashes on bootup. Now, my aim is to remove that faulty ko insmod from rc.user. However, I cannot access rc.user as my system crashes on bootup. How can I fix this issue?
If you are getting the grub screen then you can
go to runlevel 1 and change the file.
On grub screen you will have to
press a after selecting the Fedora kernel with which you want to boot.
APPEND single after space in that line and press enter
press b to boot in single user mode
You can edit any file in this mode.
The safest & easiest approach when I get into such troubles is to mount the HD (contaning the faulty kernel) as an external hard drive to another linux machine and manually edit the files causing trouble.
Do a fdisk -l with your HD plugged in (thorugh USB). Take notice of the </dev/sd*#>.
mount /media /dev/<sd*#>
Now, you can access your boot or root partition to access the files.

mounting fs with errors, running e2fsck is recommended

I am using CentOS 5.10 version with 64 bit, unfortunately when remove particular file under root directory the below error was strange :
**EXT3-fs warning: mounting fs with errors, running e2fsck is recommended,rm: cannot remove `/var/lock/subsys/xinetd': Read-only file system**
it has changed to read-only file system, how to fix it ???
If / is in read-only mode, first try remounting it with:
mount -o remount,rw /
You'll still need to check the partition for errors. If you have physical access to server or a KVM
-boot it in single mode
-unmount all the partitions and run fsck
If you dont have a physical access, try
touch /forcefsck
This will force File System Check on the next reboot.

Resources