I have a problem with UBUNTU 10.04 filesystem, which reports different results when a drive is mounted and checked with FSCK as the system drive and when it is checked by another system drive:
sudo fsck /dev/sdb1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
57217 inodes used (24.81%)
42 non-contiguous files (0.1%)
65 non-contiguous directories (0.1%)
# of inodes with ind/dind/tind blocks: 0/0/0
Extent depth histogram: 50910/20
293868 blocks used (31.88%)
0 bad blocks
1 large file
43327 regular files
7242 directories
59 character device files
26 block device files
0 fifos
509 links
6549 symbolic links (6187 fast symbolic links)
5 sockets
--------
57717 files
sudo fsck -n -t ext4 /dev/sda1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
Warning! /dev/sda1 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
/dev/sda1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (628134, counted=628213).
Fix? no
Free inodes count wrong (173391, counted=173379).
Fix? no
/dev/sda1: ********** WARNING: Filesystem still has errors **********
/dev/sda1: 57217/230608 files (0.1% non-contiguous), 293722/921856 blocks
Whenever the drive is mounted as the system / boot drive, running fsck (check don't fix) shows INODE and BLOCK counts are wrong, yet checking the same drive from another system disk reports that it's fine.
Any ideas ?
This should probably be on SuperUser or ServerFault, not StackOverflow, but anyway:
you can only fsck -n a file system that is mounted read-only. While the file system is mounted read-write, it will be inconsistent until cleanly unmounted, or the journal is recovered. The important message is
Warning: skipping journal recovery because doing a read-only filesystem check.
Related
'uname -a' on my machine gives:
Linux ct-lt-966 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux
Currently the filesystem of my USB is MS-DOS 'FAT32' which has a ~4.5 GB maximum size for individual files. I want to change this filesystem to something else, which does not have a limit. (I am trying to put a 35GB file onto a 64GB USB but I believe most USB filesystems do not limit the size of individual files).
I have not found it clear what choices of USB filesystem that I have. I tried to change the filesystem to 'NTFS', but I could not install or locate 'mkfs.ntfs' or even 'ntfsprogs'. (I also tried installing with 'pacman' and 'yum' but apparently 'pacman' requires an aarch architecture and I could not get access to 'yum-config-manager' in order to enable any repos).
So to conclude, with my minimal prowess I am just looking for any way to change the filesystem of my 64GB USB to anything which will accept a 35GB file from my machine.
Thanks
Edit 1: Just planning to use the USB on this Linux machine, not Windows.
If there's nothing on the stick you want, or it's safe to delete it then basically:
delete the current FAT32 partition from the stick
add a new partition, utilising the full size of the device
create an ext4 filesystem on the new partition
PLEASE BE CAREFUL WITH THIS PROCESS: selecting the wrong device can obliterate a disk you needed such as a $HOME or your root OS
All the following is from memory and untested: I don't have a USB stick available right now to test fully.
Start by plugging in the stick while tailing the syslog in a console and see where it gets mounted (hopefully it automounts which it should if it's a desktop based Linux you're running. Possibly not if it's a server)..
sudo tail -f /var/log/syslog
(it might be /var/log/messages depending on distro)
then plug the stick. syslog should show it being allocated a device and a mount point. A file manager window may open depending on your config if you are in a GUI. For example, you might see it being loaded on /dev/sdc1 and mounted at /media/<yourusername>/USBKEY or something.
Confirm by running lsblk and note the device for the key, i.e.
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 167.7G 0 disk
├─sda1 8:1 0 69.9G 0 part /
└─sda2 8:2 0 97.9G 0 part /home
sdb 8:16 0 149.1G 0 disk
└─sdb1 8:17 0 149.1G 0 part /mnt/snapshots
sdc 8:32 0 931.5G 0 disk
└─sdc1 8:33 0 931.5G 0 part /storage
sdd 8:48 0 465.8G 0 disk
└─sdd1 8:49 0 465.8G 0 part /mnt/backup
sr0 11:0 1 1024M 0 rom
Unmount the stick (if it mounted) but leave it plugged in. Assuming again your device is at /dev/sdc1...
umount /dev/sdc1
Now run cfdisk in a terminal if you have it (friendlier) or fdisk if not, passing it the device related to your USB stick, without the partition number.
man cfdisk
sudo cfdisk /dev/sdc
This should show the current FAT32 partition. Delete it, then create a new partition of type 'Linux', following the defaults for start and end blocks which will be suggested in such a way as to fill the available space.
When done, select the option to Write the changes. Again, DOUBLE AND TRIPLE CHECK you have the right device or you will blow away your main disk probably.
Once the changes are written, you can create the ext4 file system;
sudo mkfs.ext4 /dev/sdc1
And after it completes, you should be able to re-plug your stick and find that it remounts, this time with a file system that can take your large files.
This isn't the only way to achieve this, but it's probably the least fiddly. For the sake of repetition, don't make a mistake with the device identifiers. If you're unsure, ask.
I was trying to execute qemu while following the qemu/linaro tutorial,
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Virtual_ARM_Linux_environment
I was executing the command,
sudo mount -o loop,offset=106496 -t auto vexpress.img /mnt/tmp
mount: you must specify the filesystem type
so i did fdisk on the img file and got the following,
Device Boot Start End Blocks Id System
vexpress.img1 * 63 106494 53216 e W95 FAT16 (LBA)
vexpress.img2 106496 6291455 3092480 83 Linux
The filesystem is Linux according to the fdisk command. But I get error,
sudo mount -o loop,offset=106496 -t Linux vexpress.img /mnt/tmp
mount: unknown filesystem type 'Linux'
Kindly help.
You correctly decide to mount the particular partition by specifying its offset but the offset parameter is in bytes and fdisk shows the offset in blocks (the block size is shown before the partition list --- usually 512). For the block size 512 the command would be:
sudo mount -o loop,offset=$((106496*512)) -t auto vexpress.img /mnt/tmp
If the automatic file system type detection does not still work there is another problem. Linux is not really a file system type. In the partition table it is a collective type used for multiple possible particular file systems. For mount you must specify the particular file system. In Linux you can list the supported ones by cat /proc/filesystems.
I'm trying to recover some data from a hard drive extracted from a broken laptop, and I'm having problems mounting the disk to my current system (Linux Mint). The hard disk I'm recovering from ran Debian. Simply, I'm confused as to how I can mount the hard drive to access the files, however it's not as simple as any other mount I've done. The following details struggles and information I've encountered.
I get the following outputs when trying to mount the hard drive with different file-system tags. I should add that the file-system type isn't automatically detected when using auto, and "sdb" is definitely the correct address for the disk (taken it from dmesg).
$ mount /dev/sdb /mnt/usb -t ntfs
NTFS signature is missing.
Failed to mount '/dev/sdb': Invalid argument
The device '/dev/sdb' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
The following returns the same message when all other common file-system tags are used:
$ sudo mount /dev/sdb usb -t ext2
mount: wrong fs type, bad option, bad superblock on /dev/sdb,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
The results from these commands led me to believe that there was an issue with the hard disk and it's partitions, however fdisk proved that it's partition's do seem to be valid and correct:
$ sudo fdisk /dev/sdb -l
Disk /dev/sdb: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002da94
Device Boot Start End Blocks Id System
/dev/sdb1 * 2048 475920383 237959168 83 Linux
/dev/sdb2 475922430 488396799 6237185 5 Extended
/dev/sdb5 475922432 488396799 6237184 82 Linux swap / Solaris
I then decided to try verify the file-system type of the hard drive, which seems to be "devtmpfs", which I got from the following command using df:
$ df /dev/sdb -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
udev devtmpfs 1014764 4 1014760 1% /dev
And so finally, I mount the hard drive using -t devtmpfs, which is successful in mounting however I'm left with a confusing file system very unlike from what I would expect from what was a standard debian set up.
It contains file folders such as "block","bus","char","disk","dri","mapper"... and files like "sda1","sdb","sdb1","tty","vcs".
I'm totally stumped as to how I should progress, and I'm pretty convinced the hard disk isn't broken and that I'm just mounting it incorrectly. How can I successfully mount the disk so I can access my files? Any help would be greatly appreciated.
Ok, you are trying to mount the entire disk instead of individual partitions, which is why you are getting the error. In short the command you need is:
mount /dev/sdb1 /mnt/usb
The file /dev/sdb references the entire disk as a block file. This includes the partition table at the start, which is why it can't find a filesystem. The file /dev/sdb1 references the first partition, which is where your filesystem will be. From the looks of your fdisk output, this is not an ntfs partition since this is a Windows filesystem and the partition is marked as Linux (most likely you will have ext4 unless you specifically set up something different).
To add a quick explanation of devtmpfs, this is a special filesystem which contains these block files which are specified by udev. You can google both for more information, but by now I'm sure you now know its not what you are looking for.
I have an application that uses hugepage and the application suddenly crashed due to some bug.
After crashing, since the application does not release the hugepage properly, the free hugepage number is not increased in sys filesystem.
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
0
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
1024
Is there a way to release the hugepages by force?
Sometimes need to check all directory that hugetlbfs has been mounted.
So,
find mounted directory by command mount | grep huge.
check every directory except especially /dev/hugepages.
delete all 2M-sized files. (2M is the size of hugepage)
Use ipcs -m to list the shared memory segments.
Use ipcrm to remove the left over shared memory segments.
Edit on 06/24/2019:
Ok, so, the above answer, while correct as far as it goes, was a bit brief. In particular, if you have a host with multiple DB instances, and only one is crashed how can you determine which (if any) memory segments should be cleaned up?
Well, this too, can be done. For each running instance, connect w/ / as sysdba, then do oradebug setmypid (any pid will do, as all Oracle PIDs connect to the SGA). Then do oradebug ipc. That will (hopefully) return IPC information written to the trace file. So, go to the udump (or diag_dest) directory, and look for your trace file. It will contain all the IPC information for the instance. This will include ShmId. Look through the file for the ShmId(s) that this instance is using. Now look at the output of ipcs -m.
When you have done that for all the running instances, any memory segment output by ipcs -m that shows non-zero memory allocation, and that you cannot account for in the oradebug ipc information from any running instance, must be the left over memory segments from the crashed instance. Use ipcrm to remove it/them.
When doing this on a host with multiple running instances, this can be a bit fraught. Please proceed with caution. You don't want to remove the SGA of a running instance!
Hope that helps....
HugeTLB can either be used for shared memory (and Mark J. Bobak's answer would deal with that) or the app mmaps files created in a hugetlb filesystem. If the app crashes without removing those files they survive and keep corresponding memory 'allocated'.
Check hugeTLB filesystem and see if there are any leftover files from the app. Removing them would release the memory.
If you follow the instruction below, you can get rid of the allocated hugepages:
1) Let's check the hugepages which were free at restart
dpdk#dpdkvm:~$ ls /mnt/huge/
empty
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
2) Starting a dpdk application with wrong parameters, producing an error
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo ./build/kni -c 0x03 -n 2 -- -P -p 0x03 --config="(0,0,1),(1,0,1)"
...
EAL: Error - exiting with code: 1
Cause: No supported Ethernet device found
3) When I check hugepages, there is not any free
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 0
...
4) Now, when I check the mounted hugepage directory, I can see the files which are not given back to OS by dpdk application.
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ ls /mnt/huge/
...
rtemap_0 rtemap_137 rtemap_176 rtemap_214 rtemap_253 rtemap_62
...
5) Finally, if you remove the files starting with rtemap, you can give the hugepages back
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo rm /mnt/huge/*
[sudo] password for dpdk:
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
your hugetlb may be used by shared memory or mmap files.
try to remove the shared memories or umount the hugetlb fs
I have a vps slice running centos 5.5 I am supposed to have 15 gigs of disk space, but according to df it seems to double my disk space usage.
when I run du -skh * in / as root i get:
[root#yardvps1 /]# du -skh *
0 aquota.group
0 aquota.user
5.2M bin
4.0K boot
4.0K dev
4.9M etc
2.5G home
12M lib
14M lib64
4.0K media
4.0K mnt
299M opt
0 proc
692K root
23M sbin
4.0K selinux
4.0K srv
0 sys
48K tmp
2.0G usr
121M var
this is consistent with what I have uploaded to the machine, and adds up to about 5gigs.
BUT when i run df i get:
[root#yardvps1 /]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/simfs 15728640 11659048 4069592 75% /
none 262144 4 262140 1% /dev
it is showing me using almost 12 gigs already.
what is causing this discrepancy and is there anything I can do about it, I planned the server out based on 15 gigs but now it is basically only letting me have about 7 gigs of stuff on it.
thanks.
The most common cause of this effect is open files that have been deleted.
The kernel will only free the disk blocks of a deleted file if it is not in use at the time of its deletion. Otherwise that is deferred until the file is closed, or the system is rebooted.
A common Unix-world trick to ensure that no temporary files are left around is the following:
A process creates and opens a temporary file
While still holding the open file descriptor, the process unlinks (i.e. deletes) the file
The process reads and writes to the file normally using the file descriptor
The process closes the file descriptor when it's done, and the kernel frees the space
If the process (or the system) terminates unexpectedly, the temporary file is already deleted and no clean-up is necessary.
As a bonus, deleting the file reduces the chances of naming collisions when creating temporary files and it also provides an additional layer of obscurity over the running processes - for anyone but the root user, that is.
This behaviour ensures that processes don't have to deal with files that are suddenly pulled from under their feet, and also that processes don't have to consult each other in order to delete a file. It is unexpected behaviour for those coming from Windows systems, though, since there you are not normally allowed to delete a file that is in use.
The lsof command, when run as root, will show all open files and it will specifically indicate deleted files that are deleted:
# lsof 2>/dev/null | grep deleted
bootlogd 2024 root 1w REG 9,3 58 917506 /tmp/init.0W2ARi (deleted)
bootlogd 2024 root 2w REG 9,3 58 917506 /tmp/init.0W2ARi (deleted)
Stopping and restarting the guilty processes, or just rebooting the server should solve this issue.
Deleted files could also be held open by the kernel if, for example, it's a mounted filesystem image. In this case unmounting the filesystem or rebooting the server should do the trick.
In your case, judging by the size of the "missing" space I'd look for any references to the file that you used to set up the VPS e.g. the Centos DVD image that you deleted after installing.
Another case which I've come across although it doesn't appear to be your issue is if you mount a partition "on top" of existing files.
If you do so you effectively hide existing files that exist in the directory on the mounted-to partition (the mount point) from the mounted partition.
To fix: stop any processes with open files on the mounted partition, unmount partition, find and move/remove any files that now appear in mount point directory.
I had the same trouble with FreeBSD server. The reboot helped.