Having losetup read the partition table [closed] - linux

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
For the purpose of learning, I wanted to create a mini-replica of my hard disk:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
├─sda2 8:2 0 244M 0 part /boot
└─sda3 8:3 0 930.8G 0 part
└─sda3_crypt 254:0 0 930.8G 0 crypt
├─host--vg-root 254:1 0 25G 0 lvm /
├─host--vg-var 254:2 0 2.8G 0 lvm /var
├─host--vg-swap_1 254:3 0 11.9G 0 lvm [SWAP]
├─host--vg-tmp 254:4 0 380M 0 lvm /tmp
└─host--vg-home 254:5 0 890G 0 lvm /home
In my particular case, setting up a new device so it looks like my hard disk requires familiarity with many things, creating partitions, creating LUKS devices, opening them, creating LVM volumes etc, so I regard this as a worthy exercise, at least for someone who is new to Linux.
So I first needed a new device to play with, without messing up anything else:
$ dd if=/dev/zero of=loopfile bs=1M count=1024
$ sudo losetup /dev/loop1 loopfile
(using loop1 rather than loop0 which is already taken for some other purpose, also zero good enough for this exercise so ignoring urandom).
My first objective was to mimic the partitions sda1/sda2/sda3
$ sudo blkid
/dev/sda1: UUID="08FC-EA23" TYPE="vfat" ...
/dev/sda2: UUID="30b5d595-4986-4f75-962a-7e1f5f03ed4a" TYPE="ext2" ...
/dev/sda3: UUID="a84cc598-9316-48b9-94a9-bb4885e45e9c" TYPE="crypto_LUKS" ...
$ sudo parted /dev/loop1
So I went and created three 'primary' partitions (using 'fat32' for the first one and 'ext2' for other two, not too sure why just guessing) with all sizes reduced by a factor of 1000:
(parted) print
Number Start End Size Type File system Flags
1 512B 1000kB 1000kB primary fat32 lba
2 1049kB 2097kB 1049kB primary ext2 lba
3 2097kB 1074MB 1072MB primary ext2 lba
and I then formatted the three devices in line with the previous blkid report:
sudo mkfs -t vfat /dev/loop1p1
sudo mkfs -t ext2 /dev/loop1p2
sudo cryptsetup luksFormat /dev/loop1p3
So at this point, my parted print report looks good as well as lsblk and blkid:
$ lsblk
loop1 7:1 0 1G 0 loop
├─loop1p1 259:0 0 976.5K 0 loop
├─loop1p2 259:1 0 1M 0 loop
└─loop1p3 259:2 0 1022M 0 loop
$ sudo blkid
/dev/loop1p1: SEC_TYPE="msdos" UUID="1CD8-2CA5" TYPE="vfat" ...
/dev/loop1p2: UUID="6532dba9-3101-488e-a6d1-e5e1ef4943f7" TYPE="ext2" ...
/dev/loop1p3: UUID="a0e96a54-6d6a-49c8-80fd-03217b25062f" TYPE="crypto_LUKS" ...
/dev/loop1: PTUUID="1de285f7" PTTYPE="dos"
So I thought I was on the right track. I also thought that my file loopfile which underlies my loop device would contain of the necessary metadata so I do not need to worry about rebooting. As I am only playing with devices (not mounting them) I assumed there is no need for any /etc/fstab setup...
The issue I have is that when I reboot, some of the set up seems to be lost. After re-creating the loop device from loopfile, the parted print report still shows me the partitions (albeit with lost information on type), but these partitions no longer appear on the lsblk or the blkid reports. I was wondering if there was a way of making my set up persistant. I am on Debian 8, in case this matters.

You need to run losetup -P /dev/loop1 loopfile. What this does is tell the kernel to perform a partition table scan of the newly added file.

Related

How do I assign another hard disk to root in linux? [closed]

Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 months ago.
Improve this question
The following is the current status of my disk:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 62M 1 loop /snap/core20/1587
loop1 7:1 0 62M 1 loop /snap/core20/1611
loop2 7:2 0 79.9M 1 loop /snap/lxd/22923
loop3 7:3 0 103M 1 loop /snap/lxd/23541
loop4 7:4 0 47M 1 loop /snap/snapd/16292
sda 8:0 0 60G 0 disk
├─sda1 8:1 0 1M 0 part
├─sda2 8:2 0 2G 0 part /boot
├─sda3 8:3 0 28G 0 part
│ └─ubuntu--vg-ubuntu--lv 253:0 0 28G 0 lvm /
├─sda4 8:4 0 30G 0 part
└─sda5 8:5 0 1007K 0 part
sr0 11:0 1 1.4G 0 rom
I added sda4 as above, is there any way to increase the root capacity to sda4?
You can extend the VG and LV by commands:
pvcreate /dev/sda4
vgextend ubuntu--vg /dev/sda4
lvextend -L+4GB /dev/mapper/ubuntu--vg-ubuntu--lv
The last command add 4 GB space to /
and then extend your filesystem. It it's ext4 use command:
resize2fs /
If its XFS you can use command
xfs_growfs -d /
Both resize commands can be done on the fly (w/o unmount the filesystem)

Seems like I have lots of space outside of the boot disk but I can't install packages - how do I clear space? Ubuntu GCloud VM [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 11 months ago.
Improve this question
Here is the output of df -H. I keep deleting caches and tmp directories but the problem keeps resurfacing. Any tips on how I might clear out more space?
My home directory is only taking up 3GB - including Python packages and so on, not sure where the 104GB is being taken up. I deleted snapd earlier because all the loop devices were full with vnode tables; any time I try to install a package, for instance, xdiskusage, I get the error `
After this operation, 525 MB of additional disk space will be used.
E: You don't have enough free space in /var/cache/apt/archives/...
df -H
Filesystem Size Used Avail Use% Mounted on
/dev/root 104G 104G 0 100% /
devtmpfs 180G 0 180G 0% /dev
tmpfs 180G 0 180G 0% /dev/shm
tmpfs 36G 3.6G 33G 10% /run
tmpfs 5.3M 0 5.3M 0% /run/lock
tmpfs 180G 0 180G 0% /sys/fs/cgroup
/dev/sda15 110M 5.5M 104M 5% /boot/efi
tmpfs 36G 0 36G 0% /run/user/2002
Warning: you will not be able to log in to Linux when you run out of free disk space. Immediately delete some files or reside the disk larger.
The first step is to delete everything in the /tmp folder. Ignore warnings about not being able to delete files as some processes may have files open.
Next figure out what are the largest files and review which ones can be deleted.
This command will list the 25 largest files:
find / -type f -printf '%s %p\n'| sort -nr | head -25
Be careful what you delete. Do not manually delete files located in the OS directories.
Next list the largest installed packages. Uninstall any that are not required.
dpkg-query --show --showformat='${Installed-Size}\t${Package}\n' | sort -rh | head -25 | awk '{print $1/1024, $2}'
Another item is to review the largest directories on your system:
du -a / | sort -n -r | head -n 25
The final solution(s) are up to you. You must decide which files are not longer required for your system.

Mounting instance storage corrupting ec2 instance [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
I'm trying to mount two instance storages in my ec2 instance and before creating an AMI, I just want to try it's mounting those storages at the right mount point. But as soon as I stop and start my instance after mounting, I'm unable to connect. Looks like it's unable to boot even though ec2 console shows they are running.
I get this right after I create my instance(i2.2xlarge):
[root#xxxxx ec2-user]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 300G 0 disk
└─xvda1 202:1 0 300G 0 part /
xvdb 202:16 0 745.2G 0 disk
xvdc 202:32 0 745.2G 0 disk
Then I format and mount those two to two different location.
[root#xxxx ec2-user]# mkfs -t ext4 /dev/xvdc
[root#xxxx ec2-user]# mkfs -t ext4 /dev/xvdc
Here is my fstab:
#
LABEL=/ / ext4 defaults,noatime 1 1
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/xvdb /media/ephemeral0 ext4 defaults,nofail,comment=cloudconfig 0 2
/dev/xvdc /media/ephemeral1 ext4 defaults,nofail,comment=cloudconfig 0 2
After I mount them, I get this which I want at the end:
[root#xxxxxx ec2-user]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 493G 1.2G 491G 1% /
devtmpfs 30G 68K 30G 1% /dev
tmpfs 31G 0 31G 0% /dev/shm
/dev/xvdb 734G 69M 697G 1% /media/ephemeral0
/dev/xvdc 734G 69M 697G 1% /media/ephemeral1
At this point, when I want to stop and start the instance, I'm unable to connect that instance. I know those two are ephemeral storage and I don't care it's content. But I want to recreate several similar instances like this, so before creating an AMI, I just wanted to test it to see after I restart this instance, it keeps mount configuration.
What I am doing wrong?
This issue is a major problem while working with paritioning. The root cause of problem is SElinux which is refusing SSH connection
Here are the steps which will solve your issue :
Step 1 : Create the volume in AWS Console and attach it to instance. (Assuming you know this already!)
Step 2 : By default it is always mounted on /dev/xvdc, please create the partition using fdisk and confirm the lsblk output, it should look like below:
$ sudo fdisk /dev/xvdc
Use options N to create a new partition and all the defaults for creating 1 full partition for entire volume and option W to write the partition in the filesystem
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdc 202:80 0 20G 0 disk
└─xvdc1 202:81 0 20G 0 part
*All the work ahead would be done on this xvdfc partition, make sure you are NOT using /dev/xvdc anywhere.
Step 3 : Format the below partition using
$ sudo mkfs -t ext4 /dev/xvdc1
Step 4: Make the entry in fstab as below:
/dev/xvdf1 /var ext4 defaults,noatime,nofail 0 2
Hope that helps :)
Here are some links that might help :
STEPS TO CREATE SEPARATE /VAR PARTITION ON EBS VOLUME AWS
CREATE ROOT SWAP AND LVM PARTITION ON EBS VOLUME (AWS)

upgrade from Centos 6.7 to 7 without compromise the data in a LVM [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 years ago.
Improve this question
7 Server with 4 Discs and I have this configuration:
"/" In a LVM (Physical disk 1)
"/data" In LVM (Physical disk 1 + fake raid 0 of disks 2 and 3)
"/data1" Ext4 (Physical disk 4)
The server is a supermicro (motherboard model X8DTL) with 8gb of ram.
I need to upgrade to centos 7 because the dependencies in the newer software are only in this distro but I have afraid of fuck UP With the data in "/data"
How I can upgrade safely without screw with "/data"?
PS:
I can't make a backup, the information is more than 5TB.
"/data" and "/data1" there are only standalone files (Text, spreadsheet files, multimedia files). The programs and associations are only in "/"
Edit:
Here it is how the disks are arranged:
# lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL
NAME FSTYPE SIZE MOUNTPOINT LABEL
sda linux_raid_member 931,5G GLaDOS:0
└─md0 LVM2_member 1,8T
└─vg_glados_media-lv_data (dm-3) ext4 3,6T /data
sdc linux_raid_member 931,5G GLaDOS:0
└─md0 LVM2_member 1,8T
└─vg_glados_media-lv_data (dm-3) ext4 3,6T /data
sdb 1,8T
├─sdb1 ext4 500M /boot
├─sdb2 LVM2_member 97,7G
│ ├─vg_glados-lv_root (dm-0) ext4 50G /
│ ├─vg_glados-lv_swap (dm-1) swap 7,8G [SWAP]
│ └─vg_glados-lv_home (dm-2) ext4 39,9G /home
└─sdb3 LVM2_member 1,7T
└─vg_glados_media-lv_data (dm-3) ext4 3,6T /data
sdd 931,5G
└─sdd1 ext4 931,5G /data1 /data1
sr0 1024M
# df -H
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_glados-lv_root
53G 44G 6,6G 87% /
tmpfs 4,2G 78k 4,2G 1% /dev/shm
/dev/sdb1 500M 132M 342M 28% /boot
/dev/mapper/vg_glados_media-lv_data
3,9T 3,7T 28G 100% /data
/dev/mapper/vg_glados-lv_home
42G 862M 39G 3% /home
/dev/sdd1 985G 359G 576G 39% /data1
You have two options:
1) upgrade the existing installation. You could follow this RHEL manual for for example.
2) Make a fresh install, but a) tell the anaconda that you wish to do partitioning manually and b) carefully pick the correct partitions to format and to install the OS.
The latter option is much risky than the former. Also, you will lost any history/credentials/etc and would need to configure everything again.
If you have some spare disks to make a backup of your /data partition then better do it in both cases
I think you should add a new disk to your volume group. With this new space you can create a new logical volume and there you can try new experimental installations without compromising the rest of the system.

Linux and Hadoop : Mounting a disk and increasing the cluster capacity [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 7 years ago.
Improve this question
First of all, I'm a total noob at hadoop and linux.I have a cluster of five nodes , which when starts shows a each node capacity only 46.6 GB while each machine had around 500 gb space which i dont know how to allocate to these nodes.
(1) Do I have to change the datanode and namenode file size(i checked these and it shows the same space remaining as in the Datanode information tab)? if so how should i do that.
(2)Also this 500gb disk is only shown when i do $lsblk command and not when i do $df -H command. Does that mean its not mounted? These are the results of the commands. Can someone explain what does this mean..
[hadoop#hdp1 hadoop]$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 50G 0 disk
\u251c\u2500sda1 8:1 0 500M 0 part /boot
\u2514\u2500sda2 8:2 0 49.5G 0 part
\u251c\u2500VolGroup-lv_root (dm-0) 253:0 0 47.6G 0 lvm /
\u2514\u2500VolGroup-lv_swap (dm-1) 253:1 0 2G 0 lvm [SWAP]
sdb 8:16 0 512G 0 disk
[hadoop#hdp1 hadoop]$ sudo df -H
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
51G 6.7G 41G 15% /
tmpfs 17G 14M 17G 1% /dev/shm
/dev/sda1 500M 163M 311M 35% /boot
Please help. Thanks in advance.
First can someone help me understand why its showing different memory disks and what it means and where does it reside ?! I seem to not able to figure it out
You are right. Your second disk (sdb) is not mounted anywhere. If you are going to dedicate the whole disk to hadoop data, here is how you should format and mount it:
Format your disk:
mkfs.ext4 -m1 -O dir_index,extent,sparse_super /dev/sdb
For mounting edit the file /etc/fstab. Add this line:
/dev/sdb /hadoop/disk0 ext4 noatime 1 2
After that, create the directory /hadoop/disk0 (it doesn't have to be named like that. you could use a directory of your choice).
mkdir -p /hadoop/disk0
Now you are ready to mount the disk:
mount -a
Finally, you should let hadoop know that you want to use this disk as hadoop storage. Your /etc/hadoop/conf/hdfs-site.xml should contain these config parameters
<property><name>dfs.name.dir</name><value>/hadoop/disk0/nn</value></property>
<property><name>dfs.data.dir</name><value>/hadoop/disk0/dn</value></property>

Resources