I'm trying to use sfdisk to create an image file inside a docker container and I can use below command without any problem:
root#c8e9be2eb26f:/# sfdisk bbb_image.img << EOF
> 1M,48M,0xE,*
> ,,,-
> EOF
Checking that no-one is using this disk right now ... OK
Disk bbb_image.img: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
>>> Created a new DOS disklabel with disk identifier 0x34b8e793.
bbb_image.img1: Created a new partition 1 of type 'W95 FAT16 (LBA)' and of size 48 MiB.
bbb_image.img2: Created a new partition 2 of type 'Linux' and of size 975 MiB.
bbb_image.img3: Done.
New situation:
Disklabel type: dos
Disk identifier: 0x34b8e793
Device Boot Start End Sectors Size Id Type
bbb_image.img1 * 2048 100351 98304 48M e W95 FAT16 (LBA)
bbb_image.img2 100352 2097151 1996800 975M 83 Linux
The partition table has been altered.
Syncing disks.
Now in my Dockerfile this seems doesn't work and reproduce incomplete results:
RUN sfdisk bbb_image.img << "EOF\n\
1M,48M,0xE,*\n\
,,,-\n\
EOF"
And reproduce this in the console which is wrong:
Step 4/4 : RUN sfdisk bbb_image.img << "EOF\n1M,48M,0xE,*\n,,,-\nEOF\n"
---> Running in afc86ffef92a
Checking that no-one is using this disk right now ... OK
Disk bbb_image.img: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
>>> Done.
New situation:
ERROR: Service 'test' failed to build: The command '/bin/sh -c sfdisk bbb_image.img << "EOF\n1M,48M,0xE,*\n,,,-\nEOF\n"' returned a non-zero code: 1
I'm not sure how to handle the EOF in the Dockerfile.
Maybe try this one:
RUN sfdisk bbb_image.img << "\n1M,48M,0xE,*\n,,,-\n"
Related
I am experiencing some issues when using mmls command after having created an image with dcfldd/guymager in some particular situations. Usually this approach seems to be working fine to create physical images of devices, but with some USBs (working fine and undamaged) I manage to create the .dd disk image file, but then it won't be opened by mmls, nor fsstat.
fls does open the file system structure, but it seems like it won't show me any unallocated files just as if this was a logical image.
This is the command run to create a disk image using dcfldd:
sudo dcfldd if=/dev/sda hash=sha256 hashlog=usb.sha256hash of=./usb.dd bs=512 conv=noerror,sync,notrunc
Also, this is the output of usb.info, generated by guymager:
GUYMAGER ACQUISITION INFO FILE
==============================
Guymager
========
Version : 0.8.13-1
Version timestamp : 2022-05-11-00.00.00 UTC
Compiled with : gcc 12.1.1 20220507 (Red Hat 12.1.1-1)
libewf version : 20140812 (not used as Guymager is configured to use its own EWF module)
libguytools version: 2.0.2
Host name : lucafedora
Domain name : (none)
System : Linux lucafedora 6.1.7-100.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 18 18:37:43 UTC 2023 x86_64
Device information
==================
Command executed: bash -c "search="`basename /dev/sda`: H..t P.......d A..a de.....d" && dmesg | grep -A3 "$search" || echo "No kernel HPA messages for /dev/sda""
Information returned:
----------------------------------------------------------------------------------------------------
No kernel HPA messages for /dev/sda
Command executed: bash -c "smartctl -s on /dev/sda ; smartctl -a /dev/sda"
Information returned:
----------------------------------------------------------------------------------------------------
/usr/bin/bash: line 1: smartctl: command not found
/usr/bin/bash: line 1: smartctl: command not found
Command executed: bash -c "hdparm -I /dev/sda"
Information returned:
----------------------------------------------------------------------------------------------------
/usr/bin/bash: line 1: hdparm: command not found
Command executed: bash -c "CIDFILE=/sys/block/$(basename /dev/sda)/device/cid; echo -n "CID: " ; if [ -e $CIDFILE ] ; then cat $CIDFILE ; else echo "not available" ; fi "
Information returned:
----------------------------------------------------------------------------------------------------
CID: not available
Hidden areas: unknown
Acquisition
===========
Linux device : /dev/sda
Device size : 8053063680 (8.1GB)
Format : Linux dd raw image - file extension is .dd
Image path and file name: /home/HOMEDIR/case_usb/usb.dd
Info path and file name: /home/HOMEDIR/case_usb/usb.info
Hash calculation : SHA-256
Source verification : on
Image verification : on
No bad sectors encountered during acquisition.
No bad sectors encountered during verification.
State: Finished successfully
MD5 hash : --
MD5 hash verified source : --
MD5 hash verified image : --
SHA1 hash : --
SHA1 hash verified source : --
SHA1 hash verified image : --
SHA256 hash : 7285a8b0a2b472a8f120c4ca4308a94a3aaa3e308a1dd86e3670041b07c27e76
SHA256 hash verified source: 7285a8b0a2b472a8f120c4ca4308a94a3aaa3e308a1dd86e3670041b07c27e76
SHA256 hash verified image : 7285a8b0a2b472a8f120c4ca4308a94a3aaa3e308a1dd86e3670041b07c27e76
Source verification OK. The device delivered the same data during acquisition and verification.
Image verification OK. The image contains exactely the data that was written.
Acquisition started : 2023-01-28 12:27:07 (ISO format YYYY-MM-DD HH:MM:SS)
Verification started: 2023-01-28 12:30:11
Ended : 2023-01-28 12:35:24 (0 hours, 8 minutes and 16 seconds)
Acquisition speed : 41.97 MByte/s (0 hours, 3 minutes and 3 seconds)
Verification speed : 24.62 MByte/s (0 hours, 5 minutes and 12 seconds)
Generated image files and their MD5 hashes
==========================================
No MD5 hashes available (configuration parameter CalcImageFileMD5 is off)
MD5 Image file
n/a usb.dd
Worth to mention that when mmls is run against usb.dd it produces no output whatsoever. I have to forcefully add -v option for it to spit out this kind of information:
tsk_img_open: Type: 0 NumImg: 1 Img1: usb.dd
aff_open: Error determining type of file: usb.dd
aff_open: Success
Error opening vmdk file
Error checking file signature for vhd file
tsk_img_findFiles: usb.dd found
tsk_img_findFiles: 1 total segments found
raw_open: segment: 0 size: 8053063680 max offset: 8053063680 path: usb.dd
dos_load_prim: Table Sector: 0
raw_read: byte offset: 0 len: 65536
raw_read: found in image 0 relative offset: 0 len: 65536
raw_read_segment: opening file into slot 0: usb.dd
dos_load_prim_table: Testing FAT/NTFS conditions
dos_load_prim_table: MSDOS OEM name exists
bsd_load_table: Table Sector: 1
gpt_load_table: Sector: 1
gpt_open: Trying other sector sizes
gpt_open: Trying sector size: 512
gpt_load_table: Sector: 1
gpt_open: Trying sector size: 1024
gpt_load_table: Sector: 1
gpt_open: Trying sector size: 2048
gpt_load_table: Sector: 1
gpt_open: Trying sector size: 4096
gpt_load_table: Sector: 1
gpt_open: Trying sector size: 8192
gpt_load_table: Sector: 1
gpt_open: Trying secondary table
gpt_load_table: Sector: 15728639
raw_read: byte offset: 8053063168 len: 512
raw_read: found in image 0 relative offset: 8053063168 len: 512
gpt_open: Trying secondary table sector size: 512
gpt_load_table: Sector: 15728639
gpt_open: Trying secondary table sector size: 1024
gpt_load_table: Sector: 7864319
raw_read: byte offset: 8053062656 len: 1024
raw_read: found in image 0 relative offset: 8053062656 len: 1024
gpt_open: Trying secondary table sector size: 2048
gpt_load_table: Sector: 3932159
raw_read: byte offset: 8053061632 len: 2048
raw_read: found in image 0 relative offset: 8053061632 len: 2048
gpt_open: Trying secondary table sector size: 4096
gpt_load_table: Sector: 1966079
raw_read: byte offset: 8053059584 len: 4096
raw_read: found in image 0 relative offset: 8053059584 len: 4096
gpt_open: Trying secondary table sector size: 8192
gpt_load_table: Sector: 983039
raw_read: byte offset: 8053055488 len: 8192
raw_read: found in image 0 relative offset: 8053055488 len: 8192
sun_load_table: Trying sector: 0
sun_load_table: Trying sector: 1
mac_load_table: Sector: 1
mac_load: Missing initial magic value
mac_open: Trying 4096-byte sector size instead of 512-byte
mac_load_table: Sector: 1
mac_load: Missing initial magic value
For testing purposes, I am creating a virtual machine as it follows:
qemu-img create -f qcow2 \
-o cluster_size=4096,preallocation=full \
/home/marcop/.libvirt/nvme-20G.qcow2 20G
qemu-system-x86_64 -machine q35,accel=kvm \
-m 4096 \
-smp 4 \
-cpu host \
-boot d \
-cdrom /var/lib/libvirt/isos/archlinux-2020.10.01-x86_64.iso \
-drive file=/home/marcop/.libvirt/nvme-20G.qcow2,if=none,aio=native,cache.direct=on,id=D24 \
-device nvme,drive=D24,serial=1234,logical_block_size=4096,physical_block_size=4096
When booted inside the machine, I use fdisk and nvme-cli to check the sector size, but it's always 512B.
pacman -Sy nvme-cli
fdisk -l /dev/nvme0n1
with output:
Disk /dev/nvme0n1: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: QEMU NVMe Ctrl
Units: sectors of 1 * 512 = 512 bytes
Sector size (logica/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Looking for the available sector size modes with nvme-cli (see here and here for details on NVMe)
nvme id-ns /dev/nvme0n1
return output:
NVME Identify Namespace 1:
nsze : 0x2800000
ncap : 0x2800000
nuse : 0x2800000
nsfeat : 0
nlbaf : 0
flbas : 0
mc : 0
dpc : 0
dps : 0
nmic : 0
rescap : 0
fpi : 0
dlfeat : 0
nawun : 0
nawupf : 0
nacwu : 0
nabsn : 0
nabo : 0
nabspf : 0
noiob : 0
nvmcap : 0
nsattr : 0
nvmsetid: 0
anagrpid: 0
endgid : 0
nguid : 00000000000000000000000000000000
eui64 : 0000000000000000
lbaf 0 : ms:0 lbads:9 rp:0 (in use)
Which indicates the only one sector profile exits. For comparison, the output of the same command issued for my physical NVMe returns
[...]
lbaf 0 : ms:0 lbads:9 rp:0x2 (in use)
lbaf 1 : ms:0 lbads:12 rp:0x1
Any help would be greatly appreciated!
run command to format with 4k. But beware that format will wipe out data.
nvme format /dev/nvme0n1 --lbaf=1
I have this SSD that was in another Ovirt cluster that I destroyed, and now I'm trying to re-purpose this disk but can't seem to clear it. I have tried wipefs and dd but it still shows up in fdisk -l. How can I clear all lvms from this disk?
sudo fdisk -l
Disk /dev/sda: 3.5 TiB, 3840755982336 bytes, 7501476528 sectors
Disk model: SAMSUNG MZ7LH3T8
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mmcblk0: 58.25 GiB, 62537072640 bytes, 122142720 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 61C206C7-69A3-4C51-9215-1C0B45E32661
Device Start End Sectors Size Type
/dev/mmcblk0p1 2048 788479 786432 384M Linux filesystem
/dev/mmcblk0p2 788480 1048575 260096 127M EFI System
/dev/mmcblk0p3 1048576 1050623 2048 1M BIOS boot
/dev/mmcblk0p4 1050624 122142686 121092063 57.8G Linux filesystem
[core#h2-master-01 ~]$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3.5T 0 disk
ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc_tmeta 253:0 0 15.8G 0 lvm
â ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc-tpool 253:2 0 3.5T 0 lvm
â ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc 253:3 0 3.5T 1 lvm
â ââgluster_vg_sdc-gluster_lv_data 253:4 0 3.4T 0 lvm
ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc_tdata 253:1 0 3.5T 0 lvm
ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc-tpool 253:2 0 3.5T 0 lvm
ââgluster_vg_sdc-gluster_thinpool_gluster_vg_sdc 253:3 0 3.5T 1 lvm
ââgluster_vg_sdc-gluster_lv_data 253:4 0 3.4T 0 lvm
mmcblk0 179:0 0 58.2G 0 disk
ââmmcblk0p1 179:1 0 384M 0 part /boot
ââmmcblk0p2 179:2 0 127M 0 part /boot/efi
ââmmcblk0p3 179:3 0 1M 0 part
ââmmcblk0p4 179:4 0 57.8G 0 part /sysroot
mmcblk0boot0 179:8 0 4M 1 disk
mmcblk0boot1 179:16 0 4M 1 disk
[core#h2-master-01 ~]$ sudo lvdisplay
[core#h2-master-01 ~]$ sudo vgdisplay
[core#h2-master-01 ~]$ sudo pvdisplay
wipefs -a /dev/sda with a reboot seemed looks to have cleared the disk. Thanks all!
So I tried this:
root#kali:~/Desktop/fmk# binwalk upgrade-2.4.0.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
512 0x200 LZMA compressed data, properties: 0x6D, dictionary size: 8388608 bytes, uncompressed size: 2805816 bytes
927576 0xE2758 Squashfs filesystem, little endian, version 4.0, compression:xz, size: 12316692 bytes, 2963 inodes, blocksize: 262144 bytes, created: 2015-08-04 02:40:49
And then I used the following dd:
sudo dd if=upgrade-2.4.0.bin of=pineapple.squashfs bs=1 count=12316692
And I can't unsquashfs pineapple.squashfs.
Can't find a SQUASHFS superblock on pineapple.squashfs
You have to set the offset where the squashfs is
Usage: dd [OPERAND]...
or: dd OPTION
Copy a file, converting and formatting according to the operands.
bs=BYTES read and write up to BYTES bytes at a time
cbs=BYTES convert BYTES bytes at a time
conv=CONVS convert the file as per the comma separated symbol list
count=N copy only N input blocks
ibs=BYTES read up to BYTES bytes at a time (default: 512)
if=FILE read from FILE instead of stdin
iflag=FLAGS read as per the comma separated symbol list
obs=BYTES write BYTES bytes at a time (default: 512)
of=FILE write to FILE instead of stdout
oflag=FLAGS write as per the comma separated symbol list
seek=N skip N obs-sized blocks at start of output
skip=N skip N ibs-sized blocks at start of input
status=LEVEL The LEVEL of information to print to stderr;
'none' suppresses everything but error messages,
'noxfer' suppresses the final transfer statistics,
'progress' shows periodic transfer statistics
...
So, to extract the filesystem
dd if=upgrade-2.4.0.bin of=pineapple.squashfs bs=1 skip=927576
I did it with:
binwalk -Me upgrade-2.4.0.bin
What is 'file system outputs' counting when using the Linux 'time' command with dd?
It doesn't equal dd 'count' (presumably the number of calls to fwrite?), nor the size of the output in 4096-byte pages (which should be 1024000 in this example).
An example:
> /usr/bin/time -v dd if=/dev/zero of=/tmp/dd.test bs=4M count=1000
1000+0 records in
1000+0 records out
4194304000 bytes (4.2 GB) copied, 4.94305 s, 849 MB/s
Command being timed: "dd if=/dev/zero of=/tmp/dd.test bs=4M count=1000"
User time (seconds): 0.00
System time (seconds): 4.72
Percent of CPU this job got: 95%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.94
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 5040
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 1322
Voluntary context switches: 32
Involuntary context switches: 15
Swaps: 0
File system inputs: 240
File system outputs: 8192000
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
The command time is printing out values from the rusage struct (see getrusage(2)).
And according to the source:
/*
* We approximate number of blocks, because we account bytes only.
* A 'block' is 512 bytes
*/
static inline unsigned long task_io_get_oublock(const struct task_struct *p)
{
return p->ioac.write_bytes >> 9;
}
So (at least on Linux) "File system outputs" in time output is the total number of bytes written / 512.