There are a lots of question teaching how to create a DMG file from Linux. But none of them is clear about how to add compression to it.
I usually create a DMG package to redistribute to MacOS, but I would like to add compression as Apple specifies.
Did anyone have a chance to try a tool that supports compression during DMG packing?
Similar questions without compression:
How can I generate a DMG file from a folder in Linux?
How to create dmg file in Centos through command line
It depends. Here is a good analysis of the .dmg format. From that document, the compression is specified for each block chunk:
Table: DMG blxx types
Type Scheme Meaning
0x00000000 --- Zero-Fill
0x00000001 UDRW/UDRO RAW or NULL compression (uncompressed)
0x00000002 --- Ignored/unknown
0x80000004 UDCO Apple Data Compression (ADC)
0x80000005 UDZO zLib data compression
0x80000006 UDBZ bz2lib data compression
0x7ffffffe --- No blocks - Comment: +beg and +end
0xffffffff --- No blocks - Identifies last blxx entry
To convert the gist of #Valerio's comment into a proper answer so nobody else misses it like I almost did:
According to this answer by #uckelman:
instal libdmg-hfsplus
Run dmg uncompressed.dmg compressed.dmg
Related
I'm trying to analyze and understand an embedded system that runs linux.
I'm pretty new to linux boot process, embedded linux etc, so I have to learn a lot but prefer learning by doing. So analyzing a system is the way I try to understand all this.
I already got some insight and learnt a lot in just looking at the bootmsg, checking some scripts and provided files (firmware) and trying to understand what's going on by just searching for answers here and elsewhere in the net. This might also be the reason for some wrong expressions I might use, I hope you can understand anyways. Atm I am not able to ask the creator of this system directly, so I hope you might help me out with some answers here.
So the System got a SOC (Marvell Armada-370 88F6707-A1) with some Flash (128 MiB - Hynix NAND (HY27U1G8F2BTR)) and DRAM (512 MB) and seems to load the bootload U-Boot from SPI flash (1MB).
As mentioned above as U-Boot is used as bootloader which then loads some Linux version 3.2.34. I think I already understand quite a bit of the general boot process here, but got some questions depending the provided bootargs.
Following is an excerpt of the printenv command (U-Boot environment)
image_name=uImage
mtdids=nand0=nand0
mtdparts=mtdparts=nand0:8m(kernel),6m(Initrd),-(rootfs)
select0=nand info
load0=nand read.e 0x02000000 0 360000
loadr0=nand read.e 0x04000000 800000 300000
boot=bootm 0x02000000 0x4000000
bootcmd=run beep select0 load0 loadr0 boot || run beep beep beep errled
bootargs=console=ttyS0,115200 root=ubi0:rootfs ubi.mtd=3,2048 initrd=0x12000000 rootfstype=ubifs
beep=beep
I see that they load the kernel and compressed ramdisk content into ram load0=nand read.e 0x02000000 0 360000 and loadr0=nand read.e 0x04000000 800000 300000. The kernel at start looks at 0x04000000 for the ramdisk_image and uncompresses to use the content for initial rootfs. Then the usual process with linuxrc / init begins... at the end the normal file system from nand (here rootfs partition of ubi device 0) is loaded as stated in kernel command line (root=ubi0:rootfs ubi.mtd=3,2048 rootfstype=ubifs). This is what I understood is happening here and might be pretty straight forward.
What I'm wondering now is the following bootargs part initrd=0x12000000. I don't quite get why they provide a different address here when we already let the kernel know about the real of the compressed ramdisk (as a second parameter of bootm).
I started the system with and without this parameter and it seems there are no differences.
As in my understanding of reading about that initrd= argument is that it seems to do the same as the second parameter that is already passed to bootm, it tells the kernel where to find an initrd to load the initial rootfs.
Could anyone explain why we here pass two different locations to an initrd to the kernel? Is this just some obsolete stuff that was forgotten to remove or is there a reason for?
Thanks in advance for your help.
I hope I haven't missed anything and the question wasn't already answered here somewhere, but I couldn't find anything.
edit-2:
Thanks to #sawdust in the comments I understood better what actually is going on here and saw the importance to add some more information to the question so an answer will be clearer to understand.
Here is a short excerpt of the bootmsg
> bootm 0x02000000 0x4000000
## Booting kernel from Legacy Image at 02000000 ...
Image Name: Linux-3.2.34
Created: 2013-08-15 4:18:54 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3464064 Bytes = 3.3 MB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 04000000 ...
Image Name:
Created: 2013-09-10 10:38:37 UTC
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 3030739 Bytes = 2.9 MB
Load Address: 12000000
Entry Point: 12000000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...
Uncompressing Linux... done, booting the kernel.
I also always missed some information in this part of bootm that might have already gave a hint on what's happening here. And might have in correlation to the presence of same address should have lead to the path of answer.
edit-1: Added some more information about the hardware
edit-2: Added bootmsg for more information
I have a script that needs to work on multiple platforms and machines. Some of those machines don't have any available archiving software (e.g. zip, tar). I can't download any software onto these machines.
The script creates a directory containing output files. I need to package all those files into a single file so i can download it easily.
What is the simplest possible archiving format to implement, so I can easily roll my own impl in the script. It doesn't have to support compression.
I could make up something ad-hoc, e.g.
file1 base64EncodedContents
dir1/file1 base64EncodedContents
etc.
However if one already exists then that will save me having to roll my own packing and unpacking, only packing, which would be nice. Bonus points if it's zip compatible, so that I can try zipping it with compression if possible, and them impl my own without compression otherwise, and not have to worry about which it is on the other side.
The tar archive format is extremely simple - simple enough that I was able to implement a tar archiver in powershell in a couple of hours.
It consists of a sequence of file header, file data, file header, file data etc.
The header is pure ascii, so doesn't require any bit manipulation - you can literally append strings. Once you've written the header, you then append the file bytes, and pad it with nil chars till it's a multiple of 512 bytes. You then repeat for the next file.
Wikipedia has more details on the exact format: https://en.wikipedia.org/wiki/Tar_(computing).
I'm working on a website that is supposed to store compressed archive files for downloading, for different platforms (Mac and Windows).
Unfortunately, the Mac version of the download uses "resource forks", which I understand is a vendor-specific feature of the MacOS file system that attaches extra data to a file identifier. Previously, the only solution was to create the Mac archive (at that time a .sit archive, specifically) on a Mac, and manually upload both versions.
I would now like to let the website accept only the Windows file (a regular .zip that can be decompressed on any file-system), and generate a Mac archive with resource forks automatically. Basically, all I need is some way to generate an archive file on the Linux server (in any reasonably common format that can support resource forks; not sure if .sit is still the best option) that will yield the correct file structure when decompressed on Mac. As the file system doesn't support forks, the archive probably has to be assembled in memory and written to disk, rather than using any native compression tool.
Is there some software that can do this, or at least some format specification that would allow implementing it from scratch?
(1) Resource (and other "named") forks are legacy technology in macOS. While still supported, no modern software uses resource forks for anything substantial. I'd first suggest reviewing your requirements to see if this is even necessary anymore.
(2) macOS has long settled on .zip as the standard / built-in archive format. .sit was a third-party compression application (StuffIt) that has drifted out of favor.
(3) Resource forks are translated to non-native filesystems using a naming convention. For example, lets say the file Chart.jpg has a resource fork. When macOS writes this to a filesystem that doesn't support named forks it creates two files: Chart.jpg and ._Chart.jpg, with the latter containing the resource fork and metadata. Typically all that's required is for the .zip file to contain these two files and macOS unarchiving utility will reassemble the original file, with both forks.
I found some files with resource forks and compressed them using macOS's built-in compression command. Here's the content of the archive (unzip -v Archive.zip):
Archive: /Users/james/Development/Documentation/Archive.zip
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
1671317 Defl:N 1108973 34% 12-19-2009 12:09 b1b6083c svn-book.pdf
0 Stored 0 0% 01-30-2018 12:59 00000000 __MACOSX/
263 Defl:N 157 40% 12-19-2009 12:09 9802493b __MACOSX/._svn-book.pdf
265 Defl:N 204 23% 06-01-2007 23:49 88130a77 Python Documentation.webloc
592 Defl:N 180 70% 06-01-2007 23:49 f41cd5d1 __MACOSX/._Python Documentation.webloc
-------- ------- --- -------
1672437 1109514 34% 5 files
So it appears that the special filenames are being sequestered in an invisible __MACOSX subfolder. All you'd have to do is generate a .zip file with the same structure and it would be reassembled on a macOS system into a native file with a resource fork.
I have a great amplifier that sadly no longer gets any support (a Kickstarter gone bad). It can however be upgraded with an .img file. The unit runs Linux and the OS is installed on some kind of embedded storage (not a disk drive)
I have tried writing the img to an SD card with no success. A lot of Kickstarter backers will be very happy if we can figure this one out.
Thanks
Update:
$ file usbupgrade_O.0.31.69.img
usbupgrade_O.0.31.69.img: data
Using binwalk as suggested by #duskwuff gave the following info:
$ binwalk usbupgrade_O.0.31.69.img
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
4096 0x1000 uImage header, header size: 64 bytes, header CRC: 0xC2CEE548, created: 2016-04-19 06:16:47, image size: 3219180 bytes, Data Address: 0x10008000, Entry Point: 0x10008000, data CRC: 0x47F1092, OS: Linux, CPU: ARM, image type: OS Kernel Image, compression type: none, image name: "Linux-3.0.35"
20871 0x5187 gzip compressed data, maximum compression, from Unix, NULL date (1970-01-01 00:00:00)
5246976 0x501000 OpenSSL encryption, salted, salt: 0x-187DE192-39315650
130464447 0x7C6BABF StuffIt Deluxe Segment (data): f$
148645351 0x8DC25E7 VMware4 disk image
So I guess it is an uImage. I´m a bit surprised of the VMware4 disk image, is this normal to include in this kind of images?
Is there any C# way to check an ISO file is valid or not i.e. valid Iso format or any other check possible or not.
The scenario is like, if any text file(or any other format file) is renamed to ISO and given it for further processing. I want to check weather this ISO file is a valid ISO file or not? Is there any way exist programmatically like to check any property of the file or file header or any other things
Thanks for any reply in advance
To quote the wiki gods:
There is no standard definition for ISO image files. ISO disc images
are uncompressed and do not use a particular container format; they
are a sector-by-sector copy of the data on an optical disc, stored
inside a binary file. ISO images are expected to contain the binary
image of an optical media file system (usually ISO 9660 and its
extensions or UDF), including the data in its files in binary format,
copied exactly as they were stored on the disc. The data inside the
ISO image will be structured according to the file system that was
used on the optical disc from which it was created.
reference
So you basically want to detect whether a file is an ISO file or not, and not so much check the file, to see if it's valid (e.g. incomplete, corrupted, ...) ?
There's no easy way to do that and there certainly is not a C# function (that I know of) that can do this.
The best way to approach this is to guess the amount of bytes per block stored in the ISO.
Guess, or simply try all possible situations one by one, unless you have an associated CUE file that actually stores this information. PS. If the ISO is accompanied by a same-name .CUE file then you can be 99.99% sure that it's an ISO file anyway.
Sizes would be 2048 (user data) or 2352 (raw or audio) bytes per block. Other sizes are possible as well !!!! I just mentioned the two most common ones. In case of 2352 bytes per block the user data starts at an offset in this block. Usually 16 or 24 depending on the Mode.
Next I would try to detect the CD/DVD file-systems. Assume that the image starts at sector 0 (although you could for safety implement a scan that assumes -150 to 16 for instance).
You'll need to look into specifics of ISO9660 and UDF for that. Sectors 16, 256 etc. will be interesting sectors to check !!
Bottom line, it's not an easy task to do and you will need to familiarize yourself with optical disc layouts and optical disc file-systems (ISO9660, UDF but possibly also HFS and even FAT on BD).
If you're digging into this I strongly suggest to get IsoBuster (www.isobuster.com) to help you see what the size per block is, what file systems there are, to inspect the different key blocks etc.
In addition to the answers above (and especially #peter's answer): I recently made a very simple Python tool for the detection of truncated/incomplete ISO images. Definitely not validation (which as #Jake1164 correctly points out is impossible), but possibly useful for some scenarios nevertheless. It also supports ISO images that contain Apple (HFS) partitions. For more details see the following blog post:
Detecting broken ISO images: introducing Isolyzer
And the software's Github repo is here:
Isolyzer
You may run md5sum command to check the integrity of an image
For example, here's a list of ISO: http://mirrors.usc.edu/pub/linux/distributions/centos/5.4/isos/x86_64/
You may run:
md5sum CentOS-5.4-x86_64-LiveCD.iso
The output is supposed to be the same as 1805b320aba665db3e8b1fe5bd5a14cc, which you may find from here:
http://mirrors.usc.edu/pub/linux/distributions/centos/5.4/isos/x86_64/md5sum.txt