I have a customized Linux filesystem with some binaries and new folders in the root file system. This Linux filesystem is created for small board computer.
Currently, I compress the root folder of the customized filesystem to a tar.gz file. With this tar.gz file I can share it to my friends. Then this file have to be extracted to their SD card. With this method they can also update a libraries or binary for testing.
However, this mechanism (creating and deploying) takes a lot of time.
My questions are:
1. How can I improve the creating and deploying customized linux image?
2. If I see the linux distributions, they use .iso or .img format. What is the reason using .iso or .img instead of tar.gz or zip file?
Thanks.
You have two steps to speed up:
Create an image that is shareable with your friends
Deploy the image on the board to test it
1. Step
Automate as much as you can. You could even set up a Jenkins server which should be able to automate anything that looks like a repetitive job. (compiling, compressing, uploading or sending the image to friends)
Compress your image using all your cores. AFAIK gzip does only use one core. pigz should compress/decompress faster
2. Step: Run your rootfs from an NFS.
This is the real time saver because you don't need to copy your rootfs to a sdcard anymore:
If your bootloader supports it, use nfs/tftp boot. Your board then boots directly from a network mount instead of a sdcard. I dont know which bootloader you use, but u-boot supports it, and others probably do as well.
it depends from what exactly need to be updated. If this is entire filesystem that is always changing - then just distribute the compressed img (by xz for example). if this is some small part of the system - just use the package manager, i.e. extract this part to package, next create this new package using the standard distro tools and distribute only it.
So if your SD card image has the fixed size - you can use rsync/xdelta to distribute only changes, but their efficiency depends from the exact changes percentage.
Related
We are using meta-intel layer to generate .hddimg.
We can flash the generated .hddimg using Bootable Thumb Drive, but using this mechanism in manufacturing is not a good solution.
We need some way to flash the Yocto Image using clonezilla, hddimg is not recognized by clonezilla.
Do we need to add any IMAGE_FSTYPES to make it work. Or is there any other solutions to flash Yocto Images over PXE Boot
Thanks for your time and patience
IMAGE_FSTYPES just sets the build output image extension. With the IMAGE_FSTYPES variable, you can get partitioned image or unpartitoned image separetly.
Also you can create a recipe then, you can generate a new partitioned image according to the your needs but, As I know clonezilla can handle the raw images("dd") and yocto can able to generate raw images. I use the raw image most of the time and clonezilla can handle the raw images.
Am trying to build small OS using buildroot & able to generate iso hybrid image to boot from USB. Generated iso image is working fine as live CD but not able to install it inside hard disk (like regular OS image).
I have tried to modify /init but need more guidance for doing so. Any help here will be much appreciated.
To install on a hard disk, you want a regular root filesystem + kernel. Select:
BR2_LINUX_KERNEL_INSTALL_TARGET "Install kernel image to /boot in target" to make the kernel part of the rootfs.
grub2 as the bootloader;
ext4 as the root filesystem;
host-genimage to create a partitioned hard disk image
You also need to supply:
a grub.cfg to configure grub;
a post-build script to copy the grub.cfg to the right place;
a genimage.cfg to configure the hard disk image.
Look at configs/pc_x86_64_efi_defconfig for inspiration. You might even be able to use it directly.
Is there anyway to access files from usb without mounting it (in a terminal)?
copying or accessing files without mounting it!!
First of all: I think it would be better to post this question on "superuser.com", not on "stackoverflow.com" because this is a question about usage of Linux, not about programming for Linux.
In the 1990s there was a tool which allowed you to read FAT-formatted media (floppys and hard disk partitions) under Linux.
The tool was intended for older Linux versions that did not have FAT file system support.
Because today all Linux versions have FAT file system support I think that tool is no longer maintained however the source codes should still be available somewhere!
FAT32 did not exist, yet, the time when this tool was used so the tool only supported FAT12 and FAT16.
To access the media "root" rights are required in any case.
As part of my infrastructure I have many Virtual Machines running different Linux distros, under Proxmox using OpenVz. My problem is that I need to export into a personalized installable ISOs some of the VMs I have, (installable snapshots of the current state of the VMs), some of them are running Ubuntu, some of them CentOS, so my question is:
1- Is there a way I can do this aware of the OS the VM is running?,
2- Exporting VMs to ISOs the way I just explained is the way to go or is there any other approach?
I'm open to any advice from those how has experience with this subject even if I have to setup different Virt. Technology to host the VMs.
Your question is pretty vague on your requirements. I'll try to give you some ideas:
What do you mean by "Current state"? If you really want all the running processes, then you should something like VirtualBox and take a snapshot. You can easily boot that up on another computer and continue running where you left off, and it's independent of the OS.
If you really mean just the filesystem, then just copying the filesystem and burning it on a CD is unlikely to give you good results. For instance, there are many areas that are expected to be writable (/var, /tmp. even /etc for /etc/resolv.conf)
One simple idea is to just 'tar' up the filesystem, and untar it on another OpenVz distro. (I'm sure someone has made a bootable OpenVz distro..)
If you want a real bootable ISO, there are a LOT of different options. For example, you could have the kernel mount the ISO as root. Or you could boot to a RAMDisk as root, and unpack the filesystem you need. Or you could mount the ISO as root with an AUFS overlay filesystem. Or you could mount some directories as a SquashFS filesystem onto a RAM root.
But if you really want simplicity in "moving VMs around", look into Docker. It has a simple way to push a filesystem up to a public or private server, then download it on the other side, but save bandwidth on common elements like the OS and Apache installs. (If you do it right.)
Can anyone tell how I might best mirror selected files and folders to a NAS, (Network Addrssable Storage) box from a Linux workstation in real-time?
These are very large files, (> 50GB) and are being continually modified, so I would only like to change those portions of the files that have been changed, added or deleted.
FYI: These files are actually Virtual Box virtual hard disk (VDI) files.
I discovered that my Synology DS211J NAS can run an RSync service. So I enabled that and used lsyncd for the live mirror... the VirtualBox VMs... all works very well.
Rsync only synchronises the parts of files that have change and so is very efficient at synchronising large files.
Of the solutions that #awm mentioned, only drbd provides block-level, realtime synchronization. The other tools will meet your goal of only propagating deltas, but they operate asynchronously. In fact, rsync will work just as well in this case, since you're not trying to provide bi-directional synchronization.
For drbd to provide block-level replication, you need need to install the drbd kernel modules and userspace tools on both the workstation on the NAS...which means this solution is only appropriate if your NAS is actually a fairly generic Linux box over which you have a great deal of control.
Before hand I just want to suggest that you don't do this. You can easily bottlenet your network and NAS and cause all sorts of problem on your host.
That being said, these claim they can do it:
Unison can be found at: http://www.cis.upenn.edu/~bcpierce/unison/
PeerSoft can do it too: http://www.peersoftware.com/products/peersync/peersyncserver/overview.aspx
Maybe - http://www.drbd.org/