I'm working on an embedded system where the rootfs is constructed in a tmpfs partition by the init process. After the rootfs is complete, it will do a pivot-root and start spawning processes located in the rootfs.
But it seems like XIP is not working for our tmpfs, and all the applications is therefore loaded into ram twice (in the tmpfs and again into ram when loaded).
Can this really be true?
I found an old discussion thread at https://ez.analog.com/thread/45262 which describe the same issue as I'm seeing.
How can I achieve XIP for a file-system located in memory?
What you are attempting to do should be indeed possible (though I haven't tried it myself). The problem is simply you are not going about it the correct way. If you use the block RAM device ("brd") you can create a block device that is actually RAM presented as a block device. To enable this on your kernel (sorry you do not say which kernel you have so I will just go with the kernel 4.14), you need to enable CONFIG_BLK_DEV_RAM as well as CONFIG_BLK_DEV_RAM_DAX in your kernel configuration. They are both under "Device Drivers" -> "Block Devices". Then you create such a RAM backed block device and then create for example an ext2 or ext4 or XFS file system on it and then prepare your rootfs into that file system and then pivot-root into it. Now you are executing in a RAM backed file system which has XIP (now replaced by DAX) functionality thus executing applications should now at least in theory work correctly without creating a copy of the data and simply running it out of the RAM pages of the block RAM device.
Please do beware that such approach has limitation such as for example that kernel modules themselves will still be copied into RAM, get_user_pages() may not work, O_DIRECT may not work, and neither might RDMA, sendfile() and splice().
Some relevant things to look at include:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/Kconfig?h=v3.19#n359
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/Kconfig?h=v3.19#n396
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/blockdev/ramdisk.txt?h=v3.19
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/xip.txt?h=v3.19
Note XIP was replaced by DAX since 4.0 kernel so there see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/dax.txt?h=4.14
Also note that support for DAX was removed from block RAM driver with kernel 4.15 so you will no longer be able to do this once you move to kernel 4.15 and later... See commit 7a862fbbdec665190c5ef298c0c6ec9f3915cf45 for the reasoning behind removing the functionality.
I hope this is enough to set you on the right track and sorry about the bad news that the functionality has been removed since 4.15 kernel...
Related
I am working to implement hibernation (Suspend to disk) on ARM and have successfully done that by using swsusp ARM patch by Sebastian Capella. Now I can hibernate (suspend to swap partition in sd card) the kernel using the command echo disk > /sys/power/state and the system will resume its state with the next power on. But if I press reset again the kernel follow a normal boot sequence.
My question is how can I make that swap area and hibernate image in that area permanent, so that in every reset it will awake from that permanent image? I have given the value of swapiness=0 so that I expect there wont be any swapping of pages any more while system is alive. How kernel decide whether go for a normal boot or awake from (resume=/dev/swap_partition) hibernation?
I searched a lot on internet but didn't get a clear idea about how Linux kernel is awaking from hibernation and what it will do with swap after resuming once.Thank you for your time
My kernel version is 3.14
Here is some code trace of Linux Hibernation APIs calls:
http://www.srcmap.org/sd_share/4/839d1dea/Linux_kernel_Hibernation_Resume.html
Most of the code trace is for PowerPC. But it might give you some idea on the kernel restore-from-hibernation flow.
For ARM, maybe you need to:
On hibernation, just marks the permanent swap file as the swap file.
On resume, prevents the system to "unmarks" that swap file as hibernation file.
Be very careful about kernel image upgrade, the swap file contents are tightly couple to the kernel image. Any minor chg/recompile in kernel will mark the swap file invalid and trigger a reboot action . Add a lot of printk() logs for this.
I'm studying the process of x86-system booting
and Here is the booting flow:
BIOS load the bootsect from disk MBR to 0x7c00 memory address
boosect copy itself to 0x90000 memory address and jump to 0x90000.
boosect load setup from disk to 0x90200 memory address.
Get some system peripheral device parameters (video, root disk, keyboard,…,etc.) and jump to 0x90200.
Switch system into protected mode move kernel from 0x10000(64K) to 0x0000
Jump to 0x0000 and execute head.s for kernel boot
My question is that why we need to move bootsect itself to 0x90000 first?
Why can't we just move setup and system?
Thanks.
It was (and still is) a good practice to "shadow copy" your bootloader and jump to it. This practice began early when the typical boot loader was limited to the size of a single segment on an x86 processor and a single read sector from disk. Once interrogating the hardware a boot loader could do more advanced work, like install system files (calls, hooks, TSRs, etc), be taken over by viruses, or initialize protected mode and start performing hardware paging of applications, etc.
The origin of the "behavior" predates Linux, you should find that this behavior was common to x86 bootloaders. Possibly any computer based on the IBM PC.
The code presently in Linux was probably derived from this:
Fx. https://stuff.mit.edu/afs/sipb/user/warlord/C/memtest86/bootsect.s
In which case the choice to relocate to 0x90000 is likely arbitrary, the goal was to move the loader out of the default location into a location of its own choice where it wouldn't be tampered with by programs which might allocate from "low mem" (in effect: as a matter of practice.)
I would like to see a definite reason myself :) pretty sure it's really just a remnant of a time when the x86 platform was a DOS platform, and as the hardware evolved new tricks were employed to remain backward compatible with "unfriendly" lowmem code.
I believe that moving the boot sector out of the way was mostly a matter of convenience - there is no hard technical reason that it could not be done otherwise.
That said, 0x7c00 lies less than 32KiB from the start of the memory. 32KiB is often not enough for the setup stage of the kernel, let alone the kernel itself. 0x90000 is well under the area that is reserved by the PC BIOS, while also leaving enough space for the kernel.
In any case, the process you are referring to has not been used by the Linux kernel for several years. The addresses you mentioned are used by versions of the Linux Boot Protocol before v2.02, which was first used with linux-2.4.0. I think that the kernel itself stopped being directly bootable with linux-2.6.0 or so. The arch/i386/boot/bootsect.S file of that version would output a message to that effect when someone attempted to boot the kernel directly.
These days the kernel is usually loaded by a separate bootloader, which is free to use whatever approach it wishes as long as it complies with the boot protocol. The bootloader may have several stages and may even do kernel-y things, such as switching to protected mode itself.
I'm working with ELinux kernel on ARM cortex-A8.
I know how the bootloader works and what job it's doing. But i've got a question - why do we need bootloader, why was the bootloader born?
Why we can't directly load the kernel into RAM from flash memory without bootloader? If we load it what will happen? In fact, processor will not support it, but why are we following the procedure?
In the context of Linux, the boot loader is responsible for some predefined tasks. As this question is arm tagged, I think that ARM booting might be a useful resource. Specifically, the boot loader was/is responsible for setting up an ATAG list that describing the amount of RAM, a kernel command line, and other parameters. One of the most important parameters is the machine type. With device trees, an entire description of the board is passed. This makes a stock ARM Linux impossible to boot with out some code to setup the parameters as described.
The parameters allows one generic Linux to support multiple devices. For instance, an ARM Debian kernel can support hundreds of different board types. Uboot or other boot loader can dynamically determine this information or it can be hard coded for the board.
You might also like to look at bootloader info page here at stack overflow.
A basic system might be able to setup ATAGS and copy NOR flash to SRAM. However, it is usually a little more complex than this. Linux needs RAM setup, so you may have to initialize an SDRAM controller. If you use NAND flash, you have to handle bad blocks and the copy may be a little more complex than memcpy().
Linux often has some latent driver bugs where a driver will assume that a clock is initialized. For instance if Uboot always initializes an Ethernet clock for a particular machine, the Linux Ethernet driver may have neglected to setup this clock. This can be especially true with clock trees.
Some systems require boot image formats that are not supported by Linux; for example a special header which can initialize hardware immediately; like configuring the devices to read initial code from. Additionally, often there is hardware that should be configured immediately; a boot loader can do this quickly whereas the normal structure of Linux may delay this significantly resulting in I/O conflicts, etc.
From a pragmatic perspective, it is simpler to use a boot loader. However, there is nothing to prevent you from altering Linux's source to boot directly from it; although it maybe like pasting the boot loader code directly to the start of Linux.
See Also: Coreboot, Uboot, and Wikipedia's comparison. Barebox is a lesser known, but well structured and modern boot loader for the ARM. RedBoot is also used in some ARM systems; RedBoot partitions are supported in the kernel tree.
A boot loader is a computer program that loads the main operating system or runtime environment for the computer after completion of the self-tests.
^ From Wikipedia Article
So basically bootloader is doing just what you wanted - copying data from flash into operating memory. It's really that simple.
If you want to know more about boostrapping the OS, I highly recommend you read the linked article. Boot phase consists, apart from tests, also of checking peripherals and some other things. Skipping them makes sense only on very simple embedded devices, and that's why their bootloaders are even simpler:
Some embedded systems do not require a noticeable boot sequence to begin functioning and when turned on may simply run operational programs that are stored in ROM.
The same source
The primary bootloader is usually built in into the silicon and performs the load of the first USER code that will be run in the system.
The bootloader exists because there is no standardized protocol for loading the first code, since it is chip dependent. Sometimes the code can be loaded through a serial port, a flash memory, or even a hard drive. It is bootloader function to locate it.
Once the user code is loaded and running, the bootloader is no longer used and the correctness of the system execution is user responsibility.
In the embedded linux chain, the primary bootloader will setup and run the Uboot. Then Uboot will find the linux kernel and load it.
Why we can't directly load the kernel into RAM from flash memory without bootloader? If we load it what will happen? In fact, processor will not support it, but why are we following the procedure?
Bartek, Artless, and Felipe all give parts of the picture.
Every embedded processor type (E.G. 386EX, Coretex-A53, EM5200) will do something automatically when it is reset or powered on. Sometimes that something is different depending on whether the power is cycled or the device is reset. Some embedded processors allow you to change that something based on voltages applied to different pins when the device is powered or reset.
Regardless, there is a limited amount of something that a processor can do, because of the physical space on-processor required to define that something, whether it is on-chip FLASH, instruction micro-code, or some other mechanism.
This limit means that the something is
fixed purpose, does one thing as quickly as possible.
limited in scope and capability, typically loading a small block of code (often a few kilobytes or less) into a fixed memory location and executing from the start of the loaded code.
unmodifiable.
So what a processor does in response to reset or power-cycle cannot be changed, and cannot do very much, and we don't want it to automatically copy hundreds of megabytes or gigabytes into memory which may not exist or may not be initialized, and which could take a looooong time.
So....
We set up a small program which is smaller than the smallest size permitted across all of the devices we are going to use. That program is stored wherever the something needs it to be.
Sometimes the small program is U-Boot. Sometimes even U-Boot is too big for initial load, so the small program then in turn loads U-Boot.
The point is that whatever gets loaded by the something, is modifiable as needed for a particular system. If it is U-Boot, great, if not, it knows where to load the main operating system or where to load U-Boot (or some other bootloader).
U-Boot (speaking of bootloaders in general) then configures a minimal set of devices, memory, chip settings, etc., to enable the main OS to be loaded and started. The main OS init takes care of any additional configuration or initialization.
So the sequence is:
Processor power-on or reset
Something loads initial boot code (or U-Boot style embedded bootloader)
Initial boot code (may not be needed)
U-Boot (or other general embedded bootloader)
Linux init
The kernel requires the hardware on which you are working to be in a particular state. All the hardware you used needs to be checked for its state and initialized for its further operation. This is one of the main reasons to use a boot loader in an embedded (or any other environment), apart from its use to load a kernel image into the RAM.
When you turn on a system, the RAM is also not in a useful state (fully initialized to use) for us to load kernel into it. Therefore, we cannot load a kernel directly (to answer your question)and thus arises the need for a construct to initialize it.
Apart from what is stated in all the other answers - which is correct - in some cases the system has to go through different execution modes, take as example TrustZone for secure ARM chips. It is possible to still consider it as sort of HW initialization, but what makes it peculiar is the fact that there are additional limitations (ex: memory available) that make it impractical, if not impossible, to do everything in a single binary, thus multiple stages of bootloader are available.
Furthermore, for security reason, each of them is signed and can perform its job only if it meets the security requirements.
Using linux, I can use raw access to NAND or access to files through filesystem. So, when I need to know, where my file is really located in NAND, what should I do? I cannot found any utilities providing this feature. Moreover, I cannot detect any possibility of this, besides hacking kernel with tons of "printk" (it's not nice way, I guess).
Can anybody enlighten me on this? (I'm using YAFFS2 and JFFS2 filesystems)
You can make a copy of any partition with nanddump. Transfer that partition dump to a PC. The nandsim utility can be used to mount the partitions on a PC.
modprobe nandsim first_id_byte=0x2c second_id_byte=0xda \
third_id_byte=0x90 fourth_id_byte=0x95 parts=2,64,64
flash_erase /dev/mtd3 0 0
ubiformat /dev/mtd3 -f rootfs.ubi
This command emulates a Micron 256MB NAND flash with four partitions. If you just capture the single partition and not the whole device, don't set parts. You can also do nanddump on each partition and then concatenate them all. This code targeted mtd3 with a UbiFs partition. For JFFS2 or YAFFS2, you can try nandwrite or some other appropriate flashing utility on the PC.
How to get real file offset in NAND by file name?
The files may span several NAND sectors and they are almost never contiguous. There is not much of an advantage to keep file data together as there is no disk head that takes physical time to seek. Some flash has marginally better efficiency for sequential reads; yet other flash will give better performance for reads from another erase block.
I would turn on debug at either the MTD layer or in the filesystem. In a live system, the position of the file may migrate over time on the flash even if it is not written. This is active wear leveling.
In my ARM based custom board, I flash u-boot to NAND whenever I do changes on that. (putting some debug statements/modification). Is there any way to directly load the uboot image in RAM memory instead of flashing it every time?
For linux kernel image I do load it in memory and use bootm to boot that image. Similarly for u-boot I am trying out. Kindly provide your suggestions.
Someone at Freescale has done this, for their P1022DS evaluation system (and some others as well). They have provided a somewhat useful document about the process in the file ${UBOOTROOT}/doc/README.ramboot-ppc8500 (in U-Boot V2010.12). This document is pretty terse and leaves many questions unanswered, but I found it a reasonable place to start when I needed to debug U-Boot for a new board, before the flash memory for that board was operating correctly.
In fact, having non-functional flash memory is one reason you might want to debug U-Boot in RAM. (There are a few reasons listed in the README, and they all sound pretty reasonable to me, in contrast to some of the other advice available on this subject)
In our situation, it was found that early prototype target board hardware included an error in the address bus connection to the flash memory that prevented us from using that flash memory. While the hardware was being redesigned and re-fabricated, we wanted to continue testing/debugging those parts of our U-Boot configuration that did not depend on flash memory, for example, I2C, Ethernet, FPGA configuration, PCIe, etc. (there are plenty of things that are independent of where the U-Boot image comes from).
Running U-Boot after loading it into RAM via a JTAG interface (using Codewarrior and the USB TAP) allowed us to continue our U-Boot bring-up tasks, even though we had no functional flash memory. Once we received a newer version of the target board with correctly functioning flash memory, we returned to debugging those parts of U-Boot that we hadn't been able to test earlier. After that, U-Boot was fully functional, and we did not have to wait for a board spin to make any progress.
Debugging a bootloader is a bit difficult, but with the right tools it should be relatively painless.
I deal with the PowerPC achitecture and with a BDI-3000 I can load and debug directly to RAM (of course, after initializing the DDR controller).
One option is if you have on-chip SRAM or L2 Cache that can be configured as on-chip SRAM. The BDI can copy to the SRAM area first, u-boot does it's thing (initialize DDR controller for example), then relocates itself to DDR RAM afterwards. Definitely faster that re-writing to slow Flash all the time.
It wasn't possible in 2004, at least.
It should be possible, if the U-Boot image you want to run has startup code that allows running it from arbitrary addresses. Whether or not that is the case for your board I can't tell.
If the startup code begins by copying the code section from the current (PC-relative) address to the final execution address (usually this is preceded by a check that these areas don't overlap), then you can load the .bin file to any address in RAM, and invoke it using go.
The second obstacle I could see would be unconditional RAM setup code at the beginning, which a number of boards have.
This is what can be read on the u-boot documentation FAQ:
Question:
I don't want to erase my flash memory because I'm not sure if my new U-Boot image will work. Is it possible to configure U-Boot such
that I can load it into RAM instead of flash, and start it from my old
boot loader?
Answer:
No. (Unless you're using a Blackfin processor, or Socfpga board, but you probably aren't.)
Question:
But I've been told it is possible??
Answer:
Well, yes. Of course this is possible. This is software, so everything is possible. But it is difficult, unsupported, and fraught
with peril. You are on your own if you choose to do it. And it will
not help you to solve your problem.
source:http://www.denx.de/wiki/view/DULG/CanUBootBeConfiguredSuchThatItCanBeStartedInRAM
The problem here is that what you are trying to do goes against the philosophy of what a bootloader is. Most processors require that code starts from Flash. That code is called a bootloader. That is what U-boot is.
However, if you want to modify U-boot so that it is not a true bootloader, then you can do whatever you want. It's just software. But don't expect any mainline support for the above reasons.
Just take in mind (be care of) the hardware that you are configuring in your modified U-Boot. U
Boot is intended to initialize critical modules, some of them are not able to be re-configured on the fly or they may not performe as if they were initialized/configured at startup.
If your Target board support network booting, you can load uboot image from host machine to RAM through network.
You can use usb boot. TI and Freescale provides their usb boot utilities. I don't know about other vendors.
Yes, It is possible most of the compilation structure at the end U-Boot provides a u-boot.bin file which is a flattened binary, if your target supports USB/TFTP or any other medium which current U-boot can detect on you target environment then we can load the u-boot.bin file to the static memory address area. This address is the entry point of U-Boot Code and U-boot can execute the flattened binaries by go 0x<memory_address>. The static memory address area can be deduced form u-boot.map file, This entry point is basically address to the .text area of compiled program usually can we searched in the .map file with string "Address of section .text set to 0x." Below is the example of doing it from USB.
usb start
load usb 0x<memory_address> u-boot.bin
go 0x<memory_address>
This should run you U-Boot from usb with out disturbing current code.