Using linux, I can use raw access to NAND or access to files through filesystem. So, when I need to know, where my file is really located in NAND, what should I do? I cannot found any utilities providing this feature. Moreover, I cannot detect any possibility of this, besides hacking kernel with tons of "printk" (it's not nice way, I guess).
Can anybody enlighten me on this? (I'm using YAFFS2 and JFFS2 filesystems)
You can make a copy of any partition with nanddump. Transfer that partition dump to a PC. The nandsim utility can be used to mount the partitions on a PC.
modprobe nandsim first_id_byte=0x2c second_id_byte=0xda \
third_id_byte=0x90 fourth_id_byte=0x95 parts=2,64,64
flash_erase /dev/mtd3 0 0
ubiformat /dev/mtd3 -f rootfs.ubi
This command emulates a Micron 256MB NAND flash with four partitions. If you just capture the single partition and not the whole device, don't set parts. You can also do nanddump on each partition and then concatenate them all. This code targeted mtd3 with a UbiFs partition. For JFFS2 or YAFFS2, you can try nandwrite or some other appropriate flashing utility on the PC.
How to get real file offset in NAND by file name?
The files may span several NAND sectors and they are almost never contiguous. There is not much of an advantage to keep file data together as there is no disk head that takes physical time to seek. Some flash has marginally better efficiency for sequential reads; yet other flash will give better performance for reads from another erase block.
I would turn on debug at either the MTD layer or in the filesystem. In a live system, the position of the file may migrate over time on the flash even if it is not written. This is active wear leveling.
Related
I'm working on an embedded system where the rootfs is constructed in a tmpfs partition by the init process. After the rootfs is complete, it will do a pivot-root and start spawning processes located in the rootfs.
But it seems like XIP is not working for our tmpfs, and all the applications is therefore loaded into ram twice (in the tmpfs and again into ram when loaded).
Can this really be true?
I found an old discussion thread at https://ez.analog.com/thread/45262 which describe the same issue as I'm seeing.
How can I achieve XIP for a file-system located in memory?
What you are attempting to do should be indeed possible (though I haven't tried it myself). The problem is simply you are not going about it the correct way. If you use the block RAM device ("brd") you can create a block device that is actually RAM presented as a block device. To enable this on your kernel (sorry you do not say which kernel you have so I will just go with the kernel 4.14), you need to enable CONFIG_BLK_DEV_RAM as well as CONFIG_BLK_DEV_RAM_DAX in your kernel configuration. They are both under "Device Drivers" -> "Block Devices". Then you create such a RAM backed block device and then create for example an ext2 or ext4 or XFS file system on it and then prepare your rootfs into that file system and then pivot-root into it. Now you are executing in a RAM backed file system which has XIP (now replaced by DAX) functionality thus executing applications should now at least in theory work correctly without creating a copy of the data and simply running it out of the RAM pages of the block RAM device.
Please do beware that such approach has limitation such as for example that kernel modules themselves will still be copied into RAM, get_user_pages() may not work, O_DIRECT may not work, and neither might RDMA, sendfile() and splice().
Some relevant things to look at include:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/Kconfig?h=v3.19#n359
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/block/Kconfig?h=v3.19#n396
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/blockdev/ramdisk.txt?h=v3.19
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/xip.txt?h=v3.19
Note XIP was replaced by DAX since 4.0 kernel so there see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/dax.txt?h=4.14
Also note that support for DAX was removed from block RAM driver with kernel 4.15 so you will no longer be able to do this once you move to kernel 4.15 and later... See commit 7a862fbbdec665190c5ef298c0c6ec9f3915cf45 for the reasoning behind removing the functionality.
I hope this is enough to set you on the right track and sorry about the bad news that the functionality has been removed since 4.15 kernel...
I tried to use fsync to write some file to SD card ASAP. However fsync does not actually block before the file is physically written to the SD card. It seems to take about 5-6 seconds before the data is actually on the SD card. However mount the file system (I tried ext3, ext4) with commit = 1 or sync option does seem to work, the data is safe after reboot in 1 second. My question is that is there anyway to achieve flushing without resort to partition wide solution? I'm using linux kernel 2.6.37. Thank you
If you want to be sure the content is written on the SD card, you should call blockdev with --flushbufs before exiting the program.
If you want to benchmark the writing process, you can call this after every write.
/sbin/blockdev --flushbufs $dev
Is there a way to purposefully corrupt a FAT file system using only Win32 calls or must you do it at lower level? We're encountering FAT corruption on a WinCE 5.0 device and have written a utility to detect and attempt to correct it, but don't have a means to create FAT corruption on demand. Thanks.
The media is a CF card, but it's not removable as a normal course as it's mounted internal to the device.
What's the FAT on (e.g. inserted USB, on-board flash, etc)? That's going to make a large difference.
If it's the on-board flash, you need to get underneath the file system driver (FSD), which is typically going to be the flash driver itself. It's going to have access to the raw flash sectors (it's what the FSD uses for its reads and writes) either through Xxx_Write or Xxx_Ioctl. Exactly how it works is going to depend on the flash driver in use, so looking at the driver source is your best path.
You can access device data as a raw file and write random data in that file to corrupt FAT. E.g. if you write random data on:
\?\Device\HarddiskVolume1
This would corrupt first partition.
This page has some hints on how to figure out paths for HDD/USB drive etc.
http://www.chrysocome.net/dd
Right, so I've got a Ramdisk image on a mounted device at, say, /mnt/sda1/Ramdisk.img . I want to copy the raw data directly to a ramdisk device at /dev/ram0, in such a way that there will be no need to use mke2fs: I could mount the image and device and find that both are identical.
Ideally this would use only linux commands.
My ideas so far: Mounting the image on a loopback and using basic IO to put the data from one device into the other. I could probably do this with C.
Also, somehow finding the physical address of the ramdisk device and using linux or C commands to put the file directly into the memory.
Thanks in advance.
P.S: I know that the kernel somehow flags memory which was used for ramdisks so the system does not reclaim it. Anyone know where these flags are?
Any kind of copy command is going to require a partition and filesystem at the destination. However, dd can work at the disk or partition level, by just specifying the disk (eg /dev/sda) or partition (eg (dev/sda1) level. So in theory, you could do
dd if=ramdisk_image_file of=/dev/ram0
I'm tasked with writing a reader program for windows that is able read an ext2 partition.
For my testing I'm using a drive I formatted to ext2 and a file I created using mkfs (a file that does mount and work well under linux)
For some reason when I read the superblock from the drive (the real one) I get all the right meta-data (i.e. block size, inode count etc..) but doing the exact same thing to the file returns bad results (which make no sense).
is there a difference between the 2?
I open the drive using \.\X:
and I make the file using mkfs.
There shouldn't be any difference between ext2 on a partition and stored within a file (and indeed there isn't; I just checked); however, IIRC, the offset of the primary superblock is 2048 instead of 1024, if ext2 is installed on a bare disk (e.g. /dev/sda instead of /dev/sda1). This is to accomodate the MBR and other junk. (I can't find it in the docs from skimming just now, but this sticks out in my mind as something I ran into.) However, it's somewhat unusual to install to a bare drive, so I doubt this is your problem.
I wrote some ext2 utilities a few years ago, and after starting writing it by hand, I switched to using Ted Ts'o (the ext2 filesystem creator)'s e2fsprogs, which come with headers/libraries/etc. for doing all this in a more flexible and reliable fashion.
You may also want to check at offset 0x438 into the file/partition for the magic number 0xEF53, and consider it not an ext2/3 filesystem if that's not there, before pulling in the entire superblock, just as a sanity check.
Here's some docs that will probably be helpful: http://www.nongnu.org/ext2-doc/ext2.html