Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 6 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
Why is the total in the output of ls -l printed as 64 and not 26078 which is the total of all files listed?
$ ls -l ~/test/ls
total 64
-rw-r--r-- 1 root root 15276 Oct 5 2004 a2ps.cfg
-rw-r--r-- 1 root root 2562 Oct 5 2004 a2ps-site.cfg
drwxr-xr-x 4 root root 4096 Feb 2 2007 acpi
-rw-r--r-- 1 root root 48 Feb 8 2008 adjtime
drwxr-xr-x 4 root root 4096 Feb 2 2007 alchemist
You can find the definition of that line in the ls documentation for your platform. For coreutils ls (the one found on a lot of Linux systems), the information can be found via info coreutils ls:
For each directory that is listed, preface the files with a line
`total BLOCKS', where BLOCKS is the total disk allocation for all
files in that directory.
The Formula: What is that number?
total int = Sum of (physical_blocks_in_use) * physical_block_size/ls_block_size) for each file.
Where:
ls_block_size is an arbitrary environment variable (normally 512 or 1024 bytes) which is freely modifiable with the
--block-size=<int> flag on ls, the POSIXLY_CORRECT=1 GNU
environment variable (to get 512-byte units), or the -k flag to force
1kB units.
physical_block_size is the OS dependent value of an internal block interface, which may or may not be connected to the underlying hardware. This value is normally 512b or 1k, but is completely dependent on OS. It can be revealed through the %B value on stat or fstat. Note that this value is (almost always) unrelated to the number of physical blocks on a modern storage device.
Why so confusing?
This number is fairly detached from any physical or meaningful metric. Many junior programmers haven't had experience with file holes or hard/sym links. In addition, the documentation available on this specific topic is virtually non-existent.
The disjointedness and ambiguity of the term "block size" has been a result of numerous different measures being easily confused, and the relatively deep levels of abstraction revolving around disk access.
Examples of conflicting information: du (or ls -s) vs stat
Running du * in a project folder yields the following: (Note: ls -s returns the same results.)
dactyl:~/p% du *
2 check.cc
2 check.h
1 DONE
3 Makefile
3 memory.cc
5 memory.h
26 p2
4 p2.cc
2 stack.cc
14 stack.h
Total: 2+2+1+3+3+5+26+4+2+14 = 62 Blocks
Yet when one runs stat we see a different set of values. Running stat in the same directory yields:
dactyl:~/p% stat * --printf="%b\t(%B)\t%n: %s bytes\n"
3 (512) check.cc: 221 bytes
3 (512) check.h: 221 bytes
1 (512) DONE: 0 bytes
5 (512) Makefile: 980 bytes
6 (512) memory.cc: 2069 bytes
10 (512) memory.h: 4219 bytes
51 (512) p2: 24884 bytes
8 (512) p2.cc: 2586 bytes
3 (512) stack.cc: 334 bytes
28 (512) stack.h: 13028 bytes
Total: 3+3+1+5+6+10+51+8+3+28 = 118 Blocks
Note: You can use the command stat * --printf="%b\t(%B)\t%n: %s bytes\n" > to output (in order) the number of blocks, (in parens) the size of those
blocks, the name of the file, and the size in bytes, as shown above.
There are two important things takeaways:
stat reports both the physical_blocks_in_use and physical_block_size as used in the formula above. Note that these are values based on OS interfaces.
du is providing what is generally accepted as a fairly accurate estimate of physical disk utilization.
For reference, here is the ls -l of directory above:
dactyl:~/p% ls -l
**total 59**
-rw-r--r--. 1 dhs217 grad 221 Oct 16 2013 check.cc
-rw-r--r--. 1 dhs217 grad 221 Oct 16 2013 check.h
-rw-r--r--. 1 dhs217 grad 0 Oct 16 2013 DONE
-rw-r--r--. 1 dhs217 grad 980 Oct 16 2013 Makefile
-rw-r--r--. 1 dhs217 grad 2069 Oct 16 2013 memory.cc
-rw-r--r--. 1 dhs217 grad 4219 Oct 16 2013 memory.h
-rwxr-xr-x. 1 dhs217 grad 24884 Oct 18 2013 p2
-rw-r--r--. 1 dhs217 grad 2586 Oct 16 2013 p2.cc
-rw-r--r--. 1 dhs217 grad 334 Oct 16 2013 stack.cc
-rw-r--r--. 1 dhs217 grad 13028 Oct 16 2013 stack.h
That is the total number of file system blocks, including indirect blocks, used by the listed files. If you run ls -s on the same files and sum the reported numbers you'll get that same number.
Just to mention - you can use -h (ls -lh) to convert this in human readable format.
Related
After using arm-none-eabi-gcc to build a file in ELF format, I am using arm-none-eabi-objcopy to create an S-record file. The command that my makefile runs is:
$(TOOLCHAIN)-objcopy --srec-len 10 -O srec "$<" "$#"
The makefile can build with various different settings - with debug symbols, with optimization, and with neither.
With some information such as my username removed, the output of ls -la after doing all three builds is:
-rw-r--r-- 1 4096 270330 Oct 12 18:13 outfile_Debug.mot
-rw-r--r-- 1 4096 825888 Oct 12 18:13 outfile_Debug.out
-rw-r--r-- 1 4096 270334 Oct 12 17:06 outfile_Default.mot
-rw-r--r-- 1 4096 465928 Oct 12 17:06 outfile_Default.out
-rw-r--r-- 1 4096 184776 Oct 12 19:02 outfile_Optimized.mot
-rw-r--r-- 1 4096 395672 Oct 12 19:02 outfile_Optimized.out
Now, I have read an unsourced claim that srec files canot contain debugging information, which would explain why the Default and Debug .mot files are roughly the same size while the corresponding .out file sizes differ enormously. But otherwise, the ELF file is a binary representation of the executable, while the S-record file uses hex strings in ASCII text, so surely it should be larger than the binary ELF file for a non-debug build?
I am trying to obtain a backup of 'newly' added files to a Fedora system. Files can be copied through a Windows Samba share and appear to retain the original created timestamp. However, because it retains this timestamp I am having issues identifying which files were newly added to the system.
Currently, the only way I can think of doing this is to have a master list snapshot of all the files on the system at a specific time. Then when I perform the backup I compare the previous snapshot with a current snapshot. It would detect files that were removed from the system but it seems excessive and I was thinking there must be an easier way to backup newly added files.
Terry
Try using find. Something like this:
find . -ctime -10
That will give you a list of files and directories, starting from within your current directory, that has had its state changed within the last 10 days.
Example:
My Downloads directory looks like this:
kobus#akira:~/Downloads$ ll
total 2025284
drwxr-xr-x 4 kobus kobus 4096 Nov 4 11:25 ./
drwxr-xr-x 41 kobus kobus 4096 Oct 30 09:26 ../
-rw-rw-r-- 1 kobus kobus 8042383 Oct 28 14:08 apache-maven-3.3.3- bin.tar.gz
drwxrwxr-x 2 kobus kobus 4096 Oct 14 09:55 ELKImages/
-rw-rw-r-- 1 kobus kobus 1469054976 Nov 4 11:25 Fedora-Live-Workstation-x86_64-23-10.iso
-rw------- 1 kobus kobus 351004 Sep 21 14:07 GrokConstructor-master.zip
drwxrwxr-x 11 kobus kobus 4096 Jul 11 2014 jboss-eap-6.3/
-rw-rw-r-- 1 kobus kobus 183399393 Oct 19 16:26 jboss-eap-6.3.0-installer.jar
-rw-rw-r-- 1 kobus kobus 158177216 Oct 19 16:26 jboss-eap-6.3.0.zip
-rw-rw-r-- 1 kobus kobus 71680110 Oct 13 13:51 jre-8u60-linux-x64.tar.gz
-rw-r--r-- 1 kobus kobus 4680 Oct 12 12:34 nginx-release-centos-7-0.el7.ngx.noarch.rpm
-rw-r--r-- 1 kobus kobus 3479765 Oct 12 14:22 ngx_openresty-1.9.3.1.tar.gz
-rw------- 1 kobus kobus 16874455 Sep 15 16:49 Oracle_VM_VirtualBox_Extension_Pack-5.0.4-102546.vbox-extpack
-rw-r--r-- 1 kobus kobus 7505310 Oct 6 10:29 sublime_text_3_build_3083_x64.tar.bz2
-rw------- 1 kobus kobus 41467245 Sep 7 10:37 tagspaces-1.12.0-linux64.tar.gz
-rw-rw-r-- 1 kobus kobus 42658300 Nov 4 10:14 tagspaces-2.0.1-linux64.tar.gz
-rw------- 1 kobus kobus 70046668 Sep 15 16:49 VirtualBox-5.0-5.0.4_102546_el7-1.x86_64.rpm
Here's what the find returns:
kobus#akira:~/Downloads$ find . -ctime -10
.
./tagspaces-2.0.1-linux64.tar.gz
./apache-maven-3.3.3-bin.tar.gz
./Fedora-Live-Workstation-x86_64-23-10.iso
kobus#akira:~/Downloads$
Most unices do not have a concept of file creation time. You can't make ls print it because the information is not recorded. If you need creation time, use a version control system: define creation time as the check-in time.
If your unix variant has a creation time, look at its documentation. For example, on Mac OS X (the only example I know of¹), use ls -tU. Windows also stores a creation time, but it's not always exposed to ports of unix utilities, for example Cygwin ls doesn't have an option to show it. The stat utility can show the creation time, called “birth time” in GNU utilities, so under Cygwin you can show files sorted by birth time with stat -c '%W %n' * | sort -k1n.
Note that the ctime (ls -lc) is not the file creation time, it's the inode change time. The inode change time is updated whenever anything about the file changes (contents or metadata) except that the ctime isn't updated when the file is merely read (even if the atime is updated). In particular, the ctime is always more recent than the mtime (file content modification time) unless the mtime has been explicitly set to a date in the future.
"Newly added files, Fedora" : The below examples will show a list with date and time.
Example, all installed packages : $ rpm -qa --last
Example, the latest 100 packages : $ rpm -qa --last | head -100
Example, create a text file : $ rpm -qa --last | head -100 >> last-100-packages.txt
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 7 years ago.
Improve this question
I have executed both the comments but the size seems different in both output.
ls -lh
total 147M
-rw------- 1 root root 3.4K Sep 30 14:58 anaconda-ks.cfg
-rw-r--r-- 1 root root 247 Sep 30 14:58 install.post.log
-rw-r--r-- 1 root root 54 Sep 30 14:58 install.postnochroot.log
-rw-r--r-- 1 root root 147M Sep 30 14:58 jdk-7u79-linux-x64.gz
ls -l --si
total 154M
-rw------- 1 root root 3.5k Sep 30 14:58 anaconda-ks.cfg
-rw-r--r-- 1 root root 247 Sep 30 14:58 install.post.log
-rw-r--r-- 1 root root 54 Sep 30 14:58 install.postnochroot.log
-rw-r--r-- 1 root root 154M Sep 30 14:58 jdk-7u79-linux-x64.gz
If you would have checked the manpage for ls with the command man ls you would have seen the following:
-l use a long listing format
-h, --human-readable
with -l and/or -s, print human readable sizes (e.g., 1K 234M
2G)
-i, --inode
print the index number of each file
-s, --size
print the allocated size of each file, in blocks
So you see, each parameter just defines what and how information will be put to the screen. What you see (the difference in size) is the -h or --human-readable command, which will output more readable filesizes instead of printing always the bytes. Using -s will print the filesize in blocks on your HDD, which depends on the block size of your filesystem. From the information provided, i would say your filesystem has a 1kb blocksize. So the real content of the file would be 3.4kb, but must fill up the blocks, so on your disk the file requires 4kb or 4 blocks of space.
Got a embedded system that i have root shell access to.
I can not enter the U-boot boot menu. (boot delay=0)
The device boots from a nor flash and loads the filesystem on emmc.
It does not set /dev/mtd devices.
I want to access the nor flash.
There are MTD drivers on the system, so that seems the best option.
(no experiance with this at all, so please correct me if i'm wrong)
drwxrwxr-x 2 1000 root 1024 Jul 29 2013 chips
drwxrwxr-x 2 1000 root 1024 Jul 29 2013 maps
-rw-rw-r-- 1 1000 1000 21544 Jul 29 2013 mtd.ko
-rw-rw-r-- 1 1000 1000 8560 Jul 29 2013 mtd_blkdevs.ko
-rw-rw-r-- 1 1000 1000 6132 Jul 29 2013 mtdblock.ko
-rw-rw-r-- 1 1000 1000 9648 Jul 29 2013 mtdchar.ko
If start MTD with modprobe, /proc/mtd is created.
Nothing in dmesg.
root:/proc# cat /proc/mtd
dev: size erasesize name
So no partition.
How can i configure mtd to be able to access the nor flash.
( physical addresses are known )
Thanks
You need to describe your NOR partitions in a board-specific file in the kernel. In u-boot, you should be able to see them with smeminfo.
In your linux kernel, you'll need to populate an array of mtd_partitions.
Find more here: http://free-electrons.com/blog/managing-flash-storage-with-linux/
I was compiling a custom kernel, and I wanted to test the size of the image file.
These are the results:
ls -la | grep vmlinux
-rwxr-xr-x 1 root root 8167158 May 21 12:14 vmlinux
du -h vmlinux
3.8M vmlinux
size vmlinux
text data bss dec hex filename
2221248 676148 544768 3442164 3485f4 vmlinux
Since all of them show different sizes, which one is closest to the actual image size?
Why are they different?
They are all correct, they just show different sizes.
ls shows size of the file (when you open and read it, that's how many bytes you will get)
du shows actual disk usage which can be smaller than the file size due to holes
size shows the size of the runtime image of an object/executable which is not directly related to the size of the file (bss uses no bytes in the file no matter how large, the file may contain debugging information that is not part of the runtime image, etc.)
If you want to know how much RAM/ROM an executable will take excluding dynamic memory allocation, size gives you the information you need.
Two definition need to be understood
1 runtime vs storetime (this is why size differs)
2 file depth vs directory (this is why du differs)
Look at the below example:
[root#localhost test]# ls -l
total 36
-rw-r--r-- 1 root root 712 May 12 19:50 a.c
-rw-r--r-- 1 root root 3561 May 12 19:42 a.h
-rwxr-xr-x 1 root root 71624 May 12 19:50 a.out
-rw-r--r-- 1 root root 1403 May 8 00:15 b.c
-rw-r--r-- 1 root root 1403 May 8 00:15 c.c
[root#localhost test]# du -abch --max-depth=1
1.4K ./b.c
1.4K ./c.c
3.5K ./a.h
712 ./a.c
70K ./a.out
81K .
81K total
[root#localhost test]# ls -l
total 36
-rw-r--r-- 1 root root 712 May 12 19:50 a.c
-rw-r--r-- 1 root root 3561 May 12 19:42 a.h
-rwxr-xr-x 1 root root 71624 May 12 19:50 a.out
-rw-r--r-- 1 root root 1403 May 8 00:15 b.c
-rw-r--r-- 1 root root 1403 May 8 00:15 c.c
[root#localhost test]# size a.out
text data bss dec hex filename
3655 640 16 4311 10d7 a.out
If using size not on executable, OS will report an error.
Empirically differences happen most often for sparse files and for compressed files and can go in both directions.
du < ls
Sparse files contain metadata about space needed for an application, which ls reads and applies for its result, while du doesn't. For example:
truncate -s 1m test.dat
creates a sparse file consisting entirely of nulls without disk usage, ie. du shows 0 and ls shows 1M.
du > ls
On the other hand du can indicate, as in your case, files which might occupy a lot of space on disk (ie. they spread among lots of blocks), but not all blocks are filled, i.e. their bytesize (measured by ls) is smaller than du (looking at occupied blocks). I observed this rather prominently e.g. for some python pickle files.