linux - after rsync, du shows size difference when diff does not - linux

I copied a large folder from NTFS to ext4 using 'rsync' and validating it with 'diff'. Just for the shake of curiosity, I also used 'du' command to check if folders had the same size. While 'diff' didn't show any difference, 'du' showed that folders had different sizes. I did not encounter any errors while executing the following commands.
rsync --archive --recursive "$src" "$dest" 2>rsync_error.txt
sync
diff --brief --recursive --new-file "$src" "$dest" 1>diff-log.txt 2>diff-error.txt
Then I used 'du' for each folder:
du -sb "$src"
du -sb "$dest"
Output:
137197597476
137203512004
1.Why would this happen since there is not any difference?
2.Should I be worried about my data or my system?
EDIT:
I also tried du -s --apparent-size and there is still difference.

Sparses files
Under linux, you could create so-called sparse files. They are files where full NULL block don't really exists!
Try this:
$ dd if=/dev/zero count=2048 of=normalfile
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103269 s, 102 MB/s
and
$ dd if=/dev/zero count=0 seek=2048 of=sparsefile
0+0 records in
0+0 records out
0 bytes copied, 0.000182708 s, 0.0 kB/s
then
$ ls -l sparsefile normalfile
-rw-r--r-- 1 user user 1048576 Feb 3 17:53 normalfile
-rw-r--r-- 1 user user 1048576 Feb 3 17:53 sparsefile
$ du -b sparsefile normalfile
1048576 sparsefile
1048576 normalfile
but
$ du -k sparsefile normalfile
0 sparsefile
1024 normalfile
$ du -h sparsefile normalfile
0 sparsefile
1.0M normalfile
So long block in sparsefile are not used, they will not be allocated!
$ du -k --apparent-size sparsefile normalfile
1024 sparsefile
1024 normalfile
Then
$ diff sparsefile normalfile
echo $?
0
There is virtually no difference between both files!
Further
$ /sbin/mkfs.ext4 sparsefile
mke2fs 1.44.5 (15-Dec-2018)
Filesystem too small for a journal
...
Writing superblocks and filesystem accounting information: done
$ ls -l sparsefile normalfile
-rw-r--r-- 1 user user 1048576 Feb 3 17:53 normalfile
-rw-r--r-- 1 user user 1048576 Feb 3 17:59 sparsefile
$ du -k sparsefile
32 sparsefile
$ diff sparsefile normalfile
Binary files sparsefile and normalfile differ

Greettings Invinciblecache,
Googling around I've found this:
As du reports allocation space and not absolute file space, the amount of space on a file system shown by du may vary from that shown by df if files have been deleted but their blocks not yet freed. source
Not the best source but is a great description of what du is used for.
So, I'd rely on diff to check the content of the files, but I would recommend to ignore size difference on filesystem unless it is too high, which is not this the scenario.

du is reporting space including filesystem space, not only file content size.
Also check for hidden files which might not be included in your du.

Related

Rsync Incremental Backup still copies all the files

I am currently writing a bash script for rsync. I am pretty sure I am doing something wrong. But I can't tell what it is. I will try to elaborate everything in detail so hopefully someone can help me.
The goal of script is to do full backups and incremental ones using rsync. Everything seems to work perfectly well, besides one crucial thing. It seems like even though using the --link-dest parameter, it still copies all the files. I have checked the file sizes with du -chs.
First here is my script:
#!/bin/sh
while getopts m:p: flags
do
case "$flags" in
m) mode=${OPTARG};;
p) prev=${OPTARG};;
*) echo "usage: $0 [-m] [-p]" >&2
exit 1 ;;
esac
done
date="$(date '+%Y-%m-%d')";
#Create Folders If They Do Not Exist (-p paramter)
mkdir -p /Backups/Full && mkdir -p /Backups/Inc
FullBackup() {
#Backup Content Of Website
mkdir -p /Backups/Full/$date/Web/html
rsync -av user#IP:/var/www/html/ /Backups/Full/$date/Web/html/
#Backup All Config Files NEEDED. Saving Storage Is Key ;)
mkdir -p /Backups/Full/$date/Web/etc
rsync -av user#IP:/etc/apache2/ /Backups/Full/$date/Web/etc/
#Backup Fileserver
mkdir -p /Backups/Full/$date/Fileserver
rsync -av user#IP:/srv/samba/private/ /Backups/Full/$date/Fileserver/
#Backup MongoDB
ssh user#IP /usr/bin/mongodump --out /home/DB
rsync -av root#BackupServerIP:/home/DB/ /Backups/Full/$date/DB
ssh user#IP rm -rf /home/DB
}
IncrementalBackup(){
Method="";
if [ "$prev" == "full" ]
then
Method="Full";
elif [ "$prev" == "inc" ]
then
Method="Inc";
fi
if [ -z "$prev" ]
then
echo "-p Parameter Empty";
else
#Get Latest Folder - Ignore the hacky method, it works.
cd /Backups/$Method
NewestBackup=$(find . ! -path . -type d | sort -nr | head -1 | sed s#^./##)
IFS='/'
read -a strarr <<< "$NewestBackup"
Latest_Backup="${strarr[0]}";
cd /Backups/
#Incremental-Backup Content Of Website
mkdir -p /Backups/Inc/$date/Web/html
rsync -av --link-dest /Backups/$Method/"$Latest_Backup"/Web/html/ user#IP:/var/www/html/ /Backups/Inc/$date/Web/html/
#Incremental-Backup All Config Files NEEDED
mkdir -p /Backups/Inc/$date/Web/etc
rsync -av --link-dest /Backups/$Method/"$Latest_Backup"/Web/etc/ user#IP:/etc/apache2/ /Backups/Inc/$date/Web/etc/
#Incremental-Backup Fileserver
mkdir -p /Backups/Inc/$date/Fileserver
rsync -av --link-dest /Backups/$Method/"$Latest_Backup"/Fileserver/ user#IP:/srv/samba/private/ /Backups/Inc/$date/Fileserver/
#Backup MongoDB
ssh user#IP /usr/bin/mongodump --out /home/DB
rsync -av root#BackupServerIP:/home/DB/ /Backups/Full/$date/DB
ssh user#IP rm -rf /home/DB
fi
}
if [ "$mode" == "full" ]
then
FullBackup;
elif [ "$mode" == "inc" ]
then
IncrementalBackup;
fi
The command i used:
Full-Backup
bash script.sh -m full
Incremental
bash script.sh -m inc -p full
Executing the script is not giving any errors at all. As I mentioned above, it just seems like it's still copying all the files. Here are some tests I did.
Output of du -chs
root#Backup:/Backups# du -chs /Backups/Full/2021-11-20/*
36K /Backups/Full/2021-11-20/DB
6.5M /Backups/Full/2021-11-20/Fileserver
696K /Backups/Full/2021-11-20/Web
7.2M total
root#Backup:/Backups# du -chs /Backups/Inc/2021-11-20/*
36K /Backups/Inc/2021-11-20/DB
6.5M /Backups/Inc/2021-11-20/Fileserver
696K /Backups/Inc/2021-11-20/Web
7.2M total
Output of ls -li
root#Backup:/Backups# ls -li /Backups/Full/2021-11-20/
total 12
1290476 drwxr-xr-x 4 root root 4096 Nov 20 19:26 DB
1290445 drwxrwxr-x 6 root root 4096 Nov 20 18:54 Fileserver
1290246 drwxr-xr-x 4 root root 4096 Nov 20 19:26 Web
root#Backup:/Backups# ls -li /Backups/Inc/2021-11-20/
total 12
1290506 drwxr-xr-x 4 root root 4096 Nov 20 19:28 DB
1290496 drwxrwxr-x 6 root root 4096 Nov 20 18:54 Fileserver
1290486 drwxr-xr-x 4 root root 4096 Nov 20 19:28 Web
Rsync Output when doing the incremental backup and changing/adding a file
receiving incremental file list
./
lol.html
sent 53 bytes received 194 bytes 164.67 bytes/sec
total size is 606 speedup is 2.45
receiving incremental file list
./
sent 33 bytes received 5,468 bytes 11,002.00 bytes/sec
total size is 93,851 speedup is 17.06
receiving incremental file list
./
sent 36 bytes received 1,105 bytes 760.67 bytes/sec
total size is 6,688,227 speedup is 5,861.72
*Irrelevant MongoDB Dump Text*
sent 146 bytes received 2,671 bytes 1,878.00 bytes/sec
total size is 2,163 speedup is 0.77
I suspect that the ./ has something to do with that. I might be wrong, but it looks suspicious. Though when executing the same command again, the ./ are not in the log, probably because I did it on the same day, so it was overwriting in the /Backup/Inc/2021-11-20 Folder.
Let me know for more information. I have been trying around for a long time now. Maybe I am simply wrong and there are links made and disk space economized.
I didn't read the entire code because the main problem didn't seem to lay there.
Verify the disk usage of your /Backups directory with du -sh /Backups and then compare it with the sum of du -sh /Backups/Full and du -sh /Backups/Inc.
I'll show you why with a little test:
Create a directory containing a file of 1 MiB:
mkdir -p /tmp/example/data
dd if=/dev/zero of=/tmp/example/data/zerofile bs=1M count=1
Do a "full" backup:
rsync -av /tmp/example/data/ /tmp/example/full
Do an "incremental" backup
rsync -av --link-dest=/tmp/example/full /tmp/example/data/ /tmp/example/incr
Now let's see what we got:
with ls -l
ls -l /tmp/example/*
-rw-rw-r-- 1 user group 1048576 Nov 21 00:24 /tmp/example/data/zerofile
-rw-rw-r-- 2 user group 1048576 Nov 21 00:24 /tmp/example/full/zerofile
-rw-rw-r-- 2 user group 1048576 Nov 21 00:24 /tmp/example/incr/zerofile
and with du -sh
du -sh /tmp/example/*
1.0M /tmp/example/data
1.0M /tmp/example/full
0 /tmp/example/incr
Oh? There was a 1 MiB file in /tmp/example/incr but du missed it ?
Actually no. As the file wasn't modified since the previous backup (referenced with --link-dest), rsync created a hard-link to it instead of copying its content. — Hard-links connect a same memory space to different files
And du can detect hard-links and show you the real disk usage, but only when the hard-linked files are included (even in sub-dirs) in its arguments. For example, if you use du -sh independently for /tmp/example/incr:
du -sh /tmp/example/incr
1.0M /tmp/example/incr
How do you detect that there is hard-links to a file ?
ls -l actually showed it to us:
-rw-rw-r-- 2 user group 1048576 Nov 21 00:24 /tmp/example/full/zerofile
^
HERE
This number means that there are two existing hard-links to the file: this file itself and another one in the same filesystem.
about your code
It doesn't change anything but I would replace:
#Get Latest Folder - Ignore the hacky method, it works.
cd /Backups/$Method
NewestBackup=$(find . ! -path . -type d | sort -nr | head -1 | sed s#^./##)
IFS='/'
read -a strarr <<< "$NewestBackup"
Latest_Backup="${strarr[0]}";
cd /Backups/
with:
#Get Latest Folder
glob='20[0-9][0-9]-[0-1][0-9]-[0-3][0-9]' # match a timestamp (more or less)
NewestBackup=$(compgen -G "/Backups/$Method/$glob/" | sort -nr | head -n 1)
glob makes sure that the directories/files found by compgen -G will have the right format.
Adding / at the end of a glob makes sure that it matches directories only.

Get clean list of file sizes and names using SFTP in unix

I want to fetch list of files from a server using SFTP one by one only if their size is less than 1 GB.
I am running the following command :
$sftp -oIdentityFile=/home/user/.ssh/id_rsa -oPort=22 user#hostname >list.txt <<EOF
cd upload/Example
ls -l iurygify*.zip
EOF
This results in:
$cat list.txt
sftp> cd upload/Example
sftp> ls -l iurygify*.zip
-rwxrwx--- 0 300096661 300026669 0 Mar 11 16:38 iurygify1.zip
-rwxrwx--- 0 300096661 300026669 0 Mar 11 16:38 iurygify2.zip
I could then use awk to calculate get the size and filename which I can save into logs for reference and then download only those files which meet the 1 GB criteria.
Is there any simpler approach to accomplish getting this file list and size? I want to avoid the junk entires of the prompt and commands in the list.txt and do not want to do this via expect command.
We are using SSH key authentication
You could place your sftp commands in a batch file and filter the output - no need for expect.
echo 'ls -l' > t
sftp -b t -oIdentityFile=/home/user/.ssh/id_rsa -oPort=22 user#hostname | grep -v 'sftp>' >list.txt
Or take it a step further and filter out the "right size" in the same step:
sftp -b t -oIdentityFile=/home/user/.ssh/id_rsa -oPort=22 user#hostname | awk '$1!~/sftp>/&&$5<1000000000' >list.txt
Maybe using lftp instead of sftp ?
$ lftp sftp://xxx > list.txt <<EOF
> open
> ls -l
> EOF
$ cat list.txt
drwxr-xr-x 10 ludo users 4096 May 24 2019 .
drwxr-xr-x 8 root root 4096 Dec 20 2018 ..
-rw------- 1 ludo users 36653 Mar 31 19:28 .bash_history
-rw-r--r-- 1 ludo users 220 Mar 21 2014 .bash_logout
-rw-r--r-- 1 ludo users 362 Aug 16 2018 .bash_profile
...

Identify the latest file from a file list

I have a pretty tricky task (at least for me).
I have an sftp access to a server which I need to get ONLY the latest file in the directory. Since sftp interface is very limited I have come up to list the files in the directory to a text file first.
This is the code
sftp -b - hostname >list.txt <<EOF
ls -l *.xls
EOF
My concern now is from list.txt, how do I identify the latest file?
Sample content of list.txt
cat list.txt
-rw-r--r-- 0 16777221 16777216 52141 Mar 29 08:06 samplefile1.xls
-rw-r--r-- 0 16777221 16777216 2926332 Mar 28 09:48 samplefile2.xls
-rw-r--r-- 0 16777221 16777216 40669 Mar 26 04:38 samplefile3.xls
-rw-r--r-- 0 16777221 16777216 8640 Mar 19 08:02 samplefile4.xls
-rw-r--r-- 0 16777221 16777216 146331 Mar 25 07:27 samplefile5.xls
-rw-r--r-- 0 16777221 16777216 18988 Mar 19 03:53 samplefile6.xls
-rw-r--r-- 0 16777221 16777216 36640 Apr 2 12:52 samplefile7.xls
Use ls -lt
sftp -b - hostname >list.txt <<EOF
ls -lt
EOF
Now the first line in your file will be latest file.
You can manage it like below:-
Maintain a history file in your server like history.txt
Before transferring file create a list of files that you are creating at the moment.
For the first time generate a history.txt file manually and add all the files that you have already transferred. For example samplefile6.xls and samplefile7.xls
sftp -b - hostname >list.txt <<EOF
ls -l *.xls
EOF
Now add a while loop to your above existing script
while read line
do
file=$(echo "$line" | awk '{print $9}')
if grep "$file" history.txt; then
echo "File already existed in history file -- No need to transfer"
else
sftp server_host <<EOF
cd /your/dropLocation
put $file
quit
EOF
echo "$file" >> history.txt
#add the transferred file to history file
fi
done < list.txt
With this approach, even if you have more than one latest files you can transfer them very easily.
Hope this will help you.

Formatting text and adding text-part between specific pattern

On GNU/Linux, I've following output by running ls command:
$ ls -l /dev/disk/by-label/
total 0
lrwxrwxrwx 1 root root 10 Mar 4 18:21 Documents+Edu -> ../../sda4
lrwxrwxrwx 1 root root 10 Mar 4 18:17 Ext4 -> ../../sda3
lrwxrwxrwx 1 root root 10 Mar 4 18:21 Game+Movie -> ../../sda6
lrwxrwxrwx 1 root root 10 Mar 4 18:42 OS -> ../../sda7
lrwxrwxrwx 1 root root 10 Mar 4 18:20 Software+Video -> ../../sda5
And I want the following output:
sudo mount /dev/sda4 /media/$USER/Documents+Edu
sudo mount /dev/sda3 /media/$USER/Ext4
sudo mount /dev/sda6 /media/$USER/Game+Movie
sudo mount /dev/sda7 /media/$USER/OS
sudo mount /dev/sda5 /media/$USER/Software+Video
In other words:-
Adding text sudo mount /dev/ before every /sdax and /media/$USER/ before labels like: Documents+Edu etc.
Formatting as needed.
How can I achieve desired output on Linux by using commands like: grep, cut, awk, sed, etc.?
Another option could be to avoid ls altogether iterating through the files with readlink:
$ cat ../nols.sh
cd /dev/disk/by-label
for f in *
do
v=$(basename $(readlink $f))
echo "sudo mount /dev/$v /media/\$USER/$f"
done
$ sh ../nols.sh
sudo mount /dev/sda4 /media/$USER/Documents+Edu
sudo mount /dev/sda3 /media/$USER/Ext4
sudo mount /dev/sda6 /media/$USER/Game+Movie
sudo mount /dev/sda7 /media/$USER/OS
sudo mount /dev/sda5 /media/$USER/Software+Video
Since you don't need anything apart from the names and its symbolic links, it might me best to just do ls -1 (one instead of L) ot just get the last column. Then, pipe to this awk:
awk -F"->|/" '{printf "sudo mount /dev/%s /media/$USER/%s\n", $NF, $1}'
It slices the text in either -> or / separators. Based on that, it gets the last one and the first one and populates the string accordingly.
Test
$ ls -1 /dev/disk/by-label/ | awk -F"->|/" '{printf "sudo mount /dev/%s /media/$USER/%s\n", $NF, $1}'
sudo mount /dev/sda4 /media/$USER/Documents+Edu
sudo mount /dev/sda3 /media/$USER/Ext4
sudo mount /dev/sda6 /media/$USER/Game+Movie
sudo mount /dev/sda7 /media/$USER/OS
sudo mount /dev/sda5 /media/$USER/Software+Video
# -- Prepare ---
Folder='/dev/disk/by-label/'
HeaderLen=$(ls -ld . | wc -c )
# -- execute ---
ls -l "${Folder}" | sed "1d;s#.\{${HeaderLen}\}\(.*\) -> .*\(/[^/]*\)#sudo mount /dev\2 /media/\$USER/\1#;t;d"
estimate the first partof ls that is not used (and will be removed)
list the folder pipe to a sed
remove 1st line (total info)
remove first part and take 2 group of text separate by ->, rewrtie it in new format
if substitution occur, go to end of script (print)
if no, remove the line (and cycle without print). This remove file not linked in your format

linux command to get size of files and directories present in a particular folder? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
How can I see the size of files and directories in Linux? If use df -m, then it shows the size of all the directory at the top level, but, for the directories and files inside the directory, how do I check the size?
Use ls command for files and du command for directories.
Checking File Sizes
ls -l filename #Displays Size of the specified file
ls -l * #Displays Size of All the files in the current directory
ls -al * #Displays Size of All the files including hidden files in the current directory
ls -al dir/ #Displays Size of All the files including hidden files in the 'dir' directory
ls command will not list the actual size of directories(why?). Therefore, we use du for this purpose.
Checking Directory sizes
du -sh directory_name #Gives you the summarized(-s) size of the directory in human readable(-h) format
du -bsh * #Gives you the apparent(-b) summarized(-s) size of all the files and directories in the current directory in human readable(-h) format
Including -h option in any of the above commands (for Ex: ls -lh * or du -sh) will give you size in human readable format (kb, mb,gb, ...)
For more information see man ls and man du
There is du command.
Size of a directory and/or file, in a human-friendly way:
$ du -sh .bashrc /tmp
I memorised it as a non-existent English word dush.
--apparent-size command line switch makes it measure apparent sizes (what ls shows) rather than actual disk usage.
Use ls -s to list file size, or if you prefer ls -sh for human readable sizes.
For directories use du, and again, du -h for human readable sizes.
You can use:
ls -lh
Using this command you'll see the apparent space of the directory and true space of the files and in details the names of the files displayed, besides the size and creation date of each.
There is also a great ncdu utility - it can show directory size with detailed info about subfolders and files.
Installation
Ubuntu:
$ sudo apt-get install ncdu
Usage
Just type ncdu [path] in the command line. After a few seconds for analyzing the path, you will see something like this:
$ ncdu 1.11 ~ Use the arrow keys to navigate, press ? for help
--- / ---------------------------------------------------------
. 96,1 GiB [##########] /home
. 17,7 GiB [# ] /usr
. 4,5 GiB [ ] /var
1,1 GiB [ ] /lib
732,1 MiB [ ] /opt
. 275,6 MiB [ ] /boot
198,0 MiB [ ] /storage
. 153,5 MiB [ ] /run
. 16,6 MiB [ ] /etc
13,5 MiB [ ] /bin
11,3 MiB [ ] /sbin
. 8,8 MiB [ ] /tmp
. 2,2 MiB [ ] /dev
! 16,0 KiB [ ] /lost+found
8,0 KiB [ ] /media
8,0 KiB [ ] /snap
4,0 KiB [ ] /lib64
e 4,0 KiB [ ] /srv
! 4,0 KiB [ ] /root
e 4,0 KiB [ ] /mnt
e 4,0 KiB [ ] /cdrom
. 0,0 B [ ] /proc
. 0,0 B [ ] /sys
# 0,0 B [ ] initrd.img.old
# 0,0 B [ ] initrd.img
# 0,0 B [ ] vmlinuz.old
# 0,0 B [ ] vmlinuz
Delete the currently highlighted element with d, exit with CTRL + c
File Size in MB
ls -l --b=M filename | cut -d " " -f5
File Size in GB
ls -l --b=G filename | cut -d " " -f5
Go to the chosen directory and execute:
$ du -d 1 -h
where:
-d 1 is the depth of the directories
-h is the human-readable option
You'll see like that:
0 ./proc
8.5M ./run
0 ./sys
56M ./etc
12G ./root
33G ./var
23M ./tmp
3.2G ./usr
154M ./boot
26G ./home
0 ./media
0 ./mnt
421M ./opt
0 ./srv
2.6G ./backups
80G .
ls -l --block-size=M will give you a long format listing (needed to actually see the file size) and round file sizes up to the nearest MiB.
If you want MB (10^6 bytes) rather than MiB (2^20 bytes) units, use --block-size=MB instead.
If you don't want the M suffix attached to the file size, you can use something like --block-size=1M. Thanks Stéphane Chazelas for suggesting this.
This is described in the man page for ls; man ls and search for SIZE. It allows for units other than MB/MiB as well, and from the looks of it (I didn't try that) arbitrary block sizes as well (so you could see the file size as number of 412-byte blocks, if you want to).
Note that the --block-size parameter is a GNU extension on top of the Open Group's ls, so this may not work if you don't have a GNU userland (which most Linux installations do). The ls from GNU coreutils 8.5 does support --block-size as described above.
All you need is -l and --block-size flags
Size of all files and directories under working directory (in MBs)
ls -l --block-size=M
Size of all files and directories under working directory (in GBs)
ls -l --block-size=G
Size of a specific file or directory
ls -l --block-size=M my_file.txt
ls -l --block-size=M my_dir/
ls --help
-l use a long listing format
--block-size=SIZE: scale sizes by SIZE before printing them; e.g.,
'--block-size=M' prints sizes in units of
1,048,576 bytes; see SIZE format below
SIZE is an integer and optional unit (example: 10M is 10*1024*1024).
Units are K, M, G, T, P, E, Z, Y (powers of 1024) or KB, MB, ...
(powers of 1000).
If you are using it in a script, use stat.
$ date | tee /tmp/foo
Wed Mar 13 05:36:31 UTC 2019
$ stat -c %s /tmp/foo
29
$ ls -l /tmp/foo
-rw-r--r-- 1 bruno wheel 29 Mar 13 05:36 /tmp/foo
That will give you size in bytes. See man stat for more output format options.
The OSX/BSD equivalent is:
$ date | tee /tmp/foo
Wed Mar 13 00:54:16 EDT 2019
$ stat -f %z /tmp/foo
29
$ ls -l /tmp/foo
-rw-r--r-- 1 bruno wheel 29 Mar 13 00:54 /tmp/foo
You can use below command to get list of files in easily human readable format.
ls -lrtsh
You can use ncdu disk usage analyzer here. It displays the size of the files and directories in an ncurses interface. You can navigate to each directory and see the files sizes from the same interface.
To install
$ sudo apt-get install ncdu
To analyze
$ ncdu <directory>
I do the following all the time:
$ du -sh backup-lr-May-02-2017-1493723588.tar.gz
NB:
-s, --summarize
display only a total for each argument
-h, --human-readable
print sizes in human readable format (e.g., 1K 234M 2G)
To get the total size of directory or the total size of file use,
du -csh <directory or filename*> | grep total
Use ls command with -h argument: [root#hots19 etc]# ls -lh
h : for human readable.
Exemple:
[root#CIEYY1Z3 etc]# ls -lh
total 1.4M
-rw-r--r--. 1 root root 44M Sep 15 2015 adjtime
-rw-r--r--. 1 root root 1.5K Jun 7 2013 aliases
-rw-r--r-- 1 root root 12K Nov 25 2015 aliases.db
drwxr-xr-x. 2 root root 4.0K Jan 11 2018 alternatives
-rw-------. 1 root root 541 Jul 8 2014 anacrontab
-rw-r--r--. 1 root root 55M Sep 16 2014 asound.conf
-rw-r--r--. 1 root root 1G Oct 6 2014 at.deny
du -sh [file_name]
works perfectly to get size of a particular file.
I prefer this command ll -sha.
I'm a Ubuntu 16.04 user myself and I find that the ll command is by far the easiest way to see a directory's contents. I've noticed that not all Linux distributions support this command, but there's probably a workaround/install for each distro out there.
Example:
user#user-XPS-15-9560:/$ ll
total 188
drwxr-xr-x 27 root root 4096 Jan 26 09:13 ./
drwxr-xr-x 27 root root 4096 Jan 26 09:13 ../
drwxr-xr-x 2 root root 4096 Jan 22 15:13 bin/
drwxr-xr-x 4 root root 12288 Jan 29 11:35 boot/
drwxr-xr-x 2 root root 4096 Sep 3 18:14 cdrom/
drwxr-xr-x 20 root root 4440 Feb 5 08:43 dev/
drwxr-xr-x 153 root root 12288 Feb 2 15:17 etc/
drwxr-xr-x 4 root root 4096 Sep 3 18:15 home/
...
The biggest advantage for me is that it's quick and really intuitive to use.
UPDATE: what I didn't know was that on Ubuntu it's a pre-configured alias. You can easily set it yourself by executing alias ll="ls -la" on the command line, or by adding this entry in your .bashrc config file:
sudo nano ~/.bashrc
...add line described above and save file by pressing Ctrl+X and Y...
source ~/.bashrc
ls -sh video.mp4 | sed s/video.mp4//g
output,
5.6M
You have to differenciate between file size and disk usage. The main difference between the two comes from the fact that files are "cut into pieces" and stored in blocks.
Modern block size is 4KiB, so files will use disk space multiple of 4KiB, regardless of how small they are.
If you use the command stat you can see both figures side by side.
stat file.c
If you want a more compact view for a directory, you can use ls -ls, which will give you usage in 1KiB units.
ls -ls dir
Also du will give you real disk usage, in 1KiB units, or dutree with the -u flag.
Example: usage of a 1 byte file
$ echo "" > file.c
$ ls -l file.c
-rw-r--r-- 1 nacho nacho 1 Apr 30 20:42 file.c
$ ls -ls file.c
4 -rw-r--r-- 1 nacho nacho 1 Apr 30 20:42 file.c
$ du file.c
4 file.c
$ dutree file.c
[ file.c 1 B ]
$ dutree -u file.c
[ file.c 4.00 KiB ]
$ stat file.c
File: file.c
Size: 1 Blocks: 8 IO Block: 4096 regular file
Device: 2fh/47d Inode: 2185244 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ nacho) Gid: ( 1000/ nacho)
Access: 2018-04-30 20:41:58.002124411 +0200
Modify: 2018-04-30 20:42:24.835458383 +0200
Change: 2018-04-30 20:42:24.835458383 +0200
Birth: -
In addition, in modern filesystems we can have snapshots, sparse files (files with holes in them) that further complicate the situation.
You can see more details in this article: understanding file size in Linux
you can use ls -sh in linux you can do sort also
you need to go to dir where you want to check the size of files
go to specific directory then run below command
# du -sh *
4.0K 1
4.0K anadb.sh --> Shell file
4.0K db.sh/ --> shell file
24K backup4/ --> Directory
8.0K backup6/ --> Directory
1.9G backup.sql.gz --> sql file

Resources