I can't understand why I have different total size in a folder versus in parent.
That's my folder tree
bkp
|-- raid10
| |-- folder_a
| |-- folder_b
| |-- folder_c
| |-- folder_d
| |-- folder_e
| |-- folder_f
| |-- folder_g
| |-- folder_h
| |-- folder_i
| |-- folder_j
| |-- folder_k
| |-- folder_l
| |-- script.sh
|-- vm
I previously delete a big amount of file in that folder and I want to get my new disk usage.
sudo du -shc /bkp/*
756G raid10
4.0K vm
756G total
Now I execute that command to get more info about the folder raid10:
sudo du -shc /bkp/raid10/*
13G folder_a
178M folder_b
15G folder_c
2.3G folder_d
32M folder_e
31G folder_f
31G folder_g
49G folder_h
131M folder_i
4.7G folder_j
392M folder_k
4.0K folder_l
4.0K lost+found
4.0K script.sh~
144G total
Why the total is so different ?
I checked man du and tried some command, like --apparent-size, but same result. Also try without -s sudo du -hc /bkp/raid10/*, I have the same total but I see all directory...
I have some assumptions:
There is some cache in du command ?
There is a trash or hidden file that du can't read?
Some information about my files:
Disk filesystem is ext4
File are uploaded with rsync
Disk is not in raid
To make du search for invisible just do:
#First part will get all invisible and second will get all non-invisible
du -shc /bkp/raid10/.[!.]* /bkp/raid10/*
Or cleaner command:
cd /bkp/raid10
du -sch .[!.]* *
Or enable shell option that matches hidden file with globbing
shopt -s dotglob
du -sch *
To look for hidden (names starting with a dot) files/directories recursively:
find . -name ".*" -ls
Related
I encountered a folder structure at raymii.org/s/tutorials and I'm not really sure what the symbols `-- mean. Couln't also find any syntax or documentation on how to write such structures.
$ tree -L 2 ExampleProject/
ExampleProject/
|-- build/
|-- CMakeLists.txt
|-- lib/
| `-- googletest
|-- src/
| |-- CMakeLists.txt
| |-- Formula.cpp
| |-- Formula.h
| `-- main.cpp
`-- tst/
|-- CMakeLists.txt
|-- Formula-test.cpp
`-- main.cpp
The symbols represent a polygonal chain leading from the parent directory to the file. '-' represents a horizontal segment of the chain, '`' represents a diagonal segment and '|' represents a vertical segment.
The chain conveys the parent-child relationship of a directory entry and the directory that contains it.
This particular tree shows the root directory ExampleProject which contains a sub directory src which contains a file CMakeLists.txt. And a bunch of other directories and files.
I know that a directory can be excluded with --exclude like this:
rsync -avz --exclude=dir/to/skip /my/source/path /my/backup/path
This will omit the directory dir/to/skip
However I want to copy the directory itself but not the contents | Is there a one-liner with rsync to accomplish this?
Essentially, include dir/to/skip but exclude dir/to/skip/*
NOTE: I did search for this question. I found a lot of similar posts but not exactly this. Apologies if there is a dupe already.
The --exclude option takes a PATTERN, which means you just should just be able to do this:
rsync -avz --exclude='dir/to/skip/*' /my/source/path /my/backup/path
Note that the PATTERN is quoted to prevent the shell from doing glob expansion on it.
Since dir/to/skip doesn't match the pattern dir/to/skip/*, it will be included.
Here's an example to show that it works:
> mkdir -p a/{1,2,3}
> find a -type d -exec touch {}/file \;
> tree --charset ascii a
a
|-- 1
| `-- file
|-- 2
| `-- file
|-- 3
| `-- file
`-- file
3 directories, 4 files
> rsync -r --exclude='/2/*' a/ b/
> tree --charset ascii b
b
|-- 1
| `-- file
|-- 2
|-- 3
| `-- file
`-- file
3 directories, 3 files
It is important to note that the leading / in the above PATTERN represents the root of the source directory, not the filesystem root. This is explained in the rsync man page. If you omit the leading slash, rsync will attempt to match the PATTERN from the end of each path. This could lead to excluding files unexpectedly. For example, suppose I have a directory a/3/2/ which contains a bunch of files that I do want to transfer. If I omit the leading / and do:
rsync -r --exclude='2/*' a/ b/
then the PATTERN will match both a/2/* and a/3/2/*, which is not what I wanted.
Try:
rsync -avz --include=src/dir/to/skip --exclude=src/dir/to/skip/* src_dir dest_dir
--include=src/dir/to/skip includes the directory. --exclude=src/dir/to/skip/* excludes everything under the directory.
I would like to list most fat directories by it's own size sorted by size descending.
'Directory own size' means size of the directory excluding size of all it's subdirectories.
For example, we have directory structure:
/tmp/D1
|-- 5m.file
|-- D2
| |-- 2m.file
| `-- D4
| `-- 4m.file
`-- D3
`-- 3m.file
By for executing command and passing /tmp/D1 as argument I'd like to get result like
5m /tmp/D1
4m /tmp/D1/D2/D4
3m /tmp/D1/D3
2m /tmp/D1/D2
du -Sh . | sort -rh | head -n 10
+x to limit to current filesystem only
du -Shx . | sort -rh | head -n 10
You can use du with an -S option for this
From the man page
-S, --separate-dirs
do not include size of subdirectories
$ du -Sh /foo/bar/temp2/ | sort -rh
84K /foo/bar/temp2/
40K /foo/bar/temp2/tempo
4.0K /foo/bar/temp2/opt/logs/merchantportal
4.0K /foo/bar/temp2/opt/logs
4.0K /foo/bar/temp2/opt
4.0K /foo/bar/temp2/folder
4.0K /foo/bar/temp2/bang
Now checking in the normal way with the -s option, which is inclusive of all sub-directories.
$ du -sh /foo/bar/temp2/opt
12K /foo/bar/temp2/opt
which is the summation of sizes of the sub-folders /foo/bar/temp2/opt/logs/merchantportal, /foo/bar/temp2/opt/logs and the base folder itself
The -h formats the size in the human readable format according to the man page. If you forcefully want to format the output in 1-Mbyte blocks you can use the option du -Sm
Let's say I have a directory structure like this:
# tree original_directory/
|-- sub-1
|-- sub-2
|-- ignore_this_dir
|-- sub-3
Then the tar command to exclude the directory called ignore_this_dir is actually:
# tar -cf new_archived.tar original_directory/ --exclude=ignore_this_dir
OR
# tar -cf new_archived.tar original_directory/ --exclude=original_directory/ignore_this_dir
The man page states:
--exclude=PATTERN
exclude files, given as a PATTERN
Meaning
tar -cf new_archived.tar origin_directory/ --exclude=ignore_this_dir
will be ok in your situation as the pattern ignore_this_dir will match original_directory/ignore_this_dir.
In linux or freebsd, Is there a way to copy all files under a folder and its subfolders as symbolic link ? I need to copy thousands of files into different locations as symbolic links and only have 2-3 configuration files as the actual file. The reason I'm doing this is, I have dozen of websites with with exactly the same engine code, but different configuration and look. I want to copy the engine as symbolic link so every change I make to original files will be applied to other websites as well.
I can't make symbolic link to the engine folder itself, because the configuration file is under that folder, and I can't copy files one by one ! cause obviously it's not practical.
Any suggestion ?
The command you are looking for is cp -rs /path/to/source dest.
Note that you need to provide full path to the source directory so that it can make absolute symlinks.
i don't know if this is what you want: (see example below)
dir one is your central "engine"
dir two is one of your website.
kent#ArchT60:/tmp$ tree one two
one
|-- 1.txt
|-- 2.txt
|-- 3.txt
|-- 4.txt
|-- 5.txt
|-- dirA
| |-- a
| |-- b
| `-- c
|-- dirB
`-- dirC
two
|-- myConf_a.conf
|-- myConf_b.conf
|-- myConf_c.conf
|-- myConf_d.conf
`-- myConf_e.conf
kent#ArchT60:/tmp$ ln -s /tmp/one/* /tmp/two/.
kent$ tree -l /tmp/two
/tmp/two
|-- 1.txt -> /tmp/one/1.txt
|-- 2.txt -> /tmp/one/2.txt
|-- 3.txt -> /tmp/one/3.txt
|-- 4.txt -> /tmp/one/4.txt
|-- 5.txt -> /tmp/one/5.txt
|-- dirA -> /tmp/one/dirA
| |-- a
| |-- b
| `-- c
|-- dirB -> /tmp/one/dirB
|-- dirC -> /tmp/one/dirC
|-- myConf_a.conf
|-- myConf_b.conf
|-- myConf_c.conf
|-- myConf_d.conf
`-- myConf_e.conf