Running cPanel on a server with various customer accounts under the /home directory.
Many customers' error_log files are exceeding a desired size (let's say 100MB) and I want to create a cron job to run daily to truncate any files over a certain size.
I know truncate can shrink files but it will extend files if they're smaller than the stipulated amount, so does my solution below (of first finding all files above the desired size and only shrinking those) make the most sense and will it work?
for i in $(find /home -type f -iname error_log -size +99M); do
truncate -s 100M $i
done
I'd suggest rotating and compressing logs rather than truncating them. Logs typically compress really well, and you can move the compressed logs to backup media if you like. Plus, if you do have to delete anything, delete the oldest logs, not the newest ones.
That said, for educational purposes let's explore truncate. It has the ability to only shrink files, though it's buried in the documentation:
SIZE may also be prefixed by one of the following modifying characters: '+' extend by, '-' reduce by, '<' at most, '>' at least, '/' round down to multiple of, '%' round up to multiple of.
If the files are at a fixed depth you don't need the loop nor the find call. A simple glob will do:
truncate -s '<100M' /home/*/path/to/error_log
If they're at unpredictable depths you can use extended globbing...
shopt -s extglob
truncate -s '<100M' /home/**/error_log
...or use find -exec <cmd> {} +, which tells find to invoke a command on the files it finds.
find /home -name error_log -exec truncate -s '<100M' {} +
(If there are lots and lots of files find is safest. The glob options could exceed Linux's command-line length limit whereas find guards against that possibility.)
Do not use for i in $(...). It will break on whitespaces.
Always quote your variable expansions. Do "$i".
find has -exec, just use it.
So:
find /home -type f -iname error_log -size +99M -exec truncate -s 100M {} \;
Related
I tried using tree command but I didn't know how .(I wanted to use tree because I don't want the files to show up , just the number)
Let's say c is the code for permission
For example I want to know how many files are there with the permission 751
Use find with the -perm flag, which only matches files with the specified permission bits.
For example, if you have the octal in $c, then run
find . -perm $c
The usual find options apply—if you only want to find files at the current level without recursing into directories, run
find . -maxdepth 1 -perm $c
To find the number of matching files, make find print a dot for every file and use wc to count the number of dots. (wc -l will not work with more exotic filenames with newlines as #BenjaminW. has pointed out in the comments. Source of idea of using wc -c is this answer.)
find . -maxdepth 1 -perm $c -printf '.' | wc -c
This will show the number of files without showing the files themselves.
If you're using zsh as your shell, you can do it natively without any external programs:
setopt EXTENDED_GLOB # Just in case it's not already set
c=0751
files=( **/*(#qf$c) )
echo "${#files[#]} files found"
will count all files in the current working directory and subdirectories with those permissions (And gives you all the names in an array in case you want to do something with them later). Read more about zsh glob qualifiers in the documentation.
Alright so i have a web server running CentOS at work that is hosting a few websites internally only. It's our developpement server and thus has lots [read tons] of old junk websites and whatnot.
I was trying to elaborate a command that would find files that haven't been modified for over 6 months, group them all in a tarball and then delete them. Thus far i have tried many different type of find commands with arguments and whatnot. Our structure looks like such
/var/www/joomla/username/fileshere/temp
/var/www/username/fileshere
So i tried something amongst the lines of :
find /var/www -mtime -900 ! -mtime -180 | xargs tar -cf test4.tar
Only to have a 10MB resulting tar, when the expected result would be over 50 GB's.
I tried using gzip instead, but i ended up zipping MY WHOLE SERVER thus making is unusable, had to transfer the whole filesystem and reinstall a complete new server and lots of shit and trouble and... you get the idea. So i want to find the perfect command that won't blow up our server but will find all FILES and DIRECTORIES that haven't been modified for over 6 months.
Be careful with ctime.
ctime is related to changes made to inodes (changing permissions, owner, etc)
atime when a file was last accessed (check if your file system is using noatime or relatime options, in that case the atime option may not work in the expected way)
mtime when data in a file was last modified.
Depending on what are you trying to do, the mtime option could be your best option.
Besides, you should check the print0 option. From man find:
-print0
True; print the full file name on the standard output, followed by a null character (instead of the newline character that -print uses). This allows file names that contain newlines or
other types of white space to be correctly interpreted by programs that process the find output. This option corresponds to the -0 option of xargs.
I do not know what are you trying to do but this command could be useful for you:
find /var/www -mtime +180 -print0 | xargs -0 tar -czf example.tar.gz
Try this:
find /var/www -ctime +180 | xargs tar cf test.tar
The ctime parameter tells you the difference between current time and each files modification times, and if you use the + instead of minus it will give you the "files modified in a date older than x days".
Then just pass it to tar with xargs and you should be set.
I just ran a php script which filled up my nix servers harddrive with 15GB of some sort of junk, how do i find the junk so I can delete it? I'm not sure if it's a huge error_doc file or what
One option is to use the find command.
find / -type f -size +50M
Will search downwards from the root directory for items which are files larger than 50MB. If you want to limit how many subdirectories you want to search, you can use the -maxdepth switch.
find / -maxdepth 3 -type f -size +50M
will look for files larger than 50MB, but will only recurse 3 directories down.
This assumes that you know that the files which were created are larger than a certain size, and you can pick them out if they are displayed.
You might also be able to make use of the knowledge that the files were created recently.
find / -type f -mmin 60
should find files which were modified in the past hour.
Given a directory i'm looking for a bash one-liner to get a recursive list of all files with their size and modified time tab separated for easy parsing. Something like:
cows/betsy 145700 2011-03-02 08:27
horses/silver 109895 2011-06-04 17:43
You can use stat(1) to get the information you want, if you don't want the full ls -l output, and you can use find(1) to get a recursive directory listing. Combining them into one line, you could do this:
# Find all regular files under the current directory and print out their
# filenames, sizes, and last modified times
find . -type f -exec stat -f '%N %z %Sm' '{}' +
If you want to make the output more parseable, you can use %m instead of %Sm to get the last modified time as a time_t instead of as a human-readable date.
find is perfect for recursively searching through directories. The -ls action tells it to output its results in ls -l format:
find /dir/ -ls
On Linux machines you can print customized output using the -printf action:
find /dir/ -printf '%p\t%s\t%t\n'
See man find for full details on the format specifiers available with -printf. (This is not POSIX-compatible and may not be available on other UNIX flavors.)
find * -type f -printf '%p\t%s\t%TY-%Tm-%Td %Tk:%TM\n'
If you prefer fixed-width fields rather than tabs, you can do things like changing %s to %10s.
I used find * ... to avoid the leading "./" on each file name. If you don't mind that, use . rather than * (which also shows files whose names start with .). You can also pipe the output through sed 's/^\.\///'.
Note that the output order will be arbitrary. Pipe through sort if you want an ordered listing.
You could try this for recursive listing from current folder called "/from_dir"
find /from_dir/* -print0 | xargs -0 stat -c “%n|%A|%a|%U|%G” > permissions_list.txt
Lists files and directories passes through to stat command and puts all the info into a file called permissions_list.txt
“%n|%A|%a|%U|%G” will give you the following result in the file:
from_
dir|drwxr-sr-x|2755|root|root
from_dir/filename|-rw-r–r–|644|root|root
Cheers!
I am trying to create a shell script/command that find error_log files created by PHP, check their file sizes and remove them if over a certain size.
I have only gotten as far as printing the files and file sizes with the code below
for i in `locate -r 'error_log$'`;do echo "$i|" `stat -c %s $i`;done
Can anyone help?
Thanks in advance!
find $DIR -type f -name error_log -size +${BYTES_MIN}c -print0 |xargs -0 rm
For example:
find . -type f -name error_log -size +500k -print0 |xargs -0 rm
This will quietly remove any error log file anywhere under the current directory and larger than 500k (c for bytes, k for kilobytes, M for megabytes, ...). If you wish to see the destruction done then add -v to rm.
I would recommend using logrotate, but that presupposes that you know where the log files are. I wouldn't use locate since that uses a database that may be stale or even not updated at all.