I'm trying to write a script used on a buffer box that does full packet capture of network traffic. As it's for a fairly big network we split the captures into 100MB segments. At times of high network traffic oftentimes over a one minute period we will have multiple pcaps which cover that period.
So what I want to do is have a bash script that lets the analyst who is searching for something specify a date and time and how many minutes either side of it they want to search for files. Obviously I can do something like this -
ls -al | grep "Dec 1" | grep 02:00
ls -al | grep "Dec 1" | grep 02:01
and so on, get each result and grep each file individually for the specific keyword I'm looking for, but I'd like to be able to do a wider search for all files created within a time range and then grep each of them for the keyword.
I'm not entirely sure how to do that, any help would be appreciated.
find . -maxdepth 1 -newermt "2013-10-28 00:00:00" ! -newermt "2013-10-29 00:00:00"
What I want is for an analyst to say 1st December at 11:00am with a
keyword of "foo" searching 5 minutes either side. The script should
find all files created between 10:55am and 11:05am and grep them for
the keyword "foo"
This script uses touch -d to create temporary files with time stamps of the start and end of the time range, because older versions of find have the option -newer only, not -newermt, and touch -d conveniently allows using the given time specification (with little modification) with the minutes adjustment. The necessary modifications to the given date are done with sed and consist of moving the day after the month and removing suffixes as st or nd as well as the word at.
read -p'date and time: ' dat
read -p'+/- minutes: ' min
read -p'keyword: ' key
dat=`sed 's/\([0-9]\+\)\(st\|nd\|rd\|th\|\) \([^ ]*\)/\3 \1/; s/at //' <<<$dat`
touch -d"$dat $min min" /tmp/to
touch -d"$dat -$min min" /tmp/from
find . -type f -newer /tmp/from ! -newer /tmp/to | xargs grep "$key"
rm /tmp/from /tmp/to
Check out find with the -cmin or -ctime arguments.
So,
find -iname "*.log" -mtime +30 -mtime -90 -exec grep plasma {} \;
, would find files ending in ".log" which were modified greater than 30 days ago, but less than 90 days, then run said file through grep looking for the word "plasma".
Say you want 20131130 from 0100 to 0130 -
This does that with find:
touch -t 201311300100 dummy1
touch -t 201311300130 dummy2
find /path/to/logs type -f \( -newer dummy1 -a ! -newer dummy2 \) -name '*.log'
the 201311300100 bit is a touch timestring. I posted the most vanilla version I know because of the UNIX tag....
Related
E.g., a MySQL server is running on my Ubuntu machine. Some data has been changed during the last 24 hours.
What (Linux) scripts can find the files that have been changed during the last 24 hours?
Please list the file names, file sizes, and modified time.
To find all files modified in the last 24 hours (last full day) in a particular specific directory and its sub-directories:
find /directory_path -mtime -1 -ls
Should be to your liking
The - before 1 is important - it means anything changed one day or less ago.
A + before 1 would instead mean anything changed at least one day ago, while having nothing before the 1 would have meant it was changed exacted one day ago, no more, no less.
Another, more humanist way, is to use -newermt option which understands human-readable time units.
Unlike -mtime option which requires the user to read find documentation to figure our what time units -mtime expects and then having the user to convert its time units into those, which is error-prone and plain user-unfriendly. -mtime was barely acceptable in 1980s, but in the 21st century -mtime has the convenience and safety of stone age tools.
Example uses of -newermt option with the same duration expressed in different human-friendly units:
find /<directory> -newermt "-24 hours" -ls
find /<directory> -newermt "1 day ago" -ls
find /<directory> -newermt "yesterday" -ls
You can do that with
find . -mtime 0
From man find:
[The] time since each file was last modified is divided by 24 hours and any remainder is discarded. That means that to
match -mtime 0, a file will have to have a modification in the past which is less than 24 hours ago.
On GNU-compatible systems (i.e. Linux):
find . -mtime 0 -printf '%T+\t%s\t%p\n' 2>/dev/null | sort -r | more
This will list files and directories that have been modified in the last 24 hours (-mtime 0). It will list them with the last modified time in a format that is both sortable and human-readable (%T+), followed by the file size (%s), followed by the full filename (%p), each separated by tabs (\t).
2>/dev/null throws away any stderr output, so that error messages don't muddy the waters; sort -r sorts the results by most recently modified first; and | more lists one page of results at a time.
For others who land here in the future (including myself), add a -name option to find specific file types, for instance: find /var -name "*.php" -mtime -1 -ls
This command worked for me
find . -mtime -1 -print
Find the files...
You can set type f = file
find /directory_path -type f -mtime -1 -exec ls -lh {} \;
👍
I use AIX via telnet here at work, and I'd like to know how to find files in a specific folder between a date range. For example: I want to find all files in folder X that were created between 01-Aug-13 and 31-Aug-13.
Observations:
The touch trick (where you create two empty files to use the -newer option) does not work for me, once the user roles that I have on the server does not allow me to create files.
I need to find between specific dates, not days (like: files that were created more than 30 days ago, etc...)
If you use GNU find, since version 4.3.3 you can do:
find -newerct "1 Aug 2013" ! -newerct "1 Sep 2013" -ls
It will accept any date string accepted by GNU date -d.
You can change the c in -newerct to any of a, B, c, or m for looking at atime/birth/ctime/mtime.
Another example - list files modified between 17:30 and 22:00 on Nov 6 2017:
find -newermt "2017-11-06 17:30:00" ! -newermt "2017-11-06 22:00:00" -ls
Full details from man find:
-newerXY reference
Compares the timestamp of the current file with reference. The reference argument is normally the name of a file (and one of its timestamps is used
for the comparison) but it may also be a string describing an absolute time. X and Y are placeholders for other letters, and these letters select
which time belonging to how reference is used for the comparison.
a The access time of the file reference
B The birth time of the file reference
c The inode status change time of reference
m The modification time of the file reference
t reference is interpreted directly as a time
Some combinations are invalid; for example, it is invalid for X to be t. Some combinations are not implemented on all systems; for example B is not
supported on all systems. If an invalid or unsupported combination of XY is specified, a fatal error results. Time specifications are interpreted as
for the argument to the -d option of GNU date. If you try to use the birth time of a reference file, and the birth time cannot be determined, a fatal
error message results. If you specify a test which refers to the birth time of files being examined, this test will fail for any files where the
birth time is unknown.
Try the following command:
find /var/tmp -mtime +2 -a -mtime -8 -ls
This will allow you to find files in /var/tmp folder that are older than 2 days but not older than 8 days.
Some good solutions on here. Wanted to share mine as well as it is short and simple.
I'm using find (GNU findutils) 4.5.11
$ find search/path/ -newermt 20130801 \! -newermt 20130831
You can use the below to find what you need.
Find files older than a specific date/time:
find ~/ -mtime $(echo $(date +%s) - $(date +%s -d"Dec 31, 2009 23:59:59") | bc -l | awk '{print $1 / 86400}' | bc -l)
Or you can find files between two dates. First date more recent, last date, older. You can go down to the second, and you don't have to use mtime. You can use whatever you need.
find . -mtime $(date +%s -d"Aug 10, 2013 23:59:59") -mtime $(date +%s -d"Aug 1, 2013 23:59:59")
Use stat to get the creation time. You can compare the time in the format YYYY-MM-DD HH:MM:SS lexicographically.
This work on Linux with modification time, creation time is not supported. On AIX, the -c option might not be supported, but you should be able to get the information anyway, using grep if nothing else works.
#! /bin/bash
from='2013-08-01 00:00:00.0000000000' # 01-Aug-13
to='2013-08-31 23:59:59.9999999999' # 31-Aug-13
for file in * ; do
modified=$( stat -c%y "$file" )
if [[ $from < $modified && $modified < $to ]] ; then
echo "$file"
fi
done
You can use the following commands to list files between 2 specific dates:
Search on current (.) directory:
find . -type f -newermt "2019-01-01" ! -newermt "2019-05-01"
Search on /var/any/directory/ directory:
find /var/any/directory/ -type f -newermt "2019-01-01" ! -newermt "2019-05-01"
Script oldfiles
I've tried to answer this question in a more complete way, and I ended up creating a complete script with options to help you understand the find command.
The script oldfiles is in this repository
To "create" a new find command you run it with the option -n (dry-run), and it will print to you the correct find command you need to use.
Of course, if you omit the -n it will just run, no need to retype the find command.
Usage:
oldfiles [-v...] ([-h|-V|-n] | {[(-a|-u) | (-m|-t) | -c] (-i | -d | -o| -y | -g) N (-\> | -\< | -\=) [-p "pat"]})
Where the options are classified in the following groups:
Help & Info:
-h, --help : Show this help.
-V, --version : Show version.
-v, --verbose : Turn verbose mode on (cumulative).
-n, --dry-run : Do not run, just explain how to create a "find" command
Time type (access/use, modification time or changed status):
-a or -u : access (use) time
-m or -t : modification time (default)
-c : inode status change
Time range (where N is a positive integer):
-i N : minutes (default, with N equal 1 min)
-d N : days
-o N : months
-y N : years
-g N : N is a DATE (example: "2017-07-06 22:17:15")
Tests:
-p "pat" : optional pattern to match (example: -p "*.c" to find c files) (default -p "*")
-\> : file is newer than given range, ie, time modified after it.
-< : file is older than given range, ie, time is from before it. (default)
-= : file that is exactly N (min, day, month, year) old.
Example:
Find C source files newer than 10 minutes (access time) (with verbosity 3):
oldfiles -a -i 10 -p"*.c" -\> -nvvv
Starting oldfiles script, by beco, version 20170706.202054...
oldfiles -vvv -a -i 10 -p "*.c" -\> -n
Looking for "*.c" files with (a)ccess time newer than 10 minute(s)
find . -name "*.c" -type f -amin -10 -exec ls -ltu --time-style=long-iso {} +
Dry-run
Find H header files older than a month (modification time) (verbosity 2):
oldfiles -m -o 1 -p"*.h" -\< -nvv
Starting oldfiles script, by beco, version 20170706.202054...
oldfiles -vv -m -o 1 -p "*.h" -\< -n
find . -name "*.h" -type f -mtime +30 -exec ls -lt --time-style=long-iso {} +
Dry-run
Find all (*) files within a single day (Dec, 1, 2016; no verbosity, dry-run):
oldfiles -mng "2016-12-01" -\=
find . -name "*" -type f -newermt "2016-11-30 23:59:59" ! -newermt "2016-12-01 23:59:59" -exec ls -lt --time-style=long-iso {} +
Of course, removing the -n the program will run the find command itself and save you the trouble.
I hope this helps everyone finally learn this {a,c,t}{time,min} options.
The LS output:
You will also notice that the ls option ls OPT changes to match the type of time you choose.
Link to clone/download of the oldfiles script:
https://github.com/drbeco/oldfiles
Explanation: Use unix command find with -ctime (creation time) flag.
The find utility recursively descends the directory tree for each path listed, evaluating an expression (composed of the 'primaries' and 'operands') in terms of each file in the tree.
Solution: According to official documentation:
-ctime n[smhdw]
If no units are specified, this primary evaluates to true if the difference
between the time of last change of file status information and the time find
was started, rounded up to the next full 24-hour period, is n 24-hour peri-
ods.
If units are specified, this primary evaluates to true if the difference
between the time of last change of file status information and the time find
was started is exactly n units. Please refer to the -atime primary descrip-
tion for information on supported time units.
Formula:
find <path> -ctime +[number][timeMeasurement] -ctime -[number][timeMeasurment]
Examples:
1.Find everything that were created after 1 week ago and before 2 weeks ago.
find / -ctime +1w -ctime -2w
2.Find all javascript files (.js) in current directory that were created between 1 day ago to 3 days ago.
find . -name "*\.js" -type f -ctime +1d -ctime -3d
I am trying to find the total disk space used by files older than 180 days in a particular directory. This is what I'm using:
find . -mtime +180 -exec du -sh {} \;
but the above is quiet evidently giving me disk space used by every file that is found. I want only the total added disk space used by the files. Can this be done using find and exec command ?
Please note I simply don't want to use a script for this, it will be great if there could be a one liner for this. Any help is highly appreciated.
Why not this?
find /path/to/search/in -type f -mtime +180 -print0 | du -hc --files0-from - | tail -n 1
#PeterT is right. Almost all these answers invoke a command (du) for each file, which is very resource intensive and slow and unnecessary. The simplest and fastest way is this:
find . -type f -mtime +356 -printf '%s\n' | awk '{total=total+$1}END{print total/1024}'
du wouldn't summarize if you pass a list of files to it.
Instead, pipe the output to cut and let awk sum it up. So you can say:
find . -mtime +180 -exec du -ks {} \; | cut -f1 | awk '{total=total+$1}END{print total/1024}'
Note that the option -h to display the result in human-readable format has been replaced by -k which is equivalent to block size of 1K. The result is presented in MB (see total/1024 above).
Be careful not to take into account the disk usage by the directories. For example, I have a lot of files in my ~/tmp directory:
$ du -sh ~/tmp
3,7G /home/rpet/tmp
Running the first part of example posted by devnull to find the files modified in the last 24 hours, we can see that awk will sum the whole disk usage of the ~/tmp directory:
$ find ~/tmp -mtime 0 -exec du -ks {} \; | cut -f1
3849848
84
80
But there is only one file modified in that period of time, with very little disk usage:
$ find ~/tmp -mtime 0
/home/rpet/tmp
/home/rpet/tmp/kk
/home/rpet/tmp/kk/test.png
$ du -sh ~/tmp/kk
84K /home/rpet/tmp/kk
So we need to take into account only the files and exclude the directories:
$ find ~/tmp -type f -mtime 0 -exec du -ks {} \; | cut -f1 | awk '{total=total+$1}END{print total/1024}'
0.078125
You can also specify date ranges using the -newermt parameter. For example:
$ find . -type f -newermt "2014-01-01" ! -newermt "2014-06-01"
See http://www.commandlinefu.com/commands/view/8721/find-files-in-a-date-range
You can print file size with find using the -printf option, but you still need awk to sum.
For example, total size of all files older than 365 days:
find . -type f -mtime +356 -printf '%s\n' \
| awk '{a+=$1;} END {printf "%.1f GB\n", a/2**30;}'
If i have 3 files called 1.txt,2.txt and 3.txt for example and they were created an hour apart, say 1pm 2pm and 3pm respectively. What I need is a command that finds all files modified within an hour of a specific file.
I'm in the same directory as the files in the terminal and all files are setuid permission
I've been trying:
find . -type f -perm -4000 -newer 2.txt -mmin -60 -print
This should return 3.txt but it doesn't
What would use to file created in the hour before or after 2.txt?
Try this
touch /tmp/temp -t time1
touch /tmp/ntemp -t time2
find . -newer /tmp/temp -a ! -newer /tmp/ntemp -exec ls -l {} \; 2>/dev/null
where
time1 = time of file creation - 1hr
time2 = time of file creation + 1hr
Ex:
time1 = 201210041500
time2 = 201210041700
Here is my logic -
First get time of last access of file in seconds since Epoch in some variable.
time_in_sec=$(stat -c %X 2.txt)
Get the time of last hour ( ie 3600 seconds back )
one_hr_old_time_in_sec=`expr $time_in_sec - 3600`
Convert it into format suitable for find command
newerthan=$(date -d #$one_hr_old_time_in_sec '+%m-%d-%y%n %H:%M:%S')
Convert time of original file in format suitable for find command
olderthan=$(date -d #$time_in_sec '+%m-%d-%y%n%H:%M:%S')
Get the list of files modified between two time periods using find command
find . -newermt "$newerthan" ! -newermt "$olderthan" -print
If it works you can write a small shell script which will take file name as parameter and can try for +3600 also.
Honestly, I haven't tried it. Hope it works !
The previous solution can be simplified since -newermt shall accept the same formats than 'date' including #N for N seconds since epoch.
Also for the 'stat' command it is probably better to use -Y (the time of last modification) instead of -X (the time of last access).
So the command to find the files modified + or - 1 hours of 2.txt is
N=$(stat -c %Y 2.txt)
find . -newermt #$((N-3600)) ! -newermt #$((N+3600)) -print
If you are running this after 4pm, given your example, it makes sense that it wouldn't return 3.txt, as -newer 2.txt -mmin -60 means "modified in the last 60 minutes, and more recently than 2.txt", not "modified less than 60 minutes after 2.txt". I don't think find currently has options to do what you're wanting (at least, the version I have doesn't), but it shouldn't be too hard to script in python or perl.
We have linux machine we would like to check what new files have been added between a certain date range.
I only have SSH access to this box and it's openSUSE 11.1
Is there some sort of command that can give me a list of files that have been added to the filesystem between say 04/05/2011 and 05/05/2011
Thanks
Regards
Gabriel
There are bunch of ways for doing that.
First one:
start_date=201105040000
end_date=201105042359
touch -t ${start_date} start
touch -t ${end_date} end
find /you/path -type f -name '*you*pattern*' -newer start ! -newer end -exec ls -s {} \;
Second one:
find files modified between 20 and 21 days ago:
find -ctime +20 -ctime -21
finds files modified between 2500 and 2800 minutes ago:
find -cmin +2500 -cmin -2800
And read this topic too.
Well, you could use find to get a list of all the files that were last-modified in a certain time window, but that isn't quite what you want. I don't think you can tell just from a file's metadata when it came into existence.
Edit: To list the files along with their modification dates, you can pipe the output of find through xargs to run ls -l on all the files, which will show the modification time.
find /somepath -type f ... -print0 | xargs -0 -- ls -l
I misunderstood your question. Depending on what filesystem you are using, it may or may not store creation time.
My understanding is that ext2/3/4 do not store creation time, but modified, changed (status, which is slightly different), and access times are.
Fat32 on the other hand does contain creation timestamps IIRC.
If you are using an ext filesystem, you have two options it seems:
1.Settle for finding all of the files that were modified between two dates (which will include created files, but also files that were just edited). You could do this using find.
2.Create a script/cronjob that will document the contents of your filesystem at some interval, e.g.
find / > filesystem.$(date +%s).log
and then run diffs to see what has been added. This, of course, would prevent you from looking backwards to time before you started making these logs.
You can try one of these:
find -newerct "1 Aug 2013" ! -newerct "1 Sep 2013" -ls
find . -mtime $(date +%s -d"Jan 1, 2013 23:59:59") -mtime $(date +%s -d"Jan 2, 2016 23:59:59")
find /media/WD/backup/osool/olddata/ -newermt 20120101T1200 -not -newermt 20130101T1400
find . -mtime +1 -mtime -3
find . -mtime +1 -mtime -3 > files_from_yesterday.txt 2>&1
find . -mtime +1 -mtime -3 -ls > files_from_yesterday.txt 2>&1
touch -t 200506011200 first
touch -t 200507121200 last
find / -newer first ! -newer last
#!/bin/bash
for i in `find Your_Mail_Dir/ -newermt "2011-01-01" ! -newermt "2011-12-31"`; do
mv $i /moved_emails_dir/
Hope this helps.