I'm on a RedHat Linux 6 machine, running Elasticsearch and Logstash. I have a bunch of log files that were rotated daily from back in June til August. I am trying to figure out the best way to tar them up to save some diskspace, without manually taring up each one. I'm a bit of a newbie at scripting, so I was wondering if someone could help me out? The files have the name elasticsearch-cluster.log.datestamp. Ideally they would all be in their individual tar files, so that it'd be easier to go back and take a look at that particular day's logs if needed.
You could use a loop :
for file in elasticsearch-cluster.log.*
do
tar zcvf "$file".tar.gz "$file"
done
Or if you prefer a one-liner (this is recursive):
find . -name 'elasticsearch-cluster.log.*' -print0 | xargs -0 -I {} tar zcvf {}.tar.gz {}
or as #chepner mentions with the -exec option:
find . -name 'elasticsearch-cluster.log.*' -exec tar zcvf {}.tar.gz {} \;
or if want to exclude already zipped files:
find . -name 'elasticsearch-cluster.log.*' -not -name '*.tar.gz' -exec tar zcvf {}.tar.gz {} \;
If you don't mind all the files being in a single tar.gz file, you can do:
tar zcvf backups.tar.gz elasticsearch-cluster.log.*
All these commands leave the original files in place. After you validate the tar.gz files, you can delete them manually.
Related
I am writing a shell script in which a tar file with today's date and time will be created.
tar -zvcf "log_grabber$(date '+%y-%m-%d_%H%M').tar.gz" --files-from /dev/null
Now, to add more files to this tar files after running the find command. How can I get the name of the tar file and use it in the output of the find command?
find . -type f -name 'local*' -newermt "$user_date" -exec tar -rvf <variable tar file> {} \;
Any help will be very much useful.
Instead of
tar -zvcf "log_grabber$(date '+%y-%m-%d_%H%M').tar.gz" --files-from /dev/null
Create a variable with the name first and use that:
name="log_grabber$(date '+%y-%m-%d_%H%M').tar.gz"
tar -zvcf "$name" --files-from /dev/null
And then:
find . -type f -name 'local*' -newermt "$user_date" -exec tar -rvf "$name" {} +
Note that I changed \; to + so that tar gets multiple files in one invocation, rather than one tar invocation per file.
It seems I've got a problem. I've got some different file types in my current directory, and I want to just tar the .png files. I started with this:
find -name "*.png" | tar -cvf backupp.tar
It wouldn't work because I didn't specify which files, so looking on how others did it, I added xargs:
find -name "*.png" | xargs tar -cvf backupp.tar
It did work this time, and backupp.tar file was created, but here is the problem. I can't seem to extract it. Whenever I type:
tar -xvf backupp.tar
Nothings happens. I've tried changing chmod and sudo, but nothing gives in.
So, did I type the wrong command completely or is there somethings I just missed?
tar expects a list of names as arguments. Your use of xargs can be improved by adding the -print0 option to find and adding the -0 option to xargs to insure find is providing filenames separated by a nul-character and that xargs is processing a list of filenames separated by the same. This prevents any whitespace or other stray characters in the filenames from causing problems, e.g.
find dir -type f -name "*.png" -print0 | xargs -0 tar -cf tarfile.tar
The above will find all files in or below dir matching name "*.png" and provide a list of filenames separated by the nul-character to xargs for use by tar. You can list the files contained in the resulting archive with:
tar -tf tarfile.tar
Consider using compression (if wanted) by adding the z (gzipped) j (bzip2) or J (xz compression) and the appropriate extension to reduce you archive size. e.g.
... | xargs -0 tar -czf tarfile.tar.gz
I found what I thought was a solution in this forum to being able to find my specific LOG files and then doing TAR.GZ on these files for a backup. However, when execute the command I'm getting an error. The command prior to the pipe works great and finds the files that I'm needing but when trying to create the backup file I blow up. Any suggestions/direction would be appreciated. Thanks.
Here is the command:
find /var/log/provenir -type f -name "*2014-09-08.log" | tar -cvzf backupProvLogFiles_20140908.tar.gz
Here is the error I'm getting:
find /var/log/provenir -type f -name "*2014-09-08.log" | tar -czvf backupProvLogFiles_20140908.tar.gz --null -T -
tar: Removing leading `/' from member names
tar: /var/log/provenir/BureauDE32014-09-08.log\n/var/log/provenir/DE_HTTP2014-09
-08.log\n/var/log/provenir/BureauDE22014-09-08.log\n/var/log/provenir/DE_HTTP220
14-09-08.log\n: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
You can also use gzip to do so
find /var/log/provenir -type f -name "*2014-09-08.log" | gzip > tar -cvzf backupProvLogFiles_20140908.tar EDIT
EDIT
A better solution would be to use command substituion
tar -cvzf backupProvLogFiles_20140908.tar $(find /var/log/provenir -type f -name "*2014-09-08.log")
I think you mean something like this:
find . -name "*XYZ*" -type f -print | tar -cvz -T - -f SomeFile.tgz
I was finally able to find a solution just in case someone else might be looking for another option to answer this question:
find /var/log/provenir -type f -name "*2014-09-08.log" -print0 | tar -czvf /var/log/provenir/barchive/backupProvLogFile_20140908.tar.gz --null -T -
This worked great. The answer came from this post: Find files and tar them (with spaces)
Thanks again for the help I received.
Regards.
I have a list of files with different modification times, 1_raw,2_raw,3_raw... I want to find files that are modified more than 10 days ago and zip them to release disk space. However, the command:
find . -mtime +10 |xargs tar -cvzf backup.tar.gz
will create a new file backup.tar.gz
What I want is to create a tarball for each file, so that I can easily unzip each of them when needed. After the command, my files should become: 1_raw.tar.gz, 2_raw.tar.gz, 3_raw.tar.gz...
Is there anyway to do this? Thanks!
Something like this is what you are after:
find . -mtime +10 -type f -print0 | while IFS= read -r -d '' file; do
tar -cvzf "${file}.tar.gz" "$file"
done
The -type f was added so that it doesn't also process directories, just files.
This adds a compressed archive of each file that was modified more than 10 days ago, in all subdirectories, and places the compressed archive next to its respective unarchived version (in the same folder). I assume this is what you wanted.
If you didn't need to handle whitespaces in the path, you could do with simply:
for f in $(find . -mtime +10 -type f) ; do
tar -cvzf "${f}.tar.gz" "$f"
done
Simply, try this
$ find . -mtime +10 | xargs -I {} tar czvf {}.tar.gz {}
Here, {} indicates replace-str
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.
https://linux.die.net/man/1/xargs
I have a bunch of zip files, and I'm trying to make a bash script to automate the unzipping of certain files from it.
Things is, although I know the name of the file I want, I don't know the name of the folder it's in; it is one folder depth in
How can I extract these files, preferably discarding the folder?
Here's how to unzip any given file at any depth and junk the folder paths on the way out:
unzip -j somezip.zip *somefile.txt
The -j junks any folder structure in the zip file and the asterisk gives a wildcard to match along any path.
if you're in:
some_directory/
and the zip files are in any number of subdirectories, say:
some_directory/foo
find ./ -name myfile.zip -exec unzip {} -d /directory \;
Edit: As for the second part, removing the directory that contained the zip file I assume?
find ./ -name myfile.zip -exec unzip {} -d /directory \; -exec echo rm -rf `dirname {}` \;
Notice the "echo." That's a sanity check. I always echo first when executing something destructive like rm -rf in a loop/iterative sequence like this. Good luck!
Have you tried unzip somefile.zip "*/blah.txt"?
You can use find to find the file that you need to unzip, and xargs to call unzip:
find /path/to/root/ -name 'zipname.zip' -print0 | xargs -0 unzip
print0 enables the command to work with files or paths that have white space in them. -0 is the option to xargs that makes it work with print0.