I have to tar a list of files, without path, that is a result of a find via sh (for crontab use).
In ubuntu's shell each command works fine but in .sh not.
I tried with :
#!/bin/sh
tar -zcvf /destination/one-$(date +"%Y%m%d").tgz < find /myfolder/ -iname 'one*' -printf '%f\n'
And also with
#!/bin/sh
find /myfolder/ -iname 'one*' -print0 | tar -czvf /destination/one-$(date +"%Y%m%d").tar.gz --null -T -
But both failed. May someone help? Alternatives ?
Additional scenario info:
/myfolder/ contains:
one1.log
one2.log
one3.log
two1.log
two2.log
I want one.tgz containing one1.log, one2.log, one3.log
I think you are looking to pass the filenames to tar on stdin :
find . -name \*.png -print0 | tar -cv --null -T- -f tarball.tar
In my case:
find /myfolder/ -iname "one*" -print0 | tar -czv --null -T- -f /destination/one-$(date +"%Y%m%d").tar.gz
Related
I am writing a shell script in which a tar file with today's date and time will be created.
tar -zvcf "log_grabber$(date '+%y-%m-%d_%H%M').tar.gz" --files-from /dev/null
Now, to add more files to this tar files after running the find command. How can I get the name of the tar file and use it in the output of the find command?
find . -type f -name 'local*' -newermt "$user_date" -exec tar -rvf <variable tar file> {} \;
Any help will be very much useful.
Instead of
tar -zvcf "log_grabber$(date '+%y-%m-%d_%H%M').tar.gz" --files-from /dev/null
Create a variable with the name first and use that:
name="log_grabber$(date '+%y-%m-%d_%H%M').tar.gz"
tar -zvcf "$name" --files-from /dev/null
And then:
find . -type f -name 'local*' -newermt "$user_date" -exec tar -rvf "$name" {} +
Note that I changed \; to + so that tar gets multiple files in one invocation, rather than one tar invocation per file.
find -name archive.zip -exec unzip {} file.txt \;
This command finds all files named archive.zip and unzips file.txt to the folder that I execute the command from, is there a way to unzip the file to the same folder where the .zip file was found? I would like file.txt to be unzipped to folder1.
folder1\archive.zip
folder2\archive.zip
I realize $dirname is available in a script but I'm looking for a one line command if possible.
#iheartcpp - I successfully ran three alternatives using the same base command...
find . -iname "*.zip"
... which is used to provide the list of / to be passed as an argument to the next command.
Alternative 1: find with -exec + Shell Script (unzips.sh)
File unzips.sh:
#!/bin/sh
# This will unzip the zip files in the same directory as the zip are
for f in "$#" ; do
unzip -o -d `dirname $f` $f
done
Use this alternative like this:
find . -iname '*.zip' -exec ./unzips.sh {} \;
Alternative 2: find with | xargs _ Shell Script (unzips)
Same unzips.sh file.
Use this alternative like this:
find . -iname '*.zip' | xargs ./unzips.sh
Alternative 3: all commands in the same line (no .sh files)
Use this alternative like this:
find . -iname '*.zip' | xargs sh -c 'for f in $#; do unzip -o -d `dirname $f` $f; done;'
Of course, there are other alternatives but hope that the above ones can help.
I am trying to move about 700,000 .jpg files from one directory to another in my Ubuntu server. I tried the following:
xargs mv * -t /var/www/html/
and
echo (*.jpg|*.png|*.bmp) | xargs mv -t /var/www/html/
and
echo (*.jpg) | xargs mv -t /var/www/html/
and
find . -name "*.jpg" -print0 | xargs mv * ../
and they all give me the same error: /usr/bin/xargs: Argument list too long
what should I do? Please help me out. Thanks :)
If you use find I would recommend you to use the -exec attribute. So your result should be find . -name "*.jpg" -exec mv {} /home/new/location \;.
However I would recommend to check what the find command returns you, replacing the exec part with: -exec ls -lrt {} \;
Try:
find /path/to/old-directory -type f | xargs -i mv "{}" /path/to/new-directory
You could have tried:
for f in *.jpg do;
mv -tv $f /var/www/html/
done
for f in *.png do;
mv -tv $f /var/www/html/
done
for f in *.bmp do;
mv -tv $f /var/www/html/
done
also, you should carefully read xargs(1); I strongly suspect that
find . -name "*.jpg" -print0 | xargs -n 1000 -I '{}' mv '{}' ../
should work for you
At last, learn more about rename(1). It is probably enough for the job.
I want to tar and all .php and .html files in a directory and its subdirectories. If I use
tar -cf my_archive *
it tars all the files, which I don't want. If I use
tar -cf my_archive *.php *.html
it ignores subdirectories. How can I make it tar recursively but include only two types of files?
find ./someDir -name "*.php" -o -name "*.html" | tar -cf my_archive -T -
If you're using bash version > 4.0, you can exploit shopt -s globstar to make short work of this:
shopt -s globstar; tar -czvf deploy.tar.gz **/Alice*.yml **/Bob*.json
this will add all .yml files that starts with Alice from any sub-directory and add all .json files that starts with Bob from any sub-directory.
One method is:
tar -cf my_archive.tar $( find -name "*.php" -or -name "*.html" )
There are some caveats with this method however:
It will fail if there are any files or directories with spaces in them, and
it will fail if there are so many files that the maximum command line length is full.
A workaround to these could be to output the contents of the find command into a file, and then use the "-T, --files-from FILE" option to tar.
This will handle paths with spaces:
find ./ -type f -name "*.php" -o -name "*.html" -exec tar uvf myarchives.tar {} +
If you want to produce a zipped tar file (.tgz) and want to avoid problems with spaces in filenames:
find . \( -name \*.php -o -name \*.html \) -print0 | xargs -0 tar -cvzf my_archive.tgz
The -print0 “primary” of find separates output filenames using the NULL (\0) byte, thus playing well with the -0 option of xargs, which appends its (NULL-separated, in this case) input as arguments to the command it precedes.
The parentheses around the two -name primaries are needed, because otherwise the -print0 would only output the filenames of the second -name (there is no implied printing if -print or -print0 is present, and these only have an effect if they are evaluated).
If you need to skip some filenames or directories (e.g., the node_modules directory if you work with Node.js), prepend one or more -prune primaries like this:
find . -name skipThisName -prune -o \
-name skipThisOtherName -prune -o \
\( -name \*.php -o -name \*.html \) -print0 | xargs -0 tar -cvzf my_archive.tgz
Put them in a file
find . \( -name "*.php" -o -name "*.html" \) -print > files.txt
Then use the file as input to tar, use -I or -T depending on the version of tar you use
Use h to copy symbolic links
tar cfh my.tar -I files.txt
Easy with zsh:
tar cvzf foo.tar.gz **/*.(php|html)
find ./ -type f -name "*.php" -o -name "*.html" -printf '%P\n' |xargs tar -I 'pigz -9' -cf target.tgz
for multicore or just for one core:
find ./ -type f -name "*.php" -o -name "*.html" -printf '%P\n' |xargs tar -czf target.tgz
Alright, so simple problem here. I'm working on a simple back up code. It works fine except if the files have spaces in them. This is how I'm finding files and adding them to a tar archive:
find . -type f | xargs tar -czvf backup.tar.gz
The problem is when the file has a space in the name because tar thinks that it's a folder. Basically is there a way I can add quotes around the results from find? Or a different way to fix this?
Use this:
find . -type f -print0 | tar -czvf backup.tar.gz --null -T -
It will:
deal with files with spaces, newlines, leading dashes, and other funniness
handle an unlimited number of files
won't repeatedly overwrite your backup.tar.gz like using tar -c with xargs will do when you have a large number of files
Also see:
GNU tar manual
How can I build a tar from stdin?, search for null
There could be another way to achieve what you want. Basically,
Use the find command to output path to whatever files you're looking for. Redirect stdout to a filename of your choosing.
Then tar with the -T option which allows it to take a list of file locations (the one you just created with find!)
find . -name "*.whatever" > yourListOfFiles
tar -cvf yourfile.tar -T yourListOfFiles
Try running:
find . -type f | xargs -d "\n" tar -czvf backup.tar.gz
Why not:
tar czvf backup.tar.gz *
Sure it's clever to use find and then xargs, but you're doing it the hard way.
Update: Porges has commented with a find-option that I think is a better answer than my answer, or the other one: find -print0 ... | xargs -0 ....
If you have multiple files or directories and you want to zip them into independent *.gz file you can do this. Optional -type f -atime
find -name "httpd-log*.txt" -type f -mtime +1 -exec tar -vzcf {}.gz {} \;
This will compress
httpd-log01.txt
httpd-log02.txt
to
httpd-log01.txt.gz
httpd-log02.txt.gz
Would add a comment to #Steve Kehlet post but need 50 rep (RIP).
For anyone that has found this post through numerous googling, I found a way to not only find specific files given a time range, but also NOT include the relative paths OR whitespaces that would cause tarring errors. (THANK YOU SO MUCH STEVE.)
find . -name "*.pdf" -type f -mtime 0 -printf "%f\0" | tar -czvf /dir/zip.tar.gz --null -T -
. relative directory
-name "*.pdf" look for pdfs (or any file type)
-type f type to look for is a file
-mtime 0 look for files created in last 24 hours
-printf "%f\0" Regular -print0 OR -printf "%f" did NOT work for me. From man pages:
This quoting is performed in the same way as for GNU ls. This is not the same quoting mechanism as the one used for -ls and -fls. If you are able to decide what format to use for the output of find then it is normally better to use '\0' as a terminator than to use newline, as file names can contain white space and newline characters.
-czvf create archive, filter the archive through gzip , verbosely list files processed, archive name
Edit 2019-08-14:
I would like to add, that I was also able to use essentially use the same command in my comment, just using tar itself:
tar -czvf /archiveDir/test.tar.gz --newer-mtime=0 --ignore-failed-read *.pdf
Needed --ignore-failed-read in-case there were no new PDFs for today.
Why not give something like this a try: tar cvf scala.tar `find src -name *.scala`
Another solution as seen here:
find var/log/ -iname "anaconda.*" -exec tar -cvzf file.tar.gz {} +
The best solution seem to be to create a file list and then archive files because you can use other sources and do something else with the list.
For example this allows using the list to calculate size of the files being archived:
#!/bin/sh
backupFileName="backup-big-$(date +"%Y%m%d-%H%M")"
backupRoot="/var/www"
backupOutPath=""
archivePath=$backupOutPath$backupFileName.tar.gz
listOfFilesPath=$backupOutPath$backupFileName.filelist
#
# Make a list of files/directories to archive
#
echo "" > $listOfFilesPath
echo "${backupRoot}/uploads" >> $listOfFilesPath
echo "${backupRoot}/extra/user/data" >> $listOfFilesPath
find "${backupRoot}/drupal_root/sites/" -name "files" -type d >> $listOfFilesPath
#
# Size calculation
#
sizeForProgress=`
cat $listOfFilesPath | while read nextFile;do
if [ ! -z "$nextFile" ]; then
du -sb "$nextFile"
fi
done | awk '{size+=$1} END {print size}'
`
#
# Archive with progress
#
## simple with dump of all files currently archived
#tar -czvf $archivePath -T $listOfFilesPath
## progress bar
sizeForShow=$(($sizeForProgress/1024/1024))
echo -e "\nRunning backup [source files are $sizeForShow MiB]\n"
tar -cPp -T $listOfFilesPath | pv -s $sizeForProgress | gzip > $archivePath
Big warning on several of the solutions (and your own test) :
When you do : anything | xargs something
xargs will try to fit "as many arguments as possible" after "something", but then you may end up with multiple invocations of "something".
So your attempt: find ... | xargs tar czvf file.tgz
may end up overwriting "file.tgz" at each invocation of "tar" by xargs, and you end up with only the last invocation! (the chosen solution uses a GNU -T special parameter to avoid the problem, but not everyone has that GNU tar available)
You could do instead:
find . -type f -print0 | xargs -0 tar -rvf backup.tar
gzip backup.tar
Proof of the problem on cygwin:
$ mkdir test
$ cd test
$ seq 1 10000 | sed -e "s/^/long_filename_/" | xargs touch
# create the files
$ seq 1 10000 | sed -e "s/^/long_filename_/" | xargs tar czvf archive.tgz
# will invoke tar several time as it can'f fit 10000 long filenames into 1
$ tar tzvf archive.tgz | wc -l
60
# in my own machine, I end up with only the 60 last filenames,
# as the last invocation of tar by xargs overwrote the previous one(s)
# proper way to invoke tar: with -r (which append to an existing tar file, whereas c would overwrite it)
# caveat: you can't have it compressed (you can't add to a compressed archive)
$ seq 1 10000 | sed -e "s/^/long_filename_/" | xargs tar rvf archive.tar #-r, and without z
$ gzip archive.tar
$ tar tzvf archive.tar.gz | wc -l
10000
# we have all our files, despite xargs making several invocations of the tar command
Note: that behavior of xargs is a well know diccifulty, and it is also why, when someone wants to do :
find .... | xargs grep "regex"
they intead have to write it:
find ..... | xargs grep "regex" /dev/null
That way, even if the last invocation of grep by xargs appends only 1 filename, grep sees at least 2 filenames (as each time it has: /dev/null, where it won't find anything, and the filename(s) appended by xargs after it) and thus will always display the file names when something maches "regex". Otherwise you may end up with the last results showing matches without a filename in front.