I have a directory containing gzipped datafiles. I want to run each file using the script est_abundance.py. But first i need to unzip them. So i have this bash:
for file in /home/doy.user/scratch1/Secoutput/; do
cd "$file"
gunzip *kren.gz
python analysis1.py -i /Secoutput/*kren -k gkd_output -o /bracken_output/$(basename *kren).txt
wait
done
The problem is, the bash script keeps on unzipping all of the datafiles, it does not continue to the next command after unzipping one file.
Can you help me correct this? I just want every command to be done for every file.
Use, notice that you should use $file variable, and you can get the name of the file after unzipping by stripping the .gz part using ${file%.gz}:
for file in /home/doy.user/scratch1/Secoutput/*; do
gunzip $file
python analysis1.py -i ${file%.gz} -k gkd_output -o /bracken_output/$(basename ${file%.gz}).txt
wait
done
Related
I have a file needs to copy unique directory call test
directory structure as below
/contentroot/path/a/x/test
/contentroot/path/a/y/test
/contentroot/path/a/z/test
--------------------------
as above I have more then 250 combination test directory
I have try below command ( by using asterisk) but it's only copy one test directly only and giving issue (cp: omitting directory )
cp myfile.txt /contentroot/path/a/*/test
any Help
Perhaps a for loop?
for FOLDER in /contentroot/path/a/*/test; do
cp myfile.txt $FOLDER
done
You can feed the output of an echo as an input to xargs. xargs will then run the cp command three times, appending the next directory path piped to it from the echo.
The -n 1 option on the xargs command is so it only appends one of those arguments at a time to the cp each time it runs.
echo /contentroot/path/a/x/test /contentroot/path/a/y/test /contentroot/path/a/z/test | xargs -n 1 cp myfile.txt
Warnings! Firstly this will over-write files (if they exist) and secondlt any bash command should be tested and used at the runners risk! ;)
I am unzipping multiple files in a folder like this:
gunzip -f -k *.gz
Some of the .gz files are broken which cause the command to abort.
What is a nice way of unzipping all files while ignoring the broken ones?
The original answer gives an error, because the shell tries to run the gzipped filenames as a command, because of the grave accent ` wrappers around *.gz. But if you remove them - then it works.
I cannot edit the original answer, because SO requires an edit to change at least 6 characters, so here's a new one.
for file in *.gz
do
gunzip -f -k $file
done
Use this loop as suggested by #Gene
for file in *.gz
do
gunzip -f -k $file
done
I have several .vcf.gz files:
subset_file1.vcf.vcf.gz
subset_file2.vcf.vcf.gz
subset_file3.vcf.vcf.gz
I want to gunzip these file and rename them (remove subset_ and redudant .vcf extension in one go and get these files:
file1.vcf
file2.vcf
file3.vcf
This is the script I have tried:
iFILES=/file/path/*.gz
for i in $iFILES;
do gunzip -k $i > /get/in/this/dir/"${i##*/}"
done
Since you have to three operation at your output path name
1.remove the directory part
2.remove prefix subset_
3.remove redudant extension .vcf
It's hard to accomplish with only one command.
Following is a modification version. Be CAREFUL to try it. I didn't test it thorough in my computer.
for i in /file/path/*.gz;
do
# get the output file name
o=$(echo ${i##*/} | sed 's/.*_\(.*\)\(\.[a-z]\{3\}\)\{2\}.*/\1\2/g')
gunzip -k $i > /get/in/this/dir/$o
done
I have following script (named vid2gif.sh) to convert a video file to gif:
#! /bin/bash
ffmpeg -i $1 /tmp/gif/out%04d.gif
gifsicle --delay=10 --loop /tmp/gif/*.gif > $2
I can convert a file using command:
vid2gif.sh myvid.mp4 myvid.gif
How can I make it to convert all mp4 files in a folder? That is, how can I make following command work:
vid2gif.sh *.mp4
The script should output files as *.gif. Thanks for your help.
#!/bin/sh
for f; do
tempdir=$(mktemp -t -d gifdir.XXXXXX)
ffmpeg -i "$f" "$tempdir/out%04d.gif"
gifsicle --delay=10 --loop "$tempdir"/*.gif >"${f%.*}.gif"
rm -rf "$tempdir"
done
Let's go over how this works:
Iteration
for f; do
is equivalent to for f in "$#"; that is to say, it loops over all command-line arguments. If instead you wanted to loop over all MP4s in the current directory, this would be for f in *.mp4; do, or to loop over all MP4s named in the directory passed as the first command line argument, it would be for f in "$1"/*.mp4; do. To support either usage -- but go with the first one if no directory is passed -- it would be for f in "${1:-.}"/*.mp4; do.
Temporary directory use
Because the original script would reuse /tmp/gif for everything, you'd get files from one input source being used in others. This is best avoided by creating a new temporary directory for each input file, which mktemp will automate.
Creating the .gif name
"${f%.*}" is a parameter expansion which removes everything after the last . in a file; see BashFAQ #100 for documentation on string manipulation in bash in general, including this particular form.
Thus, "${f%.*}.gif" strips the existing extension, and adds a .gif extension.
I have a bash shell script that unzips a zip file, and manipulates the resulting files. Because of the process, I expect all the content I am interested to be within a single folder like so:
file.zip
/file
/contentFolder1
/contentFolder2
stuff1.txt
stuff2.txt
...
I've noticed users on Windows typically don't create a sub folder but instead submit an exploding zip file that looks like:
file.zip
/contentFolder1
/contentFolder2
stuff1.txt
stuff2.txt
...
How can I detect these exploding zips, so that I may handle them accordingly? Is it possible without unzipping the file first?
If you want to check, unzip -l will print the contents of the zip file without extracting them. You'll have to massage the output a bit, though, since it's printing all sorts of additional crud.
Unzip to a directory first, and then remove the extra layer if the zip is not a bomb.
tempdir=`mktemp -d`
unzip -d $tempdir file.zip
if [ $(ls $tempdir | wc -l) = 1 ]; then
mv $tempdir/* .
rmdir $tempdir
else
mv $tempdir file
fi
I wouldn't try to detect it. I'd just force unzip to do what I want. With InfoZip:
$ unzip -j -d unzip-output-dir FileFromUntrustedSource.zip
-j makes it ignore any directory structure within the file, and -d tells it to put files in a particular directory, creating it if necessary.
If there are two files with the same name but in different subdirectories, the above command will make unzip ask if you want to overwrite the first with the second. You can add -o to force it to overwrite without asking, or -f to only overwrite if the second file is newer.