I have the following directory structure:
dir1/file.ogg
dir2/file.ogg
dir3/file.ogg
file2.ogg
I would like to convert all .ogg files to .wav with GNU Parallel. Here's where I got thus far:
find . -name '*.*' -type f -print0 | parallel -0 ffmpeg -i {} outputdir/{/.}.wav
The problem here is that although obviously directories have different names, the files inside have the same name. The aforementioned command will continuously overwrite the content of the directory. What I'd like instead is:
outputdir/dir1_file.ogg
outputdir/dir2_file.ogg
outputdir/dir3_file.ogg
outputdir/dir3_file2.ogg
Essentially, I'd like to extract the subdirectory name and concatenate it with file basename and put my own extension.
Any ideas?
Using perl transformation, this command should achieve the required effect :
find . -name '*.ogg' -type f -print0 | parallel -0 echo ffmpeg -i {} outputdir/{= '$_ =~ s[^\./][]; $_ =~ s[/][_]g; $_ =~ s[ogg$][wav]g;' =}
Remove echo to run ffmpeg command
Related
I've created a script to quickly analyze some logs and automatically provide advices to solve problems based on errors found.
All works as expected.
However, it's appears that folders structure containing these logs can change (depends on system configuration) and my script not work any more.
I would like to find a way to find the directory containing a specifics files like logs or appinfo.txt file.
Once obtains I could use it as variable and finally solve my problem.
Here is an example:
AppLogDirectory ='Your_Special_Command_You_Will_HelpMe_To_Find'
grep -i "Error" $AppLogDirectory/esl*.log
Log format is: ESL.randomValue.log
Files analyzed : appinfo.txt,
system.txt etc ..
A suggested in comment section, I edit my orginal post with more detail to clarify the context, below an example:
Log files (esl.xxx.tt.ss.log ) can be in random directory, like:
/var/log/ApplicationName/logs/
/opt/ApplicationName/logs/
/var/data/ApplicationName/Extended/logs/
Because of random directory, I need to find a solution to print the directory names of the files that match esl*.log patter (without esl filename)
Use find and pass the output to xargs with grep, like so, which runs grep on multiple files and prints the output together with the file name where the pattern was found:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' \) -print0 | xargs -0 grep -i 'Error'
Or simply use -exec ... \+, which gives the same effect, without the need for xargs:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' -exec grep -i 'Error' \+
To find the directories which contain the files that contain the desired pattern, use grep -l to print file names only (not the lines that match), and pipe the results to xargs dirname to print the directory names. If you need the unique dir names, pipe it further to sort -u:
find /path/to/files /another/path/to/other/files \( -name 'appinfo.txt' -o -name 'system.txt' -o -name 'esl*.log' -exec grep -il 'Error' \+ | xargs dirname | sort -u
SEE ALSO:
GNU find manual
To search for files based on their contents
xargs
Solution found thanks to you thank you again!
#Ask for extracted tar.gz folder
read -p "Where did you extract the tar.gz file? r1
#directory path where esl files is located
logpath=`find $r1 -name "esl*.log" | xargs dirname | sort -u`
#Search value (here "Error") into all esl*.log
grep 'Error' $logpath/esl*.log | awk '{print $8}'
I want to make a specific copy.
I explain
So here my main folder :
Sub-Directory-name-01\filename-01.jpg
Sub-Directory-name-01\filename-02.jpg
Sub-Directory-name-01\filename-03.jpg
Sub-Directory-name-01\special-filename-01.jpg
Sub-Directory-name-02\filename2-01.jpg
Sub-Directory-name-02\filename2-02.jpg
Sub-Directory-name-02\filename2-03.jpg
Sub-Directory-name-02\special-filename2-01.jpg
Sub-Directory-name-02\filename2-01.jpg
Sub-Directory-name-02\filename2-02.jpg
Sub-Directory-name-02\filename2-03.jpg
Sub-Directory-name-02\special-filename2-01.jpg
I want to copy all file from all dir and :
- keep original file
- copy 2 times the original file
- add a prefix to the new name
- prefix-01 for first copy
- prefix-02 for second copy
- keep the new files in the same dir as original file
I allready succes with a command to copy 1 time with 1 prefix.
It works in the sub-directory
for file in *.jpg; do cp "$file" "prefix-$file"; done
I try to do for all sub-dirs but i got an error
find . -type f \( -iname "*.jpg" ! -iname "special-*.jpg" \) | xargs cp -v "$file" "prefix-$file"
( yes i exclude a special name )
But i got error :
cp: target `./Sub-Directory-name-01/filename-01.jpg' is not a directory
i dont know how to solve my problem and how to add the 2nd copy in the cmd.
Thanks
Edit : I havent found any similar question so any answser to solve this problem.
Note that above $file is set only by the for file in ... ; do ... ;done loop, i.e. in your xargs cmdline you were just using the last leftover value from the loop.
Some things to consider:
need to process each file separately => use xargs -l1 (process each 1 line).
need to separate DIR/FILENAME as the needed command is something like 'cp $DIR/$FILENAME $DIR/prefix-01-$FILENAME' (and prefix-02 also), use find ... -printf "%h %f\n" for this
for each line, need to do couple things (prefix-01,02) => use a scriptlet via sh -c '<scriptlet>'
better skip prefix-0?-*.jpg files from find, to be able to re-run it without "accumulating" copies
A possible implementation would be:
find . -type f \( -iname "*.jpg" ! -iname "special-*.jpg" ! -name "prefix-0?-*.jpg" \) -printf "%h %f\n" | \
xargs -l1 sh -c 'cp -v "$1/$2" "$1/prefix-01-$2"; cp -v "$1/$2" "$1/prefix-02-$2"' --
As xargs runs sh -c '<scriptlet>' -- DIR FILE for each line, the scriptlet will properly evaluate $1 and $2 respectively.
--jjo
PS: directory separator in Unix-like systems is / :)
[Update: fixed to use %f instead of %P, as per comments below]
I want to write a shell script. I list my jpg files inside nested subdirectories with the following command line:
find . -type f -name "*.jpg"
How can I save the output of this command inside a variable and write a for loop for that? (I want to do some processing steps for each jpg file)
You don't want to store output containing multiple files into a variable/array and then post-process it later. You can just do those actions on the files on-the-run.
Assuming you have bash shell available, you could write a small script as
#!/usr/bin/env bash
# ^^^^ bash shell needed over any POSIX shell because
# of the need to use process-substitution <()
while IFS= read -r -d '' image; do
printf '%s\n' "$image"
# Your other actions can be done here
done < <(find . -type f -name "*.jpg" -print0)
The -print0 option writes filenames with a null byte terminator, which is then subsequently read using the read command. This will ensure the file names containing special characters are handled without choking on them.
Better than storing in a variable, use this :
find . -type f -name "*.jpg" -exec command {} \;
Even, if you want, command can be a full bloated shell script.
A demo is better than an explanation, no ? Copy paste the whole lines in a terminal :
cat<<'EOF' >/tmp/test
#!/bin/bash
echo "I play with $1 and I can replay with $1, even 3 times: $1"
EOF
chmod +x /tmp/test
find . -type f -name "*.jpg" -exec /tmp/test {} \;
Edit: new demo (from new questions from comments)
find . -type f -name "*.jpg" | head -n 10 | xargs -n1 command
(this another solution doesn't take care of filenames with newlines or spaces)
This one take care :
#!/bin/bash
shopt -s globstar
count=0
for file in **/*.jpg; do
if ((++count < 10)); then
echo "process file $file number $count"
else
break
fi
done
I am faced with a challenge that requires multiple aspects of bash. I work in Linux (precisely Debian Stretch). Here is the situation (for all points/problem I write along the solution I considered for now, but I'm open to other ideas) :
I have videos of various types (and various upper-lower case), such as .mp4, .mov, .MOV, .MP4, .avi,... located in a directory (and spread across an almost un-structured tree of directories). To find all I tried to use the find command
For each video, I need to extract some metadata (i.e. the name of the file, duration of video, size of file and date of creation/last modification). The package mediainfo yields (among a lot of other things) the required fields.
The output of mediainfo is a long list of fields with format : <Tag>\t : <value>. I need to extract values for fields Complete name, Duration, File size and Encoded date.
So with all this information, I must filter the required fields value and put them in a CSV file. I considered using sed.
My goal is to achieve all these tasks either in a script or a small amount of separate commands.
The idea code (this code is hideously wrong, but you can get an idea) :
find . -type f -name "*.[mp4|MP4|mov|MOV|avi|AVI]" -exec mediainfo {} | sed '/Complete name|Duration|File size|Encoded date/p' > myfile.csv \;
Would you have any idea how to perform this task ? I feel terribly lost in combining find, exec and sed and outputting to a csv...
Thanks in advance for your help !
So I finally managed to write a script doing that. Probably not the best way to do, but here it is :
resFile="myresult.csv"
dstDir="./destination/"
srcDir="./source/"
#first copy all files at same level in dstDir (with preserve and update)
#this is somehow necessary, relative name for MOV files and mediainfo
#do not seem to work together.
find $srcDir -type f \( -name "*.mp4" -o -name "*.mov" -o -name "*.MOV" -o -name "*.avi" \) -exec cp -up {} $dstDir \;
#then for each file, output mediainfo of file and keep only interesting tags. add ### between each file.
find $dstDir -type f \( -name "*.mp4" -o -name "*.mov" -o -name "*.MOV" -o -name "*.avi"\
-exec sh -c " mediainfo --Output=XML {} | sed '1,15!d;/Duration\|Complete\|File_size\|Encoded_date/!d' >> $resFile && echo '########' >> $resFile" \;
#removes tags : <Duration>42s 15ms</Duration> -> 42s 15ms
sed -i 's/^<.*>\(.*\)<.*>/\1/I' $resFile
#Extract exact filename (and not relative)
sed -i 's/^\.\/.*\/\(.*\)\.[mp4|MOV|mov|avi|MP4]/\1/' $resFile
#Puts fields for a file on a unique line separated with commas
sed -i 'N;s/\n/,/;N;s/\n/,/;N;s/\n/,/;N;s/\n/,/' $resFile
#remove all trailing ###
sed -i 's/,#*$//' $resFile
I would still be interested if anyone has idea to improve the code.
I "minimized" a little bit, my actual code is a bit more modular and performs a few checks
Try this. Due to less time,I was not able to complete. You just have to send output to CSV.
for c in $(locate --basename .mp4 .mkv .wmv .flv .webm .mov .avi)
do
Complete_name=$(mediainfo --Output=XML $c | xml_grep 'Complete_name' --text_only| awk 'BEGIN{FS="/"}{print $NF}')
echo $Complete_name
Duration=$(mediainfo --Output=XML $c | xml_grep 'Duration' --text_only --nb_result 1)
echo $Duration
File_size=$(mediainfo --Output=XML $c | xml_grep 'File_size' --text_only)
echo $File_size
Encoded_date=$(mediainfo --Output=XML $c | xml_grep 'Encoded_date' --text_only -nb_result 1 | awk '{print $2}')
echo $Encoded_date
done
I have multiple folders with names :
1_1,1_2,...,2_1,...,
each of these folders contains the same file with the name file.sh. The file has the following form :
job_name=NAME
Partition = Long
I want to use a search&replace command in the terminal (Linux) for all my folders, like for example the following
find . -type f -name "file.sh" -print |xargs sed -i 's/job_name/REPLACED_TEXT/g'
and in the position of the REPLACED_TEXT I want the name of the folder. For example, inside folder 1_1, there will be the file.sh file with the modified form:
job_name=1_1
Partition = Long
I haven't found a solution for that yet.
You didn't specify how many subdirectories you might have to traverse, e.g.
./1_1/file.sh
./1_2/file.sh
./a/b/c/1_1/file.sh
So for this I'll just assume one subdirectory like so:
./1_1/file.sh
./1_2/file.sh
Something like the below should be able to get you started, not tested, just writing it off the top of my head. It's bash scripted but you can turn it into one big long command. Make sure to back up your directory first in case the script has unpredictable results.
for i in `find . -type f -print "file.sh"`;
do
subdir=`echo $i | awk -F\/ '{print $2}'`
sed -e s/job_name=NAME/jobname=$subdir/ $i > $i.bak
mv $i.bak $i
done
You can try this line to print all the sed commands you want to execute:
find . -type f -name 'file.sh' | \
sed 's=\(.*\)/\([^/]*\)=sed -i "s/NAME/\1/" \"&\"='
For each file we found, it extracts the name of its directory and creates a sed command able to replace NAME with it.
Output should be something like:
sed -i "s/NAME/1_1/" "1_1/file.sh"
sed -i "s/NAME/1_2/" "1_2/file.sh"
Then, if it looks good to you, you can repeat with the e command for sed, which will make the outer sed execute its result (i.e. inner sed command), like this:
find . -type f -name 'file.sh' | \
sed 's=\(.*\)/\([^/]*\)=sed -i "s/NAME/\1/" \"&\"=e'
# 'e' command added here -------------------------^