This question already has answers here:
Bash script stops execution of ffmpeg in while loop - why?
(3 answers)
Execute "ffmpeg" command in a loop [duplicate]
(3 answers)
Closed 7 days ago.
I am trying to split audio files by their chapters. I have downloaded this as audio with yt-dlp with its chapters on. I have tried this very simple script to do the job:
#!/bin/sh
ffmpeg -loglevel 0 -i "$1" -f ffmetadata meta # take the metadata and output it to the file meta
cat meta | grep "END" | awk -F"=" '{print $2}' | awk -F"007000000" '{print $1}' > ends #
cat meta | grep "title=" | awk -F"=" '{print $2}' | cut -c4- > titles
from="0"
count=1
while IFS= read -r to; do
title=$(head -$count titles | tail -1)
ffmpeg -loglevel 0 -i "$1" -ss $from -to $to -c copy "$title".webm
echo $from $to
count=$(( $count+1 ))
from=$to
done < ends
You see that I echo out $from and $to because I noticed they are just wrong. Why is this? When I comment out the ffmpeg command in the while loop, the variables $from and $to turn out to be correct, but when it is uncommented they just become some stupid numbers.
Commented output:
0 465
465 770
770 890
890 1208
1208 1554
1554 1793
1793 2249
2249 2681
2681 2952
2952 3493
3493 3797
3797 3998
3998 4246
4246 4585
4585 5235
5235 5375
5375 5796
5796 6368
6368 6696
6696 6961
Uncommented output:
0 465
465 70
70 890
890 08
08 1554
1554 3
3 2249
2249
2952
2952 3493
3493
3998
3998 4246
4246 5235
5235 796
796 6368
6368
I tried lots of other stuff thinking that they might be the problem but they didn't change anything. One I remember is I tried havin $from and $to in the form of %H:%M:%S which, again, gave the same result.
Thanks in advance.
Here is an untested refactoring; hopefully it can at least help steer you in another direction.
Avoid temporary files.
Avoid reading the second input file repeatedly inside the loop.
Refactor the complex Awk scripts into a single script.
To be on the safe side, add a redirection from /dev/null to prevent ffmpeg from eating the input data.
#!/bin/sh
from=0
ffmpeg -loglevel 0 -i "$1" -f ffmetadata - |
awk -F '=' '/END/ { s=$2; sub(/007000000.*/, "", s); end[++i] = s }
/title=/ { t=$2; sub(/^([^-]-){3}/, "", t); title[++j] = t }
END { for(n=1; n<=i; n++) print end[n]; print title[n] }' |
while IFS="" read -r end; do
IFS="" read -r title
ffmpeg -loglevel 0 -i "$1" -ss "$from" -to "$end" -c copy "$title".webm </dev/null
from="$end"
done
The Awk script reads all the data into memory, and then prints one "end" marker followed by the corresponding title on the next line; I can't be sure what your ffmpeg -f ffmetadata command outputs, so I just blindly refactored what your scripts seemed to be doing. If the output is somewhat structured you can probably read one record at a time.
Related
I've been trying to get a one liner working updated to add the user name for the processes consuming swap on the system but having a hard time getting it to work.
The thread here gave me a pretty good script to work with and I've been partially successful but still seeing some issues.
for file in /proc/*/status ; do awk -F: '/Tgid|VmSwap|Name/{printf $2 }END{ print ""}' $file; done| grep kB | sort -k 3 -n -r | head -n 10 | awk 'BEGIN { ORS=" " };{print $0 system("ps -o user= -p " $2)}
The above produces the output:
username 0
processname0 1734 334248 kB0 username1
processname1 2314 182360 kB0 root
processname2 2069626 78292 kB0 username2
processname3 2069621 11500 kB0 root
processname4 2540 8720 kB0 root
processname5 1547 4320 kB0 root
processname6 1556 3380 kB0 root
processname7 2069656 3208 kB0 root
processname8 1398 1312 kB0 root
The username coming in a different line is an issue and the other one is the addition of the 0 after 'kB'.
I've tried a few different things but nothing seems to be working.
Also, the script is being run through another tool and has to be a one liner which is limiting the options a bit. Any help would be great.
After a lot more trial and error, I was able to get this working.
The following is the code snippet that ended up working:
for file in /proc/*/status ; do awk -F: '/Tgid|VmSwap|Name/{printf $2}END{print ""}' $file 2>/dev/null ; done | grep kB | sort -k 3 -n -r | head -n 10 | awk 'BEGIN{printf "%-20s %-10s %10s %-5s %10s\n","Process ID","PID","Size","Units","User"};{cmd="ps -o user= -p " $2; cmd | getline result; printf "%-20s %-10s %10s %-5s %10s\n",$1,$2,$3,$4,result}'
I made a short bash program to download podcasts and retrieve only last 20 seconds.
Strange thing is it fails downloading every other iteration. There seems to be a problem with the function trim_nsec, because when I get rid of it in the loop, all the rest correctly works.
Edit : addition of double quotes, which doesn't solve the problem
<!-- language: lang-bash -->
#!/bin/bash
# Get podcast list
wget -O feed http://www.rtl.fr/podcast/on-n-est-pas-forcement-d-accord.xml
function trim_nsec () {
# arguments : 1 : mp3file - 2 : duration - 3 : outputfile
duration=$(ffprobe -i "${1}" -show_entries format=duration -v quiet -of csv="p=0")
nth_second=$(echo "${duration} - ${2}"|bc)
ffmpeg -i "${1}" -ss "${nth_second}" "${3}"
}
cpt=1
# let's work only on the 4th first files
grep -Po 'http[^<]*.mp3' feed|grep admedia| head -n 4 > list
cat list | while read i
do
year=$(echo "$i" | cut -d"/" -f6)
day=$(echo "$i" | cut -d"/" -f7)
fullname=$(echo "$i" | awk -F"/" '{print $NF}')
fullnameend=$(echo "$fullname" |sed -e 's/\.mp3$/_end\.mp3/')
new_name=$(echo "$year"_"$day"_"$fullnameend")
# let's download
wget -O "$fullname" "$i"
# let's trim last 20 sec
trim_nsec "$fullname" 20 "$new_name"
echo "$cpt file processed"
#delete orig. file :
rm "$fullname"
((cpt++))
done
Any idea ?
The problem is most likely due to the fact that on errors, ffmpeg will try to get an input from user which will consume the input provided by cat list. See a similar question here or here. To prevent trim_nsec from consuming the input from cat list, you could do:
cat list | while read i
do
year=$(echo "$i" | cut -d"/" -f6)
day=$(echo "$i" | cut -d"/" -f7)
fullname=$(echo "$i" | awk -F"/" '{print $NF}')
fullnameend=$(echo "$fullname" |sed -e 's/\.mp3$/_end\.mp3/')
new_name=$(echo "$year"_"$day"_"$fullnameend")
# let's download
wget -c -O "$fullname" "$i"
# let's trim last 20 sec
trim_nsec "$fullname" 20 "$new_name" <&3
echo "$cpt file processed"
#delete orig. file :
#rm "$fullname"
((cpt++))
done 3<&1
Good day,
I need to add a Column header "TIME" that will display the current time for each time the output is executed on a new line with the following code:
top -b -n 1 -p 984 -o +PID -o +VIRT | sed -n '7,12p' | awk '{printf "%1s %-4s\n",$1,$5}'
Output I'm looking for:
TIME PID VIRT
12:00:00 984 1024
12:16:01 984 995
12:44:29 984 1008
(The values is only for display, not correct)
also it should be in a endless loop with interval of 10s until user stops it.
everything is executed from PIDandVIRT.sh
(Linux script)
Thank you for the help in advance
I would recommend to use the ps command instead of top:
echo "TIME PID VSIZE"
while true ; do
echo "$(date +%H:%I:%S) $(ps -p 984 -o pid,vsize --no-headers)"
sleep 1
done
Set an awk variable to the result of the date command:
awk -v time=$(date '+%H:%M:%S') '{printf "%s %1s %-4s\n", time, $1, $5}'
To get it in a loop, use while
while :; do
top -b -n 1 -p 984 -o +PID -o +VIRT | sed -n '7,12p' | awk -v time=$(date '+%H:%M:%S') '{printf "%s %1s %-4s\n", time, $1, $5}'
sleep 10
done
I made a simple script that divides a flv file into multiple parts, converts them all to .mp4 individually and then merge all of them to form a final mp4 file. I did this to save time and convert large files in parallel.
However, I am stuck because the command that normally runs on command line for ffmpeg, doesn't run via script.
I am kind of stuck here and will like to have some assistance.
#!/bin/bash
#sleep 5
filenametmp=$1;
filename=`echo "$filenametmp" | awk '{split($0,a,"."); print a[1]}'`
echo $filename
output="$filename-output"
filenamewithoutpath=`echo "$output" | awk '{split($0,a,"/"); print a[4]}'`
echo $output $filenamewithoutpath
/usr/bin/ffmpeg -i $filenametmp -c copy -map 0 -segment_time $2 -f segment $output%01d.flv
#sleep 10
#echo "/bin/ls -lrt /root/storage/ | /bin/grep $filenamewithoutpath | /usr/bin/wc -l"
filecounttmp=`/bin/ls -lrt /opt/storage/ | /bin/grep $filenamewithoutpath | /usr/bin/wc -l`
filecount=`expr $filecounttmp - 1`
echo $filecount
for i in `seq 0 $filecount`
do
suffix=`expr 0000 + $i`
filenametoconvert="$output$suffix.flv"
convertedfilename="$output$suffix.mp4"
echo $filenametoconvert
/usr/bin/ffmpeg -i $filenametoconvert -c:v libx264 -crf 23 -preset medium -vsync 1 -r 25 -c:a aac -strict -2 -b:a 64k -ar 44100 -ac 1 $convertedfilename > /dev/null 2>&1 &
done
sleep 5
concatstring=""
for j in `seq 0 $filecount`
do
suffix=`expr 0000 + $j`
convertedfilenamemp4="$output$suffix.mp4"
#concatstring=`concat:$concatstring|$convertedfilenamemp4`
echo "file" $convertedfilenamemp4 >> $filename.txt
#ffmpeg -i concat:"$concatstring" -codec copy $filename.mp4
#ffmpeg -f concat -i $filename.txt -c copy $filename.mp4
done
echo $concatstring
ffmpeg -f concat -i $filename.txt -c copy $filename.mp4
rm $output*
rm $filename.txt
I run any flv file like this :
./ff.sh /opt/storage/tttttssssssssss_573f5b1cd473202daf2bf694.flv 20
I get this error message :
moov atom not found
I am on Ubuntu 14.04 LTS version, standard installation of ffmpeg.
I am trying to use bash script to add resolution through xrandr and i keep getting error, here is my script:
#!/bin/bash
out=`cvt 1500 800`
out=`echo $out | sed 's/\(.*\)MHz\(.*\)/\2/g'`
input=`echo $out | sed 's/Modeline//g'`
#echo $input
xrandr --newmode $input
input2=`echo $out | cut -d\" -f2`
#echo $input2
xrandr --addmode VNC-0 $input2
running with bash -x
input=' "1504x800_60.00" 98.00 1504 1584 1736 1968 800 803 813 831 -hsync +vsync'
+ xrandr --newmode '"1504x800_60.00"' 98.00 1504 1584 1736 1968 800 803 813 831 -hsync +vsync
if you look at the last line, it adds for some reason single quote ' at the start (before ") and after ", why ??
Singe quotes are added by bash -x when printing debug output.
It won't affect your actual variables value:
out=`cvt 1500 800`
echo $out
# 1504x800 59.92 Hz (CVT) hsync: 49.80 kHz; pclk: 98.00 MHz Modeline "1504x800_60.00" 98.00 1504 1584 1736 1968 800 803 813 831 -hsync +vsync
echo $input
"1504x800_60.00" 98.00 1504 1584 1736 1968 800 803 813 831 -hsync +vsync 98.00 1504 1584 1736 1968 800 803 813 831 -hsync +vsync
What's actually happens, that quotes inside a variable's value aren't parsed when the variable is substituted.
The best way to do this sort of thing is using an array instead of a simple text variable:
xrandr_opts=() # declaring array
input=`echo $out | sed 's/Modeline//g'`
read -a xrandr_opts <<< $input # splitting $input to array
xrandr --newmode "${xrandr_opts[#]}"
As for your specific case, following change will do the trick:
#!/bin/bash
out=`cvt 1500 800`
out=`echo $out | sed 's/\(.*\)MHz\(.*\)/\2/g'`
input=`echo $out | sed 's/Modeline//g'`
#echo $input
#xrandr --verbose --newmode $input
xrandr_opts=() # declaring array
input=`echo $input | sed 's/\"//g'`
read -a xrandr_opts <<< $input # splitting $input to array
opts_size=`echo ${#xrandr_opts[#]}`
xrandr --newmode `printf \'\"%s\"\' ${xrandr_opts[0]}`
${xrandr_opts[#]:1:$opts_size}
input2=`echo $out | cut -d\" -f2`
#echo $input2
xrandr --verbose --addmode VNC-0 $input2
Looks like xrandr --newmode won't accept double quotes. I can't say exactly what's the reason, but at least the script works :)