I'm working on Discord voice and I made voice receiver, but I can't seem to be able to write the audio buffer to a file to be able to play it. Here's my code:
stopRecording() {
this.recording = false;
let output = Buffer.concat(this.recordedBuffers);
fs.writeFileSync('./out.opus', output.toString());
this.recordedBuffers = [];
console.log('done');
};
The file gets created but it is unplayable. Any ideas?
It may be worth confirming that recordedBuffers contains Ogg Opus file data. The first 32 bytes should show file header data indicating this is an Ogg, Webm, or Mp4 container. Once the mime-type is confirmed, the file extension can be changed from .opus if needed:
/* .opus = 'OggS ... Opus'
* .webm = '... webm ...'
* .mp4 = '... mp4 ...'
* .wav = '... WAVE ...'
*/
new TextDecoder().decode(concatenatedBuffers.slice(0, 32))
FFmpeg could also inspect the downloaded file:
$ ffprobe audio.opus
Input #0, ogg, from 'audio.opus':
Duration: 00:06:00.11, start: 0.000000, bitrate: 99 kb/s
Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
Related
I have an MP4 file, which I would like to convert into an MPEG4 file. TO do this, I have found the PythonVideoConvert package. On the PyPI page, the following code is given:
from converter import Converter
conv = Converter()
info = conv.probe('test/test1.avi')
PATH = 'C:/Users/.../'
convert = conv.convert(PATH +'Demo.mp4', PATH + 'Demo.mpeg4', {
'format': 'mpeg4',
'audio': {
'codec': 'aac',
'samplerate': 11025,
'channels': 2
},
'video': {
'codec': 'hevc',
'width': 720,
'height': 400,
'fps': 25
}})
When I run this code, a convert object is created. However, there is no .mpeg4 video in the PATH directory.
Therefore, I have two questions:
Is the code above correct for converting a .mp4 file into a .mpeg4 file
What do I need to run to save the converted video as a .mpeg4 file?
Based on Selcuk's comment, I ran the following code:
for timecode in convert:
pass
This gives the error:
Traceback (most recent call last):
File "<ipython-input-60-14c9225c3ac2>", line 1, in <module>
for timecode in convert:
File "C:\Users\20200016\Anaconda3\lib\site-packages\converter\__init__.py", line 229, in convert
optlist = self.parse_options(options, twopass)
File "C:\Users\20200016\Anaconda3\lib\site-packages\converter\__init__.py", line 60, in parse_options
raise ConverterError(f'Requested unknown format: {str(f)}')
ConverterError: Requested unknown format: mpeg4
So, my suggested format seems incorrect. What can I do to convert a video into .mpeg4?
I don't think PythonVideoConverter is meant to be used in Windows.
I was getting an exception AttributeError: module 'signal' has no attribute 'SIGVTALRM', because SIGVTALRM is not a valid signal in Windows.
The default path of FFmpeg an FFprobe command line tools, also doesn't make sense for Windows.
We may still use the package in Windows, but it's recommended to set ffmpeg_path and ffprobe_path.
Example:
conv = Converter(ffmpeg_path=r'c:\FFmpeg\bin\ffmpeg.exe', ffprobe_path=r'c:\FFmpeg\bin\ffprobe.exe')
We also have to disable the timeout feature, by setting timeout=None argument.
mpeg4 is not a valid FFmpeg format, but we can still use it as a file extension.
(format is FFmpeg terminology usually applies container format).
When non-standart file extension is used, we have to set the format entry.
Setting 'format': 'mp4' creates MP4 file container (may be created with the non-standart .mpeg4 file extension).
Complete code sample:
from converter import Converter
conv = Converter(ffmpeg_path=r'c:\FFmpeg\bin\ffmpeg.exe', ffprobe_path=r'c:\FFmpeg\bin\ffprobe.exe')
#info = conv.probe('test/test1.avi')
PATH = 'C:/Users/Rotem/'
convert = conv.convert(PATH + 'Demo.mp4', PATH + 'Demo.mpeg4', {
'format': 'mp4', #'format': 'mpeg4',
'audio': {
'codec': 'aac',
'samplerate': 11025,
'channels': 2
},
'video': {
'codec': 'hevc',
'width': 720,
'height': 400,
'fps': 25
}},
timeout=None)
# https://pypi.org/project/PythonVideoConverter/
for timecode in convert:
print(f'\rConverting ({timecode:.2f}) ...')
We may see the media information of Demo.mpeg4 using MediaInfo tool:
General
Complete name : C:\Users\Rotem\Demo.mpeg4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 207 KiB
Duration : 10 s 148 ms
Overall bit rate mode : Variable
Overall bit rate : 167 kb/s
Writing application : Lavf58.45.100
FileExtension_Invalid : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main#L3#Main
Codec ID : hev1
Codec ID/Info : High Efficiency Video Coding
Duration : 10 s 0 ms
Bit rate : 82.5 kb/s
Width : 720 pixels
Height : 400 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.011
Stream size : 101 KiB (49%)
Writing library : x265 3.4+28-419182243:[Windows][GCC 9.3.0][64 bit] 8bit+10bit+12bit
Encoding settings : ...
Color range : Limited
Codec configuration box : hvcC
Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 10 s 148 ms
Duration_LastFrame : -70 ms
Bit rate mode : Variable
Bit rate : 79.1 kb/s
Maximum bit rate : 128 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 11.025 kHz
Frame rate : 10.767 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 98.0 KiB (47%)
Title : IsoMedia File Produced by Google, 5-11-2011
Language : English
Default : Yes
Alternate group : 1
In MediaInfo output, the MP4 file container applies "MPEG-4" format...
Note:
The HEVC video format applies H.265 video codec - in most cases the codec is considered to be more relevant then container.
'Requested unknown format: mpeg4'
*.mpeg4 is not valid container. mpeg4 is codec, *.something (avi, mp4, mov, mkv, ...) are containers.
basicly: codec.CONTAINER or your_mpeg4_video.mkv etc.
video codec (like mpeg4) handle only video, but you need more than only visual, you need audio, many audio tracks (eng, de, nl, 2.0, 5.1, 7.1 ...), subtitles, etc and these stuff are inside container.
install ffmpeg: https://ffmpeg.org/
try this basic script:
import subprocess
input_file = 'Demo.mp4'
output_file = 'Demo.mkv' # or .mp4, .mov, ...
ffmpeg_cli = "ffmpeg -i '{}' -vcodec libx265 '{}'".format(input_file, output_file)
subprocess.call(ffmpeg_cli, shell=True)
I don't know what are you doing (what you want, what are your expectations) but if you looking for way how to degrese size of video,
look here: https://github.com/MarcelSuleiman/convert_h264_to_h265
simple.
I need to split a large video file into multiple pieces quickly and without files with errors. The basic idea is, I have a 2GB video file which I want to change to multiple formats. I have read some encoders can not make use of more than 1 core when encoding to a different format. So I had the idea to split the large file (which is very quick) into 4/8 pieces depending on how many cores I have available on individual servers. re-encode each piece to a new format and use these to display video in sequence.
so
testfile.mp4
becomes
testfile_piece_1.mp4
testfile_piece_2.mp4
testfile_piece_3.mp4
testfile_piece_4.mp4
these can then be individually be converted.
The solution should not be format specific. However I have found issues with mp4 files.
I have tried the command below, which works really well and fast but creates files with errors.
ffmpeg -i testfile.mp4 -ss 00:00:00 -t 00:20:00 -c copy testfile_piece_1.mp4
When I play the testfile_piece_1.mp4 on VLC it works fine. An issue arises when converting the split file to a different height and width mp4 file. I would get an error similar to "moov atom not found"
I tried adding -movflags faststart with no luck
I then came across this library https://code.google.com/archive/p/moovrelocator/ which fixed the moov issue but I would then get an error with regards to aac "Error while opening encoder for output stream #0.0 - maybe incorrect parameters such as bit_rate, rate, width or height"
The other way of splitting the files is useless but involved re-encoding the file. not too bad for smaller file size but the 2GB file would probably take days to complete.
Is there a way to split the largr file quickly without producing files with errors? I have been working on it for days with no luck.
Console output for comment - FFmpeg splitting large files
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/usr/share/nginx/html/uploads/testfile01.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf56.36.100
Duration: 00:05:02.08, start: 302.120000, bitrate: 3254 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709) , 1920x1080 [SAR 1:1 DAR 16:9], 3252 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (de fault)
Metadata:
handler_name : VideoHandler
[libx264 # 0x165ffc0] width not divisible by 2 (853x480)
Output #0, mp4, to '/usr/share/nginx/html/uploads/testfile01_480.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf56.36.100
Stream #0:0(und): Video: h264, none, q=2-31, 128 kb/s, SAR 2560:2559 DAR 0:0 , 25 fps (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc56.41.100 libx264
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
console output 2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/usr/share/nginx/html/uploads/testfile.mp4':
Metadata:
major_brand : dash
minor_version : 0
compatible_brands: iso6avc1mp41
creation_time : 2016-01-24 04:26:37
Duration: 01:15:58.08, start: 0.000000, bitrate: 3163 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 3161 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)
Metadata:
creation_time : 2016-01-24 04:26:37
handler_name : VideoHandler
[segment # 0x1197060] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, segment, to '/usr/share/nginx/html/uploads/testfile%02d.mp4':
Metadata:
major_brand : dash
minor_version : 0
compatible_brands: iso6avc1mp41
encoder : Lavf56.36.100
Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 3161 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
Metadata:
creation_time : 2016-01-24 04:26:37
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 # 0x11512a0] Codec for stream 0 does not use global headers but container format requires global headers
Use the segment muxer to break the input into segments:
ffmpeg -i testfile.mp4 -c copy -f segment -segment_time 1200 testfile_piece_%02d.mp4
This will split the source at keyframes, so segments may not be exactly 1200 seconds long. And the timestamps aren't reset, so some players will fail to play the 2nd and latter segments. If playability is needed, insert -reset_timestamps 1.
After the parallel encoding, you can stitch the generated segments by first creating a text file seg.txt like this
file 'encoded_testfile_piece_00.mp4'
file 'encoded_testfile_piece_01.mp4'
file 'encoded_testfile_piece_02.mp4'
file 'encoded_testfile_piece_03.mp4'
And then running
ffmpeg -f concat -i seg.txt -c copy -fflags +genpts encoded_full.mp4
I am trying to stream video live using ffmpeg. videos stream is coming from a camera over the network and there is bunch error that i got in initial part of doing this but here i am stacked with a error that is coming from ffmpeg where it says av_interleaved_write_frame() broken pipe,then at the end i get a message saying conversion failed ,when i want to capture the stream in any format it works like a charm but when it comes to streaming it gives the error.Why this is happening?
the command i am using to get the streaming is here:
ffmpeg - i rtsp://10.63.191.250/aerostat http://10.20.1.142:8080/feed.ffm
or
ffmpeg - i rtsp://10.63.191.250/aerostat -acodec copy -vcodec copy http://10.20.1.142:8080/feed.ffm
//with both the command i am getting the same error
The ffmpeg out put is Here:
[h264 # 0x3965b40] decode_slice_header error
[h264 # 0x3965b40] no frame!
[h264 # 0x3965b40] non-existing PPS 0 referenced
Last message repeated 1 times
[h264 # 0x3965b40] decode_slice_header error
[h264 # 0x3965b40] no frame!
Input #0, rtsp, from 'rtsp://10.63.191.250/aerostat':
Metadata:
title : \Aerostat
Duration: N/A, start: 63184.792467, bitrate: N/A
Program 1
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 77 kb/s
Stream #0:1: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 656x368 [SAR 1:1 DAR 41:23], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
[libvpx # 0x397a360] v1.6.0
Output #0, ffm, to 'http://10.20.1.142:8080/feed.ffm':
Metadata:
title : \Aerostat
creation_time : now
encoder : Lavf57.56.100
Stream #0:0: Audio: vorbis (libvorbis), 22050 Hz, mono, fltp, 64 kb/s
Metadata:
encoder : Lavc57.64.101 libvorbis
Stream #0:1: Video: vp8 (libvpx), yuv420p, 720x576 [SAR 164:115 DAR 41:23], q=10-42, 400 kb/s, 29.97 fps, 1000k tbn, 25 tbc
Metadata:
encoder : Lavc57.64.101 libvpx
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 800000 vbv_delay: -1
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> vorbis (libvorbis))
Stream #0:1 -> #0:1 (h264 (native) -> vp8 (libvpx))
Press [q] to stop, [?] for help
av_interleaved_write_frame(): Broken pipe
Last message repeated 6 times
Error writing trailer of http://10.20.1.142:8080/feed.ffm: Broken pipeframe= 6 fps=0.0 q=0.0 Lsize= 52kB time=00:00:00.43 bitrate= 972.0kbits/s dup=5 drop=0 speed=0.778x
video:43kB audio:3kB subtitle:0kB other streams:0kB global headers:4kB muxing overhead: 13.661202%
Conversion failed!
Here is My FFserver.conf file:
HTTPPort 8080 # Port to bind the server to
HTTPBindAddress 0.0.0.0
MaxHTTPConnections 2000
MaxClients 1000
MaxBandwidth 10000 # Maximum bandwidth per client
# set this high enough to exceed stream bitrate
CustomLog -
<Feed feed.ffm> # This is the input feed where FFmpeg will send
File ./feed.ffm # video stream.
FileMaxSize 1G # Maximum file size for buffering video
ACL allow 10.20.0.0 10.20.255.255 # Allowed IPs
ACL allow 127.0.0.1
ACL allow localhost
</Feed>
<Stream live.webm> # Output stream URL definition
Feed feed.ffm # Feed from which to receive video
Format webm
# Audio settings
AudioCodec vorbis
Audi0oBitRate 64 # Audio bitrate
# Video settings
VideoCodec libvpx
VideoSize 720x576 # Video resolution
VideoFrameRate 25 # Video FPS
AVOptionVideo flags +global_header # Parameters passed to encoder
# (same as ffmpeg command-line parameters)
AVOptionVideo cpu-used 0
AVOptionVideo qmin 10
AVOptionVideo qmax 42
AVOptionVideo quality good
AVOptionAudio flags +global_header
PreRoll 15
StartSendOnKey
VideoBitRate 400 # Video bitrate
</Stream>
<Stream status.html> # Server status URL
Format status
# Only allow local people to get the status
ACL allow 127.0.0.1
ACL allow 10.20.0.0 10.20.255.255
</Stream>
<Redirect index.html> # Just an URL redirect for index
# Redirect index.html to the appropriate site
URL http://www.ffmpeg.org/
</Redirect>
I have a long recording (hour+) in the format of MP3. The following is the info i managed to get from FFMPEG about the audio file:
[mp3 # 000001fe666da320] Skipping 0 bytes of junk at 58650.
[mjpeg # 000001fe666effe0] Changing bps to 8
[mp3 # 000001fe666da320] Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from '1.mp3':
Duration: 00:57:18.52, start: 0.000000, bitrate: 192 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, mono, s16p, 192 kb/s
Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1300x1370, 90k tbr, 90k tbn, 90k tbc
I would like to use Bing Speech API (Microsoft Oxford - Cognitive Services - Speech API) to transcribe this file (speech to text).
I believe that this is achievable by using something like the code below.
Option 1:
before sending up any audio data, you must first send up an SpeechAudioFormat descriptor to describe the layout and format of your raw audio data via DataRecognitionClient's sendAudioFormat() method. Can you provide a code sample for this option?
Option 2: converting the file to the target's acceptable format. I have done that with FFMPEG and this is what i got:
Duration: 00:57:23.67, bitrate: 256 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, 1 channels, s16, 256 kb/s
As I understand from the documentation, this should be acceptable: The audio must be PCM, mono, 16-bit sample, with sample rate of 8000 Hz or 16000 Hz.
I tried to send the audio to the server but did not get any reply. Am I on the right tracks? What is the maximum buffer size?
Do u see other, maybe easier option to get my audio file transcribed?
private void SendAudioHelper(string wavFileName)
{
using (FileStream fileStream = new FileStream(wavFileName, FileMode.Open, FileAccess.Read))
{
int bytesRead = 0;
byte[] buffer = new byte[1024];
try
{
do
{
// Get more Audio data to send into byte buffer.
bytesRead = fileStream.Read(buffer, 0, buffer.Length);
// Send of audio data to service.
this.dataClient.SendAudio(buffer, bytesRead);
}
while (bytesRead > 0);
}
finally
{
// We are done sending audio. Final recognition results will arrive in OnResponseReceived event call.
this.dataClient.EndAudio();
}
}
}
There is a limit of 15 seconds when you use the REST implementation. SDK has a limit of 2minutes.
Bing Speech team
I am trying to decode audio samples from various file formats using ffmpeg. Therefore I have started some experimenting based on the code in this discussion: How to decode audio via FFmpeg in Android . I use the latest FFMPEG release (1.0) and compile it using https://github.com/halfninja/android-ffmpeg-x264
AVFormatContext * pFormatCtx;
avcodec_register_all();
av_register_all();
int lError;
if ((lError = avformat_open_input(&pFormatCtx, filename, NULL, 0))
!= 0) {
LOGE("Error open source file: %d", lError);
return;
}
if ((lError = avformat_find_stream_info(pFormatCtx, 0)) < 0) {
LOGE("Error find stream information: %d (Streams: %d)", lError, pFormatCtx->nb_streams);
return;
}
LOGE("audio format: %s", pFormatCtx->iformat->name);
LOGE("audio bitrate: %d", pFormatCtx->bit_rate);
audioStreamIndex = av_find_best_stream(pFormatCtx, AVMEDIA_TYPE_AUDIO,
-1, -1, &codec, 0);
//if (audioStreamIndex < 0 || audioStreamIndex >= pFormatCtx->nb_streams)
// audioStreamIndex = 0;
LOGE("Stream: %d (total: %d)", audioStreamIndex, pFormatCtx->nb_streams);
LOGE("audio codec: %s", codec->name);
FFMPEG is compiled using enable-decoder=mp1/mp2/mp3/ogg/vorbis/wav/aac/theora and without any external libraries (e.g. libmp3lame, libtheora, etc.)
Opening of mp3 and wav files works without problems producing the following output for instance for mp3:
audio format: mp3
audio bitrate: 256121
stream: 0 (total: 1)
audio codec: mp3
But when I try to open an ogg file I get this:
Error find stream information: -1 (Streams: 1)
When I manually set audioStreamIndex=0 and comment out the return statement:
Error find stream information: -1 (Streams: 1)
audio format: mp3
audio bitrate: 0
stream: 0 (total: 1)
audio codec: mp3
For m4a (AAC) I get this:
audio format: mp3
audio bitrate: 288000
stream: 0 (total: 1)
audio codec: mp1
but later it fails in avcodec_decode_audio3.
I also tried to manually force a format without success:
AVInputFormat *pForceFormat= av_find_input_format("ogg");
if ((lError = avformat_open_input(&pFormatCtx, filename, pForceFormat, 0))
// continue
Is there something wrong with the loading code which makes it only work with mp3 and wav and fails for other formats?
Regards,
The problem was a missing demuxer.