Converting raw audio to ogg with gstreamer - audio

The following pipeline results in a 3kb .ogg file (I assume it's just an empty container):
gst-launch-1.0 --gst-debug=3 filesrc location=test.raw
! 'audio/x-raw, format=S16LE, channels=1, rate=32000'
! audioconvert
! vorbisenc
! oggmux
! filesink location=test.ogg
Here's the debug output:
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
0:00:00.048490941 813 0x556bf3625000 FIXME basesink gstbasesink.c:3077:gst_base_sink_default_event:<filesink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:00.048541997 813 0x556bf3625000 WARN audioencoder gstaudioencoder.c:985:gst_audio_encoder_finish_frame:<vorbisenc0> Can't copy metadata because input buffer disappeared
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.139954729 813 0x556bf3625000 WARN basesrc gstbasesrc.c:2400:gst_base_src_update_length:<filesrc0> processing at or past EOS
Got EOS from element "pipeline0".
Execution ended after 0:00:00.091883401
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
When I add this wav encode/decode, I get a good .ogg file:
gst-launch-1.0 --gst-debug=3 filesrc location=test.raw
! 'audio/x-raw, format=S16LE, channels=1, rate=32000'
! audioconvert
! wavenc
! wavparse
! audioconvert
! vorbisenc
! oggmux
! filesink location=test.ogg
debug output:
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
0:00:00.135676651 822 0x562b3cd64770 FIXME basesink gstbasesink.c:3077:gst_base_sink_default_event:<filesink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:00.135718946 822 0x562b3cd64770 WARN audioencoder gstaudioencoder.c:985:gst_audio_encoder_finish_frame:<vorbisenc0> Can't copy metadata because input buffer disappeared
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.219188746 822 0x562b3cd64770 WARN wavenc gstwavenc.c:795:gst_wavenc_write_toc:<wavenc0> have no toc
Got EOS from element "pipeline0".
Execution ended after 0:00:00.083921991
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
So my question is: what is the second pipeline, with wavenc ! wavparse, providing that the first is missing and is there a more straight-forward way to specify it or is the 2nd form actually the 'right' way to do it?

The first pipeline is fine as it works with the testaudiosrc (audio/x-raw-int)
I'm assuming your uncompressed audio file must be an uncompressed WAV file.
https://en.wikipedia.org/wiki/List_of_codecs#Audio_compression_formats
Wavenc may be preprocessing the LPCM and converting to something vorbisenc can use. I suspect the data width needs to be 32 or 64 for vorbisenc and that may be the showstopper.
PCM signed 16-bit little-endian (S16LE) >
audioconvert - Convert audio to different formats (in:audio/x-raw-int out:audio/x-raw-int)
wavenc - Encode raw audio into WAV (in:audio/x-raw-int out:audio/x-wav)
wavparse - Parse a .wav file into raw audio (in:audio/x-wav out:audio/x-raw-float width: { 32, 64 })
vorbisenc - Encodes audio in Vorbis format (in:audio/x-raw-float out:audio/x-vorbis)
gst-launch audiotestsrc num-buffers=50 \
! vorbisenc \
! oggmux \
! filesink location=test.ogg
play test.ogg
addendum: I downloaded your file and was able to confirm you were doing unrealized stream conversion from 16 to 32 bit. Vorbisenc only accepts a 32 bit width. To answer your original question, no, you do not need the wavparsing. Here is the efficient pipeline you were looking for, simplified for width conversion.
gst-launch --gst-debug=2 filesrc location=test.raw \
! audio/x-raw-int, width=16, channels=2, depth=16, rate=16000, endianness=1234, signed=true \
! audioconvert \
! audio/x-raw-float, width=32, channels=2, rate=16000, endianness=1234, signed=true \
! vorbisenc \
! oggmux \
! filesink location=test.ogg

Related

GStreamer problem with separating audio and video

I'm new to gstreamer, basically a newbie.
I want to receive an rtmp video, process the video, reencode the video, merge it with the sound from the received video and then send it out as a new rtmp-video. Somehow I can not get get the sound working:
Receiver:
"rtmpsrc location=rtmp://xx.yy.10.40:1935/orig/1 do-timestamp=true ! queue ! flvdemux name=demux demux.video ! h264parse ! video/x-h264 ! nvh264dec ! videoconvert ! appsink"
"demux.audio ! aacparse ! queue ! mp4mux streamable=true ! shmsink socket-path=/tmp/foo sync=true wait-for-connection=false shm-size=100000000"
Please note, I separated the 2 strings simply for better readability. Both strings together are the reveiver queue. I get no error or warning up to GST_DBG=3. I used mp4mux because some claim, that I need a container.
Sender:
"appsrc ! videoconvert ! nvh264enc ! h264parse ! queue ! mux.video"
" shmsrc socket-path=/tmp/foo ! qtdemux ! aacparse ! queue ! mux.audio"
" flvmux name=mux ! rtmpsink location=rtmp://xx.yy.10.50:1935/result/1"
Please note I separated the strings for better readability. Again I get no error. But reading the sound buffer from shared memory (shmsrc) simply stalls. If I remove this line everything seems to work perfectly well, stable even for hours.
Any ideas someone, because all the working solutions seem to use raw audio and caps. But actually I'm not interested in audio at all, I just need it copied to the sender...
so an update from our side:
We tried many things, but the answer is that this problem is inherent to the elements shmsink and shmsrc from gstreamer
When using the shm-communication between threads you loose all meta-data, basically the audio stream coming from shmsrc is not an audio stream any more. You can test this by using launch:
gst-launch-1.0 --verbose rtmpsrc location=rtmp://xx.yy.10.40:1935/ai/1 do-timestamp=true timeout=10 ! flvdemux name=demux demux.video ! h264parse ! video/x-h264 ! fakesink demux.audio ! queue ! faad ! shmsink socket-path=/tmp/foo sync=true wait-for-connection=false shm-size=100000
will produce a lot of info, especially the audio and video formats
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, rate=(int)48000, channels=(int)2, codec_data=(buffer)1190 ...
/GstPipeline:pipeline0/GstShmSink:shmsink0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, codec_data=(buffer)014d4029ffe10015674d402995900780227e5c04400000fa40002ee02101000468eb8f20, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)24000/1001, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4.1
When you do the same on the side of the shmsrc (pls note we only transfer the audio via shm)
gst-launch-1.0 --verbose shmsrc socket-path=/tmp/foo ! queue ! fakesink
you will get nothing, for gstreamer "nothing" looks like this:
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
If you want to use shmsrc, you need to set the meta-data via caps manually:
gst-launch-1.0 --verbose shmsrc socket-path=/tmp/foo ! 'audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2' ! queue ! fakesink
Will give you:
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2
Which is a correct, but totally useless solution. So we decided to move to libav instead of gstreamer.

Write, then read rtpopus to file with gstreamer?

Is it possible to write rtpopus to a file, then read it back with gstreamer? It seems simple but I'm getting nowhere with it and can't seem to find any information online. Here is my attempt:
gst-launch-1.0.exe audiotestsrc ! opusenc ! rtpopuspay ! filesink location=test.opus
Then, close and run:
gst-launch-1.0.exe filesrc location="test.opus" ! rtpopusdepay ! fakesink dump=true
gstreamer fails with:
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
I don't think it could work. RTP is related to UDP packetization so it would work when streaming over UDP.
You'd better use a file container supporting opus audio such as matroskamux:
gst-launch-1.0 -e audiotestsrc ! audioconvert ! opusenc ! matroskamux ! filesink location=test.mkv
# Let play for 5s and stop with Ctrl-C
# Replay:
gst-launch-1.0 filesrc location=test.mkv ! matroskademux ! opusdec ! audioconvert ! autoaudiosink

How to fix 'Lossing Stream Before End of Stream in Gstream-0.10'

I have streamed video via vlc player over rtsp and then I have displayed this video via gstreamer-0.10. However, While vlc was streaming video over rtsp, I suddenly lost stream in the first minute of stream before end of stream.
I have used following pipeline:
GST_DEBUG=2 gst-launch-0.10 rtspsrc location=rtsp://127.0.0.1:8554/test !
gstrtpjitterbuffer ! rtph264depay ! ffdec_h264 ! videorate ! xvimagesink
sync=false
I have got following output:
rtpjitterbuffer.c:428:calculate_skew: delta - skew: 0:00:01.103711536 too big, reset skew
rtpjitterbuffer.c:387:calculate_skew: backward timestamps at server, taking new base time
Got EOS from element "pipeline0".
Execution ended after 59982680309 ns.
Setting pipeline to PAUSED ...
gst_rtspsrc_send: got NOT IMPLEMENTED, disable method PAUSE
How to fix this problem ?
I have found solution. I have used rtspt://... instead of rtsp://... to enforce TCP instead of UDP.
gst-launch-0.10 rtspsrc location= rtspt://127.0.0.1:8554/test ! gstrtpjitterbuffer ! rtph264depay ! ffdec_h264 ! xvimagesink sync=false

Gstreamer 1.8.3 rtpbin and rtpjpegpayload throws internal data flow error

I'm using Gstreamer version 1.8.3 and the following pipelines to send and receive a rtp/rtcp streaming.
Vars:
export SAMPLE="overwatch.mjpeg"
export IMAGE_CAPS="image/jpeg,width=1280,height=720,framerate=1/10,format=I420"
Listener:
test_play_rtpbin(){
gst-launch-1.0 --gst-debug=3 rtpbin name=rtpbin \
udpsrc port=25000 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG,width=1280,height=720 ! rtpbin.recv_rtp_sink_0 \
rtpbin ! rtpjpegdepay ! queue ! jpegparse ! jpegdec ! videoconvert ! fpsdisplaysink \
udpsrc port=25001 ! rtpbin.recv_rtcp_sink_0 \
rtpbin.send_rtcp_src_0 ! udpsink port=25005 host="192.168.0.33" sync=false async=false
}
Publisher:
test_record_rtpbin(){
gst-launch-1.0 --gst-debug=3 rtpbin name=t \
multifilesrc location=$SAMPLE loop=true ! queue ! $IMAGE_CAPS ! jpegparse ! $IMAGE_CAPS ! queue ! rtpjpegpay pt=26 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG,width=1280,height=720 ! t.send_rtp_sink_0 \
t.send_rtp_src_0 ! udpsink port=25000 host="192.168.0.33" \
t.send_rtcp_src_0 ! udpsink port=25001 host="192.168.0.33" sync=false async=false \
udpsrc port=25005 ! t.recv_rtcp_sink_0
}
But for some reason it keeps throwing me the following error:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:22.564008753 16798 0x1f6b540 WARN rtpjitterbuffer rtpjitterbuffer.c:487:calculate_skew: delta - skew: 0:00:09.998355706 too big, reset skew
0:00:26.750010395 16798 0x1f6b540 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc0> error: Internal data flow error.
0:00:26.750027345 16798 0x1f6b540 WARN basesrc gstbasesrc.c:2948:gst_base_src_loop:<udpsrc0> error: streaming task paused, reason not-linked (-1)
ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-linked (-1)
Execution ended after 0:00:26.709638512
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
A full level 5 log is here.
The sample file have the following format:
$ mediainfo overwatch.mjpeg
General
Complete name : overwatch.mjpeg
Format : JPEG
File size : 1.63 GiB
Image
Format : JPEG
Width : 1 280 pixels
Height : 720 pixels
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Stream size : 1.63 GiB (100%)
The pipelines work when I use rtp only but when I try to run a simple rtpbin example it keeps failing. Please help!
Feel dumb after testing with other gstreamer versions/configurations, a simple miss, in the test_play_rtpbin bash function, I was missing a point.
the line
rtpbin ! rtpjpegdepay ! queue... should be
rtpbin. ! rtpjpegdepay ! queue...

gstreamer audio error on linux

i am using g streamer-0.10 on Ubuntu os for streaming an web cam video on to an rtmp server i am getting an video output but their is a problem in audio . Below command used for streaming
gst-launch-0.10 v4l2src ! videoscale method=0 ! video/x-raw-yuv,width=852,height=480,framerate=(fraction)24/1 ! ffmpegcolorspace ! x264enc pass=pass1 threads=0 bitrate=900 tune=zerolatency ! flvmux name=mux ! rtmpsink location='rtmp://..../live/testing' demux. alsasrc device="hw:0,0" ! audioresample ! audio/x-raw-int,rate=48000,channels=2,depth=16 ! pulseaudiosink
Blockquote
by running the above command i got an error
gstbaseaudiosrc.c(840): gst_base_audio_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 13920 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
Blockquote
so the audio is not audible.
Help me out to solve this problem.
Thanks in advance
Ameeth
I don't understand your pipeline. What is "demux." in the middle?
The problem you are facing is because you have not seperated your elements with queues. Keep a queue before your sinks and after your sources to give the rest all seperate threads to run. It should allow get rid of the issue.
Since I don't have pulse audio or rtmp reciever in my system i have tested out the following and it works.
gst-launch-0.10 v4l2src ! ffmpegcolorspace ! queue ! x264enc pass=pass1 threads=0 bitrate=900000 tune=zerolatency ! queue ! flvmux name=mux ! fakesink alsasrc ! queue ! audioresample ! audioconvert ! queue ! autoaudiosink
You can change it accordingly and use it. The only thing I had to do to make it work and remove the error your are facing is to add the queues.
For me (Logitech c920 on Raspberry Pi3 w/ GStreamer 1.4.4) I was able to get rid of the "Dropped samples" warning by using audioresample to set the sampling rate of the alsasrc to something that flvmux liked. From gst-inspect-1.0 flvmux, it looks like flvmux only supports 5512, 11025, 22050, 44100 sample rates for x-raw and 5512, 8000, 11025, 16000, 22050, 44100 for mp4. Here's my working pipeline
gst-launch-1.0 -v -e \
uvch264src initial-bitrate=800000 average-bitrate=800000 iframe-period=2000 device=/dev/video0 name=src auto-start=true \
src.vidsrc ! video/x-h264,width=864,height=480,framerate=30/1 ! h264parse ! mux. \
alsasrc device=hw:1 ! 'audio/x-raw, rate=32000, format=S16LE, channels=2' ! queue ! audioresample ! "audio/x-raw,rate=44100" ! queue ! voaacenc bitrate=96000 ! mux. \
flvmux name=mux ! rtmpsink location="rtmp://live-sea.twitch.tv/app/MYSTREAMKEY"
I was surprised that flvmux didn't complain about getting an audio source that was at an unsupported sampling rate. Not sure if that's expected behavior.

Resources