I am working on a session topology that records a video (with audio) while showing the video stream with an EVR. I am using SampleGrabber sinks for the video and audio streams so I can easily control when I start/end recording (using a sinkwriter) and extract sample data as I need to (for capturing stills and displaying the microphone's audio level). I am using a Tee node to split the video feed between the EVR and the video samplegrabber, and a copierMFT to deliver samples to the EVR. Both the audio and the video samplegrabbers are marked RATELESS.
Each branch of the topology work by themselves, but when I add both to the topology the media session fails to start, with a status="E_UNEXPECTED Catastrophic Error" set on the MESessionStarted event. That is the only error I find in the MFTrace logs. My camera never turns on and I don't receive any samples in my samplegrabbers. The topology seems to resolve correctly, so I'm not sure what the session didn't expect. The documentation for MediaSession::Start doesn't mention anything about this error. My current guess is that is has something to do with syncing the presentation clock, but setting the samplegrabber output nodes as rateless doesn't seem to help. After I receive the Topology Ready status event, checking the session's clock state returns MFCLOCK_STATE_INVALID.
Is there any special setup required to have a live camera source and microphone source in the same topology? Is there any way to get more info on the E_UNEXPECTED error?
Any help on getting to the bottom of this is appreciated. Here is a snippet of my mftrace log which shows the ready topology and the error:
12656,14E8 19:25:29.69496 CTopologyHelpers::Trace #02A799F8 >>>>>>>>>>>>> ready topology
12656,14E8 19:25:29.69499 CTopologyHelpers::TraceNode # Node 0 #02A7A110 ID:317000000001, 0 inputs, 1 outputs, type 1, MF_TOPONODE_MARKIN_HERE=1;MF_TOPONODE_MARKOUT_HERE=1;MF_TOPONODE_MEDIASTART=0 (0,0);MF_TOPONODE_SOURCE=#02A95200;MF_TOPONODE_PRESENTATION_DESCRIPTOR=#02A63820;MF_TOPONODE_STREAM_DESCRIPTOR=#02A662C8;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A85698
12656,14E8 19:25:29.69500 CMFTopologyNodeDetours::GetGUID #02A7A110 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69501 CTopologyHelpers::TraceObject # Source #02A95200 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MFCORE.DLL), MFMEDIASOURCE_CHARACTERISTICS=0x00000005
12656,14E8 19:25:29.69507 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A7A188 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69509 CTopologyHelpers::TraceNode # Node 1 #02A7A188 ID:317000000002, 1 inputs, 2 outputs, type 3, MF_TOPONODE_PRIMARYOUTPUT=0;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A7CE00
12656,14E8 19:25:29.69509 CMFTopologyNodeDetours::GetGUID #02A7A188 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69509 CTopologyHelpers::TraceObject # Tee #00000000 {00000000-0000-0000-0000-000000000000} ((null)), (null)
12656,14E8 19:25:29.69514 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A7A110 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69518 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A79D68 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69522 CTopologyHelpers::TraceStream # Output stream 1, connected to node #02A7B1F0 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69524 CTopologyHelpers::TraceNode # Node 2 #02A79D68 ID:317000000005, 1 inputs, 1 outputs, type 2, {89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A85908
12656,14E8 19:25:29.69524 CMFTopologyNodeDetours::GetGUID #02A79D68 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69526 CTopologyHelpers::TraceObject # MFT #02A6DAE8 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MFCORE.DLL), MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE=1;{851745D5-C3D6-476D-9527-498EF2D10D18}=4
12656,14E8 19:25:29.69531 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A7A188 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69536 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A79EA0 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69538 CTopologyHelpers::TraceNode # Node 3 #02A79DE0 ID:317000000003, 1 inputs, 0 outputs, type 0, MF_TOPONODE_STREAMID=0;MF_TOPONODE_DISABLE_PREROLL=1;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A85AE0
12656,14E8 19:25:29.69539 CMFTopologyNodeDetours::GetGUID #02A79DE0 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69539 CTopologyHelpers::TraceObject # Sink #02A6FB28 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MFCORE.DLL), (null)
12656,14E8 19:25:29.69544 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A7B1F0 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=2560;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_GEOMETRIC_APERTURE=00 00 00 00 00 00 00 00 80 02 00 00 e0 01 00 00 ;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=42949672960333333 (10000000,333333);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_SAMPLE_SIZE=1228800;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_SUBTYPE=MFVideoFormat_RGB32
12656,14E8 19:25:29.69546 CTopologyHelpers::TraceNode # Node 4 #02A7B1F0 ID:31700000000C, 1 inputs, 1 outputs, type 2, MF_TOPONODE_TRANSFORM_OBJECTID={98230571-0087-4204-B020-3282538E57D3};{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A86040
12656,14E8 19:25:29.69546 CTopologyHelpers::TraceObject # MFT #02A7A488 {98230571-0087-4204-B020-3282538E57D3} (C:\Windows\SYSTEM32\colorcnv.dll), <NULL>
12656,14E8 19:25:29.69551 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A7A188 stream 1, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69556 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A79DE0 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=2560;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_GEOMETRIC_APERTURE=00 00 00 00 00 00 00 00 80 02 00 00 e0 01 00 00 ;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=42949672960333333 (10000000,333333);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_SAMPLE_SIZE=1228800;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_SUBTYPE=MFVideoFormat_RGB32
12656,14E8 19:25:29.69559 CTopologyHelpers::TraceNode # Node 5 #02A79EA0 ID:317000000004, 1 inputs, 0 outputs, type 0, MF_TOPONODE_STREAMID=0;MF_TOPONODE_NOSHUTDOWN_ON_REMOVE=0;MF_TOPONODE_DISABLE_PREROLL=1;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A86218;{B8AA3129-DFC9-423A-8ACD-1D82850A3D1F}=#02A8B9E0
12656,14E8 19:25:29.69560 CMFTopologyNodeDetours::GetGUID #02A79EA0 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69560 CTopologyHelpers::TraceObject # Sink #02A8E5D4 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MF.dll), (null)
12656,14E8 19:25:29.69565 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A79D68 stream 0, MT: MF_MT_FRAME_SIZE=2748779069920 (640,480);MF_MT_AVG_BITRATE=147456000;MF_MT_YUV_MATRIX=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_VIDEO_LIGHTING=3;MF_MT_DEFAULT_STRIDE=1280;MF_MT_VIDEO_CHROMA_SITING=6;MF_MT_AM_FORMAT_TYPE=FORMAT_VIDEOINFO2;MF_MT_FIXED_SIZE_SAMPLES=1;MF_MT_VIDEO_NOMINAL_RANGE=2;MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297 (1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_FRAME_RATE_RANGE_MIN=128849018881 (30,1);MF_MT_SAMPLE_SIZE=614400;MF_MT_VIDEO_PRIMARIES=2;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_RATE_RANGE_MAX=128849018881 (30,1);MF_MT_SUBTYPE=MFVideoFormat_YUY2
12656,14E8 19:25:29.69568 CTopologyHelpers::TraceNode # Node 6 #02A79F18 ID:317000000006, 0 inputs, 1 outputs, type 1, MF_TOPONODE_MARKIN_HERE=1;MF_TOPONODE_MARKOUT_HERE=1;MF_TOPONODE_MEDIASTART=0 (0,0);MF_TOPONODE_SOURCE=#02A65D40;MF_TOPONODE_PRESENTATION_DESCRIPTOR=#02A63CE8;MF_TOPONODE_STREAM_DESCRIPTOR=#02A63048;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A86370
12656,14E8 19:25:29.69568 CMFTopologyNodeDetours::GetGUID #02A79F18 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69569 CTopologyHelpers::TraceObject # Source #02A65D40 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MFCORE.DLL), MFMEDIASOURCE_CHARACTERISTICS=0x00000005
12656,14E8 19:25:29.69571 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A7BFA0 stream 0, MT: MF_MT_AUDIO_AVG_BYTES_PER_SECOND=384000;MF_MT_AUDIO_BLOCK_ALIGNMENT=8;MF_MT_AUDIO_NUM_CHANNELS=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Audio;MF_MT_AUDIO_CHANNEL_MASK=3;MF_MT_AUDIO_SAMPLES_PER_SECOND=48000;MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_AUDIO_BITS_PER_SAMPLE=32;MF_MT_SUBTYPE=MFAudioFormat_Float
12656,14E8 19:25:29.69573 CTopologyHelpers::TraceNode # Node 7 #02A7BDC0 ID:317000000007, 1 inputs, 0 outputs, type 0, MF_TOPONODE_STREAMID=0;MF_TOPONODE_DISABLE_PREROLL=1;{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02A864E0
12656,14E8 19:25:29.69573 CMFTopologyNodeDetours::GetGUID #02A7BDC0 attribute not found guidKey = MF_TOPONODE_TRANSFORM_OBJECTID
12656,14E8 19:25:29.69574 CTopologyHelpers::TraceObject # Sink #02A70410 {00000000-0000-0000-0000-000000000000} (C:\WINDOWS\SYSTEM32\MFCORE.DLL), (null)
12656,14E8 19:25:29.69575 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A7BFA0 stream 0, MT: MF_MT_AUDIO_AVG_BYTES_PER_SECOND=88200;MF_MT_AUDIO_BLOCK_ALIGNMENT=2;MF_MT_AUDIO_NUM_CHANNELS=1;MF_MT_MAJOR_TYPE=MEDIATYPE_Audio;MF_MT_AUDIO_CHANNEL_MASK=4;MF_MT_AUDIO_SAMPLES_PER_SECOND=44100;MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_AUDIO_BITS_PER_SAMPLE=16;MF_MT_SUBTYPE=MFAudioFormat_PCM
12656,14E8 19:25:29.69577 CTopologyHelpers::TraceNode # Node 8 #02A7BFA0 ID:317000000010, 1 inputs, 1 outputs, type 2, MF_TOPONODE_TRANSFORM_OBJECTID={F447B69E-1884-4A7E-8055-346F74D6EDB3};{89485B85-2FFA-4547-B269-B82C79EE197C}=1;{9C86CC4E-68CE-4CFF-AA1E-9A5A40D5B4E0}=#02AAA068
12656,14E8 19:25:29.69578 CTopologyHelpers::TraceObject # MFT #02A7C4AC {F447B69E-1884-4A7E-8055-346F74D6EDB3} (C:\Windows\SYSTEM32\resampledmo.dll), <NULL>
12656,14E8 19:25:29.69581 CTopologyHelpers::TraceStream # Input stream 0, connected to node #02A79F18 stream 0, MT: MF_MT_AUDIO_AVG_BYTES_PER_SECOND=384000;MF_MT_AUDIO_BLOCK_ALIGNMENT=8;MF_MT_AUDIO_NUM_CHANNELS=2;MF_MT_MAJOR_TYPE=MEDIATYPE_Audio;MF_MT_AUDIO_CHANNEL_MASK=3;MF_MT_AUDIO_SAMPLES_PER_SECOND=48000;MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_AUDIO_BITS_PER_SAMPLE=32;MF_MT_SUBTYPE=MFAudioFormat_Float
12656,14E8 19:25:29.69583 CTopologyHelpers::TraceStream # Output stream 0, connected to node #02A7BDC0 stream 0, MT: MF_MT_AUDIO_AVG_BYTES_PER_SECOND=88200;MF_MT_AUDIO_BLOCK_ALIGNMENT=2;MF_MT_AUDIO_NUM_CHANNELS=1;MF_MT_MAJOR_TYPE=MEDIATYPE_Audio;MF_MT_AUDIO_SAMPLES_PER_SECOND=44100;MF_MT_AUDIO_PREFER_WAVEFORMATEX=1;MF_MT_ALL_SAMPLES_INDEPENDENT=1;MF_MT_AUDIO_BITS_PER_SAMPLE=16;MF_MT_SUBTYPE=MFAudioFormat_PCM
12656,14E8 19:25:29.69584 CTopologyHelpers::Trace #02A799F8 MF_TOPOLOGY_RESOLUTION_STATUS = 0
12656,14E8 19:25:29.69584 CTopologyHelpers::Trace #02A799F8 <<<<<<<<<<<<< ready topology
12656,14E8 19:25:29.69590 CKernel32ExportDetours::OutputDebugStringA # TOPOLOGY READY
12656,14E8 19:25:29.69600 CMFTopologyDetours::GetUINT32 #02A799F8 attribute not found guidKey = {9C27891A-ED7A-40E1-88E8-B22727A024EE}
12656,14E8 19:25:29.69612 CMFMediaSessionDetours::EndGetEvent #02A6DDC0 Met=103 MESessionStarted, value (empty), failed HrStatus=8000FFFF E_UNEXPECTED,
UPDATE 1:
As suggested by mofo77, I tried creating a new presentationclock with a systemtimesource and setting it as set the EVR's presentation clock. Unfortunately this didn't change anything.
Looking more into the TopologyReady event, I noticed that the clock doesn't have a presentation time source:
IMFClock* pClock;
hr = pMediaSession->GetClock(&pClock);
MFCLOCK_STATE state;
hr = pClock->GetState(0, &state);
//state == MFCLOCK_STATE_INVALID
IMFPresentationClock* pPresentationClock;
hr = pClock->QueryInterface(&pPresentationClock);
IMFPresentationTimeSource* pPresentationTimeSource;
hr = pPresentationClock->GetTimeSource(&pPresentationTimeSource);
//hr == 0xc00d9c41 : No Presentation Time Source has been specified.
I tried creating a systemtimesource and setting it on the mediasession's clock, but that also doesn't help.
With only the video branch (or only the audio branch), my samplegrabber receives the OnSetPresentationClock callback. With both added to the topology, this never happens and I get the catastrophic failure status on the MESessionStarted event.
I think you need to provide a valid clock to the EVR.
UPDATE
After your update and better reading your question, it seems that you don't use MFCreateAggregateSource to Aggregate audio and video capture.
If i'm right, you can check my project : MFNode
Under MFNodePlayer, you have source code for MFCreateAggregateSource, and the concept of using audio/video capture with the Media Session. You will not need to worry about clock, because yes, the Media Session handles the presentation clock.
This question was also asked on MSDN where Venki.thiyag replied
In case if your topology has 2 sources, then topology needs to be
added via Sequencer source.
Refer
https://msdn.microsoft.com/en-us/library/windows/desktop/ms698996(v=vs.85).aspx
for more details.
This is a quote from the Sequencer Source documentation
You can use it to create playlists, or to play streams from multiple
sources simultaneously.
Unfortunately the documentation for Sequencer Source does not detail how to "to play [capture] streams from multiple sources simultaneously". Note the emphasis on simultaneously.
A method for how to use Sequencer Source to simultaneously capture from multiple Media Sources is described in the ofxWMFVideoPlayer technical description section titled "Bonus (experimental) feature : playing multiple video in sync"
Like #mofo77, the ofxWMFVideoPlayer technical description also suggests using the MFCreateAggregateSource function as another way to simultaneously capture from multiple Media Sources.
The MFCreateAggregateSource function is a very direct approach to simultaneously capturing from multiple Media Device Sources. This a quote from the MFCreateAggregateSource documentation remarks.
The aggregated media source is useful for combining streams from
separate media sources. For example, you can use it to combine a
video capture source and an audio capture source.
Related
I have an MP4 file, which I would like to convert into an MPEG4 file. TO do this, I have found the PythonVideoConvert package. On the PyPI page, the following code is given:
from converter import Converter
conv = Converter()
info = conv.probe('test/test1.avi')
PATH = 'C:/Users/.../'
convert = conv.convert(PATH +'Demo.mp4', PATH + 'Demo.mpeg4', {
'format': 'mpeg4',
'audio': {
'codec': 'aac',
'samplerate': 11025,
'channels': 2
},
'video': {
'codec': 'hevc',
'width': 720,
'height': 400,
'fps': 25
}})
When I run this code, a convert object is created. However, there is no .mpeg4 video in the PATH directory.
Therefore, I have two questions:
Is the code above correct for converting a .mp4 file into a .mpeg4 file
What do I need to run to save the converted video as a .mpeg4 file?
Based on Selcuk's comment, I ran the following code:
for timecode in convert:
pass
This gives the error:
Traceback (most recent call last):
File "<ipython-input-60-14c9225c3ac2>", line 1, in <module>
for timecode in convert:
File "C:\Users\20200016\Anaconda3\lib\site-packages\converter\__init__.py", line 229, in convert
optlist = self.parse_options(options, twopass)
File "C:\Users\20200016\Anaconda3\lib\site-packages\converter\__init__.py", line 60, in parse_options
raise ConverterError(f'Requested unknown format: {str(f)}')
ConverterError: Requested unknown format: mpeg4
So, my suggested format seems incorrect. What can I do to convert a video into .mpeg4?
I don't think PythonVideoConverter is meant to be used in Windows.
I was getting an exception AttributeError: module 'signal' has no attribute 'SIGVTALRM', because SIGVTALRM is not a valid signal in Windows.
The default path of FFmpeg an FFprobe command line tools, also doesn't make sense for Windows.
We may still use the package in Windows, but it's recommended to set ffmpeg_path and ffprobe_path.
Example:
conv = Converter(ffmpeg_path=r'c:\FFmpeg\bin\ffmpeg.exe', ffprobe_path=r'c:\FFmpeg\bin\ffprobe.exe')
We also have to disable the timeout feature, by setting timeout=None argument.
mpeg4 is not a valid FFmpeg format, but we can still use it as a file extension.
(format is FFmpeg terminology usually applies container format).
When non-standart file extension is used, we have to set the format entry.
Setting 'format': 'mp4' creates MP4 file container (may be created with the non-standart .mpeg4 file extension).
Complete code sample:
from converter import Converter
conv = Converter(ffmpeg_path=r'c:\FFmpeg\bin\ffmpeg.exe', ffprobe_path=r'c:\FFmpeg\bin\ffprobe.exe')
#info = conv.probe('test/test1.avi')
PATH = 'C:/Users/Rotem/'
convert = conv.convert(PATH + 'Demo.mp4', PATH + 'Demo.mpeg4', {
'format': 'mp4', #'format': 'mpeg4',
'audio': {
'codec': 'aac',
'samplerate': 11025,
'channels': 2
},
'video': {
'codec': 'hevc',
'width': 720,
'height': 400,
'fps': 25
}},
timeout=None)
# https://pypi.org/project/PythonVideoConverter/
for timecode in convert:
print(f'\rConverting ({timecode:.2f}) ...')
We may see the media information of Demo.mpeg4 using MediaInfo tool:
General
Complete name : C:\Users\Rotem\Demo.mpeg4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 207 KiB
Duration : 10 s 148 ms
Overall bit rate mode : Variable
Overall bit rate : 167 kb/s
Writing application : Lavf58.45.100
FileExtension_Invalid : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main#L3#Main
Codec ID : hev1
Codec ID/Info : High Efficiency Video Coding
Duration : 10 s 0 ms
Bit rate : 82.5 kb/s
Width : 720 pixels
Height : 400 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.011
Stream size : 101 KiB (49%)
Writing library : x265 3.4+28-419182243:[Windows][GCC 9.3.0][64 bit] 8bit+10bit+12bit
Encoding settings : ...
Color range : Limited
Codec configuration box : hvcC
Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 10 s 148 ms
Duration_LastFrame : -70 ms
Bit rate mode : Variable
Bit rate : 79.1 kb/s
Maximum bit rate : 128 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 11.025 kHz
Frame rate : 10.767 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 98.0 KiB (47%)
Title : IsoMedia File Produced by Google, 5-11-2011
Language : English
Default : Yes
Alternate group : 1
In MediaInfo output, the MP4 file container applies "MPEG-4" format...
Note:
The HEVC video format applies H.265 video codec - in most cases the codec is considered to be more relevant then container.
'Requested unknown format: mpeg4'
*.mpeg4 is not valid container. mpeg4 is codec, *.something (avi, mp4, mov, mkv, ...) are containers.
basicly: codec.CONTAINER or your_mpeg4_video.mkv etc.
video codec (like mpeg4) handle only video, but you need more than only visual, you need audio, many audio tracks (eng, de, nl, 2.0, 5.1, 7.1 ...), subtitles, etc and these stuff are inside container.
install ffmpeg: https://ffmpeg.org/
try this basic script:
import subprocess
input_file = 'Demo.mp4'
output_file = 'Demo.mkv' # or .mp4, .mov, ...
ffmpeg_cli = "ffmpeg -i '{}' -vcodec libx265 '{}'".format(input_file, output_file)
subprocess.call(ffmpeg_cli, shell=True)
I don't know what are you doing (what you want, what are your expectations) but if you looking for way how to degrese size of video,
look here: https://github.com/MarcelSuleiman/convert_h264_to_h265
simple.
I'm trying to connect to a shoutcast server from a darkice client using Ubuntu. This is my configuration:
#this section describes general aspects of the live streaming session
[general]
duration = 0 # duration of encoding, in seconds. 0 means forever
bufferSecs = 10 # size of internal slip buffer, in seconds
reconnect = yes # reconnect to the server(s) if disconnected
realtime = no # run the encoder with POSIX realtime priority
rtprio = 3 # scheduling priority for the realtime threads
# this section describes the audio input that will be streamed
[input]
device = hw:CARD=PCH,DEV=0
sampleRate = 44100 # sample rate in Hz. try 11025, 22050 or 44100
bitsPerSample = 16 # bits per sample. try 16
channel = 2 # channels. 1 = mono, 2 = stereo
# this section describes a streaming connection to an IceCast2 server
# there may be up to 8 of these sections, named [icecast2-0] ... [icecast2-7]
# these can be mixed with [icecast-x] and [shoutcast-x] sections
[shoutcast-0]
bitrateMode = cbr
format = mp3
bitrate = 96
quality = 1.0
server = xxxxxxxxxxxxxxx
port = 8020
password = xxxxxxxxxxxxxxx
name = Radio website
url = https://www.mywebsite.it
genre = live
public = no
But when I run
darkice -v 10 -c /etc/darkice-shoutcast.cfg
It only shows this, without errors or similar, but there is no streaming at the url. Using BUTT it works. I've also tested with 8021 instead of 8020 for port (8020 it's the port number given by the provider) but no luck.
DarkIce 1.4 live audio streamer, http://code.google.com/p/darkice/
Copyright (c) 2000-2007, Tyrell Hungary, http://tyrell.hu/
Copyright (c) 2008-2013, Akos Maroy and Rafael Diniz
This is free software, and you are welcome to redistribute it
under the terms of The GNU General Public License version 3 or
any later version.
Using config file: /etc/darkice-shoutcast.cfg
18-May-2021 12:02:28 Using ALSA DSP input device: hw:CARD=PCH,DEV=0
18-May-2021 12:02:28 buffer size: 1764000
18-May-2021 12:02:28 encoding
18-May-2021 12:02:28 MultiThreadedConnector :: transfer, bytes 0
18-May-2021 12:02:28 MultiThreadedConnector :: ThreadData :: threadFunction, was (thread, priority, type): 0x5568a502c010 0 SCHED_OTHER
18-May-2021 12:02:28 MultiThreadedConnector :: ThreadData :: threadFunction, now is (thread, priority, type): 0x5568a502c010 0 SCHED_OTHER
ADDENDUM
I've used tcpdump to understand what could be and I just see something similar to "invalid password"
: Flags [P.], cksum 0xc379 (correct), seq 1:19, ack 3090, win 294, options [nop,nop,TS val 3348978428 ecr 531576376], length 18
E..F1O#.1.3'.}.......T.Jq.k.D......&.y.....
..Z...68Invalid Passwor
Suggestions on how to better debug or fix this?
SOLVED
It seems the error is related to the password and wrong parsing of the config file, so I've written it without spaces
[shoutcast-0]
bitrateMode = cbr
format = mp3
bitrate = 96
quality = 1.0
server = xxxxxxxxxxxxxxx
port = 8020
password=xxxxxxxxxxxxxxx
name = Radio website
url = https://www.mywebsite.it
genre = live
public = no
I recently dipped my feet into encoding with FFmpeg. It's been working fine for me apart form 1 issues I have noticed recently. When I try and re-encode an audio track to acc the media info tags are somewhat wrong. Bitrate, streamsize etc seems to be copied form the source as opposed to reading it from the new track.
Essentially I want to copy the source video, audio and subtitles and also create an AAC version of the soundtrack for playback on devices that don't support HD audio.
Below is an example of what I am doing:
ffmpeg -i source.mkv -map_chapters 0 -map 0:v -c:v copy -c:s copy -map 0:a -c:a:0 copy -map 0:a c:a:1 aac destination.mkv
This results in a media info of:
Audio #1
ID : 2
Format : MLP FBA 16-ch
Format/Info : Meridian Lossless Packing FBA with 16-channel presentation
Commercial name : Dolby TrueHD with Dolby Atmos
Codec ID : A_TRUEHD
Duration : 1 h 50 min
Bit rate mode : Variable
Bit rate : 3 545 kb/s
Maximum bit rate : 5 826 kb/s
Channel(s) : 8 channels
Channel layout : L R C LFE Ls Rs Lb Rb
Sampling rate : 48.0 kHz
Frame rate : 1 200.000 FPS (40 SPF)
Bit depth : 24 bits
Compression mode : Lossless
Delay relative to video : 21 ms
Stream size : 2.73 GiB (29%)
Title : Dolby TrueHD 7.1 (Atmos)
Language : English
Default : Yes
Forced : No
Number of dynamic objects : 11
Bed channel count : 1 channel
Bed channel configuration : LFE
Audio #2
ID : 3
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : A_AAC-2
Duration : 1 h 50 min
Bit rate : 3 545 kb/s
Channel(s) : 8 channels
Channel layout : C L R Ls Rs Lb Rb LFE
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 2.73 GiB (29%)
Title : AAC 5.1
Writing library : Lavc58.62.100 aac
Language : English
Default : Yes
Forced : No
As you can see the bit rate, stream size, etc is copied from the source track which is incorrect. I've tried the -map_metadata -1 but that removes the metadata without creating a new one and also removed it from all video and audio tracks which I don't want.
Any other ideas what I am doing wrong or is this just a quirk of the software?
Thanks in advance.
Adding -report output trimming the lines "Writing block of size..." as the file is huge:
ffmpeg started on 2019-12-04 at 14:31:25
Report written to "ffmpeg-20191204-143125.log"
Log level: 48
Command line:
"C:\\ffmpeg\\ffmpeg-20191118-d831edc-win64-static\\bin\\ffmpeg" -i input.mkv -report -map_chapters 0 -map 0:v -c:v copy -c:s copy -map 0:a -c:a:0 copy -map 0:a -c:a:1 aac destination.mkv
ffmpeg version git-2019-11-18-d831edc Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.2.1 (GCC) 20191010
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 36.100 / 56. 36.100
libavcodec 58. 62.100 / 58. 62.100
libavformat 58. 35.100 / 58. 35.100
libavdevice 58. 9.101 / 58. 9.101
libavfilter 7. 66.100 / 7. 66.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'input.mkv'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-map_chapters' ... matched as option 'map_chapters' (set chapters mapping) with argument '0'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:v'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-c:s' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:a'.
Reading option '-c:a:0' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:a'.
Reading option '-c:a:1' ... matched as option 'c' (codec name) with argument 'aac'.
Reading option 'destination.mkv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url input.mkv.
Successfully parsed a group of options.
Opening an input file: input.mkv.
[NULL # 000001b290ddad00] Opening 'input.mkv' for reading
[file # 000001b290ddbe00] Setting default whitelist 'file,crypto'
[matroska,webm # 000001b290ddad00] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm # 000001b290ddad00] Before avformat_find_stream_info() pos: 5080 bytes read:32768 seeks:0 nb_streams:2
[hevc # 000001b290ddeb80] nal_unit_type: 32(VPS), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding VPS
[hevc # 000001b290ddeb80] Main 10 profile bitstream
[hevc # 000001b290ddeb80] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding SPS
[hevc # 000001b290ddeb80] Main 10 profile bitstream
[hevc # 000001b290ddeb80] Decoding VUI
[hevc # 000001b290ddeb80] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding PPS
[hevc # 000001b290ddeb80] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding SEI
[hevc # 000001b290ddeb80] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding SEI
[hevc # 000001b290ddeb80] nal_unit_type: 39(SEI_PREFIX), nuh_layer_id: 0, temporal_id: 0
[hevc # 000001b290ddeb80] Decoding SEI
[hevc # 000001b290ddeb80] Skipped PREFIX SEI 5
[matroska,webm # 000001b290ddad00] All info found
[matroska,webm # 000001b290ddad00] After avformat_find_stream_info() pos: 10219 bytes read:32768 seeks:0 frames:4
Input #0, matroska,webm, from 'input.mkv':
Metadata:
title : input
ENCODER : Lavf58.35.100
Duration: 01:50:06.89, start: 0.000000, bitrate: 11569 kb/s
Chapter #0:0: start 0.000000, end 308.766792
Metadata:
title : Chapter 01
Chapter #0:1: start 308.766792, end 679.720708
Metadata:
title : Chapter 02
Chapter #0:2: start 679.720708, end 904.820583
Metadata:
title : Chapter 03
Chapter #0:3: start 904.820583, end 1282.781500
Metadata:
title : Chapter 04
Chapter #0:4: start 1282.781500, end 1487.736250
Metadata:
title : Chapter 05
Chapter #0:5: start 1487.736250, end 1846.177667
Metadata:
title : Chapter 06
Chapter #0:6: start 1846.177667, end 2116.531083
Metadata:
title : Chapter 07
Chapter #0:7: start 2116.531083, end 2443.858083
Metadata:
title : Chapter 08
Chapter #0:8: start 2443.858083, end 2731.979250
Metadata:
title : Chapter 09
Chapter #0:9: start 2731.979250, end 3178.383542
Metadata:
title : Chapter 10
Chapter #0:10: start 3178.383542, end 3636.507875
Metadata:
title : Chapter 11
Chapter #0:11: start 3636.507875, end 3867.488625
Metadata:
title : Chapter 12
Chapter #0:12: start 3867.488625, end 4208.037167
Metadata:
title : Chapter 13
Chapter #0:13: start 4208.037167, end 4502.372875
Metadata:
title : Chapter 14
Chapter #0:14: start 4502.372875, end 4853.431917
Metadata:
title : Chapter 15
Chapter #0:15: start 4853.431917, end 5267.470542
Metadata:
title : Chapter 16
Chapter #0:16: start 5267.470542, end 5543.329458
Metadata:
title : Chapter 17
Chapter #0:17: start 5543.329458, end 5847.132958
Metadata:
title : Chapter 18
Chapter #0:18: start 5847.132958, end 6150.269125
Metadata:
title : Chapter 19
Chapter #0:19: start 6150.269125, end 6606.891958
Metadata:
title : Chapter 20
Stream #0:0, 3, 1/1000: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x1600 [SAR 1:1 DAR 12:5], 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc
Metadata:
title : input
ENCODER : Lavc58.62.100 libx265
DURATION : 01:50:06.891000000
Stream #0:1(eng), 1, 1/1000: Audio: truehd, 48000 Hz, 7.1, s32 (24 bit) (default)
Metadata:
title : input
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
BPS-eng : 3544728
DURATION-eng : 01:50:06.893000000
NUMBER_OF_FRAMES-eng: 7928271
NUMBER_OF_BYTES-eng: 2927455224
_STATISTICS_WRITING_APP-eng: mkvmerge v40.0.0 ('Old Town Road + Pony') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-11-30 10:59:25
DURATION : 01:50:06.893000000
Successfully opened the file.
Parsing a group of options: output url destination.mkv.
Applying option map_chapters (set chapters mapping) with argument 0.
Applying option map (set input stream mapping) with argument 0:v.
Applying option c:v (codec name) with argument copy.
Applying option c:s (codec name) with argument copy.
Applying option map (set input stream mapping) with argument 0:a.
Applying option c:a:0 (codec name) with argument copy.
Applying option map (set input stream mapping) with argument 0:a.
Applying option c:a:1 (codec name) with argument aac.
Successfully parsed a group of options.
Opening an output file: destination.mkv.
[file # 000001b290ea7e40] Setting default whitelist 'file,crypto'
Successfully opened the file.
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Stream #0:1 -> #0:2 (truehd (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:2 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:2 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:2 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:2 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
detected 12 logical cores
[graph_0_in_0_1 # 000001b290dda840] Setting 'time_base' to value '1/48000'
[graph_0_in_0_1 # 000001b290dda840] Setting 'sample_rate' to value '48000'
[graph_0_in_0_1 # 000001b290dda840] Setting 'sample_fmt' to value 's32'
[graph_0_in_0_1 # 000001b290dda840] Setting 'channel_layout' to value '0x63f'
[graph_0_in_0_1 # 000001b290dda840] tb:1/48000 samplefmt:s32 samplerate:48000 chlayout:0x63f
[format_out_0_2 # 000001b290e0ea00] Setting 'sample_fmts' to value 'fltp'
[format_out_0_2 # 000001b290e0ea00] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[format_out_0_2 # 000001b290e0ea00] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_2'
[AVFilterGraph # 000001b290e1b3c0] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[auto_resampler_0 # 000001b290e0fe80] [SWR # 000001b291232000] Using fltp internally between filters
[auto_resampler_0 # 000001b290e0fe80] ch:8 chl:7.1 fmt:s32 r:48000Hz -> ch:8 chl:7.1 fmt:fltp r:48000Hz
[matroska # 000001b290dfba40] get_metadata_duration returned: 6606893000
[matroska # 000001b290dfba40] Write early duration from metadata = 6606893
Output #0, matroska, to 'destination.mkv':
Metadata:
title : input
encoder : Lavf58.35.100
Chapter #0:0: start 0.000000, end 308.766792
Metadata:
title : Chapter 01
Chapter #0:1: start 308.766792, end 679.720708
Metadata:
title : Chapter 02
Chapter #0:2: start 679.720708, end 904.820583
Metadata:
title : Chapter 03
Chapter #0:3: start 904.820583, end 1282.781500
Metadata:
title : Chapter 04
Chapter #0:4: start 1282.781500, end 1487.736250
Metadata:
title : Chapter 05
Chapter #0:5: start 1487.736250, end 1846.177667
Metadata:
title : Chapter 06
Chapter #0:6: start 1846.177667, end 2116.531083
Metadata:
title : Chapter 07
Chapter #0:7: start 2116.531083, end 2443.858083
Metadata:
title : Chapter 08
Chapter #0:8: start 2443.858083, end 2731.979250
Metadata:
title : Chapter 09
Chapter #0:9: start 2731.979250, end 3178.383542
Metadata:
title : Chapter 10
Chapter #0:10: start 3178.383542, end 3636.507875
Metadata:
title : Chapter 11
Chapter #0:11: start 3636.507875, end 3867.488625
Metadata:
title : Chapter 12
Chapter #0:12: start 3867.488625, end 4208.037167
Metadata:
title : Chapter 13
Chapter #0:13: start 4208.037167, end 4502.372875
Metadata:
title : Chapter 14
Chapter #0:14: start 4502.372875, end 4853.431917
Metadata:
title : Chapter 15
Chapter #0:15: start 4853.431917, end 5267.470542
Metadata:
title : Chapter 16
Chapter #0:16: start 5267.470542, end 5543.329458
Metadata:
title : Chapter 17
Chapter #0:17: start 5543.329458, end 5847.132958
Metadata:
title : Chapter 18
Chapter #0:18: start 5847.132958, end 6150.269125
Metadata:
title : Chapter 19
Chapter #0:19: start 6150.269125, end 6606.891958
Metadata:
title : Chapter 20
Stream #0:0, 0, 1/1000: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x1600 [SAR 1:1 DAR 12:5], q=2-31, 23.98 fps, 23.98 tbr, 1k tbn, 1k tbc
Metadata:
title : input
ENCODER : Lavc58.62.100 libx265
DURATION : 01:50:06.891000000
Stream #0:1(eng), 0, 1/1000: Audio: truehd ([255][255][255][255] / 0xFFFFFFFF), 48000 Hz, 7.1, s32 (24 bit) (default)
Metadata:
title : input
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
BPS-eng : 3544728
DURATION-eng : 01:50:06.893000000
NUMBER_OF_FRAMES-eng: 7928271
NUMBER_OF_BYTES-eng: 2927455224
_STATISTICS_WRITING_APP-eng: mkvmerge v40.0.0 ('Old Town Road + Pony') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-11-30 10:59:25
DURATION : 01:50:06.893000000
Stream #0:2(eng), 0, 1/1000: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, 7.1, fltp (24 bit), 469 kb/s (default)
Metadata:
title : input
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
BPS-eng : 3544728
DURATION-eng : 01:50:06.893000000
NUMBER_OF_FRAMES-eng: 7928271
NUMBER_OF_BYTES-eng: 2927455224
_STATISTICS_WRITING_APP-eng: mkvmerge v40.0.0 ('Old Town Road + Pony') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-11-30 10:59:25
DURATION : 01:50:06.893000000
encoder : Lavc58.62.100 aac
cur_dts is invalid st:2 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
....................
....................
[matroska # 000001b290dfba40] Writing block of size 168 with pts 6606913, dts 6606913, duration 1 at relative offset 550755 in cluster at offset 9940898622. TrackNumber 2, keyframe 1
[matroska # 000001b290dfba40] end duration = 6606914
[matroska # 000001b290dfba40] stream 0 end duration = 6606912
[matroska # 000001b290dfba40] stream 1 end duration = 6606914
[matroska # 000001b290dfba40] stream 2 end duration = 6606914
frame=158407 fps=357 q=-1.0 Lsize= 9708475kB time=01:50:06.91 bitrate=12037.7kbits/s speed=14.9x
video:6416483kB audio:3234397kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 0.596787%
Input file #0 (input.mkv):
Input stream #0:0 (video): 158407 packets read (6570478730 bytes);
Input stream #0:1 (audio): 7928271 packets read (2927455224 bytes); 7928271 frames decoded (317130840 samples);
Total: 8086678 packets (9497933954 bytes) demuxed
Output file #0 (destination.mkv):
Output stream #0:0 (video): 158407 packets muxed (6570478730 bytes);
Output stream #0:1 (audio): 7928271 packets muxed (2927455224 bytes);
Output stream #0:2 (audio): 309699 frames encoded (317130840 samples); 309700 packets muxed (384567176 bytes);
Total: 8396378 packets (9882501130 bytes) muxed
7928271 frames successfully decoded, 0 decoding errors
[AVIOContext # 000001b291230100] Statistics: 8 seeks, 39676 writeouts
[aac # 000001b290dff3c0] Qavg: 3839.668
[AVIOContext # 000001b290de4080] Statistics: 9554742230 bytes read, 0 seeks
You'll need to do it in two steps.
ffmpeg -i source.mkv -map 0:a -c:a aac -map_metadata -1 temp.mkv
ffmpeg -i source.mkv -i temp.mkv -map_chapters 0 -map 0:v -map 0:a -map 1:a -c copy destination.mkv
The reason ffmpeg doesn't give metadata 'bitrate' for the 2nd track is you didn't specifically map the metadata from the added aac audio track. And it was copied, if it had been re-encoded. The bitrate would be shown. Here is it is transcoded in one pass. Notice I set the aac bitrate to 320kb/s. Far less then the TrueHD above 3545 kb/s. 320kb/s is will sound good & your file size will be smaller.
ffmpeg -i source.mkv -map_chapters 0 -map 0:v -map 0:a:0 -map 0:a:0 -map 0:s -c:v copy -c:a:0 copy -c:a:1 aac -b:a:1 320k -c:s copy destination.mkv
Am running my ALSA Driver on Ubuntu 14.04, 64bit, 3.16.0-30-generic Kernel.
Hardware is proprietary hardware, hence cant give much details.
Following is the existing driver implementation:
Driver is provided sample format, sample rate, channel_count as input via module parameter. (Due to requirements need to provide inputs via module parameters)
Initial snd_pcm_hardware structure for playback path.
#define DEFAULT_PERIOD_SIZE (4096)
#define DEFAULT_NO_OF_PERIODS (1024)
static struct snd_pcm_hardware xxx_playback =
{
.info = SNDRV_PCM_INFO_MMAP |
SNDRV_PCM_INFO_INTERLEAVED |
SNDRV_PCM_INFO_MMAP_VALID |
SNDRV_PCM_INFO_SYNC_START,
.formats = SNDRV_PCM_FMTBIT_S16_LE,
.rates = (SNDRV_PCM_RATE_8000 | \
SNDRV_PCM_RATE_16000 | \
SNDRV_PCM_RATE_48000 | \
SNDRV_PCM_RATE_96000),
.rate_min = 8000,
.rate_max = 96000,
.channels_min = 1,
.channels_max = 1,
.buffer_bytes_max = (DEFAULT_PERIOD_SIZE * DEFAULT_NO_OF_PERIODS),
.period_bytes_min = DEFAULT_PERIOD_SIZE,
.period_bytes_max = DEFAULT_PERIOD_SIZE,
.periods_min = DEFAULT_NO_OF_PERIODS,
.periods_max = DEFAULT_NO_OF_PERIODS,
};
Similar values for captures side snd_pcm_hardware structure.
Please, note that the following below values are replaced in playback open entry point, based on the current audio test configuration:
(user provides audio format, audio rate, ch count via module parameters as inputs to the driver, which are refilled in snd_pcm_hardware structure)
xxx_playback.formats = user_format_input
xxx_playback.rates = xxx_playback.rate_min, xxx_playback.rate_max = user_sample_rate_input
xxx_playback.channels_min = xxx_playback.channels_max = user_channel_input
Similarly values are re-filled for capture snd_pcm_hardware structure in capture open entry point.
Hardware is configured for clocks based on channel_count, format, sample_rate and driver registers successfully with ALSA layer
Found aplay/arecord working fine for channel_count = 1 or 2 or 4
During aplay/arecord, in driver when "runtime->channels" value is checked, it reflects the channel_count configured, which sounds correct to me.
Record data matches with played, since its a loop back test.
But when i use channel_count = 3, Both aplay or arecord reports
"Broken configuration for this PCM: no configurations available"!! for a wave file with channel_count '3'
ex: Playing WAVE './xxx.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Channels 3
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
aplay: set_params:1204: Broken configuration for this PCM: no configurations available
With Following changes I was able to move ahead a bit:
.........................
Method1:
Driver is provided channel_count '3' as input via module parameter
Modified Driver to fill snd_pcm_hardware structure as payback->channels_min = 2 & playback->channels_min = 3; Similar values for capture path
aplay/arecord reports as 'channel count not available', though the wave file in use has 3 channels
ex: aplay -D hw:CARD=xxx,DEV=0 ./xxx.wav Playing WAVE './xxx.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Channels 3
aplay: set_params:1239: Channels count non available
Tried aplay/arecord with plughw, and aplay/arecord moved ahead
arecord -D plughw:CARD=xxx,DEV=0 -d 3 -f S16_LE -r 48000 -c 3 ./xxx_rec0.wav
aplay -D plughw:CARD=xxx,DEV=0 ./xxx.wav
Recording WAVE './xxx_rec0.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Channels 3
Playing WAVE './xxx.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Channels 3
End of Test
During aplay/arecord, In driver when "runtime->channels" value is checked it returns value 2!!! But played wavefile has ch count 3...
When data in recorded file is checked its all silence
.........................
Method2:
Driver is provided channel_count '3' as input via module parameter
Modified Driver to fill snd_pcm_hardware structure as playback->channels_min = 3 & playback->channels_min = 4; Similar values for capture path
aplay/arecord reports as 'channel count not available', though the wave file in use has 3 channels
Tried aplay/arecord with plughw, and aplay/arecord moved ahead
During aplay/arecord, In driver when "runtime->channels" value is checked it returns value 4!!! But played wavefile has ch count 3...
When data in recorded file is checked its all silence
.........................
So from above observations, the runtime->channels is either 2 or 4, but never 3 channels was used by alsa stack though requested. When used Plughw, alsa is converting data to run under 2 or 4 channel.
Can anyone help why am unable to use channel count 3.
Will provide more information if needed.
Thanks in Advance.
A period (and the entire buffer) must contain an integral number of frames, i.e., you cannot have partial frames.
With three channels, one frame has six bytes. The fixed period size (4096) is not divisible by six without remainder.
Thanks CL.
I used period size 4092 for this particular test case with channel count 3, and was able to do loop back successfully (without using plughw).
One last question, when I used plughw earlier, and when runtime->channels was either 2 or 4, why was the recorded data not showing?
My hw:1,0 is a playback-only external USB card. The recording application cannot capture the samples being sent to hw:1,0 if it tries opening hw:1,0 for capture. The alsamixer doesn't give any "[Capture]" options for hw:1,0 - just one volume control on "[Playback]" tab.
Is it possible to create some virtual "xxx" device that would accept a stream from playing application and send it out via 2 streams (copies) - first one to real "hw:1,0" (to my headphones) and the second one to another virtual device "zzz" that would provide capture ability, feeding the stream to applications that could open "zzz" for capturing?
update
I added ALSA Loopback device in kernel, now i have loopback device. Then i found working .asoundrc (see below), but when i capture screen + sound using ffmpeg, the sound goes WAY before than video, some great sync problem. So with this .asoundrc (below) i can use hw:0,0 (loopback device, hw:0,0 - play, hw:0,1 - capture) to capture what i send to "default". Also i can hear these samples via my external USB hw:2,0.
ffmpeg command:
ffmpeg -f alsa -i hw:0,1 -f x11grab -s 854x480 -r 30 -i :0.0 -vcodec libx264 -vb 4000k /tmp/screen_854x480.avi
.asoundrc:
pcm.!default {
type plug
slave.pcm "mdev"
route_policy "duplicate"
}
pcm.mdev {
type multi
slaves.a.pcm "hw:Loopback,0,0"
slaves.a.channels 2
slaves.b.pcm "dmixer"
slaves.b.channels 2
bindings.0.slave a
bindings.0.channel 0
bindings.1.slave a
bindings.1.channel 1
bindings.2.slave b
bindings.2.channel 0
bindings.3.slave b
bindings.3.channel 1
}
pcm.dmixer {
type dmix
ipc_key 1024
slave {
pcm "hw:2,0"
period_time 0
period_size 1024
buffer_size 4096
rate 44100
channels 2
format S16_LE
}
}
It is possible to create loopback devices that reroute playback data back to a capture device:
Alsa Loopback Devices for Fun, but not yet profit...
ALSA Loopback for Fun and Profit
It is also possible to create a virtual device that outputs to multiple devices with the multi plugin.
Try something like this:
pcm.xxx {
type plug # <-- no { here
slave.pcm {
type multi
slaves {
a { channels 2 pcm "hw:1,0" } # the real device
b { channels 2 pcm "hw:5,0" } # the loopback driver
}
bindings {
0 { slave a channel 0 }
1 { slave a channel 1 }
2 { slave b channel 0 }
3 { slave b channel 1 }
}
}
ttable [
[ 1 0 1 0 ] # left -> a.left, b.left
[ 0 1 0 1 ] # right -> a.right, b.right
]
}