MediaDevices.getUserMedia() How can I set audio constraints (sampling rate/bit depth)? - audio

Question

With browser Web API, I'd like to set MediaDevices.getUserMedia constraints attributes, suitable to record audio speech (voice messages), e.g. setting these parameters:
mono
16bit
16KHz
Here my code:
const mediaStreamConstraints = {
audio: {
channelCount: 1,
sampleRate: 16000,
sampleSize: 16,
volume: 1
},
video: false
}
navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
.catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )
.then( stream => {
// audio recorded as Blob
// and the binary data are sent via socketio to a nodejs server
// that store blob as a file (e.g. audio/inp/audiofile.webm)
} )
The recorded clip is grabbed and stored (using MediaRecorder API), eventually sent to a nodejs server where the blob is saved as a file and processed (the application is a voicebot).
Something goes wrong and the WebM saved file hasn't the required parameters:
$ mediainfo audio/inp/audiofile.webm
General
Complete name : audio/inp/audiofile.webm
Format : WebM
Format version : Version 4 / Version 2
File size : 2.04 KiB
Writing application : Chrome
Writing library : Chrome
IsTruncated : Yes
Audio
ID : 1
Format : Opus
Codec ID : A_OPUS
Channel(s) : 1 channel
Channel positions : Front: C
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Compression mode : Lossy
Language : English
Default : Yes
Forced : No
E.g.
Sampling rate : 48.0 kHz
Bit depth : 32 bits
But constraints would imply different values:
Sampling rate : 16 kHz
Bit depth : 16 bits
Also the blob, played with anew Audio(audioUrl(blob)).play(), doesn't play. Weird. But all works if constraints are just:
const mediaStreamConstraints = { audio: true }
I checked the browser console and I didn't see any error of navigator.mediaDevices.getUserMedia(mediaStreamConstraints) API call.
BTW, I followed guidelines here:
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia
https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints
Note that my user agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 (I'm using a last Brave browser version).
Seems to me that any audio constraints settings not allowed by the browser:
brokes the audio blob,
without raising an error exception (I catched both navigator.mediaDevices.getUserMedia() and new MediaRecorder(...). Isn't this last at least a bug?
My question is:
There is any way to have sampling rate/bit depth as requested?
Or the audio format is "hardcoded"/decided by browser implementation?
BTW, The reason of audio parameters formats is because I want to minimize audio blob size to minimize bandwidth in websocket communications between the browser client and the server, optimizing audio blob exchanges for speech (voice messages)

Answer 1

Check the capabilities of your browser first:
let stream = await navigator.mediaDevices.getUserMedia({audio: true});
let track = stream.getAudioTracks()[0];
console.log(track.getCapabilities());
demo output:
autoGainControl: (2) [true, false]
channelCount: {max: 2, min: 1}
deviceId: "default"
echoCancellation: (2) [true, false]
groupId: "1e76386ad54f9ad3548f6f6c14c08e7eff6753f9362d93d8620cc48f546604f5"
latency: {max: 0.01, min: 0.01}
noiseSuppression: (2) [true, false]
sampleRate: {max: 48000, min: 48000}
sampleSize: {max: 16, min: 16}

Answer 2

Try setting your audio constraints on the audio media track within your stream before you instantiate MediaRecorder.
Something like this, not debugged:
const constraints = {
audio: {
channelCount: 1,
sampleRate: 16000,
sampleSize: 16,
volume: 1
},
navigator.mediaDevices.getUserMedia({audio:true})
.catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )
.then( stream => {
const audioTracks = stream.getAudioTracks()
if (audioTracks.length !== 1) throw new Error ('too many tracks???')
const audioTrack = audioTracks[0]
audioTrack.applyConstraints (constraints)
.then (()=> {
const mediaRecorder = new MediaRecorder(stream)
/* etc etc etc */
} )
.catch(console.error) /* you might get constraint failure here. */
} )
All that being said, the Opus audio codec does a good job compressing voice to a reasonable size. Just because it sez 48kHz x 32bits doesn't mean it uses that much bandwidth; the audio signal is compressed.
And, try it on the most recent releases of Google Chrome and/or Firefox. This media stuff is in active development.

Answer 3

The answer by O.Jones is on the right track but can be more concise by setting the constraint directly when grabbing the userMedia object.
const constraints = {
audio: {
channelCount: 1,
sampleRate: 16000,
sampleSize: 16,
volume: 1
}
}
navigator.mediaDevices.getUserMedia({audio: constraints})
.then( stream => {
// Do something with the stream...
})
.catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )

Answer 4

As mentioned in the comments, downsampling from getUserMedia with the exact keyword in the constraints object fails with an OverconstrainedError:
navigator.mediaDevices.getUserMedia({
audio: {
sampleRate: { exact: 16000 }
},
video: false
})
.then(stream => {
// Do something with the stream...
})
.catch(error => { console.log('Error :', error) })
And without the exact keyword, the obtained stream doesn't fulfill the request:
navigator.mediaDevices.getUserMedia({
audio: {
sampleRate: 16000
},
video: false
})
.then(stream => {
console.log('Sample rate :', stream.getAudioTracks()[0].getSettings().sampleRate)
// Shows 'Sample rate : 48000'
})
.catch(error => { console.log('Error :', error) })
You may want to try to downsample the audio stream from the microphone using the WebAudio API and create a new MediaStream to record using the AudioContext.createMediaStreamDestination, documented here.
Here is a code snippet for Chrome and Safari (Firefox throws an exception, see here):
const audioContext = new (window.AudioContext || window.webkitAudioContext)({
sampleRate: 16000
});
const mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false })
const mediaStreamSource = audioContext.createMediaStreamSource(mediaStream)
const mediaStreamDestination = audioContext.createMediaStreamDestination()
mediaStreamDestination.channelCount = 1
mediaStreamSource.connect(mediaStreamDestination)
const mediaRecorder = new MediaRecorder(mediaStreamDestination.stream)