With the help of the AudioRecord API I am now able to record audio using the microphone of the watch. However, this is uncompressed audio and the file sizes get pretty big.
Around 5.3MB per minute recording at 16bit, 44.1Khz, mono. So I started looking into reducing the size by encoding it to a different format.
I looked into MediaCodec and the other low level audio multimedia support that is mentioned.
Looping over the available codecs:
MediaCodecList codecs = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
MediaCodecInfo[] infos = codecs.getCodecInfos();
for(MediaCodecInfo info : infos) {
Log.d(TAG, "Name: " + info.getName());
Log.d(TAG, "Types: " + Arrays.toString(info.getSupportedTypes()));
Log.d(TAG, "Encoder? " + info.isEncoder());
}
It turns out that none of them are encoders. Just 9 decoders. Attempting to use any of these in MediaCodec.createByCodecName() results in an exception. Any idea how I can compress my recorded audio on Android Wear?
Related
I am trying to build a web-application with the functionality of screen-recording with system audio + headphone-mic audio being captured in the saved video.
I have been thoroughly googling on a solution for this, however my findings show multiple browser solutions where the above works so long as headphones are NOT connected, meaning the microphone input is coming from the system rather than headset.
In the case that you connect headphones, all of these solutions capture the screen without video-audio, and the microphone audio from my headset. So to re-clarify on this, it should have recorded video-audio from the video being played whilst recording, and the headset-mic audio also.
This is thoroughly available in native applications, however I am searching for a way to do this on a browser.
If there are no solutions for this currently that anybody knows of, some insight on the limitations around developing this would also really help, thank you.
Your browser manages the media input being received in the selected tab/window
To receive media input, you need to ensure you have the checkbox Share Audio in the image below checked. However this will only record media-audio being played in your headphones, when it comes to receiving microphone audio, the opposite must be done i.e the checkbox should be unchecked, or merge the microphone audio separately on saving the recorded video
https://slack-files.com/T1JA07M6W-F0297CM7F32-89e7407216
create two const, one retrieving on-screen video, other retrieving audio media:
const DISPLAY_STREAM = await navigator.mediaDevices.getDisplayMedia({video: {cursor: "motion"}, audio: {'echoCancellation': true}}); // retrieving screen-media
const VOICE_STREAM = await navigator.mediaDevices.getUserMedia({ audio: {'echoCancellation': true}, video: false }); // retrieving microphone-media
Use AudioContext to retrieve audio sources from getUserMedia() and getDisplayMedia() separately:
const AUDIO_CONTEXT = new AudioContext();
const MEDIA_AUDIO = AUDIO_CONTEXT.createMediaStreamSource(DISPLAY_STREAM); // passing source of on-screen audio
const MIC_AUDIO = AUDIO_CONTEXT.createMediaStreamSource(VOICE_STREAM); // passing source of microphone audio
Use the method below to create a new audio source which will be used as as the merger or merged version of audio, then passing audios into the merger:
const AUDIO_MERGER = AUDIO_CONTEXT.createMediaStreamDestination(); // audio merger
MEDIA_AUDIO.connect(AUDIO_MERGER); // passing media-audio to merger
MIC_AUDIO.connect(AUDIO_MERGER); // passing microphone-audio to merger
Finally, connect the merged-audio and video together into one array to form a track, and pass it to the MediaStreamer:
const TRACKS = [...DISPLAY_STREAM.getVideoTracks(), ...AUDIO_MERGER.stream.getTracks()] // connecting on-screen video with merged-audio
stream = new MediaStream(TRACKS);
I am following the samples for Microsoft Cognitive Services Speech SDK, namely the Speech Translation.
The sample for dotnet core uses microphone as audio input and translates what you speak. Translated results are also available as synthesized speech. I would like to play this audio but could not find the appropriate code for that.
Tried using NAudio as sugguested in this answer but I get garbled audio. Guess there is more to the format of the audio.
Any pointers?
On .Net Core, many audio pacakges might not work. For example with NAudio, I can't play sound on my Mac.
I got it working using NetCoreAudio package (Nuget), with the following implementation in the translation Synthesizing event:
recognizer.Synthesizing += (s, e) =>
{
var audio = e.Result.GetAudio();
Console.WriteLine(audio.Length != 0
? $"AudioSize: {audio.Length}"
: $"AudioSize: {audio.Length} (end of synthesis data)");
if (audio.Length > 0)
{
var fileName = Path.Combine(Directory.GetCurrentDirectory(), $"{DateTime.Now.ToString("yyyy-MM-dd_HH-mm-ss.wav")}");
File.WriteAllBytes(fileName, audio);
var player = new Player();
player.Play(fileName).Wait();
}
};
I receive over network PCM audio data stream and this part works fine so I am ending up with
DataReader incomming = args.GetDataReader();
byte[] RcvBuffer = new byte[incomming.UnconsumedBufferLength];
incomming.ReadBytes(RcvBuffer);
I have all audio data in buffer.
How I can play this through telephone Speaker ? Can you point me in some direction ?
Thanks
There're many ways to do that.
You can prepend the WAVE header to your data, and use MediaElement for playback, see the documentation for SetSource method.
If however by “telephone speaker” you mean the earphone, then it is only possible if you are creating a VoIP app.
It took a while but I sorted it, maybe someone else will need help in the future.
First Problem - since I just started app development for Windows Phone I have chosen Blank App (Windows Phone) instead Blank App (Windows Phone Silverlight) and I did not have access to many features that are available in Silverlight projects, so my suggestions for beginners: understand what each project is for.
Like Soonts said there are many ways to do this, this is one that I used.
I simplified this code and retyped this so there can be some typos.
using Microsoft.Xna.Framework.Audio;
using System.IO;
1) Create Stream to load your incoming data:
MemoryStream stream = new MemoryStream();
2) Load data from buffer to stream:
stream.Write(RcvBuffer, 0, RcvBuffer.Length);
3) I am using SoundEfect to play this through Loud-Speaker. Sample rate that I use is 8 kHz
SoundEffect sound;
sound = new SoundEffect(stream.toArray(), 8000, AudioChannels.Mono)
sound.Play();
I am trying to run the example code of Media Codec API with H264 Encoder on 4.3 explained in following link of bigflake
http://bigflake.com/mediacodec/CameraToMpegTest.java.txt
I have faced following problem.
-> In H264 encoder code the color format,height and width are not getting updated because there is problem in getpatameter implemetation. So i applied this patch (https://code.google.com/p/android/issues/detail?id=58834).
-> After applying the patch,also encoder does not encode
-> I have seen the observation like
D/CameraToMpegTest( 3421): encoder output format changed: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=144, mime=video/avc, csd-0=java.nio.ByteArrayBuffer[position=0,limit=12,capacity=12], what=1869968451, width=176}
SO why this value is getting changed, No idea...
After that we always see encoder gives status of queueOutputBuffer as INFO_TRY_AGAIN_LATER.
So it creates the file but it does not encode anything and it stops as
I/MPEG4Writer( 3421): Received total/0-length (0/0) buffers and encoded 0 frames. - video
D/MPEG4Writer( 3421): Stopping Video track
D/MPEG4Writer( 3421): Stopping Video track source
D/MPEG4Writer( 3421): Video track stopped
D/MPEG4Writer( 3421): Stopping writer thread
D/MPEG4Writer( 3421): 0 chunks are written in the last batch
D/MPEG4Writer( 3421): Writer thread stopped
So in my understanding it should work but looks like still encoder is not getting configured properly...
Please guide on this...
Thanks
Nehal
The "encoder output format changed" message is normal in Android 4.3. That's how the encoder gives you a MediaFormat with csd-0/csd-1 keys, needed by MediaMuxer#addTrack().
Bug 58834 is for the VP8 software encoder; those patches shouldn't be needed for the hardware AVC codec.
The most common reason for INFO_TRY_AGAIN_LATER is lack of input. The encoder may queue up a fair number of input frames before producing any output, so you can't just submit one frame and then wait for output to appear. Turn on the VERBOSE flag and make sure that frames are being submitted.
I have tried running CameraToMpegTest sample on Android 4.3 emulator. As you'd have realized by now, it's not going to work as it is, and some fixes are required.
Implement getparameter properly in SoftAVCEncoder (in case of MIME type - "video/avc") for parameters like width, height, colour format. Otherwise your MediaFormat is not configured properly, and createInputSurface would fail. (I am not sure why this doesn't cause any problem when running H.264 encoding using Mediarecorder)
Fix the EGL attributes
Most importantly, if you're trying to execute this code in Activity context, make sure you don't block onFrameAvailable callback
(final void join()
Blocks the current Thread (Thread.currentThread()) until the receiver finishes its execution and dies.)
As the code snippet, you should remove th.join();
/** Entry point. */
public static void runTest(CameraToMpegTest obj) throws Throwable {
CameraToMpegWrapper wrapper = new CameraToMpegWrapper(obj);
Thread th = new Thread(wrapper, "codec test");
th.start();
// th.join();
if (wrapper.mThrowable != null) {
throw wrapper.mThrowable;
}
}
It works well for me.
Does anyone know of a good repository to get sample code for the BlackBerry? Specifically, samples that will help me learn the mechanics of recording audio, possibly even sampling it and doing some on the fly signal processing on it?
I'd like to read incoming audio, sample by sample if need be, then process it to produce a desired result, in this case a visualizer.
RIM API contains JSR 135 Java Mobile Media API for handling audio & video content.
You correct about mess on BB Knowledge Base. The only way is browse it, hoping they'll not going to change site map again.
It's Developers->Resources->Knowledge Base->Java API's&Samples->Audio&Video
Audio Recording
Basically it's simple to record audio:
create Player with correct audio encoding
get RecordControl
start recording
stop recording
Links:
RIM 4.6.0 API ref: Package javax.microedition.media
How To - Record Audio on a BlackBerry smartphone
How To - Play audio in an application
How To - Support streaming audio to the media application
How To - Specify Audio Path Routing
How To - Obtain the media playback time from a media application
What Is - Supported audio formats
What Is - Media application error codes
Audio Record Sample
Thread with Player, RecordControl and resources is declared:
final class VoiceNotesRecorderThread extends Thread{
private Player _player;
private RecordControl _rcontrol;
private ByteArrayOutputStream _output;
private byte _data[];
VoiceNotesRecorderThread() {}
private int getSize(){
return (_output != null ? _output.size() : 0);
}
private byte[] getVoiceNote(){
return _data;
}
}
On Thread.run() audio recording is started:
public void run() {
try {
// Create a Player that captures live audio.
_player = Manager.createPlayer("capture://audio");
_player.realize();
// Get the RecordControl, set the record stream,
_rcontrol = (RecordControl)_player.getControl("RecordControl");
//Create a ByteArrayOutputStream to capture the audio stream.
_output = new ByteArrayOutputStream();
_rcontrol.setRecordStream(_output);
_rcontrol.startRecord();
_player.start();
} catch (final Exception e) {
UiApplication.getUiApplication().invokeAndWait(new Runnable() {
public void run() {
Dialog.inform(e.toString());
}
});
}
}
And on thread.stop() recording is stopped:
public void stop() {
try {
//Stop recording, capture data from the OutputStream,
//close the OutputStream and player.
_rcontrol.commit();
_data = _output.toByteArray();
_output.close();
_player.close();
} catch (Exception e) {
synchronized (UiApplication.getEventLock()) {
Dialog.inform(e.toString());
}
}
}
Processing and sampling audio stream
In the end of recording you will have output stream filled with data in specific audio format. So to process or sample it you will have to decode this audio stream.
Talking about on the fly processing, that will be more complex. You will have to read output stream during recording without record commiting. So there will be several problems to solve:
synch access to output stream for Recorder and Sampler - threading issue
read the correct amount of audio data - go deep into audio format decode to find out markup rules
Also may be useful:
java.net: Experiments in Streaming Content in Java ME by Vikram Goyal
While not audio specific, this question does have some good "getting started" references.
Writing Blackberry Applications
I spent ages trying to figure this out too. Once you've installed the BlackBerry Component Packs (available from their website), you can find the sample code inside the component pack.
In my case, once I had installed the Component Packs into Eclipse, I found the extracted sample code in this location:
C:\Program
Files\Eclipse\eclipse3.4\plugins\net.rim.eide.componentpack4.5.0_4.5.0.16\components\samples
Unfortunately when I imported all that sample code I had a bunch of compile errors. To workaround that I just deleted the 20% of packages with compile errors.
My next problem was that launching the Simulator always launched the first sample code package (in my case activetextfieldsdemo), I couldn't get it to run just the package I am interested in. Workaround for that was to delete all the packages listed alphabetically before the one I wanted.
Other gotchas:
-Right click on the project in Eclipse and select Activate for BlackBerry
-Choose BlackBerry -> Build Configurations... -> Edit... and select your new project so it builds.
-Make sure you put your BlackBerry source code under a "src" folder in the Eclipse project, otherwise you might hit build issues.