I'm trying to play some mp3 files in node.js. The thing is that I manage to play them one by one, or even, as I want in parallel. But what I also want is to be able to control the amplitude (gain) to be able to create a crossfade in the end. Could anyone help me understand what it is I need to do? (I want to use it in node-webkit so I need a solution that is node.js based with no external dependencies.)
This is what I've got so far:
var lame = require('lame'), Speaker = require('speaker'), fs = require('fs');
var audioOptions = {channels: 2, bitDepth: 16, sampleRate: 44100};
var decoder = lame.Decoder();
var stream = fs.createReadStream("music/ge.mp3", audioOptions).pipe(decoder).on("format", function (format) {
this.pipe(new Speaker(format))
}).on("data", function (data) {
console.log(data)
})
I customized the npm package pcm-volume to do that. To crossfade, provide two pcm audio buffers (output of your decoders). Pipe the result to your Speaker object.
Here is the main part of the modifications. In this case the crossfade happens at the scale of the provided buffer, but you can change that.
var l = buf.length;
var out = new Buffer(l);
for (var i=0; i < l; i+=2) {
volumeSunrise = 0.5*this.volume*(1-Math.cos(pi*i/l));
volumeSunset = 0.5*this.volume*(1+Math.cos(pi*i/l));
uint = Math.round(volumeSunrise*buf.readInt16LE(i) + volumeSunset*this.sunsetBuffer.readInt16LE(i));
// you may want to ensure that -32767 <= uint <= 32768 here, in case you use a volume higher than 1
out.writeInt16LE(uint, i);
}
this.push(out);
callback()
Related
i wanted to generate some images in Node.JS, compile them to a video and stream them to youtube. To generate the images i'm using the node-canvas module. This sounds simple enough, but i wanted to generate the images continuously, and stream the result in realtime. I'm very new to this whole thing, and what i was thinking about doing, after reading a bunch of resources on the internet was:
Open ffmpeg with spawn('ffmpeg', ...args), setting the output to the destination rtmp server
Generate the image in the canvas
Convert the content of the canvas to a buffer, and write it to the ffmpeg process through stdin
Enjoy the result on Youtube
But it's not as simple as that, is it? I saw people sharing their code involving client-side JS running on the browser, but i wanted it to be a Node app so that i could run it from a remote VPS.
Is there a way for me to do this without using something like p5 in my browser and capturing the window to restream it?
Is my thought process even remotely adequate? For now i don't really care about performance/resources usage. Thanks in advance.
EDIT:
I worked on it for a bit, and i couldn't get it to work...
This is my code:
const { spawn } = require('child_process');
const { createCanvas } = require('canvas');
const fs = require('fs');
const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');
const ffmpeg = spawn("ffmpeg",
["-re", "-f", "png_pipe", "-vcodec", "png", "-i", "pipe:0", "-vcodec", "h264", "-re", "-f", "flv", "rtmp://a.rtmp.youtube.com/live2/key-i-think"],
{ stdio: 'pipe' })
const randomColor = (depth) => Math.floor(Math.random() * depth)
const random = (min, max) => (Math.random() * (max - min)) + min;
let i = 0;
let drawSomething = function () {
ctx.strokeStyle = `rgb(${randomColor(255)}, ${randomColor(255)}, ${randomColor(255)})`
let x1 = random(0, canvas.width);
let x2 = random(0, canvas.width);
let y1 = random(0, canvas.height);
let y2 = random(0, canvas.height);
ctx.moveTo(x1, y1);
ctx.lineTo(x2, y2);
ctx.stroke();
let out = canvas.toBuffer();
ffmpeg.stdin.write(out)
i++;
if (i >= 30) {
ffmpeg.stdin.end();
clearInterval(int)
};
}
drawSomething();
let int = setInterval(drawSomething, 1000);
I'm not getting any errors, neither i am getting any video data from it. I have set up an rtmp server that i can connect to, and then get the stream with VLC, but i don't get any video data. Am i doing something wrong? I Looked around for a while, and i can't seem to find anyone that tried this, so i don't really have a clue...
EDIT 2:
Apparently i was on the right track, but my approach just gave me like 2 seconds of "good" video and then it started becoming blocky and messy. i think that, most likely, my method of generating images is just too slow. I'll try to use some GPU accelerated code to generate the images, instead of using the canvas, which means i'll be doing fractals all the time, since i don't know how to do anything else with that. Also, a bigger buffer in ffmpeg might help too
Your approach is fine. Just keep in mind that whatever you're streaming to is going to expect the source end to keep up. If you can't generate frames fast enough, all sorts of things are going to go wrong. (I think this is mostly due to YouTube trying to drop the source that's too slow, and pick up again once it receives frames, only to fail again.)
You can test by outputting to a file first before trying to stream to an RTMP server.
I'm trying to separate the 4 channel that are in a Buffer i receive from a 4-mic Array ReSpeaker. I'm using nodejs and currently i use a spawn command like:
spawn('arecord -r16000 -fS16_LE -traw -c4 -Dac108')
and then pipe the output in a transformer where i split the Buffer in the 4 channels and save them into separate file for check the result
const stream = require("stream");
const fs = require('fs');
class ChannelTransformer extends stream.Transform {
constructor(options) {
var write_1 = fs.createWriteStream('ch1', {encoding: 'binary'});
var write_2 = fs.createWriteStream('ch2', {encoding: 'binary'});
var write_3 = fs.createWriteStream('ch3', {encoding: 'binary'});
var write_4 = fs.createWriteStream('ch4', {encoding: 'binary'});
options.readableObjectMode = true;
options.writableObjectMode = true;
options.highWaterMark = 20000;
options.transform = (chunk, encoding, callback) => {
let channels = [[],[],[],[]];
for(let i=0; i<source.length;i++ ){
channels[i%4].push(chunk[i])
}
write_1.write(new Uint8Array(channels[0]));
write_2.write(new Uint8Array(channels[1]));
write_3.write(new Uint8Array(channels[2]));
write_4.write(new Uint8Array(channels[3]));
callback();
};
super(options);
}
}
As result from this code i get 4 file and if I import them with Audacity i find out that ch2 and ch4 files have been correctly separated, while ch1 and ch3 are corrupted and result in a white noise file.
Am i missing something on the separation? i thought that audio was stored on the pattern :
[[ch1_0],[ch2_0],[ch3_0],[ch4_0],[ch1_1],[ch2_1],...]
Also i dont get why, if the pattern i follow is not correct, 2 of the channels where separate succesfully.
I've also tried to cast the chunk into something else like:
let source = new Int8Array(chunk);
and then in the for cicle:
channels[i%4].push(source[i])
with different Type like Float32Array, Uint8Array, Uint16Array, Int16Array
but the result are the same.
I've already tested that the 4mic is working correctly by using the command:
arecord -r16000 -fS16_LE -traw -c4 -Dac108 -I ch1 ch2 ch3 ch4
which produce 4 file as expected containing each channel.
For every test, I block with mi finger on mic each couple of seconds while speaking so i can tell the difference between every channel.
Can anyone help me? or have some hints?
Thanks!
OK i figured out what was the problem.
Basically i was recording with bitWidth = 16, and the Buffer object in Node is an instance of Uint8Array so the pattern i was following was indeed correct, but due to the 8bitArray i had to assign 2 element per channel. Cause the format of the 8bit Array is:
[[ch1],[ch1],[ch2],[ch2],[ch3],[ch3],[ch4],[ch4],...]
Also casting the Array was useless because the produced 16bitArray didn't merge 2 8bit element to create a 16bit element but instead each 8bit element create the last 8bit of a 16bit element where the first 8bit are 0 like this:
8bitArray=[[11111111],[11111111],[2222222],[22222222],...]
16bitArray Casted= [[0000000011111111],[0000000011111111],[0000000022222222],...]
So i have created a method that merge the 8bitArray into the correct Array to be handle correctly based on the bitWidth of which your are recording:
var bitMultipler = bitWidth/8; //so i can handle any bitWidth with the same code
let channelsMap = new Map<number, Array<any>>();
for (let channel: number = 0; channel < totalChannels; channel++) {
channelsMap.set(channel, new Array())
}
/**
For each channel i push as many element as needed based on the bitMultipler
*/
let i = 0
while (i < chunk.length) {
for (let channel = 0; channel < totalChannels; channel++) {
for (let indexMultipler = 0; indexMultipler < bitMultipler; indexMultipler++) {
channelsMap.get(channel).push(chunk[i]);
i++;
}
}
}
I have two PCM-streams (decoder1 + decoder2):
var readable1 = fs.createReadStream("track1.mp3");
var decoder1 = new lame.Decoder({
channels: 2,
mode: lame.STEREO
});
readable1.pipe(decoder1);
and
var readable2 = fs.createReadStream("track2.mp3");
var decoder2 = new lame.Decoder({
channels: 2,
mode: lame.STEREO
});
readable2.pipe(decoder2);
Now I want to pipe the streams into one mix-function, where I can use the buffer-function like:
function mixStream(buf1, buf2, callback) {
// The mixStream-Function is not implemented yet (dummy)
var out = new Buffer(buf1.length);
for (i = 0; i < buf1.length; i+=2) {
var uint = Math.floor(.5 * buf1.readInt16LE(i));
out.writeInt16LE(uint, i);
}
this.push(out);
callback();
}
I need something like
mixStream(decoder1.pipe(), decoder2.pipe(), function() { }).pipe(new Speaker());
for output to speaker. Is this possible?
Well, pipe() function actually means a stream is linked to another, a readable to a writable, for instance. This 'linking' process is to write() to the writable stream once any data chunk is ready on the readable stream, along with a little more complex logic like pause() and resume(), to deal with the backpressure.
So all you have to do is to create a pipe-like function, to process two readable streams at the same time, which drains data from stream1 and stream2, and once the data is ready, write them to the destination writable stream.
I'd strongly recommend you to go through Node.js docs for Stream.
Hope this is what you are looking for :)
I'm working on a nodewebkit app that uses the Web Audio API to record microphone data and save it to disk.
I've used the RecordRTC framework, but it doesn't expose a way to stream the data to disk as the recording progresses (which is necessary given that the recordings could be longer than an hour).
I can't seem to find a good way to stream the data to disk using other methods either. If there is a proper way to do this, I would appreciate tips on what the proper tool for the job is.
However, the non-working solution I have now is:
Creating a ScriptProcessorNode with WebAudio API to access the PCM data
Create a readable stream buffer (using the stream-buffer module) and pipe it to a fileWriter (made with the wav module)
Translating that data to 16 bit ints in the onaudioprocess event and add them to the readable stream so they can be written
This hasn't worked because the ReadableStreamBuffer is only piping 20 bytes at a time to the fileWriter and not queueing all of the bytes coming from the microphone for some reason.
var wav = require('wav');
var streamBuffers = require("stream-buffers");
function convertFloat32ToInt16(buffer) {
var l = buffer.length;
var buf = new Int16Array(l);
while (l--) {
buf[l] = Math.min(1, buffer[l])*0x7FFF;
}
return buf.buffer;
}
var filePath = utils.getCwd() + '/recordings/demo.wav';
var fileWriter = new wav.FileWriter( filePath, {
channels: 1,
sampleRate: 48000,
bitDepth: 16
});
var myReadableStreamBuffer = new streamBuffers.ReadableStreamBuffer({
frequency: 0, // in milliseconds.
chunkSize: 2048 // in bytes.
});
myReadableStreamBuffer.pipe(fileWriter);
source.connect(scriptNode);
scriptNode.connect(context.destination);
scriptNode.onaudioprocess = function(e) {
var arrayBuffer = convertFloat32ToInt16(e.inputBuffer.getChannelData(0));
myReadableStreamBuffer.put(arrayBuffer);
// close myReadableStreamBuffer and run fileWriter.end() when recording is done
};
I wrote the following code to define a SynthDef that records a sound into the buffer passed as one of the parameters.
(
SynthDef(\recordTone, { |freq, bufnum, duration|
var w = SinOsc.ar(freq) * XLine.ar(101,1,duration,add: -1) / 100;
RecordBuf.ar(w!2,bufnum,loop: 0,doneAction: 2);
}).add;
)
I also have the below code that invokes a Synth for the above SynthDef and tried to write the buffer into a file.
({
var recordfn = { |freq, duration, fileName|
var server = Server.local;
var buf = Buffer.alloc(server,server.sampleRate * duration,2);
Synth(\recordTone,[\freq, 440, \bufnum, buf.bufnum, \duration, duration]);
buf.write(
"/Users/minerva/Temp/snd/" ++ fileName ++ ".wav",
"WAVE",
"int16",
completionMessage: ["b_free", buf.bufnum]
);
};
recordfn.value(440,0.5,"test");
}.value)
The output file is being created, but does not contain any audible sound. What am I doing wrong? I've looked through all the SuperCollider documentation I could find, but nothing seems to work! Any pointers is greatly appreciated.
Based on what Dan S answer, I made a few changes to get this working:
(
SynthDef(\playTone, { |freq, duration|
var w = SinOsc.ar(freq) * XLine.ar(1001,1,duration,add: -1,doneAction:2) / 1000;
Out.ar(0,w!2);
}).add;
)
(
SynthDef(\recordTone, { |buffer|
RecordBuf.ar(In.ar(0,2), buffer, loop: 0, doneAction: 2);
}).add;
)
(Routine({
var recordfn = { |freq, duration|
var server = Server.local;
var buffer = Buffer.alloc(server, server.sampleRate * duration, 2);
server.sync;
server.makeBundle(func: {
var player = Synth(\playTone, [\freq, freq, \duration, duration]);
var recorder = Synth.after(player, \recordTone, [\buffer, buffer]);
});
duration.wait;
buffer.write(
"/Users/minerva/Temp/snd/test.wav",
"WAVE",
"int16",
completionMessage: ["/b_free", buffer]
);
};
recordfn.value(440,0.1);
}).next)
Your main problem is that in your recordfn function you're instantiating the SynthDef (i.e. STARTING it recording) and writing the Buffer to disk at the same time. Obviously, at the point you START recording, there's no sound in the Buffer, so SuperCollider is doing exactly as you ask and writing the empty silent Buffer out as a file.
Solutions:
The most basic solution is to invoke one function to start recording, and a separate function when it's time to write to disk.
OR if you want it all in one, consider launching a Task within your function in order to wait until the Buffer is ready to be written to disk.
OR instead of RecordBuf use DiskOut which is for directly "spooling" to disk.
A secondary thing: I can't remember right now but I think it might be "WAV" not "WAVE".