I'm trying to prepend the byte array of a sound object to a captured microphone sound byte array.
It works, but the extracted sound object get pichted down and doubled in length. I guess this is because the byte array of the sound object is in stereo, while the mic byte array is in mono.
I have this:
sound.extract(myByteArray, extract);
myByteArray now contains the stereo data. How can I turn this to mono (I'm new to ByteArrays).
UPDATE:
Here's a working solution:
existingByte.position = 0;
var mono : ByteArray = new ByteArray();
while(existingByte.bytesAvailable) {
var left : Number = existingByte.readFloat();
mono.writeFloat(left);
existingByte.position +=4;
}
Just pick a channel to extract. I think ByteArray is interleaved so if you pick all odd bytes its the left channel, if you pick all even bytes it's the right channel.
var mono : ByteArray = new ByteArray();
for( var i : int = 0; i < raw.length; i+=2 ) {
var left : int = raw[i];
var right : int = raw[i+1];
var mixed : int = left * 0.5 + right * 0.5;
if( pickLeft ) {
mono.writeByte( left );
} else if( pickRight ) {
mono.writeByte( right );
} else {
mono.writeByte( mixed );
}
}
Related
I am using Microsoft Azure Text To Speech with Unity. But there will be broken sounds at the beginning and end of the playing sound. Is this normal, or result.AudioData is broken. Below is the code.
public AudioSource audioSource;
void Start()
{
SynthesisToSpeaker("你好世界");
}
public void SynthesisToSpeaker(string text)
{
var config = SpeechConfig.FromSubscription("[redacted]", "southeastasia");
config.SpeechSynthesisLanguage = "zh-CN";
config.SpeechSynthesisVoiceName = "zh-CN-XiaoxiaoNeural";
// Creates a speech synthesizer.
// Make sure to dispose the synthesizer after use!
SpeechSynthesizer synthesizer = new SpeechSynthesizer(config, null);
Task<SpeechSynthesisResult> task = synthesizer.SpeakTextAsync(text);
StartCoroutine(CheckSynthesizer(task, config, synthesizer));
}
private IEnumerator CheckSynthesizer(Task<SpeechSynthesisResult> task,
SpeechConfig config,
SpeechSynthesizer synthesizer)
{
yield return new WaitUntil(() => task.IsCompleted);
var result = task.Result;
// Checks result.
string newMessage = string.Empty;
if (result.Reason == ResultReason.SynthesizingAudioCompleted)
{
var sampleCount = result.AudioData.Length / 2;
var audioData = new float[sampleCount];
for (var i = 0; i < sampleCount; ++i)
{
audioData[i] = (short)(result.AudioData[i * 2 + 1] << 8
| result.AudioData[i * 2]) / 32768.0F;
}
// The default output audio format is 16K 16bit mono
var audioClip = AudioClip.Create("SynthesizedAudio", sampleCount,
1, 16000, false);
audioClip.SetData(audioData, 0);
audioSource.clip = audioClip;
audioSource.Play();
}
else if (result.Reason == ResultReason.Canceled)
{
var cancellation = SpeechSynthesisCancellationDetails.FromResult(result);
}
synthesizer.Dispose();
}
The default audio format is Riff16Khz16BitMonoPcm, which has a riff header in the beginning of result.AudioData. If you pass the audioData to audioClip, it will play the header, then you hear some noise.
You can set the format to a raw format without header by speechConfig.SetSpeechSynthesisOutputFormat(SpeechSynthesisOutputFormat.Raw16Khz16BitMonoPcm);, see this sample for details.
I try to make some concatenation of buffers which are saved in a memory streams. Then, when I'm trying to play the whole buffer it gives an exception:
An exception of type 'System.ArgumentException' occurred in
Microsoft.Xna.Framework.ni.dll but was not handled in user code
Additional information: Ensure that the buffer length is non-zero and
meets the block alignment requirements for the audio format.
When I debug the mStrm is still remains 0, can't find why.
private void mySendClick(object sender, RoutedEventArgs e)
{
var mStrmStartDelimiter = new MemoryStream();
var mStrmEndDelimiter = new MemoryStream();
BinaryWriter writer1 = new BinaryWriter(mStrmStartDelimiter);
Sinus(6500, 200, writer1, 32767);
BinaryWriter writer2 = new BinaryWriter(mStrmEndDelimiter);
Sinus(6800, 200, writer2, 32767);
var mStrm = new MemoryStream();
mStrmStartDelimiter.CopyTo(mStrm);
//ToDO
mStrmEndDelimiter.CopyTo(mStrm);
mStrm.Seek(0, SeekOrigin.Begin);
SoundEffect mySoundPlay = new SoundEffect(mStrm.ToArray(), 16000, AudioChannels.Mono);
mySoundPlay.Play();
}
public static void Sinus(double frequency, int msDuration, BinaryWriter writer, int volume)
{
double TAU = 2 * Math.PI;
double samplesPerSecond = 16000;
double theta = frequency * TAU / (double)samplesPerSecond;
int samples = (int)((decimal)samplesPerSecond * msDuration / 1000);
// 'volume' is UInt16 with range 0 thru Uint16.MaxValue ( = 65 535)
// we need 'amp' to have the range of 0 thru Int16.MaxValue ( = 32 767)
double amp = volume >> 2; // so we simply set amp = volume / 2
for (int step = 0; step < samples; step++)
{
short s = (short)(amp * Math.Sin(theta * (double)step));
writer.Write(s);
}
}
I'm targeting windows phone 8.1 silverlight platform
I got the solution for the problem: do the following before calling CopyTo()
mStrmStartDelimiter.Position = 0;
mStrmEndDelimiter.Position = 0;
So basically i have a soundboard made in Flash CS5, is it possible to alternate the sound of the library's audio files with using Flash only? Like make the clips sound deeper or faster, thats the point. But if it's not pissbile
You cannot do it in AS2. But yes, you can change the pitch and playback speed of a sound in AS3. In fact you can do more than that (search for Audiotool) but this demonstrates how it is done:
http://plasticsturgeon.com/2012/05/changing-the-pitch-of-a-looped-sound-at-runtime-with-actionscript/
And in case my blog goes down or something, here is the relevant class that changes the pitch of a sound.:
package sound
{
import flash.events.Event;
import flash.events.SampleDataEvent;
import flash.media.Sound;
import flash.media.SoundChannel;
import flash.media.SoundTransform;
import flash.net.URLRequest;
import flash.utils.ByteArray;
/**
* #author Andre Michelle (andr...#gmail.com)
* Modified by Zach Foley aka The Plastic Sturgeon
*/
public class MP3Pitch
{
private const BLOCK_SIZE: int = 3072;
private var _mp3: Sound;
private var _sound: Sound;
private var _target: ByteArray;
private var _position: Number;
private var _rate: Number;
private var repeat:SoundChannel;
private var _volume:Number = 1;
private var byteArray:ByteArray;
// Pass in your looped Sound
public function MP3Pitch( pitchedSound: Sound)
{
_target = new ByteArray();
_mp3 = pitchedSound;
_position = 0.0;
_rate = 0.0;
_sound = new Sound();
_sound.addEventListener( SampleDataEvent.SAMPLE_DATA, sampleData );
repeat = _sound.play();
}
public function get rate(): Number
{
return _rate;
}
// Also added a handy volume setter
public function set volume( value: Number ): void
{
_volume = value;
repeat.soundTransform = new SoundTransform(_volume);
}
// use this to set the pitch of your sound
public function set rate( value: Number ): void
{
if( value < 0.0 )
value = 0;
_rate = value;
}
private function sampleData( event: SampleDataEvent ): void
{
//-- REUSE INSTEAD OF RECREATION
_target.position = 0;
//-- SHORTCUT
var data: ByteArray = event.data;
var scaledBlockSize: Number = BLOCK_SIZE * _rate;
var positionInt: int = _position;
var alpha: Number = _position - positionInt;
var positionTargetNum: Number = alpha;
var positionTargetInt: int = -1;
//-- COMPUTE NUMBER OF SAMPLES NEED TO PROCESS BLOCK (+2 FOR INTERPOLATION)
var need: int = Math.ceil( scaledBlockSize ) + 2;
//-- EXTRACT SAMPLES
var read: int = _mp3.extract( _target, need, positionInt );
var n: int = read == need ? BLOCK_SIZE : read / _rate;
var l0: Number;
var r0: Number;
var l1: Number;
var r1: Number;
for( var i: int = 0 ; i < n ; ++i )
{
//-- AVOID READING EQUAL SAMPLES, IF RATE < 1.0
if( int( positionTargetNum ) != positionTargetInt )
{
positionTargetInt = positionTargetNum;
//-- SET TARGET READ POSITION
_target.position = positionTargetInt << 3; //-- READ TWO STEREO SAMPLES FOR LINEAR INTERPOLATION l0 = _target.readFloat(); r0 = _target.readFloat(); l1 = _target.readFloat(); r1 = _target.readFloat(); } //-- WRITE INTERPOLATED AMPLITUDES INTO STREAM data.writeFloat( l0 + alpha * ( l1 - l0 ) ); data.writeFloat( r0 + alpha * ( r1 - r0 ) ); //-- INCREASE TARGET POSITION positionTargetNum += _rate; //-- INCREASE FRACTION AND CLAMP BETWEEN 0 AND 1 alpha += _rate; while( alpha >= 1.0 ) --alpha;
}
//-- FILL REST OF STREAM WITH ZEROs
if( i < BLOCK_SIZE )
{
while( i < BLOCK_SIZE ) { data.writeFloat( 0.0 ); data.writeFloat( 0.0 ); ++i; } } //-- INCREASE SOUND POSITION _position += scaledBlockSize; // My little addition here: if (_position > _mp3.length * 44.1) {
_position = 0;
_target.position = 0;
}
}
}
}
This is a small project for testing pixel level manipulation performance of NME for different builds (Windows c++, Flash).
It uses BitmapData.setPixel to modify the pixels one by one (320x240 for every frame). The C++ build runs at 22 FPS, and the flash build around ~100 FPS. Whats the reason for the huge performance drop for the C++ build compared to flash? How could I improve the code to get higher FPS using the C++ build?
Mandelbrot.hx
import nme.display.Sprite;
import nme.display.Bitmap;
import nme.display.BitmapData;
import nme.text.TextField;
import nme.events.Event;
import nme.events.TimerEvent;
import nme.utils.Timer;
import nme.geom.Matrix;
import nme.geom.Rectangle;
import nme.utils.ByteArray;
class Mandelbrot
{
public static function main() : Void
{
new Mandelbrot();
}
public var pixels:Array<Array<Int>>;
public var colorModifier:Int;
private var bitmapData:BitmapData;
private var bigBitmapData:BitmapData;
private var fps:TextField;
private var width:Int;
private var height:Int;
private var matrix:Matrix;
public function new()
{
width = 320; //Std.int(flash.Lib.current.stage.stageWidth/2);
height = 240; //Std.int(flash.Lib.current.stage.stageHeight/2);
var scale:Float = 2;//flash.Lib.current.stage.stageWidth/width;
matrix = new Matrix();
matrix.scale(scale, scale);
var setBitmap:Bitmap = new Bitmap();
bitmapData = new BitmapData( width , height , false , 0x000000 );
bigBitmapData = new BitmapData( nme.Lib.current.stage.stageWidth , nme.Lib.current.stage.stageHeight , false , 0x000000 );
setBitmap.bitmapData = bigBitmapData;
nme.Lib.current.addChild( setBitmap );
var maxIterations:Int = 128;
pixels = new Array();
var beforeTime = nme.Lib.getTimer();
var xtemp;
var iteration;
var x0:Float = 0;
var y0:Float = 0;
for(ix in 0...width) {
pixels[ix] = new Array();
for(iy in 0...height) {
x0 = 0;
y0 = 0;
iteration = 128;
while ( x0*x0 + y0*y0 <= 4 && iteration > 0 )
{
xtemp = x0*x0 - y0*y0 + (ix-14*5000)/50000;
y0 = 2*x0*y0 + (iy-(height/0.6))/50000;
x0 = xtemp;
iteration--;
}
pixels[ix][iy] = iteration;
}
}
var afterTime = nme.Lib.getTimer();
var tf = new TextField();
tf.width = 400;
tf.text = "Generating fractal took "+(afterTime-beforeTime)+" ms";
nme.Lib.current.addChild(tf);
fps = new TextField();
fps.width = 400;
fps.y = 10;
fps.text = "FPS: ";
nme.Lib.current.addChild(fps);
colorModifier = 2;
var timer:haxe.Timer = new haxe.Timer(10);
runLoop();
timer.run = runLoop;
}
public function runLoop() {
var r:Int=0, b:Int=0, g:Int=0;
var pixel:Int = 0;
var beforeTime = nme.Lib.getTimer();
for(iy in 0...height) {
for(ix in 0...width) {
pixel = pixels[ix][iy];
r = pixel + colorModifier;
g = pixel + colorModifier + r;
b = pixel + colorModifier + g;
bitmapData.setPixel(ix, iy, (r<<16 | g<<8 | b));
}
}
bigBitmapData.draw(bitmapData, matrix, null, null, null, false);
var afterTime = nme.Lib.getTimer();
fps.text = "FPS: "+Math.round(1000/(afterTime-beforeTime));
colorModifier += 2;
if(colorModifier > 65530)
colorModifier = 0;
}
}
Mandelbrot.nmml
<?xml version="1.0" encoding="utf-8"?>
<project>
<app
file="Mandelbrot.hx"
title="Mandelbrot sample"
package="org.haxe.nme.mandelbrot"
version="1.0.0"
company="nme"
main="Mandelbrot"
/>
<window
width="640"
height="480"
orientation="landscape"
fps="60"
background="0xffffff"
resizeable="true"
hardware="true"
/>
<classpath name="." />
<haxelib name="nme" />
<ndll name="std" />
<ndll name="regexp" />
<ndll name="zlib" />
<ndll name="nme" haxelib="nme" />
<setenv name="SHOW_CONSOLE"/>
</project>
Look into the nme.Memory API. The idea is to create a ByteArray with the correct size (or get it from a BitmapData), select it as the current virtual memory space and manipulate its bytes directly.
You'll get an approximately 10x speed boost with Flash and it should be way faster with the CPP target too. Don't forget to compile in Release mode or method inlining will be disabled and performances will suffer a lot.
Basic usage example (untested code) :
var rect:Rectangle = bitmapData.rect;
// 32bits integer = 4 bytes
var size:Int = bitmapData.width * bitmapData.height * 4;
// The virtual memory space we'll use
var pixels:ByteArray = new ByteArray();
// CPP does not support setting the length property directly
#if (cpp) pixels.setLength(size);
#else pixels.length = size; #end
// Select the memory space (call it once, not every frame)
Memory.select(pixels);
// And in your loop set your color
// Color is in BGRA mode, nme.Memory can only be used in little endian mode.
Memory.setI32((y * width + x) * 4, color);
// When you're done, render the BitmapData
// (don't forget to reset the ByteArray position)
pixels.position = 0;
bitmapData.setPixels(rect, pixels);
Keep in mind this is a very basic code example. In your case, you'd need to adapt it and actually use a double sized ByteArray because you need to store the iteration count too. Nested loops can be optimized in your main loop and you can avoid a lot of extra index/address computations :
// Note the size * 2 !
// First part of the ByteArray will be used to store the iteration count,
// the second part to draw the pixels.
#if (cpp) pixels.setLength(size * 2);
#else pixels.length = size * 2; #end
Memory.select(pixels);
// First loop storing iteration count
for (iy in 0...height)
{
for (ix in 0...width)
{
// ... do some stuff ...
Memory.setI32((iy * width + ix) << 2, iteration);
}
}
// In your runLoop :
for (i in 0...(height * width))
{
// Get the iteration count
var pixel:Int = Memory.getI32(i << 2);
r = pixel + colorModifier;
g = pixel + colorModifier + r;
b = pixel + colorModifier + g;
// Note that we're writing the pixel in the second part of our ByteArray
Memory.setI32(size + (i << 2), r | g << 8 | b << 16);
}
// Sets the position to the second part of our ByteArray
pixels.position = size;
bitmapData.setPixels(rect, pixels);
And this is it. If you really don't want to use Alchemy Opcodes on the Flash target, the next fastest way to blit pixels is to use getVector() / setVector() from the BitmapData class. But it's really not as fast.
Array itself is not true liner array in flash, more like a map.
For the per-pixel manipulation I can recommend to use getVector/setVector api of the BitmapData class, which can retrieve (and assign) a rectangular area of the image as flat pixel data. In which case you can access individual pixels in the vector as:
pixels[ix + image_width*iy] = <argb32>
Also, instead of constructing an intermediate Array of Arrays it would be faster to assign pixels directly.
Try to use ByteArray. It would be faster in Flash and C++, I think.
I've been asked to implement and MD5 hasher ActionScript-3 and as I was in the middle of debugging how I formatted my input I came across a problem. When I try and output the ByteArray as a binary string using .toString(2), the toString(2) method will perform some short cuts that alter how the binary should look.
For Example
var bytes:ByteArray = new ByteArray();
bytes.endian = Endian.LITTLE_ENDIAN;
bytes.writeUTFBytes("a");
bytes.writeByte(0x0);
var t1:String = bytes[0].toString(2); // is 1100001 when it should be 01100001
var t2:String = bytes[1].toString(2); // is 0 when it should be 00000000
so I guess my question is, might there a way to output a binary String from a ByteArray that will always shows each byte as a 8 bit block?
All you need is to pad the output of toString(2) with zeros on the left to make its length equal to 8. Use this function for padding
function padString(str:String, len:int, char:String, padLeft:Boolean = true):String{
var padLength:int = len - str.length;
var str_padding:String = "";
if(padLength > 0 && char.length == 1)
for(var i:int = 0; i < padLength; i++)
str_padding += char;
return (padLeft ? str_padding : "") + str + (!padLeft ? str_padding: "");
}
With this function the code looks like this and gives the correct output
var bytes:ByteArray = new ByteArray();
bytes.endian = Endian.LITTLE_ENDIAN;
bytes.writeUTFBytes("a");
bytes.writeByte(0x0);
var t1:String = padString(bytes[0].toString(2), 8, "0"); // is now 01100001
var t2:String = padString(bytes[1].toString(2), 8, "0"); // is now 00000000
Update
If you want to get a string representation of complete byteArray you can use a function which iterates on the byteArray. I have wrote the following function and it seems to work correctly. Give it a try
// String Padding function
function padString(str:String, len:int, char:String, padLeft:Boolean = true):String{
// get no of padding characters needed
var padLength:int = len - str.length;
// padding string
var str_padding:String = "";
// loop from 0 to no of padding characters needed
// Note: this loop will not run if padLength is less than 1
// as i < padLength will be false from begining
for(var i:int = 0; i < padLength; i++)
str_padding += char;
// return string with padding attached either to left or right depending on the padLeft Boolean
return (padLeft ? str_padding : "") + str + (!padLeft ? str_padding: "");
}
// Return a Binary String Representation of a byte Array
function byteArrayToBinaryString(bArray:ByteArray):String{
// binary string to return
var str:String = "";
// store length so that it is not recomputed on every loop
var aLen = bArray.length;
// loop over all available bytes and concatenate the padded string to return string
for(var i:int = 0; i < aLen; i++)
str += padString(bArray[i].toString(2), 8, "0");
// return binary string
return str;
}
Now you can simply use the byteArrayToBinaryString() function like this:
// init byte array and set Endianness
var bytes:ByteArray = new ByteArray();
bytes.endian = Endian.LITTLE_ENDIAN;
// write some data to byte array
bytes.writeUTFBytes("a");
bytes.writeByte(0x0);
// convert to binaryString
var byteStr:String = byteArrayToBinaryString(bytes); // returns 0110000100000000
Here is a function extended on the Hurlant library to handle hashing byteArray.
This class has a learning curve but once you get it you will love it.
As far as your ByteArray issue with toString. I know the toString method is not accurate For this very reason.
You might want to look into byteArray.readMultiByte that will give you the 01 you are looking for. Although I can't seem top get it to work on my sample code either lol
I just always get a and empty string.
var bytes:ByteArray = new ByteArray();
bytes.endian = Endian.LITTLE_ENDIAN;
bytes.writeUTFBytes("a");
bytes.writeByte(0x0);
bytes.position = 0
var t1:String = bytes.readMultiByte(1,'us-ascii'); // is 1100001 when it should be 01100001
trace(t1)
var t2:String = bytes.readMultiByte(1,'iso-8859-01'); // is 0 when it should be 00000000
trace(t2)