Clicking Sounds When Playing Clips in Rapid Succession - audio

Question

I have a very simple program that plays 4 different tones, depending on what button is pressed. I have found that if I play multiple tones or the same tone in rapid succession, there are unpleasant clicking noises produced. I have made sure that these clicks are not present in my audio samples; it is definitely caused by playing the clips quickly one after another.
After googling around, I'm fairly sure that the clicks are due to the rapid change in pitch between clips. Looking at the waveform of the playback from the offending audio, it looks like a clip is first cancelled for a fraction of a second before starting the next clip. I have highlighted the section where this seems particularly obvious.
The clip that showcases these audio clicks can also be downloaded here.
My code is very simple. I am using XInput to read input from a connected controller, which determines the tone to play, and I am using WinMM to output sound from wav files. It is written in the D programming language, but I have modified it to use no D-specific features to make it as C-like as possible and to avoid confusion.
SHORT keyPressed(int vkey)
{
enum highBit { val = 0x8000 }
return cast(SHORT)(GetKeyState(vkey) & highBit.val);
}
enum Button
{
DPAD_UP = 0x0001,
DPAD_DOWN = 0x0002,
DPAD_LEFT = 0x0004,
DPAD_RIGHT = 0x0008,
START = 0x0010,
BACK = 0x0020,
LEFT_THUMB = 0x0040,
RIGHT_THUMB = 0x0080,
LEFT_SHOULDER = 0x0100,
RIGHT_SHOULDER = 0x0200,
A = 0x1000,
B = 0x2000,
X = 0x4000,
Y = 0x8000,
}
struct XINPUT_GAMEPAD
{
WORD wButtons;
BYTE bLeftTrigger;
BYTE bRightTrigger;
SHORT sThumbLX;
SHORT sThumbLY;
SHORT sThumbRX;
SHORT sThumbRY;
}
struct XINPUT_STATE
{
DWORD dwPacketNumber;
XINPUT_GAMEPAD Gamepad;
bool isPressed(int button)
{
return cast(bool)(Gamepad.wButtons & button);
}
}
int main()
{
HANDLE xinputDLL = initXinput();
XINPUT_STATE oldState;
XINPUT_STATE newState;
while (!keyPressed(VK_ESCAPE))
{
oldState = newState;
XInputGetState(0, &newState);
enum flags { val = SND_ASYNC | SND_FILENAME | SND_NODEFAULT }
if (newState.isPressed(Button.A) && !oldState.isPressed(Button.A))
{
PlaySoundA(toStringz("Piano.ff.A4.wav"), null, flags.val);
}
if (newState.isPressed(Button.B) && !oldState.isPressed(Button.B))
{
PlaySoundA(toStringz("Piano.ff.B4.wav"), null, flags.val);
}
if (newState.isPressed(Button.X) && !oldState.isPressed(Button.X))
{
PlaySoundA(toStringz("Piano.ff.C5.wav"), null, flags.val);
}
if (newState.isPressed(Button.Y) && !oldState.isPressed(Button.Y))
{
PlaySoundA(toStringz("Piano.ff.F4.wav"), null, flags.val);
}
}
denitXinput(xinputDLL);
return 0;
}
Assuming that I'm correct in regards to the source of the clicking sounds, I think the solution is to have each sample fade into the next one. However, I am not sure how to do this as the WinMM documentation seems relatively sparse, and I am inexperienced with it.
Is the solution to my problem of clicks when playing audio samples to have each sample fade into the next one? If so, how can I accomplish this using WinMM? If not, is there another solution that I can try?

Answer 1

I know how we can solve this in theory, but I don't have actual working code yet for all cases. (When I do, I'll edit this.)
First, the simple case which kinda works: instead of using PlaySound, try mciSendStringA:
if(auto err = mciSendStringA("play test.wav", null, 0, null))
writeln(err);
I am not making that up, Windows actually has that function, and it actually works with a lot of little command strings and file formats (though if your program terminates, all sound stops, so make sure the program keeps running e.g. stay in your controller loop or call Sleep(something)).
I've used a lot of Win32 and sometimes I'm amazed by how much stuff it has. Prototype:
extern(Windows) uint mciSendStringA(in char*,char*,uint,void*);
found in winmm.lib.
That basically works, but in my test, playing the same file twice at the same time has no effect. Playing different files together mixes them though. So it is a partial solution.
Next step from that would be to use the mciSendCommand function - a bit lower level than send string, so you can open multiple devices and try to get more overlap that way:
http://msdn.microsoft.com/en-us/library/windows/desktop/dd743675%28v=vs.85%29.aspx
I haven't tried this yet, but it looks fairly simple and I suspect it might be good enough for you. Open up a few devices for each button so you can hit them a few times fast and it cycles through them, hopefully mixing the same sound more than once when needed.
The prototype to that is:
extern(Windows) uint /*MCIERROR*/ mciSendCommandA(MCIDEVICEID,UINT,DWORD,DWORD);
Yes, it casts to void* then to DWORD in the msdn example. Blargh. Relevant structs:
struct MCI_OPEN_PARMSA {
DWORD dwCallback;
MCIDEVICEID wDeviceID; // aka uint
LPCSTR lpstrDeviceType;
LPCSTR lpstrElementName;
LPCSTR lpstrAlias;
}
struct MCI_PLAY_PARMS {
DWORD dwCallback;
DWORD dwFrom;
DWORD dwTo;
}
and you can borrow some constants from here too:
https://github.com/AndrejMitrovic/DWinProgramming/blob/master/WindowsAPI/win32/mmsystem.d#L693
(if you are already using the win32 bindings, great! But I think they are kinda a pain for little things so I try to avoid them, preferring to copy/paste prototypes+structs+constants off MSDN as I need them.)
You should be able to get the MSDN example working with those definitions and core.sys.windows.windows. Don't forget pragma(lib, "winmm"); too.
I think a full solution that will certainly work, but is also quite a bit harder, will be using the low level interface to mix the sounds yourself as they happen and send that result to the device. I don't have this working yet and I'm out of time today, but hopefully I can get something to you tomorrow.
The basic steps are:
1) call waveOutOpen to get a device. Set up a callback function which it calls when it needs more data.
2) prepare a buffer - or perhaps more than one - with waveOutPrepareHeader
3) feed data with waveOutWrite when requested by your callback (might want this in a separate thread) with the current notes. Mixing two samples is simply a case of adding the values together (and clipping if they overflow - sounds awful btw but hopefully that won't actually happen) so if you are doing more than one sound, just add them as you go.
Don't forget extern(Windows) on any callback function!
4) Loading your samples probably means reading the .wav file. That's not super hard, Windows has helper functions or you can do it yourself. I'll show code for this too.
What I have so far is in my simpleaudio.d https://github.com/adamdruppe/arsd/blob/master/simpleaudio.d find struct AudioOutput and the WinMM version. It has a horrible API right now that must be radically changed - it was acceptable on Linux but sucks on Windows. A callback feeder instead of write(data) should work better on both platforms, so that's what I'll do.
Problem I'm having with the demo right now is gaps between buffers... leading to clicky sounds. Yeah. But I'm sure it is just latency that should be solved with the proper callback approach and buffer sizing.
That MCI function might work for you as a next step though, maybe even a final step if the multiple devices works.
BTW: you could also prolly make it do MIDI commands instead of playing wavs and get all kinds of cool stuff. Simpleaudio.d's low level midi is already functioning - the demo main even shows a piano scale. Rigging it into the xbox controller shouldn't be too hard... note on when the button is pressed, note off when released, and not even think about timing.. Not really an answer to the question but a cool thing to play with in the same vein!