I need to measure signal frequency while the musicians play music, and it happens to be a bit too fast for FFT (Fast Fourier Transform).
Musicians play music at 90-140 bpm. This means that there are 90-140 groups of notes each minute, up to 8 (more frequently, up to 4) notes in each group (60/140/8 = 0.0536 sec, 60/90/4 = 0.167 sec), that is, notes may change at the rate of 6-19 notes per second.
The music uses a logarithmic scale: the range between, say, 440Hz and 880Hz is divided into 12 notes, only 7 of which are used for melody. (Basically, they use only the white keys on the piano; when they want to shift the starting frequency, they use some of the black keys and don't use some white keys.)
That is, the frequency of each next note is multiplied by 2^(1/12) = 1.05946.
To make things more complicated, the A (La) frequency may vary from 438 to 446 Hz. The string instruments in theory can be tuned, while the wind instruments depend on the air temperature and humidity, so the frequency happens to be re-negotiated by the musicians during the sound check.
Sometimes musicians and vocalists make errors in frequency, they call it "out of tune". They want a device that would inform them of such "out of tune errors". They have tuners, but the tuners require playing the same sound for about 1 sec before they start showing anything. This works for tuning, but does not work while the music is played.
Most likely, the tuner is doing FFT, and due to the formula
df = 1/T
waits for 1 second to get the 1Hz resolution.
For A=440Hz, the difference in frequency between two notes is 440*0.05946 = 26.16 Hz, to get that frequency resolution, one has to use acquisition time of 0.038 sec, that is, at tempo=196bpm FFT is able to just distinguish two notes, at 98 bpm it is able to tell a 50% out-of-tune error provided that it starts acquisition at the very moment that the pitch changes. If we allow the pitch change in the course of an acquisition period, we get 49 bpm, which is just too slow. In addition, it is very desirable to be more precise about the frequency, say, detect a 25% out-of-tune error.
Is there a way to measure frequency better than FFT, that is, with better resolution in less acquisition time? (At least 2 times better, ideally, 8 times better.)
In exchange, I do not need to distinguish between notes of different octaves, e.g. both 440 and 880 may be recognized as A. (Probably, more trade-offs are possible, just nothing else comes to my mind right now.)
UPD
Here's a really good drawing:
UPD2
I have found a PhD thesis and open source software (TARTINI -- the real-time music analysis tool) at:
http://miracle.otago.ac.nz/tartini/
(The pages are also available via the web archive service: http://web.archive.org = http://archive.org = http://waybackmachine.org )
Regarding the FFT, assuming the narrow-band spectral frequency content is sparse and well separated in low enough background noise, frequency peaks can be interpolated or phase vocoded to much higher resolution than the FFT bin spacing (bin spacing as related to the inverse of the length of the segment of actual time-domain data). Parabolic interpolation is common, but there are other more accurate interpolation kernels. Phase vocoder frequency estimation methods require stationarity across 2 overlapped frames, however the total span of those 2 frames can be relatively short.
But the peak spectral frequency reported by an FFT is not the same as a pitch frequency as perceived by a human (as voices and many musical instruments can radiate more acoustic spectral energy in an overtone series than at pitch frequency, sometimes slightly inharmonically.) There are algorithms more suited for pitch estimation than FFTs (alone). A partial list is in this answer: FFT on iPhone to ignore background noise and find lower pitches
Many academic papers on pitch estimation methods for music can be found on the music-ir/MIREX site: http://www.music-ir.org/mirex/wiki/MIREX_HOME
Related
I want to do precise guitar tuner, this is usually done by many via computing FFT and getting peak. But this is of low appliance for several reasons:
Discrete precision, gives insuffient resolution for tuning bass guitar.
High computation time and complexity, when trying to increase buffer size(and/or sampling rate). Introduces visible delay(lag).
Most of frequency range where concentrates all FFT's precision is unused. Everything above 1-2 khz is not appliable for tuning musical instruments.
There should be simplier way for signals that have single-frequency sinusoidal shape. Given small enough buffer (say it 256 samples with 96khz sampling rate) - how could you measure a base(lowese) frequency?
In simple words: How to find frequency F, so that difference of "sine signal of frequency F" and "actually recorded signal" would give minimal error, than for any frequency, other than F ? (so we can definetely conclude that sinusoid of frequency F is best approximation of recorded sound buffer).
PS. Anything, but not using FFT!
Here is a simple approach based on zero crossing. It relies on being able to map the instrument signal to a simple sinuoid. This may work OK when signal to noise ratio is high, but is not a very robust method.
Bandpass filter around the fundamental frequency of the tone you want to tune for. Example 82.41 Hz for low E string on guitar.
Consider a window of the last N samples. Set it to ex 100ms to update the pitch estimate 10 times per second.
Perform zero-crossing detection with a threshold value T. T could be set to 10% of signal peak for example. Count the periods between each zero crossing, collect them in an array.
Take the median of the periods to get your pitch estimate
You can also compute the quantiles of the periods to estimate how reliable the method is. If they give very different numbers from the median, then the method is not working well.
The approach can be extended by computing autocorrelation on the zero-crossings, as described in
https://www.cycfi.com/2018/03/fast-and-efficient-pitch-detection-bitstream-autocorrelation/
I have read all the wikipedia articles and stackoverflow articles on fft and resolution. However, nothing has helped in learning how to get high resolution frequency without having a huge latency issues.
If I understand signal processing correctly:
I have a sampling rate of 44,100, and I take 256 block. Then the frequency resolution would be 44,100/2/256 = 86.1 Hz per frequency bin with FFT.
Constantly I see examples like http://www.tunelab-world.com/, and http://www.spectraplus.com/ that are able to determine the frequency down to .01 Hz.
If I did that with my above method I would need 4410,000 bins to get that kind of resolution. At 44,100 sampling rate it would take 100 seconds to fill in the data from the input.
I know I am missing something, but I can't figure what.
How can I get a signal, and then draw a graph or display the frequency of a peak with that kind of accuracy without taking a gazillion bins or waiting forever?
Thanks in advance for your help!
If you want a high frequency resolution FFT output, you have to perform the FFT over many samples: there is simply no way round that.
What you are probably seeing in other applications is overlapping: they may do a 4096 pt FFT on the first set of data, then move along 256 samples and do another 4096 pt FFT (on 3840 of the samples they have already used, plus a new 256 samples).
This allows you to show regular (different) updates with a fine frequency resolution. It will be no good for capturing transient signals, but looks good on an active display.
The reason you can get better accuracy is that the frequency estimation problem lends itself to being solved with higher accuracy than many other estimation problems.
The Cramer-Rao Lower Bound (CRLB) on the accuracy is given by:
which means that the variance of the frequency estimate (a measure of the expected error) goes down as the cube of T, the duration of the measurements. "Normal" estimation problems tend to have this measure go down as the square of T.
Using the FFT maximizer (the bin with the largest peak) will only get you the square of T.
As Adrian Taylor says, the examples you give are probably starting with a higher number of samples and then updating by a shorter duration.
For kicks, there are some frequency estimation algorithms here that might be of interest. They are quicker than the FFT, and more accurate.
SpectraPlus says "High Resolution FFT Analysis up to 1,048,576 pts"; that won't get you to 0.01 Hz resolution at 44.1 kHz.
TuneLab seems to go down to 0.01 cents, but the "spectrum display" appears to have a resolution of around 2.5 Hz at 440 Hz. The "phase display" is nothing special.
What are you trying to do? If you merely want to implement a guitar tuner, you don't need (and probably don't want) an FFT. Not knowing any better, I'd go for a PLL.
Im fairly new to onset detection. I read some papers about it and know that when working only with the time-domain, it is possible that there will be a large number of false-positives/negatives, and that it is generally advisable to work with either both the time-domain and frequency-domain or the frequency domain.
Regarding this, I am a bit confused because, I am having trouble on how the spectral energy or the results from the FFT bin can be used to determine note onsets. Because, aren't note onsets represented by sharp peaks in amplitude?
Can someone enlighten me on this? Thank you!
This is the easiest way to think about note onset:
think of a music signal as a flat constant signal. When and onset occurs you look at it as a large rapid CHANGE in signal (a positive or negative peak)
What this means in the frequency domain:
the FT of a constant signal is, well, CONSTANT! and flat
When the onset event occurs there is a rapid increase in spectrial content.
While you may think "Well you're actually talking about the peak of the onset right?" not at all. We are not actually interested in the peak of the onset, but rather the rising edge of the signal. When there is a sharp increase in the signal, the high frequency content increases.
one way to do this is using the spectrial difference function:
take your time domain signal and cut it up into overlaping strips (typically 50% overlap)
apply a hamming/hann window (this is to reduce spectrial smudging) (remember cutting up the signal into windows is like multiplying it by a pulse, in the frequency domain its like convolving the signal with a sinc function)
Apply the FFT algorithm on two sucessive windows
For each DFT bin, calculate the difference between the Xn and Xn-1 bins if it is negative set it to zero
square the results and sum all th bins together
repeat till end of signal.
look for peaks in signal using median thresholding and there are your onset times!
Source:
https://adamhess.github.io/Onset_Detection_Nov302011.pdf
and
http://www.elec.qmul.ac.uk/people/juan/Documents/Bello-TSAP-2005.pdf
You can look at sharp differences in amplitude at a specific frequency as suspected sound onsets. For instance if a flute switches from playing a G5 to playing a C, there will be a sharp drop in amplitude of the spectrum at around 784 Hz.
If you don't know what frequency to examine, the magnitude of an FFT vector will give you the amplitude of every frequency over some window in time (with a resolution dependent on the length of the time window). Pick your frequency, or a bunch of frequencies, and diff two FFTs of two different time windows. That might give you something that can be used as part of a likelihood estimate for a sound onset or change somewhere between the two time windows. Sliding the windows or successive approximation of their location in time might help narrow down the time of a suspected note onset or other significant change in the sound.
"Because, aren't note onsets represented by sharp peaks in amplitude?"
A: Not always. On percussive instruments (including piano) this is true, but for violin, flute, etc. notes often "slide" into each other as frequency changes without sharp amplitude increases.
If you stick to a single instrument like the piano onset detection is do-able. Generalized onset detection is a much more difficult problem. There are about a dozen primitive features that have been used for onset detection. Once you code them, you still have to decide how best to use them.
I have a working tone detector which uses an FFT to determine whether a tone (or tone pair) of a particular frequency is present in an audio stream (if sufficiently above the noise floor). What method could I use to more precisely locate the onset time and duration of that tone? I am looking for something far more precise than the FFT frame duration (about 50 ms). The tone is assumed to be much longer than an FFT frame.
Sounds like DTMF detection. The standard technique for this is the Goertzel algorithm. You need one Goertzel detector for each frequency of interest, so you need to know the frequencies a priori.
If the particular frequency is known ahead of time, you could design a bandpass filter centered around that frequency and then just use an energy detector on the output. You'd have to account for the bulk delay through the filter, and probably also the rise and fall times of the steady-state response.
If you're using the FFT output to actually detect the tone, and you have sufficient memory to keep the recent past samples, you could get a rough estimate of the onset from the FFT, go back in time a few hundred milliseconds before, and start mixing the samples by a sinusoid at the detected frequency. Then run the mixed samples through a low-pass filter. Your tone detection, mixer, and LPF frequency resolutions/bandwidths will have to match, and again you'll need to consider the LPF characteristics.
With limited resources such as slower CPUs, code size and RAM, how best to detect the pitch of a musical note, similar to what an electronic or software tuner would do?
Should I use:
Kiss FFT
FFTW
Discrete Wavelet Transform
autocorrelation
zero crossing analysis
octave-spaced filters
other?
In a nutshell, what I am trying to do is to recognize a single musical note, two octaves below middle-C to two octaves above, played on any (reasonable) instrument. I'd like to be within 20% of the semitone - in other words, if the user plays too flat or too sharp, I need to distinguish that. However, I will not need the accuracy required for tuning.
If you don't need that much accuracy, an FFT could be sufficient. Window the chunk of audio first so that you get well-defined peaks, then find the first significant peak.
Bin width = sampling rate / FFT size:
Fundamentals range from 20 Hz to 7 kHz, so a sampling rate of 14 kHz would be enough. The next "standard" sampling rate is 22050 Hz.
The FFT size is then determined by the precision you want. FFT output is linear in frequency, while musical tones are logarithmic in frequency, so the worst case precision will be at low frequencies. For 20% of a semitone at 20 Hz, you need a width of 1.2 Hz, which means an FFT length of 18545. The next power of two is 215 = 32768. This is 1.5 seconds of data, and takes my laptop's processor 3 ms to calculate.
This won't work with signals that have a "missing fundamental", and finding the "first significant" peak is somewhat difficult (since harmonics are often higher than the fundamental), but you can figure out a way that suits your situation.
Autocorrelation and harmonic product spectrum are better at finding the true fundamental for a wave instead of one of the harmonics, but I don't think they deal as well with inharmonicity, and most instruments like piano or guitar are inharmonic (harmonics are slightly sharp from what they should be). It really depends on your circumstances, though.
Also, you can save even more processor cycles by computing only within a specific frequency band of interest, using the Chirp-Z transform.
I've written up a few different methods in Python for comparison purposes.
If you want to do pitch recognition in realtime (and accurate to within 1/100 of a semi-tone), your only real hope is the zero-crossing approach. And it's a faint hope, sorry to say. Zero-crossing can estimate pitch from just a couple of wavelengths of data, and it can be done with a smartphone's processing power, but it's not especially accurate, as tiny errors in measuring the wavelengths result in large errors in the estimated frequency. Devices like guitar synthesizers (which deduce the pitch from a guitar string with just a couple of wavelengths) work by quantizing the measurements to notes of the scale. This may work for your purposes, but be aware that zero-crossing works great with simple waveforms, but tends to work less and less well with more complex instrument sounds.
In my application (a software synthesizer that runs on smartphones) I use recordings of single instrument notes as the raw material for wavetable synthesis, and in order to produce notes at a particular pitch, I need to know the fundamental pitch of a recording, accurate to within 1/1000 of a semi-tone (I really only need 1/100 accuracy, but I'm OCD about this). The zero-crossing approach is much too inaccurate for this, and FFT-based approaches are either way too inaccurate or way too slow (or both sometimes).
The best approach that I've found in this case is to use autocorrelation. With autocorrelation you basically guess the pitch and then measure the autocorrelation of your sample at that corresponding wavelength. By scanning through the range of plausible pitches (say A = 55 Hz thru A = 880 Hz) by semi-tones, I locate the most-correlated pitch, then do a more finely-grained scan in the neighborhood of that pitch to get a more accurate value.
The approach best for you depends entirely on what you're trying to use this for.
I'm not familiar with all the methods you mention, but what you choose should depend primarily on the nature of your input data. Are you analysing pure tones, or does your input source have multiple notes? Is speech a feature of your input? Are there any limitations on the length of time you have to sample the input? Are you able to trade off some accuracy for speed?
To some extent what you choose also depends on whether you would like to perform your calculations in time or in frequency space. Converting a time series to a frequency representation takes time, but in my experience tends to give better results.
Autocorrelation compares two signals in the time domain. A naive implementation is simple but relatively expensive to compute, as it requires pair-wise differencing between all points in the original and time-shifted signals, followed by differentiation to identify turning points in the autocorrelation function, and then selection of the minimum corresponding to the fundamental frequency. There are alternative methods. For example, Average Magnitude Differencing is a very cheap form of autocorrelation, but accuracy suffers. All autocorrelation techniques run the risk of octave errors, since peaks other than the fundamental exist in the function.
Measuring zero-crossing points is simple and straightforward, but will run into problems if you have multiple waveforms present in the signal.
In frequency-space, techniques based on FFT may be efficient enough for your purposes. One example is the harmonic product spectrum technique, which compares the power spectrum of the signal with downsampled versions at each harmonic, and identifies the pitch by multiplying the spectra together to produce a clear peak.
As ever, there is no substitute for testing and profiling several techniques, to empirically determine what will work best for your problem and constraints.
An answer like this can only scratch the surface of this topic. As well as the earlier links, here are some relevant references for further reading.
Summary of pitch detection algorithms (Wikipedia)
Pros and cons of Autocorrelation vs Harmonic Product Spectrum
A high-level overview of pitch detection methods
In my project danstuner, I took code from Audacity. It essentially took an FFT, then found the peak power by putting a cubic curve on the FFT and finding the peak of that curve. Works pretty well, although I had to guard against octave-jumping.
See Spectrum.cpp.
Zero crossing won't work because a typical sound has harmonics and zero-crossings much more than the base frequency.
Something I experimented with (as a home side project) was this:
Sample the sound with ADC at whatever sample rate you need.
Detect the levels of the short-term positive and negative peaks of the waveform (sliding window or similar). I.e. an envelope detector.
Make a square wave that goes high when the waveform goes within 90% (or so) of the positive envelope, and goes low when the waveform goes within 90% of the negative envelope. I.e. a tracking square wave with hysteresis.
Measure the frequency of that square wave with straight-forward count/time calculations, using as many samples as you need to get the required accuracy.
However I found that with inputs from my electronic keyboard, for some instrument sounds it managed to pick up 2× the base frequency (next octave). This was a side project and I never got around to implementing a solution before moving on to other things. But I thought it had promise as being much less CPU load than FFT.