Is there any software that enables me to perform LOF outlier detection towards a 2D dataset? I just implemented LOF from the original paper and want to check whether my results are correct. So far I couldn't find any tool or online service.
You should check rapidminer application. And add anomaly detection plugin.
Did you check the Wikipedia article on Local Outlier Factor? It mentions ELKI, which has a LOF implementation (with index support, so O(n log n) for large data sets).
Note that many people overlook the reachability-distance in LOF, and only approximate it. The ELKI implementation supposedly is correct, so it can help you well for testing your results.
Rapid Miner considers the IP address attribute of DARPA 1998 dataset as polynomial type data ! But LOF is not optimized for that. Interestingly RAPID MINER cannot handle such huge network traffic datasets. Any other implementation available ?
Related
How to determine sample size given there is 20% point reduction [ before change – after change =20% ] with 95% confidence level and 90% power ? Any pointer on how to solve this
A good first step is always to think about, what kind of test you plan to use. From the very little information you give a paired t-test (or a single measurement t-test comparing the difference to zero) is a likely candidate.
You can now google for "statistical power of t test" to which you can add the name of any computer language or statistics software you plan to use. Except maybe for educational purposes I'd advise to compute statics not by hand but via software.
Kind of an obvious option for statistic software on stackoverflow might be R. In Ryou'll find solutions to many sample size or power calculations in the package pwr. Here is the link to a getting started text: https://cran.r-project.org/web/packages/pwr/vignettes/pwr-vignette.html
The pwr.t.test function is good for your problem. Google will readily help you to alternatives for Python and Julia and SPSS I assume for C++, Java and Javascript as well.
However you will have to make assumptions about the variance or the effect size. Will each value be reduced by almost exactly 20% or will some be reduced a lot and some increase? That is of utmost importance to the question. You will need only one observation if there is no variance, a small amount of observations if there is little variance and a large amount of observations if there is lots of variance.
Can anyone provide sample pseudocode or share some existing link that has sample code.
Like for example I have a mix audio of 1kHz or 2kHz or 8kHz or so, and I want to boost certain frequencies like 1kHz only in real-time.
Reading some DSP books and resources confuses me.
You just need to design and implement a suitable digital filter. This is a large and complex subject area though, so you won't get a simple answer here. Probably the best thing as a first step would be to read a good introductory book on DSP, e.g. Understanding DSP by Rick Lyons, which is a very good for beginners as it's not too heavy on the math and has a more practical bent than most such introductory DSP books.
For this particular application though what you are trying to do is similar to implementing a graphic equalizer, and there are many pointers to how to implement this kind of thing if you use e.g. "graphic equalizer" as a search term.
There's a lot of math behind digital filtering. Sorry, I think it is important to at least understand basic filters (like those used in electronics). If you don't want to go through the basics: best to get an audio graphics equaliser where you can play with the (virtual) sliders. If you want to implement a very specific filter, please read on.
Real time: depends on your computing platform. If this is a small micro (like AVR, Microchip PIC,..) you'll need an efficient algorithm. This is likely a IIR band pass filter. The equivalent of a graphics equaliser consists of multiple band pass filters, all summed together. See http://en.wikipedia.org/wiki/Infinite_impulse_response
A more computing intensive algorithm uses FIR filters. In that case you can also control the phase of the filtered signal. http://en.wikipedia.org/wiki/Finite_impulse_response
If you find an algorithm (i.e. IIR), you'll need to calculate the coefficients. The algorithm is simple, calculating the coefficients is not.
I found a book matching your question: Audio digital signal processing in real time
I browsed through it; it seems to have the right answers.
So I am working on Note Onset Detection. I have implemented the method here: Note onset detection
However, I am finding some difficulty or problems regarding the 'static' nature of the method. What I am looking for is how to make the thresholding method 'dynamic'. But I am finding trouble finding suitable solutions.
Aside from that, I am also working on instead of having the amplitude value as the basis of passing the threshold, I make use of the 'difference' between 2 amplitude values, to know when a signal increases or not, and how much it increased or decreased. This is what I'm using currently.
Anyone willing to help or has worked with this kind of problem? Thank you!
Additionally, by any chance do any of you have a PDF file of this paper: http://www.mendeley.com/research/methods-detecting-impulsive-noise-speech-audio-signals-14/
Volume compression is a form of AGC (Automatic Gain Control), and AGC can be done dynamically. The are plenty of close to real-time AGC algorithms to be found in search results, although a bit of delay is required if you want an AGC attack that's smoother than a step function.
I am looking for a way to compare a user submitted audio recording against a reference recording for comparison in order to give someone a grade or percentage for language learning.
I realize that this is a very un-scientific way of doing things and is more than a gimmick than anything.
My first thoughts are some sort of audio fingerprinting, or waveform comparison.
Any ideas where I should be looking?
This is by no means a trivial problem to solve, though there is an abundance of research on the topic. Presently the most successful forms of machine learning in the speech recognition domain apply Hidden Markov Model techniques.
You may also want to take a look at existing implementations of HMM algorithms. One such library in its early stages is ghmm.
Perhaps even better and more readily applicable to your problem is HTK.
In addition to chomp's great answer, one important keyword you probably need to look up is Dynamic Time Warping (DTW). This is the wikipedia article: http://en.wikipedia.org/wiki/Dynamic_time_warping
Given images from a certain viewpoints is there some software out there which can help me interpolate the views(i.e. Viewpoint interpolation software?).
Thanks,
View Interpolation is an illposed and very difficult problem, which in general has no optimal solution in real world scenarios.
This is because of several reasons, some of which include
Wide Baseline Matching
Disparity Estimation
Occlusion Handling (Folds and Holes)
The quality of the outcome highly depends on your video footage, especially on how close together the cameras are.
Nevertheless, research is ongoing. Dyer and Seitz for example achieved nice results on constrained examples:
http://homes.cs.washington.edu/~seitz/vmorph/vmorph.htm
Stich et.al. from TU Braunschweig showed some amazing results with their Virtual Camera system, which probably is the one thing, you are looking for:
http://www.youtube.com/watch?v=uqKTbyNoaxE
And finally, for soccer enthusiasts:
http://www.youtube.com/watch?v=cUK1UobhCX0
http://www.youtube.com/watch?v=UePrOp2s31c
As a starting point, look for Image Morphing and View Morphing.
If you want, I can provide some more papers on the topic.
Good luck! :)
EDIT: Also, if you're interested, here's my work on Spatial and Temporal Interpolation of Multi-View Image-Sequence.
http://tobiasgurdan.de/research/