I am trying to determine the lag time between seeing a dip (or rise) in a predictor metric and when we would see the resulting dip (or rise) in a known response metric. I am not sure where to start, could someone put me on the right path?
For context, I would like to use R or Python and am familiar with statistics and machine learning. I am just searching for what method or modeling technique would be best to use and less about the code.
Related
So checking for Sample Ratio Mismatch is good for data quality.
But in Google Optimize i can't influence the sample size or do something against it.
My problem is, out of 15 A/B Tests I only got 2 Experiment with no SRM.
(Used this tool https://www.lukasvermeer.nl/srm/microsite/)
In the other hand the bayesian model deals with things like different sample sizes and I dont need to worry about, but the opinions on this topic are different.
Is SRM really a problem in Google Optimize or can I ignore it?
SRM affects Bayesian experiments just as much as it affects Frequentist. SRM happens when you expect a certain traffic split, but end up with a different one. Google Optimize is a black box, so it's impossible to tell if the uneven sample sizes you are experiencing are intentional or not.
Lots of things can cause a SRM, for example if your variation's javascript code has a bug in some browsers those users may not be tracked properly. Another common cause is if your variation causes page load times to increase, more people will abandon the page and you'll see a smaller sample size than expected.
That lack of statistical rigor and transparency is one of the reasons I built Growth Book, which is an open source A/B testing platform with a Bayesian stats engine and automatic SRM checks for every experiment.
How to determine sample size given there is 20% point reduction [ before change – after change =20% ] with 95% confidence level and 90% power ? Any pointer on how to solve this
A good first step is always to think about, what kind of test you plan to use. From the very little information you give a paired t-test (or a single measurement t-test comparing the difference to zero) is a likely candidate.
You can now google for "statistical power of t test" to which you can add the name of any computer language or statistics software you plan to use. Except maybe for educational purposes I'd advise to compute statics not by hand but via software.
Kind of an obvious option for statistic software on stackoverflow might be R. In Ryou'll find solutions to many sample size or power calculations in the package pwr. Here is the link to a getting started text: https://cran.r-project.org/web/packages/pwr/vignettes/pwr-vignette.html
The pwr.t.test function is good for your problem. Google will readily help you to alternatives for Python and Julia and SPSS I assume for C++, Java and Javascript as well.
However you will have to make assumptions about the variance or the effect size. Will each value be reduced by almost exactly 20% or will some be reduced a lot and some increase? That is of utmost importance to the question. You will need only one observation if there is no variance, a small amount of observations if there is little variance and a large amount of observations if there is lots of variance.
I've been working with a ClusterGAN, originally based on the one here: https://github.com/zhampel/clusterGAN
While trying to stabilize training on my data I integrated a bunch of current strategies, like making it fully convolutional, including spectral norm, using warm restarts, and adding instance noise. Though I was able to solve problems I was having with mode collapse, and I can get useful output images, it will eventually diverge and I'm wondering what might be going on (see chart below). To be clear, the output of the model is actually quite usable around epoch 69 (in this run), right before the last restart, but I'm still curious what might be wrong, partly for my own understanding, but also to inform future exploration. Is this overfitting, perhaps?
Worth noting, perhaps, is that a larger version of the model—just using more filters—did tend to diverge later in the training, but would still diverge eventually (and didn't necessarily produce clearly "better" images). I'm running on mobile, so I'd like to make the model as small as possible, both to improve inference speed and keep app size down.
UPDATE: Increasing the initial learning rate stabilized training for this number of epochs (though I'm curious whether it would still diverge if I ran it through another warm restart cycle).
I cannot figure out what's wrong. I mean, the speed is way too fast, like 1 million items vs 10 million items basically have the same 0.0005 second computation on my machine. So fast, it looks like it wasn't doing anything. But the result of the data is actually correct.
It is mind boggling because if I make similar computation on sequential loop without storing the result in an array, it is not only number of cores slower, but, like 1000 times slower than ArrayFire.
So, maybe I wasn't using the timer correctly?
Do you think they didn't actually compute the data right away? Maybe it just sets up some kind of shadow marker? And when I call the myArray.host(), it will start doing all the actual computations?
From their website, it says there is some kind of JIT to bundle the computations.
ArrayFire uses Just In Time compilation to combine many light weight functions into a single kernel launch. This along with our easy-to-use API allows users to not only quickly prototype their algorithms, but also get the best out of the underlying hardware.
I start/stop my timer right before/after few ArrayFire computations. And it is just insanely fast. Maybe I test it wrong? What's the proper way to test ArrayFire performance?
Never mind, I found out what to do,
Based on the examples, I should be using af::timeit(function) instead of using the af::timer. Using af::timeit will be very slow, but, the result scale more reasonably when I increase the size 10x. It doens't actually compute right away, that's why using af::timer myself wouldn't work.
thank you
So I am working on Note Onset Detection. I have implemented the method here: Note onset detection
However, I am finding some difficulty or problems regarding the 'static' nature of the method. What I am looking for is how to make the thresholding method 'dynamic'. But I am finding trouble finding suitable solutions.
Aside from that, I am also working on instead of having the amplitude value as the basis of passing the threshold, I make use of the 'difference' between 2 amplitude values, to know when a signal increases or not, and how much it increased or decreased. This is what I'm using currently.
Anyone willing to help or has worked with this kind of problem? Thank you!
Additionally, by any chance do any of you have a PDF file of this paper: http://www.mendeley.com/research/methods-detecting-impulsive-noise-speech-audio-signals-14/
Volume compression is a form of AGC (Automatic Gain Control), and AGC can be done dynamically. The are plenty of close to real-time AGC algorithms to be found in search results, although a bit of delay is required if you want an AGC attack that's smoother than a step function.