So checking for Sample Ratio Mismatch is good for data quality.
But in Google Optimize i can't influence the sample size or do something against it.
My problem is, out of 15 A/B Tests I only got 2 Experiment with no SRM.
(Used this tool https://www.lukasvermeer.nl/srm/microsite/)
In the other hand the bayesian model deals with things like different sample sizes and I dont need to worry about, but the opinions on this topic are different.
Is SRM really a problem in Google Optimize or can I ignore it?
SRM affects Bayesian experiments just as much as it affects Frequentist. SRM happens when you expect a certain traffic split, but end up with a different one. Google Optimize is a black box, so it's impossible to tell if the uneven sample sizes you are experiencing are intentional or not.
Lots of things can cause a SRM, for example if your variation's javascript code has a bug in some browsers those users may not be tracked properly. Another common cause is if your variation causes page load times to increase, more people will abandon the page and you'll see a smaller sample size than expected.
That lack of statistical rigor and transparency is one of the reasons I built Growth Book, which is an open source A/B testing platform with a Bayesian stats engine and automatic SRM checks for every experiment.
photos in washington VS show me photos in washington VS I wanna see all my photos in washington taken day before yesterday
what:photos
entities:washington (dont want to be too assuming)
when: 2013-03-14
I want to parse preset queries into conditions (like above). I want these qualities:
I can extract relevant terms even in presence of fluff ("I wanna see) and lowercase nouns
warm program can accept requests over HTTP or allow me to add some network communication
warm program responds in 50ms and needs atmost 500Mb of memory for reasonable sentences
I am more experienced in Python, less so in Java
Parser data structure is easy to handle
I use NLTK, but its slow. I see StanfordNLP and OpenNLP as viable alternatives but I find the program-start latency to be too high. I dont mind integrating them over servlets if I am left with no alternative.
The Stanford Parser is a solid choice, and pretty well-supported (as research code goes). But it sounds like low latency is an important requirement for you, so I'd also suggest you look at the BUBS Parser (full disclosure - I'm one of the primary researchers working on BUBS).
I haven't compared directly to NLTK, but I think you may find that the Stanford Parser doesn't meet your performance needs. This paper found a total throughput of ~60 words/second (~2-3 sentences/second). Those timings are pretty old, so newer hardware will certainly improve that, but probably still won't come close to 50 ms latency.
As you note, startup time will be an issue with any parser - a high-accuracy model is necessarily quite large. And 500 MB is probably pretty tight too (I usually run BUBS with 1-1.2 GB). But once loaded, BUBS latency is generally in the neighborhood of 10 ms per sentence (for ~20-25-word sentences), and we can push the total throughput up around 2500 words/second before accuracy starts to drop off. I think those numbers might meet your performance needs, and I don't know of any other high-accuracy (F1 >= 88-89) parser that comes close in speed.
Note: the fastest results are with recent pruning models that aren't yet posted to the website, but I can get you a model if you need. Hope that helps, and if you have more questions, feel free to ask.
I am currently helping someone with a reaction time experiment. For this experiment reaction times on the keyboard are measured. For this experiment it might be important to know, how much error could be introduced because of the delay between the key-press and the processing in the software.
Here are some factors that I found out using google already:
The USB-bus is polled at 125Hz at minimum and 1000Hz at maximum (depending on settings, see this link).
There might be some additional keyboard buffers in Windows that might delay the keypresses further, but I do not know about the logic behind those.
Unfortunately it is not possible to control the low level logic of the experiment. The experiment is written in E-Prime a software that is often used for this kind of experiments. However the company that offers E-Prime also offers additional hardware, that they advertise for precise reaction-timing. Hence they seem to be aware about this effect (but do not tell how large it is).
Unfortunately it is necessary to use a standart keyboard, so I need to provide ways to reduce the latency.
any latency from key presses can be attributed to the debounce routine (i usually use 30ms to be safe) and not to the processing algorithms themselves (unless you are only evaluating the first press).
If you are running an experiment where millisecond timing is important you may want to use http://www.blackboxtoolkit.com/ to find sources of error.
Your needs also depend on the nature of your study. I've run RT experiments in Eprime with a keyboard. Since any error should be consistent on average across participants, for some designs it is not a big problem. If you need to sync up the data though with something else (like Eye tracking or EEG) or want to draw conclusions about RT where specific magnitude is important then E-Primes serial resp box (or another brand, though I have had compatibility issues in the past with other brand boxes and eprime) is a must.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Ethical Dilemma: Should I still cater for IE6 as a web-developer.
5% of IE users last month use IE6. So I am curious if people think I should support it....because I haven't been.
http://www.w3schools.com/browsers/browsers_explorer.asp
I wouldn't..Its not worth your time and effort for 5%.
What is your product or service?
What percentage of conversions come
from that 5% of ie users?
What percentage would come from
those users if your site had
better support for their
browser?
If you answer those questions, you won't have to ask anybody else whether you should support IE6 for your site or not.
No. But someone is going to say you will lose customers if you don't.
It's time for IE6 to die, already. Those of you who are still using tube monitors and abacuses, please give us a break and come into the 21st century.
As Bobince points out in his comment above, it's unrealistic to put a coat of wax on a VW Beetle, and expect it to look more like a Porsche.
Businesses tend to keep IE6 on as legacy applications might only support IE6 so you tend to find they are in no hurry to roll out newer versions of IE as they would have to update in-house applications.
Probably not, but if your consumers consist of people that use old browsers, it might be a good idea.
It's not that they're 5% of your user base, they're 5% of your IE user base. In otherwords, IE6 accounts for 5% of 30% of your traffic.
I would say no, because you'd be sacrificing too much time and features for too few people. That and big websites like youtube and firefox have dropped support already.
IE6 users are almost exclusively corporate users whose update schedule is mandated by the corp. If your userbase is the general public then that 5% figure will be much lower and as such I dont think supporting IE6 is worthwhile.
I personally loathe background music on a website. My client has opposite feelings on the subject. I added music because the customer is always right, though I'd like to revisit the subject with them.
Almost everyone would agree that it is annoying and wastes precious bandwidth but are there any usability studies or a recommendation for someone esteemed in the profession that can provide a valid argument against background music?
Usability is not the only concern. Consider the following scenarios:
1 - Someone browses to the site while at work in a shared office, and now all of their co-workers think "Gee, he's wasting time".
2 - Someone browses to the site while in a room with a sleeping baby, and now they have to spend an hour getting him/her back to sleep.
3 - Someone browses to the site while they are listening to their own music, and now they hear a cacaphony of shrieks until one source is muted.
Also, consider that any benefit gained from the music on your website will be totally lost on anyone who has their speakers muted. So your audience can be divided between:
A - People who cannot hear the music
B - People who can hear it, but do not like it
C - People who can hear it, and do like it
I would not care to estimate the percentages associated with each of these groups, but keep in mind that category "B" is actively offended by your website. To take a line from the hippocratic oath, one rule of web design should be "do no harm".
Metrics. You'll never be able to convince a business person with an emotional answer.
If you investigate the situation empirically you'll be able to give them something irrefutable.
I would would try an experiment: (get google analytics)
have one site with the music as-is, measure the bounce rate,etc
have an identical site without music, measure the bounce rate,etc
Have the server randomly serve up the different pages for a couple weeks (until you get a significant data) and see what happens.
Maybe we're wrong (I hate music too). I hope your customer is wrong, but who knows.
You could also add a survey link and try to get people to answer that as well (but without an incentive that might not work)
Stats can be your friend here :)
I would also:
(calculate the size of the audio file(s)*the number of hits*months)/cost of GB per month
Then tell them how much money they are wasting.
Basically, it boils down to this:
Audio on websites is a bad idea. No one likes it.
Try to educate your client that it is a bad idea. (It's annoying, different levels of sound can cause problems, yadda yadda) Mention that most users don't take sites seriously if they use sound. It's a very '99 thing to do.
If you client does not budge, (politely) remind him/her that they are paying you for your expertise as an internet professional. You are the expert on the web, and they have hired you to give your expertise.
If they still won't budge, keep the sound and make sure they are happy. The bottom line is keeping the client happy.
Music also interferes with screen reader users. I'm a blind computer user and nothing annoys me more then having music start playing and drowned out my speech program that's trying to read the site. Nothing will make me close a website quicker then unwanted audio.
It took a bit but I found a site that talks about usability on web sites.
They have a video on the right hand side of this page:
http://www.ciaromano.com/evaluating/testing.php
It shows why audio ads are not a good idea on websites.
Hope this helps.
G-Man
Just make sure that there is a way to turn it off. It really depends on the type of Website, because multimedia-heavy sites (i.e. sites for Movies or Games) can benefit from it, but if I'm listening to some of my own music, I definitely want a way to turn it off.
Oh and please, no crappy MIDI-Files that people already hated in 1993 when they were novel.
This is a tough one -- and what's amazing is that at the moment, I have a client who's demanding the exact same thing.
Personally I don't know of any usability studies addressing this topic specifically, but there's plenty of anecdotal evidence out there from users complaining about the intrusiveness or outright corniness of unrequested background music. * That said, clients still ask for it. Best you can do is try to explain the situation to them, try to gather a few good examples of people complaining about it from the Web at large, build a case, and hope the client goes for it.
In my case, she completely agrees that it's potentially annoying, understands it cuts against the grain of user expectations and politeness, but wants it anyway. So I'm building it. Whaddyagonnado.
* Indeed, you could probably use this thread as evidence! Good luck.
Consider taking a different path with the client.
Ask them what the purpose for the music is...
If it is to install a particular feeling or mood with the visitor of the site, consider taking them through all the points mentioned in answers here and discuss how that may violate the intended for the music.
Then you will be able to talk to the client about different ways to instill the same "ambience" to the website without resorting to music. This is really a design issue and not usability.
If the background music/sound was to convey some information, then it is a usability issue as people who for technological or biological reasons cannot hear the sound at the correct volume will miss out on that. Therefore the site is not as usable as it should be.
Unfortunately, as a service provider of sorts, all we can do is cringe and give the customer what they want - after documenting your disapproval both commented in the code and in writing to the client, of course.
Pardon me, but i have a different opinion about loading music in the website. With all due respect I have for the answer posters of this thread.
I see visits to e-commerce websites like going to a shopping complex. Where you have a cart, varieties of products, checkout counters and background music to make your stay as comfortable and interesting as possible.
There's a whole psychological reason as to what certain slow paced music can do to certain parts of the brain. Some studies even suggested that certain music play a role in motivating customers to purchase more items. Check this site
This can definitely be a plus point in a website. Of course it depends on what kind of website it is. However, a slow and non-vocal music shouldn't necessarily disrupt one's attention; rather it might have the opposite effect.
My justification is that when a potential customer visits a site, he is only using one of his senses while browsing through the pages. His eyes! I'm saying why not allow him (if he wants) to use his sense of hearing that would encourage him (not only through the means of displaying fancy texts, design and animations that looks nice to the eyes) but also to capture his attention through music (allowing him to be more in touch with the site).
Its obviously not possible to trigger his sense of smell and taste. But why limit it to only the eyes. Why not use the ears too!
Whether you choose to put music into your site or not, MichaelStum's post about having an option to turn off the music is highly essential.
Of course in the end its all about the amount of traffic that comes to your website. For this matter, #Cbrulak's idea of using Google Analytics would be a realistic approach for different individuals.