Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> The purpose of an A/B test isn't to always show the best performing result

The key part is 'not always'.

Using A/B tests just to find the better performing version is a valid use case too. And if you have the traffic, multi-armed bandits are a nice way of automating the whole procedure. Even scientifically speaking there is nothing wrong with them. Their biggest issue is that they require a lot of traffic for significant results.



In practice, MAB should require less traffic than an A/B test for the same level of significance. Although it's much more difficult to rigorously describe that significance level with a MAB.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: