Talk:A/B testing/Archives/2019

External links modified
Hello fellow Wikipedians,

I have just modified one external link on A/B testing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
 * Added archive https://web.archive.org/web/20150921174256/https://developer.amazon.com/public/apis/manage/ab-testing/doc/math-behind-ab-testing to https://developer.amazon.com/public/apis/manage/ab-testing/doc/math-behind-ab-testing

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

Cheers.— InternetArchiveBot  (Report bug) 19:16, 23 June 2017 (UTC)

The rest of the story
I'm not asking a question for discussion here on the talk page, but rather asking for an article improvement.

Consider this scenario:

Web page "A" is seen by 100 people. 50 continue to purchase the product. But the analysis should not end there, because of the remaining 50, 25 come back the next week and purchase the product. Total sales: 75.

Web page "B" is seen by a different group of 100 people. 60 continue to purchase the product. Hurrah, page "B" is victorious! But the obnoxious aggressive advertising of page "B" is so annoying that the remaining 40 vow never to return. Total sales: 60.

How is this accounted for in A/B testing? 72.208.150.248 (talk) 15:54, 1 March 2017 (UTC)


 * You can set the sales as your conversion objective, so you'll see 75 conversions in Page A and 60 sales in page B, that's a matter of what you are meaning to optimize and it's very important to get this right, optimizing for the wrong metric can be very hurtful to your business.
 * --Sicarul (talk) 00:10, 1 July 2017 (UTC)

= Common Test Statistics

Gibbs sampling is not in any sense a test statistic. It's a simulation method, and it's generally not even going to be a sensible approach for simulating the distribution of whichever test statistic you might want to insert here; generally straight simulation will be easier and faster. It literally has no place in this table - its presence here is a category error. Further, if a distribution is actually unknown how are you going to write a Gibbs sampler (or indeed any kind of simulation) for it? Bootstrapping (or a permutation test, or some other form of resampling) could work as an approach if you had a statistic but bootstrapping wouldn't belong in the table either -- whatever test statistic you bootstrapped would. Glenbarnett (talk) 22:38, 7 May 2019 (UTC)