Arggg - A/B testing gone wrong...

[Tomer]

Last week I designed a new product page for my site [landing page] as that is the first page users see when coming from other sites and that's the page that decides if they buy or not [pretty much], so I designed a new one and used A/B testing to send 50% to each, monitoring the results.

A week later, nothing! The results are exactly ~50/50%. How can that be? To me, the two pages [page A, page B] are very different, how can they have the same results? Weird huh.

So I guess it's up to me to pick the one I like... Maybe I'll leave it a bit longer to see results.

[Dan Grossman]

Better than finding out one doesn't work and the traffic you sent there was wasted.

[Tomer]

Guess so, but I personally thought the second one would bring in MUCH better results.

[Dan Grossman]

I don't see why you would expect very different results... same product, same product image, same product copy, almost exactly the same text around the product, same prices.... very little is different between the pages.

[Tomer]

Guess you're right - I thought just the little different design would make a difference.

- Tomer

[lerxtjr]

Better than finding out one doesn't work and the traffic you sent there was wasted.

I would LOVE for one page to make sales and the other to make none. That way you would have a "control" page to use in continued split testing.

You're missing the opportunity of what split testing is all about. You always need a control page (similar to having a control "group" when testing a new pharmaceutical drug in that you need one group that takes the real drug and the other group gets a placebo.)

Once you have a control group with a proven conversion rate, make a change to one thing, most commonly the headline. You could spend the next six months seeing different conversion rates just by testing new headlines. But, just change one or two things at a time.

When you get some tests that prove better than your control page, add those changes to your control page.....and then continue testing other things.

So, your control page always evolves but only with changes that have been proven from your test page.

[Tomer]

Yeah thats what I did - I copied the regular page [#2] and changed the upper part.

- Tomer