Archives for November 2013

Radical A/B Testing

Optimizely argues that A/B testing should be used to test radically different designs and that multi-variant testing should be used to test the combinations of smaller variations.

The main problem with A/B testing only small changes is that you don’t know what you’re leaving on the table. Small incremental changes most likely yield small incremental returns. But radical changes can yield radical returns. Changing a button color may get you a 3% increase, but changing your entire page may get you a 100% increase.

This article summarizes it well:

Button color tests. Font size tests. Headline tests. Run them all you want. But remember that your website’s conversion rate is only as limited as the risks you take. Small tweaks = small wins. If you’re craving for big wins, you’ll have to make big changes.

Many people argue that you should start with radical and move to incremental.

Here’s another argument for radical A/B testing. Here’s one from Moz’s Rand Fishkin, in which he advises, “Start with the big ideas and big projects, nail down the grand plans worth testing, let your audience pick a winner and then try to tweak, tune and improve.”

A few other articles and tools with checking out:

What Our Marketing Is Missing

I’ve been pondering recently what’s missing in our marketing. There are many things we could do better, but the main thing I’ve been thinking about is this: no one knows who we are or what we stand for (and perhaps we don’t either).

When you think of successful companies that have devoted followers, you almost always think of companies that have a clear identity and take a clear stand for something. Apple and Google are both great example. Apple is a disruptor. Apple thinks different(ly) and believes products should be beautiful, intuitive, and just work. Google is a disruptor. Google believes in making things fast and free, while not being evil.

A company I’ve been intrigued by recently is Ting. They embody many of the same things. Ting is a disruptor. They believe the mobile phone industry is corrupt, and they believe people deserve a better alternative. As with Apple and Google, who Ting is and what Ting stands for are critical factors in Ting’s traction in a crowded market.

I had a great interview with a guy who’s applying to work for us, and our discussion landed on this subject. He pointed me to Simon Sinek’s TED talk on “How Great Leaders Inspire Action.” Sinek articulates a lot of the things I’ve been thinking about, better than I could. I’d strongly encourage you to watch it. It’s well worth your 18 minutes.

I haven’t read it yet, but you may also want to check out Sinek’s book, Start with Why: How Great Leaders Inspire Everyone to Take Action.

Thoughts? Agree? Disagree? What do you think we’re missing?