Have you ever heard this ad? "They laughed when I sat down at the piano - but when I started to play!…" If it sounds familiar, it may be because AdAge declared it to be arguably the best advertising headline in the 20th century. It was written in 1925 by copywriter John Caples back when something called "direct mail" and lengthy, entertaining sales letters were staples of marketing (By the way – he was advertising the US School of Music whose sales went through the roof due to his ad).
Fast forward 90 years to today’s internet marketing and anybody with a website and $100 for Google AdWords can try to sell you something. Today, we are going to take a core advertising principle, benchmark (i.e. blatantly steal) it, and see how to apply it to our process changes.
That principle is called A-B testing. You have been exposed to this all your life without realizing it. Especially prevalent in internet marketing, savvy marketers don’t have ONE version of a sign-up screen, they have MULTIPLE. When you click on an ad, you are randomly assigned to either the "A" page or the "B" page. Both do the same thing but may experiment with different wording, colors, etc. For example, click on these to see the difference of an experiment I have run: A-Page / B-Page. Marketers experiment with a variety of X-factors including wording, font, message tone, colors, position, calls to action, etc. It makes me want to take a nap when I see the full list of factors they try to optimize!
So what? If you and I don’t implement A-B testing when rolling out process change, we are missing one of the biggest learning opportunities of our careers!
Just think of all the possible applications. A-B testing can improve your training, reporting, presenting, communications, influence approaches, and so on. For example: In one organization I was conducting some basic process improvement training prior to a new process rollout. At the conclusion of one class, they applauded. That had not happened before and caught my attention. I was saying the same basic words in closing, but had inadvertently slightly changed my timing and emphasis on a couple of words. I later found this to be something repeatable and started experimenting with many other things. Many X-factors didn’t pan out, but others surprisingly did. We still learn something from the failures.
Your experiment may not be on an internet page. In any process change rollout, think of the factors you control. You might craft several versions of:
- Communications (different wording or senders, written vs. face-to-face, 1:1 vs. mass, ...)
- Process maps or forms (try one version for Office A and another for Office B)
- Reporting (frequency, content, delivery methods)
- How deadlines / performance monitoring are handled (loosely vs. tightly controlled, ...)
- Calls to action (different next steps; you find us vs. we’ll find you, ...)
- Training (classroom vs. self-paced; instructor; leader message – live vs. video, ...)
- Rollout timing (one office now, another later, ...)
- Rollout method (flash-cut, parallel, sequential, phased, etc.)
For my Lean Six Sigma friends who are now conjuring images of 2-sample t-tests, ANOVAs, and Full Factorial DOE designs – knock you self out. However, it need not be that scientific – a PhD in statistics is definitely not required. Start simple and see what your results are. This is a down and dirty journey – don’t let perfection get in the way of progress. In no time at all, you’ll be saying: "They laughed when I stood up to present, but when I showed my results!…"