We frequently receive questions from our email subscribers asking marketing advice. Instead of hiding those answers in one-to-one email communication, we occasionally publish edited excerpts of some of them here as well as our sister publication, MarketingSherpa, so they can help other readers as well. If you have any questions, let us know.
Dear MarketingExperiments: Hi. I read a lot of your articles and see your videos. Your research techniques are based on A/B testing. What’s your opinion about design sprint and design thinking process and qualitative-based research.
Dear Reader: Thank you for your question. A/B testing is not mutually exclusive from the other techniques you mentioned. In fact, they are often paired together by practitioners who are either aware of specific methodologies incorporated by these techniques, or are simply engaging in many of the tactics in those methodologies without specifically following (or even being aware of) them.
Here is some information to answer your question paired with
some related resources so you can dig further.
Qualitative research can be valuable to help you build a
theory about your customer. You should then form hypotheses to test your
theories. That’s where A/B testing comes in. The qualitative research can also
help you interpret those test results.
So, for example, you might assume customers care more about the environmental aspects of a product than the luxury aspects of a product based on your qualitative research. You can run an A/B test to determine if you are correct. The qualitative research can then help you answer why customers have the preference. For example, if they care about the environmental aspects for a deeper reason — maybe they are concerned about getting cancer from chemicals — it can lead you to explore other ways to serve those customer motivations with your messaging and your products.
Some sources you might consider for your qualitative
- Social media — How to use social media to help discover why customers buy from you
- Online reviews — How to start discovering your value proposition with Yelp
- Focus groups — Procter & Gamble Reveals How a New Focus Group Tactic Solved Four Flaws in Their R&D Process
- Surveys — 6 Tips for Creating an Effective Survey
- Customer and employee interviews — Every Conversion Copywriter Should Be Skilled at Interviewing
- Usability testing — Usability Testing: 5 tips for augmenting A/B testing
You can build a database of your qualitative research discoveries to help you track your findings and prioritize what you would like to test.
While the MECLABS methodology employs quantitative methods to study objective evidence in pursuit of a robust Customer Theory, those quantitative methods are enhanced by qualitative methods. Quantitative and qualitative methods are not at odds with each other, rather they work better in tandem.
Design thinking is a methodology that focuses on the user. A methodology like design thinking can go hand-in-hand with customer-first marketing.
Key elements of design thinking include empathizing with the
prospective customer or user, defining the problem, ideating a solution,
prototyping that solution and then testing it.
Again, we see where A/B testing can fit into this
Design thinking is similar to the MECLABS Conversion Sequence Heuristic — a patented methodology to help you empathize with the customer, define the problem and come up with solutions. And ultimately you want to test to see how your suppositions (hypotheses) behave in the real world.
A design sprint incorporates many of the ideas we’ve already
discussed. It is in many ways a focused application of design thinking in a
condensed amount of time.
That focus can be essential to the modern marketer and
business person. We frequently find our time and attention splintered by all of
the tasks we’re trying to keep up with — from long-term strategy to immediate
fire drills. A surge of effort on a precise task has become a popular way of
attacking a specific problem or opportunity. It can be personally fulfilling as
well, bringing the team closer together and uniting everyone around a common
goal with a clear success metric — building something specific to solve a
As such, this sprint mindset isn’t unique to design or marketing. In fact, it is especially popular in information technology (IT) where practitioners, for example, use the Scrum project-management system for website redesign and software development.
If you are going to engage in a design sprint, it helps to
begin with insights about the challenges and opportunities for the product and/or
user experience based on some information. This is where qualitative research can come
It also helps to get an understanding of how the thing you
create performs with users and customers — this is where A/B testing comes in.
After all, a design sprint can be a good way to build something based on the
best internal gut thinking, external opinions and other data. But again, this should
lead to a hypothesis which you can ultimately test with real-world behavior.
While we never use the term or official methodology of a design sprint, MECLABS also engages in focused activities with business and marketing teams to get a specific, customer-focused output that can then be tested — our Quick Win Intensives and Value Proposition Workshops.
A common thread
I’ve found that some people in life prefer a specific
methodology or canonical approach to follow.
I’m more of a big-picture guy myself — what can I learn from different methodologies, approaches and belief systems? And how can I apply that in my life, in my career and in specific situations?
If we were to step back for a minute, there is a common
thread in all of the research tactics we’ve discussed:
- How can you focus on the other (i.e., user or customer)?
- How can you discover what will best serve the
other (which can be very different from your own opinion)?
- How can you structure your time, efforts and
resources to serve the other?
- How can you know if your suppositions and effort
were effective? (What is your single source or truth?)
Thanks again for your question and best of luck in wherever
your research journeys take you.
MECLABS Institute Online Testing on-demand certification course — Learn where to test, what to test, and how to turn basic testing data into customer wisdom