Home / Internet Marketing News / A/B Testing Prioritization: The surprising ROI impact of test order

A/B Testing Prioritization: The surprising ROI impact of test order

I want everything. And I want it now.

I’m sure you do, too.

But let me tell you about my marketing department. Resources aren’t infinite. I can’t do everything right away. I need to focus myself and my team on the right things.

Unless you found a genie in a bottle and wished for an infinite marketing budget (right after you wished for unlimited wishes, natch), I’m guessing you’re in the same boat.

When it comes to your conversion rate optimization program, it means running the most impactful tests. As Stephen Walsh said when he wrote about 19 possible A/B tests for your website on Neil Patel’s blog, “testing every random aspect of your website can often be counter-productive.”

Of course, you probably already know that. What may surprise you is this …

It’s not enough to run the right tests, you will get a higher ROI if you run them in the right order

To help you discover the optimal testing sequence for your marketing department, we’ve created the free MECLABS Institute Test Planning Scenario Tool (MECLABS is the parent research organization of MarketingExperiments).

Let’s look at a few example scenarios.

Scenario #1: Level of effort and level of impact

Tests will have different levels of effort to run. For example, it’s easier to make a simple copy change to a headline than to change a shopping cart.

This level of effort (LOE) sometimes correlates to the level of impact the test will have to your bottom line. For example, a radical redesign might be a higher LOE to launch, but it will also likely produce a higher lift than a simple, small change.

So how does the order you run a high effort, high return, and low effort, low return test sequence affect results? Again, we’re not saying choose one test over another. We’re simply talking about timing. To the test planning scenario tool …

Test 1 (Low LOE, low level of impact)

  • Business impact — 15% more revenue than the control
  • Build Time — 2 weeks

Test 2 (High LOE, high level of impact)

  • Business impact — 47% more revenue than the control
  • Build Time — 6 weeks

Let’s look at the revenue impact over a six-month period. According to the test planning tool, if the control is generating $30,000 in revenue per month, running a test where the treatment has a low LOE and a low level of impact (Test 1) first will generate $22,800 more revenue than running a test where the treatment has a high LOE and a high level of impact (Test 2) first.

Scenario #2: An even larger discrepancy in the level of impact

It can be hard to predict the exact level of business impact. So what if the business impact differential between the higher LOE test is even greater than in Scenario #1, and both treatments perform even better than they did in Scenario #1? How would test sequence affect results in that case?

Let’s run the numbers in the Test Planning Scenario Tool.

Test 1 (Low LOE, low level of impact)

  • Business impact — 25% more revenue than the control
  • Build Time — 2 weeks

Test 2 (High LOE, high level of impact)

  • Business impact — 125% more revenue than the control
  • Build Time — 6 weeks

According to the test planning tool, if the control is generating $30,000 in revenue per month, running Test 1 (low LOE, low level of impact) first will generate $45,000 more revenue than running Test 2 (high LOE, high level of impact) first.

Again, same tests (over a six-month period) just a different order. And you gain $45,000 more in revenue.

“It is particularly interesting to see the benefits of running the lower LOE and lower impact test first so that its benefits could be reaped throughout the duration of the longer development schedule on the higher LOE test. The financial impact difference — landing in the tens of thousands of dollars — may be particularly shocking to some readers,” said Rebecca Strally, Director, Optimization and Design, MECLABS Institute.

Scenario #3: Fewer development resources

In the above two examples, the tests were able to be developed simultaneously. What if the test cannot be developed simultaneously (must be developed sequentially) and can’t be developed until the previous test has been implemented? Perhaps this is because of your organization’s development methodology (Agile vs. Waterfall, etc.), or there is just simply a limit on your development resources. (They likely have many other projects than just developing your tests.)

Let’s look at that scenario, this time with three treatments.

Test 1 (Low LOE, low level of impact)

  • Business impact — 10% more revenue than the control
  • Build Time — 2 weeks

Test 2 (High LOE, high level of impact)

  • Business impact — 360% more revenue than the control
  • Build Time — 6 weeks

Test 3 (Medium LOE, medium level of impact)

  • Business impact — 70% more revenue than the control
  • Build Time — 3 weeks

In this scenario, Test 2 first, then Test 1 and finally Test 3, along with Test 2, then Test 3, then Test 1 were the highest-performing scenarios. The lowest-performing scenario was Test 3, Test 1, Test 2. The difference was $894,000 more revenue from using one of the highest-performing test sequences versus the lowest-performing test sequence.

“If development for tests could not take place simultaneously, there would be a bigger discrepancy in overall revenue from different test sequences,” Strally said.

“Running a higher LOE test first suddenly has a much larger financial payoff. This is notable because once the largest impact has been achieved, it doesn’t matter in what order the smaller LOE and impact tests are run, the final dollar amounts are the same. Development limitations (although I’ve rarely seen them this extreme in the real world) created a situation where whichever test went first had a much longer opportunity to impact the final financial numbers. The added front time certainly helped to push running the highest LOE and impact test first to the front of the financial pack,” she added.

The Next Scenario Is Up To You: Now forecast your own most profitable test sequences

You likely don’t have the exact perfect information we provided in the scenarios. We’ve provided model scenarios above, but the real world can be trickier. After all, as Nobel Prize-winning physicist Niels Bohr said, “Prediction is very difficult, especially if it’s about the future.”

“We rarely have this level of information about the possible financial impact of a test prior to development and launch when working to optimize conversion for MECLABS Research Partners. At best, the team often only has a general guess as to the level of impact expected, and it’s rarely translated into a dollar amount,” Strally said.

That’s why we’re providing the Test Planning Scenario Tool as a free, instant download. It’s easy to run a few different scenarios in the tool based on different levels of projected results and see how the test order can affect overall revenue. You can then use the visual charts and numbers created by the tool to make the case to your team, clients and business leaders about what order you should run your company’s tests.

Don’t put your tests on autopilot

Of course, things don’t always go according to plan. This tool is just a start. To have a successful conversion optimization practice, you have to actively monitor your tests and advocate for the results because there are a number of additional items that could impact an optimal testing sequence.

“There’s also the reality of testing which is not represented in these very clean charts. For example, things like validity threats popping up midtest and causing a longer run time, treatments not being possible to implement, and Research Partners requesting changes to winning treatments after the results are in, all take place regularly and would greatly shift the timing and financial implications of any testing sequence,” Strally said.

“In reality though, the number one risk to a preplanned DOE (design of experiments) in my experience is an unplanned result. I don’t mean the control winning when we thought the treatment would outperform. I mean a test coming back a winner in the main KPI (key performance indicator) with an unexpected customer insight result, or an insignificant result coming back with odd customer behavior data. This type of result often creates a longer analysis period and the need to go back to the drawing board to develop a test that will answer a question we didn’t even know we needed to ask. We are often highly invested in getting these answers because of their long-term positive impact potential and will pause all other work — lowering financial impact — to get these questions answered to our satisfaction,” she said.

Related Resources

MECLABS Institute Online Testing on-demand certification course

Offline and Online Optimization: Cabela’s shares tactics from 51 years of offline testing, 7 years of digital testing

Landing Page Testing: Designing And Prioritizing Experiments

Email Optimization: How To Prioritize Your A/B Testing

The post A/B Testing Prioritization: The surprising ROI impact of test order appeared first on MarketingExperiments.

Click Here For Original Source Of The Article

Ads by WOW Trk

About Daniel Rodgers

A lot of news that you will not see in the paper. A lot of technology that is coming out that will not see in the paper.

Check Also

Interactive Content Marketing: Why B2B Marketers Should Take Their Content from Boring to Bold

Why B2B Marketers Should Consider Interactive Content

Why B2B Marketers Should Consider Interactive Content

Show of hands, B2B marketers: How many of you know which Hogwarts house you belong in? Or which Disney princess best represents your personality and relationship ambitions?

via GIPHY

Don’t be shy. I’m a Gryffindor wizard through and through—and apparently, I’m more of a Jasmine than an Ariel. And I know all this thanks to the rise of interactive content.

From quirky quizzes to ROI calculators to guided eBooks, interactive content is a rising content marketing star. BuzzFeed is perhaps the most prolific example, creating dozens of quizzes each week that are making their way into social feeds and search results. (And almost all users reportedly finish them).

But why should B2B marketers consider adding interactive content to their mix?

Because B2B is often pegged as bland and boring. And in a crowded content market, not to mention the fact that buyers' content preferences are turning increasingly visual, interactive content is the next evolution.

But if that doesn't convince you, read on for a few more reasons why its time for B2B to embrace interactive content.

#1 – Interactive content is more engaging than static content—for the long-term.

Interactive content may seem a little gimmicky for some marketers—especially those in the B2B space. But the vast majority of marketers who use interactive content agree that it not only grabs attention, but can also hold that attention beyond that initial view.

In fact, according to the Content Marketing Institute (CMI) and Ion Interactive 2017 Interactive Content Study:

  • 87% of marketers agree that interactive content grabs the attention of the reader more effectively than static content
  • 77% of marketers agree that interactive content has reusable value, resulting in repeat visitors and multiple exposures
  • 73% of marketers agree that interactive content, when combined with other more traditional content marketing tactics, enhances message retention among their audiences.

So, if you’re aiming for awareness, engagement, and attention, interactive content holds incredible potential. But I’d also add that this is only true if you deliver quality, relevant content in an interactive format.

As my colleague Josh Nite points out: “[Interactive content is] absolutely designed to grab attention. But if your content provides value—if it’s worth paying attention to—interactive elements can help you bring in an audience.”

[bctt tweet="#InteractiveContent is absolutely designed to grab attention. But if your content provides value—if it’s worth paying attention to—interactive elements can help you bring in an audience. - @NiteWrites" username="toprank"]

#2 – Interactive content can differentiate you from your competitors.

Content has always been a foundational element of B2B marketing. Buyers don’t make hard and fast decisions. Instead, they do their research, weigh their options, and have multiple engagements with sales reps before they sign on the dotted line. Interactive content can help you make an impression and stand out in a crowed, competitive content landscape.

In addition, according to the aforementioned report, just 46% of marketers report using interactive content right now—which was flat year-over-year. And if history is any indicator, I’d wager that interactive content adoption among B2B marketers is far lower since the industry is typically slower to adopt new tactics.

But that won’t always be the case. Harnessing the opportunity right now has the potential to differentiate your B2B brand from the competition early on, showcasing your commitment to innovation.

For example, Prophix, a leading provider of corporate performance management (CPM) software solutions in the FP&A industry, wanted to drive awareness in a unique way around its report on the evolution of financial planning and analysis, as well as its solutions.

By repurposing its original research and adding influencer perspectives, we created an interactive quiz to help empower their audience to "crush" their jobs to succeed now and into the future.

Prophix Crush It Interactive Quiz

This anchor asset, which was promoted using a supporting mix of blog content, social amplification, email, and more, saw a view rate 6-times higher than the benchmark for a similar resource. In addition, the page where it lived garnered 3-times the average share rate. This unique approach to interactive influencer content made Prophix stand out from the competition and deliver a great resource that performed.

#3 – Interactive tools can provide you with exclusive data and analytics.

Savvy marketers are driven by data insights. And many of the interactive content tools you’d leverage for an asset come with their own analytics dashboards, allowing you to get near real-time data on how your audience is interacting and absorbing your content.

For example, Ceros, an interactive content software that simplifies the creation process, provides all the basic KPIs such as visitors, opens, and pageviews, as well as engagement metrics like time spent and interaction clicks. But they also track inbound referrals, social shares, video plays, and outbound link clicks.

Oh, and that data is viewable in its Analytics Dashboard within second of it happening.

Ceros Interactive Content Tool

While traditional analytics platforms and the data within them is invaluable, from my perspective, this more niche data can help uncover some insights that can help you refine your asset on the fly or consider how to improve other content types moving forward.

#4 – Interactive content can drive results at every stage of the funnel.

From educating buyers to creating customer loyalty, interactive content can serve a purpose (and drive results) at every stage of the funnel. Interactive content users report using the tactic for lead generation, lead nurturing, customer retention, and the list goes on.

And interactive is especially powerful, when combined with other tried-and-true content marketing tactics.

To help promote the Influencer Marketing 2.0 report based on research conducted by Traackr, Altimeter Group and TopRank Marketing, we partnered with interactive content platform Ceros to produce an Influence 2.0 interactive infographic.

Influence 2.0

The infographic was promoted via blog posts, social channels, email, and through the influencers that contributed. With calls to action embedded within the interactive infographic, this content succeeded at attracting over 1,700 prospective customers to download the full report with a 43% conversion rate.

What Opportunity Does Interactive Content Hold for Your B2B Brand?

Interactive content is here to stay. But the real opportunity doesn’t lay in the interactivity itself. The real value creation is in the excitement or connection that you can make with your audience, as well as the potential to hold their attention for long enough to engrain your message or inspire action.

[bctt tweet="The real opportunity with #interactivecontent doesn’t lay in the interactivity itself. The real value creation is in the excitement or connection that you can make with your audience. - @CaitlinMBurgess" username="toprank"]

So, B2B marketers. If you’re ready to break away from boring and drive better engagement, interactive content deserves your consideration.

How can you leverage interactive content? Check out our post featuring five ways of making marketing magic with interactive content.

The post Interactive Content Marketing: Why B2B Marketers Should Take Their Content from Boring to Bold appeared first on Online Marketing Blog - TopRank®.

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php