Home / Internet Marketing News / The Importance of Testing: How one test applied to two email sends resulted in different audience responses

The Importance of Testing: How one test applied to two email sends resulted in different audience responses

At MarketingExperiments, sister company of MarketingSherpa and parent company of MECLABS, we believe in testing. Our Web clinics are a testament to this belief – every month we share research that is designed to help marketers do their jobs better.

This culture of testing encouraged me to run my own test.

First, I ran the test on a MarketingSherpa send. After the results of that test came back, the same test was then applied to a MarketingExperiments’ newsletter.

By running virtually the same test twice for two different audiences not only did I learn more about the preferences of ourreaders, but I also learned how incredibly important it is to test, even when you are sure you know what the results will be.

 

The MarketingSherpa test

As the copy editor at MECLABS, I get to see all of the copy produced by both MarketingExperiments and MarketingSherpa. One of my responsibilities is overseeing the email newsletter sends. Every Monday, MarketingSherpa sends out a Best of the Week newsletter which features the most popular articles of the previous week and introduces the new book for the MarketingSherpa Book Giveaway.

The copy in these newsletters was extensive. Every article listed had a full summary which, as a reader, I imagined would seem overwhelming when opening this email on a Monday morning.

With the help of Selena Blue, Manager of Editoral Content, and Daniel Burstein, Director of Editorial Content, I decided to change reduce the length of the Best of the Week newsletter and use an A/B split test to determine which format our readers preferred.

Below is a side-by-side of the Control and the Treatment. 

 

After running this test for six weeks, there was a definite trend of higher total and unique clickthrough happening for the shorter treatment versions. My hypothesis proved to be true — less copy in the email meant higher engagement rates with readers.

 

The MarketingExperiments’ test

Based on the success of the MarketingSherpa test, I thought it would be interesting to run the same test on MarketingExperiments’ Best of the Month newsletter send, which is sent to readers on the first of every month. Similar to the Best of the Week email send, the Best of the Month features the most popular articles of the month, gives a recap of the latest Web clinic and provides a description of the upcoming Web clinic. This was done with the help of Ken Bowen, Managing Editor, MarketingExperiments.

The Best of the Month send has more information in it, so it is naturally a longer newsletter than the Best of the Week email previously tested. Therefore, it felt like a natural mental leap to assume that the shorter Treatment would once again get higher clickthrough rates as it seemed to be more reader-friendly.

As you can see with the below Control of the Best of the Month from August, there was a lot of copy in these sends.

 

For the Treatment, we once again cut the article summaries. In addition to losing this copy, we also cut the recap of the previous month’s research and instead just had the upcoming month’s Web clinic description. We ran this A/B split test for three months.

 

Due to the results of the Best of the Week testing, I knew what the results of this testing would be. If one audience responded positively the lessened copy, certainly so would another audience. The fact that we were virtually re-running the same test seemed to me to just be a formality.

Spoiler Alert: This is why testing is so imperative.

If I had applied the results from one test to all newsletter sends, I would have been doing our readers a disservice. Below are the results broken down by month.

 

Of the three months of testing, only one month validated. Although the Treatment had slightly higher clickthrough rates, it wasn’t significant enough to regularly validate as it did with the Best of the Week send. It appears that readers of the Best of the Month newsletter may find value in the summaries and extended copy that the Control version had.

Testing allowed us to change the MarketingSherpa Best of the Week send to better serve our readers and also learn that the same changes made to the Best of the Month send did not make a statistically significant difference to MarketingExperiments’ readers.

 

Results

Testing is so important. I work for a company that values testing and is continually striving to best suit the needs of our readers. The editors and directors overseeing both MarketingSherpa and MarketingExperiments were very open to my experimenting with the format of a send that has been with the company much longer than I have.

And while the change of the format itself is important, so was the testing involved. It allowed me to really see how our readers were digesting the information that I was sending to them on a weekly or monthly basis. To better understand the patterns and behaviors of our audience only benefits me as a copy editor.

This meant that one audience preferred the lessened copy while the other may find value with more text . Without re-running the test, I would have blindly assumed that the MarketingExperiments audience had the same reading preferences as our MarketingSherpa audience.

Testing allows us to validate even that which we are sure to be true, and to humble us when this in fact turns out to be false.

 

You might also like

MarketingSherpa Summit 2016 — At the Bellagio in Las Vegas, February 22-24

A/B Testing: Ecommerce site’s 3,000 positive comments show why you can’t trust just one test

Testing and Optimization: Welcome send test results in 46% open rate for CNET

Click Here For Original Source Of The Article

Ads by WOW TRK

About Daniel Rodgers

A lot of news that you will not see in the paper. A lot of technology that is coming out that will not see in the paper.

Check Also

Digital Marketing News: Coronavirus’ Impact on Marketing Budgets & Social Media Usage, LinkedIn’s New Conversation Ads, & Instagram Story Changes

2020 March 27 MarketingCharts Chart

2020 March 27 MarketingCharts Chart

LinkedIn rolls out Conversation Ads, aimed at improving personalization in messaging
LinkedIn has begun launching its new Conversation Ads, with real-time engagement ad options that include multiple calls-to-action. Conversation Ads are rolling out to all LinkedIn (client) advertisers over the next several weeks, the firm recently announced. Marketing Land

Social Media Consumption Grows Amid Coronavirus
Social media usage has sharply increased in the midst of the COVID-19 pandemic. Certain Instagram #ad campaign post likes have seen an increase of up to 76 percent during the past two weeks, as the majority of workers have shifted to temporary remote work. AList

CX in B2B Marketing: Top-of-Mind Strategy in 2020
54 percent of B2B marketers say their organizations are focusing most on customer experience and loyalty in their marketing efforts, according to newly-released report data examining global B2B customer experience priorities. MarketingProfs

Coronavirus: Emarketer lowers global ad spend projections for 2020
Worldwide ad spending will decrease due to the COVID-19 pandemic, with a previously expected $712 billion total being revised to $691 billion for 2020, a 3 percent decrease, according to newly-released forecast data, with many expecting even sharper forthcoming ad spend decreases. Marketing Land

B2B Senior Marketer Survey: The Most Effective Approaches for 2020 [Infographic]
LinkedIn (client) is tops among senior B2B marketers for nurturing leads, with some 84 percent saying the Microsoft-owned platform is the most effective social media channel, followed by Facebook at 43 percent and YouTube at 25 percent, according to recently-released survey data of interest to B2B marketers. MarketingProfs

Facebook Warns of Ad Approval Delays Amid Staff Changes Due to COVID-19 Impacts
Facebook was among the first social media platform to announce that the COVID-19 pandemic would affect how it reviews and approves certain content including ads, with a full slate of Facebook monetized content options being impacted. Social Media Today

2020 March 27 Statistics Image

Sales And Marketing Are Misaligned As Email Lags In B2B Nurture Mix: Study
While some 64 percent of marketing teams said that their organizations use account based marketing (ABM) for at least half of their marketing, 90 percent consider it challenging to successfully use it across multiple channels including sales — two of several findings of interest to digital marketers in recently-released survey data. MediaPost

Instagram's Testing the Capability to Add Hashtags and Location to Stories Highlights
Marketers may soon be able to add location information and up to four hashtags to the Highlights option of Instagram Stories, which stay active beyond the usual 24-hour active period, according to recent test observations — changes that could eventually bring an array of new targeting options to digital marketers' toolkits. Social Media Today

Amazon could win big in the post-coronavirus retail economy
Marketers may swivel more ad dollars to Amazon, which has seen sizable demand increases in response to the COVID-19 pandemic. The firm announced that it was hiring more than 100,000 new workers, and a jump in ad revenue also appeared likely as some marketers have shifted more spend to the e-commerce giant. Marketing Land

Data Hub: Coronavirus and Marketing
28 percent of industrial marketers plan to shift marketing budgets from trade shows cancelled due to COVID-19 to digital advertising instead, while 14 percent will allot that spending to content marketing — one of several statistics of interest to digital marketers contained in newly-released report data from MarketingCharts. MarketingCharts

ON THE LIGHTER SIDE:

2020 March 27 Marketoonist Comic

A lighthearted look at how not to communicate right now by Marketoonist Tom Fishburne — Marketoonist

Diabolical Mailing List Admin Twirling Mustache and Cackling at Dead “Unsubscribe” Link — The Hard Times

TOPRANK MARKETING & CLIENTS IN THE NEWS:

  • SAP, Dell Technologies — 3 Tips For B2B Marketers When Your Big Trade Show Is Cancelled Due To Coronavirus — Forbes
  • SAP — Event industry and COVID-19: An insider’s point of view — The Future of Commerce
  • Lee Odden — Marketing During a Pandemic – Resources for Small Businesses in the Coronavirus Crisis [Roundup] — Simple Machines

Do you have your own top B2B content marketing or digital advertising stories from the past week? Please let us know in the comments below.

Thank you for taking the time to join us, and please return again next Friday for a new selection of the most relevant B2B and digital marketing industry news. In the meantime, you can follow us at @toprank on Twitter for even more timely daily news. Also, don't miss the full video summary on our TopRank Marketing TV YouTube Channel.

The post Digital Marketing News: Coronavirus’ Impact on Marketing Budgets & Social Media Usage, LinkedIn’s New Conversation Ads, & Instagram Story Changes appeared first on Online Marketing Blog - TopRank®.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

css.php