Home / Internet Marketing News / 5 a/b test ideas for mobile gaming apps

5 a/b test ideas for mobile gaming apps

We talk a lot about a/b testing here on MarketingExperiments. What we don’t usually talk about is a/b testing for the mobile web…especially testing within mobile apps.

I thought we should change that. As I was scouring the web looking for mobile a/b tests, I found this 2-year old video by Amazon.

Apparently, Amazon Web Services (AWS) at one point had an a/b testing feature that is now closed
.
When they had testing, however, one app developer used it extensively and shared their experiences in a promotional video for the feature on Amazon. The developers were behind the game Air Patriots. Russell Caroll was the Senior Producer for the game and Julio Gorge was the Game Development Engineer. The game is a kind of aerial take on the classic tower defense game genre.

air patriots screenshot

Air Patriots

Now granted, this was a promotional video, but the content still speaks for itself. These guys had (and still have by the looks of it) a fairly successful mobile app and they ran some successful tests. It’s a great starting place for what you can test in your mobile app.

By the way, while Amazon has shut down its a/b testing feature, there are a lot of other tools for testing mobile apps that will accomplish the same thing the developers talk about in the video.

Test #1: What is the impact of ads on customer experience? (1:34)

The first thing the team tested was the impact of ads are on their customers. They wanted to make sure the ads did not harm the customer experience. So they tested a single ad in the main menu near the bottom of the screen.

air patriots menu ad

They found that the ads didn’t affect customer retention. This meant that they could insert ads and generate more revenue without hurting their customers.

Test #2: Will in-game ad placement affect customer retention? (2:56)

In the second test, the team put ads in the game screen.

air patriots in-game ad

In both the first and second tests, the ads had a little “X” that the customers could tap to hypothetically dismiss the ads. When they tapped, a pop up came up that told customers they could eliminate ads with any purchase in the game’s store.

In this test, there was again, no impact on customer retention, but there was a statistically significant increase in revenue.

Test #3: Simple game-circle icon test (4:20)

In this test, the team wanted to know whether an icon to the game-circle (Amazon’s game stats and leaderboards portal) would improve performance.

air patriots game-circle

It’s not clear which icon won, or even why this particular test was useful for the team, but they did get a favorable result, and the lesson they wanted to drive home was that simple changes like icons can make a difference. We’ve, of course found that to be the case in a large number of our tests on MarketingExperiments.

Test #4: Does game difficulty affect revenue? (4:58)

In this 4th test, Caroll made a mistake. He accidentally changed the game difficulty to make it about 10% harder. As a result, every metric that was important to them tanked.

air patriots metrics

The team of course fixed it as fast as possible, but it gave them an idea.

What would happen to revenue if they made the game easier?

So they ran a test that had 5 treatments: The control and then 4 difficulty levels that were easier than the control.

It turned out that the easiest difficulty performed the best. By making it easier, they players playing 20% longer and revenue went up 20%.

Test #5: When is the best time to have push-notifications for re-engagement in inactive players? (7:43)

The team then tested a push-notification that offered inactive players and incentive for picking the game back up.

air patriots push

They wanted to know when the best time to send the notification would be. So they tested a few different variables and found that the best time was 3 days after the last game play.

They also found that sending the notification 7 days after the last game play negatively impacted their performance metrics.

With these 5 tests and probably a few more that have been happening off the record, the team was able to develop a great app for their customers and steadily increase their revenue. At the end of the video Carol gives a few key takeaways for marketers who are a/b testing their apps.

You might also like:

Email Research Chart: Email opens trends on mobile devices in 2015

Mobile Marketing Chart: Amount of revenue from the mobile channel, by merchant type

3 A/B Testing Case Studies from Smart Brand-Side Marketers

Click Here For Original Source Of The Article

Ads by WOW TRK

About Daniel Rodgers

A lot of news that you will not see in the paper. A lot of technology that is coming out that will not see in the paper.

Check Also

Tales from the Trenches: How to Transition from Marketing Doer to Marketing Leader

I was roughly five years into my marketing career when I began managing my first direct report. It was the biggest challenge I faced yet. I was now being evaluated on the actions, successes, and failures of another person—and I also knew it was my responsibility to give them the support and tools they needed to have more successes than failures.

I felt as if I didn’t know how to influence, motivate, or persuade another person. But I was given the opportunity to try and to learn. I had a great group of bosses, mentors, and peers giving me advice, listening to my concerns or wins, and allowing me to make mistakes.

Quite a few years (and many direct reports) later, today I have a much better handle on how to manage a team. And as I’ve grown, I’ve learned that my job isn’t just to manage people, time, projects, or priorities, my job is to lead.

But it can be hard to make the transition from a “doer” to a leader. And the stakes are high. In fact, a recent study from TINYpulse found that nearly 50% of employees have quit a job because of a less than stellar manager. In addition, those who don’t feel recognized for their work are two-times as likely to be job hunting.

Whether you’re stepping into your first management role, moving onto middle management, or you have your eye on the CMO office, as a leader it’s your job to inspire, motivate, and grow a happy and high-functioning team. The insights below are designed to help guide you down a successful path to a fruitful career and happy, supported, and motivated employees. 

Tip #1: Understand the landscape

Whether you’re managing one team member or an entire department, you’ll be setting goals and playing an integral role in setting the marketing strategy your team is responsible for driving results with. But to do that, you must understand the broad and niche context in which your organization, department, or service line operates. This means getting to know your customers, prospects, and competitors more deeply, so you thoughtfully can guide and educate your team:

  • Seek out opportunities to hold monthly or quarterly one-on-one calls with your priority customers. Ask them what they value most about your organization or product, as well as where you can do better. 
  • Regularly research your competitors. Subscribe to emails, follow them on social media, and attend industry events where they might be speaking. This will give you unique intel that you can bring back to your team.
  • Get out of the marketing silo. Brainstorm with the sales team. Talk to your customer service team. These teams are intimately familiar with the challenges your customers and prospects face.

Tip #2: Set goals … and exceed them

Yes, you’ve probably be setting goals at all stages of your career. As an individual contributor, your goals were likely focused on what you could individually achieve. In a leadership role, you’re likely responsible for setting goals for your team that will ladder to corporate goals. If you are new to a leadership role, achieving goals that map directly to the success of the company, can be a quick win to build trust within leadership and grow your team and influence. 

  • Keep your goals top of mind. Discuss progress, roadblocks, and wins with your team, your boss, and other leaders. The more discussion around goals, the more likely you and your team are to remain focused and accountable on achieving them. 
  • Incentivize if you can. Big and small incentives can keep your team motivated to achieve their goals.
  • Make it a number. In my experience, setting and achieving a numerical goal has more impact on the organization and is generally more impressive than an accomplishment-based goal. For example, make the goal double MQLs, instead of rolling out a new marketing automation system. The marketing automation system is a stepping stone to reach the goal, not the actual goal. 
  • Set goals quarterly. Ninety days is long enough to achieve something big-ish, but short enough to keep you focused. We’ve found quarterly goals helps us track for the year and keep the team more motivated. 

[bctt tweet="The more discussion around goals, the more likely you and your team are to remain focused and accountable on achieving them. @Alexis5484" username="toprank"]

Tip #3: Focus on scalability

Once it’s time to step out of day-to-day execution and supervision and into leadership, you should focus more on optimizing and solving issues on a systematic basis, rather than local basis. When I was a new manager, I found myself constantly on the run putting out fires as they would pop up, instead of focusing on why it started and how to prevent it going forward.

  • Create make-sense processes. Identify the things your team does over and over again such as campaign launches, attending events, or adding new content to the website. These are replicable events that you can create process around and then optimize for efficiency, results, and so on.
  • Don’t feel like you have to stick to the status quo. Just because the marketing team has always had six copywriters, two content strategists, and an analyst, doesn’t mean that’s the ideal structure. Document the needs and functions of the organization and then map out the most make-sense roles to those needs. For the sake of the exercise, take the current situation out of it. You can employ a phased approach to get you from current situation to ideal. 

Tip #4: Shift the spotlight to your team

As you’re moving into leadership, you’re likely trying to build trust and show value to upper leadership, and it can be easy to lose focus on serving your team. Fostering a happy, well-functioning team is your top priority. Not only can you not do your job without them, but it is one of the best indicators of success to your boss and your boss’s boss. 

  • Shift how you find personal value from work. Most of us have moved into leadership, after being highly successful individual contributors and supervisors. As leaders, we must find more value from the task, result, or project we helped someone else achieve, rather than the work we did ourselves. 
  • Clear obstacles. Be transparent when you can; have your employees’ backs. These things build trust and create a secure, happy, and productive team. 
  • Cultivate the next round of leaders. Understand what your team wants to achieve personally within their careers within the next five or 10 years, and help them do that. As leaders, we should always be identifying and growing the team members who want to move to the next round in their careers. 

[bctt tweet="Most of us have moved into leadership, after being highly successful individual contributors and supervisors. As leaders, we must find more value from the task, result, or project we helped someone else achieve. @Alexis5484" username="toprank"]

Tip #5: Stay fresh on the job

At all levels of my career, I’ve found the best way to build trust with a team is to help them solve a problem. The more you understand your team’s job function, the more able you will be able to help them solve problems, innovate, and provide feedback to improve the function of their performance. 

  • Stay fresh. I find the best way to do this is to jump in and help execute from time to time. So, write a blog post or create the tactical plan. This keeps you from getting rusty, but also helps you empathize with your team and the challenges within their roles. 
  • Ask questions. Sometimes you won’t understand the details of what they’re working on, particularly if you’re leading a cross functional team. But ask questions. Help them look at the problem critically, and it’s likely you’ll guide them to their own answer. 

Tip #6: Be the leader

One of the toughest transitions from individual contributor to leader, is owning your role as the leader. For the first few years that I was managing a small team, I was more likely to be found deep in the weeds, doing the tasks I did in my previous job titles, than actually doing my work as a leader.

There were a couple reasons for this. It was comfortable doing the work; I already knew how to do it and I was good at. I also felt like I was most helpful to my team if I was helping them get the work done by actually doing the work. 

This was not true. See tip No. 3. You (and I) are most helpful to your team when you’re solving systematic problems, optimizing workflow and production, and creating a happy and secure work environment. If you’re always in the weeds, all you can see is the weeds. 

[bctt tweet="You're most helpful to your team when you’re solving systematic problems, optimizing workflow and production, and creating a happy and secure work environment. @Alexis5484 on being a #marketing leader" username="toprank"]

Tip #7: Keep learning

The leaders I am most inspired by inside and outside of my organization are probably the most voracious learners. Continuous learning through a variety of mediums will help you continue to evolve your skill set, bring in fresh ideas, and help you be inspired to test something new. Here are a couple of the resources that I go to:

  • Read: HBR is a go to for great content on how to lead, manage and shape a business. 
  • Listen: Dear HBR has a great Q&A format about navigating workplace challenges. 
  • Attend: Industry events are great for providing outside perspective, networking with other leaders and inspiring the evolution of your tactics. MarketingProfs is a great event for marketers.

Take Your Place at the Leadership Table

Each stage of your career offers a unique set of challenges and opportunities. The way in which you handle those situations—tackling them head-on or leaving them for someone else—has the potential to make or break your success in that position… and the one that may or may not come after. Keep these pieces of advice in mind as you work to build your team, your organization, and career as a leader.

Looking for more tips on how to inspire, motivate, and build a more effective marketing team? Check out our tips for getting your marketing team to work better together.

The post Tales from the Trenches: How to Transition from Marketing Doer to Marketing Leader appeared first on Online Marketing Blog - TopRank®.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

css.php