Everything You Need to Know about A/B Testing Emails

Businesses are booming and the market is constantly changing. In order to keep up with these ever-fluctuating times, it is necessary to keep your marketing tactics up to date. It is essential that you don’t drop the ball; instead, always remain at the top with respect to keeping and encouraging new customers to patronize your business.

Research has shown that psychology plays a large role in marketing, so people’s choices are often influenced by a company’s webpage and the emails they send out. The style and design will give visitors an idea of the personality of a company. The colors, designs, choice of words and even the layout of key features can have an impact upon the existing client base.

So with every change to the webpage or email format, there are multiple variables to consider:

  • Would it affect the revenue earned per month?
  • What about unsubscribes?
  • If it’s a membership site you’d be wondering if the free users would convert to paid ones if the price was right.

Up until now, answering such questions would simply have been mere speculation. You would only get a vague idea until your new webpage or email finally hit the market, and the result would then determine whether implementing the change was a good or bad decision. This takes a lot of time and mistakes are expensive to put right.

The damage would already have been done if the change affected people negatively.

The A/B split test is here to change this game. It will change the ‘maybe’ in your mind to an ‘absolutely’, giving you a definite prediction for the kind of impact of your change will have.

You no longer need to feel uncertain about the impact that implementing new changes might have. You will be able to determine whether a change would be profitable or not before it is exposed to your entire client base, thereby saving you from missteps and lost profits.

The way you can achieve this is by A/B testing.

What is A/B testing and how does it work?

More importantly why should you care?

You should care because A/B Split Testing is the answer to your troubles. Simply put, it compares two versions of a website or an email design or a landing page. Half of the traffic is sent to version A and the other half to version B, in order to see whether the new design would have a positive or negative impact upon the audience.

What is A/B testing and how does it work?

Since the target population differs from business to business, this test takes a sample from your client base. You can determine the percentage of clients to send the old and new email format. For example, if you have a client base of 10000 customers you can draw a sample of 3000 contacts to show the split tests to.

Experimenting with emails

Emails, in particular, are a personal way to reach out to your client base. They help you advertise your products directly to the consumer and have become a viral method of spreading the latest news.

Therefore, these need to be crafted carefully. The choice of words is imperative. The subject line itself will have a big impact upon how many people open their email, whereas the email content will affect their enthusiasm for whatever is advertised.

If you wish them to click through to your webpage, this can be achieved through the content. Here’s where the email design plays a part.

Emails are the easiest and fastest way of reaching out to multiple people, but formulating them requires great care.

The subject line must be carefully crafted to entice the recipients into opening the mail. The content must be written to encourage customers to take advantage of the new offer or deal.

An email is composed of multiple aspects, each with the power to change and influence opinions. Whether it is the subject line, personalizing the email, questions present in the content or the attached images, everything can directly impact the actions of the receiver. While the A/B Split test is meant to uncover these changing impacts, for clarity and sanity, it is necessary to test these one at a time. This will directly attribute the results to the changed variable alone instead of giving you multiple possibilities for the change.

Things to look for when Split Testing Emails:

Subject Line: To Read or Not to Read

The subject line is the first thing the recipient will notice. Its phrasing and catchiness will determine whether the email will be opened or not. Try to be specific, directly stating the purpose of the email instead of providing a long subject. For example, stating ‘SALE’ is definitely attractive, but starting with ‘75% off’ might spark a higher level of interest instead. It clearly states the gist of the email and is definitely something that will pique the customer’s curiosity.

Having questions present in the text that will encourage the recipients to continue reading further. But too many questions can just make the email sound uncertain. Use multiple techniques to spice up your sample email, but using them in good measure is the key to success.


When addressing the email personally to the recipient, it encourages them to read further. It adds a touch of effort to the email, showing that it was clearly intended for them and is therefore worth their time. Showing that you care is the best way to grab someone’s attention. ‘Dear Diana’ is more attention grabbing than ‘Dear Customer’, and is more likely to encourage Diana to read the email.

An Experian study conducted in March 2013 revealed that personalized mails had a 29% higher open rate and 41% unique click-through rate.

“Personalization is much like a matchmaking exercise with the ultimate goal of pinpointing the best product or service from your catalog that best satisfies your customers’ needs,” says Scott Brave of Baynote in, “The Human Need for Personalization: Psychology, Technology and Science.”

You can take it a step further and include a sender’s name in emails you send. For example, instead of saying that the email is from XYZ company, make that email say “Marjorie” from XYZ company.

Call to Action (CTA)

This is an important part of the email, as it enthusiastically encourages the clients to take up your offer. For example, VAIO used ‘Create Your Own VAIO Laptop’ in one of their campaigns to encourage personalizing their models and advertise this new service. A short, direct CTA is more likely to encourage the clients than a long, drawn out statement. Including multiple CTAs in one email, though, can have a negative impact upon the reader. Try sticking to presenting only one, attention-grabbing CTA per an email.

Fab, an online retail community, discovered the importance of having a clear and direct CTA. They changed their ‘Add to Cart’ icon. The original, featured a cart with a small ‘+’ sign next to it and no text. The new icon stated ‘ADD TO CART’ instead. This resulted in a clear increase in their cart sales by a whopping 49%!

Another case study showed that that while a clear, interesting picture can catch people’s attention, a video might be even better. An engaging video which is to the point and concise can often have an even greater impact.

Doggyloot is online company that sells dog toys and chewables. In this email the big orange button spells out very clearly that stocks may be limited and encourages action.

Doggyloot Call to Action

Day and Time

It is important to determine what time of the day and day of the week would result in the maximum response. The answer for this can positively impact your marketing campaign. For example, a weekend might lead to many people not checking their inbox. This is even more important if your target audience is a population whose work life starts and ends with emails.

Weekends are holidays, and being so, many people actively attempt to dissociate these from their work days. Monday is the first day of the week, and therefore might also have the heaviest workload, making it difficult for many to check their emails. Perhaps emails sent on a Tuesday or Friday might do the trick.

Due to the ambiguity surrounding these factors, it is necessary to verify the option that would yield the best results before applying it to all of your ongoing marketing campaign.

According to GetResponse’s research over 23% of email opens happen right during the first hour after the email being sent.

After 24 hours, the chances of the email being opened, ever, drops below 1%.

This chart may help you decide when to send emails.

when to send emails?

Content of the Email

This includes the text, the font, layout and much more. You need the most important content to be presented in an eye-catching manner. It should not be lost in the text. The layout will help to classify the information’s importance and separate it, making it easier on the eyes.

The font, too, can have a big impact.

Studies have shown that the Typography used influences decisions. For example, the font Georgia can make readers feel that a particular statement is true.

This being primarily because a font which is clear is easier to read. Different fonts can convey different messages. While some are too formal, others are informal, so it is important to choose one with care.

While you may want to stand out from all the other emails the recipient is likely to receive, you don’t want your message to be discarded to the junk folder. Another key aspect is the color combinations of an email. Different combinations are more likely to grab the attention of the receiver, but colors which make the text difficult to read should be actively avoided.

Have a look at the welcome email sent out by Adidas to its new customers. They are rich in design, color and if you have just signed up you may want to splurge some cash in your excitement.

Adidas welcome email

Free shipping

Another popular idea is through the incentive of free shipping. No matter the price, adding this incentive can increase sales by 97%, simply because customers would much prefer paying only for the product than for both the product as well as its shipping. Knowing that they are getting something for free is always a good incentive.

Many other case studies over time have shown the importance of multiple aspects of marketing. It goes to show that everything, ranging from the company logo and the layout of your email to the background of your webpage can affect your business tremendously.

Choosing and Using Split-Testing Tools

Your email auto-responder probably has a split-testing facility you can use to test different templates and headlines organized in a neat dashboard. A good tool allows you to experiment with the subject line, form field, content, day of the week and time of day to determine which would give you the best results.

You can determine the percentage of your test population, subjecting the rest of your clients to the winning email from this test.

You can tweak your template or create an entirely new one to see the outcome. While the most conclusive results will be found when presenting it to a larger test population, it is necessary to allocate a majority of your customers to be subjected to the winning email instead.

Since the email recipients for the test are chosen randomly, it’s likely that you will see an unbiased result. After submitting your test email, you can decide whether the winner of the two should be picked based on the ‘open rate’ or ‘click rate’.

Due to the versatility of this test, there is greater scope for change and creativity. It gives you more freedom to experiment with newer ideas so that you may receive a clearer perspective.

This is the perfect way to increase and improve your business as you reach out to your customers.

Concluding thoughts!

A/B tests when done right can significantly increase your revenues and profits. It helps you identify the weak spots in your design and marketing and helps you correct them.

With a proper email A/B testing strategy you should immediately see improved results from your email marketing campaign.

In today’s ever-changing world, we all need to keep altering our business tactics.

There is no attaining perfection but these slight changes are likely to boost your business tremendously. There will always be scope for more improvement, and there are various websites and SaaS applications that enable you to conduct A/B split testing to help you with that improvement.

What’s Your Take on A/B Testing?

Do you find A/B split testing difficult? What split testing tools have you tried? Please share your experiences using the comments box below.

Like the article? Share it.

LinkedIn Pinterest


  1. Good job

  2. This is great, thank you for the share! This is something I will have to go over with my outbound marketing person. Thanks!

Leave a Comment Yourself

Your email address will not be published. Required fields are marked *