Using A/B test campaigns
A/B testing (or multivariate testing) can be used to evaluate the performance of your campaign on a sample of your filter. The testing possibilities are up to you, whether you want to test the subject of the design, the content, or even your recipient’s behavior, be it the number of clicks or total revenue from the campaign.
You can choose to send only one message to a small percentage of your database or test many designs at once, simultaneously.
The best-performing message can be selected manually, or Splio can automatically select and send your best message, based on the criteria you chose.
Creating an A/B test campaign
To create an A/B Test, create a regular email campaign and choose your initial population.
The A/B Test will take only a sample of the population and will keep the remaining recipients for the final action.
Once your population is computed, add the first sequence, select A/B Test, and input the date and time you wish to launch the test.
Your campaign will contain 2 actions by default, but you can add more with the configuration panel. Click on the A/B Test Configuration box to open the modal window.
Configuring the A/B test
Volumes
The Volumes tab is where you select the number of branches for your test, one action (message) per branch.
If you wish to test 3 messages, click on 3 branches. If you wish to test more than 6 messages, you can set a custom number in the field.
You can then select the number of recipients that will be selected for each action.
The sample recipients are selected at random from the initial population.
A/B Test Settings
The settings tab contains the validation mode. You can either keep the final message manual selection, or set a criterion, a date, and a time for the automatic selection and sending of the final message.
Automatic selection criteria
Splio can select the best-performing message for you. The available criteria are:
Best open rate (unique openers)
Best click rate (unique clickers)
Most buyers
Highest total sales
To make its selection, Splio will analyze, at the date and time set, all the tested message's statistics, and select the best according to the criteria.
If there is a tie in the criteria selected, Splio will send you an email and ask for your manual input.
What happens if you want to stop the sending of the automatic winner?
If you have selected the option of the winner design to be sent out automatically at the end of the A/B test but for some reason you wish to stop this action and not send it. It is possible to do so by editing the campaign and switching back to a manual selection of the winner.
Choosing the best design manually
If you chose the automatic A/B Test validation, your message is selected and sent automatically at the time and date you selected. When the final message is sent, it is available in the campaigns list within your AB Test campaign.
If you have to choose the final message manually, go to the campaigns list and click on the Edit button at the campaign level.
Open the campaign editor, and click on Choose final action. From there, you will be able to compare the statistics of tested messages, select the best-performing message, and select the date and time to send the message to the remaining recipients.
Comparing the results
To make the best decision, you need to compare the results of your A/B Test and evaluate each message's performance.
Go to the campaign's list, find your A/B Test, and click on Compare A/B Tests.
Timing the campaign well
Test one thing at a time
In order to run a successful A/B test, you should only test one thing at a time so you can analyze your results: either the subject line, the pre-header, or both or the body of your design. It is important to isolate the tested area to be able to identify what has delivered results afterward.
Measuring
Make sure that you have something to base your analysis on. If you have existing statistics, you will be able to measure this against your test.
Testing simultaneously and continuously
Challenge yourself with new tests: although you should focus on one thing at a time, you can still run several A/B test campaigns in parallel. For example, you can test the length, urgency, mention of a promotion, or use of the recipient’s name.
Timing
Timing is everything. Make sure you test different days and times of the day: emails don’t perform the same depending on when you send them.
Have a clear goal in mind
You need to increase the open rate: focus on the subject line and pre-headers
You need to increase the click-through rate or subscriptions: focus on your content.
Subject line
word order (try reversing the order)
tone (neutral, friendly, provocative)
length (try shorter, try longer)
personalization (try including their first name)
Pre-header
Try and introduce pre-headers to trigger the customer’s curiosity and open your emails, the more they are linked to the subject line, the better.
Content
It's all about the visuals. Re-arrange the positioning of the product catalog to achieve a different impact and boost the click rate.