A/B Testing

You don’t know what will be the best newsletter to send? Since AcyMailing 4.8.0, we included in our Enterprise version the A/B Split testing feature! It allows you to compare different newsletters and send the best one to your subscribers. You can not only compare two Newsletters but many of them at once! So it can be a A/B/C/D/E Testing...

Send different newsletters to a part of your receivers, compare statistics to determine what's the best Newsletter and complete the send process with the optimized version. You can choose manually the Newsletter you want to send to the rest of your subscribers or let Acy automatically select the best one based on the open rate, click rate or a mix of both statistics. AcyMailing will create a new newsletter according to the choice and complete the sending!

Create a test

In your newsletters listing, select the newsletters you want to compare (at least 2) and then click on the A/B Testing button.

In the popup window, you can first choose the percentage of your receivers that will be part of the test. The users are chosen randomly, so it won’t be the same users each time you perform a test. Below this, you will see a recap of the selected newsletters and the lists and filters of the first newsletter. The number of users who will receive the test and the total number of users concerned by the sending are displayed next. If the receivers do not satisfy you, you can use the link to edit the lists and filters applied to the newsletter.

The second part will let you choose what to do after the test. You can choose the number of days before taking action (only for automatic actions), and the action to take:

• Do nothing: manual action, you will have to come back and choose which newsletter to send.

• Generate and send automatically the newsletter with the best open rate

• Generate and send automatically the newsletter with the best click rate

• Generate and send automatically a newsletter with the subject of the best open rate and the body of the best click rate

Once the test is finished, the selected action will be executed (i.e the newsletter will be added to the queue to be sent), if you change the value of this field to "Do nothing" after the action has been executed, it will not remove the email from the queue, you will have to do it manually.

Then click on the test button to launch it. When accessing again the newsletter listing, you will see a new icon next to the newsletter name for newsletters included in the test. Just click on it to open the related test.

Compare results

After you have launched the test, some emails will be added in the queue. If you use manual sending, then go process the queue. If you use the automatic sending, then you just have to wait for the sending to proceed on its own. After the test has been sent to the testing part of the list, you can go back to the A/B Testing window and see the result of the test. It will display the number of emails sent of each newsletter, their number of opens and clicks.

Final sending

If you have chosen not to take any action, then you will be able to manually choose which newsletter to finally send. This will create a copy of the newsletter you chose and send it to the rest of the receivers. If you chose an automatic action, don’t do anything. Acy will automatically choose the newsletter when the time has come according to what you chose when configuring the test. If you choose the mix action, it will create a newsletter with the subject and the sending information of the newsletter with the best open rate, and the body of the newsletter with the best click rate.