How do I send an A/B test campaign?

A/B Test campaigns allow you to test the subject line or email design of a campaign to determine what changes impact the recipients' engagement with the campaign.  

You will find the A/B test feature on the Campaigns Dashboard.  Follow the steps below to setup your A/B test campaign.

*Note: A/B test campaigns are available on Professional license or above. 

How-To Steps

Step 1 - Setup tab

1. Provide a name for your campaign.
2. Choose A/B Test Email under "Campaign Type". This will generate an option to test Subject Lines or Email Designs. Choose whichever is applicable.
3. You may note that you are automatically editing Version A of your A/B test. There is no limit to how many versions you create for the test.
4. Use the Version drop-down to complete the setup for all of your different versions. Sender names and footers chosen for Version A will be the default for all subsequent versions unless changed.
   *Note: Versions can be removed by choosing the Remove button. If you remove Version B then Version C will become Version B, etc. 
   
   
   
   
5. Once you complete your setup for all planned versions hit "Save and Next".

Step 2 - Design tab

1. You will be prompted to choose how you want to start your design for version A with the following options:
   • Start from Blank: Start in the drag-and-drop editor with no template. 
   • Start from Paste: Paste previously created HTML to create your email design. This will bypass the drag-and-drop editor. 
   • Start from Upload: Upload a previously created HTML file to create your email design. This will bypass the drag-and-drop editor.
   • Start from Template: Choose either a stock template, saved custom account template, or a template saved by your site admin available to all related sub-accounts to complete your design in the drag-and-drop editor.
2. If you've chosen to test Subject Lines you will only need to complete one version of the email design. If you've chosen to test email design you will be able to design each of your versions separately. 
   *Note: If your designs are similar with minimal changes, we recommend creating a default version as an Email Template. Be sure to "complete" that template so it will be available for selection under "Start From Template" > Custom Templates. 
   
   
   
   
3. Once you are done designing the email copy, select "Save and Next".
   
   

Step 3 - Contacts tab

1. Under the Contacts step you can choose one or more existing categories or segments. If you do not already have the appropriate segment you can build one from this page with the "Create a New Segment" button. 
   • If applicable, also choose your exclusions or suppression lists.
2. Click Save & Next.
   
   

Step 4 - Preview tab

1. Use the Preview step to preview the rendering for all of your different email design versions. If you have email rendering credits (additional fee) you may preview rendering with specific email clients. You can also test dynamic content, merge tags, or see the online version of the campaign. 
2. Click Next.
   
   

Step 5 - Test tab

Use the Test step to test your content live. You will receive each version with 1 test (you do not need to send multiple copies). You MUST test your content before moving to the Schedule step.

Step 6 - Schedule tab

In the Schedule step you have a few settings to configure that will determine how the audience will be split, when the campaign will be sent, and more:

1. 1. Distribute to: You can distribute to an exact number of recipients or a percentage of your segment(s) selected for the campaign. "Percentage of Segment" is the most commonly used selection. Examples of using percentage for A/B test campaigns include:
      • Partial segment (less than 100) - Enter 10 in the box to split & send version A to 5% and version B to 5% of your segment.  
      • Full segment splitting methods (100) - Enter 100 in the percentage box to split & send to the whole segment based on the # of email designs / subject lines you have. Examples:
        • Two versions: The segment will be split in half for the campaign; 50% will receive version A and 50% will receive version B.
        • Three versions: The segment will be split in thirds for the campaign; 33.3% will receive version A, 33.3% will receive version B, and finally 33.3% will receive version C.
        • Note: When testing to the full segment, you will ignore the Winner selection.
        • Tip: You can click on the "View Details" button in this section to see the distribution visually. 
   2. When do you want to sent the campaign: Set a date & time to send the campaign.
      • Partial segment: You are scheduling only the smaller portion of the segment to be sent and there will be a remaining audience that did not receive anything. In this case, you will also need to determine a winner so that the best performing campaign is sent to the remaining contacts in the segment.
      • Full segment: You are scheduling the send to the entire segment. In this case, you will ignore the Winner section.
   3. Winner: Choose how the winning campaign will be identified: automatically or manually.
      • Automatic: Chosen based on opens, clicks, or purchase amount (Clickstream customers only). If you choose Automatic you must set a send time to schedule when the system will evaluate the A/B parts and determine which has the most activity.
      • Manual: If you choose manual you will need to view the results of the initial test and manually send out the winning version on your own. To do this, at some point after the sending of the smaller portion of the campaign, return to the Campaigns dashboard. Use the filter to find your sent A/B campaign. Hover over the campaign and select "View". On the winning campaign, hover and click "Send This Mailing as Winner".
        
        
        
        
2. Click the Save when your scheduling settings are complete.

Resources

Visit our e-Learning hub for online modules to learn more about A/B test campaigns & other Delivra features.