In this article we’ll review the key steps to testing out different campaigns on a specific audience segment.
What is an A/B Campaign?
If you’re interested in running a campaign but wish to test different versions of the campaign, you may want to consider running an A/B test campaign. A/B tests are a great way to look at how different subsets of a selected group, who are linked by a specific account filter, react to variables such as:
Different creative
Alternate subject lines
Sending the campaigns on different days/times
Two different offers
By randomly breaking your audience into segments, you should be able to get a reliable read on which version was more effective.
How do I set up an A/B campaign?
To perform an effective A/B test it is important to randomize the segment audience. If the audience is split by a more arbitrary condition such as favorite store, last activity, age, etc., the results of the A/B test will not be as conclusive as with a randomized test. To achieve randomization, To achieve randomization, you'll use the control group functionality to split your audience in half. Your target group will receive the "A" version and through a second campaign your control group will ultimately receive the "B" version.
I. Create the “A” Campaign
For this campaign, your target group will receive the initial version, or the “A” version.
Begin by building an account filter for the segment you wish to target with your campaign. See What are Account Filters? for more information on creating filters.
Once your filter is created, navigate to Marketing Tools> Campaign Center> Create New Campaign. See the Campaign Builder Guide for more info on creating a campaign.
Build your campaign.
When you click on the Segment tab, choose the account filter you just built or you can use an existing filter.
Once you’ve added the segment, a toggle will appear labeled: Set a Control Group.
After toggling on this button, enter your percentage of guest to be held back as the control group, to eventually receive the “B” campaign. These control group guests will be selected at random from the original account filter audience and left off of the “A” campaign. It is recommended that you segment the guests 50/50 to ensure that half of the audience receives the “A” campaign while the other half receives the “B” campaign.
Save and publish your campaign.
II. Create the “B” Campaign:
For this campaign, your control group from Campaign A will be your target group here and will receive version B of your test.
Navigate to Marketing Tools> Campaign Center> Create New Campaign.
Build your campaign.
When you reach the Segment tab, select Create New Segment.
You’ll need to use the Relative Campaign condition to target customers based on their inclusion in the last occurrence of your campaign.
Type in Relative Campaign in box above the available account filter conditions.
Relative Campaign will appear under the Engagement condition. Drag it to the open box on the left to apply the condition.
When the list of campaigns appears, select the original “A” campaign title into the condition by clicking the radio button or double clicking on the name. When selected, the Campaign name and details will appear further down the screen.
NOTE: Do not change the default setting for 'You have selected accounts in the last occurrence of the following campaign.
Click the radio button next to Control Group to filter for accounts that were in the initial control group.
All default settings listed under Advanced Campaign Messaging Filter can be left as is.
NOTE: If the filter is showing an estimated segment size of zero matching accounts, this is normal because the "A" campaign has not executed yet. This amount will remain at zero until the campaign has run in its entirety.
To finalize the "B" Campaign, schedule it to run at a time at least 1 minute after the scheduled time of the "A" campaign.” Do not worry if the “A” campaign is delayed, as the system will know that the “B” campaign is meant to run after “A” has finished, and will delay it accordingly.