Use A/B Tests to experiment with different variations, or variants, of a message for a given audience, and provide engagement reporting data for each variant.
What You'll Do
In this tutorial, you will:
- Define who will receive your message variants.
- Name your test, enter the number of variants, and adjust the audience vs. control group sizes.
- For each variant, choose which message type to send and enter its content.
- Direct the users' next step after they tap or swipe the message variants.
- Decide how and when you want the message variants delivered.
- Preview and send the message variants.
Features and options are explained along the way.
Open your project from the dashboard, then click Create and select the A/B Test workflow.
The audience you select in this step defines the entire set of users for the A/B test, which will include the control group (users who receive no message) and users who will receive different messages, according to the variants that you define.
- Optionally modify which Channels are included in your audience.All channels configured for your project that are available for A/B testing will be listed and enabled by default, but you can toggle to limit your audience to specific channels. You must select at least one.
- Select the radio button for your desired recipient group.
- All Users sends the message to your entire audience.
- Target Specific Users lets you create a recipient group based on segmentation data. See: Target Specific Users.
- Test Users are predefined recipient groups. After choosing Test Users, select from the Test Groups dropdown menu that appears. See: Audience: Test Groups.
- Click Variants in the header to move on.
- Enter a descriptive Test Name.
- Select the Number of Variants for your test, up to 26. The default is two. You will be able to add or delete variants in later steps.
- Use the slider to adjust the percentages of your target audience vs. control group. By default we send your test to 80% of your audience, keeping a control group of 20%
Variant A Set Up
- Enter a name for the first Variant.
- Choose between sending a Push Notification or an In-App Message, or choose Silent Push Notification to send a push without notification text.After you make your selection you have the option to combine message types, then click Continue to confirm your choices.
- Enter the text that will display in your message. A preview will
display as you type.
If you chose to combine a Push Notification and In-App Message, an In-App Message Text section appears below the message truncation indicator. By default, the In-App Message's alert text is the same as the Push Notification text. If you'd prefer to use a different message, select the radio button for Write alternative, and enter your text in the box that appears below.
- Select a Message Action from the dropdown menu.
- Optionally Set a tag.
- Set Optional Features.
- Configure additional variants. Select the next variant by using the
lettered tabs, and complete the name and message setup for each.
For variants B-Z, you have the option to either use the content from a previously created variant, or start from a blank message.
You may also add and delete variants using the tab X and + options.
- Click Delivery in the header to move on.
- Click to select Send Now or Schedule.
- Schedule: Enter a date in YYYY-MM-DD format and make your selections
from the dropdown menus.
- Schedule: Enter a date in YYYY-MM-DD format and make your selections from the dropdown menus.
- Set Optional Delivery Features.
- Click Review in the header to move on.
- Select a variant from the dropdown menu in the Content section or above the preview.
- Review the device preview and message summary. Click the arrows to page
through the various previews.
The channel and display type dynamically update in the dropdown menu
above. You can also select a preview directly from the dropdown menu.
If you would like to make changes, click the associated step in the progress header, make your changes, then return to Review.
- Send your message! Click Send Message or Schedule Message at the bottom of the window.