Handling a large control group

When creating your Canvas, you may have expected your audience to split evenly between your control group and your variant group, like in the following example. We can explain why that is and how to fix it!

The group that a user joins depends on their settings. This can be either the control group or variant group. A user will enter a Canvas when they fit all of your criteria defined in the Entry Step. When setting up your Canvas, you define what percentage of users will enter each variant and the control group.

If your control group is large compared to your variant group (and this is not your intent), we recommend the following:

1. Set your entry audience filter to “Is Push Enabled”.
2. Set your entry audience filter to “is Opted In or Subscribed”.

When creating a Canvas with a control group, ensure that all users in the Entry Audience are able to receive messages within the Canvas (such as the Canvas contains push and email messages).

Use case

Let’s imagine the following scenario:

• A Canvas has a single variant and a control group.
• The first step of the variant is a push notification.
• 90% of users were selected to enter the variant, and 10% to enter the control group.

In this scenario, 90% of the users who enter the Canvas will enter the variant.

If we look back to the active users, we can see that even though it contains 39.8k users with only 73% of them push enabled:

This means that even though we specified 90% of users to enter the variant, not all of those users are actually able to receive a push notification. These users who are unable to receive a push notification will still enter the variant regardless.

Still need help? Open a support ticket.

Last updated on December 3, 2020