Global Control Group

Use the Global Control Group to specify a percentage of all users that shouldn’t receive any campaigns or Canvases, allowing you to analyze the overall impact of your messaging efforts over time.

By comparing the behaviors of users that receive messaging to those that don’t, you can further understand how your marketing campaigns and Canvases result in an uplift in sessions and custom events.

How the Global Control Group works

With the Global Control Group, you can set a percentage of all users as a control group. When saved, users in the group will not receive any campaigns or Canvases.

Your Global Control Group is applied to all channels, campaigns, and Canvases, with the exception of API campaigns and News Feed Cards (deprecating). Users in your control group will still receive API campaigns and News Feed Cards. This exception doesn’t extend to Content Cards—if you’re using Content Cards, users in your control group won’t receive them.

Assign users randomly to the Global Control Group

Braze randomly selects multiple ranges of random bucket numbers and includes users from those selected buckets. If you are currently using Random Bucket Numbers for any other purposes, check out Things to watch out for.

Data tracking for reporting

Braze measures the behaviors of users in your control group and users in your treatment sample. Your treatment sample is a random selection of users not in your control group, generated using the same Random Bucket Number method.

Exclude users from feature flags

You can’t enable feature flags for users in your Global Control Group. This means users in your Global Control Group also can’t be part of feature flag experiments.

Create a Global Control Group

Step 1: Navigate to the Global Control Group Settings

From the dashboard, go to Audience > Global Control Group.

Step 2: Assign a percentage of all users to this control group

Input a percentage for your control group and click Save. When entered, Braze shows you an estimate of how many users will fall into your Global Control, treatment, and treatment sample. Keep in mind that the more users you have in your workspace, the more accurate this estimate will be.

The number of users in your Global Control Group is automatically updated after its initial setup to remain proportionate to this audience percentage when more users are added to your workspace. For instance, if the number of users in your workspace grows, then the number of users in your Global Control Group will also grow so that your Control Group remains a constant percentage of your workspace audience. For percentage guidelines, refer to the following best practices section.

Step 3: Assign exclusion settings

Use tags to add exclusion settings to your Global Control Group. Any campaigns or Canvases that use the tags included in the exclusion settings don’t use your Global Control Group. These campaigns and Canvases continue to send to every user in the target audience, including those in your Global Control Group.

Step 4: Save your control group

At this point, Braze generates a randomly selected group of users to comprise the selected percentage of your total user base. When saved, all currently active and future campaigns and Canvases no longer send to users in this group, except for campaigns or Canvases that contain any of the tags in your exclusion settings.

Disable your Global Control Group

You can disable your Global Control Group at any time from the Global Control Group Settings tab, but keep in mind that doing so will result in users in this group immediately becoming eligible for campaigns and Canvases.

Before disabling your Control Group, we recommend exporting a CSV of users in that group in case you need to reference it at a later point. When you disable a control group, there is no way for Braze to restore the group or identify which users were in this group.

After disabling your Control Group, you can save a new one. When you enter a percentage and save it, Braze generates a new randomly selected group of users. If you enter the same percentage as before, Braze still generates a new group of users for your control and treatment groups.

Export your control group members

If you’d like to see which users are in your Global Control Group, you can export your Group’s members via CSV or API.

To run a CSV export, navigate to the Global Control Group Settings tab and click  Export. To export via API, use the /users/export/global_control_group endpoint.

Reporting

Refer to Global Control Group Reporting for information on report metrics.

Troubleshooting

As you set up your global control groups and view reporting, here are errors you may run into:

Issue	Troubleshooting
Unable to save the percentage entered when designating a Global Control Group.	This issue occurs if you enter a non-integer or an integer that is not between 1 and 15 (inclusive).
“Braze is not able to update your Global Control Group” error on the Global Control settings page.	This usually indicates that some component of this page has changed, likely due to actions taken by another user on your Braze account. In this case, refresh the page and retry.
Global Control Group report does not have any data.	If you access the Global Control Group Report without having saved a Global Control Group, you will not see any data in the report. Create and save a Global Control Group and try again.
My conversion rate is 0% or I’m not seeing the graph display, even though there are more than zero events occurring.	If the number of conversions is very small and your control or treatment groups are very large, then the conversion rate may round to 0%, and thus not show up in the graph. You can verify this by checking the Total Number of Events metric. You could compare the effectiveness of your two groups using the incremental uplift percent metric.
My conversion rate (or other metrics) are changing drastically depending on the time period I’m viewing data for.	If you’re viewing data over short time periods, it’s possible for your metrics to fluctuate day to day or week to week. We recommend that you view metrics over the course of at least one month.

Things to watch out for

Overlapping Random Bucket Numbers

Your Global Control Group is formed using Random Bucket Numbers, and thus if you are running any other tests using Random Bucket Numbers segment filters, keep in mind that there could be an overlap between those segments you create, and your Global Control Group users.

Duplicate email addresses

If two users who have different external user IDs have the same email address, and one of these users is in the control group and the other is not, then an email will still be sent to that email address whenever the non-control group user is eligible for an email. When this occurs, we will mark both user profiles as having received the campaign or Canvas containing that email.

Global Control Group and message-specific control groups

It’s possible to have both a Global Control Group and also use a campaign-specific or Canvas-specific control group. Having a campaign-specific or Canvas-specific control group lets you measure the impact of a particular message.

Users in your Global Control Group are withheld from receiving any messages other than those with tag exceptions, and if you add a control to a campaign or Canvas, Braze withholds a portion of your global treatment group from receiving that particular campaign or Canvas. That means if a member of the Global Control Group is not eligible to receive a particular campaign or Canvas, they will also not be present in the control group for that particular campaign or Canvas.

In short, users in the Global Control Group are filtered out of the campaign or Canvas audience prior to entry. Of the users who enter the campaign or Canvas, a percentage of those are then assigned to the control variant.

Global Control Group segments on the Developer Console

You may see multiple Global Control segments in the Additional API Identifiers section of the API Keys page. This is because each time the Global Control Group is enabled or disabled, a new Global Control Group is formed. This leads to multiple segments labeled “Global Control Group”.

Only one of these segments is active and can be queried using the /users/export/global_control_group endpoint, or exported from the dashboard. The export from the dashboard specifically states which subsegments make up this Global Control Group.

Testing best practices

Optimal control group size

Two main rules to keep in mind are**:

1. Your control group should be no smaller than 1000 users.
2. Your control group should be no more than 10% of your entire audience.

If you have a total audience that’s smaller than 10,000, you should increase your percentage to create a group of over 1000 users; in this case, you should not increase your percentage higher than 15%. Keep in mind that the smaller your overall workspace size is, the more challenging it will be to run a statistically rigorous test.

• Some trade-offs to consider when thinking about your control group size are that you need a significantly large number of customers in your control group so that any behavior analysis created is trustworthy. However, the larger your control group is, the fewer customers are getting your campaigns, which is a downside if you’re using your campaigns to drive engagement and conversions.
• The ideal percentage of your total audience will depend on how large your total audience is. The bigger your total audience is, the smaller your percentage can be. If you have a small audience, however, you will need a larger percentage for your control group.

Experiment duration

Choose an ideal duration

How long to run your experiment before reshuffling control group membership depends on what you’re testing and what your users’ baseline behaviors are. If you aren’t sure, a good place to start is one quarter (three months), but you should not go shorter than one month.

To determine the appropriate length of time for your experiment, consider what questions you’re hoping to answer. For instance, are you looking to see if there’s a difference in sessions? If so, think about how often your users have sessions organically. Brands whose users have sessions every day can run shorter experiments than brands whose users have sessions only a couple of times a month.

Or, you might be interested in a custom event, so your experiment may need to run for longer than an experiment where you’re examining sessions if it’s likely your users trigger that custom event less frequently.

Try to limit ending experiments prematurely

You should decide how long to run your experiment before beginning it, and then you should only end your experiment and gather final results after you reach this pre-determined point. Ending your experiment early, or whenever you see promising data, will introduce bias.

Think about valuable metrics

Consider any baseline behaviors for the metrics you’re most interested in. Are you interested in purchase rates for subscription plans that are renewed only on an annual basis? Or do customers have a weekly habit for the event you’d like to measure? Think about how long it takes users to potentially alter their behaviors due to your messaging. After you decide how long your experiment should run, be sure to not end your experiment or record final results early, or your findings may be biased.