From Hacker to Bug Bounty Program Owner: A Learning Experience

Security has been a major priority here at Braze since our founding back in 2011. Our focus on protecting our customers’ data led us to embrace a mindset of security and privacy by design when building our product, ensure we were certified in connection with ISO 27001 and SOC 2 Type 2, and to build out a dedicated HIPAA cluster to help brands safeguard protected health information. But while all those steps are important, we’re not resting on our laurels. There’s always more that you can do, and that focus on finding new ways to strengthen our security is a big reason why I’m here.

Back in 2020, Mark Shasha, Head of Security at Braze, brought me in to help launch a bug bounty program here, with the goal of making it easier to identify security issues that could impact our product. And now that the program has been up and running for just over a year, I thought it was time to look back at what we built and what I’ve learned along the way.

What Is a Bug Bounty Program?

In the Braze bug bounty program, outside parties are invited to try to compromise a sanitized, customer data-free version of the Braze platform and are paid when they identify a valid, actionable, security issue. Creating a bug bounty program makes it possible for a company like Braze to leverage external security researchers and professionals to identify potential security issues, allowing us to proactively address vulnerabilities.

These programs are widely seen as one of the most important cybersecurity measures that a company can embrace—in part, because it allows brands to crowdsource security insights from a vast number of different people with a wide variety of skill sets. As a rule, launching and maintaining a successful public bug bounty is a sign that an organization has a healthy security program, since running a program like this without incident requires a level of security maturity that takes a lot of thought and care to achieve.

My Days as a Bug Bounty Participant

I first heard about bug bounties back in 2014, but because I thought they sounded too good to be true, I didn’t end up actually participating in one until 2016. Inspired by some blog posts written by other ethical hackers, I took part in the Yahoo! bug bounty program and found that I had a real knack for the work. For one thing, they gave me an opportunity to leverage my hacking skills to help safeguard computer systems, rather than compromise them. For another, they gave me the opportunity to make large sums of money while I was at it.

As the number of companies and governmental organizations launching bug bounty programs grew, I ended up specializing in hunting down server-side request forgery (SSRF) bugs for a bug bounty program associated with a major telecommunications company and made just under $1 million dollars just from identifying those vulnerabilities. But while the financial side of working bug bounties was really paying off for me, I found myself missing the structure and personal connections that come with having a day job. So when Braze offered me the opportunity to launch and oversee a bug bounty program of my own, I jumped at the chance.

How We Launched a Bug Bounty Program at Braze

At Braze, we had to go through a number of steps before we were able to make our vision of a bug bounty program a reality. For one thing, because participants are paid for every valid, actionable bug they find, launching a bounty program without addressing any and all known vulnerabilities can lead companies to pay top dollar for information that they already have, reducing the impact of the program while simultaneously driving up its cost. To that end, we carried out the following steps before preparing for an official launch:

• Deploying internal security service-level agreements (SLAs) with development teams
• Creating a vulnerability management program
• Deploying dynamic analysis security testing (DAST) tooling
• Performing internal penetration tests
• Conduct third-party penetration tests
• Ensuring that all known issues have been remediated

Then, once we were confident that the duplicated version of the Braze platform created for the bug bounty program was as buttoned up as possible, we initiated a private, limited-scope program using the Bugcrowd platform. We launched this two-week-long, on-demand program so that we could both use it as a proof of concept and to help introduce the Braze organization to the realities of running a bug bounty program. After all, the idea of inviting hackers and security researchers to look for security flaws in your product can seem strange or confusing if you haven’t encountered bug bounties before.

4 Big Learnings from Launching a New Bug Bounty Program

I learned pretty quickly that launching a new bug bounty program is much more difficult than taking over an existing one. There are so many different factors that go into creating a successful program that don’t get as much attention—in part, because they aren’t as exciting as identifying critical bugs, fixing them quickly, and paying out large bounties. Given that, let’s talk through a few of my biggest learnings:

1. Launching a Bug Bounty Program Takes Cross-Team Collaboration

Originally, I’d hoped that launching the program would be as simple as deciding to do it, picking the right platform, deciding on the scope and bounty amounts, and then just kicking things off. But doing it right takes far more planning, preparation, and attention to deal than I thought. For one thing, I hadn’t taken into account all of the other teams within Braze that had a role to play in supporting the bug bounty program launch—from the work our Legal team did to make sure we had the proper wording in place for our Safe Harbor agreement to the work done to create our SLAs and ensure we had the right escalation process when violations happen. This work can be challenging, but it’s absolutely essential. A bug bounty program that hasn’t been carefully planned and executed will inevitably run into many problems, which, in turn, can lead to bad word of mouth about the program, making it harder to attract top-tier hackers and security researchers and potentially dooming the whole endeavor.

2. Never Lose Sight of Your Relationship With Hackers and Researchers

It’s important for brands to remember that a successful bug bounty program depends on the relationship between the program and the hackers/researchers who participate in it. Happy hackers are far more willing to spend their time on your program and given that every bounty program is fighting for a share of a finite resource—namely, hackers/researchers’ time and attention—it’s important to make sure you’re setting yourself up for success by prioritizing this relationship and doing what you can to set yourself apart from other programs. Some companies do this by paying larger bounties than the industry average for various bug classes and severities, but that’s not the only (or best) way to do it.

Because of my background as a bug bounty hunter, I’ve been able to use my experiences to help inform how Braze nurtures that relationship. For instance, I was able to get buy-in to ensure that Braze runs both public and private bug bounty programs concurrently. That allows us to identify individuals involved in our public program who are reporting good, valid reports and then reward them by inviting them to our private program. These participants have additional functionality to test and get the first crack at new scope additions before we add them to the public program. I believe that by doing little things like this, companies can show their appreciation for the researchers who are contributing to their program and encourage them to deepen their engagement in the future.

3. Bug Bounties Look Different From the Company Side

Before I started running my own bug bounty program, I wasn’t a fan of working with programs that were managed by a third-party platform. For programs set up like this, it’s common for companies to rely on third-party triagers provided by the platform, who review submissions from hackers/researchers and determine the severity of each identified bug, and I’d had some experiences where the triagers had made calls that I didn’t agree with.

However, now that I’m on the other side, I can see just how much value this kind of platform-driven approach provides to the companies using it. While we're still involved in directly overseeing the work being done by the third-party triagers that we use, I've found that leveraging them can do a lot to reduce the time and energy burden associated with running a program like this. The triagers we work with are pros and have proven to be a great asset to our program, helping to make our successful rollout possible.

4. The Work Doesn’t End When a Bug is Identified

Before joining Braze, I would often get frustrated with bug bounty programs that took weeks or months to fix the vulnerabilities I’d submitted to them. From my perspective, the issues generally seemed pretty cut and dry and I felt like the patch for those bugs should take little or no time to implement.

But now that I’ve witnessed what happens behind the scenes when one of these bugs is submitted, I realized that I failed to take into account all the discussions and work that’s done behind the scenes during the lifecycle of a security vulnerability—from the investigation and confirmation of the bug and the delivery of those details to the responsible team internally to the actual coding changes, testing, and releases that have to happen before the bug is truly addressed. The reality is that these things take time and, as a hacker, I didn’t always take the work involved into consideration, in part because I wanted the entire process to be done as quickly as possible so that I could be paid.

Final Thoughts

Running a bug bounty program for the last year has shifted my entire outlook on the industry and even changed how I select what bug bounty programs I focus on in my free time. This peek behind the curtain has given me a lot more respect for the essential work being done by platform triagers and a better understanding of what realistic timelines are for companies to assess and address the bugs I submit. With this new insight, I’m hopeful that I can continue to improve and grow the Braze bug bounty program while also seeing even more success as a bug bounty hunter going forward.

Interested in joining the team here at Braze? Check out our open roles!