GitLab’s Top 5 Tips for Running a Bug Bounty Program – InApps Technology 2022

GitLab sponsored this post.

GitLab went public with our bug bounty program in December 2018, and since then we’ve had 2,110 reports submitted and thanked 246 hackers. We certainly do not know everything when it comes to running a bug bounty program; and what we do know, we’ve learned and iterated on as we’ve gone along. In the spirit of collaboration and transparency, we thought we’d share a few tips we’ve learned about building a bug bounty program that’s both successful and aligned with our values.

Tip #1: Prioritize What’s Critical (and High)

Let’s be honest, no one in your organization wants to spend numerous hours triaging, scheduling, investigating, debugging, or fixing low severity issues. The cost-benefit likely just doesn’t make sense.

The high and critical severity vulnerabilities are where you want to incentivize research, as by definition they are more impactful to your organization. In terms of high and critical severity vulnerabilities, being aware of these potential company extinction events is incredibly important. Some first steps to treating them as first-class entities are to pay competitively for them and reward the bounty as quickly as possible. Expediting them with high payouts should incentivize additional reports of a similar nature.

Back in November 2019 we raised bounties for High and Critical severity vulnerabilities. We went from 12K to 20K for Criticals and 7K to 10K for High vulnerabilities. We also cut the time to bounty payout on these same classes of vulnerabilities, based directly on feedback from some of our bug bounty researchers.

It’s not much of a surprise, but increased bounties paid quickly works. In fact, we saw a 45% increase in high severity submissions and a 73% increase for critical vulnerability submissions in the 7 months following our bounty increases. We were grateful for the influx of reports, as each one got us closer to a more secure environment; and ultimately, the cost of a responsibly disclosed bug of this type to our program is a fraction of the risk and costs associated with a large, malicious attack.

To further incentivize around high and critical severity vulnerabilities, as well as to achieve cost savings during a somewhat unpredictable time in global business, we recently (June 30, 2020) reduced our bounties on low and medium severity vulnerabilities by half. At this same time, we reduced the time to bounty in our program from 90 days to 45 days max and intend to continue iterating on this so that we can shorten this time frame further.

Tip #2: Listen to Stakeholder Feedback

There are many stakeholders with respect to your bug bounty program. To name a few, at least from GitLab’s perspective, there are:

• Bug bounty reporters: those who find and report vulnerabilities
• Security team: those who triage the vulnerabilities
• Product: those who schedule vulnerabilities for fixing
• Engineering: those who fix the vulnerabilities
• Bug bounty program partner/vendor: (in our case, that’s HackerOne) the platform and support team hosting your program.

Each one of these stakeholders are incredibly valuable to the security of our product and the success of our program. So it’s important to connect with them and listen to their feedback and needs regarding your program, as these are opportunities to improve, streamline and possibly innovate.

Quick examples of stakeholder feedback that led to tangible improvements:

• Reporters asking for frequent updates on when fixes would be scheduled led our team to develop an automation bot that seamlessly shares fix ETAs, based on the related GitLab.com issue’s due date and targeted milestone.
• Security and product teams expressed difficulties related to tracking multiple security issues simultaneously, so we created an escalation engine that effectively surfaces outstanding security issues and efficiently tracks them from triage to remediation and closure. You can read more on this below.
• In addition, as we mentioned above, you can thank your fellow hackers who requested faster payouts for medium, high and critical vulnerabilities.
• Also, thanks to the advice of HackerOne, we’ve been able to effectively calibrate bounty amounts, time to triage and time to bounty rates. They’ve also counseled us on best practices surrounding triage, maneuvering challenging situations with reporters and even, when the best time would be to take our bug bounty program public.

Tip #3: Be Responsive and Focus on Time

We learned pretty quickly that being responsive, both in our communications and our fixes, was important to hacker engagement and overall program success.

Key focus areas for us are:

Time to initial response

Driving this down helps establish expectations and keep reporters engaged and informed. We built in automation which provides a ETA on our follow-up response and an ETA on the fix.

Time to triage

The more quickly you triage, the faster the reporters know whether their bug is valid or duplicate. This speeds up payment too, which is always good.

Time to bounty

We reward partial bounties at triage, $500 for medium and $1000 for high or critical reports. This was an improvement we made based on hacker feedback. The assumption here is that reporters (and all of us) like money in our pocket for a job well done sooner, rather than later.

Time to fix

We can all agree, nothing drains the excitement out of finding a bug like hearing that you weren’t the first one to find it. Quickly fixing a bug reduces the chance for duplicate reports. You can check out our handbook to get a better idea of how we prioritize and track remediation as it relates to vulnerability severity. We’re continually working to tackle our backlog and consistently meet or exceed industry standard timelines for responsible disclosure. Of course, there are still improvements to be made!

Tip #4: Scale with Automation

With a massive pool of potential bug bounty reporters and only a small security team, it’s easy to become overwhelmed with the incoming volume of reports. This is why it’s important to automate where possible. Communication and report management are key target areas for automation.

Using our program as an example, one of our challenges was tracking and driving flaws awaiting patch through to a fixed state. We realized pretty quickly that they could easily become lost in the volume and noise. To help manage bugs in this state, our automation team developed an escalation engine that tracks each bug’s progress and ensures they are properly triaged, labeled, scheduled and are meeting our mean time to remediation (MTTR) goals. This helps notify all teams involved in the vulnerability lifecycle when vulnerabilities are overdue, have insufficient labels, or need to be escalated.

Tip #5: Be Transparent About Security Issues

No one product or application is 100% secure.

I’d encourage organizations to be transparent about your security issues. We acknowledge that this can be difficult at times. Just check out this handbook entry: transparency is only a value if you do it when its hard, which applies to our security practices too.

Transparency in Security

• Clearly and publicly illustrates the importance and commitment you place on securing your product and your organization.
• Can further secure the product by inspiring new reports. Public disclosure of security vulnerabilities reduces the threshold to contribute because it allows researchers to learn and develop on top of other researchers’ findings. Very often, GitLab receives reports which are inspired by or based on previous reports that were published publicly after they were resolved.
• Extends public recognition to the security researcher. Often it’s more than about money for bug bounty reporters. They want the public recognition and the ability to talk about, show off or even blog about their findings. As a result, public disclosures can increase engagement for a bug bounty program if researchers know their work will be published.

Figuring out when to make your security issues public is a different matter. We’ve found the sweet spot for our own vulnerability disclosure to be 30 days or less. For organizations with SaaS offerings, if you’ve already upgraded your software and users have updated, releasing details immediately should be the goal. If you offer managed or on-premise solutions, your disclosure timeline will depend on your product. We base our disclosure schedule around the number of users on each version and how that aligns with our release schedule.

We’re proud of the growth and success our program has seen so far and grateful for the continued contributions and ingenuity of the security researchers who contribute to our program. We haven’t figured it all out and we definitely still have areas for improvement, based on metrics like those listed above and user feedback. We regularly post our monthly program metrics to twitter and you can always check out our Hacktivity and program statistics on HackerOne. Our aim is to continue to improve and strengthen our program for the benefit of our hackers, our customers and the open source community. If you have questions or ideas on how we can continue to improve, please let us know. Happy hacking!

List of Keywords users find our article on Google: