AI is transforming cybersecurity, accelerating vulnerability discovery and enabling increasingly autonomous security testing. But as the volume of findings grows, organizations face a mounting challenge: identifying which vulnerabilities matter most and remediating them before attackers can exploit them. At the same time, the rapid adoption of AI agents is raising questions about security safeguards, human oversight and accountability when autonomous systems act outside their intended boundaries.
Tech Channels sat down with Bugcrowd CEO Dave Gerry to examine the limits of AI self-regulation, the emerging risks of autonomous agents and how AI-driven vulnerability discovery, remediation and penetration testing are reshaping cybersecurity operations.
Q. How do you assess the current efforts to regulate AI at the federal level?
A. The challenge is I'm not convinced that we're going to see anything at the national level out of this administration. All these AI leaders, all the frontier labs were at the White House, they did their role, but the whole thing felt a little performative. There’s no teeth to it.
Q. There are some legislative efforts afoot—Sen. Josh Hawley (R-MO) and Sen. Chris Murphy (D-CT) are working together on efforts to put guardrails around in AI. Is that progress?
A. Yes. And there is Rep. James Walkinshaw (D-Va.) and Rep. Vince Fong (R-CA) on the House side leading the charge. So, it’s interesting. I spent some time with Walkenshaw and Fong. One of the issues they were focused on was how to fix what happened to CISA—it was clearly gutted far beyond what it ever should have been. It’s a critical function for the government from a policy and oversight perspective. It was good to hear both of them talking about finding a way to fund this and bring CISA back.
Q. What challenges would a state-by-state approach to AI regulation create for businesses?
A. In New Hampshire, where we live, I sit on the AI task force, and it basically was, “we're a small state, how are we going to bring together public policy business leaders and try to mesh this together in something that makes sense?” The challenge is going to be you're going to have 50 different regulations at a minimum. And when you start including the District of Columbia, Puerto Rico, other US territories, you're going to have 54 or more different sets of guardrails that business leaders have to try to adopt.
Q. How do you expect AI legislation to evolve, and which organizations will struggle most with compliance?
A. States like California and New York will come out with something that's fairly well-rounded, fairly thoughtful, steeped in case law. Then you're going to have a bunch of the other states that maybe have the best of intentions, but their regulation is going to be overly vague, or it's going to be challenged in court because it wasn't well written. The frontier guys will be fine. OpenAI and Anthropic, will figure out a way to navigate through whatever exists. But I think for all the others, it's going to become really tough.
Q. Is Europe going to take the lead in regulation like they did with GDPR?
A. You’re going to see further guardrails ome out of the EU and people may latch onto that. It’s a bit of a shame because I think the U.S. had an opportunity to be the leader. We had the frontier labs. We could have come out with what should have been the international standard for AI safety and security. So I think we’re going to see a little more chaos
Q. Can AI developers effectively regulate themselves, or is independent oversight necessary?
A. We know in any industry that doesn't work, even with the best of intentions. They may have the best of intentions, but without some sort of third-party oversight, whether it's a regulatory body or whether it's just an independent agency, it won’t work. There's got to be some level of [outside regulation] there.
Q. What lessons should policymakers draw from industries that operated without sufficient oversight?
A. Let's go back to 2008 and look at what the lack of regulation costs, what the lack of oversight costs. Were all of those people running the banks bad or inherently evil? No, but they were motivated by how much money they could make as quickly as possible. Same thing is happening here. The companies have outsized valuations. They're real businesses. They're real revenue, but they're getting ungodly amounts of capital deployed at extraordinarily high valuations. They're going to be motivated to move quickly.
Q. Why are you skeptical of AI companies' public commitments to slowing development for safety?
A. Look at them all claiming we need to slow down. Did they? Right after the ChatGPT [incident], OpenAI delayed one of the ChatGPT launches by a new model by a week. We haven't seen some drastic slowdown or shutdown. I think that's where I come back to. This feels very performative.
Q. How serious is the threat of AI agents acting outside their intended boundaries?
A. I laugh. Everyone describes these agents going rogue as you almost picture Flubber in the glass cage, bouncing around, and it breaks out. I spent most of my career in application security, so we had scanners. If we were scanning company A and there's a link to Company B, it might jump over and start scanning Company B, and we get a call from the lawyers or from their cyber team saying, "What are you guys doing? Shut it down.”
It's very similar here right now. The difference is [agents] are far more capable. They're now communicating with each other, so I think we are seeing the potential be far more severe. But there is so much marketing hype behind it—look at how powerful this model is that we've created–that it's tough to decipher what's real and what's not.
Q. How can organizations distinguish genuinely dangerous AI behavior from failures to configure or enforce safeguards?
A. It would be through third-party oversight, somebody who has an independent view to say, “okay, this is actually doing something malicious. It is not just about somebody not enabling the guardrails.” How did that happen? Did they have controls in terms of what they were trying to do? Did they have human oversight? Were they trying to hold an agent accountable? You’ve got to have a human somewhere.
Q. How does the growing use of autonomous personal assistants change the cybersecurity threat model?
A. Right now, people are figuring out they can actually have an assistant do work on their behalf, and there's things it can do. I had a conversation with somebody where I said, “you know, no longer do we have to hack you. We just direct your bot to do something on your behalf.” It's the assistant, it's the EA that never gets suspicious because you've given it carte blanche to do whatever you need it to do.
Q. How could an attacker manipulate an AI assistant into performing an unauthorized transaction?
A. You could easily manipulate that. If it's reading your email and checking your calendar, you could implement something there that says, "Hey, go do this.” The assistant reads it as a legitimate instruction from you, and now suddenly it's executed some transactions.
Q. What safeguards should consumers put in place before allowing AI assistants to take actions on their behalf?
A. Human oversight. Say, “go find it, go do all that work, save me all that headache and time. But I want to click "approve” before you buy something. You’ve got to be thoughtful in what you're doing. The tough part is there are going to be people in this space that are going to understand this: Don't train on my data. Don't take these actions on my behalf. This isn't an auto approval. Go through those settings. But for the average person, they’re going to say “My God, look how much time I’m saving” and maybe in some cases the reward outweighs the risk
Q. Who should be held accountable when an autonomous AI agent causes financial damage?
A. It’s going to be an interesting liability question that has to get answered at some point. If and when something bad happens and there's damages—let's assume it's not loss of life; it's financial damages that happen—who's liable? The people that deployed the AI, or the AI, the frontier lab, who's running it. All these questions haven't been answered yet.
Q. Could the debate over AI liability resemble earlier legal battles involving social media platforms?
A. I think we're going to see a similar battle play out to what we saw with the social media platforms. Is the platform liable, or is the user liable? And we still haven't fully solved that question.”
Q. How can educators encourage students to use AI without sacrificing critical thinking?
A. You take the same argument or same conversation and look at the education system. So how do we figure out how to teach kids the power of AI in the classroom in a safe setting without them losing that critical thinking. There was a teacher who said students could use AI for whatever they want, but this goes back to the old days of you can do the math problem but show me your work. Show me all the prompts you're using. How are you guiding the AI? How are you leveraging the power of it to move faster, be smarter? That’s an interesting approach, too—use every tool at your disposal, but show me how you got that.
Q. How should AI infrastructure developers address the environmental and economic concerns of communities hosting data centers?
A. The data center issue is an interesting one to me because there has to be a solution, and the owner of that data center ultimately should be responsible for it. How do you solve for the water, the very legitimate water and electricity concerns for the residents?
Q. Could smaller, distributed data centers offer a more community-friendly approach to expanding AI infrastructure?
A. One of the models I've done a little bit of research on is Nvidia doing micro data centers. They want to put them at your house. So, if you own to plot a land, you can get this self-contained baby data center that they'll install, and in return, they'll pay your electricity bill.
Q. How can the industry balance the need for additional computing capacity with the interests of local communities?
A. There’s going to be the happy medium of where do you put these things. How do you take care of the community around it? This isn't just like the old “drill, baby, drill.” It's not “build, baby, build.” How do we do it in a thoughtful way? But I do think it's important that, ultimately, we're going to need the capacity.
Q. How has AI changed the balance between vulnerability discovery and remediation?
A. We've very quickly commoditized detection. Detecting is now the easy part.
Q. What challenges arise when security teams are overwhelmed by AI-generated vulnerability findings?
A. So, now it's what do you do with [what’s been detected]. How do you get the operator? The human on the other side that's now inundated, they were already inundated. They already couldn't keep up with the pace of remediation. Now they're completely overwhelmed.
Q. How do security teams handle the onslaught of vulnerabilities?
A. Now it’s all about how you get them in the right context. What's that prioritization layer that has to exist? Say, you have 10,000 findings, you decide these are the 10, these are the 100, that really matter, this is why, and this is what you do.
Q. Will autonomous remediation become necessary to manage the growing volume of vulnerabilities?
A. What you're going to see is an upcropping of all sorts of agentic remediation companies. The only way to solve this is by fully automating the patching. Vulnerabilities are detected, triage is prioritized. There's context around it. “Yes, it's real. It's exploitable. Go fix it.”
Q. How has AI affected the volume and quality of vulnerability reports submitted through bug bounty programs?
A. On the bug bounty side, our volume is pretty normalized now at about 200%. So we're seeing more than almost triple what we used to see. Now it was easier in the early days where the volumes were a little bit more lumpy. But at least you could say “okay, if it was found by automation, it's found by AI. It's probably junk.” Now the findings are actually pretty good.
Q. How are you adapting your vulnerability triage processes to handle the increase in findings?
A. We’ve hired more people to help get through some of this. That's where the critical findings are going to be triaged just by the humans. Then we've also enabled all of those humans by saying here's custom built bots to help you get through it even faster, and things like deduplication—that's a perfect use case for AI.
Q. Which vulnerability triage tasks are particularly well suited to AI assistance?
A. You don't need a human to look at deduplication. Is it the same thing? Yeah. Summarization is a great use case for an LLM to come in and just summarize something for me. What are the researchers trying to say? It makes it easier for them, makes it easier for the triager, and now they can get to a resolution faster.
Q. How is AI changing the economics of bug bounty research and the opportunities available to skilled hackers?
A. For the top 20% of security researchers, the earnings are going to go through the roof. But the payment per vulnerability on a unit basis is going to come down. with those folks that now, 24/7, churning out findings and submitting them on their behalf, now they're finding six-figure [bugs].
Q. How widely have security researchers adopted AI, and what does that mean for defenders?
A. It’s been a number of months now since we released our Inside the Mind of the Hacker report, but at that point, 82% of researchers were already using AI and that number is only going to go up. We've sped up quite a bit on the detection side, now it's about how do you actually fix them.
Q. How will agentic penetration testing change the value organizations receive from their security testing budgets?
A. I think what we'll see in the pen test space is, if a company was spending GDZMP, they've spent $100,000 on pen testing, probably still going to spend $100,000 on pen testing, but they're going to get a lot more coverage.
Q. Why will human penetration testers remain important even as autonomous testing becomes more capable?
A. The analogy I use is you're a big bank, you can let your agentic pen testing run across everything, but for your online banking portal, for your wire transfer portal, for investor relations, you're probably going to want UNET to do some testing.
Q. How quickly do you expect auditors and regulators to accept agentic penetration testing reports?
A. I don't think we're going to see the auditors or the regulators start to accept agentic testing reports right away. I think we'll get there, but they're going to want human overlay. That’s in the same way that they didn't accept a dynamic scan report as your pen test. They wanted human testing.
Q. Will autonomous penetration testing replace human testers or make them more productive?
A. I get asked the question all the time: Aren’t you nervous that this agentic pen testing may cannibalize your pen testing business? And, I go, “Really? It's much better margin profile than humans.” We'll always still have a human somewhere in the loop on that when we call it a true pen test. But you can make them a heck of a lot more efficient. You don't need a human writing a report.”
As AI makes vulnerability discovery faster and more accessible, the focus is shifting toward prioritization, remediation and meaningful oversight. Automated tools can help overwhelmed security teams manage growing workloads, but human expertise remains essential, particularly when assessing critical systems and validating security decisions.
Gerry notes that the broader challenge is ensuring that governance and accountability keep pace with AI's expanding capabilities. Ultimately, the interviewee argues that realizing AI's potential will require more than powerful technology: organizations must pair automation with effective safeguards, independent oversight and clear responsibility for the actions AI systems take.