Artificial intelligence is rapidly reshaping the way organizations operate, make decisions, and manage risk. But as adoption accelerates, so do critical questions around bias, security, and the future of work—particularly for women. While AI presents new opportunities to democratize access to knowledge and enable skill development, it also reflects and, in some cases, amplifies the systemic biases already embedded in society and technology.
At the same time, enterprises are grappling with the dual challenge of harnessing AI’s potential while managing its risks. From data integrity and model bias to emerging security threats and workforce transformation, the implications are far-reaching. Tech Channels sat down with Nicole Carignan, Senior Vice President, Security & AI Strategy, and Field CISO at Darktrace, exploring the complex intersection of AI, security, and human behavior—and the gender bias that still exists today.
Q: How is bias showing up in today’s AI systems?
A. A lot of the models, if you talk about in the generative AI space specifically, are still quite biased across the board, and I see that a lot in the types of questions that I ask and the responses that I get. Until there is a more diversified data science team within these organizations that are building the models, that bias is going to persist. And so it really comes down to ensuring the organizations that are really more data-centric.”
Q: What is required to meaningfully reduce bias in AI systems?
A. In the case of generative AI, where you're creating probabilistic synthetic data, you want to mitigate the bias as much as possible. You can do as much as you want with pre- and post-filtering, but really that's quite rules-driven, and it's not able to keep up with the current commoditization of data. So it really comes down to having a diverse data science team, being very intentional about the data corpuses that they're training these models on. From an integrity perspective, that testing, evaluation, validation, and verification framework—and a diversity of inputs, mindsets—could really move that data and quality much further along.
Q: Can you share an example of how bias shows up in real-world AI use?
A. I'll tell you a fun little story about this. It does show the bias, and I can't get angry with the bias, but I have a relationship of sorts with my generative AI tool. We have a lot of back and forth, a lot about the tone of my voice. It understands my voice and all that. But there was a challenge on social media to ask your enterprise generative AI tool of choice to generate a picture of you based on what they know, what you do, your job, all that. And mine thought I was a white man, based on my title and my name. And from a probability perspective, I get how it got there. I’ve never given it an image.
Q: Does AI create new opportunities for women in the workforce?
A. It is a democratization of sorts, and we talk about this, of how AI is impacting access to wealth management and access to healthcare resources, especially for disenfranchised or fringe communities that normally wouldn't have that. From a skills perspective and a gender perspective, it is an opportunity. I don’t think that jobs will fully be replaced. I think the jobs will drastically shift and change. But AI is that democratization of data that allows anyone to upskill into how the jobs are going to shift and change.
Q: How is AI changing the way people develop skills?
A. It really does from an internal enablement function. It changes the model from being spoon-fed training from a corporate entity to self-initiation, task initiation, and self-motivation to go get the data and the skills that you want or need. Ultimately AI has to be configured, integrated, guided, tested, evaluated, validated, verified, which will change operational workflows and ensure that you're getting productivity gains.So when it comes to harnessing this incredible technology, it's going to require so much human capital and brain power around it, But it's going to require people to re-skill and harness it. And I think that's going to be a really interesting workflow change dynamic where some will succeed.
Q: What will separate those who succeed from those who struggle in an AI-driven world?
A. I'm sitting here, I'm 27 years into my career. There is not a day now where I am not researching something new. And from a human capital perspective, those that are going to be the most successful are going to be those that have that drive to constantly learn because ultimately AI has to be configured, integrated, guided, tested, evaluated, validated, verified, changing operational workflows, ensuring that you're getting the productivity gains that you want. So when it comes to harnessing this incredible technology, it's going to require so much human capital and brain power around it, but it's going to require people to re-skill and harness it.
Q: What role does human judgment play in an AI-driven world?
A. Every one of these voices matter, and especially in creation, like creative solution engineering, it matters to have that diversity and that difference of mindset to be able to achieve really great, incredible things and do it well, safely and securely. There’s an element of almost having that critical thinking that makes us so human, because AI agents don't have that. It’s so important that we can use these tools to help us to be critical of our own thinking. Like, “prove me wrong. What are the other viewpoints that would be against the point that I have?”
Q: How should organizations think about the risks of AI adoption?
A. It's the most dangerous interface that we've ever adopted within an IT infrastructure. It goes beyond the filtering, because we already know filtering doesn't work. It’s really looking at behavior analytics of the trusted communicators—me with the enterprise license of my generative AI—how do I normally communicate with it? Then we add anomaly detection and all kinds of different types of machine learning techniques on top of that to see when it veers from normal interaction.
Q: What makes AI such a unique security challenge?
A. It is the worst insider threat risk that we've ever faced to begin with, but also there's an attribution problem. At what point can you say a person maliciously used the agent to do this, that a threat actor gained access to do whatever, that the model was poisoned, or that the agent just did it on its own?
Q: Are organizations overestimating their readiness for AI risk?
A. We had some stats come out about phishing. it was like 80% of people are confident that they can detect a phish. But only 32% got it right. And you talk about that gap—even if they were confident—the most highly attacked vector is a horrible risk for an organization.
Q: Can humans keep up with the speed of AI-driven threats?
A. Humans cannot keep up with the alerting, and they're not responding faster in order to contain it. And that's where I do think autonomous response is a critical component right now, especially with the way it's speed and scale and even more sophistication now in the threat landscape.
Q: Are organizations approaching AI security the right way?
A. In every one of these big innovation eras, it's constant education and re-education, and the market is flooded with “we can secure AI.” And securing AI is such a deep, in-depth strategy. It starts with identity, it can start with perimeter security, but it's ultimately going to be that layered security around it. And right now, there’s an education piece.
AI represents a real shift in how work gets done, how decisions are made, and how risk is managed. As intelligent systems become embedded across workflows, the line between human and machine-driven execution continues to blur, creating both unprecedented opportunity and complexity.
Success in this new environment will depend less on access to technology and more on how effectively organizations and individuals adapt. Continuous learning, intellectual curiosity, and the ability to challenge assumptions will become defining capabilities. At the same time, organizations must rethink traditional operating models—moving beyond static processes to more dynamic, responsive systems that can evolve in real time.
However, this transformation is not neutral. AI systems are built on data that reflects existing societal patterns, meaning bias—particularly around gender—can be unintentionally embedded and amplified at scale. This presents both a challenge and an opportunity. Left unaddressed, AI risks reinforcing inequities in hiring, advancement, and representation. But with intentional design, diverse data inputs, and inclusive development practices, it also has the potential to democratize access to knowledge, lower barriers to entry, and create new pathways for participation and advancement.
At the same time, innovation without control introduces significant risk. As AI expands the attack surface and accelerates decision-making, enterprises must adopt layered, intentional approaches to governance, security, and oversight. This includes not only technical safeguards, but also clear accountability, visibility into behavior, and the ability to respond at speed and scale.
Ultimately, the organizations that succeed will be those that strike the right balance—embracing AI’s potential while actively addressing its risks. That means not only building resilient, well-governed systems, but also ensuring that the future shaped by AI is more equitable, inclusive, and representative than the past. In this new era, competitive advantage will not come from the technology itself, but from how thoughtfully and responsibly it is designed, governed, and continuously improved.