6 ways to make AI accountability stick

Forward-thinking IT leaders are redesigning ownership, observability, and operational controls to make accountability enforceable.

As intelligent systems move into production environments and begin taking actions, organizations quickly discover that accountability becomes much harder. Unlike traditional enterprise software, these tools can produce unpredictable outcomes as they interact dynamically with data, APIs, and business workflows.

“When something goes wrong with AI, it is generally assigned to whoever was closest to the pain point,” says David DuChene, manager of data and AI pre-sales at SHI International, which works with enterprises on AI deployments and governance.

As these systems shift from advisor to actor within workflows, accountability becomes harder to enforce through policies alone. IT leaders must build it directly into the fabric of their operations through clear ownership, continuous observability, defined escalation paths, and infrastructure designed to make responsibility visible when things go wrong.

Here are six ways to make AI accountability enforceable in production.

1. Assign direct ownership from the beginning

Many enterprises still view AI accountability as a shared responsibility, but some experts argue that this is the first assumption to fail when systems enter production.

“Shared accountability is not accountability,” says Joe Wilson, SVP and CIO of CSG, a customer experience, billing, and payments software provider. “You need a direct owner.”

He says that at CSG, AI initiatives go through governance reviews involving executive leadership, and direct ownership is assigned at the start of projects. Wilson, who oversees the AI governance and deployment strategy for CSG, says the company also created “CIO reps” embedded inside business units and product groups to ensure accountability spans the entire lifecycle of AI initiatives.

According to SHI’s DuChene, many enterprises still lack formalized accountability structures for those environments. “They may have responsible parties on paper, but once a system actually breaks down, everything gets relitigated,” he says. “It goes back to who’s closest to the pain point.”

One diagnostic question, he argues, reveals whether organizations are truly prepared: “If your AI deployment generates a wrong answer and costs the business money tomorrow, who’s going to write the postmortem?”

If leaders cannot answer that question quickly, accountability structures likely don’t yet exist in practice.

2. Build governance before scaling deployments

In the past few years, many enterprises deployed AI systems before establishing the governance and operational foundations necessary to support them safely. “The biggest gap we see is a sequencing problem,” says DuChene. “We’ve gone around and built a bunch of houses where we’re standing up the walls before we’re pouring the foundations.”

That sequencing problem creates expensive retrofitting efforts later. DuChene says teams frequently discover they lack data classification systems, AI-aware identity and access controls, lineage and provenance tracking, audit capabilities, and escalation channels for failures.

According to Seth Dobrin, CEO of deterministic AI model maker Arya Labs and former global AI leader at IBM, governance often fails because organizations treat it as a policy layer rather than something embedded directly into operational workflows. “How do you integrate it into the workflow?” he asks. “If you don’t get that right, it’s going to fall apart.”

Dobrin recalls working with an insurance company that spent 18 months building an intelligent system before legal teams blocked deployment entirely. The problem was not the technology itself, but the absence of governance early in the process. “They had to throw it away,” Dobrin says. “Had they started earlier, they would have steered it to a place where they could have gotten to yes.”

Dobrin says governance should not slow projects down. Instead it should be integrated deeply enough into workflows that teams can move quickly without downstream compliance or operational failures. “The objective should never be to say no,” he says. “It should always be to figure out how to say yes.”

Wilson at CSG makes a similar point, arguing that governance should help teams absorb complexity rather than simply restrict what they can do. He compares it to a vehicle suspension system rather than a braking mechanism. “Our intention is not to slow things down,” he says. “Our intention is to speed stuff up, but also when you get into rough terrain, to be able to navigate that terrain.”

3. Treat data governance as the foundation of accountability

According to Wilson, CSG focused on governing its data before scaling AI initiatives across the business. Those efforts started with data synchronization and privacy impact assessments.

“The foundation is data,” Wilson says. “If we don’t have clean, synchronized, and governed data across the board, we’re not going to win this battle.”

Many organizations underestimate how difficult it becomes to maintain accountability once AI systems begin interacting with fragmented enterprise data environments, says Quais Taraki, CTO of EnterpriseDB, a company that works with enterprises on data infrastructure and governance.

An AI assistant summarizing customer interactions, for example, may pull regulated or confidential data from systems that were never intended to feed generative AI tools.

Strong data governance practices — including lineage, provenance tracking, classification systems, and access controls — not only help head off such problems but also create the foundation for accountability when something does go wrong. Otherwise, teams struggle to determine what data an AI system accessed, how outputs were generated, and whether sensitive information influenced a decision.

“Without lineage and provenance, you can’t do root-cause analysis,” Taraki says. “You won’t know what to change, or how things mutated in ways you didn’t expect.”

Taraki argues that accountability should follow governed data products rather than organizational silos. When ownership is split across infrastructure teams, data scientists, and application developers, responsibility can become difficult to establish after failures occur. Assigning clear ownership to the data products that feed AI systems helps create accountability throughout the AI lifecycle.

4. Build observability into (and beyond) AI systems

Traditional enterprise monitoring systems were designed primarily to track uptime, infrastructure health, and application performance. AI introduces a different challenge: tracing reasoning paths, decision chains, and behavioral drift.

Nik Kale, a member of the Coalition for Secure AI (CoSAI) and participant in AI security and agent identity standards efforts, describes this through what he calls an “Investigation Graph.” This is a reasoning trail showing what an AI system observed, what tools it accessed, what conclusions it reached, and what actions it ultimately took.

“When something breaks, the first instinct is always to ask, ‘Why did the AI make that decision?’” Kale says. “Honestly, I think that’s the wrong question. The right question is, ‘What did the system actually do?’”

That distinction is increasingly important because AI failures rarely originate from a model alone. Instead, they emerge from interactions between models, credentials, APIs, workflows, policies, and downstream systems.

“The model didn’t act,” Kale says. “The system around the model acted.”

That broader view of accountability is also changing how IT leaders think about observability. Rather than monitoring AI models in isolation, enterprises increasingly need visibility across the systems those models interact with, including data sources, APIs, applications, security controls, and downstream workflows.

In practice, that starts with comprehensive logging of prompts, model outputs, tool calls, data access events, and agent actions. Combined with traditional application and infrastructure telemetry, those logs create an auditable record of how AI systems behaved and why decisions were made.

That visibility becomes especially important when IT leaders try to identify unauthorized AI usage. While governance policies define which tools employees should use, observability helps reveal which tools they are actually using. Unusual data access patterns, unexpected API calls, traffic to external AI services, and unexplained movement of sensitive data can all be indicators of shadow AI.

Even well-governed organizations can struggle when employees adopt unauthorized AI tools outside approved workflows. “If it’s shadow IT, we don’t even know it exists,” says DuChene. “We don’t know what data of ours is going into it, how it’s being used, or how it’s being distributed.”

By extending observability beyond AI models to the broader enterprise environment, IT can detect those activities earlier, investigate them more quickly, and reduce the accountability gaps that shadow AI creates.

5. Create ‘escalate’ and ‘stop’ mechanisms

The most important accountability question may not be what an AI system can see or do, but when it should stop and ask for help.

According to Kale, that’s often the most underdeveloped part of enterprise AI deployments. “Most enterprises have figured out how to monitor their AI systems,” he says. “But nobody has really built the third piece, which is, when does the system actually stop and ask for help?”

Kale argues that enterprises need explicit escalation paths, human decision points, and clearly defined stop mechanisms for systems operating in production.

“You don’t want a rubber stamp — you want a human in the loop,” he says, adding that the human should be named and have the authority to say no.

According to Wilson, incident response processes also need to evolve, because AI failures behave differently from traditional IT outages. “A traditional IT incident typically looks like it’s an up or down scenario,” he says. “AI failures are a little more subtle than that.”

Models may drift gradually, outputs may degrade over time, or workflows may begin producing unexpected results without systems technically failing. The result, says Wilson, is a growing need for multidisciplinary response processes involving legal, communications, security, audit, business teams, and IT operations simultaneously.

6. Treat AI systems more like workers than software

Some enterprises still govern AI like traditional applications. But according to Kale, AI systems behave more like workers and less like deterministic software.

“You cannot just deploy once and be done,” he says. “Like workers, they need ongoing oversight.”

That ongoing oversight is becoming a core accountability function. Employees are not hired, trained, and then left unsupervised indefinitely. Managers monitor performance, provide feedback, evaluate changing responsibilities, and intervene when behavior drifts from expectations. Kale argues that AI systems increasingly require similar treatment.

Traditional software can often be reviewed and approved at release time because its behavior remains relatively stable between versions. AI systems are different. Models evolve, prompts change, retrieval systems are updated, and the information available to agents changes continuously.

That challenge extends beyond internally developed systems. Enterprises must also monitor the third-party AI services they rely on. Not only do vendor models evolve on their own, but vendors also update software and capabilities behind the scenes.

“The vendor we approved last quarter is functionally a different vendor this quarter,” Kale says.

As a result, accountability cannot end when a system is deployed. Someone must remain responsible for monitoring performance, reviewing changes, assessing risk, and determining whether systems continue to operate within acceptable boundaries. Kale points to CoSAI’s AI Shared Responsibility Framework as one emerging effort to clarify those responsibilities across enterprises, software vendors, model providers, and infrastructure operators.

The organizations making the most progress are discovering that accountability cannot be assigned on paper and forgotten. As AI systems become more autonomous, accountability is becoming an operational capability built into data governance, observability, escalation processes, and ongoing oversight. For IT leaders, the challenge is no longer defining responsibility. It is making responsibility enforceable.