The AI productivity trap hiding behind every 'time saved' stat

AI is driving up individual productivity, but why isn’t it translating into business outcomes?

The answer, according to new data from AI company Glean, is two related phenomena: ‘botsitting’ and ‘botshitting’. Together they are draining the AI productivity dividend.

Glean’s Work AI Institute surveyed 6,000 workers in the US, UK & Australia and found that 75% of workers said AI is driving up productivity, with them saving 11 hours a week.

However, they are also spending 6.4 hours botsitting, the hidden labor involved in managing the tools and their output, including cleaning up AI-generated work, debugging errors and switching between tools.

This frustration makes them more likely to look for another job (73%), as well as leading to the more concerning second concept: ‘botshitting’.

This is where workers simply cognitively check out and start delivering AI-generated work without checking or verifying it. 69% of those surveyed admitted to the practice.

Furthermore, 41% said they delivered work they couldn’t explain or defend if asked, and four in ten blamed AI for the mistake.

It is easy to assume that better tools will lead to less botshitting – but the opposite is true. Glean’s data found that the tools that workers said brought the most productivity gains – ChatGPT (67%) and Claude (59%) – were the same tools that drove the most lax behavior: 71% and 92% respectively.

Glean’s data shows that the 13% of organizations breaking these cycles are not those using more AI, but those who redesign work.

As the report stated: “That’s the choice in front of every organization. Build the human infrastructure that makes AI worth using. Or keep paying the bill — in botsitting, in botshitting, and in the steady departure of the people who got tired of cleaning up after the bots.”

Building that human infrastructure is easy to say, but much harder to do.

Here are five things successful companies are doing as they redesign work for the AI era.

It’s easy to lean into vanity metrics around quantity of AI use – these metrics are easy to count.

Rebecca Hinds, Head of the Work AI Institute at Glean, tells UNLEASH the issue is that "metrics don't just measure behavior, they incentivize certain types of behavior."

“Whatever we measure in the organization, it tells our employees what the organization values.”

So, if organizations just measure how much people are using AI, they are saying “it’s more important to click the tool than really be thoughtful about how you are using the technology to advance business objectives.”

74% of employees in organizations that only measure productivity with AI admit to ‘botshitting’, but when productivity and quality of AI use are measured that figure drops to 64%.

While there’s no silver bullet metric for AI, Hinds shares that the most effective companies “measure more things on average” and anchor their data collection in “existing KPIs, business objectives.”

They also listen to employees and collect qualitative data about their experience with the tool, asking questions such as ‘Do you like using the tool?’ and ‘Do you feel worn out by the tool?’

More than half (53%) of workers told Glean that the essential information they need to do their jobs is not accessible through their AI systems – this is “exhausting” as individuals have to “go on a detective mission to figure where the information exists” and share it with the AI, Hinds says.

Organizations will argue that they have given their AI tools access to data, but that’s not the same as giving it context.

When organizations successfully provide AI with context, workers spend 9% less time ‘botsitting’ and are 31% less likely to ‘botshit’.

“The fact that LLMs are trained on the vast corpus of the internet doesn’t do a lot of service for AI that’s trying to deeply understand how your organization works” – that’s why Hinds recommends that organizations embed a context layer into their tech stack.

This layer “is very important in being able to surface not only any information, but relevant information to you as an employee, to you as a team, to you as an organization.”

Glean’s report notes that leading organizations don’t start with the tools, but with the work itself. They select “tools and platforms that fit the job instead of letting vendor contracts dictate their AI strategy.”

It is easy for enterprise companies who have built up relationships with vendors to continue to work with those partners – “it’s the path of least resistance,” notes Hinds.

This is especially true as leaders in HR and beyond are “facing pressure to adopt the technology” at pace.

However, leading organizations are “much less likely to be held back by those existing vendor relationships...they know that AI touches every function within the organization [therefore] it requires a holistic strategy” when choosing tools.

It could very well be the case that your existing technology partners don't have technology that's going to work across the organization, notes Hinds.

She advises organizations not to think “about the solution first” and instead focus on the problem they want to solve with AI, then find the best technology to solve that challenge.

“This is such a complex technology that is evolving very quickly. If you choose a partner who is going to help you understand the evolution, then, as best as possible, you’ll start to weatherproof your tech stack for the inevitable changes,” concludes Hinds.

Humans are naturally resistant to change. When it comes to AI, people are worried both about being replaced by the technology, and becoming obsolete if they don’t embrace the tools.

This explains why they are disengaging at work by ‘botshitting’. To break the cycle, organizations – specifically the HR function in Hinds’ view – need to step up their upskilling and reskilling around AI.

They need to introduce formal learning programs around AI, but for Hinds and Glean, it's also important to address how employees think about the technology. Hinds explains this in terms of ‘mental models’.

“Mental models can significantly influence how workers use the technology,” Hinds tells UNLEASH. To break unwanted behaviors, HR needs to encourage employees to not see AI as a tool, which always gives perfect answers, but instead as a teammate they collaborate with.

Glean's data showed that 75% of the highest-achieving employees trust AI as a teammate, versus just 32% of low achievers — a gap of 2.3x.

Having an AI policy is not the same as having AI governance.

14% of workers admitted they had never read their organization’s AI policy – this helps explain why 50% use AI in a non-compliant manner at work.

Hinds tells UNLEASH that leading organizations “not only have a policy, they also tend to have a clear rationale behind the policy” (91% vs 57%).

Sharing the why behind decisions is essential to building trust and compliance with employees around AI – as discussed in depth during a recent UNLEASH webinar in partnership with Qualtrics, available to watch on demand now.

“Another characteristic is that transformative organizations are continuing to review that policy” – 93% of them do this regularly, compared to 55% of standard organizations.

As a result, AI governance “is not something that’s one and done, it’s something that is continuously evaluated over time,” adds Hinds.