Webinar recap · 23 September 2026 · 10 minute read
Phil Gray, Director of AI and Change Capability, in conversation with Mavis Chan, Change Lead, Blue Seed Consulting.
Most organisations have now run an AI pilot that worked. Very few can show it changed anything. In our September webinar, Phil and Mavis unpacked why scaling AI pilots so rarely delivers, and what the small group seeing a real return does differently.
“Don’t scale the pilot. Scale the conditions.” Phil Gray
Here’s the conversation in ten minutes, plus answers to the questions attendees asked.
01 · The paradox
Three numbers that tell the whole story
Phil Gray picked three statistics from this year’s research. Together, they describe a strange situation.
Use is normal now
Percentage of US employees now use AI at work, up from 50% a quarter earlier.
Say AI delivers value
Belief is close to universal among senior leaders that AI is delivering meaningful business value, up 12 points in a quarter.
Value has landed almost nowhere
Percentage of leaders who report an established return on investment from AI.
These are self-reported surveys. Phil’s rule: defend the shape, not the decimal places.
Everyone’s using it, everyone’s committed, and almost nobody is better off.
Why both numbers are true
Savings don’t add up
A team that saves four hours a week doesn’t deliver four hours of value. Unless something changes around that team, the time simply disappears. As Phil put it: “When a team uses AI well and finishes at 3pm, AI hasn’t created value. It’s created 3pm.”
Speed moves the bottleneck
A legal team that drafts a contract in an hour instead of a day hasn’t sped up the deal. It has created a queue in someone else’s approvals. At pilot scale, a dozen people reach that approver. At rollout scale, it’s closer to 3,000, and the approver hasn’t changed.
Only the pilot got measured
Most organisations tracked savings inside the pilot, because that’s what the pilot existed to do. Almost nobody measures them at scale.
02 · Why the playbook stall
Why the usual playbook for scaling AI pilots stalls
The flashlight problem
Conor Grennan teaches AI at NYU. He once gave his elderly father an iPhone to replace a much-loved flip phone.
When he checked in a couple of weeks later, his dad said the phone had transformed his life. His big discovery? It had a torch.
We do the same thing when we scale an AI pilot. We reach for the parts we can already see.
He got a supercomputer, but he only found the flashlight
We haven’t written a new playbook for AI. We’ve rolled it out the way we roll out everything else.
Four familiar moves after a pilot, and why each one stalls
The lighthouse case study
REALITY: Communications
The pilot report, polished into a story, with the conditions that made it work stripped out.
The tool rollout
REALITY: Deployment
Thousands of people get the tool. None of them get the sponsor, the permissions or the Friday retro.
The use case library
REALITY: Enablement
People frame their use cases around what you showed them, not what the tool can actually do.
The champion network
REALITY: change network
Without authority, forums or escalation paths, it becomes “enthusiasm plus a calendar invite”.
POLL RESULTS
Effort isn’t the problem
We asked attendees: Which of these has your organisation done in the last 12 months?
Tool rollout or licences:
Champion network:
Use case library:
Case study or benchmarking exercise:
None of the above:
Reasons the pilot does’t scale
What the pilot had
- A sponsor who answered questions within the hour
- Explicit permission to bend the normal rules
- A weekly session on what worked, what didn’t, and what to change
- A team who carried all the context
What the rollout gets
- The tool
That’s the gap. Scale the conditions, not the pilot.
9 in 10 organisations in the room have already rolled out the tool. Nobody is sitting on their hands, which makes the 7% harder to explain, not easier.
03 · Where the constraint sits
It’s not skills. It’s what the organisation rewards.
Atlassian ran a controlled experiment with 961 knowledge workers. Everyone saw the same piece of work. The only thing that changed was whether the colleague said they’d used AI.
Colleague who didn’t mention AI
Colleagues judged work on its merits and rated the person accordingly.
Colleague who disclosed AI use
Source: Atlassian, “Honesty about AI use at work is backfiring”, 2026 (961 US knowledge workers).
Identical work. The only difference was the honesty
So people keep using it, and stop telling you.
During the pilot, saying you used AI was the whole point. The day after it closes, it stops being safe, and nobody sends a note saying the rules have changed back.
What removes the constraints
A policy telling people to disclose AI use won’t outweigh the consequences described above.
Here are four things that will make a difference:
Leaders go first
When senior people talk openly and specifically about how they use AI, everyone else learns that disclosure is safe.
Frame it as a team effort, not a shortcut
People are judged more kindly when AI use is framed as helping the team. In the study, that lifted ratings by 11 points.
Celebrate good use
In companies that actively celebrate AI use, the effect flips. People who disclose are rated more efficient than those who stay quiet.
Separate “must disclose” from “good to share”
Client-facing, image and regulated content need a clear rule. Everyday drafting needs a culture, not a rule.
At your next team meeting, ask the most senior person in the room to share one thing they used AI for this week, and what they changed in the output.
04 · What the 7% do differently
Doesn’t it just take a good CEO?
A good leader helps, but it isn’t what separates them. Phil’s favourite example is IKEA, because it shows what happens when you plan where the capacity goes.
2021
IKEA launches Billie, a customer service bot named after the bookcase.
Billie handles close to half of customer enquiries, saving about €13m a year.
Cost modelling suggested IKEA only needs half the number of contact centres; however, IKEA asked a second question:
“What can our people do that the bot can’t?”
The results:
€1.3bn
In remote design sales, 3.3% of IKEA’s total (FY22), with a target of 10% by 2028. Roughly a hundred times the €13m Billie itself saved.
IKEA’s 2026 corporate job cuts (around 1,650 roles) are a separate group, driven by sales and tariff pressure rather than automation.
IKEA didn’t run a better pilot. It decided where the capacity would go before the bot scaled.
POLL RESULTS
We asked attendees:
Who is actually accountable for AI adoption?
Not who’s named in the strategy – who gets the call when it goes wrong?
CIO or CTO:
CEO or exec sponsor:
Nobody, or not sure:
Business unit leaders:
A transformation or change lead:
Several attendees said they’d have picked more than one. That accountability feels so shared is part of the finding.
Over half the room said adoption sits with the people who own the tools, or with nobody at all. The CIOs we speak to put it plainly: “I’m responsible for putting it in. I’m not responsible for making sure it’s used, or for signing off on the benefits.”
Why it matters: organisations where the CEO is accountable for AI are nearly three times as likely to realise meaningful value.
Source: KPMG Global AI Pulse, Q2 2026 (57% with CEO accountability vs 21% without).
05 · The real risk
Getting it wrong just got cheaper
Big decisions used to be expensive, so organisations built machinery to protect themselves from bad ones. AI takes the brakes off. Rolling out something that changes how whole teams work is now easy. So is getting it wrong at scale.
Business case
Govenance
Testing
Budget cycle
BEFORE
Applies
Applies
Applies
Applies
AFTER
Skipped
Skipped
Skipped
Skipped
One question for your exec
“How old is ‘bad news’ by the time it reaches someone who can act on it?”
-
Ask it about the pilot. What was the worst thing that happened during the pilot?
-
Then time it. How long did that news take to reach someone who could act on it?
-
Read the answer. If nothing went wrong, you didn’t run a pilot. You ran a product demo.
A pilot that doesn’t scale isn’t a failed pilot. And a pilot that “worked” while nothing changed around it hasn’t told you much at all.
06 · Your questions
Your questions, answered
Attendees asked more questions than we had time for. Here are full answers to the questions that matter most.
Most organisations don't do standard change well. How do we use AI to help land change better?
Short answer: don’t use AI to produce more change material faster. Use it to shorten the feedback loop.
The four moves above apply here too. If AI just writes the comms plan, the case study and the training deck quicker, you’ve sped up the playbook that isn’t working. AI earns its place in sensing, not broadcasting:
- Synthesise pulse surveys, open-text feedback and Q&A in hours, not weeks, so you see resistance while you can still act on it.
- Tailor one message by role and impact, so each group hears what changes for them.
- Stress-test a change before launch by asking AI to play a sceptical frontline team leader.
- Keep impact assessments live as the change evolves, instead of a document that dates the week you sign it off.
Try this: pick one live initiative and use AI to turn last month’s open-text feedback into themes in an afternoon. Then check how long those themes take to reach someone who can act on them.
These tools are expensive. Does cost explain why so few leaders report a return on investment? And what about intangible benefits?
Short answer: cost is part of it, but it’s not the main story.
Most organisations don’t have a clear picture of what they spend on AI, so they can’t prove what they got back. Leaders who do have that visibility report a return about five times more often. But even they mostly don’t.
The bigger gap is on the other side of the ledger. Most organisations count the hours AI saves and stop there. Nobody decides in advance what those hours will become. That’s why “soft” benefits get waved away. They’re not too vague to count. Nobody has linked them to anything finance already tracks.
Try this: before your next pilot, finish this sentence. “The capacity this frees up will go to ___, and we’ll know it’s working when ___ moves.” If you can’t fill both gaps, you don’t have a business case yet.
Is there an AI change readiness index that looks at maturity, including capability at every level?
Short answer: not one that covers the full picture on its own.
The Australian Responsible AI Index (Fifth Quadrant, with the National AI Centre) is a solid starting point. It measures responsible practice: fairness, accountability, transparency, explainability and safety. What it doesn’t measure is whether your people and operating model are ready to change how work gets done. A true readiness baseline asks:
- Leadership: who owns adoption and benefits, not just the platform?
- Capability: can people at every level use AI with judgement, including leaders asking the right questions of a pilot?
- Culture: do people feel safe to disclose AI use and raise bad news early?
- Ways of working: where does freed-up capacity go, and has anyone redesigned the work around it?
- Data and decision rights: are the rules an agent would follow written down, agreed and current?
- Change capability: can the organisation absorb change weekly, not quarterly?
At Blue Seed, we baseline this through our Change Blueprint, a six-lens view of change maturity.
Try this: if you’ve completed the Responsible AI Index, keep it. Then take the six lenses above and see how many you can answer with evidence. The ones you can’t are your baseline.
Source: Fifth Quadrant, Australian Responsible AI Index 2025.
How many organisations use AI agents successfully, and how does Australia compare?
Short answer: fewer than the surveys suggest, and Australia isn’t lagging on appetite.
Industry surveys report over half of organisations “deploying” agents. But “deployed” usually means someone switched it on, not that it has changed how work gets done. Phil’s estimate from the organisations he works with: about one in five are piloting agents, and only a quarter of those have scaled them. That’s well under one in ten, close to the 7% ROI figure, reached a completely different way.
What holds agents back will sound familiar: data an agent can trust, decision rules that live in one place, and people with the skills to oversee them.
On Australia, research places us in an “intermediate tier” alongside the US, UK and Japan. Australian and New Zealand customer experience leaders are far more likely than global peers to call agentic AI critical (39% vs 22%). The real pace-setters here are regulation, industrial awards and consultation obligations, and that clock runs however keen the organisation is.
Try this: pick one process you’d consider for an agent and check whether its rules live in one agreed place. If they live in three, start there. If the agent could change roles covered by an award or enterprise agreement, bring your industrial relations and people teams in at design, not rollout.
07 · Watch
Watch the full conversation
Key takeaways – 30 mins
Full recording: 50 mins
We’re working with more and more organisations on AI, from early pilots to real adoption.
If you want to find out more about how we can help:
