Ep 220: Engagement scores don’t tell the full story | CPOs of DraftKings, Klaviyo & Smartcat

Listen to this episode

The companies moving fastest aren’t asking “can we use AI?”

The brutal truth about AI in HR isn’t that bots will take jobs…

It’s that people who know how to leverage AI will outperform those who don’t.

Here’s what the group said is actually working:

1) Linda said to start small and stay specific

  • DraftKings replaced 85% of their basic CX tickets with AI
  • Now agents focus on complex problems that actually need human touch
  • Result: Better service, happier employees

2) Reward innovation attempts (not just outcomes)

  • Stacey shared how Smartcat gives quarterly awards for coolest IDEAS not results - Creates psychological safety to experiment w/ AI
  • Managers measured on innovation adoption

3) Give explicit permission

  • DraftKings embedded AI in performance reviews
  • Built a ChatGPT bot for goal setting
  • Quality of reviews improved dramatically

The companies moving fastest aren’t asking “can we use AI?”

They’re asking “how can we use AI to multiply human impact?”

Stop obsessing over engagement scores

We had a fascinating convo with the crew about why traditional engagement surveys don’t tell the full story…

Key insights from Linda, Carmel, and Stacey:

1) Surveys should drive listening, not action

Most companies rush to create action plans from their bottom 3 scores. But what if employees don’t actually care about those things? Focus on understanding what truly matters.

2) Timing matters more than you think

Don’t launch surveys right after:

  • Bad earnings calls
  • Comp cycles
  • Layoffs (yeah this happens way too often)

3) The Oura Ring effect

Carmel made a brilliant point - engagement surveys are like Oura rings.

They tell you what you already know (like yeah, those 3 glasses of wine killed my sleep score).

The real value is using them as signals to dig deeper.

4) Quality > Quantity

  • Stop doing massive annual surveys.
  • Start having targeted conversations about specific experiences.
  • DraftKings does customer centricity surveys.
  • Smartcat measures innovation scores.

My favorite quote from Linda: “I fundamentally believe you should have a bias to listening”

Talent density - the metric nobody talks about but should

Stacey, Smartcat’s Global VP of People, shared how the teams w/ highest talent density were consistently:

  • Most engaged
  • Most productive
  • Most innovative

But here’s the interesting part… they measure talent density by triangulating:

  • 9-box performance data
  • Ramp-up time metrics
  • Pre-employment assessments
  • Departmental success metrics

Traditional engagement surveys miss this completely. She says we’re all chasing eNPS scores while ignoring the fundamental connection between talent quality and engagement.

Think about it - when you cluster high performers together, they naturally:

  • Push each other to innovate
  • Raise the bar on quality
  • Create positive pressure

Smartcat found this so impactful they now use talent density as a leading indicator for engagement… way before survey results come in.

Measuring quality of hire

Carmel shared the three key focus areas they look at to measure quality of hire. Instead of tracking dozens of disconnected metrics, they focused on three key areas:

1) Time-to-value metrics

  • Sales: Days to first quota attainment
  • Engineering: Time to first PR
  • Support: Time to handle tickets independently

2) Manager feedback loop

  • Direct manager rating vs team baseline
  • Skip-level assessment (manager’s manager)
  • Clear expectations set during onboarding

3) Leading indicators

  • 30/60/90 day milestone completion
  • Peer feedback on collaboration
  • Onboarding satisfaction scores

The magic? They back-tested this framework against 6 months of hires and found it highly predictive of future performance.

Too many companies try to measure everything and end up measuring nothing well.

Pick your core metrics. Test them. Iterate.

Go listen to the full episode: Spotify | Apple

See you next week!

P.S. If you like MPL, help us grow the show by giving us a 5 star rating on Apple or Spotify.

Get weekly insights & playbooks from top Chief People Officers

    We won't send you spam. Unsubscribe at any time.