How to Automate Customer Health Scores With AI

Health scores are only as good as the data behind them. And for most CS teams, that data is incomplete, stale, or scattered across three different tools that no one has connected yet.

AI doesn't fix that by magic. But AI tools do give you a way to centralize, standardize, and analyze signals across your customer lifecycle, so your health scores reflect what's actually happening, not just what made it into the last manual update.

This guide breaks down how to build AI-powered health scoring: which signals to use, how to set it up, and how to know if it's working.

Why Manual Health Scoring Fails at Scale

Manual scoring works fine for 20 accounts, but at 100+ accounts, the math breaks down.

There are a few simple reasons for that:

• There is only so much data you can take in and manage.
• The signals become more complex, or get lost in the noise, and something inevitably slips through the cracks.
• Manually updated health scores are based on old data, not what’s happening right now.

The score going into a QBR reflects whatever data was checked in the 48 hours before the call, not the full quarter, not the signal that got noticed but not documented. Accounts look healthy until they churn because a drop in login trend or a champion's departure never made it into the formula. And with a large book, a meaningful review of every account isn't realistic.

Unless, of course, you want to spend every weekend reviewing spreadsheets.

Manual health scoring may seem like it can give you a semi-accurate number to fall back on, but in practice, you’re missing something – and that something could be the difference between an upsell and a churn.

What Signals Belong in Your Health Score Model

Not even the slickest AI tool can save you if you have bad data or poor signals.

Good health scores are built on two things: 

1. Signals that predict churn (negative)
2. Signals that predict renewals or upsells (positive)

A customer who logs into your product multiple times each week, adds her team members, or tries a new feature you promoted is a happy customer. A customer who exports her data, logs in infrequently, if at all, or asks when their current contract expires could be an unhappy customer.

The trick is to identify the signals so you can manage your customers at scale. We recommend starting with the following:

• Product usage: Login frequency matters, but usage depth matters too. An account with high logins but low feature adoption is using the product out of habit, not because they are getting value.
  
  • Track: Active users against licensed seats, feature adoption depth, and time-to-value milestones.
• Engagement: An account where the main contact stopped responding to emails two weeks ago should raise some flags, but an account where your buyer has not attended a meeting in six months sends a much louder signal.
  
  • Track: QBR attendance, response rates to CSM outreach, and executive sponsor activity.
• Support health: Open ticket count is a starting point, but severity trends and time-to-resolution provide more insight. A cluster of escalated tickets in a short period is a different risk profile than a steady, low-level support relationship.
  
  • Track: Support tickets opened, NPS, resolution times, and sentiment (if you have a tool that can manage that).
• Financial signals: An account on net-30 that has gone to 75 days is worth watching, even if the product usage looks fine.
  
  • Track: Payment status, contract renewal timeline, and any history of expansion or contraction.
• Sentiment signals: An NPS score that drops from 9 to 7 over three surveys is meaningful in a way that a single data point is not.
  
  • Track: NPS/CSAT scores and trends, survey responses, and (if you have the tools for it) tone patterns in recent communications.

Treat the above as datapoints, not mandates. The factors you include in your health score and the way you weigh them will differ from ours and probably from your competitor’s.

Some of the signals above will be stronger than others: Support volume may not be too big a deal, but consistently low NPS scores and not logging into the product are. You’ll need to choose your signals and assign a “weight” to them so they are appropriately balanced in your score calculation.

Here’s a reasonable starting framework for weighting your different health signals: 

• Product usage: around 30%
• Feature adoption: 20%
• Support health: 20%
• Payment status: 15%
• Engagement: 15%. 

Again, this is not a universal formula. It should shift based on your customer segment, your product, and what your own churn data actually predicts about risk. But it is a better starting point than equal weighting across a handful of signals you happen to have access to.

The signals AI unlocks beyond this baseline are the ones that are hard to quantify manually: 

• Sentiment drift in email and call transcripts
• Organizational changes, like a champion leaving or a new buyer
• Cross-account pattern recognition 

But beyond that, the right AI tool can also help you spot trends and weight signals accordingly. For example, suppose 70% of the accounts that churned last year showed the same three-signal pattern in the 60 days before they left. A good AI model can pick that up and weight it appropriately.

An Example of Dynamic Health Scoring in Action

Let’s look at a real-life example of how powerful this type of data can be.

Linnea Olson and Nina Wilkinson, founders of ScaleUp CS, joined us for a recent webinar on how to scale your CS team and invest in tools as you grow. During the webinar, Linnea and Nina shared their firsthand experience leading CS at Apollo. 

Apollo is a sales data platform with four primary product lines, plus add-ons and upsells. While at Apollo, Linnea and Nina focused the onboarding team on ensuring customers tried at least two base products and one add-on. Then, the CSM team (measured against product adoption) focused on getting accounts to engage with the product, providing the team with a leading indicator for upcoming renewals.

The CS team also collaborated with Apollo’s BI team to run an analysis of their customers to predict renewals and churn based on product adoption and answer questions like:

• If a customer used three products, what was their likelihood of renewing? 
• How many of those customers actually got to renew or potentially expand on the platform? 
• If they used all four products, what did that number jump up to? What did our retention metrics look like? 
• What could they predict in terms of that renewed health? 

This information was critical and enabled the CS team to have strategic conversations about increasing adoption to drive renewals, rather than looking back and wondering why customers churned.

With the power of an AI-powered Customer Success Platform, you can run an analysis like this for your team. Read on as we give you the exact steps to follow to set up automated health scoring for your CS org.

How to Set Up Automated Health Scoring: A Practical Guide

Signals, weights… It’s enough to make your head spin.

Fear not. The above is actually quite simple to set up, as long as you have the right tools for the job. If you have an AI-powered Customer Success Platform like Vitally, the following five steps can take minutes rather than months:

1. Audit your data
2. Define your signals, weights, and thresholds
3. Connect your data sources
4. Build playbooks for score changes
5. Validate and adjust

Audit Your Data

Start with your data. Are you tracking what you need to track to support health scores that cover your customers at every stage of their journey with your product?

At a baseline, this means having:

• A CRM
• Product analytics
• Support data
• Billing data
• NPS and survey data

Remember from the example above with Linnea and Nina, every single datapoint you can collect matters. Before you build a scoring model, know which signals have clean, accessible data and which have gaps. A signal you cannot reliably pull is a signal you cannot score on.

Define Your Signals, Weights, and Thresholds

Once you’ve audited your data, it’s time to choose your signals and weigh them.

You don’t need a PhD in mathematics to figure this one out. We recommend starting with 4-6 signals as a baseline to iterate on, focusing on the clearest signals you have that point to either churn or renewal. It helps to document your reasoning for each signal and its weight, so you can adjust this as you go.

If you use a CSP like Vitally with built-in health scoring features, you can adjust signals and weights directly in the app, no code needed. Then, set your thresholds. These can and should change, but it helps to define what “healthy” and “at-risk” look like for your org and map your scores accordingly.

Vitally makes this incredibly easy by allowing you to define signals and weights on the fly. You can even customize the traits that contribute to these signals, including those from the tools you’ve integrated with Vitally.

Connect Your Data Sources

You’re going to need data to power all of this, and lots of it.

Your next step is to connect your tools – your CRM, billing system, support system, product data – in one place so that you can feed data from across sources into your health score model.

Fragmented data produces a fragmented score. This is where having a CSP like Vitally that integrates with all your tools and standardizes your data in one place for deeper analysis is so helpful. For example, Vitally can ingest product data to trigger real-time alerts, integrate with your CRM, collect survey responses, and even integrate with your data warehouse to capture every signal and power your health scores.

Build Playbooks and Automations

Automation is where those health scores transform from a nice dashboard to look at into something that drives Customer Success at scale.

A health score that drops without triggering a CSM action is still a manual process; someone still has to notice. With built-in playbooks, you can automate actions based on score changes without having to lift a finger. 

Vitally’s Playbooks, for example, can fire automatically when a health score crosses a threshold, routing the account to the right CSM, creating a follow-up task, or kicking off a save play without anyone having to catch the signal manually.

Validate Before You Trust at Scale

Before we get carried away with the benefits of AI-powered health scoring here, you need to validate that your scores are accurate before running all the above at scale.

The best way to do that is to back-test your scoring model against accounts that churned or expanded in the last 12 months. If the scoring model accurately flags at-risk accounts, you’re golden (at least, with a gentle hand on the wheel). If accounts slip through the cracks or your model raises false flags, something is missing, and you need to adjust and retest.

Even with an AI-powered platform like Vitally, your health scores may take some tinkering and refinement to get right. They will also evolve as you uncover new signals in your data over time, so don’t feel like you need to get everything right on day one.

To give you a head start, check out our AI prompt book below for AI prompts you can use to analyze your customer data for trends. 

Get 23 AI Prompts Designed to Help CS Teams Scale

Let’s set the record straight on AI: it’s not about removing you from the equation. It's about removing as much from your plate as possible so you can focus on what matters.

Health scoring and data analysis are prime examples of how AI can do the heavy lifting for your team, freeing you from time-consuming, error-prone processes and allowing you to do your best work.

To help you even further, we pulled together our 23 favorite AI prompts for Customer Success. In this guide, we share prompts for QBR prep, call follow-up, and churn mitigation. Click the button below to get your copy.