generate buyer journeys that seem clean and interesting, however evaluating whether or not these journeys are structurally sound stays difficult for present strategies.
This text introduces Continuity, Deepening, and Development (CDP) — three deterministic, content-structure-based metrics for evaluating multi-step journeys utilizing a predefined taxonomy quite than stylistic judgment.
Historically, optimizing customer-engagement programs has concerned fine-tuning supply mechanics comparable to timing, channel, and frequency to attain engagement and enterprise outcomes.
In observe, this meant you skilled the mannequin to grasp guidelines and preferences, comparable to “Don’t contact clients too usually”, “Shopper Alfa responds higher to telephone calls”, and “Shopper Beta opens emails largely within the night.”
To handle this, you constructed a cool-off matrix to stability timing, channel constraints, and enterprise guidelines to manipulate buyer communication.
To date, so good. The mechanics of supply are optimized.
At this level, the core problem arises when the LLM generates the journey itself. The problem is not only about channel or timing, however whether or not the sequence of messages kinds a coherent, efficient narrative that meets enterprise targets.
And all of the sudden you notice:
There isn’t any normal metric to find out if an AI-generated journey is coherent, significant, or advances enterprise objectives.
What We Count on From a Profitable Buyer Journey
From a enterprise perspective, the sequence of contents per journey step can’t be random: it should be a guided expertise that feels coherent, strikes the shopper ahead by means of significant phases, and deepens the connection over time.
Whereas this instinct is frequent, it’s also supported by customer-engagement analysis. Brodie et al. (2011) describe engagement as “a dynamic, iterative course of” that varies in depth and complexity as worth is co-created over time.
In observe, we consider journey high quality alongside three complementary dimensions:
Continuity — whether or not every message matches the context established by prior interactions.
Deepening — whether or not content material turns into extra particular, related, or customized quite than remaining generic.
Development — whether or not the journey advances by means of phases (e.g., from exploration to motion) with out pointless backtracking.
Why Present LLM Analysis Metrics Fall Quick
If we have a look at normal analysis strategies for LLMs, comparable to accuracy metrics, similarity metrics, human-evaluation standards, and even LLM-as-a-judge, it turns into clear that none present a dependable, unambiguous technique to consider buyer journeys generated as multi-step sequences.
Let’s look at what normal buyer journey metrics can and may’t present.
Accuracy Metrics (Perplexity, Cross-Entropy Loss)
These metrics measure confidence degree in predicting the following token given the coaching knowledge. They don’t seize whether or not a generated sequence kinds a coherent or significant journey.
Similarity Metrics (BLEU, ROUGE, METEOR, BERTScore, MoveScore)
These metrics examine the generated consequence to a reference textual content. Nonetheless, buyer journeys hardly ever have a single right reference, as they adapt to context, personalization, and prior interactions. Structurally legitimate journeys might differ considerably whereas remaining efficient.
Undoubtedly, semantic similarity has its benefits, and we’ll use cosine similarity, however extra on that later.
Human Analysis (Fluency, Relevance, Coherence)
Human judgment usually outperforms automated metrics in assessing language high quality, however it’s poorly fitted to steady journey analysis. It’s costly, suffers from cultural bias and ambiguity, and doesn’t operate as a everlasting a part of the workflow however quite as a one-time effort to bootstrap a fine-tuning stage.
LLM-as-a-Choose (AI suggestions scoring)
Utilizing LLMs to judge outputs from different LLM programs is a powerful course of.
This method tends to focus extra on model, readability, and tone quite than structural analysis.
LLM-as-a-Choose could be utilized in multi-stage use circumstances, however outcomes are sometimes much less exact as a result of elevated threat of context overload. Moreover, fine-grained analysis scores from this technique are sometimes unreliable. Like human evaluators, LAAJ additionally carries biases and ambiguities.
A Structural Method to Evaluating Buyer Journeys
Finally, the first lacking component in evaluating advisable content material sequences throughout the buyer journey is construction.
Probably the most pure technique to symbolize content material construction is as a taxonomic tree, a hierarchical mannequin consisting of phases, content material themes, and ranges of element.
As soon as buyer journeys are mapped onto this tree, CDP metrics could be outlined as structural variations:
- Continuity: clean motion throughout branches
- Deepening: transferring into extra particular nodes
- Development: transferring ahead by means of buyer journey phases
The answer is to symbolize a journey as a path by means of a hierarchical taxonomy derived from the content material area. As soon as this illustration is established, CDP metrics could be computed deterministically from the trail. The diagram under summarizes the whole pipeline.
Establishing the Taxonomy Tree
To guage buyer journeys structurally, we first require a structured illustration of content material. We assemble this illustration as a multi-level taxonomy derived instantly from customer-journey textual content utilizing semantic embeddings.
The taxonomy is anchored by a small set of high-level phases (e.g., motivation, buy, supply, possession, and loyalty). Each anchors and journey messages are embedded into the identical semantic vector area, permitting content material to be organized by semantic proximity.
Inside every anchor, messages are grouped into progressively extra particular themes, forming deeper ranges of the taxonomy. Every degree refines the earlier one, capturing rising topical specificity with out counting on handbook labeling.
The result’s a hierarchical construction that teams semantically associated journey messages and gives a steady basis for evaluating how journeys stream, deepen, and progress over time.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is established, particular person buyer journeys are mapped onto it as ordered sequences of messages. Every step is embedded in the identical semantic area and matched to the closest taxonomy node utilizing cosine similarity.
This mapping converts a temporal sequence of messages right into a path by means of the taxonomy, enabling the structural evaluation of journey evolution quite than treating the journey as a flat listing of texts.
Defining the CDP Metrics
The CDP framework consists of three complementary metrics: Continuity, Deepening, and Development. Every captures a definite side of journey high quality. We describe these metrics conceptually earlier than defining them formally primarily based on the taxonomy-mapped journey.
Setup and Computation
Earlier than analyzing actual journeys, we make clear two facets of the setup.
(1) how journey content material is structurally represented, and
(2) how CDP metrics are derived from that construction.
Buyer-journey content material is organized right into a hierarchical taxonomy consisting of anchors (L1 journey phases), thematic heads (L2 subjects), and deeper nodes that symbolize rising specificity:
Anchor (L1)
└── Head (L2)
└── Youngster (L3)
└── Grandchild (L4+)As soon as a journey is mapped onto this hierarchy, Continuity, Deepening, and Development are computed deterministically from the journey’s path by means of the tree.
Let a buyer journey be an ordered sequence of steps:
J = (x₁, x₂, …, xₙ)
Every step xᵢ is assigned:
- aᵢ — anchor (L1 journey stage)
- tᵢ — thematic head (L2 matter), the place tᵢ = 0 means “unknown”
- ℓᵢ — taxonomy depth degree (L1 = 0, L2 = 1, L3 = 2, …)
Continuity (C)
Continuity evaluates whether or not consecutive messages stay contextually and thematically coherent.
For every transition (xᵢ →xᵢ₊₁), a step-level continuity rating cᵢ ∈ [0, 1] is assigned primarily based on taxonomy alignment, with larger weights given to transitions that keep throughout the identical matter or intently associated branches.
Transitions are ranked from strongest to weakest (e.g., identical matter, associated matter, ahead stage transfer, backward transfer), and
assigned lowering weights:
1 ≥ α₁ > α₂ > α₃ > α₄ > α₅ > α₆ ≥ 0
The general continuity rating is computed as:
C(J) = (1 / (n − 1)) · Σ cᵢ for i = 1 … n−1
Deepening (D)
Deepening measures whether or not a journey accumulates worth by transferring from normal content material towards extra particular or detailed
interactions. It’s computed utilizing two complementary parts.
Journey-based deepening captures how depth adjustments alongside the noticed path:
Δᵢᵈᵉᵖᵗʰ = ℓᵢ₊₁ − ℓᵢ, dᵢ = max(Δᵢᵈᵉᵖᵗʰ, 0)
D_journey(J) = (1 / (n − 1)) · Σ dᵢ
Taxonomy-aware deepening measures how deeply a journey explores the precise taxonomy tree, primarily based on the heads it visits.
It evaluates how most of the attainable deeper content material gadgets (kids, sub-children, and so forth.) underneath every visited head are later seen
in the course of the journey.
D_taxonomy(J) = |D_seen(J)| / |D_exist(J)|
The ultimate deepening rating is a weighted mixture:
D(J) = λ₁ · D_taxonomy(J) + λ₂ · D_journey(J), λ₁ + λ₂ = 1.
Deepening lies in [0, 1].
Development (P)
Development measures directional motion by means of journey phases. For every transition, we compute:
Δᵢ = aᵢ₊₁ − aᵢ.
Solely transferring steps (Δᵢ ≠ 0) are thought-about. Let wᵢ denote the relative significance of the present stage.
If Δᵢ > 0 (ahead motion):
cᵢ = wᵢ / Δᵢ
If Δᵢ < 0 (backward motion):
cᵢ = Δᵢ · wᵢ
The uncooked development rating is:
P_raw(J) = Σ cᵢ for all i the place Δᵢ ≠ 0
To sure the rating to[−1, +1], we apply a tanh normalization:
P(J) = (e^(P_raw) − e^(−P_raw)) / (e^(P_raw) + e^(−P_raw))
Making use of CDP Metrics to an Automotive Buyer Journey
To display how structured analysis works on life like journeys, we generated an artificial automotive customer-journey dataset overlaying the primary phases of the shopper lifecycle.
Enter Knowledge: Anchors and Journey Content material
The CDP framework makes use of two major inputs: anchors, which outline journey phases, and customer-journey content material, which gives the messages to judge.
Anchors symbolize significant phases within the lifecycle, comparable to motivation, buy, supply, possession, and loyalty. Every anchor is augmented with a small set of consultant key phrases to floor it semantically. Anchors serve each as reference factors for taxonomy development and because the anticipated directional stream used later within the Development metric.
anchor Phrases:
motivation exploration analysis discovery curiosity take a look at drive wants evaluation expertise
buy financing comparability quotes mortgage negotiation credit score pre-approval deposit
supply paperwork signing deposit logistics handover activation
possession upkeep guarantee restore supplier help service inspections
loyalty suggestions satisfaction survey referral improve retention advocacyBuyer-journey content material consists of quick, action-oriented CRM-style messages (emails, calls, chats, in-person interactions) with various ranges of specificity and spanning a number of phases. Though this dataset is synthetically generated, anchor info isn’t used throughout taxonomy development or CDP scoring.
CJ messages:
Discover fashions that match your way of life and private objectives.
Take a digital tour to find key options and trims.
Examine physique kinds to evaluate area, consolation, and utility.
Guide a take a look at drive to expertise dealing with and visibility.
Use the wants evaluation to rank must-have options.
Filter fashions by vary, mpg, or towing to slender selections.Taxonomy Development Outcomes
Right here, we utilized the taxonomy development course of to the automotive customer-engagement dataset. The determine under reveals the ensuing customer-journey taxonomy, constructed from message content material and anchor semantics.
Every top-level department corresponds to a journey anchor (L1), which represents main journey phases comparable to Motivation, Buy, Supply, Possession, and Loyalty.
Deeper ranges (L2, L3+) group messages by thematic similarity and rising specificity.
What the Taxonomy Reveals
Even on this compact dataset, the taxonomy highlights a number of useful patterns:
- Early-stage messages cluster round exploration and comparability, step by step narrowing towards concrete actions comparable to reserving a take a look at drive.
- Buy-related content material separates naturally into monetary planning, doc dealing with, and finalization.
- Possession content material reveals a transparent development from upkeep scheduling to diagnostics, value estimation, and guarantee analysis.
- Loyalty content material shifts from transactional actions towards suggestions, upgrades, and advocacy.
Whereas these patterns align with how practitioners usually purpose about journeys, they come up instantly from the information quite than from predefined guidelines.
Why This Issues for Analysis
This taxonomy now gives a shared structural reference:
- Any buyer journey could be mapped as a path by means of the tree.
- Motion throughout branches, depth ranges, and anchors turns into measurable.
- Continuity, Deepening, and Development are not summary ideas; they now correspond to concrete structural adjustments.
Within the subsequent part, we use this taxonomy to map actual journey examples and compute CDP scores in steps.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is constructed, evaluating a buyer journey turns into a structural downside.
Every journey is represented as an ordered sequence of customer-facing messages.
As an alternative of judging these messages in isolation, we challenge them onto the taxonomy and analyze the ensuing path.
Formally, a journey J = (x₁, x₂, …, xₙ) is mapped to a sequence of taxonomy nodes: (x₁→v₁),(x₂→v₂),…,(xₙ→vₙ) the place every vᵢ is the closest taxonomy node primarily based on embedding similarity.
A Step-by-Step Walkthrough: From Journey Textual content to CDP Scores
To make the CDP framework concrete, let’s stroll by means of a single buyer journey instance and present how it’s evaluated step-by-step.
Step 1 — The Buyer Journey Enter
We start with an ordered sequence of customer-facing messages generated by an LLM.
Every message represents a touchpoint in a practical automotive buyer journey:
journey = ['Take a virtual tour to discover key features and trims.';
'We found a time slot for a test drive that fits your schedule.';
'Upload your income verification and ID to finalize the pre-approval decision.';
'Estimate costs for upcoming maintenance items.';
'Track retention offers as your lease end nears.';
'Add plates and registration info before handover.']Step 2 — Mapping the Journey into the Taxonomy
For structural analysis, every journey step is mapped into the customer-journey taxonomy. Utilizing textual content embeddings, every message is matched to its closest taxonomy node. This produces a journey map (jmap), a structured illustration of how the journey traverses the taxonomy.
Desk 2: Every message is assigned to an anchor (stage), a thematic head, and a depth degree within the taxonomy primarily based on semantic similarity within the shared embedding area. This desk acts as the inspiration for all future evaluations.
Step 3 — Making use of CDP Metrics to This Journey
As soon as the journey is mapped, we compute Continuity, Deepening, and Development deterministically from step-to-step transitions.
Desk 3: Every row represents a transition between consecutive journey steps, annotated with indicators for continuity, deepening, and development.
Remaining CDP scores (this journey):
Taken collectively, the CDP indicators point out a journey that’s largely coherent and forward-moving, with one clear second of
deepening and one seen structural regression. Importantly, these insights are derived solely from construction, not from
stylistic judgments in regards to the textual content.
Conclusion: From Scores to Profitable Journeys
Continuity, Deepening, and Development are decided by construction and could be utilized wherever LLMs generate multi-step
content material:
- to check various journeys generated by completely different prompts or fashions,
- to supply automated suggestions for bettering journey technology over time.
On this method, CDP scores provide structural suggestions for LLMs. They complement, quite than substitute, stylistic or fluency-based analysis by offering indicators that mirror enterprise logic and buyer expertise.
Though this text focuses on automotive commerce, the idea is broadly relevant. Any system that generates ordered, goal-oriented content material requires robust structural foundations.
Giant language fashions are already able to producing fluent, persuasive textual content.
The higher problem is making certain that textual content sequences kind coherent narratives that align with enterprise logic and person expertise.
CDP gives a technique to make construction specific, measurable, and actionable.
Thanks for staying with me by means of this journey. Hopefully, this idea helps you assume in a different way about evaluating AI-generated sequences and conjures up you to deal with construction as a main sign in your personal programs. All logic offered on this article is applied within the accompanying Python code on GitHub. In case you have any questions or feedback, please go away them within the feedback part or attain out by way of LinkedIn.
References
Brodie, R. J., et al. (2011). Buyer engagement: Conceptual area, basic propositions, and implications for analysis.
