LLM Themes Are Not Observations

themes from a name corpus to the shopper desk. Prospects with out transcripts get NULL. NULL will get crammed with zero, or with “no concern talked about,” or quietly omitted as a reference class. In a single line of preprocessing, the pipeline converts didn’t name assist into didn’t expertise billing frustration.

The regression that follows appears clear. The coefficient on “billing frustration” is critical, signed the best way the product staff anticipated, massive sufficient to matter. It will get pasted right into a roadmap doc. No one asks the place the variable got here from.

This text is about what acquired smuggled in with that fill worth, and about three different strikes that look simply as innocuous in a pocket book however relaxation on assumptions the evaluation by no means names. The setup isn’t particular to assist calls. It applies to talk logs, ticket summaries, product critiques, gross sales transcripts, and survey free-response fields, anyplace a contemporary pipeline turns textual content right into a tidy column. The pipeline could possibly be a fine-tuned classifier, a zero-shot LLM, or an embedding-plus-cluster. The conceptual drawback is identical: the column isn’t an commentary of a buyer attribute. It’s the output of a generative course of utilized to a self-selected subset of buyer conduct.

Practitioners more and more deal with outputs like these as in the event that they had been direct readings of buyer state. They aren’t. They’re generated variables: measurements produced by a pipeline, conditional on a buyer doing one thing that left a textual hint, conditional on that hint surviving the extraction mannequin. Each step of that conditional has penalties for what the variable means in a downstream causal mannequin, and most of these penalties are invisible within the joined desk.

4 issues are likely to go mistaken, and the NULL transfer makes all 4 seen directly.

Choice. A theme exists for a buyer as a result of that buyer known as, complained, posted, or replied. No matter drove that motion can also be most likely correlated with the remedy, the result, or each. The NULL fill collapses “didn’t generate textual content” into the reference class, and the evaluation is not estimating an impact over the shopper base. It’s estimating an impact over a redefined inhabitants, and the redefinition occurred in preprocessing.

Timing. Was the decision earlier than the remedy, throughout it, or after? Pre-treatment textual content is a candidate confounder. Publish-treatment textual content is a candidate mediator or end result, and treating it as a pre-treatment management is a traditional supply of post-treatment bias. The joined desk hardly ever makes this seen.

Measurement. The label “billing frustration” isn’t billing frustration. It’s what the pipeline detected as billing-frustration-shaped language. Classifier accuracy is finite, and accuracy can differ throughout remedy arms, as a result of a remedy that adjustments how clients discuss additionally adjustments how the mannequin reads them. The label noise isn’t orthogonal to the factor being studied.

Function. Is the theme performing as a confounder, a mediator, a remedy, an end result, or a descriptive function? The DAG decides this, not the column title. A variable that’s methodologically legitimate in a single function turns into a bias supply in one other.

These 4 issues usually are not impartial. They work together. An LLM-detected theme inherits a variety footprint from the channel it got here by way of, a timing footprint from when the textual content was generated, and a measurement footprint from the pipeline that extracted it. The downstream regression sees a column of zeros and ones.

The issue isn’t that the pipeline produced a foul label. The issue is that the label inherited a data-generating course of the downstream evaluation by no means modeled.

The remainder of this text works by way of what meaning in apply, the place the usual workflow goes mistaken, and what the minimal diagnostic appears like. We begin with the role-and-timing query, as a result of it’s the one analysts get mistaken first.

Function and timing are the identical query

The primary transfer an analyst makes with a transcript-derived theme is implicit: they deal with it as a covariate. Themes go into the right-hand aspect of the regression. The remedy is the variable of curiosity. The end result is on the left. The theme is “managed for.”

That phrase, “managed for,” is doing work the analyst hasn’t checked. Controlling for a variable adjusts away the a part of the treatment-outcome relationship that flows by way of it. Whether or not that adjustment helps or hurts relies upon solely on the place the variable sits within the causal graph, and that place is decided by timing.

Pre-treatment textual content, generated earlier than the remedy was assigned, can play the function of a confounder. If a buyer known as about billing in January and the retention provide went out in March, the decision captures one thing about buyer state that will affect each who acquired the provide and who churned. Conditioning on the theme right here can cut back bias from omitted variables, offered the theme really proxies for the related assemble and the choice points within the subsequent part are dealt with.

Concurrent textual content, generated as a part of the remedy itself, isn’t a covariate in any respect. If the remedy is a name from a retention agent and the theme comes from that very same name, the theme is a part of the intervention. Conditioning on it doesn’t modify for confounding; it removes a part of the impact the analyst is making an attempt to measure.

Publish-treatment textual content, generated after the remedy, is probably the most harmful class, as a result of it’s the one probably to be misclassified as a confounder by an analyst working from a flat desk with no time index. A buyer who acquired a retention provide in March and known as complaining in April produced a transcript that displays, not less than partially, their response to the remedy. Conditioning on a theme extracted from that decision is conditioning on a post-treatment variable. That may block mediation paths, induce collider associations, or in any other case shift the estimand away from the remedy impact the analyst thinks they’re estimating.

A labored instance makes this concrete. Think about an artificial however business-realistic setup. Prospects are focused right into a retention provide based mostly on a mannequin that picks up value sensitivity. Each the provide project and buyer churn depend upon this underlying value sensitivity, which the analyst doesn’t observe. Prospects who’re extra price-sensitive usually tend to obtain the provide (as a result of the focusing on mannequin chosen them) and extra more likely to churn regardless. They’re additionally extra more likely to name assist and specific invoice shock. The theme “invoice shock” is generated from these post-treatment calls.

The naive analyst joins the theme onto the shopper desk, fills NULL as zero, and runs a logistic regression of churn on provide plus bill-shock:

import numpy as np
import pandas as pd
import statsmodels.api as sm
 
rng = np.random.default_rng(7)
n = 20000
 
price_sens = rng.regular(0, 1, n)
provide = rng.binomial(1, 1 / (1 + np.exp(-(0.8 * price_sens))))
churn = rng.binomial(1, 1 / (1 + np.exp(-(-1.0 + 1.2 * price_sens - 0.5 * provide))))
known as = rng.binomial(1, 1 / (1 + np.exp(-(-1.5 + 0.7 * price_sens + 0.9 * churn))))
 
theme_prob = 1 / (1 + np.exp(-(-0.5 + 0.8 * price_sens)))
bill_shock = np.the place(known as == 1, rng.binomial(1, theme_prob), 0)
 
df = pd.DataFrame({"churn": churn, "provide": provide, "bill_shock": bill_shock})
 
X = sm.add_constant(df[["offer", "bill_shock"]])
naive = sm.Logit(df["churn"], X).match(disp=0)
print(naive.params)

The true impact of the provide on churn is −0.50 in log-odds. The provide is meant to cut back churn, and within the data-generating course of it does. Here’s what 4 specs return:

As a result of provide project is already confounded by value sensitivity, eradicating the dangerous management doesn’t make the design legitimate. It solely removes one further supply of distortion. Two observations from this desk.

First, the naive specification is mistaken in course. Including the bill-shock management to a mannequin that was already biased flips the signal on the provide coefficient. The product staff studying this output would conclude that retention gives trigger churn. They’d be mistaken.

Second, dropping the bill-shock variable doesn’t repair the evaluation. The dropped specification can also be constructive, and solely the oracle specification, which situations on the unobserved confounder instantly, recovers the true impact. In an actual evaluation the analyst doesn’t have that column. That’s the level. Eradicating a foul management is important however not enough, and a post-treatment theme extracted from a self-selected calling subpopulation isn’t an alternative to identification.

The mechanism behind the signal flip within the naive specification is price strolling by way of. Churn impacts the probability of calling, as a result of clients who’re leaving usually tend to name. Invoice-shock is just noticed for purchasers who known as, because the theme requires a transcript to exist. Conditioning on bill-shock due to this fact situations on a downstream consequence of churn. Amongst clients with bill-shock equal to 1, the connection between provide and value sensitivity has been distorted, as a result of each variables now assist clarify why the shopper ended up flagged. The coefficient on provide absorbs that induced affiliation.

The methodological level generalizes. A transcript-derived variable has a place within the causal graph decided by when the textual content was generated relative to the remedy, who generated it, and what course of produced the label. Function and timing are the identical query considered by way of completely different lenses. These variables include a structural footprint the analyst is answerable for tracing, and the joined desk isn’t the place the tracing occurs.

The choice query

Most trade analyses utilizing assist transcripts implicitly redefine the inhabitants from “clients” to “clients who generated assist language.” The estimand adjustments earlier than the regression even begins.

That is the half that tends to matter most in practitioner workflows, and it’s the place the usual workflow is most fragile.

The textual content exists as a result of the shopper did one thing: known as, posted, complained, replied. That one thing is a conduct, not a measurement. It’s influenced by buyer traits, by the channel that was out there, by the urgency of the underlying concern, and sometimes by the remedy itself. None of those are random. None are usually orthogonal to the result.

The NULL dealing with determination is the place this turns into operational. There are three widespread strikes, and every carries an assumption.

Filling NULL as zero or “no concern talked about” assumes that not producing textual content is informative concerning the absence of the underlying assemble. The analyst is claiming that clients who didn’t name didn’t expertise the factor the theme is detecting. For many themes price detecting, that is implausible on its face. Prospects who didn’t name might have skilled billing frustration and resolved it by canceling, by switching to a competitor, by complaining on social media, or by giving up. The zero-fill turns all of those into “no frustration.”

Dropping rows with NULL themes, limiting the evaluation to the calling subpopulation, is not less than sincere concerning the inhabitants, nevertheless it adjustments the estimand. The remedy impact amongst clients who known as isn’t the remedy impact amongst clients, and the distinction between the 2 is usually your complete level of the enterprise query. A retention provide’s impact on churn-prone callers is a helpful amount. It isn’t the amount most analyses declare to estimate.

Treating text-presence as a missingness mechanism and making use of inverse likelihood weighting based mostly on a mannequin of who calls is, methodologically, the fitting form of transfer. The catch is the propensity mannequin itself. Modeling who generates textual content requires writing down what drives calling, and that mannequin will depend on demographics, tenure, prior points, remedy publicity, and unmeasured frustration, which is the assemble the theme was supposed to assist measure within the first place. The IPW transfer is principled, and it’s also hardly ever as principled because it appears.

The deeper level is that choice into textual content is a conduct that interacts with the remedy. A retention provide might change calling charges. A pricing change might change criticism charges. A function launch might change the sorts of points clients articulate. Any of those makes the choice mechanism itself treatment-dependent, which implies even a wonderfully extracted, completely timed theme is being measured on a inhabitants whose composition shifts with the remedy. Commonplace observational corrections assume the choice mechanism is steady. When the remedy strikes the choice, the corrections don’t.

None of this implies transcript-derived variables are ineffective. It means the analyst owes the reader an specific assertion of which inhabitants the evaluation is estimating an impact over, what mechanism produced the textual content, and what assumption was made about everybody whose textual content doesn’t exist.

The measurement query

Outdated NLP outputs seemed noisy. TF-IDF weights, sparse key phrase counts, LDA subject vectors: none of them seemed like issues a buyer felt. Practitioners distrusted them by reflex, and that reflex saved numerous dangerous analyses.

LLM outputs don’t look noisy. They appear like latent constructs. A label like “billing frustration” or “belief erosion” or “renewal nervousness” reads like an outline of a buyer’s psychological state. The label is articulate, the classes are semantically coherent, and the failure modes don’t announce themselves within the column. The persuasion drawback is actual earlier than the statistical drawback begins.

The statistical drawback is extra acquainted. An LLM theme is a loud proxy for the underlying assemble. The label “invoice shock” isn’t invoice shock. It’s what the mannequin determined was bill-shock-shaped language within the transcripts it processed. Classifier accuracy is finite even for clear duties, and the accuracy on the precise inhabitants, not the held-out analysis set, is usually unknown. Plugging a loud proxy right into a regression instead of the true variable attenuates coefficients towards zero in some setups and distorts them in others, relying on whether or not the noise is differential.

Differential measurement error is the place the true injury lives. If a remedy adjustments how clients discuss, and most therapies price working do, then the classifier’s accuracy on theme detection can differ between remedy and management. A retention provide that softens buyer sentiment might cut back the speed at which the mannequin flags “invoice shock” language with out lowering the underlying frustration. A pricing change that shifts how clients articulate complaints might transfer classifier accuracy extra in a single arm than the opposite. The label noise is not mean-zero. It’s correlated with the remedy, and conditioning on the noisy label biases the estimated remedy impact in a course the analyst can not simply signal.

There’s a literature on correcting for classifier-induced measurement error. Egami and colleagues develop a split-sample workflow for causal inference with text-discovered measures as therapies or outcomes in “How one can Make Causal Inferences Utilizing Texts”. Mozer and colleagues apply text-augmented matching to digital well being data and present how text-based covariates change estimated results in an actual medical examine in “Leveraging textual content information for causal inference utilizing digital well being data”. For the broader panorama, Keith, Jensen, and O’Connor assessment how textual content has been used to take away confounding throughout functions in “Textual content and Causal Inference: A Evaluate of Utilizing Textual content to Take away Confounding from Causal Estimates”. These strategies exist, and they’re price utilizing when the evaluation issues. In addition they require the analyst to acknowledge {that a} label is a measurement with error within the first place, which is the transfer most workflows skip.

The practitioner mistake isn’t utilizing the label. The practitioner mistake is treating a label that got here out of a generative mannequin as if it had been a column learn off a sensor.

A practitioner guidelines

A causal evaluation that makes use of a generated variable derived from transcripts can nonetheless be defensible. It simply has to reply 5 questions earlier than the regression runs.

1. What function am I assuming this variable performs?

Confounder, mediator, remedy, end result, or descriptive function. The DAG decides. The column title doesn’t.

2. When was the textual content generated relative to the remedy?

Pre-treatment, concurrent, or post-treatment. If the analyst can not reply this from the information, the variable doesn’t enter the mannequin as a confounder.

3. What choice mechanism produced the textual content, and what am I assuming about everybody whose textual content doesn’t exist?

Zero-fill, drop, IPW: every is an assumption. Decide one and state it.

4. How was the label produced, and will its reliability differ throughout remedy arms?

If the remedy plausibly adjustments how clients specific the underlying assemble, classifier accuracy isn’t fixed throughout the comparability the evaluation is making.

5. What does the end result appear like below a stress take a look at?

Refit the mannequin with out the transcript-derived variable. If the headline coefficient is fragile, the end result isn’t steady sufficient to hold a causal declare by itself.

These 5 questions usually are not an answer. They’re a diagnostic. An analyst who can reply them isn’t assured an recognized impact. An analyst who can not reply them is doing descriptive work with causal language connected.

The broader sample is older than LLMs. Generated variables are pipeline outputs that appear like observations however are literally mannequin outputs conditioned on choice. They present up in fraud scores, recommender relevance metrics, sentiment indices, propensity scores reused as covariates, and any latent-trait estimate produced by an upstream mannequin and consumed by a downstream evaluation. LLMs didn’t invent this error. They made it accessible at a scale and a fluency that older NLP outputs by no means reached. The labels appear like latent constructs, the columns appear like measurements, and the workflow appears like causal inference.

The assumptions didn’t disappear. They simply moved upstream.

Workers Information Scientist targeted on causal inference, experimentation, and determination science. I write about turning ambiguous enterprise questions into decision-ready evaluation.

Extra like this on LinkedIn 👇

🔗 LinkedIn