Ask a coding agent to vary a part and it’ll do it. Ask it what that change broke and it’ll guess.
Up entrance: I co-founded Bit Cloud, which works on this. It’s the place the numbers come from, and I title it the place it’s the related reply. Weigh it accordingly.
Here’s what that appears like in a system I keep. One of many merchandise is a studying platform, and the mentor you speak to inside it was by no means constructed for that platform in any respect: it’s a separate product of mine, put in as a broadcast part, a chat widget speaking to its personal retrieval service.
Now ask what breaks if I modify that widget. The web page that renders it does import it, so the sting is written down someplace: within the different product’s repository, in a press release that resolves into node_modules. Working from the widget, nothing in attain says that web page exists. Cursor, Claude Code and Copilot all sit in entrance of the identical hole, no matter every of them indexes, as a result of the knowledge will not be in any repository they will see.
This is not only a story about my very own setup. DORA’s 2025 State of AI-assisted Software program Growth report describes time saved throughout technology being re-spent on auditing and verification, and finds round 30% of builders reporting little or no belief within the code AI writes for them. That tax has a number of causes; I’m after one in all them.
Two units, and they aren’t the identical set
Each code-retrieval system in manufacturing solutions one query: given this question, which components of the codebase most bear a resemblance to it? Embed the chunks, embed the question, return the closest neighbours, rerank, widen the window. Each a type of enhancements makes the identical output higher: the set of code that resembles what you requested about.
An agent making an attempt to vary one thing safely wants a unique set, which is the set of code that breaks when the factor it’s altering modifications.
These two units overlap, however far lower than the tooling assumes. A client that imports a perform shares its names, so retrieval often finds it. However a theme file and a button share nothing in any respect, and altering the theme’s contract breaks the button anyway, so the one relationship that issues is the one an index scores lowest. It fails the opposite manner too, handing again two type validators written months aside that look alike and are related to nothing, which invitations the agent to reconcile them.
And none of that is an accident of how some codebase grew. It’s what you get when the engineering is finished correctly. The identical system has an authentication equipment in it: a impartial contract written as plain sorts, with a Descope implementation and a Clerk one behind it. Parts rely upon the contract and by no means on the supplier, which is why swapping one for the opposite value me a day, and why the supplier’s title seems nowhere in client code. Ask a similarity index what makes use of Descope and also you get precisely one file: the supplier wrapping the SDK. Each contract you place between two issues intentionally removes a vocabulary overlap the index was counting on.
Resemblance is a property of textual content, and dependency is a property of a system. The artifact in entrance of you carries the primary one, and it can not carry the second, as a result of the system it belongs to is some other place.
The part on the left is a construct atmosphere, which determines compilation, dependency decision and check setup for all the pieces utilizing it, so a change genuinely reaches all twenty-eight. Not a type of twenty-eight incorporates textual content resembling an atmosphere configuration, so the similarity rating is accurately low and utterly ineffective.
That’s one part. The sample holds throughout sorts. Right here is one occasion of every factor I share throughout these merchandise. The design system, the product that consumes most of it and the mentor’s personal scope are public, so each row however the final could be checked; that one sits in two scopes I’ve not opened.
|
What’s shared |
An actual occasion |
Consumed by |
Does it resemble its shoppers? |
|---|---|---|---|
|
A visible primitive |
|
8 parts |
Considerably |
|
A theme |
|
14 parts |
Barely |
|
A construct atmosphere |
|
28 parts |
No |
|
A operating product |
|
2 pages in a unique product |
No |
|
A backend service |
|
one other product’s service |
No |
Learn the final column downward.
The highest row is what folks image once they hear “part reuse”, and retrieval handles it least badly, since a button and a web page that renders buttons share vocabulary. The center rows already break inside one product, as a result of a hook fetching person progress shares no phrases with a theme or a construct atmosphere. The underside two are those that matter: a operating service and a stay backend, not primitives. Essentially the most beneficial issues being reused resemble their shoppers least, which is backwards from what a similarity index is sweet at, and they’re those nothing else can see. No person reuses a service they can’t uncover. They rebuild it.
Modularity breaks the import graph in two alternative ways
So the vocabulary is gone. Learn the imports as a substitute, which is the affordable subsequent transfer and helps lower than you’ll assume: it breaks in two methods, and each worsen the higher the system is assembled.
First, the import survives however stops being followable. In a component-based codebase the import is true there within the file:
Declared, greppable, unambiguous, and it resolves into node_modules, the place each native analyser offers up. That printed part has its personal dependencies, its personal dependents and a model historical past, none of it in your machine. You’ll be able to see the sting exists; what sits on the far finish, and who else holds that finish proper now, you can not.
Second, greater up, the import disappears completely. The place complete functions and companies get assembled as a substitute of imported, there’s typically no assertion to learn: a characteristic registering itself right into a platform slot at startup, an edge dwelling in a config array, a deploy-time selection between two auth suppliers, or two parts agreeing on a form no one declared.
One platform part in that system assembles a React software, two Node companies and a gateway. Right here is the way it names them:
import.meta.resolve palms again a path. Nothing is imported, no sort flows throughout, and “discover all references” on the app returns nothing, as a result of the argument is a string. Nx builds its graph from TypeScript imports, so Nx doesn’t see this both until somebody declares the sting by hand.
Earlier than calling that unique, discover what it’s. An actual import of the frontend would pull browser code right into a Node course of, and importing any of them would run module unwanted side effects at load time. The platform doesn’t devour these models, it arranges them, so it names them by identification as a substitute of by worth. Each composition root does this: webpack entry factors, pictures: in a compose file, service references in Kubernetes, module sources in Terraform. Anybody assembling individually deployed issues already has this edge, and in most stacks that string is recorded nowhere besides the config file it sits in.
Right here it will get resolved towards the dependency graph, pinned, and written into the model report when that model is created:
That third entry, agents-4-all/agent-service, is the mentor’s backend from the opening, consumed as a bundle and pinned, and each ends of that edge are public if you wish to verify it. So the sting will not be sitting in a manifest ready to be parsed. It was computed as soon as, when the factor was constructed, and saved as a truth, which is why studying it again is a lookup.
Be equally actual about what it doesn’t carry. It declares that these models ship collectively, an actual blast-radius sign: change one and the others are implicated. It says nothing concerning the frontend’s assumption concerning the service’s response form, which is agreed over HTTP at a deploy-time deal with and written down nowhere.
The co-assembly edge is asserted and written down. It’s simply not written the place an indexer seems. Registration and composition each produce edges sitting in an artifact no one indexes: unread, not unknowable. A deploy-time selection and an undeclared structural contract are in no artifact in any respect, and people keep correctly unsolved.
What with the ability to ask truly purchased
There was a interval earlier than this one the place the graph existed and nothing may attain it, and each session I re-stated the identical boundaries by hand, pulling the agent again inside an structure that was, from its perspective, not there. It labored, it didn’t scale, and it was a silly use of an individual.
I’ve spent the final a number of months sustaining the present system by myself: a dozen or so functions and shared libraries, the most important round forty parts, sharing the design system above, a typed API consumer, one product’s chat service, and a seize library put in from its bundle.
The one motive any of it was workable is that the agent may ask. Each part data what it is dependent upon when its model is created, and people data arrive as a instrument name, not as a paragraph pasted right into a immediate 9 turns earlier and hoped to nonetheless be related. Earlier than touching something shared it may ask what else held the opposite finish; earlier than writing one thing new, whether or not that factor already existed.
Here is the form of it over MCP. The form of the decision is generic, a instrument returning recorded edges, and inside one repository a mission graph solutions the intra-repo half of it. The pinned variations and the scope crossings are the half it can not:
Three of these cross a scope boundary, and as earlier than this can be a lookup, nothing embedded and nothing parsed. Run it the opposite manner, what is dependent upon this, and it’s the similar information backwards. That’s the place the counts right here got here from: I requested an agent to stroll the graph.
None of this made the mannequin smarter, but it surely made one class of mistake unavailable: the agent couldn’t quietly duplicate one thing that already existed, and it couldn’t change a shared contract whereas leaving the dependents implicit, as a result of itemizing them is a lookup. Throughout these merchandise that’s the distinction between 19 shared parts carrying 97 usages and 97 separate implementations quietly drifting aside. A utilization there’s one part naming a part from one other scope in its printed report, so it counts reuse and nothing else. One particular person can maintain the primary of these; no one holds the second at any measurement.
These two asks are completely different queries, and conflating them is why this layer often will get constructed badly. What is dependent upon this is a traversal over recorded edges. Does one thing equal exist already is a search over declared APIs, a unique index and the extra beneficial of the 2: an agent requested so as to add a chat interface will construct one, with no method to know a completed, deployed one exists two merchandise over. That duplicate will not be a failure of reasoning; it’s the right output given what the agent may see. Similarity search continues to be proper for the place do I begin, and the error is asking all of it three.
What modifications with multiple staff
The arithmetic inverts the second multiple particular person is concerned. In a one-person system a publication boundary is one thing you select to create, and I didn’t need many. In a multi-team organisation each staff boundary is a publication boundary by development, so the rely stops monitoring anybody’s self-discipline and begins monitoring the org chart. What fails there’s visibility and never care: whoever modifications a shared contract has by no means seen most of what holds the opposite finish. At a dozen merchandise and one particular person a queryable graph is a comfort; at a number of hundred parts and multiple staff, the place no one has seen many of the system, it stops being one.
“We’ve got had import graphs for a decade”
A good objection: if the perimeters are recorded someplace, as I maintain insisting they’re, why has this not already been solved by instruments which have existed for years?
Partly it has, and people instruments do actual work: language servers resolve references, Nx and Turborepo keep mission graphs, and for any wiring you expressed as an import inside one repository the traversal already exists. Cross-repository instruments go additional, with Sourcegraph resolving references throughout repos, GitHub displaying “Utilized by” and npm itemizing dependents. Most of them ship MCP servers now, so an agent can attain them whereas it really works, and reachability has stopped being the differentiator. What’s reachable nonetheless is.
Two issues differ. They index at repository or bundle granularity, so that you get “this bundle is utilized by that product”, not “this part’s contract change breaks these two pages at this model”. And so they maintain the sting solely when each side had been listed, which fails for personal and cross-organisation code.
Beneath each sits a 3rd distinction: these instruments derive the graph by evaluation at learn time, from no matter is in entrance of them, whereas an edge recorded when the model was created wants no evaluation and doesn’t rely upon having each side in view.
Which brings again the boundary from earlier, from the opposite facet. Going out, native tooling stops at node_modules. Going again, run those self same instruments contained in the printed library’s personal repository and there’s no report the patron exists. Neither path crosses, and reverse edges solely exist if one thing data them the place the publishing occurs.
That’s the class, and it’s the one I work in. Bit Cloud data the sting at part granularity when the model is created and holds it throughout repositories and scopes, which is why each question here’s a lookup and never an evaluation. It isn’t the one form that might work: something that data at that granularity, throughout these boundaries, solutions the identical questions.
What this doesn’t repair, and what survives it
A dependency graph doesn’t let you know whether or not a change is right, solely what’s affected. Figuring out the blast radius will not be figuring out the result.
It doesn’t assist the place the construction genuinely will not be there: a codebase with no part boundaries has no graph to show, and no recorded edge will floor a contract no one declared.
It requires the unit of labor to be one thing the system can report, which is an actual design resolution even when it isn’t a lot of a tax.
And that is one system, self-reported, with no management situation. Deal with it as a discipline report, not a benchmark.
What survives all of that’s the form of the issue. Technology is in good condition, and fashions write right code from an outline extra reliably than most individuals anticipated this quick. Modification will not be, and the reason being not mannequin functionality. We ask a text-similarity system a query about system construction, get a poor reply, and deal with it as a context-length downside. Then we draw a clear boundary, compose as a substitute of import, and make the reply worse.
The numbers above come from techniques operating on what we construct at Bit Cloud, and none of it requires our product: solely that edges get recorded wherever the work occurs, and that an agent can ask about them whereas it really works.
If you’re in a single repository with odd imports, most of what you want is already computable and your agent is solely not asking for it. Wire these queries to no matter your language server or mission graph already is aware of, and deal with that import graph as a flooring: it covers the wiring you expressed as imports and leaves you registration, deploy-time choice, and all the pieces previous your personal repository.
The studying platform I opened with continues to be operating, and its mentor continues to be one other product. It stayed that manner for one motive: one thing recorded the sting, so no one needed to rebuild it. Construction your agent can not question doesn’t constrain your agent. It solely constrains you.
