TL;DR
variable rename passes Git, greenlights your mocked take a look at suite, and will get accredited in code overview—proper up till it crashes each actual manufacturing name the second it executes.
This failure mode isn’t an anomaly. It’s what inevitably occurs when a immediate’s enter variables change and 0 tooling checks the precise name websites counting on them.
To repair this, I constructed promptctl: a zero-dependency, three-pass Python static analyzer that catches damaged immediate signatures earlier than you deploy:
- PromptDiff: Detects variable adjustments in your immediate information.
- Contract Validation: Verifies schema boundaries.
- Affect Evaluation: Traces the place these variables are referenced throughout your codebase.
It requires no LLM calls, no API keys, no mannequin evaluations, and 0 coaching. It runs strictly utilizing Python’s built-in ast, string.Formatter, and set arithmetic straight in opposition to your supply code.
Each terminal output and timing benchmark proven on this article comes from dwell runs of the instrument—no pretend mockups. It’s an deliberately slim instrument, and the article explicitly covers its limitations upfront moderately than hiding them within the high-quality print.
A Frequent Failure Sample, Not a Hypothetical
This failure mode doesn’t want a dramatic story. The mechanics alone clarify it.
You rename one variable in a immediate template to match a schema replace elsewhere in your codebase. {ticket} turns into {ticket_id}.
Git flags it as a clear one-line diff. Your unit checks move as a result of they mock the LLM consumer and by no means really invoke .format() with actual inputs. Code overview glides by means of as a result of the change appears to be like trivial. The deployment finishes and not using a hitch.
Then, each single name website nonetheless passing ticket= as a substitute of ticket_id= fails the second it runs in manufacturing.
Nothing in your normal setup catches this beforehand. Your linter, kind checker, take a look at runner, and overview course of all did their jobs, however none of them deal with a immediate template like an interface with a inflexible contract. To Python, your immediate is only a string. Strings don’t have contracts. Nothing verifies that the caller formatting the string really matches what that string expects.
As an alternative of simply speaking about the issue, I constructed a small take a look at repo with this actual bug: one immediate, three caller features. Then I ran a static analyzer in opposition to it.
Each terminal block under comes straight from executing that instrument—no hardcoded examples or pretend mockups. I wrote promptctl, a small, zero-dependency Python instrument, to shut this particular hole earlier than code hits manufacturing.
Right here is the way it works, step-by-step, with actual output from the terminal. All of the supply code, mock setup, and baseline snapshots are up on GitHub. https://github.com/Emmimal/promptctl
Who This Is For
Construct or undertake one thing like this if you happen to ship immediate templates as code (a .py, .yaml, or template file in Git) and a number of locations in your codebase format or render that very same template. The second a immediate has multiple caller, which is typical for manufacturing agent programs previous the prototype part, contract enforcement turns into price it.
Skip this if each immediate in your system has precisely one caller. A contract break there’s simply a normal bug, not a cross-file coordination failure. Skip it in case your prompts dwell in a database or exterior CMS. Static evaluation over a file tree can not see them, so this method is not going to work. And positively skip it in order for you one thing to guage immediate high quality. This instrument doesn’t care in case your immediate is well-written; it solely checks whether or not its interface contract is revered.
In case you are uncertain whether or not you want this, run a fast take a look at: grep your codebase for what number of distinct information name .format(), .render(), or an equal on the identical immediate template. Two or extra callers means a variable rename is a hidden coordination downside your present toolchain in all probability misses. One caller means you should not have this downside but.
Immediate Engineering Solved Writing Prompts. It By no means Solved Altering Them Safely.
Nearly all the pieces written about immediate engineering is about v1: immediate construction, few-shot examples, chain-of-thought, and output formatting. That stuff issues, nevertheless it treats prompts like static artifacts.
In manufacturing, prompts are by no means static. Schemas evolve. Fields get renamed. Somebody splits a single immediate into two, merges two into one, or tweaks wording and by accident drops an enter variable alongside the best way.
Each a type of edits alters a contract between two decoupled items of code: the file defining the immediate and each caller invoking .format() or .render() on it. Python enforces nothing about that relationship. It’s the actual kind of implicit coupling that trendy infrastructure stopped accepting years in the past.
| Class | Examples | The way it’s checked earlier than it runs |
|---|---|---|
| Executable code | Python, Go, Rust | Compiled or type-checked; CI fails loudly |
| Infrastructure config | Terraform, Kubernetes YAML | terraform validate [1], schema validators like kubeconform [2] |
| Supply type and kinds | Python supply itself | Ruff, mypy [3][4] |
| Immediate templates | System prompts, agent directions | Nothing, by default |
You don’t run terraform apply and hope your syntax holds up. You don’t push a Kubernetes manifest with out validating it in opposition to a schema first. You don’t ship Python code with out working ruff or mypy. We constructed all of this tooling for one plain purpose: breaking prod at runtime is painful and costly, however catching a mismatch in CI is affordable.
Immediate templates are not any totally different. They’re structured configs driving an exterior system. They is perhaps plain textual content, however they act like API contracts, and people contracts break the second surrounding code strikes on.
Positive, the LLM ecosystem is flooded with instruments proper now. We’ve immediate registries, analysis frameworks, and tracing platforms out of the field. However virtually none of them reply the obvious query on a pull request: does altering this string break any of the code presently calling it?
That’s the lacking piece promptctl tackles. The core concept is straightforward: if you happen to edit a immediate template, you run a verify in opposition to each caller in your codebase earlier than you merge, similar to you’d for a database schema migration.
This Is a Change Administration Drawback, Not a Repository Drawback
It’s straightforward to misdiagnose this concern as one thing that solely hits massive engineering groups. Organizing prompts, writing them, tweaking the wording, and protecting them in folders is simply CRUD work. Most groups handle that high-quality with clear file constructions and some naming conventions. That isn’t what breaks in manufacturing.
What breaks is managing adjustments to these prompts: realizing what modified, checking whether or not downstream callers fulfill the brand new contract, and verifying if a pull request is definitely secure to merge.
You don’t want 300 prompts throughout a posh agent community to set off this. You want precisely one immediate, one caller operate, and one unverified edit. That applies to a solo developer constructing a easy assist bot simply as a lot as a staff of fifty engineers. Scale solely will increase the variety of locations a bug can disguise. It doesn’t create the hole.
For this reason promptctl stays locked to a few passes as a substitute of making an attempt to change into an all-in-one platform. Instruments that attempt to clear up immediate versioning, storage, high quality scoring, and contract security normally find yourself doing all 4 poorly. The aim right here stays slim: validate immediate adjustments statically in CI earlier than deployment, precisely such as you verify a database migration earlier than working it in opposition to manufacturing.
Structure: Three Passes, One Exit Code

promptctl diff, promptctl validate, and promptctl influence every execute a single move on their very own. promptctl verify runs all three collectively and exits with standing 1 if something is damaged, or standing 0 if the construct passes.
That exit code is intentionally formed on your CI pipeline.
Nothing on this instrument touches an exterior API, runs mannequin evaluations, or requires fine-tuning. The execution path depends fully on ast.parse for Python supply code, string.Formatter to tug variable names out of immediate templates, and set arithmetic to check declared parameters in opposition to name websites. There are not any pip dependencies, no community calls, and no LLMs wherever within the course of.
The take a look at repository used all through this walk-through retains issues easy: one immediate file (prompts.py) and three caller information (router.py, evaluator.py, and checks/test_router.py). Any string fixed ending in _PROMPT is robotically picked up as a registered immediate. You do not want decorators or separate registry configs.
The CLI makes use of normal argparse. Each command takes --repo and --baseline flags so you’ll be able to level it at any listing or snapshot. That’s how the before-and-after checks later on this article run without having guide file edits between passes.
Right here is the baseline snapshot, which is only a compact JSON file storing the final accepted state of the immediate:
{
"CUSTOMER_ROUTER_PROMPT": {
"symbol_id": "customer_router",
"variables": ["ticket", "domain"]
}
}
No database, no exterior historical past service. Only a checked-in file that claims “that is what the immediate regarded just like the final time we reviewed it.” Updating it requires a deliberate commit, which supplies you a transparent, trackable log of each schema change.
The three passes are impartial features tied collectively by one grasp command that executes them so as and aggregates the findings. verify is only a comfort wrapper, not a separate function.
You’ll be able to run promptctl diff, promptctl validate, or promptctl influence on their very own. That distinction issues if you solely care about one particular verify as a substitute of working the entire suite, just like how terraform plan and terraform validate keep break up as a substitute of forcing a full run each time.
The worth isn’t in how these instructions chain collectively. It’s in what every move inspects underneath the hood. The AST snippets under break down how that parsing really works.
Element 1: PromptDiff
The primary move solutions a fundamental query: did the variables required by this immediate change for the reason that final accepted state?
It parses prompts.py utilizing Python’s native ast module, extracts each variable identify from _PROMPT string constants with string.Formatter().parse(), and runs set arithmetic in opposition to the baseline file.
def _extract_prompt_vars(supply: str) -> Dict[str, Set[str]]:
tree = ast.parse(supply)
discovered: Dict[str, Set[str]] = {}
for node in tree.physique:
if not isinstance(node, ast.Assign):
proceed
if not (isinstance(node.worth, ast.Fixed)
and isinstance(node.worth.worth, str)):
proceed
for goal in node.targets:
if isinstance(goal, ast.Title) and goal.id.endswith("_PROMPT"):
template = node.worth.worth
variables = {
identify for _, identify, _, _ in
string.Formatter().parse(template) if identify
}
discovered[target.id] = variables
return discovered
Nothing right here executes the immediate or calls .format(). It merely walks the syntax tree and inspects the nodes straight. That pure static method is why execution completes in single-digit milliseconds, no matter template measurement or complexity.
What Modified, Earlier than vs. After
| Earlier than (baseline.json) | After (present prompts.py) | |
|---|---|---|
| Required variables | {ticket}, {area} |
{ticket_id}, {area} |
| Eliminated | – | {ticket} |
| Added | – | {ticket_id} |
| Breaking? | – | YES |
Actual output from working promptctl diff after making that actual change:
$ promptctl diff
Immediate: customer_router
Contract Diff
Eliminated: {ticket}
Added: {ticket_id}
Breaking Change: YES
A change will get flagged as breaking particularly when a variable will get eliminated. Any current caller that continues passing that outdated parameter will trigger str.format() to throw a KeyError as a result of the placeholder it expects is gone.
That removing is the place the runtime danger lies. Including a brand new variable by itself received’t break downstream code. The true downside is renaming.
Including a variable received’t break something. Eradicating one will. Whenever you rename a placeholder, you’re deleting the outdated key. If a caller nonetheless sends that deleted key phrase argument, str.format() throws a KeyError immediately:
@property
def is_breaking(self) -> bool:
return len(self.removed_vars) > 0
Element 2: Contract Validation
Recognizing a modified immediate is just half the job. What really issues is realizing whether or not that change broke downstream code.
This move makes use of ast.stroll to examine each .py file in your repository. It locates calls formatted like SOME_PROMPT.format(...) and compares the key phrase arguments being handed in opposition to the precise placeholders the up to date immediate calls for.
for node in ast.stroll(tree):
if not (isinstance(node, ast.Name)
and isinstance(node.func, ast.Attribute)):
proceed
if node.func.attr != "format":
proceed
if not isinstance(node.func.worth, ast.Title):
proceed
constant_name = node.func.worth.id
if constant_name not in contracts:
proceed
required_vars = contracts[constant_name]
provided_vars = {kw.arg for kw in node.key phrases if kw.arg shouldn't be None}
lacking = required_vars - provided_vars
This maps straight onto the sample we walked by means of earlier. Right here is the precise per-call-site output:
| Caller | Offers | Immediate Requires | Lacking | Standing |
|---|---|---|---|---|
router.py:8 |
ticket, area |
ticket_id, area |
ticket_id |
✗ FAIL |
evaluator.py:8 |
ticket, area |
ticket_id, area |
ticket_id |
✗ FAIL |
checks/test_router.py |
(by no means calls .format()) |
ticket_id, area |
– | not checked |
$ promptctl validate
Immediate: customer_router
Required Variables
{area}
{ticket_id}
Name Website Variables
{area}
{ticket}
Lacking
{ticket_id}
Affected Name Websites
✗ evaluator.py:8
✗ router.py:8
2 contract violations
Discover how the take a look at file isn’t on that record? That’s not a bug.
The take a look at simply checks that the immediate fixed exists. It by no means calls .format(), which is why a mocked take a look at suite lets this actual bug slip proper by means of. In case your unit checks mock the mannequin consumer as a substitute of really formatting the immediate string, you’re skipping the precise name that might have caught the crash.
To ensure this wasn’t only a staged demo designed to fail, I mounted each name websites to move ticket_id= as a substitute of ticket= and ran it once more.
End result? Zero contract violations, exit code 0. A instrument that solely is aware of how one can fail isn’t a validator—it’s simply an annoying script caught on crimson.
Element 3: Affect Evaluation
Contract validation exhibits you what’s damaged proper now. Affect evaluation exhibits you all the pieces related to a immediate, damaged or not. That manner, you see all the blast radius earlier than merging a Pull Request, not simply the items that already blew up.
for node in ast.stroll(tree):
if isinstance(node, ast.Title) and node.id == constant_name:
affected.append(str(py_file.relative_to(root)).substitute("", "/"))
break
$ promptctl influence customer_router
Immediate:
customer_router
Affected Modules
- evaluator.py
- router.py
- checks/test_router.py
Complete: 3 modules
Visualized as a dependency graph, here’s what influence evaluation really walks:

All three information present up within the affected modules record, together with the take a look at file that contract validation skipped.
That’s the reason these are two separate passes. A file is perhaps high-quality technically proper now, however a developer ought to nonetheless take a look at it every time a immediate adjustments in a serious manner.
The Full Pipeline, Finish to Finish
Working promptctl verify executes all three passes and merges all the pieces right into a single report.
The core of that output is an identical to what we simply walked by means of: the variable diff from PromptDiff, the lacking variable breakdown from Contract Validation, and the module record from Affect Evaluation.
What verify provides on prime is a top-level header exhibiting what baseline you might be evaluating in opposition to, plus a backside abstract with a clear, specific exit code:
$ promptctl verify
promptctl verify
===============================
Repository
Prompts scanned: 1
Python information: 4
Scan time: 0.003 s
Evaluating
Baseline: baseline.json
Present : mock_repo/
... with the three evaluation sections proven earlier ...
========================================
CHECK FAILED
Breaking Adjustments 1
Contract Violations 2
Affected Modules 3
Exit Code 1
Merge blocked.
========================================
That header is price pausing on. Earlier than any move runs, a developer a CI log immediately is aware of what baseline and repository state are being in contrast, no guessing required. It feels like a minor element, however six months down the road if you’re digging by means of another person’s construct logs, it solutions the one query everybody forgets to log: in comparison with what, precisely?
The abstract on the finish is the place this begins feeling like ruff verify or terraform validate. Nothing is hardcoded right here:
- Breaking Adjustments counts prompts the place variables had been deleted.
- Contract Violations counts every particular person name website that can fail.
- Affected Modules tracks the overall distinctive information flagged throughout each impacted immediate.
Plug that exit code straight right into a pre-commit hook or CI pipeline, and people three strains give your merge gate all the pieces it must move or fail a construct.
What This Really Buys You, In comparison with the Standing Quo
| No validation (the sample above) | Unit checks that mock the LLM | promptctl verify | |
|---|---|---|---|
| Catches a renamed immediate variable | No | No, mocks skip the true .format() name |
Sure |
| Catches it earlier than deploy | No | Technically sure, however doesn’t hearth | Sure |
| Requires a mannequin name | No | No | No |
| Requires community entry | No | No | No |
| Tells you which of them information are affected | No | No | Sure, by file and line |
| Time to run in opposition to this repo | N/A | Seconds (current suite) | ~3 ms of scan work |
That center column is the important thing right here.
Groups aren’t skipping checks. A codebase can have 100% inexperienced checks and nonetheless ship this actual crash to manufacturing. The problem is that mocking the mannequin consumer creates a blind spot. For the reason that take a look at mocks out the decision earlier than .format() ever runs on the modified template, the KeyError by no means fires.
Mocking LLM calls in unit checks remains to be the suitable transfer—it retains checks quick and deterministic. But it surely leaves a spot. A fast, static structural verify fills that hole with out forcing anybody to rewrite their take a look at suite or spin up precise API calls.
Efficiency Traits
Measured on Python 3.12 utilizing normal library modules in opposition to the four-file mock repository, averaged throughout a number of runs:
| Operation | Inner scan time | What dominates |
|---|---|---|
| PromptDiff | ~1 ms | Parsing one file’s AST |
| Contract Validation | ~1 ms | Strolling 3 caller information |
| Affect Evaluation | ~1 ms | Title-reference scan throughout 3 information |
Full verify (all three) |
~3 ms | Sum of the above |
verify, full course of incl. Python startup |
~50–90 ms | Python interpreter startup, not the instrument |
At this scale, the scan itself is virtually free. Each move runs in underneath 10 milliseconds.
That barely bigger quantity within the final row isn’t the evaluation chunking by means of code—it’s simply Python’s startup overhead. It stays roughly the identical whether or not your codebase has one immediate or fifty.
Since each move is only a linear stroll over the file tree, the runtime stays low-cost because the repository grows. You get quick checks with out the large overhead or latency of instruments that spin up API calls to examine your prompts.
The Repair, and Why “Zero Breaking Adjustments” Is Nonetheless Trustworthy
The fascinating half isn’t the failing run—it’s what occurs after you repair it. Getting this fallacious would smash the instrument’s credibility quicker than any bug may.
When you replace the 2 damaged name websites to move ticket_id=, the immediate template itself stays untouched. So what does a second promptctl verify report?
$ promptctl verify --repo mock_repo_after_fix --baseline baseline_after_fix.json
promptctl verify
===============================
Repository
Prompts scanned: 1
Python information: 4
Scan time: 0.003 s
Evaluating
Baseline: baseline_after_fix.json
Present : mock_repo_after_fix/
PromptDiff
----------
No breaking variable adjustments.
Contract Validation
-------------------
No contract violations.
Affect Evaluation
---------------
No immediate adjustments to verify.
========================================
CHECK PASSED
Breaking Adjustments 0
Contract Violations 0
Exit Code 0
Repository is secure.
========================================
Zero breaking adjustments. Wait, the immediate genuinely did change earlier: {ticket} actually grew to become {ticket_id}. Why does the diff report zero this time?
As a result of this run compares in opposition to a brand new baseline, baseline_after_fix.json, which represents the immediate’s state after the change was reviewed and merged.
That isn’t a shortcut; it’s the core design. A baseline isn’t a everlasting report of what the code regarded like on day one. It’s simply the final state everybody agreed on. As quickly as you overview and merge a change, that new state turns into your start line for the subsequent verify. It’s the identical precept database migration instruments use: as soon as a schema replace runs, you examine future adjustments in opposition to that new structure, not the setup from three years in the past.

PromptDiff asks if the immediate modified for the reason that final accredited state. Contract Validation asks if the present code works proper now. They aim fully totally different issues. If a instrument merely reported “no adjustments” after each fast repair, it wouldn’t really be validating your code. It will simply be telling you what you need to hear.
Trustworthy Design Choices
A number of selections listed below are easy conventions moderately than common truths. I might moderately name them out straight than fake they’re extra principled than they are surely.
The _PROMPT Suffix Conference
Image registration depends fully on a easy naming rule: any top-level string fixed ending in _PROMPT will get handled as a registered immediate. That retains issues easy and avoids further configuration, nevertheless it means any immediate named with out that suffix stays invisible to the instrument. A broader system would possibly depend on an specific registry or a decorator. I went with the naming conference as a result of it takes zero effort to undertake and suits how most codebases I work with already identify these constants.
Static AST Parsing, Not Execution
Each move inspects the supply code with out working it. That makes it utterly secure for CI pipelines with no uncomfortable side effects, and it retains execution lightning quick. The trade-off is obvious: something constructed at runtime, like a immediate key constructed dynamically from a variable, slips previous a static parser by design.
Flat JSON Baselines Over Git Historical past
Utilizing a flat JSON file for baselines retains snapshot diffs trivial to overview inside a pull request. The draw back is that this file can drift out of sync if no person updates it after a merge, which this model leaves as a guide step.
What It Detects (And What It Misses)
It catches:
- Variable-level breaking adjustments in contrast in opposition to a baseline.
- Name websites that break the present immediate’s contract.
- Each file that imports or references a particular immediate fixed.
It misses:
- Dynamic immediate keys constructed at runtime, like
registry.get(f"prompt_{position}"). - Templates saved outdoors your Python information, equivalent to in a database or a distant CMS.
- The precise semantic high quality or reasoning capacity of the textual content your immediate generates.
In case your stack depends closely on these setups, this instrument is not going to assist with these elements. A static analyzer that overpromises on protection is harmful as a result of as soon as a staff sees a inexperienced verify mark, they cease watching out for the bugs the instrument was by no means constructed to catch.
Commerce-offs and What’s Lacking
It is a v1, and I saved its scope slim on objective. Listed here are a couple of options I intentionally omitted moderately than forgot about:
- Runtime regression testing: This instrument strictly checks construction earlier than something runs. Determining whether or not a immediate rewrite degraded precise mannequin output is an analysis downside, not a static validation one. You would wish to make dwell mannequin calls to reply that.
- Multi-version baselines: The instrument presently tracks a single baseline: the final accepted state. In the event you run a number of environments with totally different immediate variations deployed concurrently, like staging versus manufacturing, you would possibly must diff in opposition to these targets independently.
- Automated immediate migrations: Whenever you deliberately rename a variable like
{ticket}to{ticket_id}, the instrument may theoretically refactor name websites for you. It presently features like a schema checker moderately than a migration script. - Computerized baseline updates: Updating the baseline snapshot after a PR merges remains to be a guide step. Ideally, a CI set off would replace and commit that state robotically on merge.
None of these options exist on this construct. Every represents a basically totally different scope. Packing all of them in from the beginning is how light-weight validators flip into bloated instruments that folks cease trusting.
Why Not Simply Use an Present Device for This?
Earlier than writing one thing new, it’s price asking: doesn’t an current kind checker or schema library already cowl this?
It helps to stroll by means of the principle alternate options and see why every one misses this particular hole.
1. Sort Checkers (mypy, pyright)
Sort checkers consider sorts, not the particular characters inside a string literal.
str.format(ticket=x) passes kind checks with none points, no matter which key phrase arguments you feed it. That’s as a result of .format() is designed to just accept arbitrary key phrase arguments (**kwargs). The production-breaking bug isn’t a sort error; it’s a value-level mismatch between string placeholders and key phrase arguments. That sits fully outdoors what a sort checker appears to be like for.
2. Schema Libraries (Pydantic)
You may wrap each immediate inside a Pydantic mannequin with specific fields as a substitute of utilizing uncooked string templates. That may catch this class of bug.
Nevertheless, doing that forces you to rewrite how each immediate throughout your codebase is outlined and invoked. That incurs a heavy migration value as a substitute of offering a lightweight, drop-in verify you’ll be able to run on current code. promptctl was designed particularly to work with the usual sample builders already use—module-level string constants and .format()—with out forcing architectural refactors.
3. Handbook Code Evaluate Checklists
In apply, that is what most groups depend on: a pull request reviewer is predicted to note variable adjustments and manually confirm each caller.
This method fails as quickly as a PR will get too massive to carry in your head. It assumes the reviewer is aware of each location the place that immediate will get referred to as. That assumption breaks down quickly as a repository grows—which is exactly when these bugs change into most harmful.
The Core Hole: These current instruments aren’t ineffective; they simply reply totally different questions. The query right here is hyper-specific: Does this string template’s contract nonetheless match each single caller, checked robotically in CI, with out rewriting current code?
promptctl isn’t about treating prompts as particular instances. It exists as a result of this actual, slim contract verify is lacking from most immediate administration platforms, even when these platforms deal with versioning, eval suites, and observability effectively.
The place This Differs From Regression Testing
Checking mannequin conduct means working inputs by means of an LLM to guage issues like tone, output formatting, or accuracy. That could be a actual engineering problem, however answering these questions forces you to ship dwell API requests. It’s important to cope with community latency, token prices, and variable mannequin responses.
promptctl stays fully native. It inspects the AST to confirm that the key phrase arguments in your code match the placeholders inside your template, effectively earlier than any API payload will get constructed.
If a immediate revision causes the mannequin to present weaker solutions, this instrument is not going to spot that. What it offers you is a zero-cost assure that your Python code is not going to crash on a lacking variable the second a person triggers that path.
These checks belong at totally different levels of your workflow. Quick, static checks belong on each native decide to catch damaged code paths early. Heavy behavioral evaluations make sense downstream, as soon as you recognize the applying itself executes cleanly.
Attempting It Your self
git clone https://github.com/Emmimal/promptctl
cd promptctl
python3 promptctl.py verify
That reproduces the CHECK FAILED output above, in opposition to the identical mock repo with the identical intentional rename. The passing case:
python3 promptctl.py verify --repo mock_repo_after_fix --baseline baseline_after_fix.json
It runs out of the field on any trendy Python 3 set up. You do not want a digital surroundings, a config file, or an API key to get began.
Closing
Making a immediate is straightforward sufficient if you first write it. The true bother begins months later when another person modifies that string, lengthy after the unique context is forgotten and the staff has shifted.
Most immediate administration instruments concentrate on preliminary creation, model histories, or dwell evaluations. Few deal with the quiet danger of small textual content edits breaking utility name websites additional down the road.
promptctl fills that particular hole with out making an attempt to do all the pieces else. It depends on three static evaluation passes, returns a single exit code on your CI pipeline, and stays utterly specific about its limitations.
References
[1] HashiCorp. (n.d.). terraform validate command reference. Terraform Documentation. https://developer.hashicorp.com/terraform/cli/instructions/validate
[2] Hamon, Y. (n.d.). kubeconform: A FAST Kubernetes manifests validator. GitHub. https://github.com/yannh/kubeconform
[3] Astral. (n.d.). Ruff: An especially quick Python linter and code formatter. GitHub. https://github.com/astral-sh/ruff
[4] python/mypy. (n.d.). mypy: Non-compulsory static typing for Python. GitHub. https://github.com/python/mypy
[5] Python Software program Basis. (n.d.). ast: Summary Syntax Bushes. Python 3 documentation. https://docs.python.org/3/library/ast.html
[6] Python Software program Basis. (n.d.). string.Formatter. Python 3 documentation. https://docs.python.org/3/library/string.html#string.Formatter
[7] Python Software program Basis. (n.d.). argparse: Parser for command-line choices. Python 3 documentation. https://docs.python.org/3/library/argparse.html
The total supply code for PromptCTL, together with the mock repository and each baseline snapshots used on this article, is out there on GitHub: https://github.com/Emmimal/promptctl
Disclosure
All code on this article was written by me and is unique work. This implementation was developed and examined on Python 3.12 (Home windows 11), normal library solely, no exterior dependencies. Benchmark numbers are from precise runs in opposition to the mock repository included within the supply, and are reproducible by cloning the repository and working promptctl.py verify, besides the place the article explicitly notes a quantity is measured otherwise. I’ve no monetary relationship with any instrument, library, or firm talked about on this article.
Get in Contact
In the event you discovered this text helpful or have ideas on immediate reliability and static validation, I’d love to listen to from you.
Join with me:
















