Stopping Spec Drift in the Era of Fast AI Coding: Building an OSS Called jev-spec

The Reality of Fast AI Coding and Spec Drift

About six months ago, I wrote an article about contextlint, a linter that statically analyzes the consistency of Markdown documents.

In Specification-Driven Development (SDD), teams translate business “requirements” into system behaviors as “specifications (acceptance criteria)”, and further flesh them out into “designs” such as table schemas and screen layouts, capturing them in Markdown and feeding them into AI coding agents like Cursor or Claude Code.

AI writes code at breathtaking speed. However, as the velocity of code generation increases, the frequency with which human developers reread original specifications drops proportionally. In day-to-day code reviews, attention naturally gravitates toward immediate diffs and runtime smoke tests. Reviewing whether every sentence in the specification still holds against the generated code’s behavior is rarely feasible. As a result, discrepancies between specifications and code accumulate unnoticed beneath the surface.

Static analysis (contextlint and similar tools) can confirm that a specification contains an entry such as REQ-AUTH-02 (used here as an example of a requirement or specification identifier), follows the expected format, and is properly referenced across documents. But whether src/auth/session.ts still behaves according to REQ-AUTH-02 cannot be verified by parsing document syntax alone.

To counter this drift, some teams create custom slash commands or Agent Skills in Cursor or Claude Code to periodically crawl the codebase and check alignment against specifications. In practice, this means asking an agent: “Check whether this code satisfies all requirements in the spec and flag any omissions.”

Yet running this approach on every commit is impractical. Stuffing substantial codebases and full specifications into prompts and running general-purpose LLM agents takes minutes per run and consumes significant token costs. It is far too slow and expensive to act as a pre-commit hook gating git commit. Consequently, drift detection remains an occasional, heavy audit run only when someone remembers, while discrepancies between specifications and code continue to accumulate unnoticed during daily development.

“Could we catch the drift between specifications and code in tenths of a second on every single commit?” This question led me to develop the OSS jev-spec.

jev-spec

Catch spec drift on every commit: check your code against your Markdown specs with TypeSafe AI's Jev model.

🔗github.com

The Limits of Traditional LLMs and the Arrival of Jev

When attempting to build a pre-commit gate that reconciles code against specifications, traditional LLMs face structural bottlenecks.

General-purpose LLMs are fundamentally autoregressive: they choose the next token conditioned on tokens already produced. JSON mode and Structured Outputs still sample the output as a token sequence that fits the schema. The schema only constrains which tokens are legal next. It does not change the generation method. Longer output increases wait time and output-token cost. Wait time also includes input processing, reasoning-token generation, and the first schema compilation. Billing is not limited to output tokens. Input tokens are charged separately. To avoid breaking a developer’s flow during a local pre-commit check, verification must finish in under a second. High latency and recurrent token billing make traditional LLMs ill-suited for every commit.

A fresh architectural direction arrived with Jev, a model developed by TypeSafe AI. TypeSafe categorizes Jev as a “System One model.”

System One - TypeSafe

System One models make fast, structured decisions for software. Jev is TypeSafe's flagship model and the first System One model.

🔗docs.typesafe.ai

To describe it accurately: Jev does not perform free-form autoregressive text generation or sequential token decoding. Instead, it directly evaluates given context or state (text, data, application state, and similar inputs) in a single pass and outputs typed decision primitives: well-calibrated probabilities and scores. jev-spec uses this capability by passing specifications and code as context and reading probabilities for each spec item’s Rubric.

Because it does not generate text token by token, there is no waiting on output decoding, and output tokens are free. Input token costs are negligible ($0.042 per million input tokens), and response times clock in at just 70 to 400ms.

Jev provides three core decision primitives:

• noul (Boolean proposition)

  evaluates a proposition answerable with yes or no, returning a probability between 0 and 1

• choice (Categorical distribution)

  evaluates a discrete probability distribution across declared, mutually exclusive options (such as compliant, vulnerable, or inconclusive)

• score (Ordinal rubric)

  evaluates an ordered rubric (such as a multi-tier maturity scale from stub to production-grade), returning an expected score and level probabilities

Because Jev’s output is numeric and grounded from the start, there is no need to parse natural language. Comparing the returned probability against a deterministic threshold (such as minProbability: 0.85) produces an immediate pass or fail verdict.

Note

jev-spec is an independent open-source project that I develop on my own. It is not affiliated with or endorsed by TypeSafe AI. Descriptions of Jev’s properties and pricing reflect the official documentation as of September 2026.

Expanding Horizons: Jev’s Broader Potential and the Genesis of jev-spec

It is worth emphasizing that Jev was not designed exclusively for specification verification.

Observing the developer community on X, engineers are leveraging this ultra-fast, low-cost decision primitive to build software interactions that were previously unviable.

Marcus Lowe’s smart clipboard demo instantly evaluates copied text, recognizes whether it contains contact details or personal credentials, and routes the content to corresponding form fields in milliseconds. OkinaAudio’s creative audio workflow for Ableton demonstrates fast audio and UI interaction.

Unlike open-ended text generation models, executing fast, inexpensive, type-safe decisions in milliseconds opens up distinct possibilities for software interaction. As an engineer, watching this architectural direction unfold is genuinely exciting.

Seeing Jev’s characteristics firsthand sparked an immediate question from my background building contextlint:

“If we have a decision primitive this fast and lightweight, could we verify whether code satisfies the behavior and requirements written in specifications on every single git commit?”

Exploring whether this fast decision model could be applied to Specification-Driven Development is what motivated me to start building jev-spec.

How jev-spec Works

The operational pipeline in jev-spec consists of four steps:

1. Read the spec

   extracts the statements to verify from a Markdown specification based on sections, headings, or specification/requirement IDs

2. Read the code

   loads the source files implementing that specification

3. Evaluate rubrics

   submits focused evaluation questions (Rubrics) per spec item to Jev, which evaluates the spec and code as context in a single pass and returns probabilities in parallel

4. Compare

   evaluates the probabilities against defined threshold assertions, stopping the build as a validation error (check failure) if any assertion fails

Each evaluation question is called a Rubric, and the comparison with a threshold is an Assertion. While binary verification with noul is most common for specs, the DSL also supports choice and score.

For speed and rigorous type safety, jev-spec adopts TypeScript 7, powered by the new Go compiler. Furthermore, it provides official dual runtime support for both Node.js (22+) and Bun.

Basic Usage

Getting started with jev-spec in your project is straightforward.

Agent Skills clients (Claude Code, Cursor, Codex, Antigravity CLI, and GitHub Copilot) can install a skill with the following command:

gh skill install nozomi-koborinai/jev-spec <skill-name>

Skill	Role
jev-spec-init	Maps the spec to the code and drafts the Rubrics
jev-spec-fix	Helps fix a failed check

1. Installation

Add jev-spec to your devDependencies using your package manager of choice:

npm install -D jev-spec
# or
bun add -d jev-spec

2. Configuration (jev.config.ts)

Create a jev.config.ts file in your project root and declare the mapping between specifications and code in targets:

import { defineConfig, noul } from 'jev-spec';

export default defineConfig({
  client: { model: 'jev-1.13.0' },
  targets: {
    auth: {
      specPath: 'docs/specs/auth-requirements.md',
      codePaths: ['src/auth/**/*.ts', '!src/auth/**/*.test.ts'],
      rubrics: {
        'REQ-AUTH-01': noul(
          'Is the signature of a session token checked before access to a protected resource is granted?'
        ),
        'REQ-AUTH-02': noul('Is a token rejected when its ID is on the revocation list?'),
      },
      assertions: {
        'REQ-AUTH-01': { minProbability: 0.85 },
        'REQ-AUTH-02': { minProbability: 0.85 },
      },
    },
  },
});

Key options configured in defineConfig include:

• specPath

  path to the target Markdown specification file

• codePaths

  glob patterns for code files implementing that specification (with negation support to exclude test files)

• rubrics

  spec and behavior checks written as natural-language questions using noul(...) (keyed by IDs such as REQ-AUTH-01 or target spec item names)

  Note: Keying by IDs is merely one example. Even if your project does not strictly assign requirement IDs, you can organize rubrics with arbitrary key names.

• assertions

  minimum confidence threshold for passing (such as minProbability: 0.85)

3. Setting the API Key

Export your TypeSafe AI API key as an environment variable:

export TYPESAFE_AI_API_KEY="your-api-key"

4. Running Checks

Once configured, run checks from the CLI.

To verify all targets across the entire repository:

npx jev-spec check
# or
bunx jev-spec check

To verify only targets affected by staged Git changes before committing, use the --staged flag. Targets unaffected by the staged diff are automatically SKIPPED:

npx jev-spec check --staged

The CLI report prints clear, structured feedback:

$ npx jev-spec check

=== jev-spec Check Report ===

Target: auth [✖ FAILED]
  Spec files: docs/specs/auth-requirements.md
  Code files: src/auth/session.ts
  Model: jev-1.13.0
    ✔ REQ-AUTH-01: probability: 0.97
    ✖ REQ-AUTH-02: probability: 0.08
       └─ Violation: Probability 0.08 is below minimum threshold 0.85

Overall: ✖ CHECKS FAILED

$ echo $?
1

Displaying comparison results between probabilities and thresholds per spec item (ID) makes it straightforward to identify exactly which specification and code have drifted.

5. Workflow Integration

jev-spec shines most when embedded into your routine development workflow as an everyday gate:

• Staged checks in pre-commit hooks

  using tools like husky or simple-git-hooks, run npx jev-spec check --staged on every commit. Because it evaluates only modified targets in milliseconds, it catches drift before code is committed without interrupting your flow

• Full checks in CI

  in your CI pipeline (such as GitHub Actions), run a full npx jev-spec check across all targets to guarantee spec compliance before merging pull requests

What Real-World Measurements Revealed

Gating builds on a probabilistic model raised questions about cost and reproducibility. To investigate, I conducted empirical measurements on jev-spec’s own repository, which maintains its own specs (docs/specs/). The numbers below were recorded on September 21, 2026 using jev-1.13.0.

Measured Cost

Checking all 8 targets and 22 Rubrics across the entire repository completed in approximately 4 seconds. Individual targets took between 0.3 and 0.9 seconds. CLI startup alone took roughly 0.1 seconds on Node.js (and was even faster on Bun); the vast majority of runtime was spent on network requests.

The estimated cost reported by the CLI was approximately $0.0006 per full run. Rather than stuffing the entire codebase into context, jev-spec transmits only the relevant spec excerpt and a handful of files per target. Jev evaluates all Rubrics in that target in a single pass. The cost of running this on every commit is negligible.

Reproducibility and Model Pinning

Repeating identical Rubrics over identical code five times yielded a confidence variance of 0.00 to 0.03 on clean code. A probability moving from 0.96 to 0.95 remains safely above the example threshold of 0.85 (configured as minProbability in the settings).

Thresholds are not hardcoded values; they can be configured individually in the settings based on the nature of the specifications and requirements. In addition, because aliases such as latest can resolve to a newer model version and cause unintended shifts in evaluations, specifying an explicit versioned ID (such as jev-1.13.0) in client.model is recommended. Pinning the model ensures consistent evaluations over time and prevents unexpected behavioral changes, with the report explicitly displaying the model ID used.

Static Analysis vs. Decision Models, and Conclusion

Deploying jev-spec effectively requires recognizing its operational boundaries:

• A check, not a mathematical proof

  passing indicates a probability surpassed an empirical threshold, not formal correctness. Internal terminology deliberately uses “check” rather than “verify”

• English is officially recommended

  while official documentation recommends English and notes that CJK languages may not reach the same accuracy, in practice Jev also understands Japanese specifications and Rubrics quite well and can evaluate them effectively

• Adversarial code comments can bias evaluations

  comments stating “this function satisfies the specification” can skew decisions. TypeSafe lists this as a known model limitation

• Arithmetic, dates, and multi-step reasoning are poor fits

  counting items or comparing calendar dates should be verified via standard unit tests rather than Rubrics

The complementary roles of contextlint and jev-spec can be summarized as follows:

Dimension	contextlint	jev-spec
Scope	Consistency across documents	Alignment between specs and code
Method	Rule-based static analysis	Model Rubric evaluation and threshold comparison
Determinism	Deterministic	Probabilistic
AI Dependency	None	Yes (TypeSafe AI Jev)

In an era where AI generates code at unprecedented speed, guarding against specification drift without slowing down development rhythm is essential.

jev-spec started as a personal experiment, but if you maintain specifications in Markdown and worry about drift from code, consider giving it a try.

jev-spec - npm

Catch spec drift on every commit: check your code against your Markdown specs with TypeSafe AI's Jev model.

🔗npmjs.com