docs(changelog): add semantic similarity entries

Document validators module with Check protocol, metric validators,
constraint validators, composite validators, and factory functions.

CLAUDE.md

@@ -0,0 +1,116 @@
+# CLAUDE.md
+
+Guidelines for working on the Veritext project.
+
+## Project Overview
+
+Veritext is a semantic text validation framework for Python. It validates text outputs
+against quality criteria using metrics like BLEU, ROUGE, and semantic similarity.
+
+## Directory Structure
+
+```
+veritext/
+├── src/veritext/          # Package source
+│   ├── core/              # Shared types, tokenisation, config
+│   ├── metrics/           # BLEU, ROUGE, lexical, readability
+│   ├── semantic/          # Optional embedding-based similarity
+│   ├── validators/        # Composable validation checks
+│   ├── benchmark/         # Quality tracking & regression detection
+│   ├── pytest_plugin/     # Native pytest integration
+│   └── cli/               # Command-line interface
+├── tests/                 # Test suite (mirrors src structure)
+├── docs/                  # Project documentation
+└── examples/              # Usage examples
+```
+
+## Code Style
+
+### Python Conventions
+
+- **Python 3.11+** with modern type hints
+- **UK English** in all text (colour, behaviour, summarisation, tokenisation)
+- **snake_case** for variables, functions, modules
+- **PascalCase** for classes
+- Absolute imports from package root: `from veritext.core.types import ...`
+
+### Quality Gates
+
+All must pass with zero issues before any commit:
+
+```bash
+uv run ruff check .              # Linting
+uv run ruff format --check .     # Formatting
+uv run mypy src/                 # Type checking
+uv run pytest                    # Tests
+```
+
+### Documentation
+
+- Docstrings for all public APIs (Google style)
+- Type hints on all function signatures
+- Keep docstrings concise; let types speak where possible
+
+## Architecture
+
+### Layer Dependencies
+
+```
+CLI / pytest_plugin  (presentation)
+        ↓
+validators / benchmark  (decision logic)
+        ↓
+metrics  (pure computation)
+        ↓
+core  (shared types, tokenisation)
+```
+
+Each layer depends only on layers below it.
+
+### Metrics vs Validators
+
+| Concept | Responsibility | Output |
+|---------|----------------|--------|
+| **Metric** | Compute a score | Typed result (e.g., `BleuResult`) |
+| **Validator** | Make pass/fail decision | `ValidationResult` with diagnostics |
+
+### Edge Case Handling
+
+- Empty text: Metrics return zero scores; validators fail
+- Empty reference: Comparison metrics raise `ValueError`
+- Whitespace-only: Treated as empty after tokenisation
+- Unicode: NFC normalisation by default
+
+## Git Workflow
+
+### Commits
+
+- Format: `type(scope): description`
+- Types: feat, fix, chore, refactor, docs, test
+- Atomic: ≤3 new files, ≤150 LOC per commit
+- Update changelog.md before completing a task
+
+### Branches
+
+- `feat/kebab-case` — new features
+- `fix/kebab-case` — bug fixes
+- `chore/` — maintenance
+- `refactor/` — code restructure
+- `docs/` — documentation only
+
+## Testing
+
+- Test files mirror source structure: `tests/test_core/test_types.py`
+- Use pytest fixtures for common setup
+- Target ≥80% coverage
+- Include edge cases: empty input, Unicode, boundary values
+
+## Pre-Completion Checklist
+
+Before marking ANY task complete:
+
+- [ ] All linting/formatting/type checks pass
+- [ ] Tests pass with adequate coverage
+- [ ] changelog.md updated if user-facing changes
+- [ ] Filenames are lowercase (except CLAUDE.md)
+- [ ] Commit follows `type(scope): description` format

changelog.md

@@ -7,108 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-### Changed
-
-- Refactored CLI metric computation to eliminate code duplication
-- Version format updated from `0.1.0-dev` to `0.1.0.dev0` (PEP 440 compliance)
-- Settings instance is now cached via `@lru_cache` for better performance
-- Documented composite validators' intentional deviation from `Check` protocol return type
-
-### Fixed
-
-- Consolidated redundant empty checks in ROUGE-L computation
-- Fixed README example using incorrect property names (`grade_level` → `flesch_kincaid_grade`, `reading_ease` → `flesch_reading_ease`)
-
-### Documentation
-
-- Added Phase 10 (Portfolio Demos) to implementation plan: Streamlit demo and Jupyter notebooks
-- Updated project plan with portfolio demo section
-- Fixed potential crash in ROUGE metric when all references are empty after tokenisation
-- Fixed potential division by zero in readability metric when text has no sentence endings
-- Fixed unbounded cache growth in `SemanticSimilarity` by implementing LRU eviction with configurable max size
-- Fixed mutable list aliasing in `AllOf` and `AnyOf` composite validators
-- Fixed regex pattern validation in `ContainsValidator` and `ExcludesValidator` to fail at init time rather than during `check()`
-- Fixed pytest plugin tests failing with duplicate plugin registration error
-
 ### Added
 
-- Added `.score` property to `LexicalResult` for API consistency with other result types
-- Added `cache_max_size` parameter to `SemanticSimilarity` (default: 1000 embeddings)
-- Added test coverage for `core/config.py` and `core/logging.py` modules
-
-## [0.1.0] — 2025-05-17
-
-Initial release of Veritext, a semantic text validation framework for Python.
-
-### Added
-
-#### Core
-
 - Project scaffold with pyproject.toml and development tooling
 - Core exception hierarchy (`VeritextError` and subclasses)
 - Core types: `ValidationContext`, `CheckResult`, `ValidationResult`
 - Word tokeniser with Unicode normalisation support
 - Configuration module with pydantic-settings
 - Structured logging with structlog
-
-#### Metrics
-
 - Metrics module with `Metric` protocol, `AggregateStats`, and `BatchResult` types
 - BLEU metric implementation (BLEU-1 through BLEU-4 with brevity penalty)
-- ROUGE metric (ROUGE-1, ROUGE-2, ROUGE-L with precision/recall/F-measure)
 - Lexical similarity metric (Jaccard similarity and token overlap)
+- ROUGE metric (ROUGE-1, ROUGE-2, ROUGE-L with precision/recall/F-measure)
 - Flesch-Kincaid readability metrics (grade level and reading ease)
 - Batch scoring with aggregate statistics for all metrics
-
-#### Validators
-
 - Validators module with `Check` protocol for validation checks
 - Metric-based validators: `BleuValidator`, `RougeValidator`, `LexicalValidator`
 - Constraint validators: `LengthValidator`, `ReadabilityValidator`, `ContainsValidator`, `ExcludesValidator`
 - Composite validators: `AllOf` (all checks must pass), `AnyOf` (any check must pass)
 - Factory functions for clean validator API (`bleu()`, `rouge()`, `lexical()`, `length()`, `readability()`, `contains()`, `excludes()`, `all_of()`, `any_of()`)
-
-#### Semantic Similarity
-
 - Semantic similarity module with embedding-based text comparison (requires `veritext[semantic]` extra)
 - `SemanticSimilarity` metric using sentence-transformers for semantic relatedness
 - `SemanticValidator` for threshold-based semantic similarity validation
 - `semantic()` factory function for creating semantic validators
 - Embedding caching for performance optimisation in repeated comparisons
-
-#### Pytest Plugin
-
-- Native pytest plugin for CI/CD integration (entry point: `pytest11`)
-- `validate_text()` assertion function for expressive test assertions
-- `text_validation` marker for filtering validation tests
-- Pytest fixtures: `text_validator` factory and `validation_context` helper
-- Detailed failure messages with text preview and check diagnostics
-
-#### Benchmarking
-
-- Benchmark module for quality tracking and regression detection
-- `Benchmark` class for evaluating text quality over time with metric storage
-- `BenchmarkRun` and `RegressionReport` data models for tracking runs
-- SQLite storage backend with WAL mode for concurrent access
-- Rolling window baseline computation for historical comparison
-- `check_regression()` for statistical comparison against baseline
-- `assert_no_regression()` raises `RegressionDetectedError` for CI integration
-- Customisable tolerance threshold and window size for regression detection
-- Metadata support for tracking git SHA, model versions, etc.
-
-#### CLI
-
-- Command-line interface (CLI) via `veritext` command
-- `veritext validate` command for inline and file-based text validation
-- JSONL input format support for batch validation (`--file` option)
-- Separate candidate/reference file support (`--reference-file` option)
-- Multiple output formats: table (default), JSON, and simple text
-- `veritext benchmark run` command for running evaluations and storing results
-- `veritext benchmark show` command for viewing benchmark history
-- `veritext benchmark check` command for regression detection with exit code 1 on failure
-- Rich-formatted terminal output with tables and coloured panels
-
-#### Documentation
-
-- Readme with usage examples
-- Example scripts: basic validation, chatbot testing, benchmark regression

docs/implementation-plan.md

@@ -0,0 +1,949 @@
+# Implementation Plan: Veritext
+
+Semantic text validation framework for Python — validates text outputs against quality criteria.
+
+## Project Overview
+
+**Location:** `portfolio/veritext/` (relative to workspace root)
+**Remote:** `https://gitea.kschappell.com/kschappell/veritext.git`
+
+**Purpose:** A Python library for validating text outputs against semantic criteria. Designed for developers building systems that produce text (chatbots, content generators, summarisation tools) who need automated quality assurance beyond simple string matching.
+
+---
+
+## Architectural Decisions
+
+### 1. Layered Architecture
+
+```
+┌─────────────────────────────────────────────────────┐
+│  CLI / pytest_plugin  (presentation layer)          │
+├─────────────────────────────────────────────────────┤
+│  validators/          (decision logic)              │
+│  benchmark/           (tracking & regression)       │
+├─────────────────────────────────────────────────────┤
+│  metrics/             (pure computation)            │
+├─────────────────────────────────────────────────────┤
+│  core/                (shared types, tokenisation)  │
+└─────────────────────────────────────────────────────┘
+```
+
+**Dependency rule:** Each layer depends only on layers below it.
+
+### 2. Metrics vs Validators (Clear Separation)
+
+| Concept | Responsibility | Output |
+|---------|----------------|--------|
+| **Metric** | Compute a score | Typed result object (e.g., `BleuResult`) |
+| **Validator** | Make pass/fail decision | `ValidationResult` with diagnostics |
+
+Validators wrap metrics and apply thresholds.
+
+### 3. Optional Heavy Dependencies
+
+`sentence-transformers` (~2GB with PyTorch) is optional:
+
+```toml
+[project.optional-dependencies]
+semantic = ["sentence-transformers>=2.2"]
+```
+
+Core library works without ML dependencies.
+
+### 4. Typed Result Objects
+
+Each metric returns a specific result type, not just `float`:
+
+```python
+@dataclass(frozen=True)
+class BleuResult:
+    bleu1: float
+    bleu2: float
+    bleu3: float
+    bleu4: float
+    brevity_penalty: float
+
+@dataclass(frozen=True)
+class RougeScore:
+    precision: float
+    recall: float
+    fmeasure: float
+
+@dataclass(frozen=True)
+class RougeResult:
+    rouge1: RougeScore
+    rouge2: RougeScore
+    rouge_l: RougeScore
+```
+
+### 5. Shared Tokenisation
+
+Single tokeniser used by all n-gram metrics:
+
+```python
+class Tokeniser(Protocol):
+    def tokenise(self, text: str) -> list[str]: ...
+
+class WordTokeniser:
+    def __init__(self, lowercase: bool = True, remove_punctuation: bool = True): ...
+```
+
+### 6. Explicit Context Object
+
+Validation context is explicit, not `**kwargs`:
+
+```python
+@dataclass
+class ValidationContext:
+    reference: str | list[str] | None = None
+    metadata: dict[str, Any] = field(default_factory=dict)
+```
+
+---
+
+## Directory Structure
+
+```
+veritext/
+├── src/
+│   └── veritext/
+│       ├── __init__.py              # Public API exports
+│       ├── py.typed                 # PEP 561 marker
+│       ├── core/
+│       │   ├── __init__.py
+│       │   ├── types.py             # ValidationContext, CheckResult, BatchResult
+│       │   ├── exceptions.py        # Exception hierarchy
+│       │   ├── tokenisation.py      # Shared tokeniser
+│       │   ├── config.py            # pydantic-settings
+│       │   └── logging.py           # structlog configuration
+│       ├── metrics/
+│       │   ├── __init__.py          # Metric exports
+│       │   ├── base.py              # Metric protocol
+│       │   ├── results.py           # BleuResult, RougeResult, etc.
+│       │   ├── bleu.py              # BLEU implementation
+│       │   ├── rouge.py             # ROUGE implementation
+│       │   ├── lexical.py           # Jaccard, token overlap
+│       │   └── readability.py       # Flesch-Kincaid, etc.
+│       ├── semantic/                # Optional (requires sentence-transformers)
+│       │   ├── __init__.py
+│       │   └── similarity.py        # Embedding-based similarity
+│       ├── validators/
+│       │   ├── __init__.py          # Validator exports
+│       │   ├── base.py              # Check protocol, ValidationResult
+│       │   ├── metric.py            # Validators wrapping metrics
+│       │   ├── constraint.py        # Length, content checks
+│       │   └── composite.py         # Validator composition
+│       ├── benchmark/
+│       │   ├── __init__.py
+│       │   ├── models.py            # BenchmarkRun, RegressionReport
+│       │   ├── storage.py           # SQLite backend
+│       │   ├── runner.py            # Benchmark execution
+│       │   └── regression.py        # Statistical detection
+│       ├── pytest_plugin/
+│       │   ├── __init__.py          # Plugin entry point
+│       │   ├── fixtures.py          # Pytest fixtures
+│       │   ├── assertions.py        # validate_text(), assert_similar()
+│       │   └── plugin.py            # Pytest hooks
+│       └── cli/
+│           ├── __init__.py
+│           └── main.py              # Typer CLI app
+├── tests/
+│   ├── conftest.py
+│   ├── test_core/
+│   │   ├── test_tokenisation.py
+│   │   └── test_types.py
+│   ├── test_metrics/
+│   │   ├── test_bleu.py
+│   │   ├── test_rouge.py
+│   │   ├── test_lexical.py
+│   │   └── test_readability.py
+│   ├── test_semantic/
+│   │   └── test_similarity.py
+│   ├── test_validators/
+│   │   ├── test_metric_validators.py
+│   │   ├── test_constraint_validators.py
+│   │   └── test_composite.py
+│   ├── test_benchmark/
+│   │   ├── test_storage.py
+│   │   ├── test_runner.py
+│   │   └── test_regression.py
+│   ├── test_pytest_plugin/
+│   │   └── test_integration.py
+│   └── test_cli/
+│       └── test_commands.py
+├── examples/
+│   ├── basic_validation.py
+│   ├── chatbot_testing.py
+│   └── benchmark_regression.py
+├── docs/
+│   ├── project-plan.md
+│   └── implementation-plan.md
+├── pyproject.toml
+├── readme.md
+├── changelog.md
+└── CLAUDE.md
+```
+
+---
+
+## Exception Hierarchy
+
+```python
+class VeritextError(Exception):
+    """Base exception for all Veritext errors."""
+
+class MetricError(VeritextError):
+    """Error during metric computation."""
+
+class TokenisationError(MetricError):
+    """Error during text tokenisation."""
+
+class EmbeddingError(MetricError):
+    """Error computing embeddings (semantic similarity)."""
+
+class ValidationError(VeritextError):
+    """Error during validation."""
+
+class InvalidThresholdError(ValidationError):
+    """Invalid threshold value provided."""
+
+class BenchmarkError(VeritextError):
+    """Error during benchmarking."""
+
+class StorageError(BenchmarkError):
+    """Error reading/writing benchmark storage."""
+
+class RegressionDetectedError(BenchmarkError):
+    """Quality regression detected (used in CI)."""
+
+class ConfigurationError(VeritextError):
+    """Invalid configuration."""
+
+class DependencyError(VeritextError):
+    """Optional dependency not installed."""
+```
+
+---
+
+## Core Interfaces
+
+### Metric Protocol
+
+```python
+from typing import Protocol, TypeVar, Generic
+
+T = TypeVar("T")
+
+class Metric(Protocol[T]):
+    """Protocol for text comparison metrics."""
+
+    @property
+    def name(self) -> str: ...
+
+    @property
+    def requires_reference(self) -> bool:
+        """Whether this metric requires a reference text."""
+        ...
+
+    def score(self, candidate: str, reference: str | list[str] | None = None) -> T:
+        """
+        Compute metric score.
+
+        Args:
+            candidate: The text to evaluate.
+            reference: Reference text(s) for comparison. Required for comparison
+                       metrics (BLEU, ROUGE, semantic). Ignored for standalone
+                       metrics (readability).
+
+        Raises:
+            ValueError: If reference is required but not provided.
+        """
+        ...
+
+    def batch_score(
+        self,
+        candidates: list[str],
+        references: list[str] | list[list[str]] | None = None,
+    ) -> BatchResult[T]: ...
+
+@dataclass
+class AggregateStats:
+    mean: float
+    std: float
+    min: float
+    max: float
+    percentiles: dict[int, float]  # {25: 0.65, 50: 0.72, 75: 0.81, 95: 0.89}
+
+@dataclass
+class BatchResult(Generic[T]):
+    results: list[T]                     # Individual results per sample
+    count: int
+    stats: dict[str, AggregateStats]     # Aggregated stats for numeric fields
+```
+
+**Note:** Standalone metrics like readability return `False` for `requires_reference` and ignore the `reference` parameter. Comparison metrics (BLEU, ROUGE, semantic) return `True` and raise `ValueError` if `reference` is `None`.
+
+### Validator Protocol
+
+```python
+class Check(Protocol):
+    """Protocol for individual validation checks."""
+
+    @property
+    def name(self) -> str: ...
+
+    def check(self, text: str, context: ValidationContext) -> CheckResult: ...
+
+@dataclass
+class CheckResult:
+    name: str
+    passed: bool
+    actual: Any
+    threshold: Any | None
+    message: str
+
+@dataclass
+class ValidationResult:
+    passed: bool
+    checks: list[CheckResult]
+
+    @property
+    def failure_summary(self) -> str: ...
+
+    @property
+    def failed_checks(self) -> list[CheckResult]: ...
+```
+
+### Benchmark Models
+
+```python
+@dataclass
+class BenchmarkRun:
+    id: str                          # UUID
+    benchmark_name: str
+    timestamp: datetime
+    veritext_version: str            # Track library version
+    metrics: dict[str, float]        # {"rouge_l": 0.82, "bleu4": 0.71}
+    sample_count: int
+    metadata: dict[str, Any]         # {"git_sha": "abc123", "model": "v2"}
+
+@dataclass
+class RegressionReport:
+    detected: bool
+    baseline: dict[str, float]
+    current: dict[str, float]
+    deltas: dict[str, float]         # {"rouge_l": -0.05}
+    tolerance: float
+
+    @property
+    def summary(self) -> str: ...
+```
+
+---
+
+## Edge Case Handling
+
+All components must handle edge cases consistently:
+
+### Empty Text
+
+| Input | Behaviour |
+|-------|-----------|
+| Empty candidate (`""`) | Metrics return zero scores; validators fail unless explicitly configured |
+| Empty reference (`""`) | Comparison metrics raise `ValueError` |
+| Whitespace-only text | Treated as empty after tokenisation |
+
+### None Reference
+
+| Component | Behaviour |
+|-----------|-----------|
+| Comparison metrics (BLEU, ROUGE, semantic) | Raise `ValueError("Reference required for {metric_name}")` |
+| Standalone metrics (readability) | Ignore, compute normally |
+| Validators wrapping comparison metrics | Raise `ValidationError` if `context.reference` is `None` |
+
+### Unicode & Encoding
+
+- All text assumed to be valid UTF-8 strings
+- Normalisation: NFC by default (configurable in `Tokeniser`)
+- Emoji and non-Latin scripts: Supported, tokenised as words where applicable
+
+### Very Long Text
+
+- No hard limits enforced by default
+- `Tokeniser` can accept `max_tokens: int | None` for truncation
+- Semantic similarity: Truncates to model's max sequence length (typically 512 tokens) with warning logged
+
+### Multiple References
+
+BLEU and ROUGE support multiple references (`list[str]`):
+- BLEU: Computes against each reference, uses maximum n-gram matches
+- ROUGE: Computes against each reference, returns best score
+
+---
+
+## Validator Naming Convention
+
+Consistent short names:
+
+```python
+from veritext import validators as v
+
+# Metric-based validators
+v.bleu(min_score=0.7)                    # BLEU-4 by default
+v.bleu(min_score=0.7, variant=1)         # BLEU-1
+v.rouge(min_score=0.7)                   # ROUGE-L by default
+v.rouge(min_score=0.7, variant="1")      # ROUGE-1
+v.semantic(min_score=0.8)                # Semantic similarity
+
+# Constraint validators
+v.length(max_chars=500)
+v.length(min_chars=100, max_chars=500)
+v.readability(max_grade=8)
+v.contains(terms=["hello", "world"])
+v.excludes(terms=["error", "fail"])
+v.pattern(regex=r"^\d{4}-\d{2}-\d{2}$")
+
+# Composition
+v.all_of([...])                          # All must pass
+v.any_of([...])                          # At least one must pass
+v.weighted(                              # Weighted score threshold
+    checks=[
+        (v.bleu(min_score=0.7), 0.6),    # (check, weight) tuples
+        (v.readability(max_grade=8), 0.4),
+    ],
+    min_score=0.75,                      # Minimum weighted score to pass
+)
+```
+
+---
+
+## Implementation Phases
+
+### Phase 1: Project Scaffold & Core
+
+**Goal:** Set up project structure with shared types and tokenisation.
+
+**Tasks:**
+1. Create directory structure
+2. Write `pyproject.toml` with optional dependencies
+3. Create `CLAUDE.md` with project guidelines
+4. Implement `core/exceptions.py` (full hierarchy)
+5. Implement `core/types.py` (`ValidationContext`, `CheckResult`, `ValidationResult`)
+6. Implement `core/tokenisation.py` (`WordTokeniser` with NFC normalisation)
+7. Implement `core/config.py` (pydantic-settings)
+8. Implement `core/logging.py` (structlog configuration)
+9. Create `__init__.py` with `__version__` and `__all__` exports
+10. Write tests for tokenisation (including Unicode, empty input, whitespace-only)
+11. Write tests for types (including edge cases)
+12. Initial commit to Gitea
+
+**Files:**
+- `pyproject.toml`
+- `CLAUDE.md`
+- `readme.md` (stub)
+- `changelog.md`
+- `src/veritext/__init__.py`
+- `src/veritext/py.typed`
+- `src/veritext/core/__init__.py`
+- `src/veritext/core/exceptions.py`
+- `src/veritext/core/types.py`
+- `src/veritext/core/tokenisation.py`
+- `src/veritext/core/config.py`
+- `src/veritext/core/logging.py`
+- `tests/conftest.py`
+- `tests/test_core/test_tokenisation.py`
+- `tests/test_core/test_types.py`
+
+**Verification:**
+```bash
+uv sync
+uv run ruff check .
+uv run ruff format --check .
+uv run mypy src/
+uv run pytest tests/test_core/ -v
+```
+
+---
+
+### Phase 2: Metrics — BLEU & Lexical
+
+**Goal:** Implement BLEU and lexical similarity metrics.
+
+**Tasks:**
+1. Implement `metrics/base.py` (Metric protocol, `BatchResult`, `AggregateStats`)
+2. Implement `metrics/results.py` (`BleuResult`, `LexicalResult`)
+3. Implement `metrics/bleu.py` (BLEU-1 through BLEU-4)
+4. Implement `metrics/lexical.py` (Jaccard, token overlap)
+5. Add batch processing with aggregate statistics (mean, std, percentiles)
+6. Write comprehensive tests:
+   - Single-pair scoring with reference values from NLTK
+   - Batch scoring with statistical aggregation
+   - Edge cases: empty text, single-word inputs, identical texts
+   - Multiple references support
+7. Define `__all__` exports in each module's `__init__.py`
+8. Update changelog
+
+**Key Design:**
+```python
+class Bleu:
+    def __init__(self, tokeniser: Tokeniser | None = None, max_n: int = 4): ...
+
+    def score(self, candidate: str, reference: str | list[str]) -> BleuResult: ...
+```
+
+**Files:**
+- `src/veritext/metrics/__init__.py`
+- `src/veritext/metrics/base.py`
+- `src/veritext/metrics/results.py`
+- `src/veritext/metrics/bleu.py`
+- `src/veritext/metrics/lexical.py`
+- `tests/test_metrics/test_bleu.py`
+- `tests/test_metrics/test_lexical.py`
+
+**Verification:**
+```bash
+uv run pytest tests/test_metrics/ -v --cov=src/veritext/metrics
+# Verify BLEU matches nltk.translate.bleu_score reference
+```
+
+---
+
+### Phase 3: Metrics — ROUGE & Readability
+
+**Goal:** Implement ROUGE and readability metrics.
+
+**Tasks:**
+1. Implement `metrics/rouge.py` (ROUGE-1, ROUGE-2, ROUGE-L with precision/recall/F1)
+2. Implement `metrics/readability.py` (Flesch-Kincaid grade level)
+   - Set `requires_reference = False` for standalone operation
+3. Add `RougeResult`, `RougeScore`, `ReadabilityResult` to results.py
+4. Write comprehensive tests:
+   - Single-pair scoring with reference values from `rouge-score` library
+   - Batch scoring with statistical aggregation
+   - Edge cases: empty text, very short text, identical texts
+   - Readability on various grade levels (children's text → academic)
+5. Update changelog
+
+**Files:**
+- `src/veritext/metrics/rouge.py`
+- `src/veritext/metrics/readability.py`
+- `tests/test_metrics/test_rouge.py`
+- `tests/test_metrics/test_readability.py`
+
+**Verification:**
+```bash
+uv run pytest tests/test_metrics/ -v
+# Verify ROUGE matches rouge-score library reference
+```
+
+---
+
+### Phase 4: Validators
+
+**Goal:** Build composable validation system.
+
+**Tasks:**
+1. Implement `validators/base.py` (`Check` protocol, `ValidationResult`)
+2. Implement `validators/metric.py` (`BleuValidator`, `RougeValidator`)
+   - Raise `ValidationError` if `context.reference` is `None`
+3. Implement `validators/constraint.py` (`LengthValidator`, `ContainsValidator`, etc.)
+4. Implement `validators/composite.py` (`AllOf`, `AnyOf`, `Weighted`)
+5. Create validator factory functions (`v.bleu()`, `v.length()`, etc.)
+6. Define `__all__` exports in `validators/__init__.py`
+7. Write comprehensive tests:
+   - Individual validators with passing/failing cases
+   - Composition (all_of, any_of, weighted)
+   - Edge cases: missing reference, empty text, boundary thresholds
+8. Update changelog
+
+**Key Design:**
+```python
+# validators/metric.py
+class BleuValidator:
+    def __init__(
+        self,
+        min_score: float,
+        variant: int = 4,
+        tokeniser: Tokeniser | None = None,
+    ): ...
+
+    def check(self, text: str, context: ValidationContext) -> CheckResult: ...
+
+# validators/__init__.py (factory functions)
+def bleu(min_score: float, variant: int = 4) -> BleuValidator: ...
+def rouge(min_score: float, variant: str = "l") -> RougeValidator: ...
+def length(min_chars: int | None = None, max_chars: int | None = None) -> LengthValidator: ...
+```
+
+**Files:**
+- `src/veritext/validators/__init__.py`
+- `src/veritext/validators/base.py`
+- `src/veritext/validators/metric.py`
+- `src/veritext/validators/constraint.py`
+- `src/veritext/validators/composite.py`
+- `tests/test_validators/test_metric_validators.py`
+- `tests/test_validators/test_constraint_validators.py`
+- `tests/test_validators/test_composite.py`
+
+**Verification:**
+```bash
+uv run pytest tests/test_validators/ -v --cov=src/veritext/validators
+```
+
+---
+
+### Phase 5: Semantic Similarity (Optional Dependency)
+
+**Goal:** Add embedding-based semantic similarity as optional feature.
+
+**Tasks:**
+1. Implement `semantic/similarity.py` with lazy import
+2. Add embedding caching for repeated texts
+3. Add `DependencyError` for missing sentence-transformers
+4. Add `SemanticResult` to `metrics/results.py`
+5. Add `SemanticValidator` to `validators/metric.py` (extends existing file)
+6. Add `v.semantic()` factory function to `validators/__init__.py`
+7. Write tests (skipped if dependency missing via `pytest.importorskip`)
+8. Update changelog
+
+**Key Design:**
+```python
+# semantic/similarity.py
+class SemanticSimilarity:
+    def __init__(
+        self,
+        model: str = "all-MiniLM-L6-v2",
+        cache_embeddings: bool = True,
+    ):
+        try:
+            from sentence_transformers import SentenceTransformer
+        except ImportError:
+            raise DependencyError(
+                "Install veritext[semantic] for semantic similarity: "
+                "pip install veritext[semantic]"
+            )
+        self._model = SentenceTransformer(model)
+        self._cache: dict[str, Any] = {} if cache_embeddings else None
+```
+
+**Files:**
+- `src/veritext/semantic/__init__.py`
+- `src/veritext/semantic/similarity.py`
+- `src/veritext/metrics/results.py` (add `SemanticResult`)
+- `src/veritext/validators/metric.py` (add `SemanticValidator`)
+- `src/veritext/validators/__init__.py` (add `semantic()` factory)
+- `tests/test_semantic/test_similarity.py`
+
+**Verification:**
+```bash
+# Without semantic dependency — tests should skip gracefully
+uv run pytest tests/ -v
+
+# With semantic dependency
+uv sync --extra semantic
+uv run pytest tests/test_semantic/ -v
+```
+
+---
+
+### Phase 6: Pytest Plugin
+
+**Goal:** Native pytest integration for CI/CD.
+
+**Tasks:**
+1. Create plugin structure with entry points
+2. Implement fixtures: `text_validator`
+3. Implement `validate_text()` assertion function
+4. Create detailed failure formatting
+5. Add `@pytest.mark.text_validation` marker
+6. Write integration tests
+7. Update changelog
+
+**Entry point:**
+```toml
+[project.entry-points.pytest11]
+veritext = "veritext.pytest_plugin"
+```
+
+**Key Design:**
+```python
+# pytest_plugin/assertions.py
+def validate_text(
+    text: str,
+    *,
+    reference: str | None = None,
+    min_bleu: float | None = None,
+    min_rouge: float | None = None,
+    min_semantic: float | None = None,
+    max_length: int | None = None,
+    max_reading_grade: float | None = None,
+    contains: list[str] | None = None,
+    excludes: list[str] | None = None,
+) -> None:
+    """
+    Assert text passes all specified validation criteria.
+
+    Raises:
+        AssertionError: With detailed failure information if validation fails.
+        ValueError: If comparison metrics requested but reference not provided.
+    """
+```
+
+**Error handling:** If `min_bleu`, `min_rouge`, or `min_semantic` is specified without a `reference`, raise `ValueError` immediately with a clear message rather than failing inside the metric.
+
+**Files:**
+- `src/veritext/pytest_plugin/__init__.py`
+- `src/veritext/pytest_plugin/fixtures.py`
+- `src/veritext/pytest_plugin/assertions.py`
+- `src/veritext/pytest_plugin/plugin.py`
+- `tests/test_pytest_plugin/test_integration.py`
+
+**Verification:**
+```bash
+uv pip install -e .
+uv run pytest --co -q  # Should show veritext plugin
+uv run pytest tests/test_pytest_plugin/ -v
+```
+
+---
+
+### Phase 7: Benchmark & Regression
+
+**Goal:** Track quality over time, detect regressions.
+
+**Tasks:**
+1. Implement `benchmark/models.py` (`BenchmarkRun`, `RegressionReport`)
+2. Implement `benchmark/storage.py` (SQLite backend)
+   - Handle concurrent writes gracefully (SQLite WAL mode)
+   - Raise `StorageError` on corruption with recovery guidance
+3. Implement `benchmark/runner.py` (`Benchmark` class)
+4. Implement `benchmark/regression.py` (statistical detection using rolling window)
+5. Add `assert_no_regression()` for CI integration
+6. Write comprehensive tests:
+   - Storage CRUD operations
+   - Regression detection with known degradation
+   - Edge cases: first run (no baseline), empty metrics
+7. Update changelog
+
+**Key Interface:**
+```python
+class Benchmark:
+    def __init__(self, name: str, storage_path: str | Path = "benchmarks/"): ...
+
+    def evaluate(
+        self,
+        candidates: list[str],
+        references: list[str],
+        metrics: list[str] | None = None,  # Default: ["rouge_l", "bleu4"]
+    ) -> BenchmarkRun:
+        """Evaluate candidates, store results, return the run record."""
+        ...
+
+    def check_regression(
+        self,
+        tolerance: float = 0.05,
+        window: int = 10,
+    ) -> RegressionReport:
+        """Compare current run against historical baseline."""
+        ...
+
+    def assert_no_regression(self, tolerance: float = 0.05) -> None:
+        """Raise RegressionDetectedError if quality dropped."""
+        ...
+```
+
+**SQLite Schema:**
+```sql
+CREATE TABLE benchmark_runs (
+    id TEXT PRIMARY KEY,
+    benchmark_name TEXT NOT NULL,
+    timestamp TEXT NOT NULL,
+    veritext_version TEXT NOT NULL,
+    sample_count INTEGER NOT NULL,
+    metadata TEXT  -- JSON
+);
+
+CREATE TABLE benchmark_metrics (
+    run_id TEXT REFERENCES benchmark_runs(id),
+    metric_name TEXT NOT NULL,
+    value REAL NOT NULL,
+    PRIMARY KEY (run_id, metric_name)
+);
+
+CREATE INDEX idx_benchmark_name ON benchmark_runs(benchmark_name, timestamp);
+```
+
+**Files:**
+- `src/veritext/benchmark/__init__.py`
+- `src/veritext/benchmark/models.py`
+- `src/veritext/benchmark/storage.py`
+- `src/veritext/benchmark/runner.py`
+- `src/veritext/benchmark/regression.py`
+- `tests/test_benchmark/test_storage.py`
+- `tests/test_benchmark/test_runner.py`
+- `tests/test_benchmark/test_regression.py`
+
+**Verification:**
+```bash
+uv run pytest tests/test_benchmark/ -v --cov=src/veritext/benchmark
+```
+
+---
+
+### Phase 8: CLI
+
+**Goal:** Command-line interface for validation and benchmarking.
+
+**Tasks:**
+1. Implement Typer CLI app
+2. Add `validate` command
+3. Add `benchmark run` command
+4. Add `benchmark show` command
+5. Add rich output formatting
+6. Write CLI tests
+7. Update changelog
+
+**Commands:**
+```bash
+veritext validate "text" --reference "ref" --metrics bleu,rouge
+veritext validate --file outputs.jsonl --reference-file refs.jsonl
+veritext benchmark run my_benchmark --inputs data/ --references refs/
+veritext benchmark show my_benchmark --last 20
+veritext benchmark check my_benchmark --tolerance 0.05
+```
+
+**Input Formats:**
+- **JSONL:** One JSON object per line with `candidate` and `reference` fields:
+  ```json
+  {"candidate": "The cat sat on the mat.", "reference": "A cat is sitting on a mat."}
+  {"candidate": "Hello world.", "reference": "Greetings, world."}
+  ```
+- **Directories:** Matching filenames in `--inputs` and `--references` directories:
+  ```
+  data/sample1.txt ↔ refs/sample1.txt
+  data/sample2.txt ↔ refs/sample2.txt
+  ```
+
+**Files:**
+- `src/veritext/cli/__init__.py`
+- `src/veritext/cli/main.py`
+- `tests/test_cli/test_commands.py`
+
+**Verification:**
+```bash
+uv run veritext --help
+uv run veritext validate "hello world" --reference "hello world" --metrics bleu
+uv run pytest tests/test_cli/ -v
+```
+
+---
+
+### Phase 9: Documentation & Polish
+
+**Goal:** Make portfolio-ready.
+
+**Tasks:**
+1. Write comprehensive `readme.md` with examples
+2. Add docstrings to all public APIs
+3. Create example scripts
+4. Ensure ≥80% test coverage
+5. Final linting/type checking
+6. Update `changelog.md` with 0.1.0 release
+7. Update project docs in `docs/`
+
+**Files:**
+- `readme.md` (comprehensive)
+- `examples/basic_validation.py`
+- `examples/chatbot_testing.py`
+- `examples/benchmark_regression.py`
+- Update all docstrings
+- `docs/project-plan.md` (update)
+- `docs/implementation-plan.md` (update)
+
+**Verification:**
+```bash
+uv run ruff check .
+uv run ruff format --check .
+uv run mypy src/
+uv run pytest --cov=src/veritext --cov-report=term-missing
+# Verify ≥80% coverage
+```
+
+---
+
+## Dependencies
+
+```toml
+[project]
+name = "veritext"
+version = "0.1.0"
+description = "Semantic text validation framework"
+readme = "readme.md"
+requires-python = ">=3.11"
+dependencies = [
+    "pydantic>=2.0",
+    "pydantic-settings>=2.0",
+    "structlog>=23.0",
+    "typer>=0.9",
+    "rich>=13.0",
+]
+
+[project.optional-dependencies]
+semantic = ["sentence-transformers>=2.2"]
+dev = [
+    "pytest>=7.0",
+    "pytest-cov>=4.0",
+    "mypy>=1.0",
+    "ruff>=0.1",
+]
+all = ["veritext[semantic]"]
+
+[project.scripts]
+veritext = "veritext.cli.main:app"
+
+[project.entry-points.pytest11]
+veritext = "veritext.pytest_plugin"
+```
+
+---
+
+## Conventions
+
+### Code Quality
+- `ruff check .` — zero issues
+- `ruff format --check .` — zero changes
+- `mypy src/` — passes (strict mode)
+- `pytest --cov=src/veritext` — ≥80% coverage
+
+### Git
+- **Author:** Kai Chappell <git@kschappell.com>
+- **Signed commits:** GPG key 219AA60F0638489B
+- **Format:** `type(scope): description`
+- **Atomic:** ≤3 files, ≤150 LOC per commit
+- **No AI/LLM attribution**
+
+### Python
+- Python 3.11+ with modern type hints
+- Absolute imports from package root
+- structlog for logging
+- UK English (colour, behaviour, summarisation)
+
+---
+
+## Verification Checklist (Per Phase)
+
+```bash
+cd /home/kai/work/dev/portfolio/veritext
+
+# Code quality
+uv run ruff check .
+uv run ruff format --check .
+uv run mypy src/
+
+# Tests
+uv run pytest --cov=src/veritext --cov-report=term-missing
+
+# Package installation
+uv pip install -e .
+uv run python -c "import veritext; print(veritext.__version__)"
+```

docs/project-plan.md

@@ -0,0 +1,490 @@
+# Project Plan: Veritext — Semantic Text Validation Framework
+
+## Overview
+
+A Python library for validating text outputs against semantic criteria. Designed for
+developers building any system that produces text — chatbots, content generators,
+translation pipelines, summarisation tools — who need automated quality assurance
+beyond simple string matching.
+
+**Origin story:** "I was building a feature that generated article summaries and got
+tired of manually checking if they captured the key points. Existing tools could tell
+me if two strings matched, but not if they *meant* the same thing. So I built a
+validation framework that understands semantics."
+
+**Portfolio role:** A practical developer tool that demonstrates Python framework
+design, NLP evaluation techniques, and test automation integration. The project
+solves a real problem any developer working with text processing encounters.
+
+**Target users:** Developers building content pipelines, chatbot teams validating
+responses, ML engineers evaluating model outputs, QA teams testing text-based features.
+
+---
+
+## Problem Statement
+
+Text validation is hard. Traditional testing approaches fall short:
+
+| Approach | Problem |
+|----------|---------|
+| Exact string match | Fails on semantically equivalent variations |
+| Substring/regex | Brittle, misses meaning entirely |
+| Manual review | Doesn't scale, subjective |
+| Generic diff tools | Show *what* changed, not *if it matters* |
+
+**Example:** A summarisation system produces "The CEO announced layoffs affecting 500
+employees" one day and "500 workers will lose their jobs, the company's chief executive
+said" the next. These are semantically equivalent, but every traditional test would
+flag this as a failure.
+
+Veritext answers: "Is this text output *good enough* according to my criteria?" — not
+"Is it identical?"
+
+---
+
+## Core Concepts
+
+### Metrics (Pure Computation)
+
+Metrics compute scores comparing candidate text to reference text:
+
+```python
+from veritext.metrics import Bleu, Rouge
+
+bleu = Bleu()
+result = bleu.score(
+    candidate="The cat sat on the mat",
+    reference="A cat is sitting on a mat"
+)
+# BleuResult(bleu1=0.71, bleu2=0.58, bleu3=0.45, bleu4=0.41, brevity_penalty=1.0)
+
+rouge = Rouge()
+result = rouge.score(candidate, reference)
+# RougeResult(rouge1=RougeScore(...), rouge2=RougeScore(...), rouge_l=RougeScore(...))
+```
+
+**Built-in metrics:**
+
+| Metric | What it measures | Use case |
+|--------|------------------|----------|
+| BLEU-1 to BLEU-4 | N-gram precision | Translation, generation |
+| ROUGE-1, ROUGE-2 | N-gram recall | Summarisation |
+| ROUGE-L | Longest common subsequence | Summarisation |
+| Semantic similarity | Cosine distance of embeddings | Any meaning comparison |
+| Lexical overlap | Jaccard similarity of tokens | Simple similarity |
+| Reading level | Flesch-Kincaid grade | Accessibility |
+
+**Note:** Reading level is a standalone metric (`requires_reference = False`) that analyses only the candidate text. Comparison metrics (BLEU, ROUGE, semantic) require a reference and raise `ValueError` if none provided.
+
+### Validators (Decision Logic)
+
+Validators wrap metrics and apply thresholds to make pass/fail decisions:
+
+```python
+from veritext import validators as v
+
+# Compose multiple checks
+validator = v.all_of([
+    v.bleu(min_score=0.7),
+    v.length(max_chars=500),
+    v.readability(max_grade=8),
+])
+
+from veritext.core.types import ValidationContext
+
+context = ValidationContext(reference="The quick brown fox jumps over the lazy dog")
+result = validator.validate("The fast brown fox leaped over the lazy dog", context)
+# ValidationResult(passed=True, checks=[...])
+```
+
+### Pytest Integration
+
+Native pytest fixtures and assertions for CI/CD:
+
+```python
+from veritext.pytest_plugin import validate_text
+
+def test_summary_quality(summariser, document):
+    summary = summariser.summarise(document)
+
+    validate_text(
+        summary,
+        reference=expected_summary,
+        min_rouge=0.7,
+        min_semantic=0.85,
+    )
+```
+
+### Regression Detection
+
+Track output quality over time, catch degradations before users do:
+
+```python
+from veritext.benchmark import Benchmark
+
+benchmark = Benchmark("summarisation_quality", storage_path="benchmarks/")
+results = benchmark.evaluate(outputs, references, metrics=["rouge_l", "bleu4"])
+benchmark.assert_no_regression(tolerance=0.05)
+```
+
+---
+
+## Tech Stack
+
+| Component | Technology | Rationale |
+|-----------|------------|-----------|
+| Core | Python 3.11+ | Target ecosystem, modern type hints |
+| Metrics | Custom implementations | Full control, understanding of algorithms |
+| Embeddings | sentence-transformers | Semantic similarity (optional) |
+| Test integration | pytest | Fixtures, plugins, assertions |
+| CLI | typer | Consistent with portfolio projects |
+| Data handling | pydantic | Validation, serialisation |
+| Storage | SQLite | Benchmark history, lightweight |
+| Output | rich | Terminal formatting |
+
+---
+
+## Architecture
+
+### Layered Design
+
+```
+┌─────────────────────────────────────────────────────┐
+│  CLI / pytest_plugin  (presentation layer)          │
+├─────────────────────────────────────────────────────┤
+│  validators/          (decision logic)              │
+│  benchmark/           (tracking & regression)       │
+├─────────────────────────────────────────────────────┤
+│  metrics/             (pure computation)            │
+├─────────────────────────────────────────────────────┤
+│  core/                (shared types, tokenisation)  │
+└─────────────────────────────────────────────────────┘
+```
+
+**Dependency rule:** Each layer depends only on layers below it.
+
+### Key Design Decisions
+
+1. **Metrics vs Validators separation** — Metrics compute scores; validators make
+   pass/fail decisions. Clear separation of concerns.
+
+2. **Typed result objects** — Each metric returns a specific result type (e.g.,
+   `BleuResult`, `RougeResult`), not just `float`. Full information preserved.
+
+3. **Optional heavy dependencies** — `sentence-transformers` (~2GB with PyTorch) is
+   optional. Core library works without ML dependencies.
+
+4. **Shared tokenisation** — Single `Tokeniser` protocol used by all n-gram metrics.
+   Consistent behaviour across BLEU and ROUGE.
+
+5. **Explicit context** — `ValidationContext` dataclass instead of `**kwargs`.
+   Type-safe, discoverable API.
+
+6. **Graceful edge case handling** — Empty text returns zero scores (not errors).
+   Missing reference raises clear `ValueError` for comparison metrics. Unicode
+   normalised to NFC by default.
+
+---
+
+## Project Components
+
+### Component 1: Core Module
+
+Shared types, exceptions, and tokenisation.
+
+**Types:**
+- `ValidationContext` — reference text and metadata for validation
+- `CheckResult` — individual check result with diagnostics
+- `ValidationResult` — aggregate result with pass/fail and all checks
+- `BatchResult` — statistics over multiple evaluations
+
+**Tokeniser:**
+```python
+class Tokeniser(Protocol):
+    def tokenise(self, text: str) -> list[str]: ...
+
+class WordTokeniser:
+    def __init__(self, lowercase: bool = True, remove_punctuation: bool = True): ...
+```
+
+---
+
+### Component 2: Metric Engine
+
+Pure implementations of text evaluation metrics.
+
+**Interface:**
+```python
+class Metric(Protocol[T]):
+    @property
+    def name(self) -> str: ...
+
+    @property
+    def requires_reference(self) -> bool: ...
+
+    def score(self, candidate: str, reference: str | list[str] | None = None) -> T:
+        """Raises ValueError if reference required but not provided."""
+        ...
+
+    def batch_score(
+        self,
+        candidates: list[str],
+        references: list[str] | list[list[str]] | None = None,
+    ) -> BatchResult[T]: ...
+```
+
+**Metrics:**
+- `Bleu` — BLEU-1 through BLEU-4 with brevity penalty
+- `Rouge` — ROUGE-1, ROUGE-2, ROUGE-L with precision/recall/F1
+- `Lexical` — Jaccard similarity, token overlap
+- `Readability` — Flesch-Kincaid grade level
+- `SemanticSimilarity` — Embedding cosine distance (optional dependency)
+
+---
+
+### Component 3: Validator Framework
+
+Composable validation rules with clear pass/fail semantics.
+
+**Built-in validators:**
+
+| Validator | Description |
+|-----------|-------------|
+| `v.bleu(min_score, variant)` | BLEU score above minimum |
+| `v.rouge(min_score, variant)` | ROUGE score above minimum |
+| `v.semantic(min_score)` | Semantic similarity above threshold |
+| `v.length(min_chars, max_chars)` | Length constraints |
+| `v.readability(max_grade)` | Reading level constraint |
+| `v.contains(terms)` | Required terms present |
+| `v.excludes(terms)` | Forbidden terms absent |
+| `v.pattern(regex)` | Regex pattern match |
+
+**Composition:**
+
+```python
+# All validators must pass
+v.all_of([v.bleu(min_score=0.7), v.length(max_chars=500)])
+
+# At least one must pass
+v.any_of([v.contains(["error"]), v.contains(["failed"])])
+
+# Weighted scoring
+v.weighted([
+    (v.bleu(min_score=0.7), 0.6),
+    (v.readability(max_grade=8), 0.4),
+], min_score=0.75)
+```
+
+**Reference requirements:** Validators wrapping comparison metrics (`bleu`, `rouge`, `semantic`) require `context.reference` to be set. If `None`, they raise `ValidationError` with a clear message. Constraint validators (`length`, `readability`, `contains`) do not require a reference.
+
+---
+
+### Component 4: Pytest Plugin
+
+First-class pytest integration for CI/CD pipelines.
+
+**Features:**
+- Custom assertions with detailed failure messages
+- Fixtures for common validation patterns
+- Markers for categorising text tests
+
+**Usage:**
+
+```python
+from veritext.pytest_plugin import validate_text
+
+def test_chatbot_response():
+    response = chatbot.respond("What are your hours?")
+
+    validate_text(
+        response,
+        reference="We're open Monday to Friday, 9am to 5pm.",
+        min_bleu=0.6,
+        min_semantic=0.8,
+        max_length=500,
+    )
+```
+
+**Failure output:**
+
+```
+FAILED test_summary.py::test_summary_quality
+    AssertionError: Text failed 2 of 4 checks:
+
+    ✗ rouge: 0.58 (minimum: 0.70)
+    ✗ semantic: 0.72 (minimum: 0.85)
+    ✓ length: 342 (maximum: 500)
+    ✓ readability: 6.2 (maximum: 8)
+
+    Candidate: "The company reported losses..."
+    Reference: "Financial results showed significant decline..."
+```
+
+---
+
+### Component 5: Benchmark & Regression Detection
+
+Track quality over time, catch degradations automatically.
+
+**Features:**
+- Store historical metric values in SQLite
+- Statistical regression detection
+- Configurable tolerance thresholds
+- CI integration for blocking degradations
+
+**Usage:**
+
+```python
+from veritext.benchmark import Benchmark
+
+benchmark = Benchmark("chatbot_quality", storage_path="benchmarks/")
+
+# Record current run (returns BenchmarkRun with metrics and metadata)
+run = benchmark.evaluate(
+    candidates=current_outputs,
+    references=expected_outputs,
+    metrics=["rouge_l", "semantic"]
+)
+# run.metrics = {"rouge_l": 0.82, "semantic": 0.89}
+
+# Compare against historical baseline
+regression = benchmark.check_regression(tolerance=0.05, window=10)
+
+if regression.detected:
+    print(f"Quality dropped: {regression.summary}")
+
+# In CI: fail the build on regression
+benchmark.assert_no_regression(tolerance=0.05)
+```
+
+---
+
+### Component 6: CLI Tool
+
+Command-line interface for quick validation and benchmarking.
+
+```bash
+# Validate a single text
+$ veritext validate "Your text here" --reference "Expected text" --metrics bleu,rouge
+
+# Validate from files
+$ veritext validate --file outputs.jsonl --reference-file expected.jsonl
+
+# Run benchmark
+$ veritext benchmark run summarisation --inputs docs/ --references refs/
+
+# Show benchmark history
+$ veritext benchmark show summarisation --last 20
+
+# Check for regression
+$ veritext benchmark check summarisation --tolerance 0.05
+```
+
+---
+
+## Example Use Cases
+
+### Use Case 1: Chatbot Response Validation
+
+```python
+from veritext import validators as v
+from veritext.core.types import ValidationContext
+
+# Define acceptable response criteria
+response_validator = v.all_of([
+    v.length(max_chars=500),
+    v.readability(max_grade=8),
+    v.excludes(terms=["I don't know", "I'm not sure"]),
+])
+
+def test_chatbot_responds_helpfully():
+    response = chatbot.respond("How do I reset my password?")
+    context = ValidationContext()
+    result = response_validator.validate(response, context)
+    assert result.passed, result.failure_summary
+```
+
+### Use Case 2: Summarisation Quality Gate
+
+```python
+from veritext.pytest_plugin import validate_text
+
+def test_summary_captures_key_points():
+    article = load_article("financial_report.txt")
+    summary = summariser.summarise(article)
+
+    validate_text(
+        summary,
+        reference=load_reference_summary("financial_report_summary.txt"),
+        min_rouge=0.65,
+        min_semantic=0.80,
+        max_length=300,
+    )
+```
+
+### Use Case 3: Translation Quality Monitoring
+
+```python
+from veritext.benchmark import Benchmark
+
+benchmark = Benchmark("translation_en_de", storage_path="benchmarks/")
+
+# Nightly CI job
+results = benchmark.evaluate(
+    candidates=translate_batch(test_documents),
+    references=human_translations,
+    metrics=["bleu4", "semantic"]
+)
+
+# Block deployment if quality drops
+benchmark.assert_no_regression(tolerance=0.03)
+```
+
+---
+
+## Success Criteria
+
+- [ ] BLEU/ROUGE implementations match reference implementations (nltk, rouge-score)
+- [ ] Semantic similarity correlates with human judgement on test pairs
+- [ ] Pytest plugin installs cleanly via `uv pip install veritext`
+- [ ] Validation of 1000 text pairs completes in <5 seconds (excluding embeddings)
+- [ ] Benchmark regression detection has <5% false positive rate
+- [ ] Edge cases handled gracefully (empty text, None reference, Unicode)
+- [ ] Documentation includes working examples for each use case
+- [ ] All code passes ruff, mypy strict, and pytest with ≥80% coverage
+- [ ] Can explain design decisions and metric theory in interview
+
+---
+
+## Skills Demonstrated
+
+| Skill | How Veritext demonstrates it |
+|-------|------------------------------|
+| Python framework design | Composable validators, clean API, plugin architecture |
+| Test automation | Native pytest integration, CI/CD workflows |
+| NLP evaluation metrics | BLEU, ROUGE, semantic similarity implementations |
+| Data analysis | Statistical regression detection, batch processing |
+| CLI development | Typer-based interface, rich output |
+| Software architecture | Layered design, clear separation of concerns |
+| Documentation | Comprehensive readme, examples |
+| Quality engineering | High test coverage, type safety, linting |
+
+---
+
+## What Makes This Project Credible
+
+1. **Solves a real problem** — Anyone building text-based features faces validation
+   challenges.
+
+2. **Not tied to a specific technology** — Works with any text source (chatbots, LLMs,
+   translation APIs, content generators). It's a general-purpose tool, not an "LLM
+   testing framework."
+
+3. **Practical scope** — Not trying to reinvent pytest or build an ML platform. Focused
+   on one thing: validating text quality.
+
+4. **Demonstrates depth** — Implementing BLEU/ROUGE from understanding (not just
+   wrapping libraries) shows knowledge of how these metrics work.
+
+5. **Natural portfolio narrative** — "I was building X and needed a better way to test
+   it, so I built this tool." Every interviewer has faced similar problems.

examples/basic_validation.py

@@ -1,135 +0,0 @@
-"""Basic text validation examples.
-
-Demonstrates core Veritext functionality:
-- Single metric scoring (BLEU, ROUGE)
-- Validator usage with thresholds
-- Composite validators (all_of, any_of)
-- Constraint validators (length, readability)
-"""
-
-from veritext.core.types import ValidationContext
-from veritext.metrics import Bleu, Rouge
-from veritext.validators import (
-    all_of,
-    any_of,
-    bleu,
-    contains,
-    excludes,
-    length,
-    readability,
-    rouge,
-)
-
-
-def metric_scoring_example() -> None:
-    """Score text using individual metrics."""
-    candidate = "The quick brown fox jumps over the lazy dog."
-    reference = "A fast brown fox leaps over a sleepy dog."
-
-    # BLEU scoring (translation quality)
-    bleu_metric = Bleu()
-    bleu_result = bleu_metric.score(candidate, reference)
-    print("BLEU Scores:")
-    print(f"  BLEU-1: {bleu_result.bleu1:.3f}")
-    print(f"  BLEU-4: {bleu_result.bleu4:.3f}")
-    print(f"  Brevity penalty: {bleu_result.brevity_penalty:.3f}")
-
-    # ROUGE scoring (summary quality)
-    rouge_metric = Rouge()
-    rouge_result = rouge_metric.score(candidate, reference)
-    print("\nROUGE Scores:")
-    print(f"  ROUGE-1 F1: {rouge_result.rouge1.fmeasure:.3f}")
-    print(f"  ROUGE-L F1: {rouge_result.rouge_l.fmeasure:.3f}")
-
-
-def validator_example() -> None:
-    """Use validators to make pass/fail decisions."""
-    reference = "Machine learning models require training data."
-    candidate = "ML models need training data to learn patterns."
-
-    context = ValidationContext(reference=reference)
-
-    # BLEU validator with minimum threshold
-    bleu_validator = bleu(min_score=0.3)
-    result = bleu_validator.check(candidate, context)
-    print(f"\nBLEU validation (min 0.3): {'PASS' if result.passed else 'FAIL'}")
-
-    # ROUGE validator
-    rouge_validator = rouge(min_score=0.5)
-    result = rouge_validator.check(candidate, context)
-    print(f"ROUGE validation (min 0.5): {'PASS' if result.passed else 'FAIL'}")
-
-
-def composite_validator_example() -> None:
-    """Combine validators with all_of and any_of."""
-    reference = "The product launch exceeded all expectations."
-    candidate = "The product release performed beyond expectations."
-
-    context = ValidationContext(reference=reference)
-
-    # All checks must pass
-    strict_validator = all_of(
-        [
-            bleu(min_score=0.2),
-            rouge(min_score=0.4),
-            length(max_chars=100),
-        ]
-    )
-    result = strict_validator.check(candidate, context)
-    print(f"\nStrict (all_of): {'PASS' if result.passed else 'FAIL'}")
-    if not result.passed:
-        print(f"  Failures: {result.failure_summary}")
-
-    # At least one check must pass
-    flexible_validator = any_of(
-        [
-            bleu(min_score=0.8),  # Unlikely to pass
-            rouge(min_score=0.4),  # More likely
-        ]
-    )
-    result = flexible_validator.check(candidate, context)
-    print(f"Flexible (any_of): {'PASS' if result.passed else 'FAIL'}")
-
-
-def constraint_validator_example() -> None:
-    """Use constraint validators for text properties."""
-    text = "This short guide explains the basics clearly."
-    context = ValidationContext()  # No reference needed for constraints
-
-    # Length constraints
-    length_validator = length(min_chars=20, max_chars=100, min_words=5, max_words=20)
-    result = length_validator.check(text, context)
-    print(f"\nLength check: {'PASS' if result.passed else 'FAIL'}")
-
-    # Readability (Flesch-Kincaid)
-    readability_validator = readability(max_grade=10.0)
-    result = readability_validator.check(text, context)
-    print(f"Readability (grade <= 10): {'PASS' if result.passed else 'FAIL'}")
-
-    # Content patterns
-    contains_validator = contains(patterns=["guide", "basics"])
-    result = contains_validator.check(text, context)
-    print(f"Contains required terms: {'PASS' if result.passed else 'FAIL'}")
-
-    excludes_validator = excludes(patterns=["error", "warning"])
-    result = excludes_validator.check(text, context)
-    print(f"Excludes forbidden terms: {'PASS' if result.passed else 'FAIL'}")
-
-
-def main() -> None:
-    """Run all examples."""
-    print("=" * 60)
-    print("Veritext Basic Validation Examples")
-    print("=" * 60)
-
-    metric_scoring_example()
-    validator_example()
-    composite_validator_example()
-    constraint_validator_example()
-
-    print("\n" + "=" * 60)
-    print("All examples completed.")
-
-
-if __name__ == "__main__":
-    main()

examples/benchmark_regression.py

@@ -1,160 +0,0 @@
-"""Benchmark quality tracking with regression detection.
-
-Demonstrates Veritext's benchmark module for CI integration:
-- Creating a benchmark suite
-- Running evaluations and storing results
-- Checking for quality regression
-- CI integration pattern with exit codes
-"""
-
-import tempfile
-from pathlib import Path
-
-from veritext.benchmark import Benchmark
-from veritext.core.exceptions import RegressionDetectedError
-
-
-def create_sample_data() -> tuple[list[str], list[str]]:
-    """Create sample candidate/reference pairs for benchmarking."""
-    # Simulated summarisation outputs and references
-    candidates = [
-        "The new policy aims to reduce carbon emissions by 50% by 2030.",
-        "Scientists discovered a new species of deep-sea fish.",
-        "The company reported record profits in the third quarter.",
-        "Researchers developed a breakthrough treatment for the disease.",
-        "The city plans to expand public transportation routes.",
-    ]
-    references = [
-        "The policy targets a 50% reduction in carbon emissions by 2030.",
-        "A new deep-sea fish species was discovered by marine biologists.",
-        "Record profits were announced by the company for Q3.",
-        "A breakthrough disease treatment was developed by researchers.",
-        "Public transport expansion is planned for the city.",
-    ]
-    return candidates, references
-
-
-def run_benchmark_example() -> None:
-    """Run a benchmark evaluation and view results."""
-    # Use a temp directory for this example
-    with tempfile.TemporaryDirectory() as tmpdir:
-        storage_path = Path(tmpdir) / "benchmarks"
-
-        # Create benchmark suite
-        bench = Benchmark("summariser_quality", storage_path=storage_path)
-
-        candidates, references = create_sample_data()
-
-        # Run evaluation
-        print("Running benchmark evaluation...")
-        run = bench.evaluate(
-            candidates=candidates,
-            references=references,
-            metrics=["rouge_l", "bleu4"],
-            metadata={"model": "v1.0", "dataset": "test"},
-        )
-
-        print("\nBenchmark run completed:")
-        print(f"  Run ID: {run.id[:8]}...")
-        print(f"  Samples: {run.sample_count}")
-        print("  Metrics:")
-        for name, value in run.metrics.items():
-            print(f"    {name}: {value:.4f}")
-
-
-def regression_detection_example() -> None:
-    """Demonstrate regression detection with historical comparison."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        storage_path = Path(tmpdir) / "benchmarks"
-        bench = Benchmark("summariser_quality", storage_path=storage_path)
-
-        candidates, references = create_sample_data()
-
-        # Simulate historical runs with stable quality
-        print("\nBuilding baseline with historical runs...")
-        for i in range(5):
-            bench.evaluate(
-                candidates=candidates,
-                references=references,
-                metrics=["rouge_l", "bleu4"],
-                metadata={"run": f"baseline_{i}"},
-            )
-            print(f"  Baseline run {i + 1} recorded")
-
-        # Check regression (no degradation expected)
-        report = bench.check_regression(tolerance=0.05, window=5)
-        print(f"\nRegression check: {'DETECTED' if report.detected else 'NONE'}")
-
-        # Simulate a degraded model
-        print("\nSimulating degraded model output...")
-        degraded_candidates = [
-            "Policy carbon emissions.",  # Much shorter/worse
-            "Fish discovered.",
-            "Company profits.",
-            "Treatment developed.",
-            "Transport expansion.",
-        ]
-        bench.evaluate(
-            candidates=degraded_candidates,
-            references=references,
-            metrics=["rouge_l", "bleu4"],
-            metadata={"model": "v1.1-broken"},
-        )
-
-        # Check regression (should detect)
-        report = bench.check_regression(tolerance=0.05, window=5)
-        print(f"Regression check: {'DETECTED' if report.detected else 'NONE'}")
-        if report.detected:
-            print("\nRegression details:")
-            for metric, delta in report.deltas.items():
-                baseline = report.baseline.get(metric, 0)
-                current = report.current.get(metric, 0)
-                print(f"  {metric}: {baseline:.4f} -> {current:.4f} ({delta:+.4f})")
-
-
-def ci_integration_example() -> None:
-    """CI integration pattern using assert_no_regression()."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        storage_path = Path(tmpdir) / "benchmarks"
-        bench = Benchmark("ci_check", storage_path=storage_path)
-
-        candidates, references = create_sample_data()
-
-        # Build baseline
-        for _ in range(3):
-            bench.evaluate(candidates, references, metrics=["rouge_l"])
-
-        # Simulate CI check
-        print("\n" + "=" * 50)
-        print("CI Integration Example")
-        print("=" * 50)
-
-        print("\nRunning evaluation...")
-        bench.evaluate(candidates, references, metrics=["rouge_l"])
-
-        print("Checking for regression...")
-        try:
-            bench.assert_no_regression(tolerance=0.05, window=3)
-            print("No regression detected.")
-            print("CI status: EXIT 0")
-        except RegressionDetectedError as e:
-            print(f"Regression detected: {e}")
-            print("CI status: EXIT 1")
-
-
-def main() -> None:
-    """Run all benchmark examples."""
-    print("=" * 60)
-    print("Veritext Benchmark & Regression Detection Examples")
-    print("=" * 60)
-
-    run_benchmark_example()
-    regression_detection_example()
-    ci_integration_example()
-
-    print("\n" + "=" * 60)
-    print("All examples completed.")
-
-
-if __name__ == "__main__":
-    main()

examples/chatbot_testing.py

@@ -1,135 +0,0 @@
-"""Pytest integration for chatbot testing.
-
-Demonstrates Veritext's pytest plugin for testing chatbot responses:
-- validate_text() assertion function
-- Custom test fixtures
-- Test organisation with markers
-"""
-
-import pytest
-
-from veritext.pytest_plugin import validate_text
-
-# Sample chatbot responses for testing
-CHATBOT_RESPONSES = {
-    "greeting": {
-        "input": "Hello!",
-        "response": "Hi there! How can I help you today?",
-        "expected_keywords": ["help", "hi"],
-    },
-    "weather": {
-        "input": "What's the weather like?",
-        "response": "I don't have access to real-time weather data, but you can "
-        "check a weather service like weather.com for current conditions.",
-        "expected_keywords": ["weather", "check"],
-    },
-    "farewell": {
-        "input": "Goodbye!",
-        "response": "Goodbye! Have a great day!",
-        "expected_keywords": ["goodbye", "day"],
-    },
-}
-
-
-# Fixtures for common test setup
-@pytest.fixture
-def greeting_response() -> str:
-    return CHATBOT_RESPONSES["greeting"]["response"]
-
-
-@pytest.fixture
-def weather_response() -> str:
-    return CHATBOT_RESPONSES["weather"]["response"]
-
-
-# Basic validation tests
-class TestResponseQuality:
-    """Test chatbot response quality using Veritext."""
-
-    def test_greeting_length(self, greeting_response: str) -> None:
-        validate_text(
-            greeting_response,
-            min_length=10,
-            max_length=100,
-        )
-
-    def test_greeting_readability(self, greeting_response: str) -> None:
-        validate_text(
-            greeting_response,
-            max_reading_grade=8.0,
-        )
-
-    def test_greeting_contains_keywords(self, greeting_response: str) -> None:
-        validate_text(
-            greeting_response,
-            must_contain=["help"],
-        )
-
-    def test_weather_response_quality(self, weather_response: str) -> None:
-        """Weather response should be informative and readable."""
-        validate_text(
-            weather_response,
-            min_length=50,
-            max_length=500,
-            max_reading_grade=10.0,
-            must_contain=["weather"],
-        )
-
-
-# Tests with reference comparison
-class TestResponseSimilarity:
-    """Test response similarity against reference texts."""
-
-    def test_greeting_similarity(self) -> None:
-        """Greeting should match expected style."""
-        reference = "Hello! How may I assist you today?"
-        response = CHATBOT_RESPONSES["greeting"]["response"]
-
-        validate_text(
-            response,
-            reference=reference,
-            min_rouge=0.3,  # Allow variation in wording
-            min_length=10,
-        )
-
-    def test_farewell_similarity(self) -> None:
-        """Farewell should match expected style."""
-        reference = "Goodbye! Have a wonderful day!"
-        response = CHATBOT_RESPONSES["farewell"]["response"]
-
-        validate_text(
-            response,
-            reference=reference,
-            min_rouge=0.5,
-            must_contain=["goodbye"],
-        )
-
-
-# Content safety tests
-class TestContentSafety:
-    """Test responses for inappropriate content."""
-
-    @pytest.mark.parametrize("response_key", ["greeting", "weather", "farewell"])
-    def test_no_profanity(self, response_key: str) -> None:
-        """Responses should not contain profanity."""
-        response = CHATBOT_RESPONSES[response_key]["response"]
-        validate_text(
-            response,
-            must_exclude=["damn", "hell", "crap"],
-            min_length=1,
-        )
-
-    @pytest.mark.parametrize("response_key", ["greeting", "weather", "farewell"])
-    def test_no_harmful_content(self, response_key: str) -> None:
-        """Responses should not contain harmful instructions."""
-        response = CHATBOT_RESPONSES[response_key]["response"]
-        validate_text(
-            response,
-            must_exclude=["hack", "exploit", "attack"],
-            min_length=1,
-        )
-
-
-# Run tests when executed directly
-if __name__ == "__main__":
-    pytest.main([__file__, "-v"])

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "veritext"
-version = "0.1.0.dev0"
+version = "0.1.0-dev"
 description = "Semantic text validation framework"
 readme = "readme.md"
 requires-python = ">=3.11"

readme.md

@@ -2,398 +2,48 @@
 
 Semantic text validation framework for Python.
 
-Veritext validates text outputs against quality criteria using metrics like BLEU,
+Validates text outputs against quality criteria using metrics like BLEU, ROUGE,
-ROUGE, and semantic similarity. Designed for developers building systems that produce
+and semantic similarity. Designed for developers building systems that produce
-text (chatbots, content generators, summarisation tools) who need automated quality
+text (chatbots, content generators, summarisation tools) who need automated
-assurance beyond simple string matching.
+quality assurance beyond simple string matching.
 
-## Features
+## Status
 
-- **Multiple metrics** — BLEU, ROUGE, lexical similarity, readability, semantic
+Under active development. See [changelog.md](changelog.md) for progress.
-  embeddings
-- **Composable validators** — Build complex checks from simple primitives
-- **Native pytest integration** — `validate_text()` assertion for test suites
-- **Quality benchmarking** — Track metrics over time with regression detection
-- **CLI tools** — Command-line validation and benchmark management
 
 ## Installation
 
 ```bash
 pip install veritext
 
-# With semantic similarity support (sentence-transformers)
+# With semantic similarity support
 pip install veritext[semantic]
 ```
 
 ## Quick Start
 
 ```python
+from veritext import validators as v
 from veritext.core.types import ValidationContext
-from veritext.validators import all_of, bleu, length, rouge
 
-# Create a validator
+# Create validators
-validator = all_of([
+validator = v.all_of([
-    bleu(min_score=0.5),
+    v.bleu(min_score=0.7),
-    rouge(min_score=0.6),
+    v.length(max_chars=500),
-    length(max_chars=500),
 ])
 
 # Validate text
-context = ValidationContext(reference="The quick brown fox jumps over the lazy dog.")
+context = ValidationContext(reference="The cat sat on the mat.")
-result = validator.check("A fast brown fox leaps over a sleepy dog.", context)
+result = validator.check("A cat is sitting on the mat.", context)
 
-if result.passed:
+if not result.passed:
-    print("Validation passed!")
-else:
     print(result.failure_summary)
 ```
 
-## Metrics
+## Documentation
 
-Veritext provides several metrics for text evaluation.
+- [Project Plan](docs/project-plan.md)
-
+- [Implementation Plan](docs/implementation-plan.md)
-### BLEU
-
-Measures n-gram precision against reference text. Useful for translation and
-generation quality.
-
-```python
-from veritext.metrics import Bleu
-
-bleu = Bleu()
-result = bleu.score(
-    candidate="The cat sat on the mat.",
-    reference="A cat is sitting on the mat.",
-)
-print(f"BLEU-4: {result.bleu4:.3f}")  # Uses 1-4 gram precision
-print(f"BLEU-1: {result.bleu1:.3f}")  # Unigram precision only
-```
-
-### ROUGE
-
-Measures recall-oriented overlap with reference text. Useful for summarisation.
-
-```python
-from veritext.metrics import Rouge
-
-rouge = Rouge()
-result = rouge.score(
-    candidate="Scientists found a new planet.",
-    reference="Researchers discovered a new planet in the solar system.",
-)
-print(f"ROUGE-1 F1: {result.rouge1.fmeasure:.3f}")  # Unigram overlap
-print(f"ROUGE-L F1: {result.rouge_l.fmeasure:.3f}")  # Longest common subsequence
-```
-
-### Lexical Similarity
-
-Measures token overlap using Jaccard similarity.
-
-```python
-from veritext.metrics import Lexical
-
-lexical = Lexical()
-result = lexical.score(
-    candidate="The quick brown fox",
-    reference="The fast brown fox",
-)
-print(f"Jaccard: {result.jaccard:.3f}")
-print(f"Token overlap: {result.token_overlap:.3f}")
-```
-
-### Readability
-
-Computes Flesch-Kincaid scores for text complexity.
-
-```python
-from veritext.metrics import Readability
-
-readability = Readability()
-result = readability.score("This is a simple sentence.")
-print(f"Grade level: {result.flesch_kincaid_grade:.1f}")
-print(f"Reading ease: {result.flesch_reading_ease:.1f}")
-```
-
-### Semantic Similarity (Optional)
-
-Requires `pip install veritext[semantic]`.
-
-```python
-from veritext.semantic import SemanticSimilarity
-
-semantic = SemanticSimilarity()
-result = semantic.score(
-    candidate="The dog is running in the park.",
-    reference="A canine is jogging through the garden.",
-)
-print(f"Similarity: {result.score:.3f}")
-```
-
-## Validators
-
-Validators wrap metrics with thresholds to make pass/fail decisions.
-
-### Metric-Based Validators
-
-```python
-from veritext.core.types import ValidationContext
-from veritext.validators import bleu, lexical, rouge
-
-context = ValidationContext(reference="Reference text here.")
-
-# BLEU validation
-validator = bleu(min_score=0.5, variant=4)  # BLEU-4
-result = validator.check("Candidate text here.", context)
-
-# ROUGE validation
-validator = rouge(min_score=0.6, variant="l")  # ROUGE-L
-result = validator.check("Candidate text here.", context)
-
-# Lexical validation
-validator = lexical(min_jaccard=0.3, min_overlap=0.5)
-result = validator.check("Candidate text here.", context)
-```
-
-### Constraint Validators
-
-These don't require reference text.
-
-```python
-from veritext.core.types import ValidationContext
-from veritext.validators import contains, excludes, length, readability
-
-context = ValidationContext()  # No reference needed
-
-# Length constraints
-validator = length(min_chars=50, max_chars=500, min_words=10)
-result = validator.check("Your text here...", context)
-
-# Readability constraints
-validator = readability(max_grade=8.0, min_ease=60.0)
-result = validator.check("Your text here...", context)
-
-# Content requirements
-validator = contains(patterns=["important", "keyword"])
-result = validator.check("This important text has a keyword.", context)
-
-# Content exclusions
-validator = excludes(patterns=["forbidden", "banned"])
-result = validator.check("This text is clean.", context)
-```
-
-### Composite Validators
-
-Combine multiple checks with logical operators.
-
-```python
-from veritext.validators import all_of, any_of, bleu, length, rouge
-
-# All checks must pass
-validator = all_of([
-    bleu(min_score=0.5),
-    rouge(min_score=0.6),
-    length(max_chars=500),
-])
-
-# At least one check must pass
-validator = any_of([
-    bleu(min_score=0.7),
-    rouge(min_score=0.7),
-])
-```
-
-## Pytest Plugin
-
-Veritext provides native pytest integration for testing text quality.
-
-### Basic Usage
-
-```python
-from veritext.pytest_plugin import validate_text
-
-
-def test_response_quality():
-    response = "This is a helpful response to your question."
-
-    validate_text(
-        response,
-        min_length=20,
-        max_length=200,
-        max_reading_grade=10.0,
-        must_contain=["helpful"],
-        must_exclude=["error", "sorry"],
-    )
-
-
-def test_summary_similarity():
-    summary = "Scientists discovered a new planet."
-    reference = "Researchers found a new planet in our solar system."
-
-    validate_text(
-        summary,
-        reference=reference,
-        min_rouge=0.5,
-        min_length=10,
-    )
-```
-
-### Available Parameters
-
-| Parameter | Description |
-|-----------|-------------|
-| `reference` | Reference text for comparison metrics |
-| `min_bleu` | Minimum BLEU-4 score (0.0-1.0) |
-| `min_rouge` | Minimum ROUGE-L F1 score (0.0-1.0) |
-| `min_semantic` | Minimum semantic similarity (0.0-1.0) |
-| `min_length` | Minimum character count |
-| `max_length` | Maximum character count |
-| `max_reading_grade` | Maximum Flesch-Kincaid grade level |
-| `must_contain` | List of required patterns |
-| `must_exclude` | List of forbidden patterns |
-
-## Benchmarking
-
-Track text quality over time and detect regressions.
-
-### Running Benchmarks
-
-```python
-from veritext.benchmark import Benchmark
-
-# Create a benchmark suite
-bench = Benchmark("summariser_quality", storage_path="benchmarks/")
-
-# Evaluate a batch of outputs
-candidates = ["Summary 1...", "Summary 2...", "Summary 3..."]
-references = ["Reference 1...", "Reference 2...", "Reference 3..."]
-
-run = bench.evaluate(
-    candidates=candidates,
-    references=references,
-    metrics=["rouge_l", "bleu4"],
-    metadata={"model": "v1.2", "git_sha": "abc123"},
-)
-
-print(f"ROUGE-L: {run.metrics['rouge_l']:.4f}")
-print(f"BLEU-4: {run.metrics['bleu4']:.4f}")
-```
-
-### Regression Detection
-
-```python
-from veritext.benchmark import Benchmark
-from veritext.core.exceptions import RegressionDetectedError
-
-bench = Benchmark("summariser_quality")
-
-# Check for regression against historical baseline
-report = bench.check_regression(tolerance=0.05, window=10)
-if report.detected:
-    print("Quality regression detected!")
-    for metric, delta in report.deltas.items():
-        print(f"  {metric}: {delta:+.4f}")
-
-# Or raise an exception for CI integration
-try:
-    bench.assert_no_regression(tolerance=0.05)
-except RegressionDetectedError as e:
-    print(f"CI failure: {e}")
-    exit(1)
-```
-
-### Viewing History
-
-```python
-bench = Benchmark("summariser_quality")
-
-for run in bench.get_history(limit=10):
-    print(f"{run.timestamp}: rouge_l={run.metrics.get('rouge_l', 0):.4f}")
-```
-
-## CLI
-
-Veritext provides a command-line interface for validation and benchmarking.
-
-### Validate Text
-
-```bash
-# Inline validation
-veritext validate "Candidate text" -r "Reference text" -m bleu,rouge
-
-# File-based batch validation (JSONL with "candidate" and "reference" fields)
-veritext validate -f outputs.jsonl -m bleu,rouge,lexical
-
-# With threshold for pass/fail
-veritext validate "Text" -r "Reference" -m bleu -t 0.5 -o simple
-
-# Output formats: table (default), json, simple
-veritext validate "Text" -r "Reference" -m bleu -o json
-```
-
-### Benchmark Commands
-
-```bash
-# Run a benchmark evaluation
-veritext benchmark run my_bench -f data.jsonl -m rouge_l,bleu4
-
-# View benchmark history
-veritext benchmark show my_bench --last 10
-
-# Check for regression (exits with code 1 if detected)
-veritext benchmark check my_bench --tolerance 0.05 --window 10
-```
-
-### JSONL Format
-
-For file-based operations, use JSONL with `candidate` and `reference` fields:
-
-```json
-{"candidate": "Model output 1", "reference": "Expected output 1"}
-{"candidate": "Model output 2", "reference": "Expected output 2"}
-```
-
-## Configuration
-
-Veritext uses environment variables for configuration:
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `VERITEXT_LOG_LEVEL` | `INFO` | Logging level |
-| `VERITEXT_LOG_FORMAT` | `console` | Log format (`console` or `json`) |
-
-## Development
-
-### Setup
-
-```bash
-git clone https://gitea.kschappell.com/kschappell/veritext.git
-cd veritext
-uv sync --all-extras
-```
-
-### Quality Checks
-
-```bash
-# Linting
-uv run ruff check .
-
-# Formatting
-uv run ruff format --check .
-
-# Type checking
-uv run mypy src/
-
-# Tests
-uv run pytest
-```
-
-### Running Examples
-
-```bash
-uv run python examples/basic_validation.py
-uv run pytest examples/chatbot_testing.py -v
-uv run python examples/benchmark_regression.py
-```
 
 ## Licence
 

src/veritext/benchmark/__init__.py

@@ -1,12 +0,0 @@
-"""Benchmark module for quality tracking and regression detection."""
-
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-from veritext.benchmark.runner import Benchmark
-from veritext.benchmark.storage import BenchmarkStorage
-
-__all__ = [
-    "Benchmark",
-    "BenchmarkRun",
-    "BenchmarkStorage",
-    "RegressionReport",
-]

src/veritext/benchmark/models.py

@@ -1,72 +0,0 @@
-"""Benchmark data models."""
-
-from datetime import datetime
-from typing import Any
-
-from pydantic import BaseModel, ConfigDict, Field
-
-
-class BenchmarkRun(BaseModel):
-    """Record of a single benchmark execution."""
-
-    model_config = ConfigDict(frozen=True)
-
-    id: str
-    """UUID for this run."""
-
-    benchmark_name: str
-    """Name identifying this benchmark suite."""
-
-    timestamp: datetime
-    """When the benchmark was executed."""
-
-    veritext_version: str
-    """Version of veritext used."""
-
-    metrics: dict[str, float]
-    """Metric results, e.g. {"rouge_l": 0.82, "bleu4": 0.71}."""
-
-    sample_count: int
-    """Number of samples evaluated."""
-
-    metadata: dict[str, Any] = Field(default_factory=dict)
-    """Optional metadata (git_sha, model version, etc.)."""
-
-
-class RegressionReport(BaseModel):
-    """Report comparing current run against baseline."""
-
-    model_config = ConfigDict(frozen=True)
-
-    detected: bool
-    """Whether a regression was detected."""
-
-    baseline: dict[str, float]
-    """Baseline metric values (rolling average)."""
-
-    current: dict[str, float]
-    """Current run metric values."""
-
-    deltas: dict[str, float]
-    """Difference from baseline (negative = regression)."""
-
-    tolerance: float
-    """Tolerance threshold used for detection."""
-
-    @property
-    def summary(self) -> str:
-        """Human-readable summary of the report."""
-        if not self.detected:
-            return "No regression detected. All metrics within tolerance."
-
-        regressions = [
-            f"  {metric}: {self.current.get(metric, 0.0):.4f} "
-            f"(baseline: {self.baseline.get(metric, 0.0):.4f}, "
-            f"delta: {delta:+.4f})"
-            for metric, delta in self.deltas.items()
-            if delta < -self.tolerance
-        ]
-
-        return f"Regression detected (tolerance: {self.tolerance:.2%}):\n" + "\n".join(
-            regressions
-        )

src/veritext/benchmark/regression.py

@@ -1,82 +0,0 @@
-"""Regression detection using rolling window comparison."""
-
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-
-
-def compute_baseline(
-    runs: list[BenchmarkRun],
-    window: int = 10,
-) -> dict[str, float]:
-    """
-    Compute rolling average baseline from recent runs.
-
-    Args:
-        runs: List of benchmark runs (most recent first).
-        window: Number of runs to include in the baseline.
-
-    Returns:
-        Dictionary of metric names to their average values.
-    """
-    if not runs:
-        return {}
-
-    recent_runs = runs[:window]
-
-    metric_values: dict[str, list[float]] = {}
-    for run in recent_runs:
-        for metric_name, value in run.metrics.items():
-            if metric_name not in metric_values:
-                metric_values[metric_name] = []
-            metric_values[metric_name].append(value)
-
-    return {
-        metric: sum(values) / len(values) for metric, values in metric_values.items()
-    }
-
-
-def detect_regression(
-    current: dict[str, float],
-    baseline: dict[str, float],
-    tolerance: float = 0.05,
-) -> RegressionReport:
-    """
-    Compare current metrics against baseline.
-
-    A regression is detected if any metric drops by more than the tolerance
-    threshold (relative to its baseline value).
-
-    Args:
-        current: Current metric values.
-        baseline: Baseline metric values.
-        tolerance: Maximum allowed drop before regression is flagged (e.g., 0.05 = 5%).
-
-    Returns:
-        RegressionReport with comparison results.
-    """
-    if not baseline:
-        return RegressionReport(
-            detected=False,
-            baseline=baseline,
-            current=current,
-            deltas={},
-            tolerance=tolerance,
-        )
-
-    deltas: dict[str, float] = {}
-    detected = False
-
-    for metric, baseline_value in baseline.items():
-        current_value = current.get(metric, 0.0)
-        delta = current_value - baseline_value
-        deltas[metric] = delta
-
-        if delta < -tolerance:
-            detected = True
-
-    return RegressionReport(
-        detected=detected,
-        baseline=baseline,
-        current=current,
-        deltas=deltas,
-        tolerance=tolerance,
-    )

src/veritext/benchmark/runner.py

@@ -1,179 +0,0 @@
-"""Benchmark execution and tracking."""
-
-import uuid
-from datetime import UTC, datetime
-from pathlib import Path
-from typing import Any
-
-import veritext
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-from veritext.benchmark.regression import compute_baseline, detect_regression
-from veritext.benchmark.storage import BenchmarkStorage
-from veritext.core.exceptions import RegressionDetectedError
-from veritext.metrics.bleu import Bleu
-from veritext.metrics.rouge import Rouge
-
-DEFAULT_METRICS = ["rouge_l", "bleu4"]
-
-
-class Benchmark:
-    """Track text quality over time."""
-
-    def __init__(
-        self,
-        name: str,
-        storage_path: str | Path = "benchmarks/",
-    ) -> None:
-        """
-        Initialise a benchmark tracker.
-
-        Args:
-            name: Name identifying this benchmark suite.
-            storage_path: Directory for storing benchmark data.
-        """
-        self._name = name
-        self._storage_path = Path(storage_path)
-        self._storage = BenchmarkStorage(self._storage_path / f"{name}.db")
-
-        self._bleu = Bleu()
-        self._rouge = Rouge()
-
-    @property
-    def name(self) -> str:
-        return self._name
-
-    def _compute_metrics(
-        self,
-        candidates: list[str],
-        references: list[str] | list[list[str]],
-        metric_names: list[str],
-    ) -> dict[str, float]:
-        results: dict[str, float] = {}
-
-        for metric_name in metric_names:
-            if metric_name in ("bleu1", "bleu2", "bleu3", "bleu4"):
-                batch_result = self._bleu.batch_score(candidates, references)
-                stats = batch_result.stats.get(metric_name)
-                if stats:
-                    results[metric_name] = stats.mean
-
-            elif metric_name in (
-                "rouge1",
-                "rouge2",
-                "rouge_l",
-                "rouge1_fmeasure",
-                "rouge2_fmeasure",
-                "rouge_l_fmeasure",
-            ):
-                rouge_result = self._rouge.batch_score(candidates, references)
-                stat_name = metric_name
-                if metric_name == "rouge1":
-                    stat_name = "rouge1_fmeasure"
-                elif metric_name == "rouge2":
-                    stat_name = "rouge2_fmeasure"
-                elif metric_name == "rouge_l":
-                    stat_name = "rouge_l_fmeasure"
-
-                stats = rouge_result.stats.get(stat_name)
-                if stats:
-                    results[metric_name] = stats.mean
-
-        return results
-
-    def evaluate(
-        self,
-        candidates: list[str],
-        references: list[str] | list[list[str]],
-        metrics: list[str] | None = None,
-        metadata: dict[str, Any] | None = None,
-    ) -> BenchmarkRun:
-        """
-        Evaluate candidates against references, store results, and return the run.
-
-        Args:
-            candidates: List of candidate texts to evaluate.
-            references: Reference text(s) for each candidate.
-            metrics: List of metrics to compute. Defaults to ["rouge_l", "bleu4"].
-            metadata: Optional metadata (git_sha, model version, etc.).
-
-        Returns:
-            The BenchmarkRun record that was created and stored.
-        """
-        metric_names = metrics or DEFAULT_METRICS
-        metric_results = self._compute_metrics(candidates, references, metric_names)
-
-        run = BenchmarkRun(
-            id=str(uuid.uuid4()),
-            benchmark_name=self._name,
-            timestamp=datetime.now(UTC),
-            veritext_version=veritext.__version__,
-            metrics=metric_results,
-            sample_count=len(candidates),
-            metadata=metadata or {},
-        )
-
-        self._storage.save_run(run)
-        return run
-
-    def check_regression(
-        self,
-        tolerance: float = 0.05,
-        window: int = 10,
-    ) -> RegressionReport:
-        """
-        Compare latest run against historical baseline.
-
-        Args:
-            tolerance: Maximum allowed metric drop before regression is flagged.
-            window: Number of historical runs to include in baseline.
-
-        Returns:
-            RegressionReport with comparison results.
-        """
-        runs = self._storage.get_runs(self._name)
-
-        if not runs:
-            return RegressionReport(
-                detected=False,
-                baseline={},
-                current={},
-                deltas={},
-                tolerance=tolerance,
-            )
-
-        current_run = runs[0]
-        historical_runs = runs[1:]
-        baseline = compute_baseline(historical_runs, window=window)
-
-        return detect_regression(current_run.metrics, baseline, tolerance)
-
-    def assert_no_regression(
-        self,
-        tolerance: float = 0.05,
-        window: int = 10,
-    ) -> None:
-        """
-        Raise RegressionDetectedError if quality dropped.
-
-        Args:
-            tolerance: Maximum allowed metric drop before regression is flagged.
-            window: Number of historical runs to include in baseline.
-
-        Raises:
-            RegressionDetectedError: If a regression is detected.
-        """
-        report = self.check_regression(tolerance=tolerance, window=window)
-        if report.detected:
-            raise RegressionDetectedError(report.summary)
-
-    def get_history(self, limit: int = 20) -> list[BenchmarkRun]:
-        """
-        Get recent benchmark runs.
-
-        Args:
-            limit: Maximum number of runs to return.
-
-        Returns:
-            List of BenchmarkRun objects, most recent first.
-        """
-        return self._storage.get_runs(self._name, limit=limit)

src/veritext/benchmark/storage.py

@@ -1,158 +0,0 @@
-"""SQLite storage for benchmark history."""
-
-import json
-import sqlite3
-from datetime import datetime
-from pathlib import Path
-
-from veritext.benchmark.models import BenchmarkRun
-from veritext.core.exceptions import StorageError
-
-
-class BenchmarkStorage:
-    """SQLite-backed storage for benchmark runs."""
-
-    def __init__(self, db_path: Path) -> None:
-        """
-        Initialise storage, creating tables if needed.
-
-        Args:
-            db_path: Path to the SQLite database file.
-        """
-        self._db_path = db_path
-        self._db_path.parent.mkdir(parents=True, exist_ok=True)
-        try:
-            with self._get_connection() as conn:
-                conn.executescript("""
-                    CREATE TABLE IF NOT EXISTS benchmark_runs (
-                        id TEXT PRIMARY KEY,
-                        benchmark_name TEXT NOT NULL,
-                        timestamp TEXT NOT NULL,
-                        veritext_version TEXT NOT NULL,
-                        sample_count INTEGER NOT NULL,
-                        metadata TEXT
-                    );
-
-                    CREATE TABLE IF NOT EXISTS benchmark_metrics (
-                        run_id TEXT REFERENCES benchmark_runs(id) ON DELETE CASCADE,
-                        metric_name TEXT NOT NULL,
-                        value REAL NOT NULL,
-                        PRIMARY KEY (run_id, metric_name)
-                    );
-
-                    CREATE INDEX IF NOT EXISTS idx_benchmark_name
-                    ON benchmark_runs(benchmark_name, timestamp DESC);
-                """)
-        except sqlite3.Error as e:
-            raise StorageError(f"Failed to initialise database: {e}") from e
-
-    def _get_connection(self) -> sqlite3.Connection:
-        conn = sqlite3.connect(str(self._db_path), timeout=30.0)
-        conn.execute("PRAGMA journal_mode=WAL")
-        conn.execute("PRAGMA foreign_keys=ON")
-        conn.row_factory = sqlite3.Row
-        return conn
-
-    def save_run(self, run: BenchmarkRun) -> None:
-        """
-        Persist a benchmark run.
-
-        Args:
-            run: The benchmark run to save.
-
-        Raises:
-            StorageError: If the save operation fails.
-        """
-        try:
-            with self._get_connection() as conn:
-                conn.execute(
-                    """
-                    INSERT INTO benchmark_runs
-                    (id, benchmark_name, timestamp, veritext_version, sample_count, metadata)
-                    VALUES (?, ?, ?, ?, ?, ?)
-                    """,
-                    (
-                        run.id,
-                        run.benchmark_name,
-                        run.timestamp.isoformat(),
-                        run.veritext_version,
-                        run.sample_count,
-                        json.dumps(run.metadata) if run.metadata else None,
-                    ),
-                )
-
-                for metric_name, value in run.metrics.items():
-                    conn.execute(
-                        """
-                        INSERT INTO benchmark_metrics (run_id, metric_name, value)
-                        VALUES (?, ?, ?)
-                        """,
-                        (run.id, metric_name, value),
-                    )
-        except sqlite3.IntegrityError as e:
-            raise StorageError(f"Run with id '{run.id}' already exists") from e
-        except sqlite3.Error as e:
-            raise StorageError(f"Failed to save benchmark run: {e}") from e
-
-    def get_runs(
-        self,
-        benchmark_name: str,
-        limit: int | None = None,
-    ) -> list[BenchmarkRun]:
-        """
-        Retrieve runs for a benchmark, most recent first.
-
-        Args:
-            benchmark_name: Name of the benchmark to retrieve runs for.
-            limit: Maximum number of runs to return.
-
-        Returns:
-            List of BenchmarkRun objects, most recent first.
-
-        Raises:
-            StorageError: If the retrieval fails.
-        """
-        try:
-            with self._get_connection() as conn:
-                query = """
-                    SELECT id, benchmark_name, timestamp, veritext_version,
-                           sample_count, metadata
-                    FROM benchmark_runs
-                    WHERE benchmark_name = ?
-                    ORDER BY timestamp DESC
-                """
-                if limit is not None:
-                    query += " LIMIT ?"
-                    rows = conn.execute(query, (benchmark_name, limit)).fetchall()
-                else:
-                    rows = conn.execute(query, (benchmark_name,)).fetchall()
-
-                runs = []
-                for row in rows:
-                    metrics_rows = conn.execute(
-                        "SELECT metric_name, value FROM benchmark_metrics WHERE run_id = ?",
-                        (row["id"],),
-                    ).fetchall()
-                    metrics = {m["metric_name"]: m["value"] for m in metrics_rows}
-
-                    metadata = json.loads(row["metadata"]) if row["metadata"] else {}
-
-                    runs.append(
-                        BenchmarkRun(
-                            id=row["id"],
-                            benchmark_name=row["benchmark_name"],
-                            timestamp=datetime.fromisoformat(row["timestamp"]),
-                            veritext_version=row["veritext_version"],
-                            sample_count=row["sample_count"],
-                            metrics=metrics,
-                            metadata=metadata,
-                        )
-                    )
-
-                return runs
-        except sqlite3.Error as e:
-            raise StorageError(f"Failed to retrieve benchmark runs: {e}") from e
-
-    def get_latest_run(self, benchmark_name: str) -> BenchmarkRun | None:
-        runs = self.get_runs(benchmark_name, limit=1)
-        return runs[0] if runs else None

src/veritext/cli/__init__.py

@@ -1,5 +0,0 @@
-"""CLI module: Command-line interface for Veritext."""
-
-from veritext.cli.main import app
-
-__all__ = ["app"]

src/veritext/cli/benchmark.py

@@ -1,166 +0,0 @@
-"""Benchmark commands for quality tracking."""
-
-from pathlib import Path
-from typing import Annotated
-
-import typer
-
-from veritext.benchmark import Benchmark
-from veritext.cli.formatters import (
-    console,
-    format_benchmark_history,
-    format_regression_report,
-)
-from veritext.cli.readers import read_jsonl
-
-benchmark_app = typer.Typer(
-    name="benchmark",
-    help="Track and compare text quality over time.",
-    no_args_is_help=True,
-)
-
-
-@benchmark_app.command("run")
-def benchmark_run(
-    name: Annotated[
-        str,
-        typer.Argument(help="Name for this benchmark suite."),
-    ],
-    file: Annotated[
-        Path,
-        typer.Option("--file", "-f", help="JSONL file with candidate/reference pairs."),
-    ],
-    metrics: Annotated[
-        str,
-        typer.Option(
-            "--metrics",
-            "-m",
-            help="Comma-separated metrics to track (e.g., rouge_l,bleu4).",
-        ),
-    ] = "rouge_l,bleu4",
-    storage_path: Annotated[
-        Path,
-        typer.Option(
-            "--storage",
-            "-s",
-            help="Directory for benchmark data storage.",
-        ),
-    ] = Path("benchmarks"),
-) -> None:
-    """
-    Run a benchmark evaluation and store the results.
-
-    Example:
-        veritext benchmark run my_bench -f data.jsonl -m rouge_l,bleu4
-    """
-    # Read text pairs
-    try:
-        pairs = read_jsonl(file)
-    except (FileNotFoundError, ValueError) as e:
-        console.print(f"[red]Error:[/red] {e}")
-        raise typer.Exit(code=1) from e
-
-    if not pairs:
-        console.print("[yellow]Warning:[/yellow] No text pairs found in file.")
-        raise typer.Exit(code=0)
-
-    # Parse metrics
-    metric_names = [m.strip() for m in metrics.split(",")]
-
-    candidates = [p.candidate for p in pairs]
-    references = [p.reference for p in pairs]
-
-    # Run benchmark
-    bench = Benchmark(name, storage_path=storage_path)
-    run = bench.evaluate(candidates, references, metrics=metric_names)
-
-    console.print(f"[green]Benchmark '{name}' completed.[/green]")
-    console.print(f"Samples: {run.sample_count}")
-    console.print("\nMetrics:")
-    for metric_name, value in sorted(run.metrics.items()):
-        console.print(f"  {metric_name}: {value:.4f}")
-
-
-@benchmark_app.command("show")
-def benchmark_show(
-    name: Annotated[
-        str,
-        typer.Argument(help="Name of the benchmark suite."),
-    ],
-    last: Annotated[
-        int,
-        typer.Option("--last", "-n", help="Number of recent runs to show."),
-    ] = 20,
-    storage_path: Annotated[
-        Path,
-        typer.Option(
-            "--storage",
-            "-s",
-            help="Directory for benchmark data storage.",
-        ),
-    ] = Path("benchmarks"),
-) -> None:
-    """
-    Show benchmark history for a suite.
-
-    Example:
-        veritext benchmark show my_bench --last 10
-    """
-    bench = Benchmark(name, storage_path=storage_path)
-    runs = bench.get_history(limit=last)
-
-    if not runs:
-        console.print(f"[yellow]No benchmark runs found for '{name}'.[/yellow]")
-        raise typer.Exit(code=0)
-
-    table = format_benchmark_history(runs)
-    console.print(table)
-
-
-@benchmark_app.command("check")
-def benchmark_check(
-    name: Annotated[
-        str,
-        typer.Argument(help="Name of the benchmark suite."),
-    ],
-    tolerance: Annotated[
-        float,
-        typer.Option(
-            "--tolerance",
-            "-t",
-            help="Maximum allowed metric drop (e.g., 0.05 = 5%).",
-        ),
-    ] = 0.05,
-    window: Annotated[
-        int,
-        typer.Option(
-            "--window",
-            "-w",
-            help="Number of historical runs for baseline.",
-        ),
-    ] = 10,
-    storage_path: Annotated[
-        Path,
-        typer.Option(
-            "--storage",
-            "-s",
-            help="Directory for benchmark data storage.",
-        ),
-    ] = Path("benchmarks"),
-) -> None:
-    """
-    Check for quality regression against historical baseline.
-
-    Exits with code 1 if regression detected (for CI integration).
-
-    Example:
-        veritext benchmark check my_bench --tolerance 0.05
-    """
-    bench = Benchmark(name, storage_path=storage_path)
-    report = bench.check_regression(tolerance=tolerance, window=window)
-
-    panel = format_regression_report(report)
-    console.print(panel)
-
-    if report.detected:
-        raise typer.Exit(code=1)

src/veritext/cli/formatters.py

@@ -1,150 +0,0 @@
-"""Rich output formatters for CLI display."""
-
-import json
-
-from rich.console import Console
-from rich.panel import Panel
-from rich.table import Table
-
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-
-console = Console()
-
-
-def format_validation_table(
-    results: dict[str, float],
-    threshold: float | None = None,
-) -> Table:
-    """
-    Format validation results as a Rich table.
-
-    Args:
-        results: Dictionary of metric names to scores.
-        threshold: Optional threshold for pass/fail colouring.
-
-    Returns:
-        Rich Table object.
-    """
-    table = Table(title="Validation Results", show_header=True, header_style="bold")
-    table.add_column("Metric", style="cyan")
-    table.add_column("Score", justify="right")
-
-    if threshold is not None:
-        table.add_column("Status", justify="center")
-
-    for metric, score in sorted(results.items()):
-        score_str = f"{score:.4f}"
-
-        if threshold is not None:
-            status = "[green]PASS[/green]" if score >= threshold else "[red]FAIL[/red]"
-            table.add_row(metric, score_str, status)
-        else:
-            table.add_row(metric, score_str)
-
-    return table
-
-
-def format_validation_json(results: dict[str, float]) -> str:
-    return json.dumps(results, indent=2)
-
-
-def format_validation_simple(results: dict[str, float]) -> str:
-    lines = [f"{metric}: {score:.4f}" for metric, score in sorted(results.items())]
-    return "\n".join(lines)
-
-
-def format_benchmark_history(runs: list[BenchmarkRun]) -> Table:
-    """
-    Format benchmark run history as a Rich table.
-
-    Args:
-        runs: List of BenchmarkRun objects (most recent first).
-
-    Returns:
-        Rich Table object.
-    """
-    if not runs:
-        table = Table(title="Benchmark History")
-        table.add_column("No runs found")
-        return table
-
-    metric_names: set[str] = set()
-    for run in runs:
-        metric_names.update(run.metrics.keys())
-    sorted_metrics = sorted(metric_names)
-
-    table = Table(title="Benchmark History", show_header=True, header_style="bold")
-    table.add_column("Timestamp", style="cyan")
-    table.add_column("Samples", justify="right")
-    for metric in sorted_metrics:
-        table.add_column(metric, justify="right")
-
-    for run in runs:
-        timestamp = run.timestamp.strftime("%Y-%m-%d %H:%M")
-        samples = str(run.sample_count)
-        metric_values = [f"{run.metrics.get(m, 0.0):.4f}" for m in sorted_metrics]
-        table.add_row(timestamp, samples, *metric_values)
-
-    return table
-
-
-def format_regression_report(report: RegressionReport) -> Panel:
-    """
-    Format a regression report as a Rich panel.
-
-    Args:
-        report: RegressionReport object.
-
-    Returns:
-        Rich Panel object with formatted report.
-    """
-    if not report.detected:
-        content = (
-            f"[green]No regression detected.[/green]\nTolerance: {report.tolerance:.2%}"
-        )
-        return Panel(content, title="Regression Check", border_style="green")
-
-    lines = [
-        "[red]Regression detected![/red]",
-        f"Tolerance: {report.tolerance:.2%}",
-        "",
-        "Metric details:",
-    ]
-
-    for metric in sorted(report.deltas.keys()):
-        baseline = report.baseline.get(metric, 0.0)
-        current = report.current.get(metric, 0.0)
-        delta = report.deltas[metric]
-
-        if delta < -report.tolerance:
-            status = "[red]REGRESSED[/red]"
-        else:
-            status = "[green]OK[/green]"
-
-        lines.append(
-            f"  {metric}: {current:.4f} (baseline: {baseline:.4f}, "
-            f"delta: {delta:+.4f}) {status}"
-        )
-
-    return Panel("\n".join(lines), title="Regression Check", border_style="red")
-
-
-def print_validation_output(
-    results: dict[str, float],
-    output_format: str = "table",
-    threshold: float | None = None,
-) -> None:
-    """
-    Print validation results in the specified format.
-
-    Args:
-        results: Dictionary of metric names to scores.
-        output_format: Output format ('table', 'json', or 'simple').
-        threshold: Optional threshold for pass/fail colouring (table only).
-    """
-    if output_format == "json":
-        console.print(format_validation_json(results))
-    elif output_format == "simple":
-        console.print(format_validation_simple(results))
-    else:
-        console.print(format_validation_table(results, threshold))

src/veritext/cli/main.py

@@ -1,37 +0,0 @@
-"""Veritext CLI entry point."""
-
-import typer
-
-import veritext
-from veritext.cli.benchmark import benchmark_app
-from veritext.cli.validate import validate
-
-app = typer.Typer(
-    name="veritext",
-    help="Semantic text validation framework.",
-    no_args_is_help=True,
-)
-
-# Register commands
-app.command()(validate)
-app.add_typer(benchmark_app)
-
-
-@app.callback(invoke_without_command=True)
-def main(
-    version: bool | None = typer.Option(
-        None,
-        "--version",
-        "-V",
-        help="Show version and exit.",
-        is_eager=True,
-    ),
-) -> None:
-    """Veritext: Semantic text validation framework for Python."""
-    if version:
-        typer.echo(f"veritext {veritext.__version__}")
-        raise typer.Exit()
-
-
-if __name__ == "__main__":
-    app()

src/veritext/cli/readers.py

@@ -1,118 +0,0 @@
-"""Input readers for CLI operations."""
-
-import json
-from dataclasses import dataclass
-from pathlib import Path
-
-
-@dataclass
-class TextPair:
-
-    candidate: str
-    reference: str
-
-
-def read_jsonl(path: Path) -> list[TextPair]:
-    """
-    Read text pairs from a JSONL file.
-
-    Each line must be a JSON object with 'candidate' and 'reference' keys.
-
-    Args:
-        path: Path to the JSONL file.
-
-    Returns:
-        List of TextPair objects.
-
-    Raises:
-        FileNotFoundError: If the file does not exist.
-        ValueError: If any line is malformed or missing required keys.
-    """
-    if not path.exists():
-        raise FileNotFoundError(f"File not found: {path}")
-
-    pairs: list[TextPair] = []
-    with path.open() as f:
-        for line_num, line in enumerate(f, start=1):
-            line = line.strip()
-            if not line:
-                continue
-
-            try:
-                data = json.loads(line)
-            except json.JSONDecodeError as e:
-                raise ValueError(f"Invalid JSON on line {line_num}: {e}") from e
-
-            if "candidate" not in data:
-                raise ValueError(f"Missing 'candidate' key on line {line_num}")
-            if "reference" not in data:
-                raise ValueError(f"Missing 'reference' key on line {line_num}")
-
-            pairs.append(
-                TextPair(
-                    candidate=str(data["candidate"]),
-                    reference=str(data["reference"]),
-                )
-            )
-
-    return pairs
-
-
-def read_paired_jsonl(candidates_path: Path, references_path: Path) -> list[TextPair]:
-    """
-    Read text pairs from separate candidate and reference JSONL files.
-
-    Each file should contain one JSON object per line with a 'text' key.
-
-    Args:
-        candidates_path: Path to the candidates JSONL file.
-        references_path: Path to the references JSONL file.
-
-    Returns:
-        List of TextPair objects.
-
-    Raises:
-        FileNotFoundError: If either file does not exist.
-        ValueError: If files have different lengths or are malformed.
-    """
-    candidates = _read_text_jsonl(candidates_path, "candidates")
-    references = _read_text_jsonl(references_path, "references")
-
-    if len(candidates) != len(references):
-        raise ValueError(
-            f"Number of candidates ({len(candidates)}) does not match "
-            f"number of references ({len(references)})"
-        )
-
-    return [
-        TextPair(candidate=c, reference=r)
-        for c, r in zip(candidates, references, strict=True)
-    ]
-
-
-def _read_text_jsonl(path: Path, label: str) -> list[str]:
-    if not path.exists():
-        raise FileNotFoundError(f"{label.capitalize()} file not found: {path}")
-
-    texts: list[str] = []
-    with path.open() as f:
-        for line_num, line in enumerate(f, start=1):
-            line = line.strip()
-            if not line:
-                continue
-
-            try:
-                data = json.loads(line)
-            except json.JSONDecodeError as e:
-                raise ValueError(
-                    f"Invalid JSON in {label} file on line {line_num}: {e}"
-                ) from e
-
-            if "text" not in data:
-                raise ValueError(
-                    f"Missing 'text' key in {label} file on line {line_num}"
-                )
-
-            texts.append(str(data["text"]))
-
-    return texts

src/veritext/cli/validate.py

@@ -1,222 +0,0 @@
-"""Validate command for computing text metrics."""
-
-from pathlib import Path
-from typing import Annotated
-
-import typer
-
-from veritext.cli.formatters import console, print_validation_output
-from veritext.cli.readers import read_jsonl, read_paired_jsonl
-from veritext.metrics.bleu import Bleu
-from veritext.metrics.lexical import Lexical
-from veritext.metrics.rouge import Rouge
-
-AVAILABLE_METRICS = frozenset(
-    {"bleu", "bleu1", "bleu2", "bleu3", "bleu4", "rouge", "rouge_l", "lexical"}
-)
-
-_bleu: Bleu | None = None
-_rouge: Rouge | None = None
-_lexical: Lexical | None = None
-
-
-def _get_bleu() -> Bleu:
-    global _bleu
-    if _bleu is None:
-        _bleu = Bleu()
-    return _bleu
-
-
-def _get_rouge() -> Rouge:
-    global _rouge
-    if _rouge is None:
-        _rouge = Rouge()
-    return _rouge
-
-
-def _get_lexical() -> Lexical:
-    global _lexical
-    if _lexical is None:
-        _lexical = Lexical()
-    return _lexical
-def _bleu_single(candidate: str, reference: str, key: str) -> dict[str, float]:
-    result = _get_bleu().score(candidate, reference)
-    return {key: getattr(result, key)}
-
-
-def _bleu_batch(
-    candidates: list[str], references: list[str], key: str
-) -> dict[str, float]:
-    batch = _get_bleu().batch_score(candidates, references)
-    stats = batch.stats.get(key)
-    return {key: stats.mean} if stats else {}
-
-
-def _rouge_single(candidate: str, reference: str) -> dict[str, float]:
-    result = _get_rouge().score(candidate, reference)
-    return {"rouge_l": result.rouge_l.fmeasure}
-
-
-def _rouge_batch(candidates: list[str], references: list[str]) -> dict[str, float]:
-    batch = _get_rouge().batch_score(candidates, references)
-    stats = batch.stats.get("rouge_l_fmeasure")
-    return {"rouge_l": stats.mean} if stats else {}
-
-
-def _lexical_single(candidate: str, reference: str) -> dict[str, float]:
-    result = _get_lexical().score(candidate, reference)
-    return {"jaccard": result.jaccard, "token_overlap": result.token_overlap}
-
-
-def _lexical_batch(candidates: list[str], references: list[str]) -> dict[str, float]:
-    batch = _get_lexical().batch_score(candidates, references)
-    results: dict[str, float] = {}
-    jaccard_stats = batch.stats.get("jaccard")
-    overlap_stats = batch.stats.get("token_overlap")
-    if jaccard_stats:
-        results["jaccard"] = jaccard_stats.mean
-    if overlap_stats:
-        results["token_overlap"] = overlap_stats.mean
-    return results
-
-
-def _compute_metrics(
-    candidate: str,
-    reference: str,
-    metric_names: list[str],
-) -> dict[str, float]:
-    results: dict[str, float] = {}
-
-    for metric in metric_names:
-        if metric in ("bleu", "bleu4"):
-            results.update(_bleu_single(candidate, reference, "bleu4"))
-        elif metric in ("bleu1", "bleu2", "bleu3"):
-            results.update(_bleu_single(candidate, reference, metric))
-        elif metric in ("rouge", "rouge_l"):
-            results.update(_rouge_single(candidate, reference))
-        elif metric == "lexical":
-            results.update(_lexical_single(candidate, reference))
-
-    return results
-
-
-def _compute_batch_metrics(
-    candidates: list[str],
-    references: list[str],
-    metric_names: list[str],
-) -> dict[str, float]:
-    results: dict[str, float] = {}
-
-    for metric in metric_names:
-        if metric in ("bleu", "bleu4"):
-            results.update(_bleu_batch(candidates, references, "bleu4"))
-        elif metric in ("bleu1", "bleu2", "bleu3"):
-            results.update(_bleu_batch(candidates, references, metric))
-        elif metric in ("rouge", "rouge_l"):
-            results.update(_rouge_batch(candidates, references))
-        elif metric == "lexical":
-            results.update(_lexical_batch(candidates, references))
-
-    return results
-
-
-def _parse_metrics(metrics_str: str) -> list[str]:
-    metrics = [m.strip().lower() for m in metrics_str.split(",")]
-    invalid = [m for m in metrics if m not in AVAILABLE_METRICS]
-    if invalid:
-        raise typer.BadParameter(
-            f"Unknown metrics: {', '.join(invalid)}. "
-            f"Available: {', '.join(sorted(AVAILABLE_METRICS))}"
-        )
-
-    return metrics
-
-
-def validate(
-    text: Annotated[
-        str | None,
-        typer.Argument(help="Candidate text to validate (inline mode)."),
-    ] = None,
-    reference: Annotated[
-        str | None,
-        typer.Option("--reference", "-r", help="Reference text for comparison."),
-    ] = None,
-    file: Annotated[
-        Path | None,
-        typer.Option("--file", "-f", help="JSONL file with candidate/reference pairs."),
-    ] = None,
-    reference_file: Annotated[
-        Path | None,
-        typer.Option(
-            "--reference-file",
-            "-R",
-            help="Separate JSONL file with references (requires --file).",
-        ),
-    ] = None,
-    metrics: Annotated[
-        str,
-        typer.Option(
-            "--metrics",
-            "-m",
-            help="Comma-separated metrics: bleu, bleu1-4, rouge, rouge_l, lexical.",
-        ),
-    ] = "bleu,rouge",
-    output: Annotated[
-        str,
-        typer.Option("--output", "-o", help="Output format: table, json, or simple."),
-    ] = "table",
-    threshold: Annotated[
-        float | None,
-        typer.Option("--threshold", "-t", help="Score threshold for pass/fail status."),
-    ] = None,
-) -> None:
-    """
-    Validate text quality using various metrics.
-
-    Use inline mode for single texts:
-        veritext validate "text" -r "reference" -m bleu,rouge
-
-    Use file mode for batches:
-        veritext validate -f outputs.jsonl -m bleu,rouge
-    """
-    try:
-        metric_names = _parse_metrics(metrics)
-    except typer.BadParameter as e:
-        console.print(f"[red]Error:[/red] {e}")
-        raise typer.Exit(code=1) from e
-
-    if output not in ("table", "json", "simple"):
-        console.print(f"[red]Error:[/red] Invalid output format: {output}")
-        raise typer.Exit(code=1)
-
-    if file is not None:
-        try:
-            if reference_file is not None:
-                pairs = read_paired_jsonl(file, reference_file)
-            else:
-                pairs = read_jsonl(file)
-        except (FileNotFoundError, ValueError) as e:
-            console.print(f"[red]Error:[/red] {e}")
-            raise typer.Exit(code=1) from e
-
-        if not pairs:
-            console.print("[yellow]Warning:[/yellow] No text pairs found in file.")
-            raise typer.Exit(code=0)
-
-        candidates = [p.candidate for p in pairs]
-        references = [p.reference for p in pairs]
-
-        results = _compute_batch_metrics(candidates, references, metric_names)
-        console.print(f"[dim]Evaluated {len(pairs)} text pairs.[/dim]\n")
-
-    elif text is not None and reference is not None:
-        results = _compute_metrics(text, reference, metric_names)
-
-    else:
-        console.print(
-            "[red]Error:[/red] Provide either text and --reference, "
-            "or --file for batch mode."
-        )
-        raise typer.Exit(code=1)
-
-    print_validation_output(results, output, threshold)

src/veritext/core/config.py

@@ -1,6 +1,5 @@
 """Configuration management using pydantic-settings."""
 
-from functools import lru_cache
 from pathlib import Path
 from typing import Literal
 
@@ -55,6 +54,6 @@ class VeritextSettings(BaseSettings):
     )
 
 
-@lru_cache
 def get_settings() -> VeritextSettings:
+    """Get the current settings instance."""
     return VeritextSettings()

src/veritext/core/logging.py

@@ -24,12 +24,14 @@ def configure_logging(
     level = level or settings.log_level
     log_format = log_format or settings.log_format
 
+    # Configure standard library logging
     logging.basicConfig(
         format="%(message)s",
         stream=sys.stderr,
         level=getattr(logging, level),
     )
 
+    # Shared processors
     shared_processors: list[Any] = [
         structlog.contextvars.merge_contextvars,
         structlog.processors.add_log_level,
@@ -40,12 +42,14 @@ def configure_logging(
     ]
 
     if log_format == "json":
+        # JSON output for production/log aggregation
         processors = [
             *shared_processors,
             structlog.processors.format_exc_info,
             structlog.processors.JSONRenderer(),
         ]
     else:
+        # Console output for development
         processors = [
             *shared_processors,
             structlog.dev.ConsoleRenderer(colors=True),
@@ -63,4 +67,13 @@ def configure_logging(
 
 
 def get_logger(name: str | None = None) -> structlog.stdlib.BoundLogger:
+    """
+    Get a logger instance.
+
+    Args:
+        name: Logger name. Uses 'veritext' if not provided.
+
+    Returns:
+        A bound logger instance.
+    """
     return structlog.get_logger(name or "veritext")

src/veritext/core/tokenisation.py

@@ -10,7 +10,17 @@ NormalisationForm = Literal["NFC", "NFD", "NFKC", "NFKD"]
 class Tokeniser(Protocol):
     """Protocol for text tokenisers."""
 
-    def tokenise(self, text: str) -> list[str]: ...
+    def tokenise(self, text: str) -> list[str]:
+        """
+        Tokenise text into a list of tokens.
+
+        Args:
+            text: The text to tokenise.
+
+        Returns:
+            List of tokens.
+        """
+        ...
 
 
 class WordTokeniser:
@@ -38,6 +48,7 @@ class WordTokeniser:
         self.remove_punctuation = remove_punctuation
         self.normalisation_form: NormalisationForm = normalisation_form
 
+        # Pattern for punctuation removal (keeps alphanumeric and Unicode letters)
         self._punctuation_pattern = re.compile(r"[^\w\s]", re.UNICODE)
 
     def tokenise(self, text: str) -> list[str]:
@@ -53,14 +64,18 @@ class WordTokeniser:
         if not text or not text.strip():
             return []
 
+        # Unicode normalisation
         normalised = unicodedata.normalize(self.normalisation_form, text)
 
+        # Lowercase if requested
         if self.lowercase:
             normalised = normalised.lower()
 
+        # Remove punctuation if requested
         if self.remove_punctuation:
             normalised = self._punctuation_pattern.sub(" ", normalised)
 
+        # Split on whitespace and filter empty strings
         tokens = normalised.split()
 
         return tokens

src/veritext/core/types.py

@@ -51,10 +51,12 @@ class ValidationResult(BaseModel):
 
     @property
     def failed_checks(self) -> list[CheckResult]:
+        """Return only the checks that failed."""
         return [check for check in self.checks if not check.passed]
 
     @property
     def failure_summary(self) -> str:
+        """Return a summary of all failed checks."""
         failed = self.failed_checks
         if not failed:
             return "All checks passed."

src/veritext/metrics/base.py

@@ -61,6 +61,7 @@ class AggregateStats(BaseModel):
         sorted_values = sorted(values)
 
         def percentile(p: int) -> float:
+            """Compute percentile using linear interpolation."""
             if n == 1:
                 return sorted_values[0]
             k = (n - 1) * p / 100
@@ -99,10 +100,12 @@ class Metric(Protocol[T]):
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         ...
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         ...
 
     def score(self, candidate: str, reference: str | list[str] | None = None) -> T:

src/veritext/metrics/bleu.py

@@ -7,10 +7,9 @@ from veritext.core.tokenisation import WordTokeniser
 from veritext.metrics.base import AggregateStats, BatchResult
 from veritext.metrics.results import BleuResult
 
-MAX_NGRAM_ORDER = 4
-
 
 def _get_ngrams(tokens: list[str], n: int) -> Counter[tuple[str, ...]]:
+    """Extract n-grams from a list of tokens."""
     if n > len(tokens):
         return Counter()
     return Counter(tuple(tokens[i : i + n]) for i in range(len(tokens) - n + 1))
@@ -31,12 +30,14 @@ def _modified_precision(
     if not candidate_ngrams:
         return 0, 0
 
+    # Get max count for each n-gram across all references
     max_ref_counts: Counter[tuple[str, ...]] = Counter()
     for ref_tokens in reference_token_lists:
         ref_ngrams = _get_ngrams(ref_tokens, n)
         for ngram, count in ref_ngrams.items():
             max_ref_counts[ngram] = max(max_ref_counts[ngram], count)
 
+    # Clip candidate counts to max reference counts
     clipped_count = 0
     for ngram, count in candidate_ngrams.items():
         clipped_count += min(count, max_ref_counts[ngram])
@@ -53,6 +54,7 @@ def _brevity_penalty(candidate_len: int, reference_lens: list[int]) -> float:
     if candidate_len == 0:
         return 0.0
 
+    # Find closest reference length
     closest_ref_len = min(reference_lens, key=lambda r: (abs(r - candidate_len), r))
 
     if candidate_len >= closest_ref_len:
@@ -80,10 +82,12 @@ class Bleu:
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         return "bleu"
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         return True
 
     def score(
@@ -105,21 +109,26 @@ class Bleu:
         if reference is None:
             raise ValueError("BLEU requires reference text")
 
+        # Normalise reference to list
         references = [reference] if isinstance(reference, str) else reference
 
+        # Tokenise
         candidate_tokens = self._tokeniser.tokenise(candidate)
         reference_token_lists = [self._tokeniser.tokenise(r) for r in references]
 
+        # Handle empty candidate
         if not candidate_tokens:
             return BleuResult(
                 bleu1=0.0, bleu2=0.0, bleu3=0.0, bleu4=0.0, brevity_penalty=0.0
             )
 
+        # Handle empty references
         if all(not ref for ref in reference_token_lists):
             raise ValueError("Reference text cannot be empty")
 
+        # Compute modified precisions for n=1,2,3,4
         precisions = []
-        for n in range(1, MAX_NGRAM_ORDER + 1):
+        for n in range(1, 5):
             clipped, total = _modified_precision(
                 candidate_tokens, reference_token_lists, n
             )
@@ -128,19 +137,22 @@ class Bleu:
             else:
                 precisions.append(clipped / total)
 
+        # Compute BLEU scores using geometric mean
         bp = _brevity_penalty(
             len(candidate_tokens),
             [len(ref) for ref in reference_token_lists],
         )
 
         def geometric_mean(p_list: list[float]) -> float:
+            """Compute geometric mean with smoothing for zeros."""
             if any(p == 0.0 for p in p_list):
                 return 0.0
             log_sum = sum(math.log(p) for p in p_list)
             return math.exp(log_sum / len(p_list))
 
         bleu_scores = []
-        for n in range(1, MAX_NGRAM_ORDER + 1):
+        for n in range(1, 5):
+            # BLEU-n uses precisions 1 through n
             bleu_n = bp * geometric_mean(precisions[:n])
             bleu_scores.append(bleu_n)
 
@@ -184,6 +196,7 @@ class Bleu:
             ref: str | list[str] = references[i]
             results.append(self.score(cand, ref))
 
+        # Compute aggregate statistics for each score type
         stats = {
             "bleu1": AggregateStats.from_values([r.bleu1 for r in results]),
             "bleu2": AggregateStats.from_values([r.bleu2 for r in results]),

src/veritext/metrics/lexical.py

@@ -13,14 +13,22 @@ class Lexical:
     """
 
     def __init__(self, tokeniser: WordTokeniser | None = None) -> None:
+        """
+        Initialise the lexical metric.
+
+        Args:
+            tokeniser: Tokeniser to use. Defaults to WordTokeniser().
+        """
         self._tokeniser = tokeniser or WordTokeniser()
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         return "lexical"
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         return True
 
     def score(
@@ -43,14 +51,18 @@ class Lexical:
         if reference is None:
             raise ValueError("Lexical similarity requires reference text")
 
+        # Use first reference if list provided
         ref_text = reference[0] if isinstance(reference, list) else reference
 
+        # Tokenise
         candidate_tokens = self._tokeniser.tokenise(candidate)
         reference_tokens = self._tokeniser.tokenise(ref_text)
 
+        # Handle empty reference
         if not reference_tokens:
             raise ValueError("Reference text cannot be empty")
 
+        # Handle empty candidate
         if not candidate_tokens:
             return LexicalResult(jaccard=0.0, token_overlap=0.0)
 
@@ -97,6 +109,7 @@ class Lexical:
             ref: str | list[str] = references[i]
             results.append(self.score(cand, ref))
 
+        # Compute aggregate statistics
         stats = {
             "jaccard": AggregateStats.from_values([r.jaccard for r in results]),
             "token_overlap": AggregateStats.from_values(

src/veritext/metrics/readability.py

@@ -5,19 +5,25 @@ import re
 from veritext.metrics.base import AggregateStats, BatchResult
 from veritext.metrics.results import ReadabilityResult
 
+# Sentence-ending punctuation pattern
 _SENTENCE_ENDINGS = re.compile(r"[.!?]+")
+
+# Vowel pattern for syllable counting
 _VOWELS = re.compile(r"[aeiouy]+", re.IGNORECASE)
 
-FK_GRADE_WORDS_PER_SENTENCE = 0.39
-FK_GRADE_SYLLABLES_PER_WORD = 11.8
-FK_GRADE_CONSTANT = 15.59
-
-FRE_CONSTANT = 206.835
-FRE_WORDS_PER_SENTENCE = 1.015
-FRE_SYLLABLES_PER_WORD = 84.6
-
 
 def _count_syllables(word: str) -> int:
+    """
+    Count syllables in a word using a heuristic approach.
+
+    Uses vowel group counting with adjustments for common patterns.
+
+    Args:
+        word: The word to count syllables for.
+
+    Returns:
+        Estimated syllable count (minimum 1 for non-empty words).
+    """
     if not word:
         return 0
 
@@ -25,33 +31,62 @@ def _count_syllables(word: str) -> int:
     if not word:
         return 0
 
+    # Count vowel groups
     vowel_groups = _VOWELS.findall(word)
     count = len(vowel_groups)
 
+    # Adjust for silent 'e' at end
     if word.endswith("e") and count > 1:
         count -= 1
 
+    # Adjust for 'le' ending (e.g., "table", "able")
     if word.endswith("le") and len(word) > 2 and word[-3] not in "aeiouy":
         count += 1
 
+    # Adjust for 'ed' ending when not adding syllable
     if word.endswith("ed") and len(word) > 2 and word[-3] not in "dt":
         count = max(count - 1, 1)
 
+    # Ensure at least 1 syllable for any word
     return max(count, 1)
 
 
 def _count_sentences(text: str) -> int:
+    """
+    Count sentences in text.
+
+    Splits on sentence-ending punctuation (.!?).
+
+    Args:
+        text: The text to count sentences in.
+
+    Returns:
+        Number of sentences (minimum 1 for non-empty text).
+    """
     if not text or not text.strip():
         return 0
 
+    # Split on sentence endings and filter empty strings
     sentences = _SENTENCE_ENDINGS.split(text)
+    # Filter out empty segments
     sentences = [s for s in sentences if s.strip()]
 
     return max(len(sentences), 1)
 
 
 def _count_words(text: str) -> tuple[list[str], int]:
+    """
+    Extract words from text and count them.
+
+    Args:
+        text: The text to process.
+
+    Returns:
+        Tuple of (word list, word count).
+    """
+    # Extract words (sequences of letters and apostrophes)
     words = re.findall(r"[a-zA-Z']+", text)
+    # Filter out standalone apostrophes
     words = [w for w in words if w.replace("'", "")]
     return words, len(words)
 
@@ -69,10 +104,12 @@ class Readability:
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         return "readability"
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         return False
 
     def score(
@@ -90,31 +127,33 @@ class Readability:
         Returns:
             ReadabilityResult with Flesch-Kincaid scores.
         """
+        # Extract words and count
         words, word_count = _count_words(candidate)
 
+        # Handle empty or trivial text
         if word_count == 0:
             return ReadabilityResult(
                 flesch_kincaid_grade=0.0,
                 flesch_reading_ease=0.0,
             )
 
-        sentence_count = max(_count_sentences(candidate), 1)
+        # Count sentences
+        sentence_count = _count_sentences(candidate)
+
+        # Count syllables
         syllable_count = sum(_count_syllables(word) for word in words)
 
+        # Compute ratios
         words_per_sentence = word_count / sentence_count
         syllables_per_word = syllable_count / word_count
 
-        grade_level = (
+        # Flesch-Kincaid Grade Level
-            FK_GRADE_WORDS_PER_SENTENCE * words_per_sentence
+        # Formula: 0.39 * (words/sentences) + 11.8 * (syllables/words) - 15.59
-            + FK_GRADE_SYLLABLES_PER_WORD * syllables_per_word
+        grade_level = 0.39 * words_per_sentence + 11.8 * syllables_per_word - 15.59
-            - FK_GRADE_CONSTANT
-        )
 
-        reading_ease = (
+        # Flesch Reading Ease
-            FRE_CONSTANT
+        # Formula: 206.835 - 1.015 * (words/sentences) - 84.6 * (syllables/words)
-            - FRE_WORDS_PER_SENTENCE * words_per_sentence
+        reading_ease = 206.835 - 1.015 * words_per_sentence - 84.6 * syllables_per_word
-            - FRE_SYLLABLES_PER_WORD * syllables_per_word
-        )
 
         return ReadabilityResult(
             flesch_kincaid_grade=grade_level,
@@ -143,6 +182,7 @@ class Readability:
         for cand in candidates:
             results.append(self.score(cand))
 
+        # Compute aggregate statistics
         stats = {
             "flesch_kincaid_grade": AggregateStats.from_values(
                 [r.flesch_kincaid_grade for r in results]

src/veritext/metrics/results.py

@@ -25,6 +25,7 @@ class BleuResult(BaseModel):
 
     @property
     def score(self) -> float:
+        """Return the composite BLEU-4 score with brevity penalty."""
         return self.bleu4
 
 
@@ -39,10 +40,6 @@ class LexicalResult(BaseModel):
     token_overlap: float
     """Proportion of candidate tokens found in reference."""
 
-    @property
-    def score(self) -> float:
-        return self.jaccard
-
 
 class RougeScore(BaseModel):
     """Individual ROUGE variant score with precision, recall, F-measure."""
@@ -75,6 +72,7 @@ class RougeResult(BaseModel):
 
     @property
     def score(self) -> float:
+        """Return ROUGE-L F-measure as the primary score."""
         return self.rouge_l.fmeasure
 
 
@@ -91,6 +89,7 @@ class ReadabilityResult(BaseModel):
 
     @property
     def score(self) -> float:
+        """Return Flesch reading ease as the primary score."""
         return self.flesch_reading_ease
 
 
@@ -107,4 +106,5 @@ class SemanticResult(BaseModel):
 
     @property
     def score(self) -> float:
+        """Return the primary score for this result."""
         return self.similarity

src/veritext/metrics/rouge.py

@@ -8,6 +8,7 @@ from veritext.metrics.results import RougeResult, RougeScore
 
 
 def _get_ngrams(tokens: list[str], n: int) -> Counter[tuple[str, ...]]:
+    """Extract n-grams from a list of tokens."""
     if n > len(tokens):
         return Counter()
     return Counter(tuple(tokens[i : i + n]) for i in range(len(tokens) - n + 1))
@@ -17,6 +18,7 @@ def _ngram_overlap(
     candidate_ngrams: Counter[tuple[str, ...]],
     reference_ngrams: Counter[tuple[str, ...]],
 ) -> int:
+    """Compute the overlap count between candidate and reference n-grams."""
     overlap = 0
     for ngram, count in candidate_ngrams.items():
         overlap += min(count, reference_ngrams.get(ngram, 0))
@@ -70,11 +72,13 @@ def _lcs_length(seq1: list[str], seq2: list[str]) -> int:
     if not seq1 or not seq2:
         return 0
 
+    # Optimise by using shorter sequence for columns
     if len(seq1) < len(seq2):
         seq1, seq2 = seq2, seq1
 
     m, n = len(seq1), len(seq2)
 
+    # Only need two rows at a time
     prev = [0] * (n + 1)
     curr = [0] * (n + 1)
 
@@ -103,6 +107,9 @@ def _compute_rouge_l(
     Returns:
         RougeScore with precision, recall, and F-measure.
     """
+    if not candidate_tokens and not reference_tokens:
+        return RougeScore(precision=0.0, recall=0.0, fmeasure=0.0)
+
     if not candidate_tokens or not reference_tokens:
         return RougeScore(precision=0.0, recall=0.0, fmeasure=0.0)
 
@@ -120,6 +127,7 @@ def _compute_rouge_l(
 
 
 def _max_rouge_scores(scores: list[RougeScore]) -> RougeScore:
+    """Select the RougeScore with the highest F-measure from a list."""
     return max(scores, key=lambda s: s.fmeasure)
 
 
@@ -142,10 +150,12 @@ class Rouge:
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         return "rouge"
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         return True
 
     def score(
@@ -168,14 +178,18 @@ class Rouge:
         if reference is None:
             raise ValueError("ROUGE requires reference text")
 
+        # Normalise reference to list
         references = [reference] if isinstance(reference, str) else reference
 
+        # Tokenise
         candidate_tokens = self._tokeniser.tokenise(candidate)
         reference_token_lists = [self._tokeniser.tokenise(r) for r in references]
 
+        # Handle empty references
         if all(not ref for ref in reference_token_lists):
             raise ValueError("Reference text cannot be empty")
 
+        # Handle empty candidate
         if not candidate_tokens:
             return RougeResult(
                 rouge1=RougeScore(precision=0.0, recall=0.0, fmeasure=0.0),
@@ -183,6 +197,7 @@ class Rouge:
                 rouge_l=RougeScore(precision=0.0, recall=0.0, fmeasure=0.0),
             )
 
+        # Compute scores for each reference and take max
         rouge1_scores = []
         rouge2_scores = []
         rouge_l_scores = []
@@ -194,9 +209,6 @@ class Rouge:
             rouge2_scores.append(_compute_rouge_score(candidate_tokens, ref_tokens, 2))
             rouge_l_scores.append(_compute_rouge_l(candidate_tokens, ref_tokens))
 
-        if not rouge1_scores:
-            raise ValueError("Reference text cannot be empty")
-
         return RougeResult(
             rouge1=_max_rouge_scores(rouge1_scores),
             rouge2=_max_rouge_scores(rouge2_scores),
@@ -235,6 +247,7 @@ class Rouge:
             ref: str | list[str] = references[i]
             results.append(self.score(cand, ref))
 
+        # Compute aggregate statistics for each score type
         stats = {
             "rouge1_precision": AggregateStats.from_values(
                 [r.rouge1.precision for r in results]

src/veritext/pytest_plugin/__init__.py

@@ -1,22 +0,0 @@
-"""Pytest plugin for text validation.
-
-This plugin provides native pytest integration for Veritext, enabling
-text validation assertions in test suites.
-
-Example:
-    >>> from veritext.pytest_plugin import validate_text
-    >>>
-    >>> def test_summary_quality():
-    ...     text = "The quick brown fox jumps over the lazy dog."
-    ...     validate_text(
-    ...         text,
-    ...         min_length=10,
-    ...         max_length=100,
-    ...         max_reading_grade=8.0,
-    ...     )
-"""
-
-from veritext.pytest_plugin.assertions import validate_text
-from veritext.pytest_plugin.plugin import pytest_configure
-
-__all__ = ["pytest_configure", "validate_text"]

src/veritext/pytest_plugin/assertions.py

@@ -1,135 +0,0 @@
-"""Assertion functions for text validation in pytest."""
-
-from typing import TYPE_CHECKING
-
-from veritext.core.types import ValidationContext, ValidationResult
-from veritext.validators import all_of
-
-if TYPE_CHECKING:
-    from veritext.validators.base import Check
-
-
-def validate_text(
-    text: str,
-    *,
-    reference: str | list[str] | None = None,
-    min_bleu: float | None = None,
-    min_rouge: float | None = None,
-    min_semantic: float | None = None,
-    max_length: int | None = None,
-    min_length: int | None = None,
-    max_reading_grade: float | None = None,
-    must_contain: list[str] | None = None,
-    must_exclude: list[str] | None = None,
-) -> None:
-    """Assert text passes all specified validation criteria.
-
-    This is the primary assertion function for text validation in pytest.
-    It builds validators from keyword arguments and raises AssertionError
-    with detailed failure information if validation fails.
-
-    Args:
-        text: The text to validate.
-        reference: Reference text for comparison metrics (BLEU, ROUGE, semantic).
-        min_bleu: Minimum BLEU-4 score required (0.0 to 1.0).
-        min_rouge: Minimum ROUGE-L F-measure required (0.0 to 1.0).
-        min_semantic: Minimum semantic similarity required (0.0 to 1.0).
-        max_length: Maximum character count allowed.
-        min_length: Minimum character count required.
-        max_reading_grade: Maximum Flesch-Kincaid grade level.
-        must_contain: Patterns that must be present in the text.
-        must_exclude: Patterns that must not be present in the text.
-
-    Raises:
-        AssertionError: With detailed failure information if validation fails.
-        ValueError: If comparison metrics requested but reference not provided,
-            or if no validation criteria are specified.
-
-    Example:
-        >>> validate_text(
-        ...     "The quick brown fox jumps over the lazy dog.",
-        ...     min_length=10,
-        ...     max_length=100,
-        ...     max_reading_grade=8.0,
-        ... )
-    """
-    if any([min_bleu, min_rouge, min_semantic]) and reference is None:
-        raise ValueError(
-            "Reference text required for comparison metrics "
-            "(min_bleu, min_rouge, min_semantic)"
-        )
-
-    checks: list[Check] = []
-
-    if min_bleu is not None:
-        from veritext.validators import bleu
-
-        checks.append(bleu(min_score=min_bleu))
-
-    if min_rouge is not None:
-        from veritext.validators import rouge
-
-        checks.append(rouge(min_score=min_rouge))
-
-    if min_semantic is not None:
-        from veritext.validators import semantic
-
-        checks.append(semantic(min_score=min_semantic))
-
-    if max_length is not None or min_length is not None:
-        from veritext.validators import length
-
-        checks.append(length(min_chars=min_length, max_chars=max_length))
-
-    if max_reading_grade is not None:
-        from veritext.validators import readability
-
-        checks.append(readability(max_grade=max_reading_grade))
-
-    if must_contain is not None:
-        from veritext.validators import contains
-
-        checks.append(contains(patterns=must_contain))
-
-    if must_exclude is not None:
-        from veritext.validators import excludes
-
-        checks.append(excludes(patterns=must_exclude))
-
-    if not checks:
-        raise ValueError("At least one validation criterion must be specified")
-
-    context = ValidationContext(reference=reference)
-    validator = all_of(checks)
-    result = validator.check(text, context)
-
-    if not result.passed:
-        raise AssertionError(_format_failure(text, result))
-
-
-def _format_failure(text: str, result: ValidationResult) -> str:
-    """Format a detailed failure message for pytest output.
-
-    Args:
-        text: The text that was validated.
-        result: The validation result containing check failures.
-
-    Returns:
-        Formatted failure message with check details.
-    """
-    lines = ["Text validation failed:"]
-    lines.append("")
-
-    preview = text[:100] + "..." if len(text) > 100 else text
-    lines.append(f"  Text: {preview!r}")
-    lines.append("")
-
-    lines.append("  Failed checks:")
-    for check in result.failed_checks:
-        lines.append(f"    - {check.name}:")
-        lines.append(f"        {check.message}")
-        if check.threshold is not None:
-            lines.append(f"        Expected: >= {check.threshold}")
-            lines.append(f"        Actual:   {check.actual}")
-
-    return "\n".join(lines)

src/veritext/pytest_plugin/fixtures.py

@@ -1,80 +0,0 @@
-"""Pytest fixtures for text validation."""
-
-from typing import TYPE_CHECKING, Any
-
-import pytest
-
-from veritext.core.types import ValidationContext, ValidationResult
-from veritext.validators import all_of
-from veritext.validators.base import Check
-
-if TYPE_CHECKING:
-    from collections.abc import Callable
-
-
-class ValidatorFactory:
-    """Factory for building validators from keyword arguments."""
-
-    def __call__(
-        self,
-        checks: list[Check],
-        reference: str | list[str] | None = None,
-    ) -> "Callable[[str], ValidationResult]":
-        """Create a validator function from a list of checks.
-
-        Args:
-            checks: List of validation checks to apply.
-            reference: Optional reference text for comparison metrics.
-
-        Returns:
-            A callable that takes text and returns a ValidationResult.
-        """
-        validator = all_of(checks)
-        context = ValidationContext(reference=reference)
-
-        def validate(text: str) -> ValidationResult:
-            return validator.check(text, context)
-
-        return validate
-
-
-@pytest.fixture
-def text_validator() -> ValidatorFactory:
-    """Provide a factory for building validators.
-
-    Example:
-        >>> def test_with_factory(text_validator):
-        ...     from veritext.validators import bleu, length
-        ...     validate = text_validator(
-        ...         checks=[bleu(min_score=0.5), length(min_words=10)],
-        ...         reference="The reference text.",
-        ...     )
-        ...     result = validate("Some candidate text.")
-        ...     assert result.passed
-
-    Returns:
-        ValidatorFactory instance.
-    """
-    return ValidatorFactory()
-
-
-@pytest.fixture
-def validation_context() -> "Callable[..., ValidationContext]":
-    """Provide a factory for creating ValidationContext objects.
-
-    Example:
-        >>> def test_with_context(validation_context):
-        ...     ctx = validation_context(reference="The reference text.")
-        ...     assert ctx.reference == "The reference text."
-
-    Returns:
-        A callable that creates ValidationContext objects.
-    """
-
-    def _create(
-        reference: str | list[str] | None = None,
-        **metadata: Any,
-    ) -> ValidationContext:
-        return ValidationContext(reference=reference, metadata=metadata)
-
-    return _create

src/veritext/pytest_plugin/plugin.py

@@ -1,18 +0,0 @@
-"""Pytest hooks for Veritext plugin."""
-
-from typing import TYPE_CHECKING
-
-if TYPE_CHECKING:
-    import pytest
-
-
-def pytest_configure(config: "pytest.Config") -> None:
-    """Register Veritext markers.
-
-    Args:
-        config: Pytest configuration object.
-    """
-    config.addinivalue_line(
-        "markers",
-        "text_validation: mark test as a text validation test",
-    )

src/veritext/semantic/similarity.py

@@ -1,14 +1,11 @@
 """Embedding-based semantic similarity using sentence-transformers."""
 
-from collections import OrderedDict
 from typing import Any
 
 from veritext.core.exceptions import DependencyError
 from veritext.metrics.base import AggregateStats, BatchResult
 from veritext.metrics.results import SemanticResult
 
-DEFAULT_CACHE_MAX_SIZE = 1000
-
 
 class SemanticSimilarity:
     """
@@ -24,7 +21,6 @@ class SemanticSimilarity:
         self,
         model: str = "all-MiniLM-L6-v2",
         cache_embeddings: bool = True,
-        cache_max_size: int = DEFAULT_CACHE_MAX_SIZE,
     ) -> None:
         """
         Initialise the semantic similarity metric.
@@ -34,8 +30,6 @@ class SemanticSimilarity:
                    Defaults to "all-MiniLM-L6-v2" (22MB, good quality/size tradeoff).
             cache_embeddings: Whether to cache embeddings for repeated texts.
                               Defaults to True.
-            cache_max_size: Maximum number of embeddings to cache. Oldest entries
-                            are evicted when the limit is reached. Defaults to 1000.
 
         Raises:
             DependencyError: If sentence-transformers is not installed.
@@ -50,37 +44,53 @@ class SemanticSimilarity:
 
         self._model_name = model
         self._model: Any = SentenceTransformer(model)
-        self._cache: OrderedDict[str, Any] | None = (
+        self._cache: dict[str, Any] | None = {} if cache_embeddings else None
-            OrderedDict() if cache_embeddings else None
-        )
-        self._cache_max_size = cache_max_size
 
     @property
     def name(self) -> str:
+        """Return the name of this metric."""
         return "semantic"
 
     @property
     def requires_reference(self) -> bool:
+        """Return whether this metric requires reference text."""
         return True
 
     def _get_embedding(self, text: str) -> Any:
+        """
+        Get embedding for text, using cache if available.
+
+        Args:
+            text: The text to embed.
+
+        Returns:
+            The embedding tensor.
+        """
         if self._cache is not None and text in self._cache:
-            self._cache.move_to_end(text)
             return self._cache[text]
 
         embedding = self._model.encode(text, convert_to_tensor=True)
 
         if self._cache is not None:
-            while len(self._cache) >= self._cache_max_size:
-                self._cache.popitem(last=False)
             self._cache[text] = embedding
 
         return embedding
 
     def _cosine_similarity(self, embedding1: Any, embedding2: Any) -> float:
+        """
+        Compute cosine similarity between two embeddings.
+
+        Args:
+            embedding1: First embedding tensor.
+            embedding2: Second embedding tensor.
+
+        Returns:
+            Cosine similarity score (0.0 to 1.0).
+        """
         from sentence_transformers import util
 
         similarity: float = util.cos_sim(embedding1, embedding2).item()
+        # Clamp to [0, 1] as negative similarities are possible but not meaningful
         return max(0.0, min(1.0, similarity))
 
     def score(
@@ -105,21 +115,26 @@ class SemanticSimilarity:
         if reference is None:
             raise ValueError("Semantic similarity requires reference text")
 
+        # Normalise reference to list
         references = [reference] if isinstance(reference, str) else reference
 
         if not references:
             raise ValueError("Reference text cannot be empty")
 
+        # Handle empty candidate
         candidate_stripped = candidate.strip()
         if not candidate_stripped:
             return SemanticResult(similarity=0.0, model=self._model_name)
 
+        # Handle empty references
         valid_references = [r for r in references if r.strip()]
         if not valid_references:
             raise ValueError("Reference text cannot be empty")
 
+        # Get candidate embedding
         candidate_embedding = self._get_embedding(candidate_stripped)
 
+        # Compute similarity against each reference, take maximum
         max_similarity = 0.0
         for ref in valid_references:
             ref_embedding = self._get_embedding(ref.strip())
@@ -160,6 +175,7 @@ class SemanticSimilarity:
             ref: str | list[str] = references[i]
             results.append(self.score(cand, ref))
 
+        # Compute aggregate statistics
         stats = {
             "similarity": AggregateStats.from_values([r.similarity for r in results]),
         }
@@ -167,5 +183,6 @@ class SemanticSimilarity:
         return BatchResult(results=results, count=len(results), stats=stats)
 
     def clear_cache(self) -> None:
+        """Clear the embedding cache."""
         if self._cache is not None:
             self._cache.clear()

src/veritext/validators/__init__.py

@@ -35,6 +35,7 @@ from veritext.validators.metric import (
 )
 
 
+# Factory functions for clean API
 def bleu(
     min_score: float,
     variant: Literal[1, 2, 3, 4] = 4,

src/veritext/validators/base.py

@@ -14,7 +14,9 @@ class Check(Protocol):
     """
 
     @property
-    def name(self) -> str: ...
+    def name(self) -> str:
+        """Return the name of this check."""
+        ...
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:
         """Run the check and return a result.

src/veritext/validators/composite.py

@@ -1,20 +1,11 @@
-"""Composite validators for combining multiple checks.
+"""Composite validators for combining multiple checks."""
-
-Note: CompositeCheck classes (AllOf, AnyOf) intentionally return ValidationResult
-rather than CheckResult. This allows callers to inspect individual check results
-for detailed error reporting. They implement a compatible interface but are not
-substitutable where Check is expected as a type constraint.
-"""
 
 from veritext.core.types import CheckResult, ValidationContext, ValidationResult
 from veritext.validators.base import Check
 
 
 class AllOf:
-    """Passes only if all checks pass.
+    """Passes only if all checks pass."""
-
-    Note: Returns ValidationResult (not CheckResult) to expose child results.
-    """
 
     def __init__(self, checks: list[Check]) -> None:
         """
@@ -29,10 +20,11 @@ class AllOf:
         if not checks:
             raise ValueError("checks list cannot be empty")
 
-        self._checks = list(checks)
+        self._checks = checks
 
     @property
     def name(self) -> str:
+        """Return the name of this composite check."""
         return "all_of"
 
     def check(self, text: str, context: ValidationContext) -> ValidationResult:
@@ -56,10 +48,7 @@ class AllOf:
 
 
 class AnyOf:
-    """Passes if any check passes.
+    """Passes if any check passes."""
-
-    Note: Returns ValidationResult (not CheckResult) to expose child results.
-    """
 
     def __init__(self, checks: list[Check]) -> None:
         """
@@ -74,10 +63,11 @@ class AnyOf:
         if not checks:
             raise ValueError("checks list cannot be empty")
 
-        self._checks = list(checks)
+        self._checks = checks
 
     @property
     def name(self) -> str:
+        """Return the name of this composite check."""
         return "any_of"
 
     def check(self, text: str, context: ValidationContext) -> ValidationResult:

src/veritext/validators/constraint.py

@@ -61,6 +61,7 @@ class LengthValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "length"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:  # noqa: ARG002
@@ -145,6 +146,7 @@ class ReadabilityValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "readability"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:  # noqa: ARG002
@@ -211,16 +213,6 @@ class ReadabilityValidator:
         )
 
 
-def _compile_patterns(patterns: list[str], flags: int) -> list[re.Pattern[str]]:
-    compiled = []
-    for pattern in patterns:
-        try:
-            compiled.append(re.compile(pattern, flags))
-        except re.error as e:
-            raise InvalidThresholdError(f"Invalid regex pattern '{pattern}': {e}") from e
-    return compiled
-
-
 class ContainsValidator:
     """Validates text contains required patterns."""
 
@@ -237,7 +229,7 @@ class ContainsValidator:
             case_sensitive: Whether matching is case-sensitive. Defaults to False.
 
         Raises:
-            InvalidThresholdError: If patterns list is empty or contains invalid regex.
+            InvalidThresholdError: If patterns list is empty.
         """
         if not patterns:
             raise InvalidThresholdError("patterns list cannot be empty")
@@ -245,10 +237,10 @@ class ContainsValidator:
         self._patterns = patterns
         self._case_sensitive = case_sensitive
         self._flags = 0 if case_sensitive else re.IGNORECASE
-        self._compiled_patterns = _compile_patterns(patterns, self._flags)
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "contains"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:  # noqa: ARG002
@@ -263,10 +255,8 @@ class ContainsValidator:
             CheckResult with pass/fail status.
         """
         missing = []
-        for pattern, compiled in zip(
+        for pattern in self._patterns:
-            self._patterns, self._compiled_patterns, strict=True
+            if not re.search(pattern, text, self._flags):
-        ):
-            if not compiled.search(text):
                 missing.append(pattern)
 
         passed = len(missing) == 0
@@ -301,7 +291,7 @@ class ExcludesValidator:
             case_sensitive: Whether matching is case-sensitive. Defaults to False.
 
         Raises:
-            InvalidThresholdError: If patterns list is empty or contains invalid regex.
+            InvalidThresholdError: If patterns list is empty.
         """
         if not patterns:
             raise InvalidThresholdError("patterns list cannot be empty")
@@ -309,10 +299,10 @@ class ExcludesValidator:
         self._patterns = patterns
         self._case_sensitive = case_sensitive
         self._flags = 0 if case_sensitive else re.IGNORECASE
-        self._compiled_patterns = _compile_patterns(patterns, self._flags)
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "excludes"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:  # noqa: ARG002
@@ -327,10 +317,8 @@ class ExcludesValidator:
             CheckResult with pass/fail status.
         """
         found = []
-        for pattern, compiled in zip(
+        for pattern in self._patterns:
-            self._patterns, self._compiled_patterns, strict=True
+            if re.search(pattern, text, self._flags):
-        ):
-            if compiled.search(text):
                 found.append(pattern)
 
         passed = len(found) == 0

src/veritext/validators/metric.py

@@ -43,6 +43,7 @@ class BleuValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return f"bleu-{self._variant}"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:
@@ -64,6 +65,7 @@ class BleuValidator:
 
         result = self._metric.score(text, context.reference)
 
+        # Select the appropriate BLEU variant
         score_map = {
             1: result.bleu1,
             2: result.bleu2,
@@ -128,6 +130,7 @@ class RougeValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return f"rouge-{self._variant}"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:
@@ -149,6 +152,7 @@ class RougeValidator:
 
         result = self._metric.score(text, context.reference)
 
+        # Select the appropriate ROUGE variant (use F-measure)
         score_map = {
             "1": result.rouge1.fmeasure,
             "2": result.rouge2.fmeasure,
@@ -218,6 +222,7 @@ class LexicalValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "lexical"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:
@@ -239,6 +244,7 @@ class LexicalValidator:
 
         result = self._metric.score(text, context.reference)
 
+        # Check each threshold that was specified
         failures = []
         if self._min_jaccard is not None and result.jaccard < self._min_jaccard:
             failures.append(
@@ -265,6 +271,7 @@ class LexicalValidator:
         else:
             message = "Lexical similarity: " + "; ".join(failures)
 
+        # Build actual value dict
         actual = {"jaccard": result.jaccard, "token_overlap": result.token_overlap}
         threshold = {}
         if self._min_jaccard is not None:
@@ -311,6 +318,7 @@ class SemanticValidator:
             )
 
         self._min_score = min_score
+        # Lazy import to avoid loading PyTorch unless needed
         from veritext.semantic.similarity import SemanticSimilarity
 
         self._metric: SemanticSimilarity = SemanticSimilarity(
@@ -319,6 +327,7 @@ class SemanticValidator:
 
     @property
     def name(self) -> str:
+        """Return the name of this check."""
         return "semantic"
 
     def check(self, text: str, context: ValidationContext) -> CheckResult:

tests/conftest.py

@@ -7,14 +7,17 @@ from veritext.core.tokenisation import WordTokeniser
 
 @pytest.fixture
 def word_tokeniser() -> WordTokeniser:
+    """Provide a default word tokeniser."""
     return WordTokeniser()
 
 
 @pytest.fixture
 def word_tokeniser_no_lowercase() -> WordTokeniser:
+    """Provide a word tokeniser without lowercasing."""
     return WordTokeniser(lowercase=False)
 
 
 @pytest.fixture
 def word_tokeniser_keep_punctuation() -> WordTokeniser:
+    """Provide a word tokeniser that keeps punctuation."""
     return WordTokeniser(remove_punctuation=False)

tests/test_benchmark/__init__.py

@@ -1 +0,0 @@
-"""Tests for the benchmark module."""

tests/test_benchmark/test_models.py

@@ -1,132 +0,0 @@
-"""Tests for benchmark data models."""
-
-from datetime import UTC, datetime
-
-import pytest
-from pydantic import ValidationError
-
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-
-
-class TestBenchmarkRun:
-    def test_create_benchmark_run(self) -> None:
-        run = BenchmarkRun(
-            id="test-id-123",
-            benchmark_name="test-benchmark",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0-dev",
-            metrics={"bleu4": 0.75, "rouge_l": 0.82},
-            sample_count=100,
-        )
-
-        assert run.id == "test-id-123"
-        assert run.benchmark_name == "test-benchmark"
-        assert run.veritext_version == "0.1.0-dev"
-        assert run.metrics == {"bleu4": 0.75, "rouge_l": 0.82}
-        assert run.sample_count == 100
-        assert run.metadata == {}
-
-    def test_create_with_metadata(self) -> None:
-        run = BenchmarkRun(
-            id="test-id-456",
-            benchmark_name="test-benchmark",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0-dev",
-            metrics={"bleu4": 0.75},
-            sample_count=50,
-            metadata={"git_sha": "abc123", "model_version": "gpt-4"},
-        )
-
-        assert run.metadata == {"git_sha": "abc123", "model_version": "gpt-4"}
-
-    def test_frozen_model(self) -> None:
-        run = BenchmarkRun(
-            id="test-id",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-
-        with pytest.raises(ValidationError):
-            run.id = "new-id"  # type: ignore[misc]
-
-    def test_serialisation(self) -> None:
-        run = BenchmarkRun(
-            id="test-id",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-
-        data = run.model_dump()
-        assert data["id"] == "test-id"
-        assert data["benchmark_name"] == "test"
-        assert data["metrics"] == {"bleu4": 0.5}
-
-
-class TestRegressionReport:
-    def test_no_regression_summary(self) -> None:
-        report = RegressionReport(
-            detected=False,
-            baseline={"bleu4": 0.75, "rouge_l": 0.80},
-            current={"bleu4": 0.76, "rouge_l": 0.81},
-            deltas={"bleu4": 0.01, "rouge_l": 0.01},
-            tolerance=0.05,
-        )
-
-        assert "No regression detected" in report.summary
-
-    def test_regression_summary(self) -> None:
-        report = RegressionReport(
-            detected=True,
-            baseline={"bleu4": 0.75, "rouge_l": 0.80},
-            current={"bleu4": 0.65, "rouge_l": 0.78},
-            deltas={"bleu4": -0.10, "rouge_l": -0.02},
-            tolerance=0.05,
-        )
-
-        assert "Regression detected" in report.summary
-        assert "bleu4" in report.summary
-        assert "0.6500" in report.summary
-        assert "baseline: 0.7500" in report.summary
-
-    def test_regression_within_tolerance(self) -> None:
-        report = RegressionReport(
-            detected=True,
-            baseline={"bleu4": 0.75, "rouge_l": 0.80},
-            current={"bleu4": 0.65, "rouge_l": 0.78},
-            deltas={"bleu4": -0.10, "rouge_l": -0.02},
-            tolerance=0.05,
-        )
-
-        # rouge_l is -0.02, within tolerance of 0.05, so shouldn't appear
-        assert "rouge_l" not in report.summary
-        # bleu4 is -0.10, exceeds tolerance, so should appear
-        assert "bleu4" in report.summary
-
-    def test_frozen_model(self) -> None:
-        report = RegressionReport(
-            detected=False,
-            baseline={},
-            current={},
-            deltas={},
-            tolerance=0.05,
-        )
-
-        with pytest.raises(ValidationError):
-            report.detected = True  # type: ignore[misc]
-
-    def test_tolerance_in_summary(self) -> None:
-        report = RegressionReport(
-            detected=True,
-            baseline={"metric": 0.80},
-            current={"metric": 0.50},
-            deltas={"metric": -0.30},
-            tolerance=0.10,
-        )
-
-        assert "10.00%" in report.summary

tests/test_benchmark/test_regression.py

@@ -1,207 +0,0 @@
-"""Tests for regression detection."""
-
-from datetime import UTC, datetime
-
-import pytest
-
-from veritext.benchmark.models import BenchmarkRun
-from veritext.benchmark.regression import compute_baseline, detect_regression
-
-
-def make_run(
-    run_id: str,
-    metrics: dict[str, float],
-    day: int = 1,
-) -> BenchmarkRun:
-    return BenchmarkRun(
-        id=run_id,
-        benchmark_name="test",
-        timestamp=datetime(2025, 1, day, 12, 0, 0, tzinfo=UTC),
-        veritext_version="0.1.0",
-        metrics=metrics,
-        sample_count=10,
-    )
-
-
-class TestComputeBaseline:
-    def test_empty_runs(self) -> None:
-        baseline = compute_baseline([])
-        assert baseline == {}
-
-    def test_single_run(self) -> None:
-        runs = [make_run("r1", {"bleu4": 0.75, "rouge_l": 0.80})]
-
-        baseline = compute_baseline(runs)
-
-        assert baseline["bleu4"] == 0.75
-        assert baseline["rouge_l"] == 0.80
-
-    def test_multiple_runs_average(self) -> None:
-        runs = [
-            make_run("r1", {"bleu4": 0.70}, day=3),
-            make_run("r2", {"bleu4": 0.80}, day=2),
-            make_run("r3", {"bleu4": 0.90}, day=1),
-        ]
-
-        baseline = compute_baseline(runs, window=3)
-
-        assert baseline["bleu4"] == pytest.approx(0.80)  # (0.70+0.80+0.90)/3
-
-    def test_window_limits_runs(self) -> None:
-        runs = [
-            make_run("r1", {"bleu4": 0.70}, day=5),  # most recent
-            make_run("r2", {"bleu4": 0.80}, day=4),
-            make_run("r3", {"bleu4": 0.90}, day=3),
-            make_run("r4", {"bleu4": 0.60}, day=2),  # excluded
-            make_run("r5", {"bleu4": 0.50}, day=1),  # excluded
-        ]
-
-        baseline = compute_baseline(runs, window=3)
-
-        # Only first 3 runs: (0.70 + 0.80 + 0.90) / 3 = 0.80
-        assert baseline["bleu4"] == pytest.approx(0.80)
-
-    def test_partial_history(self) -> None:
-        runs = [
-            make_run("r1", {"bleu4": 0.70}),
-            make_run("r2", {"bleu4": 0.80}),
-        ]
-
-        baseline = compute_baseline(runs, window=10)
-
-        # Only 2 runs available: (0.70 + 0.80) / 2 = 0.75
-        assert baseline["bleu4"] == pytest.approx(0.75)
-
-    def test_multiple_metrics(self) -> None:
-        runs = [
-            make_run("r1", {"bleu4": 0.70, "rouge_l": 0.75}),
-            make_run("r2", {"bleu4": 0.80, "rouge_l": 0.85}),
-        ]
-
-        baseline = compute_baseline(runs)
-
-        assert baseline["bleu4"] == pytest.approx(0.75)
-        assert baseline["rouge_l"] == pytest.approx(0.80)
-
-    def test_varying_metrics(self) -> None:
-        runs = [
-            make_run("r1", {"bleu4": 0.70, "rouge_l": 0.75}),
-            make_run("r2", {"bleu4": 0.80}),  # No rouge_l
-        ]
-
-        baseline = compute_baseline(runs)
-
-        # bleu4 appears in both runs
-        assert baseline["bleu4"] == pytest.approx(0.75)
-        # rouge_l only appears in one run
-        assert baseline["rouge_l"] == pytest.approx(0.75)
-
-
-class TestDetectRegression:
-    def test_no_baseline(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.70},
-            baseline={},
-            tolerance=0.05,
-        )
-
-        assert not report.detected
-        assert report.deltas == {}
-
-    def test_no_regression_stable(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.75},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        assert not report.detected
-        assert report.deltas["bleu4"] == pytest.approx(0.0)
-
-    def test_no_regression_improved(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.85},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        assert not report.detected
-        assert report.deltas["bleu4"] == pytest.approx(0.10)
-
-    def test_no_regression_within_tolerance(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.73},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        assert not report.detected
-        assert report.deltas["bleu4"] == pytest.approx(-0.02)
-
-    def test_regression_detected(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.65},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        assert report.detected
-        assert report.deltas["bleu4"] == pytest.approx(-0.10)
-
-    def test_regression_at_tolerance_boundary(self) -> None:
-        # Use a value clearly at the boundary (accounting for float precision)
-        # The implementation checks delta < -tolerance (strictly less than)
-        report = detect_regression(
-            current={"bleu4": 0.50},
-            baseline={"bleu4": 0.50},
-            tolerance=0.05,
-        )
-
-        # Delta is 0.0, well within tolerance
-        assert not report.detected
-        assert report.deltas["bleu4"] == 0.0
-
-    def test_regression_just_beyond_tolerance(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.6999},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        # Delta is -0.0501, which is < -tolerance
-        assert report.detected
-
-    def test_multiple_metrics_any_regresses(self) -> None:
-        report = detect_regression(
-            current={"bleu4": 0.65, "rouge_l": 0.80},
-            baseline={"bleu4": 0.75, "rouge_l": 0.80},
-            tolerance=0.05,
-        )
-
-        assert report.detected
-        # Only bleu4 regressed
-        assert report.deltas["bleu4"] == pytest.approx(-0.10)
-        assert report.deltas["rouge_l"] == pytest.approx(0.0)
-
-    def test_report_contains_all_values(self) -> None:
-        baseline = {"bleu4": 0.75, "rouge_l": 0.80}
-        current = {"bleu4": 0.65, "rouge_l": 0.82}
-
-        report = detect_regression(current, baseline, tolerance=0.05)
-
-        assert report.baseline == baseline
-        assert report.current == current
-        assert report.tolerance == 0.05
-        assert "bleu4" in report.deltas
-        assert "rouge_l" in report.deltas
-
-    def test_missing_metric_in_current(self) -> None:
-        report = detect_regression(
-            current={},
-            baseline={"bleu4": 0.75},
-            tolerance=0.05,
-        )
-
-        # 0.0 - 0.75 = -0.75, which is a regression
-        assert report.detected
-        assert report.deltas["bleu4"] == pytest.approx(-0.75)

tests/test_benchmark/test_runner.py

@@ -1,218 +0,0 @@
-"""Tests for benchmark runner."""
-
-from pathlib import Path
-
-import pytest
-
-from veritext.benchmark.models import BenchmarkRun
-from veritext.benchmark.runner import Benchmark
-from veritext.core.exceptions import RegressionDetectedError
-
-
-@pytest.fixture
-def benchmark(tmp_path: Path) -> Benchmark:
-    return Benchmark("test-suite", storage_path=tmp_path / "benchmarks")
-
-
-@pytest.fixture
-def sample_data() -> tuple[list[str], list[str]]:
-    candidates = [
-        "The quick brown fox jumps over the lazy dog.",
-        "A fast auburn fox leaps above the sleepy hound.",
-    ]
-    references = [
-        "The quick brown fox jumps over the lazy dog.",
-        "The swift brown fox jumps over the lazy dog.",
-    ]
-    return candidates, references
-
-
-class TestBenchmarkInit:
-    def test_creates_storage_directory(self, tmp_path: Path) -> None:
-        storage_path = tmp_path / "benchmarks"
-        Benchmark("my-suite", storage_path=storage_path)
-
-        assert storage_path.exists()
-
-    def test_name_property(self, benchmark: Benchmark) -> None:
-        assert benchmark.name == "test-suite"
-
-
-class TestEvaluate:
-    def test_evaluate_stores_run(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        run = benchmark.evaluate(candidates, references)
-
-        assert isinstance(run, BenchmarkRun)
-        assert run.benchmark_name == "test-suite"
-        assert run.sample_count == 2
-
-    def test_evaluate_returns_metrics(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        run = benchmark.evaluate(candidates, references)
-
-        # Default metrics are rouge_l and bleu4
-        assert "rouge_l" in run.metrics
-        assert "bleu4" in run.metrics
-        assert 0.0 <= run.metrics["rouge_l"] <= 1.0
-        assert 0.0 <= run.metrics["bleu4"] <= 1.0
-
-    def test_evaluate_custom_metrics(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        run = benchmark.evaluate(
-            candidates, references, metrics=["bleu1", "bleu2", "rouge1"]
-        )
-
-        assert "bleu1" in run.metrics
-        assert "bleu2" in run.metrics
-        assert "rouge1" in run.metrics
-        assert "bleu4" not in run.metrics  # Not requested
-
-    def test_evaluate_with_metadata(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        run = benchmark.evaluate(
-            candidates, references, metadata={"git_sha": "abc123", "model": "gpt-4"}
-        )
-
-        assert run.metadata == {"git_sha": "abc123", "model": "gpt-4"}
-
-    def test_evaluate_stores_retrievable(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-        run = benchmark.evaluate(candidates, references)
-
-        history = benchmark.get_history()
-
-        assert len(history) == 1
-        assert history[0].id == run.id
-
-
-class TestCheckRegression:
-    def test_check_no_runs(self, benchmark: Benchmark) -> None:
-        report = benchmark.check_regression()
-
-        assert not report.detected
-        assert report.baseline == {}
-        assert report.current == {}
-
-    def test_check_single_run(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-        benchmark.evaluate(candidates, references)
-
-        report = benchmark.check_regression()
-
-        # First run has no baseline to compare against
-        assert not report.detected
-
-    def test_check_stable_metrics(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        # Run multiple times with same data
-        for _ in range(3):
-            benchmark.evaluate(candidates, references)
-
-        report = benchmark.check_regression()
-        assert not report.detected
-
-    def test_check_reports_regression(self, tmp_path: Path) -> None:
-        benchmark = Benchmark("regress-test", storage_path=tmp_path / "benchmarks")
-
-        # First run with good metrics
-        good_candidates = ["The quick brown fox jumps."]
-        good_references = ["The quick brown fox jumps."]
-        benchmark.evaluate(good_candidates, good_references)
-
-        # Second run with worse metrics (different text)
-        bad_candidates = ["Something completely different here."]
-        benchmark.evaluate(bad_candidates, good_references)
-
-        report = benchmark.check_regression(tolerance=0.05)
-
-        # Should detect regression since second run is very different
-        assert report.detected or any(d < -0.05 for d in report.deltas.values())
-
-
-class TestAssertNoRegression:
-    def test_passes_when_stable(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        for _ in range(3):
-            benchmark.evaluate(candidates, references)
-
-        # Should not raise
-        benchmark.assert_no_regression()
-
-    def test_raises_on_regression(self, tmp_path: Path) -> None:
-        benchmark = Benchmark("regress-test", storage_path=tmp_path / "benchmarks")
-
-        # Establish baseline with perfect match
-        perfect = ["The quick brown fox."]
-        benchmark.evaluate(perfect, perfect)
-
-        # Second run with terrible match
-        terrible = ["Completely unrelated text."]
-        benchmark.evaluate(terrible, perfect)
-
-        with pytest.raises(RegressionDetectedError):
-            benchmark.assert_no_regression(tolerance=0.05)
-
-
-class TestGetHistory:
-    def test_empty_history(self, benchmark: Benchmark) -> None:
-        history = benchmark.get_history()
-        assert history == []
-
-    def test_returns_runs(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        run1 = benchmark.evaluate(candidates, references)
-        run2 = benchmark.evaluate(candidates, references)
-
-        history = benchmark.get_history()
-
-        assert len(history) == 2
-        assert history[0].id == run2.id  # Most recent first
-        assert history[1].id == run1.id
-
-    def test_respects_limit(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        for _ in range(5):
-            benchmark.evaluate(candidates, references)
-
-        history = benchmark.get_history(limit=3)
-        assert len(history) == 3
-
-    def test_default_limit(
-        self, benchmark: Benchmark, sample_data: tuple[list[str], list[str]]
-    ) -> None:
-        candidates, references = sample_data
-
-        for _ in range(25):
-            benchmark.evaluate(candidates, references)
-
-        history = benchmark.get_history()
-        assert len(history) == 20

tests/test_benchmark/test_storage.py

@@ -1,268 +0,0 @@
-"""Tests for benchmark SQLite storage."""
-
-import sqlite3
-import threading
-from datetime import UTC, datetime
-from pathlib import Path
-
-import pytest
-
-from veritext.benchmark.models import BenchmarkRun
-from veritext.benchmark.storage import BenchmarkStorage
-from veritext.core.exceptions import StorageError
-
-
-@pytest.fixture
-def db_path(tmp_path: Path) -> Path:
-    return tmp_path / "benchmarks" / "test.db"
-
-
-@pytest.fixture
-def storage(db_path: Path) -> BenchmarkStorage:
-    return BenchmarkStorage(db_path)
-
-
-@pytest.fixture
-def sample_run() -> BenchmarkRun:
-    return BenchmarkRun(
-        id="run-001",
-        benchmark_name="test-suite",
-        timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-        veritext_version="0.1.0-dev",
-        metrics={"bleu4": 0.75, "rouge_l": 0.82},
-        sample_count=100,
-        metadata={"git_sha": "abc123"},
-    )
-
-
-class TestDatabaseCreation:
-    def test_creates_database_file(self, db_path: Path) -> None:
-        assert not db_path.exists()
-        BenchmarkStorage(db_path)
-        assert db_path.exists()
-
-    def test_creates_parent_directories(self, tmp_path: Path) -> None:
-        nested_path = tmp_path / "deep" / "nested" / "path" / "test.db"
-        BenchmarkStorage(nested_path)
-        assert nested_path.exists()
-
-    def test_creates_tables(self, db_path: Path) -> None:
-        BenchmarkStorage(db_path)
-
-        conn = sqlite3.connect(str(db_path))
-        cursor = conn.execute("SELECT name FROM sqlite_master WHERE type='table'")
-        tables = {row[0] for row in cursor.fetchall()}
-        conn.close()
-
-        assert "benchmark_runs" in tables
-        assert "benchmark_metrics" in tables
-
-    def test_creates_index(self, db_path: Path) -> None:
-        BenchmarkStorage(db_path)
-
-        conn = sqlite3.connect(str(db_path))
-        cursor = conn.execute("SELECT name FROM sqlite_master WHERE type='index'")
-        indices = {row[0] for row in cursor.fetchall()}
-        conn.close()
-
-        assert "idx_benchmark_name" in indices
-
-
-class TestSaveRun:
-    def test_save_run(
-        self, storage: BenchmarkStorage, sample_run: BenchmarkRun
-    ) -> None:
-        storage.save_run(sample_run)
-
-        runs = storage.get_runs("test-suite")
-        assert len(runs) == 1
-        assert runs[0].id == "run-001"
-
-    def test_save_preserves_all_fields(
-        self, storage: BenchmarkStorage, sample_run: BenchmarkRun
-    ) -> None:
-        storage.save_run(sample_run)
-
-        runs = storage.get_runs("test-suite")
-        run = runs[0]
-
-        assert run.id == sample_run.id
-        assert run.benchmark_name == sample_run.benchmark_name
-        assert run.timestamp == sample_run.timestamp
-        assert run.veritext_version == sample_run.veritext_version
-        assert run.metrics == sample_run.metrics
-        assert run.sample_count == sample_run.sample_count
-        assert run.metadata == sample_run.metadata
-
-    def test_save_duplicate_id_raises(
-        self, storage: BenchmarkStorage, sample_run: BenchmarkRun
-    ) -> None:
-        storage.save_run(sample_run)
-
-        with pytest.raises(StorageError, match="already exists"):
-            storage.save_run(sample_run)
-
-    def test_save_run_empty_metadata(self, storage: BenchmarkStorage) -> None:
-        run = BenchmarkRun(
-            id="run-no-meta",
-            benchmark_name="test-suite",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0-dev",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-
-        storage.save_run(run)
-        retrieved = storage.get_latest_run("test-suite")
-
-        assert retrieved is not None
-        assert retrieved.metadata == {}
-
-
-class TestGetRuns:
-    def test_get_runs_empty_database(self, storage: BenchmarkStorage) -> None:
-        runs = storage.get_runs("nonexistent")
-        assert runs == []
-
-    def test_get_runs_filters_by_name(self, storage: BenchmarkStorage) -> None:
-        run1 = BenchmarkRun(
-            id="run-1",
-            benchmark_name="suite-a",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-        run2 = BenchmarkRun(
-            id="run-2",
-            benchmark_name="suite-b",
-            timestamp=datetime(2025, 1, 15, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.6},
-            sample_count=10,
-        )
-
-        storage.save_run(run1)
-        storage.save_run(run2)
-
-        runs_a = storage.get_runs("suite-a")
-        runs_b = storage.get_runs("suite-b")
-
-        assert len(runs_a) == 1
-        assert runs_a[0].id == "run-1"
-        assert len(runs_b) == 1
-        assert runs_b[0].id == "run-2"
-
-    def test_get_runs_ordered_by_timestamp(self, storage: BenchmarkStorage) -> None:
-        run_old = BenchmarkRun(
-            id="run-old",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 10, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-        run_new = BenchmarkRun(
-            id="run-new",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 20, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.6},
-            sample_count=10,
-        )
-
-        # Save in reverse order
-        storage.save_run(run_new)
-        storage.save_run(run_old)
-
-        runs = storage.get_runs("test")
-        assert runs[0].id == "run-new"
-        assert runs[1].id == "run-old"
-
-    def test_get_runs_with_limit(self, storage: BenchmarkStorage) -> None:
-        for i in range(5):
-            run = BenchmarkRun(
-                id=f"run-{i}",
-                benchmark_name="test",
-                timestamp=datetime(2025, 1, i + 1, 12, 0, 0, tzinfo=UTC),
-                veritext_version="0.1.0",
-                metrics={"bleu4": 0.5 + i * 0.1},
-                sample_count=10,
-            )
-            storage.save_run(run)
-
-        runs = storage.get_runs("test", limit=3)
-        assert len(runs) == 3
-
-
-class TestGetLatestRun:
-    def test_get_latest_run_empty(self, storage: BenchmarkStorage) -> None:
-        result = storage.get_latest_run("nonexistent")
-        assert result is None
-
-    def test_get_latest_run(self, storage: BenchmarkStorage) -> None:
-        run_old = BenchmarkRun(
-            id="run-old",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 10, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.5},
-            sample_count=10,
-        )
-        run_new = BenchmarkRun(
-            id="run-new",
-            benchmark_name="test",
-            timestamp=datetime(2025, 1, 20, 12, 0, 0, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"bleu4": 0.6},
-            sample_count=10,
-        )
-
-        storage.save_run(run_old)
-        storage.save_run(run_new)
-
-        latest = storage.get_latest_run("test")
-        assert latest is not None
-        assert latest.id == "run-new"
-
-
-class TestConcurrentAccess:
-    def test_concurrent_writes(self, db_path: Path) -> None:
-        errors: list[Exception] = []
-
-        def write_run(run_id: int) -> None:
-            try:
-                storage = BenchmarkStorage(db_path)
-                run = BenchmarkRun(
-                    id=f"run-{run_id}",
-                    benchmark_name="test",
-                    timestamp=datetime(2025, 1, 15, 12, 0, run_id, tzinfo=UTC),
-                    veritext_version="0.1.0",
-                    metrics={"bleu4": 0.5},
-                    sample_count=10,
-                )
-                storage.save_run(run)
-            except Exception as e:
-                errors.append(e)
-
-        threads = [threading.Thread(target=write_run, args=(i,)) for i in range(10)]
-        for t in threads:
-            t.start()
-        for t in threads:
-            t.join()
-
-        assert not errors, f"Concurrent writes failed: {errors}"
-
-        storage = BenchmarkStorage(db_path)
-        runs = storage.get_runs("test")
-        assert len(runs) == 10
-
-    def test_wal_mode_enabled(self, db_path: Path) -> None:
-        BenchmarkStorage(db_path)
-
-        conn = sqlite3.connect(str(db_path))
-        cursor = conn.execute("PRAGMA journal_mode")
-        mode = cursor.fetchone()[0]
-        conn.close()
-
-        assert mode.lower() == "wal"

tests/test_cli/__init__.py

@@ -1 +0,0 @@
-"""CLI test suite."""

tests/test_cli/test_benchmark.py

@@ -1,316 +0,0 @@
-"""Tests for CLI benchmark commands."""
-
-from pathlib import Path
-
-from typer.testing import CliRunner
-
-from veritext.cli.main import app
-
-runner = CliRunner()
-
-
-class TestBenchmarkRun:
-    def test_benchmark_run_basic(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text(
-            '{"candidate": "hello world today", "reference": "hello world today"}\n'
-            '{"candidate": "foo bar baz qux", "reference": "foo bar baz qux"}'
-        )
-        storage_path = tmp_path / "benchmarks"
-
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                str(data_file),
-                "-m",
-                "rouge_l,bleu4",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert "Benchmark 'test_bench' completed" in result.stdout
-        assert "Samples: 2" in result.stdout
-        assert "rouge_l:" in result.stdout
-        assert "bleu4:" in result.stdout
-
-    def test_benchmark_run_file_not_found(self, tmp_path: Path) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                "/nonexistent/file.jsonl",
-                "-s",
-                str(tmp_path / "benchmarks"),
-            ],
-        )
-        assert result.exit_code == 1
-        assert "Error" in result.stdout
-
-    def test_benchmark_run_creates_storage(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text('{"candidate": "hello", "reference": "hello"}')
-        storage_path = tmp_path / "new_benchmarks"
-
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                str(data_file),
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert storage_path.exists()
-
-
-class TestBenchmarkShow:
-    def test_benchmark_show_no_runs(self, tmp_path: Path) -> None:
-        storage_path = tmp_path / "benchmarks"
-        storage_path.mkdir()
-
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "show",
-                "nonexistent_bench",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert "No benchmark runs found" in result.stdout
-
-    def test_benchmark_show_with_runs(self, tmp_path: Path) -> None:
-        # First create some runs
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text('{"candidate": "hello world", "reference": "hello world"}')
-        storage_path = tmp_path / "benchmarks"
-
-        # Run benchmark twice
-        for _ in range(2):
-            runner.invoke(
-                app,
-                [
-                    "benchmark",
-                    "run",
-                    "test_bench",
-                    "-f",
-                    str(data_file),
-                    "-s",
-                    str(storage_path),
-                ],
-            )
-
-        # Show history
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "show",
-                "test_bench",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert "Benchmark History" in result.stdout
-
-    def test_benchmark_show_limit(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text('{"candidate": "hello", "reference": "hello"}')
-        storage_path = tmp_path / "benchmarks"
-
-        # Run benchmark 3 times
-        for _ in range(3):
-            runner.invoke(
-                app,
-                [
-                    "benchmark",
-                    "run",
-                    "test_bench",
-                    "-f",
-                    str(data_file),
-                    "-s",
-                    str(storage_path),
-                ],
-            )
-
-        # Show only last 2
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "show",
-                "test_bench",
-                "--last",
-                "2",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-
-
-class TestBenchmarkCheck:
-    def test_benchmark_check_no_regression(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text(
-            '{"candidate": "hello world today", "reference": "hello world today"}'
-        )
-        storage_path = tmp_path / "benchmarks"
-
-        # Run benchmark twice with same data (no regression)
-        for _ in range(2):
-            runner.invoke(
-                app,
-                [
-                    "benchmark",
-                    "run",
-                    "test_bench",
-                    "-f",
-                    str(data_file),
-                    "-s",
-                    str(storage_path),
-                ],
-            )
-
-        # Check for regression
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "check",
-                "test_bench",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert "No regression detected" in result.stdout
-
-    def test_benchmark_check_with_regression(self, tmp_path: Path) -> None:
-        storage_path = tmp_path / "benchmarks"
-
-        # First run with good data
-        good_file = tmp_path / "good.jsonl"
-        good_file.write_text(
-            '{"candidate": "hello world today", "reference": "hello world today"}'
-        )
-        runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                str(good_file),
-                "-s",
-                str(storage_path),
-            ],
-        )
-
-        # Second run with bad data (regression)
-        bad_file = tmp_path / "bad.jsonl"
-        bad_file.write_text(
-            '{"candidate": "completely different", "reference": "hello world today"}'
-        )
-        runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                str(bad_file),
-                "-s",
-                str(storage_path),
-            ],
-        )
-
-        # Check for regression
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "check",
-                "test_bench",
-                "-t",
-                "0.05",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 1
-        assert "Regression detected" in result.stdout
-
-    def test_benchmark_check_custom_tolerance(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text('{"candidate": "hello", "reference": "hello"}')
-        storage_path = tmp_path / "benchmarks"
-
-        runner.invoke(
-            app,
-            [
-                "benchmark",
-                "run",
-                "test_bench",
-                "-f",
-                str(data_file),
-                "-s",
-                str(storage_path),
-            ],
-        )
-
-        result = runner.invoke(
-            app,
-            [
-                "benchmark",
-                "check",
-                "test_bench",
-                "--tolerance",
-                "0.10",
-                "-s",
-                str(storage_path),
-            ],
-        )
-        assert result.exit_code == 0
-        assert "10.00%" in result.stdout
-
-
-class TestBenchmarkHelp:
-    def test_benchmark_help(self) -> None:
-        result = runner.invoke(app, ["benchmark", "--help"])
-        assert result.exit_code == 0
-        assert "run" in result.stdout
-        assert "show" in result.stdout
-        assert "check" in result.stdout
-
-    def test_benchmark_run_help(self) -> None:
-        result = runner.invoke(app, ["benchmark", "run", "--help"])
-        assert result.exit_code == 0
-        assert "--file" in result.stdout
-        assert "--metrics" in result.stdout
-
-    def test_benchmark_show_help(self) -> None:
-        result = runner.invoke(app, ["benchmark", "show", "--help"])
-        assert result.exit_code == 0
-        assert "--last" in result.stdout
-
-    def test_benchmark_check_help(self) -> None:
-        result = runner.invoke(app, ["benchmark", "check", "--help"])
-        assert result.exit_code == 0
-        assert "--tolerance" in result.stdout
-        assert "--window" in result.stdout

tests/test_cli/test_formatters.py

@@ -1,118 +0,0 @@
-"""Tests for CLI output formatters."""
-
-from datetime import UTC, datetime
-
-from veritext.benchmark.models import BenchmarkRun, RegressionReport
-from veritext.cli.formatters import (
-    format_benchmark_history,
-    format_regression_report,
-    format_validation_json,
-    format_validation_simple,
-    format_validation_table,
-)
-
-
-class TestFormatValidationTable:
-    def test_format_empty_results(self) -> None:
-        table = format_validation_table({})
-        assert table.title == "Validation Results"
-        assert table.row_count == 0
-
-    def test_format_single_metric(self) -> None:
-        results = {"bleu4": 0.8523}
-        table = format_validation_table(results)
-        assert table.row_count == 1
-
-    def test_format_multiple_metrics(self) -> None:
-        results = {"bleu4": 0.85, "rouge_l": 0.92, "jaccard": 0.75}
-        table = format_validation_table(results)
-        assert table.row_count == 3
-
-    def test_format_with_threshold(self) -> None:
-        results = {"bleu4": 0.85, "rouge_l": 0.45}
-        table = format_validation_table(results, threshold=0.5)
-        # Should have 3 columns: Metric, Score, Status
-        assert table.row_count == 2
-
-
-class TestFormatValidationJson:
-    def test_format_empty_results(self) -> None:
-        result = format_validation_json({})
-        assert result == "{}"
-
-    def test_format_results(self) -> None:
-        results = {"bleu4": 0.85, "rouge_l": 0.92}
-        result = format_validation_json(results)
-        assert '"bleu4": 0.85' in result
-        assert '"rouge_l": 0.92' in result
-
-
-class TestFormatValidationSimple:
-    def test_format_empty_results(self) -> None:
-        result = format_validation_simple({})
-        assert result == ""
-
-    def test_format_results(self) -> None:
-        results = {"bleu4": 0.8523, "rouge_l": 0.9234}
-        result = format_validation_simple(results)
-        assert "bleu4: 0.8523" in result
-        assert "rouge_l: 0.9234" in result
-
-
-class TestFormatBenchmarkHistory:
-    def test_format_empty_history(self) -> None:
-        table = format_benchmark_history([])
-        assert table.title == "Benchmark History"
-
-    def test_format_single_run(self) -> None:
-        run = BenchmarkRun(
-            id="test-id",
-            benchmark_name="test",
-            timestamp=datetime(2024, 1, 15, 10, 30, tzinfo=UTC),
-            veritext_version="0.1.0",
-            metrics={"rouge_l": 0.85, "bleu4": 0.72},
-            sample_count=100,
-        )
-        table = format_benchmark_history([run])
-        assert table.row_count == 1
-
-    def test_format_multiple_runs(self) -> None:
-        runs = [
-            BenchmarkRun(
-                id=f"test-id-{i}",
-                benchmark_name="test",
-                timestamp=datetime(2024, 1, i + 1, 10, 30, tzinfo=UTC),
-                veritext_version="0.1.0",
-                metrics={"rouge_l": 0.8 + i * 0.01},
-                sample_count=100,
-            )
-            for i in range(3)
-        ]
-        table = format_benchmark_history(runs)
-        assert table.row_count == 3
-
-
-class TestFormatRegressionReport:
-    def test_format_no_regression(self) -> None:
-        report = RegressionReport(
-            detected=False,
-            baseline={"rouge_l": 0.85},
-            current={"rouge_l": 0.86},
-            deltas={"rouge_l": 0.01},
-            tolerance=0.05,
-        )
-        panel = format_regression_report(report)
-        assert panel.title == "Regression Check"
-        assert panel.border_style == "green"
-
-    def test_format_with_regression(self) -> None:
-        report = RegressionReport(
-            detected=True,
-            baseline={"rouge_l": 0.85, "bleu4": 0.72},
-            current={"rouge_l": 0.70, "bleu4": 0.70},
-            deltas={"rouge_l": -0.15, "bleu4": -0.02},
-            tolerance=0.05,
-        )
-        panel = format_regression_report(report)
-        assert panel.title == "Regression Check"
-        assert panel.border_style == "red"

tests/test_cli/test_readers.py

@@ -1,126 +0,0 @@
-"""Tests for CLI input readers."""
-
-import json
-from pathlib import Path
-
-import pytest
-
-from veritext.cli.readers import TextPair, read_jsonl, read_paired_jsonl
-
-
-class TestTextPair:
-    def test_create_text_pair(self) -> None:
-        pair = TextPair(candidate="hello", reference="world")
-        assert pair.candidate == "hello"
-        assert pair.reference == "world"
-
-
-class TestReadJsonl:
-    def test_read_valid_jsonl(self, tmp_path: Path) -> None:
-        data = [
-            {"candidate": "foo", "reference": "bar"},
-            {"candidate": "baz", "reference": "qux"},
-        ]
-        jsonl_file = tmp_path / "data.jsonl"
-        jsonl_file.write_text("\n".join(json.dumps(d) for d in data))
-
-        pairs = read_jsonl(jsonl_file)
-
-        assert len(pairs) == 2
-        assert pairs[0].candidate == "foo"
-        assert pairs[0].reference == "bar"
-        assert pairs[1].candidate == "baz"
-        assert pairs[1].reference == "qux"
-
-    def test_read_empty_file(self, tmp_path: Path) -> None:
-        jsonl_file = tmp_path / "empty.jsonl"
-        jsonl_file.write_text("")
-
-        pairs = read_jsonl(jsonl_file)
-
-        assert pairs == []
-
-    def test_read_file_with_blank_lines(self, tmp_path: Path) -> None:
-        jsonl_file = tmp_path / "data.jsonl"
-        content = '{"candidate": "a", "reference": "b"}\n\n{"candidate": "c", "reference": "d"}\n'
-        jsonl_file.write_text(content)
-
-        pairs = read_jsonl(jsonl_file)
-
-        assert len(pairs) == 2
-
-    def test_read_file_not_found(self, tmp_path: Path) -> None:
-        with pytest.raises(FileNotFoundError):
-            read_jsonl(tmp_path / "nonexistent.jsonl")
-
-    def test_read_invalid_json(self, tmp_path: Path) -> None:
-        jsonl_file = tmp_path / "invalid.jsonl"
-        jsonl_file.write_text("not valid json")
-
-        with pytest.raises(ValueError, match="Invalid JSON on line 1"):
-            read_jsonl(jsonl_file)
-
-    def test_read_missing_candidate_key(self, tmp_path: Path) -> None:
-        jsonl_file = tmp_path / "data.jsonl"
-        jsonl_file.write_text('{"reference": "bar"}')
-
-        with pytest.raises(ValueError, match="Missing 'candidate' key on line 1"):
-            read_jsonl(jsonl_file)
-
-    def test_read_missing_reference_key(self, tmp_path: Path) -> None:
-        jsonl_file = tmp_path / "data.jsonl"
-        jsonl_file.write_text('{"candidate": "foo"}')
-
-        with pytest.raises(ValueError, match="Missing 'reference' key on line 1"):
-            read_jsonl(jsonl_file)
-
-
-class TestReadPairedJsonl:
-    def test_read_paired_valid(self, tmp_path: Path) -> None:
-        candidates_file = tmp_path / "candidates.jsonl"
-        references_file = tmp_path / "references.jsonl"
-
-        candidates_file.write_text('{"text": "foo"}\n{"text": "bar"}')
-        references_file.write_text('{"text": "baz"}\n{"text": "qux"}')
-
-        pairs = read_paired_jsonl(candidates_file, references_file)
-
-        assert len(pairs) == 2
-        assert pairs[0].candidate == "foo"
-        assert pairs[0].reference == "baz"
-        assert pairs[1].candidate == "bar"
-        assert pairs[1].reference == "qux"
-
-    def test_read_paired_length_mismatch(self, tmp_path: Path) -> None:
-        candidates_file = tmp_path / "candidates.jsonl"
-        references_file = tmp_path / "references.jsonl"
-
-        candidates_file.write_text('{"text": "foo"}\n{"text": "bar"}')
-        references_file.write_text('{"text": "baz"}')
-
-        with pytest.raises(ValueError, match="does not match"):
-            read_paired_jsonl(candidates_file, references_file)
-
-    def test_read_paired_candidates_not_found(self, tmp_path: Path) -> None:
-        references_file = tmp_path / "references.jsonl"
-        references_file.write_text('{"text": "baz"}')
-
-        with pytest.raises(FileNotFoundError, match="Candidates file not found"):
-            read_paired_jsonl(tmp_path / "nonexistent.jsonl", references_file)
-
-    def test_read_paired_references_not_found(self, tmp_path: Path) -> None:
-        candidates_file = tmp_path / "candidates.jsonl"
-        candidates_file.write_text('{"text": "foo"}')
-
-        with pytest.raises(FileNotFoundError, match="References file not found"):
-            read_paired_jsonl(candidates_file, tmp_path / "nonexistent.jsonl")
-
-    def test_read_paired_missing_text_key(self, tmp_path: Path) -> None:
-        candidates_file = tmp_path / "candidates.jsonl"
-        references_file = tmp_path / "references.jsonl"
-
-        candidates_file.write_text('{"value": "foo"}')
-        references_file.write_text('{"text": "baz"}')
-
-        with pytest.raises(ValueError, match="Missing 'text' key in candidates file"):
-            read_paired_jsonl(candidates_file, references_file)

tests/test_cli/test_validate.py

@@ -1,214 +0,0 @@
-"""Tests for CLI validate command."""
-
-import json
-from pathlib import Path
-
-from typer.testing import CliRunner
-
-from veritext.cli.main import app
-
-runner = CliRunner()
-
-
-class TestValidateInline:
-    def test_validate_inline_basic(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "The quick brown fox jumps",
-                "-r",
-                "The quick brown fox jumps",
-                "-m",
-                "bleu",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "bleu4" in result.stdout
-
-    def test_validate_inline_with_rouge(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello world today",
-                "-r",
-                "hello world here",
-                "-m",
-                "rouge",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "rouge_l" in result.stdout
-
-    def test_validate_inline_with_lexical(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello world",
-                "-r",
-                "hello everyone",
-                "-m",
-                "lexical",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "jaccard" in result.stdout
-        assert "token_overlap" in result.stdout
-
-    def test_validate_inline_json_output(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello world today",
-                "-r",
-                "hello world today",
-                "-m",
-                "bleu",
-                "-o",
-                "json",
-            ],
-        )
-        assert result.exit_code == 0
-        data = json.loads(result.stdout)
-        assert "bleu4" in data
-
-    def test_validate_inline_simple_output(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello world today",
-                "-r",
-                "hello world today",
-                "-m",
-                "rouge",
-                "-o",
-                "simple",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "rouge_l:" in result.stdout
-
-    def test_validate_inline_missing_reference(self) -> None:
-        result = runner.invoke(
-            app,
-            ["validate", "hello world", "-m", "bleu"],
-        )
-        assert result.exit_code == 1
-        assert "Error" in result.stdout
-
-    def test_validate_inline_invalid_metric(self) -> None:
-        result = runner.invoke(
-            app,
-            ["validate", "hello", "-r", "world", "-m", "invalid_metric"],
-        )
-        assert result.exit_code == 1
-        assert "Unknown metrics" in result.stdout
-
-
-class TestValidateFile:
-    def test_validate_file_basic(self, tmp_path: Path) -> None:
-        data_file = tmp_path / "data.jsonl"
-        data_file.write_text(
-            '{"candidate": "hello world today", "reference": "hello world today"}\n'
-            '{"candidate": "foo bar baz", "reference": "foo bar baz"}'
-        )
-
-        result = runner.invoke(
-            app,
-            ["validate", "-f", str(data_file), "-m", "bleu"],
-        )
-        assert result.exit_code == 0
-        assert "bleu4" in result.stdout
-        assert "Evaluated 2 text pairs" in result.stdout
-
-    def test_validate_file_not_found(self) -> None:
-        result = runner.invoke(
-            app,
-            ["validate", "-f", "/nonexistent/file.jsonl", "-m", "bleu"],
-        )
-        assert result.exit_code == 1
-        assert "Error" in result.stdout
-
-    def test_validate_paired_files(self, tmp_path: Path) -> None:
-        candidates_file = tmp_path / "candidates.jsonl"
-        references_file = tmp_path / "references.jsonl"
-
-        candidates_file.write_text(
-            '{"text": "hello world today"}\n{"text": "foo bar baz"}'
-        )
-        references_file.write_text(
-            '{"text": "hello world today"}\n{"text": "foo bar baz"}'
-        )
-
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "-f",
-                str(candidates_file),
-                "-R",
-                str(references_file),
-                "-m",
-                "bleu",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "Evaluated 2 text pairs" in result.stdout
-
-
-class TestValidateOptions:
-    def test_validate_with_threshold(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello world today",
-                "-r",
-                "hello world today",
-                "-m",
-                "bleu",
-                "-t",
-                "0.5",
-            ],
-        )
-        assert result.exit_code == 0
-        # Table output should include Status column
-        assert "Status" in result.stdout or "PASS" in result.stdout
-
-    def test_validate_invalid_output_format(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "hello",
-                "-r",
-                "world",
-                "-m",
-                "bleu",
-                "-o",
-                "invalid",
-            ],
-        )
-        assert result.exit_code == 1
-        assert "Invalid output format" in result.stdout
-
-    def test_validate_multiple_metrics(self) -> None:
-        result = runner.invoke(
-            app,
-            [
-                "validate",
-                "The quick brown fox",
-                "-r",
-                "The quick brown fox",
-                "-m",
-                "bleu,rouge,lexical",
-            ],
-        )
-        assert result.exit_code == 0
-        assert "bleu4" in result.stdout
-        assert "rouge_l" in result.stdout
-        assert "jaccard" in result.stdout

tests/test_core/test_config.py

@@ -1,58 +0,0 @@
-"""Tests for configuration module."""
-
-from pathlib import Path
-
-import pytest
-
-from veritext.core.config import VeritextSettings, get_settings
-
-
-class TestVeritextSettings:
-    def test_default_log_level(self) -> None:
-        settings = VeritextSettings()
-        assert settings.log_level == "INFO"
-
-    def test_default_log_format(self) -> None:
-        settings = VeritextSettings()
-        assert settings.log_format == "console"
-
-    def test_default_benchmark_path(self) -> None:
-        settings = VeritextSettings()
-        assert settings.benchmark_storage_path == Path("benchmarks")
-
-    def test_default_tokeniser_lowercase(self) -> None:
-        settings = VeritextSettings()
-        assert settings.tokeniser_lowercase is True
-
-    def test_default_remove_punctuation(self) -> None:
-        settings = VeritextSettings()
-        assert settings.tokeniser_remove_punctuation is True
-
-    def test_default_semantic_model(self) -> None:
-        settings = VeritextSettings()
-        assert settings.semantic_model == "all-MiniLM-L6-v2"
-
-    def test_default_semantic_cache_enabled(self) -> None:
-        settings = VeritextSettings()
-        assert settings.semantic_cache_embeddings is True
-
-    def test_env_var_override(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.setenv("VERITEXT_LOG_LEVEL", "DEBUG")
-        settings = VeritextSettings()
-        assert settings.log_level == "DEBUG"
-
-    def test_env_var_override_log_format(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.setenv("VERITEXT_LOG_FORMAT", "json")
-        settings = VeritextSettings()
-        assert settings.log_format == "json"
-
-
-class TestGetSettings:
-    def test_get_settings_returns_instance(self) -> None:
-        settings = get_settings()
-        assert isinstance(settings, VeritextSettings)
-
-    def test_get_settings_returns_valid_defaults(self) -> None:
-        settings = get_settings()
-        assert settings.log_level in ("DEBUG", "INFO", "WARNING", "ERROR")
-        assert settings.log_format in ("console", "json")

tests/test_core/test_logging.py

@@ -1,45 +0,0 @@
-"""Tests for logging module."""
-
-from veritext.core.logging import configure_logging, get_logger
-
-
-class TestGetLogger:
-    def test_get_logger_returns_logger(self) -> None:
-        logger = get_logger()
-        assert logger is not None
-
-    def test_get_logger_default_name(self) -> None:
-        logger = get_logger()
-        # The logger should be a bound logger from structlog
-        assert hasattr(logger, "info")
-        assert hasattr(logger, "debug")
-        assert hasattr(logger, "warning")
-        assert hasattr(logger, "error")
-
-    def test_get_logger_custom_name(self) -> None:
-        logger = get_logger("custom.module")
-        assert logger is not None
-        assert hasattr(logger, "info")
-
-
-class TestConfigureLogging:
-    def test_configure_logging_console_format(self) -> None:
-        configure_logging(level="INFO", log_format="console")
-        logger = get_logger()
-        assert logger is not None
-
-    def test_configure_logging_json_format(self) -> None:
-        configure_logging(level="DEBUG", log_format="json")
-        logger = get_logger()
-        assert logger is not None
-
-    def test_configure_logging_uses_defaults(self) -> None:
-        configure_logging()
-        logger = get_logger()
-        assert logger is not None
-
-    def test_configure_logging_different_levels(self) -> None:
-        for level in ("DEBUG", "INFO", "WARNING", "ERROR"):
-            configure_logging(level=level)
-            logger = get_logger()
-            assert logger is not None

tests/test_core/test_tokenisation.py

@@ -9,41 +9,52 @@ if TYPE_CHECKING:
 
 
 class TestWordTokeniser:
+    """Tests for WordTokeniser."""
+
     def test_basic_tokenisation(self, word_tokeniser: WordTokeniser) -> None:
+        """Test basic word tokenisation."""
         tokens = word_tokeniser.tokenise("The cat sat on the mat")
         assert tokens == ["the", "cat", "sat", "on", "the", "mat"]
 
     def test_lowercasing(self, word_tokeniser: WordTokeniser) -> None:
+        """Test that tokens are lowercased by default."""
         tokens = word_tokeniser.tokenise("Hello WORLD")
         assert tokens == ["hello", "world"]
 
     def test_no_lowercasing(self, word_tokeniser_no_lowercase: WordTokeniser) -> None:
+        """Test tokenisation without lowercasing."""
         tokens = word_tokeniser_no_lowercase.tokenise("Hello WORLD")
         assert tokens == ["Hello", "WORLD"]
 
     def test_punctuation_removal(self, word_tokeniser: WordTokeniser) -> None:
+        """Test that punctuation is removed by default."""
         tokens = word_tokeniser.tokenise("Hello, world! How are you?")
         assert tokens == ["hello", "world", "how", "are", "you"]
 
     def test_keep_punctuation(
         self, word_tokeniser_keep_punctuation: WordTokeniser
     ) -> None:
+        """Test tokenisation keeping punctuation."""
         tokens = word_tokeniser_keep_punctuation.tokenise("Hello, world!")
         assert tokens == ["hello,", "world!"]
 
     def test_empty_string(self, word_tokeniser: WordTokeniser) -> None:
+        """Test that empty string returns empty list."""
         tokens = word_tokeniser.tokenise("")
         assert tokens == []
 
     def test_whitespace_only(self, word_tokeniser: WordTokeniser) -> None:
+        """Test that whitespace-only string returns empty list."""
         tokens = word_tokeniser.tokenise("   \t\n  ")
         assert tokens == []
 
     def test_multiple_spaces(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of multiple spaces between words."""
         tokens = word_tokeniser.tokenise("hello    world")
         assert tokens == ["hello", "world"]
 
     def test_unicode_nfc_normalisation(self) -> None:
+        """Test NFC Unicode normalisation."""
         # 'é' can be composed (U+00E9) or decomposed (e + U+0301)
         composed = "caf\u00e9"  # café with composed é
         decomposed = "cafe\u0301"  # café with decomposed é
@@ -57,33 +68,40 @@ class TestWordTokeniser:
         assert tokens_composed == ["café"]
 
     def test_unicode_emoji(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of emoji characters."""
         # Emoji are removed as punctuation by default
         tokens = word_tokeniser.tokenise("Hello 👋 world 🌍")
         assert tokens == ["hello", "world"]
 
     def test_unicode_non_latin(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of non-Latin scripts."""
         tokens = word_tokeniser.tokenise("日本語 テスト")
         assert tokens == ["日本語", "テスト"]
 
     def test_unicode_mixed_scripts(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of mixed scripts."""
         tokens = word_tokeniser.tokenise("Hello 世界 Bonjour мир")
         assert tokens == ["hello", "世界", "bonjour", "мир"]
 
     def test_numbers_preserved(self, word_tokeniser: WordTokeniser) -> None:
+        """Test that numbers are preserved."""
         tokens = word_tokeniser.tokenise("I have 42 apples")
         assert tokens == ["i", "have", "42", "apples"]
 
     def test_contractions(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of contractions (apostrophe replaced with space)."""
         tokens = word_tokeniser.tokenise("I can't don't won't")
         # Apostrophe is replaced with space, splitting the words
         assert tokens == ["i", "can", "t", "don", "t", "won", "t"]
 
     def test_hyphenated_words(self, word_tokeniser: WordTokeniser) -> None:
+        """Test handling of hyphenated words."""
         tokens = word_tokeniser.tokenise("state-of-the-art")
         # Hyphens are removed as punctuation
         assert tokens == ["state", "of", "the", "art"]
 
     def test_custom_normalisation_form(self) -> None:
+        """Test custom Unicode normalisation form."""
         tokeniser = WordTokeniser(normalisation_form="NFKC")
         # NFKC normalises compatibility characters
         # ™ (U+2122) is compatibility equivalent to 'TM'
@@ -92,7 +110,10 @@ class TestWordTokeniser:
 
 
 class TestTokeniserProtocol:
+    """Tests for Tokeniser protocol compliance."""
+
     def test_word_tokeniser_implements_protocol(self) -> None:
+        """Test that WordTokeniser implements the Tokeniser protocol."""
         tokeniser: Tokeniser = WordTokeniser()
         # Check that it has the required method
         assert hasattr(tokeniser, "tokenise")

tests/test_core/test_types.py

@@ -7,34 +7,44 @@ from veritext.core.types import CheckResult, ValidationContext, ValidationResult
 
 
 class TestValidationContext:
+    """Tests for ValidationContext."""
+
     def test_create_empty_context(self) -> None:
+        """Test creating a context with no reference."""
         context = ValidationContext()
         assert context.reference is None
         assert context.metadata == {}
 
     def test_create_with_single_reference(self) -> None:
+        """Test creating a context with a single reference string."""
         context = ValidationContext(reference="The cat sat on the mat.")
         assert context.reference == "The cat sat on the mat."
 
     def test_create_with_multiple_references(self) -> None:
+        """Test creating a context with multiple reference strings."""
         references = ["The cat sat on the mat.", "A cat was sitting on a mat."]
         context = ValidationContext(reference=references)
         assert context.reference == references
         assert len(context.reference) == 2
 
     def test_create_with_metadata(self) -> None:
+        """Test creating a context with metadata."""
         metadata = {"source": "test", "timestamp": "2024-01-01"}
         context = ValidationContext(metadata=metadata)
         assert context.metadata == metadata
 
     def test_context_is_immutable(self) -> None:
+        """Test that ValidationContext is immutable (frozen)."""
         context = ValidationContext(reference="test")
         with pytest.raises(PydanticValidationError):
             context.reference = "new value"  # type: ignore[misc]
 
 
 class TestCheckResult:
+    """Tests for CheckResult."""
+
     def test_create_passing_result(self) -> None:
+        """Test creating a passing check result."""
         result = CheckResult(
             name="bleu",
             passed=True,
@@ -49,6 +59,7 @@ class TestCheckResult:
         assert "0.85" in result.message
 
     def test_create_failing_result(self) -> None:
+        """Test creating a failing check result."""
         result = CheckResult(
             name="length",
             passed=False,
@@ -62,6 +73,7 @@ class TestCheckResult:
         assert result.threshold == 500
 
     def test_create_result_without_threshold(self) -> None:
+        """Test creating a result for checks without thresholds."""
         result = CheckResult(
             name="contains",
             passed=True,
@@ -72,6 +84,7 @@ class TestCheckResult:
         assert result.threshold is None
 
     def test_result_is_immutable(self) -> None:
+        """Test that CheckResult is immutable (frozen)."""
         result = CheckResult(
             name="test",
             passed=True,
@@ -83,7 +96,10 @@ class TestCheckResult:
 
 
 class TestValidationResult:
+    """Tests for ValidationResult."""
+
     def test_all_checks_passed(self) -> None:
+        """Test validation result when all checks pass."""
         checks = [
             CheckResult(name="bleu", passed=True, actual=0.8, message="OK"),
             CheckResult(name="length", passed=True, actual=100, message="OK"),
@@ -96,6 +112,7 @@ class TestValidationResult:
         assert "All checks passed" in result.failure_summary
 
     def test_some_checks_failed(self) -> None:
+        """Test validation result when some checks fail."""
         checks = [
             CheckResult(name="bleu", passed=True, actual=0.8, message="OK"),
             CheckResult(
@@ -122,6 +139,7 @@ class TestValidationResult:
         assert result.failed_checks[1].name == "readability"
 
     def test_failure_summary_format(self) -> None:
+        """Test the format of failure summary."""
         checks = [
             CheckResult(
                 name="bleu",
@@ -140,6 +158,7 @@ class TestValidationResult:
         assert "BLEU score 0.5 < threshold 0.7" in summary
 
     def test_empty_checks_list(self) -> None:
+        """Test validation result with no checks."""
         result = ValidationResult(passed=True, checks=[])
         assert result.passed is True
         assert result.checks == []
@@ -147,11 +166,13 @@ class TestValidationResult:
         assert "All checks passed" in result.failure_summary
 
     def test_result_is_immutable(self) -> None:
+        """Test that ValidationResult is immutable (frozen)."""
         result = ValidationResult(passed=True, checks=[])
         with pytest.raises(PydanticValidationError):
             result.passed = False  # type: ignore[misc]
 
     def test_check_actual_can_be_any_type(self) -> None:
+        """Test that CheckResult.actual can hold any type."""
         # List
         result1 = CheckResult(
             name="contains",

tests/test_metrics/test_bleu.py

@@ -6,17 +6,23 @@ from veritext.metrics import Bleu, BleuResult
 
 
 class TestBleu:
+    """Tests for the Bleu metric class."""
+
     @pytest.fixture
     def bleu(self) -> Bleu:
+        """Provide a BLEU metric instance."""
         return Bleu()
 
     def test_name(self, bleu: Bleu) -> None:
+        """Test that name returns 'bleu'."""
         assert bleu.name == "bleu"
 
     def test_requires_reference(self, bleu: Bleu) -> None:
+        """Test that BLEU requires reference text."""
         assert bleu.requires_reference is True
 
     def test_identical_texts(self, bleu: Bleu) -> None:
+        """Test that identical texts produce perfect scores."""
         text = "The cat sat on the mat"
         result = bleu.score(text, text)
 
@@ -27,6 +33,7 @@ class TestBleu:
         assert result.brevity_penalty == 1.0
 
     def test_no_overlap(self, bleu: Bleu) -> None:
+        """Test that texts with no overlap produce zero scores."""
         candidate = "The cat sat"
         reference = "A dog runs fast"
         result = bleu.score(candidate, reference)
@@ -37,34 +44,44 @@ class TestBleu:
         assert result.bleu4 == 0.0
 
     def test_partial_overlap(self, bleu: Bleu) -> None:
+        """Test partial overlap produces intermediate scores."""
         candidate = "The cat sat on the mat"
         reference = "The cat is on the floor"
         result = bleu.score(candidate, reference)
 
+        # Should have some overlap but not perfect
         assert 0.0 < result.bleu1 < 1.0
         assert 0.0 < result.bleu2 < 1.0
 
     def test_brevity_penalty_applied(self, bleu: Bleu) -> None:
+        """Test that brevity penalty is applied for short candidates."""
         candidate = "cat"
         reference = "The cat sat on the mat"
         result = bleu.score(candidate, reference)
 
+        # Candidate is much shorter, so brevity penalty should be < 1
         assert result.brevity_penalty < 1.0
 
-    def test_no_brevity_penalty_long_cand(self, bleu: Bleu) -> None:
+    def test_no_brevity_penalty_for_long_candidate(self, bleu: Bleu) -> None:
+        """Test that no brevity penalty for longer candidates."""
         candidate = "The cat sat on the mat today"
         reference = "The cat sat"
         result = bleu.score(candidate, reference)
 
+        # Candidate is longer, so no brevity penalty
         assert result.brevity_penalty == 1.0
 
     def test_single_word_texts(self, bleu: Bleu) -> None:
+        """Test BLEU with single word texts."""
         result = bleu.score("cat", "cat")
 
         assert result.bleu1 == 1.0
+        # Higher order n-grams can't be computed for single words
+        # but with geometric mean, they still produce a score
         assert result.brevity_penalty == 1.0
 
     def test_empty_candidate(self, bleu: Bleu) -> None:
+        """Test that empty candidate returns zero scores."""
         result = bleu.score("", "The cat sat")
 
         assert result.bleu1 == 0.0
@@ -74,20 +91,24 @@ class TestBleu:
         assert result.brevity_penalty == 0.0
 
     def test_whitespace_only_candidate(self, bleu: Bleu) -> None:
+        """Test that whitespace-only candidate returns zero scores."""
         result = bleu.score("   \t\n  ", "The cat sat")
 
         assert result.bleu1 == 0.0
         assert result.bleu4 == 0.0
 
     def test_empty_reference_raises(self, bleu: Bleu) -> None:
+        """Test that empty reference raises ValueError."""
         with pytest.raises(ValueError, match="cannot be empty"):
             bleu.score("The cat sat", "")
 
     def test_none_reference_raises(self, bleu: Bleu) -> None:
+        """Test that None reference raises ValueError."""
         with pytest.raises(ValueError, match="requires reference"):
             bleu.score("The cat sat", None)
 
     def test_multiple_references(self, bleu: Bleu) -> None:
+        """Test BLEU with multiple references uses max n-gram matches."""
         candidate = "The cat sat on the mat"
         references = [
             "The cat is on the floor",
@@ -95,9 +116,11 @@ class TestBleu:
         ]
         result = bleu.score(candidate, references)
 
+        # Should get perfect score due to exact match reference
         assert result.bleu4 == 1.0
 
     def test_multiple_references_partial(self, bleu: Bleu) -> None:
+        """Test multiple references with partial matches."""
         candidate = "The quick brown fox"
         references = [
             "The fast brown fox",
@@ -105,27 +128,35 @@ class TestBleu:
         ]
         result = bleu.score(candidate, references)
 
+        # Should benefit from both references
         assert result.bleu1 > 0.0
 
     def test_result_score_property(self, bleu: Bleu) -> None:
+        """Test that result.score returns bleu4."""
         result = bleu.score("The cat sat", "The cat sat")
         assert result.score == result.bleu4
 
     def test_case_insensitivity(self, bleu: Bleu) -> None:
+        """Test that BLEU is case insensitive by default."""
         result = bleu.score("THE CAT SAT ON THE MAT", "the cat sat on the mat")
         assert result.bleu4 == 1.0
 
     def test_punctuation_ignored(self, bleu: Bleu) -> None:
+        """Test that punctuation is ignored by default."""
         result = bleu.score("The cat sat on the mat.", "The cat sat on the mat!")
         assert result.bleu4 == 1.0
 
 
 class TestBleuBatch:
+    """Tests for BLEU batch scoring."""
+
     @pytest.fixture
     def bleu(self) -> Bleu:
+        """Provide a BLEU metric instance."""
         return Bleu()
 
     def test_batch_score_basic(self, bleu: Bleu) -> None:
+        """Test basic batch scoring."""
         candidates = ["The cat sat on the mat", "A quick brown dog runs fast"]
         references = ["The cat sat on the mat", "A quick brown dog runs fast"]
         result = bleu.batch_score(candidates, references)
@@ -135,6 +166,7 @@ class TestBleuBatch:
         assert all(r.bleu4 == 1.0 for r in result.results)
 
     def test_batch_score_statistics(self, bleu: Bleu) -> None:
+        """Test that batch scoring computes statistics."""
         candidates = ["The cat sat", "A different text entirely"]
         references = ["The cat sat", "The cat sat"]
         result = bleu.batch_score(candidates, references)
@@ -149,6 +181,7 @@ class TestBleuBatch:
         assert stats.min <= stats.mean <= stats.max
 
     def test_batch_score_percentiles(self, bleu: Bleu) -> None:
+        """Test that batch scoring computes percentiles."""
         candidates = ["a", "b", "c", "d", "e"]
         references = ["a", "b", "c", "d", "e"]
         result = bleu.batch_score(candidates, references)
@@ -160,14 +193,17 @@ class TestBleuBatch:
         assert 95 in stats.percentiles
 
     def test_batch_score_none_references_raises(self, bleu: Bleu) -> None:
+        """Test that batch scoring raises for None references."""
         with pytest.raises(ValueError, match="requires reference"):
             bleu.batch_score(["text"], None)
 
     def test_batch_score_length_mismatch_raises(self, bleu: Bleu) -> None:
+        """Test that batch scoring raises for mismatched lengths."""
         with pytest.raises(ValueError, match="must match"):
             bleu.batch_score(["a", "b"], ["a"])
 
-    def test_batch_score_multi_refs(self, bleu: Bleu) -> None:
+    def test_batch_score_with_multiple_references(self, bleu: Bleu) -> None:
+        """Test batch scoring with multiple references per candidate."""
         candidates = [
             "The cat sat on the mat",
             "A quick brown dog runs fast",
@@ -184,7 +220,10 @@ class TestBleuBatch:
 
 
 class TestBleuResult:
+    """Tests for BleuResult type."""
+
     def test_frozen(self) -> None:
+        """Test that BleuResult is frozen."""
         from pydantic import ValidationError
 
         result = BleuResult(
@@ -194,6 +233,7 @@ class TestBleuResult:
             result.bleu1 = 0.6  # type: ignore[misc]
 
     def test_score_property(self) -> None:
+        """Test that score property returns bleu4."""
         result = BleuResult(
             bleu1=0.9, bleu2=0.8, bleu3=0.7, bleu4=0.6, brevity_penalty=1.0
         )

tests/test_metrics/test_lexical.py

@@ -6,17 +6,23 @@ from veritext.metrics import Lexical, LexicalResult
 
 
 class TestLexical:
+    """Tests for the Lexical metric class."""
+
     @pytest.fixture
     def lexical(self) -> Lexical:
+        """Provide a lexical metric instance."""
         return Lexical()
 
     def test_name(self, lexical: Lexical) -> None:
+        """Test that name returns 'lexical'."""
         assert lexical.name == "lexical"
 
     def test_requires_reference(self, lexical: Lexical) -> None:
+        """Test that lexical requires reference text."""
         assert lexical.requires_reference is True
 
     def test_identical_texts(self, lexical: Lexical) -> None:
+        """Test that identical texts produce perfect scores."""
         text = "The cat sat on the mat"
         result = lexical.score(text, text)
 
@@ -24,6 +30,7 @@ class TestLexical:
         assert result.token_overlap == 1.0
 
     def test_no_overlap(self, lexical: Lexical) -> None:
+        """Test that texts with no overlap produce zero scores."""
         candidate = "apple banana cherry"
         reference = "dog elephant fox"
         result = lexical.score(candidate, reference)
@@ -32,6 +39,7 @@ class TestLexical:
         assert result.token_overlap == 0.0
 
     def test_partial_overlap_jaccard(self, lexical: Lexical) -> None:
+        """Test Jaccard with partial overlap."""
         candidate = "the cat sat"
         reference = "the dog sat"
         result = lexical.score(candidate, reference)
@@ -41,6 +49,7 @@ class TestLexical:
         assert result.jaccard == 0.5
 
     def test_partial_overlap_token_overlap(self, lexical: Lexical) -> None:
+        """Test token overlap with partial overlap."""
         candidate = "the cat sat"
         reference = "the dog sat"
         result = lexical.score(candidate, reference)
@@ -51,6 +60,7 @@ class TestLexical:
         assert abs(result.token_overlap - 2 / 3) < 1e-10
 
     def test_candidate_subset_of_reference(self, lexical: Lexical) -> None:
+        """Test when candidate is a subset of reference."""
         candidate = "the cat"
         reference = "the cat sat on the mat"
         result = lexical.score(candidate, reference)
@@ -61,6 +71,7 @@ class TestLexical:
         assert result.jaccard < 1.0
 
     def test_reference_subset_of_candidate(self, lexical: Lexical) -> None:
+        """Test when reference is a subset of candidate."""
         candidate = "the cat sat on the mat"
         reference = "the cat"
         result = lexical.score(candidate, reference)
@@ -71,26 +82,31 @@ class TestLexical:
         assert result.token_overlap < 1.0
 
     def test_empty_candidate(self, lexical: Lexical) -> None:
+        """Test that empty candidate returns zero scores."""
         result = lexical.score("", "The cat sat")
 
         assert result.jaccard == 0.0
         assert result.token_overlap == 0.0
 
     def test_whitespace_only_candidate(self, lexical: Lexical) -> None:
+        """Test that whitespace-only candidate returns zero scores."""
         result = lexical.score("   \t\n  ", "The cat sat")
 
         assert result.jaccard == 0.0
         assert result.token_overlap == 0.0
 
     def test_empty_reference_raises(self, lexical: Lexical) -> None:
+        """Test that empty reference raises ValueError."""
         with pytest.raises(ValueError, match="cannot be empty"):
             lexical.score("The cat sat", "")
 
     def test_none_reference_raises(self, lexical: Lexical) -> None:
+        """Test that None reference raises ValueError."""
         with pytest.raises(ValueError, match="requires reference"):
             lexical.score("The cat sat", None)
 
     def test_multiple_references_uses_first(self, lexical: Lexical) -> None:
+        """Test that multiple references uses the first one."""
         candidate = "the cat sat"
         references = ["the dog ran", "the cat sat"]  # First differs
         result = lexical.score(candidate, references)
@@ -99,16 +115,19 @@ class TestLexical:
         assert result.jaccard < 1.0
 
     def test_case_insensitivity(self, lexical: Lexical) -> None:
+        """Test that lexical is case insensitive by default."""
         result = lexical.score("THE CAT SAT", "the cat sat")
         assert result.jaccard == 1.0
         assert result.token_overlap == 1.0
 
     def test_punctuation_ignored(self, lexical: Lexical) -> None:
+        """Test that punctuation is ignored by default."""
         result = lexical.score("The cat sat.", "The cat sat!")
         assert result.jaccard == 1.0
         assert result.token_overlap == 1.0
 
     def test_repeated_tokens(self, lexical: Lexical) -> None:
+        """Test handling of repeated tokens."""
         candidate = "the the the"
         reference = "the cat"
         result = lexical.score(candidate, reference)
@@ -121,11 +140,15 @@ class TestLexical:
 
 
 class TestLexicalBatch:
+    """Tests for lexical batch scoring."""
+
     @pytest.fixture
     def lexical(self) -> Lexical:
+        """Provide a lexical metric instance."""
         return Lexical()
 
     def test_batch_score_basic(self, lexical: Lexical) -> None:
+        """Test basic batch scoring."""
         candidates = ["The cat sat", "A dog runs"]
         references = ["The cat sat", "A dog runs"]
         result = lexical.batch_score(candidates, references)
@@ -135,6 +158,7 @@ class TestLexicalBatch:
         assert all(r.jaccard == 1.0 for r in result.results)
 
     def test_batch_score_statistics(self, lexical: Lexical) -> None:
+        """Test that batch scoring computes statistics."""
         candidates = ["The cat sat", "Completely different words"]
         references = ["The cat sat", "The cat sat"]
         result = lexical.batch_score(candidates, references)
@@ -151,6 +175,7 @@ class TestLexicalBatch:
         assert result.stats["jaccard"].mean == 0.5
 
     def test_batch_score_percentiles(self, lexical: Lexical) -> None:
+        """Test that batch scoring computes percentiles."""
         candidates = ["a", "b", "c", "d", "e"]
         references = ["a", "b", "c", "d", "e"]
         result = lexical.batch_score(candidates, references)
@@ -162,16 +187,21 @@ class TestLexicalBatch:
         assert 95 in stats.percentiles
 
     def test_batch_score_none_references_raises(self, lexical: Lexical) -> None:
+        """Test that batch scoring raises for None references."""
         with pytest.raises(ValueError, match="requires reference"):
             lexical.batch_score(["text"], None)
 
     def test_batch_score_length_mismatch_raises(self, lexical: Lexical) -> None:
+        """Test that batch scoring raises for mismatched lengths."""
         with pytest.raises(ValueError, match="must match"):
             lexical.batch_score(["a", "b"], ["a"])
 
 
 class TestLexicalResult:
+    """Tests for LexicalResult type."""
+
     def test_frozen(self) -> None:
+        """Test that LexicalResult is frozen."""
         from pydantic import ValidationError
 
         result = LexicalResult(jaccard=0.5, token_overlap=0.7)
@@ -179,6 +209,7 @@ class TestLexicalResult:
             result.jaccard = 0.6  # type: ignore[misc]
 
     def test_values(self) -> None:
+        """Test that values are stored correctly."""
         result = LexicalResult(jaccard=0.5, token_overlap=0.7)
         assert result.jaccard == 0.5
         assert result.token_overlap == 0.7

tests/test_metrics/test_readability.py

@@ -6,17 +6,23 @@ from veritext.metrics import Readability, ReadabilityResult
 
 
 class TestReadability:
+    """Tests for the Readability metric class."""
+
     @pytest.fixture
     def readability(self) -> Readability:
+        """Provide a readability metric instance."""
         return Readability()
 
     def test_name(self, readability: Readability) -> None:
+        """Test that name returns 'readability'."""
         assert readability.name == "readability"
 
     def test_requires_reference(self, readability: Readability) -> None:
+        """Test that readability does NOT require reference text."""
         assert readability.requires_reference is False
 
     def test_simple_text(self, readability: Readability) -> None:
+        """Test readability of simple, easy text."""
         # Simple children's text - short sentences, simple words
         text = "The cat sat. The dog ran. I see a bird."
         result = readability.score(text)
@@ -26,10 +32,11 @@ class TestReadability:
         assert result.flesch_reading_ease > 80.0
 
     def test_complex_text(self, readability: Readability) -> None:
+        """Test readability of complex, academic text."""
         # Complex academic text - long sentences, polysyllabic words
         text = (
             "The implementation of sophisticated computational methodologies "
-            "necessitates thorough understanding of algorithmic complexity "
+            "necessitates comprehensive understanding of algorithmic complexity "
             "and architectural considerations."
         )
         result = readability.score(text)
@@ -39,6 +46,7 @@ class TestReadability:
         assert result.flesch_reading_ease < 30.0
 
     def test_medium_text(self, readability: Readability) -> None:
+        """Test readability of medium-difficulty text."""
         text = (
             "The weather today is quite pleasant. "
             "Many people are enjoying the sunshine in the park. "
@@ -51,6 +59,7 @@ class TestReadability:
         assert 50.0 < result.flesch_reading_ease < 90.0
 
     def test_single_sentence(self, readability: Readability) -> None:
+        """Test readability with a single sentence."""
         text = "The cat sat on the mat."
         result = readability.score(text)
 
@@ -59,6 +68,7 @@ class TestReadability:
         assert result.flesch_reading_ease is not None
 
     def test_single_word(self, readability: Readability) -> None:
+        """Test readability with a single word."""
         text = "Cat"
         result = readability.score(text)
 
@@ -67,18 +77,21 @@ class TestReadability:
         assert result.flesch_reading_ease is not None
 
     def test_empty_text(self, readability: Readability) -> None:
+        """Test that empty text returns zero scores."""
         result = readability.score("")
 
         assert result.flesch_kincaid_grade == 0.0
         assert result.flesch_reading_ease == 0.0
 
     def test_whitespace_only(self, readability: Readability) -> None:
+        """Test that whitespace-only text returns zero scores."""
         result = readability.score("   \t\n  ")
 
         assert result.flesch_kincaid_grade == 0.0
         assert result.flesch_reading_ease == 0.0
 
     def test_reference_ignored(self, readability: Readability) -> None:
+        """Test that reference parameter is ignored."""
         text = "The cat sat on the mat."
 
         # Score with no reference
@@ -94,6 +107,7 @@ class TestReadability:
         assert result1.flesch_kincaid_grade == result3.flesch_kincaid_grade
 
     def test_punctuation_handling(self, readability: Readability) -> None:
+        """Test that punctuation affects sentence counting."""
         # Same words, different sentence structure
         text1 = "The cat sat on the mat"  # 1 sentence
         text2 = "The cat sat. On the mat."  # 2 sentences
@@ -105,6 +119,7 @@ class TestReadability:
         assert result1.flesch_kincaid_grade != result2.flesch_kincaid_grade
 
     def test_question_marks_count_sentences(self, readability: Readability) -> None:
+        """Test that question marks end sentences."""
         text = "What is this? It is a test."
         result = readability.score(text)
 
@@ -113,6 +128,7 @@ class TestReadability:
         assert result.flesch_kincaid_grade is not None
 
     def test_exclamation_marks_count_sentences(self, readability: Readability) -> None:
+        """Test that exclamation marks end sentences."""
         text = "Wow! That is amazing!"
         result = readability.score(text)
 
@@ -120,6 +136,7 @@ class TestReadability:
         assert result.flesch_kincaid_grade is not None
 
     def test_multiple_punctuation(self, readability: Readability) -> None:
+        """Test handling of multiple punctuation marks."""
         text = "What?! That's crazy... Well then."
         result = readability.score(text)
 
@@ -127,10 +144,12 @@ class TestReadability:
         assert result.flesch_kincaid_grade is not None
 
     def test_result_score_property(self, readability: Readability) -> None:
+        """Test that result.score returns flesch_reading_ease."""
         result = readability.score("The cat sat on the mat.")
         assert result.score == result.flesch_reading_ease
 
     def test_contractions(self, readability: Readability) -> None:
+        """Test handling of contractions."""
         text = "I'm going to the store. It's not far away."
         result = readability.score(text)
 
@@ -140,11 +159,15 @@ class TestReadability:
 
 
 class TestReadabilityBatch:
+    """Tests for readability batch scoring."""
+
     @pytest.fixture
     def readability(self) -> Readability:
+        """Provide a readability metric instance."""
         return Readability()
 
     def test_batch_score_basic(self, readability: Readability) -> None:
+        """Test basic batch scoring."""
         candidates = [
             "The cat sat on the mat.",
             "A dog ran through the park.",
@@ -155,6 +178,7 @@ class TestReadabilityBatch:
         assert len(result.results) == 2
 
     def test_batch_score_statistics(self, readability: Readability) -> None:
+        """Test that batch scoring computes statistics."""
         candidates = [
             "Cat sat.",  # Very simple
             "The implementation of sophisticated methodologies requires expertise.",
@@ -172,6 +196,7 @@ class TestReadabilityBatch:
         )
 
     def test_batch_score_percentiles(self, readability: Readability) -> None:
+        """Test that batch scoring computes percentiles."""
         candidates = ["a", "b", "c", "d", "e"]
         result = readability.batch_score(candidates)
 
@@ -182,6 +207,7 @@ class TestReadabilityBatch:
         assert 95 in stats.percentiles
 
     def test_batch_score_references_ignored(self, readability: Readability) -> None:
+        """Test that batch scoring ignores references."""
         candidates = ["The cat sat.", "A dog ran."]
 
         result1 = readability.batch_score(candidates)
@@ -193,12 +219,16 @@ class TestReadabilityBatch:
         )
 
     def test_batch_score_empty_list_raises(self, readability: Readability) -> None:
+        """Test that empty candidate list raises ValueError."""
         with pytest.raises(ValueError, match="empty"):
             readability.batch_score([])
 
 
 class TestReadabilityResult:
+    """Tests for ReadabilityResult type."""
+
     def test_frozen(self) -> None:
+        """Test that ReadabilityResult is frozen."""
         from pydantic import ValidationError
 
         result = ReadabilityResult(flesch_kincaid_grade=5.0, flesch_reading_ease=70.0)
@@ -206,21 +236,27 @@ class TestReadabilityResult:
             result.flesch_kincaid_grade = 6.0  # type: ignore[misc]
 
     def test_values(self) -> None:
+        """Test that values are stored correctly."""
         result = ReadabilityResult(flesch_kincaid_grade=8.5, flesch_reading_ease=65.0)
         assert result.flesch_kincaid_grade == 8.5
         assert result.flesch_reading_ease == 65.0
 
     def test_score_property(self) -> None:
+        """Test that score property returns flesch_reading_ease."""
         result = ReadabilityResult(flesch_kincaid_grade=8.5, flesch_reading_ease=65.0)
         assert result.score == 65.0
 
 
 class TestSyllableCounting:
+    """Tests for syllable counting heuristics."""
+
     @pytest.fixture
     def readability(self) -> Readability:
+        """Provide a readability metric instance."""
         return Readability()
 
     def test_monosyllabic_words(self, readability: Readability) -> None:
+        """Test that monosyllabic words don't inflate scores."""
         # All one-syllable words
         text = "The cat sat on the mat."
         result = readability.score(text)
@@ -229,6 +265,7 @@ class TestSyllableCounting:
         assert result.flesch_reading_ease > 90.0
 
     def test_polysyllabic_words(self, readability: Readability) -> None:
+        """Test that polysyllabic words affect scores."""
         # Words with multiple syllables
         text = "International communication facilitates understanding."
         result = readability.score(text)

tests/test_metrics/test_rouge.py

@@ -6,17 +6,23 @@ from veritext.metrics import Rouge, RougeResult, RougeScore
 
 
 class TestRouge:
+    """Tests for the Rouge metric class."""
+
     @pytest.fixture
     def rouge(self) -> Rouge:
+        """Provide a ROUGE metric instance."""
         return Rouge()
 
     def test_name(self, rouge: Rouge) -> None:
+        """Test that name returns 'rouge'."""
         assert rouge.name == "rouge"
 
     def test_requires_reference(self, rouge: Rouge) -> None:
+        """Test that ROUGE requires reference text."""
         assert rouge.requires_reference is True
 
     def test_identical_texts(self, rouge: Rouge) -> None:
+        """Test that identical texts produce perfect scores."""
         text = "The cat sat on the mat"
         result = rouge.score(text, text)
 
@@ -27,6 +33,7 @@ class TestRouge:
         assert result.rouge_l.fmeasure == 1.0
 
     def test_no_overlap(self, rouge: Rouge) -> None:
+        """Test that texts with no overlap produce zero scores."""
         candidate = "apple banana cherry"
         reference = "dog elephant fox"
         result = rouge.score(candidate, reference)
@@ -38,6 +45,7 @@ class TestRouge:
         assert result.rouge_l.fmeasure == 0.0
 
     def test_partial_overlap_rouge1(self, rouge: Rouge) -> None:
+        """Test ROUGE-1 with partial overlap."""
         candidate = "the cat sat"
         reference = "the dog sat"
         result = rouge.score(candidate, reference)
@@ -49,6 +57,7 @@ class TestRouge:
         assert abs(result.rouge1.recall - 2 / 3) < 1e-10
 
     def test_partial_overlap_rouge2(self, rouge: Rouge) -> None:
+        """Test ROUGE-2 (bigram) with partial overlap."""
         candidate = "the cat sat on the mat"
         reference = "the cat lay on the mat"
         result = rouge.score(candidate, reference)
@@ -61,6 +70,7 @@ class TestRouge:
         assert abs(result.rouge2.recall - 3 / 5) < 1e-10
 
     def test_rouge_l_basic(self, rouge: Rouge) -> None:
+        """Test ROUGE-L (LCS) computation."""
         candidate = "the cat sat on the mat"
         reference = "the cat sat"
         result = rouge.score(candidate, reference)
@@ -71,6 +81,7 @@ class TestRouge:
         assert result.rouge_l.recall == 1.0
 
     def test_rouge_l_non_contiguous(self, rouge: Rouge) -> None:
+        """Test ROUGE-L with non-contiguous LCS."""
         candidate = "the big cat sat"
         reference = "the cat sat"
         result = rouge.score(candidate, reference)
@@ -81,6 +92,7 @@ class TestRouge:
         assert result.rouge_l.recall == 1.0
 
     def test_precision_vs_recall(self, rouge: Rouge) -> None:
+        """Test that precision and recall differ appropriately."""
         # Short candidate, long reference
         candidate = "the cat"
         reference = "the cat sat on the mat"
@@ -92,6 +104,7 @@ class TestRouge:
         assert result.rouge1.recall < 1.0
 
     def test_empty_candidate(self, rouge: Rouge) -> None:
+        """Test that empty candidate returns zero scores."""
         result = rouge.score("", "The cat sat")
 
         assert result.rouge1.fmeasure == 0.0
@@ -99,20 +112,24 @@ class TestRouge:
         assert result.rouge_l.fmeasure == 0.0
 
     def test_whitespace_only_candidate(self, rouge: Rouge) -> None:
+        """Test that whitespace-only candidate returns zero scores."""
         result = rouge.score("   \t\n  ", "The cat sat")
 
         assert result.rouge1.fmeasure == 0.0
         assert result.rouge_l.fmeasure == 0.0
 
     def test_empty_reference_raises(self, rouge: Rouge) -> None:
+        """Test that empty reference raises ValueError."""
         with pytest.raises(ValueError, match="cannot be empty"):
             rouge.score("The cat sat", "")
 
     def test_none_reference_raises(self, rouge: Rouge) -> None:
+        """Test that None reference raises ValueError."""
         with pytest.raises(ValueError, match="requires reference"):
             rouge.score("The cat sat", None)
 
     def test_multiple_references_uses_max(self, rouge: Rouge) -> None:
+        """Test that multiple references use max scores."""
         candidate = "the cat sat on the mat"
         references = [
             "a dog ran across the room",  # Low overlap
@@ -125,6 +142,7 @@ class TestRouge:
         assert result.rouge_l.fmeasure == 1.0
 
     def test_multiple_references_partial(self, rouge: Rouge) -> None:
+        """Test multiple references with partial matches."""
         candidate = "the quick brown fox"
         references = [
             "the fast brown fox",  # 3/4 match
@@ -136,19 +154,23 @@ class TestRouge:
         assert result.rouge1.fmeasure > 0.0
 
     def test_result_score_property(self, rouge: Rouge) -> None:
+        """Test that result.score returns rouge_l.fmeasure."""
         result = rouge.score("The cat sat", "The cat sat")
         assert result.score == result.rouge_l.fmeasure
 
     def test_case_insensitivity(self, rouge: Rouge) -> None:
+        """Test that ROUGE is case insensitive by default."""
         result = rouge.score("THE CAT SAT", "the cat sat")
         assert result.rouge1.fmeasure == 1.0
         assert result.rouge_l.fmeasure == 1.0
 
     def test_punctuation_ignored(self, rouge: Rouge) -> None:
+        """Test that punctuation is ignored by default."""
         result = rouge.score("The cat sat.", "The cat sat!")
         assert result.rouge1.fmeasure == 1.0
 
     def test_single_word(self, rouge: Rouge) -> None:
+        """Test ROUGE with single word texts."""
         result = rouge.score("cat", "cat")
 
         assert result.rouge1.fmeasure == 1.0
@@ -157,6 +179,7 @@ class TestRouge:
         assert result.rouge_l.fmeasure == 1.0
 
     def test_fmeasure_calculation(self, rouge: Rouge) -> None:
+        """Test that F-measure is calculated correctly."""
         # Create a case where P != R
         candidate = "the cat sat on"
         reference = "the cat"
@@ -169,11 +192,15 @@ class TestRouge:
 
 
 class TestRougeBatch:
+    """Tests for ROUGE batch scoring."""
+
     @pytest.fixture
     def rouge(self) -> Rouge:
+        """Provide a ROUGE metric instance."""
         return Rouge()
 
     def test_batch_score_basic(self, rouge: Rouge) -> None:
+        """Test basic batch scoring."""
         candidates = ["The cat sat", "A dog runs"]
         references = ["The cat sat", "A dog runs"]
         result = rouge.batch_score(candidates, references)
@@ -183,6 +210,7 @@ class TestRougeBatch:
         assert all(r.rouge_l.fmeasure == 1.0 for r in result.results)
 
     def test_batch_score_statistics(self, rouge: Rouge) -> None:
+        """Test that batch scoring computes statistics."""
         candidates = ["The cat sat", "Completely different words"]
         references = ["The cat sat", "The cat sat"]
         result = rouge.batch_score(candidates, references)
@@ -199,6 +227,7 @@ class TestRougeBatch:
         assert result.results[1].rouge1.fmeasure == 0.0
 
     def test_batch_score_percentiles(self, rouge: Rouge) -> None:
+        """Test that batch scoring computes percentiles."""
         candidates = ["a", "b", "c", "d", "e"]
         references = ["a", "b", "c", "d", "e"]
         result = rouge.batch_score(candidates, references)
@@ -210,14 +239,17 @@ class TestRougeBatch:
         assert 95 in stats.percentiles
 
     def test_batch_score_none_references_raises(self, rouge: Rouge) -> None:
+        """Test that batch scoring raises for None references."""
         with pytest.raises(ValueError, match="requires reference"):
             rouge.batch_score(["text"], None)
 
     def test_batch_score_length_mismatch_raises(self, rouge: Rouge) -> None:
+        """Test that batch scoring raises for mismatched lengths."""
         with pytest.raises(ValueError, match="must match"):
             rouge.batch_score(["a", "b"], ["a"])
 
-    def test_batch_score_multi_refs(self, rouge: Rouge) -> None:
+    def test_batch_score_with_multiple_references(self, rouge: Rouge) -> None:
+        """Test batch scoring with multiple references per candidate."""
         candidates = [
             "The cat sat on the mat",
             "A quick brown fox",
@@ -235,7 +267,10 @@ class TestRougeBatch:
 
 
 class TestRougeResult:
+    """Tests for RougeResult and RougeScore types."""
+
     def test_rouge_score_frozen(self) -> None:
+        """Test that RougeScore is frozen."""
         from pydantic import ValidationError
 
         score = RougeScore(precision=0.5, recall=0.6, fmeasure=0.55)
@@ -243,6 +278,7 @@ class TestRougeResult:
             score.precision = 0.7  # type: ignore[misc]
 
     def test_rouge_result_frozen(self) -> None:
+        """Test that RougeResult is frozen."""
         from pydantic import ValidationError
 
         score = RougeScore(precision=0.5, recall=0.6, fmeasure=0.55)
@@ -251,6 +287,7 @@ class TestRougeResult:
             result.rouge1 = score  # type: ignore[misc]
 
     def test_score_property(self) -> None:
+        """Test that score property returns rouge_l.fmeasure."""
         r1 = RougeScore(precision=0.9, recall=0.9, fmeasure=0.9)
         r2 = RougeScore(precision=0.8, recall=0.8, fmeasure=0.8)
         rl = RougeScore(precision=0.7, recall=0.7, fmeasure=0.7)

tests/test_pytest_plugin/__init__.py

@@ -1 +0,0 @@
-"""Tests for the Veritext pytest plugin."""

tests/test_pytest_plugin/conftest.py

@@ -1,30 +0,0 @@
-"""Pytest configuration for pytest_plugin tests."""
-
-import pytest
-
-from veritext.pytest_plugin.fixtures import ValidatorFactory
-
-# Enable the pytester fixture for plugin testing
-pytest_plugins = ["pytester"]
-
-# Re-export fixtures from the plugin module for testing
-
-
-@pytest.fixture
-def text_validator() -> ValidatorFactory:
-    return ValidatorFactory()
-
-
-@pytest.fixture
-def validation_context() -> type:
-    from typing import Any
-
-    from veritext.core.types import ValidationContext
-
-    def _create(
-        reference: str | list[str] | None = None,
-        **metadata: Any,
-    ) -> ValidationContext:
-        return ValidationContext(reference=reference, metadata=metadata)
-
-    return _create

tests/test_pytest_plugin/test_assertions.py

@@ -1,171 +0,0 @@
-"""Tests for the validate_text assertion function."""
-
-import pytest
-
-from veritext.pytest_plugin import validate_text
-
-
-class TestValidateTextBasicValidation:
-    def test_valid_length(self) -> None:
-        text = "The quick brown fox jumps over the lazy dog."
-        validate_text(text, min_length=10, max_length=100)
-
-    def test_too_short(self) -> None:
-        text = "Short."
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, min_length=50)
-        assert "length" in str(exc_info.value).lower()
-
-    def test_too_long(self) -> None:
-        text = "A" * 100
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, max_length=50)
-        assert "length" in str(exc_info.value).lower()
-
-
-class TestValidateTextReadability:
-    def test_simple_text_passes(self) -> None:
-        text = "The cat sat on the mat. It was a nice day."
-        validate_text(text, max_reading_grade=10.0)
-
-    def test_complex_text_fails(self) -> None:
-        text = (
-            "The implementation of sophisticated metacognitive strategies "
-            "necessitates the thorough understanding of epistemological "
-            "frameworks and their corresponding methodological implications."
-        )
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, max_reading_grade=3.0)
-        assert "readability" in str(exc_info.value).lower()
-
-
-class TestValidateTextPatterns:
-    def test_contains_pattern(self) -> None:
-        text = "Please contact support@example.com for assistance."
-        validate_text(text, must_contain=["support@example.com"])
-
-    def test_missing_pattern(self) -> None:
-        text = "Please contact us for assistance."
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, must_contain=["@example.com"])
-        assert "contains" in str(exc_info.value).lower()
-
-    def test_excludes_pattern(self) -> None:
-        text = "The report is complete and reviewed."
-        validate_text(text, must_exclude=["TODO", "FIXME"])
-
-    def test_forbidden_pattern(self) -> None:
-        text = "The report is almost done. TODO: add conclusion."
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, must_exclude=["TODO"])
-        assert "excludes" in str(exc_info.value).lower()
-
-
-class TestValidateTextComparisonMetrics:
-    def test_high_bleu_passes(self) -> None:
-        reference = "The quick brown fox jumps over the lazy dog."
-        text = "The quick brown fox jumps over the lazy dog."
-        validate_text(text, reference=reference, min_bleu=0.9)
-
-    def test_low_bleu_fails(self) -> None:
-        reference = "The quick brown fox jumps over the lazy dog."
-        text = "A slow red cat sleeps under the active mouse."
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, reference=reference, min_bleu=0.5)
-        assert "bleu" in str(exc_info.value).lower()
-
-    def test_high_rouge_passes(self) -> None:
-        reference = "Machine learning models require extensive training data."
-        text = "Machine learning models need extensive training data."
-        validate_text(text, reference=reference, min_rouge=0.5)
-
-    def test_low_rouge_fails(self) -> None:
-        reference = "The algorithm processes input data efficiently."
-        text = "Cats enjoy sleeping in sunny spots."
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, reference=reference, min_rouge=0.5)
-        assert "rouge" in str(exc_info.value).lower()
-
-
-class TestValidateTextErrorHandling:
-    def test_requires_criteria(self) -> None:
-        with pytest.raises(ValueError, match="At least one validation criterion"):
-            validate_text("Some text")
-
-    def test_bleu_requires_reference(self) -> None:
-        with pytest.raises(ValueError, match="Reference text required"):
-            validate_text("Some text", min_bleu=0.5)
-
-    def test_rouge_requires_reference(self) -> None:
-        with pytest.raises(ValueError, match="Reference text required"):
-            validate_text("Some text", min_rouge=0.5)
-
-    def test_semantic_requires_reference(self) -> None:
-        with pytest.raises(ValueError, match="Reference text required"):
-            validate_text("Some text", min_semantic=0.5)
-
-
-class TestValidateTextMultipleCriteria:
-    def test_all_criteria_pass(self) -> None:
-        reference = "The quick brown fox jumps over the lazy dog."
-        text = "The quick brown fox jumps over the lazy dog."
-        validate_text(
-            text,
-            reference=reference,
-            min_bleu=0.9,
-            min_length=10,
-            max_length=100,
-        )
-
-    def test_one_failure_fails_all(self) -> None:
-        reference = "The quick brown fox jumps over the lazy dog."
-        text = "The quick brown fox jumps over the lazy dog."
-        with pytest.raises(AssertionError):
-            validate_text(
-                text,
-                reference=reference,
-                min_bleu=0.9,
-                max_length=10,  # This will fail
-            )
-
-
-class TestValidateTextFailureMessage:
-    def test_includes_text_preview(self) -> None:
-        text = "Short text"
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, min_length=100)
-        assert "Short text" in str(exc_info.value)
-
-    def test_truncates_long_text(self) -> None:
-        text = "A" * 200
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, max_length=50)
-        message = str(exc_info.value)
-        assert "..." in message
-        assert "A" * 200 not in message
-
-    def test_includes_check_details(self) -> None:
-        text = "Short"
-        with pytest.raises(AssertionError) as exc_info:
-            validate_text(text, min_length=100)
-        message = str(exc_info.value)
-        assert "Failed checks:" in message
-        assert "length" in message.lower()
-
-
-class TestValidateTextListReference:
-    def test_bleu_multi_reference(self) -> None:
-        references = [
-            "The quick brown fox jumps over the lazy dog.",
-            "A fast brown fox leaps over a sleepy dog.",
-        ]
-        text = "The quick brown fox jumps over the lazy dog."
-        validate_text(text, reference=references, min_bleu=0.9)
-
-    def test_rouge_multi_reference(self) -> None:
-        references = [
-            "Machine learning requires data.",
-            "ML models need training data.",
-        ]
-        text = "Machine learning models require training data."
-        validate_text(text, reference=references, min_rouge=0.3)

tests/test_pytest_plugin/test_fixtures.py

@@ -1,74 +0,0 @@
-"""Tests for the pytest plugin fixtures."""
-
-from veritext.core.types import ValidationContext
-from veritext.pytest_plugin.fixtures import ValidatorFactory
-from veritext.validators import bleu, length
-
-
-class TestValidatorFactory:
-    def test_creates_validator_from_checks(self) -> None:
-        factory = ValidatorFactory()
-        validate = factory(checks=[length(min_chars=5)])
-
-        result = validate("Hello, World!")
-        assert result.passed
-
-    def test_validator_uses_provided_reference(self) -> None:
-        factory = ValidatorFactory()
-        reference = "The quick brown fox."
-        validate = factory(
-            checks=[bleu(min_score=0.5)],
-            reference=reference,
-        )
-
-        # Exact match should pass
-        result = validate("The quick brown fox.")
-        assert result.passed
-
-    def test_validator_returns_validation_result(self) -> None:
-        factory = ValidatorFactory()
-        validate = factory(checks=[length(min_chars=100)])
-
-        result = validate("Short")
-        assert not result.passed
-        assert len(result.checks) == 1
-        assert result.checks[0].name == "length"
-
-
-class TestTextValidatorFixture:
-    def test_fixture_returns_factory(self, text_validator: ValidatorFactory) -> None:
-        assert isinstance(text_validator, ValidatorFactory)
-
-    def test_fixture_can_create_validators(
-        self,
-        text_validator: ValidatorFactory,
-    ) -> None:
-        validate = text_validator(checks=[length(min_chars=5, max_chars=50)])
-
-        assert validate("Hello, World!").passed
-        assert not validate("Hi").passed
-
-
-class TestValidationContextFixture:
-    def test_fixture_creates_context(
-        self,
-        validation_context: type,
-    ) -> None:
-        ctx = validation_context(reference="Test reference")
-        assert isinstance(ctx, ValidationContext)
-        assert ctx.reference == "Test reference"
-
-    def test_fixture_accepts_metadata(
-        self,
-        validation_context: type,
-    ) -> None:
-        ctx = validation_context(reference="Test", source="unit_test", version=1)
-        assert ctx.metadata["source"] == "unit_test"
-        assert ctx.metadata["version"] == 1
-
-    def test_fixture_allows_no_reference(
-        self,
-        validation_context: type,
-    ) -> None:
-        ctx = validation_context()
-        assert ctx.reference is None

tests/test_pytest_plugin/test_plugin.py

@@ -1,89 +0,0 @@
-"""Tests for the pytest plugin hooks."""
-
-import pytest
-
-
-@pytest.fixture
-def plugin_pytester(pytester: pytest.Pytester) -> pytest.Pytester:
-    return pytester
-
-
-def test_plugin_registers_marker(plugin_pytester: pytest.Pytester) -> None:
-    plugin_pytester.makepyfile(
-        """
-        import pytest
-
-        @pytest.mark.text_validation
-        def test_example():
-            pass
-        """
-    )
-    # Run with strict markers - this will fail if marker isn't registered
-    result = plugin_pytester.runpytest("--strict-markers")
-    result.assert_outcomes(passed=1)
-
-
-def test_marker_can_be_used(plugin_pytester: pytest.Pytester) -> None:
-    plugin_pytester.makepyfile(
-        """
-        import pytest
-
-        @pytest.mark.text_validation
-        def test_marked():
-            pass
-
-        def test_unmarked():
-            pass
-        """
-    )
-    # Run only marked tests
-    result = plugin_pytester.runpytest("-m", "text_validation")
-    result.assert_outcomes(passed=1)
-
-
-def test_validate_text_is_importable(plugin_pytester: pytest.Pytester) -> None:
-    plugin_pytester.makepyfile(
-        """
-        from veritext.pytest_plugin import validate_text
-
-        def test_import():
-            assert callable(validate_text)
-        """
-    )
-    result = plugin_pytester.runpytest()
-    result.assert_outcomes(passed=1)
-
-
-def test_validate_text_works_in_tests(plugin_pytester: pytest.Pytester) -> None:
-    plugin_pytester.makepyfile(
-        """
-        from veritext.pytest_plugin import validate_text
-
-        def test_validation_passes():
-            validate_text(
-                "The quick brown fox jumps over the lazy dog.",
-                min_length=10,
-                max_length=100,
-            )
-        """
-    )
-    result = plugin_pytester.runpytest()
-    result.assert_outcomes(passed=1)
-
-
-def test_validate_text_failure_in_tests(plugin_pytester: pytest.Pytester) -> None:
-    plugin_pytester.makepyfile(
-        """
-        from veritext.pytest_plugin import validate_text
-
-        def test_validation_fails():
-            validate_text(
-                "Short",
-                min_length=100,
-            )
-        """
-    )
-    result = plugin_pytester.runpytest()
-    result.assert_outcomes(failed=1)
-    # Check that failure message contains useful information
-    result.stdout.fnmatch_lines(["*Text validation failed*"])

tests/test_semantic/test_similarity.py

@@ -10,17 +10,23 @@ from veritext.semantic import SemanticSimilarity
 
 
 class TestSemanticSimilarity:
+    """Tests for the SemanticSimilarity metric class."""
+
     @pytest.fixture
     def semantic(self) -> SemanticSimilarity:
+        """Provide a SemanticSimilarity metric instance."""
         return SemanticSimilarity()
 
     def test_name(self, semantic: SemanticSimilarity) -> None:
+        """Test that name returns 'semantic'."""
         assert semantic.name == "semantic"
 
     def test_requires_reference(self, semantic: SemanticSimilarity) -> None:
+        """Test that semantic similarity requires reference text."""
         assert semantic.requires_reference is True
 
     def test_identical_texts(self, semantic: SemanticSimilarity) -> None:
+        """Test that identical texts produce high similarity."""
         text = "The cat sat on the mat"
         result = semantic.score(text, text)
 
@@ -29,6 +35,7 @@ class TestSemanticSimilarity:
         assert result.model == "all-MiniLM-L6-v2"
 
     def test_semantically_similar_texts(self, semantic: SemanticSimilarity) -> None:
+        """Test that semantically similar texts have high similarity."""
         candidate = "The cat sat on the mat"
         reference = "A feline rested on the rug"
         result = semantic.score(candidate, reference)
@@ -37,6 +44,7 @@ class TestSemanticSimilarity:
         assert result.similarity > 0.3
 
     def test_unrelated_texts(self, semantic: SemanticSimilarity) -> None:
+        """Test that unrelated texts have low similarity."""
         candidate = "The quick brown fox"
         reference = "Quantum physics describes particle behaviour"
         result = semantic.score(candidate, reference)
@@ -45,26 +53,32 @@ class TestSemanticSimilarity:
         assert result.similarity < 0.5
 
     def test_empty_candidate(self, semantic: SemanticSimilarity) -> None:
+        """Test that empty candidate returns zero similarity."""
         result = semantic.score("", "The cat sat on the mat")
         assert result.similarity == 0.0
 
     def test_whitespace_only_candidate(self, semantic: SemanticSimilarity) -> None:
+        """Test that whitespace-only candidate returns zero similarity."""
         result = semantic.score("   \t\n  ", "The cat sat on the mat")
         assert result.similarity == 0.0
 
     def test_none_reference_raises(self, semantic: SemanticSimilarity) -> None:
+        """Test that None reference raises ValueError."""
         with pytest.raises(ValueError, match="requires reference"):
             semantic.score("The cat sat", None)
 
     def test_empty_reference_raises(self, semantic: SemanticSimilarity) -> None:
+        """Test that empty reference raises ValueError."""
         with pytest.raises(ValueError, match="cannot be empty"):
             semantic.score("The cat sat", "")
 
     def test_whitespace_reference_raises(self, semantic: SemanticSimilarity) -> None:
+        """Test that whitespace-only reference raises ValueError."""
         with pytest.raises(ValueError, match="cannot be empty"):
             semantic.score("The cat sat", "   \t\n  ")
 
     def test_multiple_references(self, semantic: SemanticSimilarity) -> None:
+        """Test semantic similarity with multiple references uses max."""
         candidate = "The cat sat on the mat"
         references = [
             "A dog ran through the park",
@@ -76,6 +90,7 @@ class TestSemanticSimilarity:
         assert result.similarity >= 0.99
 
     def test_multiple_references_takes_max(self, semantic: SemanticSimilarity) -> None:
+        """Test that multiple references returns maximum similarity."""
         candidate = "The cat sat on the mat"
         references = [
             "Quantum physics is complex",  # Low similarity
@@ -87,10 +102,12 @@ class TestSemanticSimilarity:
         assert result.similarity > 0.3
 
     def test_result_score_property(self, semantic: SemanticSimilarity) -> None:
+        """Test that result.score returns similarity."""
         result = semantic.score("The cat sat", "The cat sat")
         assert result.score == result.similarity
 
     def test_caching_behaviour(self) -> None:
+        """Test that caching works for repeated texts."""
         semantic = SemanticSimilarity(cache_embeddings=True)
 
         # Score same texts multiple times
@@ -107,6 +124,7 @@ class TestSemanticSimilarity:
         assert result3.similarity == result1.similarity
 
     def test_caching_disabled(self) -> None:
+        """Test that caching can be disabled."""
         semantic = SemanticSimilarity(cache_embeddings=False)
 
         text = "The cat sat on the mat"
@@ -120,6 +138,7 @@ class TestSemanticSimilarity:
         semantic.clear_cache()
 
     def test_custom_model(self) -> None:
+        """Test that custom model name is recorded in result."""
         # Use the same model but verify it's recorded correctly
         semantic = SemanticSimilarity(model="all-MiniLM-L6-v2")
         result = semantic.score("Test text", "Test text")
@@ -127,11 +146,15 @@ class TestSemanticSimilarity:
 
 
 class TestSemanticSimilarityBatch:
+    """Tests for semantic similarity batch scoring."""
+
     @pytest.fixture
     def semantic(self) -> SemanticSimilarity:
+        """Provide a SemanticSimilarity metric instance."""
         return SemanticSimilarity()
 
     def test_batch_score_basic(self, semantic: SemanticSimilarity) -> None:
+        """Test basic batch scoring."""
         candidates = ["The cat sat on the mat", "A quick brown dog runs fast"]
         references = ["The cat sat on the mat", "A quick brown dog runs fast"]
         result = semantic.batch_score(candidates, references)
@@ -142,6 +165,7 @@ class TestSemanticSimilarityBatch:
         assert all(r.similarity >= 0.99 for r in result.results)
 
     def test_batch_score_statistics(self, semantic: SemanticSimilarity) -> None:
+        """Test that batch scoring computes statistics."""
         candidates = ["The cat sat", "Quantum physics is complex"]
         references = ["The cat sat", "The cat sat"]
         result = semantic.batch_score(candidates, references)
@@ -154,6 +178,7 @@ class TestSemanticSimilarityBatch:
         assert stats.min <= stats.mean <= stats.max
 
     def test_batch_score_percentiles(self, semantic: SemanticSimilarity) -> None:
+        """Test that batch scoring computes percentiles."""
         candidates = ["a", "b", "c", "d", "e"]
         references = ["a", "b", "c", "d", "e"]
         result = semantic.batch_score(candidates, references)
@@ -167,18 +192,21 @@ class TestSemanticSimilarityBatch:
     def test_batch_score_none_references_raises(
         self, semantic: SemanticSimilarity
     ) -> None:
+        """Test that batch scoring raises for None references."""
         with pytest.raises(ValueError, match="requires reference"):
             semantic.batch_score(["text"], None)
 
     def test_batch_score_length_mismatch_raises(
         self, semantic: SemanticSimilarity
     ) -> None:
+        """Test that batch scoring raises for mismatched lengths."""
         with pytest.raises(ValueError, match="must match"):
             semantic.batch_score(["a", "b"], ["a"])
 
-    def test_batch_score_multi_refs(
+    def test_batch_score_with_multiple_references(
         self, semantic: SemanticSimilarity
     ) -> None:
+        """Test batch scoring with multiple references per candidate."""
         candidates = [
             "The cat sat on the mat",
             "A quick brown dog runs fast",
@@ -196,7 +224,10 @@ class TestSemanticSimilarityBatch:
 
 
 class TestSemanticResult:
+    """Tests for SemanticResult type."""
+
     def test_frozen(self) -> None:
+        """Test that SemanticResult is frozen."""
         from pydantic import ValidationError
 
         result = SemanticResult(similarity=0.85, model="test-model")
@@ -204,5 +235,6 @@ class TestSemanticResult:
             result.similarity = 0.9  # type: ignore[misc]
 
     def test_score_property(self) -> None:
+        """Test that score property returns similarity."""
         result = SemanticResult(similarity=0.75, model="test-model")
         assert result.score == 0.75

tests/test_validators/test_composite.py

@@ -8,7 +8,10 @@ from veritext.validators.composite import AllOf, AnyOf
 
 
 class TestAllOf:
+    """Tests for AllOf composite validator."""
+
     def test_all_of_passes_when_all_checks_pass(self) -> None:
+        """Test that AllOf passes when all checks pass."""
         validator = AllOf(
             checks=[
                 length(min_words=2),
@@ -23,6 +26,7 @@ class TestAllOf:
         assert all(c.passed for c in result.checks)
 
     def test_all_of_fails_when_one_check_fails(self) -> None:
+        """Test that AllOf fails when any check fails."""
         validator = AllOf(
             checks=[
                 length(min_words=2),
@@ -37,6 +41,7 @@ class TestAllOf:
         assert len(result.failed_checks) == 1
 
     def test_all_of_fails_when_all_checks_fail(self) -> None:
+        """Test that AllOf fails when all checks fail."""
         validator = AllOf(
             checks=[
                 length(min_words=10),
@@ -50,6 +55,7 @@ class TestAllOf:
         assert len(result.failed_checks) == 2
 
     def test_all_of_with_metric_validators(self) -> None:
+        """Test AllOf with metric-based validators."""
         validator = AllOf(
             checks=[
                 bleu(min_score=0.5),
@@ -63,6 +69,7 @@ class TestAllOf:
         assert len(result.checks) == 2
 
     def test_all_of_failure_summary(self) -> None:
+        """Test the failure summary property."""
         validator = AllOf(
             checks=[
                 length(min_words=10),
@@ -78,20 +85,26 @@ class TestAllOf:
         assert "contains" in summary
 
     def test_all_of_raises_on_empty_checks(self) -> None:
+        """Test that empty checks list raises error."""
         with pytest.raises(ValueError, match="cannot be empty"):
             AllOf(checks=[])
 
     def test_all_of_name_property(self) -> None:
+        """Test the name property."""
         validator = AllOf(checks=[length(min_chars=1)])
         assert validator.name == "all_of"
 
     def test_all_of_factory_function(self) -> None:
+        """Test the all_of() factory function."""
         validator = all_of(checks=[length(min_chars=1)])
         assert isinstance(validator, AllOf)
 
 
 class TestAnyOf:
+    """Tests for AnyOf composite validator."""
+
     def test_any_of_passes_when_any_check_passes(self) -> None:
+        """Test that AnyOf passes when any check passes."""
         validator = AnyOf(
             checks=[
                 length(min_words=10),  # Will fail
@@ -107,6 +120,7 @@ class TestAnyOf:
         assert any(c.passed for c in result.checks)
 
     def test_any_of_passes_when_all_checks_pass(self) -> None:
+        """Test that AnyOf passes when all checks pass."""
         validator = AnyOf(
             checks=[
                 length(min_words=2),
@@ -120,6 +134,7 @@ class TestAnyOf:
         assert all(c.passed for c in result.checks)
 
     def test_any_of_fails_when_all_checks_fail(self) -> None:
+        """Test that AnyOf fails when all checks fail."""
         validator = AnyOf(
             checks=[
                 length(min_words=10),
@@ -133,6 +148,7 @@ class TestAnyOf:
         assert not any(c.passed for c in result.checks)
 
     def test_any_of_with_metric_validators(self) -> None:
+        """Test AnyOf with metric-based validators."""
         validator = AnyOf(
             checks=[
                 bleu(min_score=0.9),  # Might fail
@@ -145,6 +161,7 @@ class TestAnyOf:
         assert result.passed is True  # Length check passes
 
     def test_any_of_with_excludes(self) -> None:
+        """Test AnyOf with excludes validator."""
         validator = AnyOf(
             checks=[
                 excludes(patterns=["error"]),
@@ -166,13 +183,16 @@ class TestAnyOf:
         assert result.passed is False
 
     def test_any_of_raises_on_empty_checks(self) -> None:
+        """Test that empty checks list raises error."""
         with pytest.raises(ValueError, match="cannot be empty"):
             AnyOf(checks=[])
 
     def test_any_of_name_property(self) -> None:
+        """Test the name property."""
         validator = AnyOf(checks=[length(min_chars=1)])
         assert validator.name == "any_of"
 
     def test_any_of_factory_function(self) -> None:
+        """Test the any_of() factory function."""
         validator = any_of(checks=[length(min_chars=1)])
         assert isinstance(validator, AnyOf)

tests/test_validators/test_constraint.py

@@ -14,7 +14,10 @@ from veritext.validators.constraint import (
 
 
 class TestLengthValidator:
-    def test_min_chars_passes(self) -> None:
+    """Tests for LengthValidator."""
+
+    def test_length_validator_min_chars_passes(self) -> None:
+        """Test that validator passes when char count meets minimum."""
         validator = LengthValidator(min_chars=10)
         context = ValidationContext()
         result = validator.check("hello world!", context)
@@ -23,7 +26,8 @@ class TestLengthValidator:
         assert result.name == "length"
         assert result.actual["chars"] == 12
 
-    def test_min_chars_fails(self) -> None:
+    def test_length_validator_min_chars_fails(self) -> None:
+        """Test that validator fails when char count below minimum."""
         validator = LengthValidator(min_chars=20)
         context = ValidationContext()
         result = validator.check("hello", context)
@@ -31,7 +35,8 @@ class TestLengthValidator:
         assert result.passed is False
         assert "< min" in result.message
 
-    def test_max_chars_passes(self) -> None:
+    def test_length_validator_max_chars_passes(self) -> None:
+        """Test that validator passes when char count within maximum."""
         validator = LengthValidator(max_chars=20)
         context = ValidationContext()
         result = validator.check("hello world", context)
@@ -39,7 +44,8 @@ class TestLengthValidator:
         assert result.passed is True
         assert result.actual["chars"] == 11
 
-    def test_max_chars_fails(self) -> None:
+    def test_length_validator_max_chars_fails(self) -> None:
+        """Test that validator fails when char count exceeds maximum."""
         validator = LengthValidator(max_chars=5)
         context = ValidationContext()
         result = validator.check("hello world", context)
@@ -47,7 +53,8 @@ class TestLengthValidator:
         assert result.passed is False
         assert "> max" in result.message
 
-    def test_min_words_passes(self) -> None:
+    def test_length_validator_min_words_passes(self) -> None:
+        """Test that validator passes when word count meets minimum."""
         validator = LengthValidator(min_words=3)
         context = ValidationContext()
         result = validator.check("the quick brown fox", context)
@@ -55,7 +62,8 @@ class TestLengthValidator:
         assert result.passed is True
         assert result.actual["words"] == 4
 
-    def test_min_words_fails(self) -> None:
+    def test_length_validator_min_words_fails(self) -> None:
+        """Test that validator fails when word count below minimum."""
         validator = LengthValidator(min_words=10)
         context = ValidationContext()
         result = validator.check("hello world", context)
@@ -63,14 +71,16 @@ class TestLengthValidator:
         assert result.passed is False
         assert "words < min" in result.message
 
-    def test_max_words_passes(self) -> None:
+    def test_length_validator_max_words_passes(self) -> None:
+        """Test that validator passes when word count within maximum."""
         validator = LengthValidator(max_words=5)
         context = ValidationContext()
         result = validator.check("hello world", context)
 
         assert result.passed is True
 
-    def test_max_words_fails(self) -> None:
+    def test_length_validator_max_words_fails(self) -> None:
+        """Test that validator fails when word count exceeds maximum."""
         validator = LengthValidator(max_words=2)
         context = ValidationContext()
         result = validator.check("the quick brown fox", context)
@@ -78,7 +88,8 @@ class TestLengthValidator:
         assert result.passed is False
         assert "words > max" in result.message
 
-    def test_combined_constraints(self) -> None:
+    def test_length_validator_combined_constraints(self) -> None:
+        """Test validator with multiple constraints."""
         validator = LengthValidator(
             min_chars=5, max_chars=50, min_words=2, max_words=10
         )
@@ -91,11 +102,13 @@ class TestLengthValidator:
         assert "min_words" in result.threshold
         assert "max_words" in result.threshold
 
-    def test_requires_constraints(self) -> None:
+    def test_length_validator_raises_when_no_constraints(self) -> None:
+        """Test that validator raises when no constraints provided."""
         with pytest.raises(InvalidThresholdError, match="At least one"):
             LengthValidator()
 
-    def test_rejects_negative_values(self) -> None:
+    def test_length_validator_raises_on_negative_values(self) -> None:
+        """Test that negative constraint values raise error."""
         with pytest.raises(InvalidThresholdError, match="min_chars must be >= 0"):
             LengthValidator(min_chars=-1)
 
@@ -108,21 +121,26 @@ class TestLengthValidator:
         with pytest.raises(InvalidThresholdError, match="max_words must be >= 0"):
             LengthValidator(max_words=-1)
 
-    def test_rejects_inverted_range(self) -> None:
+    def test_length_validator_raises_on_invalid_range(self) -> None:
+        """Test that min > max raises error."""
         with pytest.raises(InvalidThresholdError, match="cannot exceed max_chars"):
             LengthValidator(min_chars=100, max_chars=50)
 
         with pytest.raises(InvalidThresholdError, match="cannot exceed max_words"):
             LengthValidator(min_words=20, max_words=5)
 
-    def test_length_factory(self) -> None:
+    def test_length_factory_function(self) -> None:
+        """Test the length() factory function."""
         validator = length(min_chars=10, max_words=100)
         assert isinstance(validator, LengthValidator)
         assert validator.name == "length"
 
 
 class TestReadabilityValidator:
-    def test_max_grade_passes(self) -> None:
+    """Tests for ReadabilityValidator."""
+
+    def test_readability_validator_max_grade_passes(self) -> None:
+        """Test that validator passes when grade level within maximum."""
         validator = ReadabilityValidator(max_grade=12.0)
         context = ValidationContext()
         # Simple text should have low grade level
@@ -132,13 +150,14 @@ class TestReadabilityValidator:
         assert result.name == "readability"
         assert "grade" in result.actual
 
-    def test_max_grade_fails(self) -> None:
+    def test_readability_validator_max_grade_fails(self) -> None:
+        """Test that validator fails when grade level exceeds maximum."""
         validator = ReadabilityValidator(max_grade=1.0)
         context = ValidationContext()
         # Complex text
         result = validator.check(
             "The implementation of sophisticated methodologies necessitates "
-            "detailed analytical frameworks for systematic evaluation.",
+            "comprehensive analytical frameworks for systematic evaluation.",
             context,
         )
 
@@ -146,7 +165,8 @@ class TestReadabilityValidator:
         assert "grade level" in result.message
         assert "> max" in result.message
 
-    def test_min_ease_passes(self) -> None:
+    def test_readability_validator_min_ease_passes(self) -> None:
+        """Test that validator passes when reading ease meets minimum."""
         validator = ReadabilityValidator(min_ease=30.0)
         context = ValidationContext()
         # Simple text should have high reading ease
@@ -155,12 +175,13 @@ class TestReadabilityValidator:
         assert result.passed is True
         assert "ease" in result.actual
 
-    def test_min_ease_fails(self) -> None:
+    def test_readability_validator_min_ease_fails(self) -> None:
+        """Test that validator fails when reading ease below minimum."""
         validator = ReadabilityValidator(min_ease=100.0)
         context = ValidationContext()
         result = validator.check(
             "The implementation of sophisticated methodologies necessitates "
-            "detailed analytical frameworks.",
+            "comprehensive analytical frameworks.",
             context,
         )
 
@@ -168,7 +189,8 @@ class TestReadabilityValidator:
         assert "reading ease" in result.message
         assert "< min" in result.message
 
-    def test_combined_grade_and_ease(self) -> None:
+    def test_readability_validator_combined_constraints(self) -> None:
+        """Test validator with both grade and ease constraints."""
         validator = ReadabilityValidator(max_grade=12.0, min_ease=30.0)
         context = ValidationContext()
         result = validator.check("The cat sat on the mat.", context)
@@ -176,18 +198,23 @@ class TestReadabilityValidator:
         assert "max_grade" in result.threshold
         assert "min_ease" in result.threshold
 
-    def test_requires_constraints(self) -> None:
+    def test_readability_validator_raises_when_no_constraints(self) -> None:
+        """Test that validator raises when no constraints provided."""
         with pytest.raises(InvalidThresholdError, match="At least one"):
             ReadabilityValidator()
 
-    def test_readability_factory(self) -> None:
+    def test_readability_factory_function(self) -> None:
+        """Test the readability() factory function."""
         validator = readability(max_grade=8.0, min_ease=60.0)
         assert isinstance(validator, ReadabilityValidator)
         assert validator.name == "readability"
 
 
 class TestContainsValidator:
-    def test_finds_all_patterns(self) -> None:
+    """Tests for ContainsValidator."""
+
+    def test_contains_validator_passes_when_pattern_found(self) -> None:
+        """Test that validator passes when all patterns are found."""
         validator = ContainsValidator(patterns=["hello", "world"])
         context = ValidationContext()
         result = validator.check("Hello World!", context)
@@ -197,7 +224,8 @@ class TestContainsValidator:
         assert result.actual["found"] == 2
         assert result.actual["missing"] == []
 
-    def test_fails_on_missing_pattern(self) -> None:
+    def test_contains_validator_fails_when_pattern_missing(self) -> None:
+        """Test that validator fails when a pattern is missing."""
         validator = ContainsValidator(patterns=["hello", "goodbye"])
         context = ValidationContext()
         result = validator.check("Hello World!", context)
@@ -206,43 +234,47 @@ class TestContainsValidator:
         assert "goodbye" in result.actual["missing"]
         assert "missing" in result.message
 
-    def test_case_insensitive_default(self) -> None:
+    def test_contains_validator_case_insensitive_by_default(self) -> None:
+        """Test that matching is case-insensitive by default."""
         validator = ContainsValidator(patterns=["HELLO"])
         context = ValidationContext()
         result = validator.check("hello world", context)
 
         assert result.passed is True
 
-    def test_case_sensitive(self) -> None:
+    def test_contains_validator_case_sensitive(self) -> None:
+        """Test case-sensitive matching."""
         validator = ContainsValidator(patterns=["HELLO"], case_sensitive=True)
         context = ValidationContext()
         result = validator.check("hello world", context)
 
         assert result.passed is False
 
-    def test_regex_patterns(self) -> None:
+    def test_contains_validator_regex_patterns(self) -> None:
+        """Test regex pattern matching."""
         validator = ContainsValidator(patterns=[r"\d{3}-\d{4}"])
         context = ValidationContext()
         result = validator.check("Call 555-1234 for info", context)
 
         assert result.passed is True
 
-    def test_rejects_empty_patterns(self) -> None:
+    def test_contains_validator_raises_on_empty_patterns(self) -> None:
+        """Test that empty patterns list raises error."""
         with pytest.raises(InvalidThresholdError, match="cannot be empty"):
             ContainsValidator(patterns=[])
 
-    def test_rejects_invalid_regex(self) -> None:
+    def test_contains_factory_function(self) -> None:
-        with pytest.raises(InvalidThresholdError, match="Invalid regex"):
+        """Test the contains() factory function."""
-            ContainsValidator(patterns=[r"[invalid"])
-
-    def test_contains_factory(self) -> None:
         validator = contains(patterns=["test"], case_sensitive=True)
         assert isinstance(validator, ContainsValidator)
         assert validator.name == "contains"
 
 
 class TestExcludesValidator:
-    def test_passes_when_absent(self) -> None:
+    """Tests for ExcludesValidator."""
+
+    def test_excludes_validator_passes_when_pattern_absent(self) -> None:
+        """Test that validator passes when all patterns are absent."""
         validator = ExcludesValidator(patterns=["bad", "forbidden"])
         context = ValidationContext()
         result = validator.check("This is good text.", context)
@@ -251,7 +283,8 @@ class TestExcludesValidator:
         assert result.name == "excludes"
         assert result.actual["found"] == []
 
-    def test_fails_when_found(self) -> None:
+    def test_excludes_validator_fails_when_pattern_found(self) -> None:
+        """Test that validator fails when a forbidden pattern is found."""
         validator = ExcludesValidator(patterns=["bad", "forbidden"])
         context = ValidationContext()
         result = validator.check("This is bad text.", context)
@@ -260,21 +293,24 @@ class TestExcludesValidator:
         assert "bad" in result.actual["found"]
         assert "forbidden" in result.message
 
-    def test_case_insensitive_default(self) -> None:
+    def test_excludes_validator_case_insensitive_by_default(self) -> None:
+        """Test that matching is case-insensitive by default."""
         validator = ExcludesValidator(patterns=["BAD"])
         context = ValidationContext()
         result = validator.check("This is bad text.", context)
 
         assert result.passed is False
 
-    def test_case_sensitive(self) -> None:
+    def test_excludes_validator_case_sensitive(self) -> None:
+        """Test case-sensitive matching."""
         validator = ExcludesValidator(patterns=["BAD"], case_sensitive=True)
         context = ValidationContext()
         result = validator.check("This is bad text.", context)
 
         assert result.passed is True
 
-    def test_regex_patterns(self) -> None:
+    def test_excludes_validator_regex_patterns(self) -> None:
+        """Test regex pattern matching."""
         validator = ExcludesValidator(patterns=[r"\b\d{4}\b"])  # 4-digit numbers
         context = ValidationContext()
 
@@ -286,15 +322,13 @@ class TestExcludesValidator:
         result = validator.check("No numbers here", context)
         assert result.passed is True
 
-    def test_rejects_empty_patterns(self) -> None:
+    def test_excludes_validator_raises_on_empty_patterns(self) -> None:
+        """Test that empty patterns list raises error."""
         with pytest.raises(InvalidThresholdError, match="cannot be empty"):
             ExcludesValidator(patterns=[])
 
-    def test_rejects_invalid_regex(self) -> None:
+    def test_excludes_factory_function(self) -> None:
-        with pytest.raises(InvalidThresholdError, match="Invalid regex"):
+        """Test the excludes() factory function."""
-            ExcludesValidator(patterns=[r"[invalid"])
-
-    def test_excludes_factory(self) -> None:
         validator = excludes(patterns=["test"], case_sensitive=True)
         assert isinstance(validator, ExcludesValidator)
         assert validator.name == "excludes"

tests/test_validators/test_metric.py

@@ -9,7 +9,10 @@ from veritext.validators.metric import BleuValidator, LexicalValidator, RougeVal
 
 
 class TestBleuValidator:
-    def test_bleu_passes_above_threshold(self) -> None:
+    """Tests for BleuValidator."""
+
+    def test_bleu_validator_passes_when_score_meets_threshold(self) -> None:
+        """Test that validator passes when BLEU score meets threshold."""
         validator = BleuValidator(min_score=0.5, variant=4)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("the cat sat on the mat", context)
@@ -19,7 +22,8 @@ class TestBleuValidator:
         assert result.actual == 1.0  # Identical text
         assert result.threshold == 0.5
 
-    def test_bleu_fails_below_threshold(self) -> None:
+    def test_bleu_validator_fails_when_score_below_threshold(self) -> None:
+        """Test that validator fails when BLEU score is below threshold."""
         validator = BleuValidator(min_score=0.9, variant=4)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("a dog ran through the park", context)
@@ -29,7 +33,8 @@ class TestBleuValidator:
         assert result.actual < 0.9
         assert "below minimum" in result.message
 
-    def test_bleu_variant_selection(self) -> None:
+    def test_bleu_validator_variant_selection(self) -> None:
+        """Test different BLEU variants."""
         context = ValidationContext(reference="the quick brown fox jumps")
 
         for variant in (1, 2, 3, 4):
@@ -37,32 +42,39 @@ class TestBleuValidator:
             result = validator.check("the quick brown fox", context)
             assert result.name == f"bleu-{variant}"
 
-    def test_bleu_requires_reference(self) -> None:
+    def test_bleu_validator_raises_on_missing_reference(self) -> None:
+        """Test that validator raises when reference is missing."""
         validator = BleuValidator(min_score=0.5)
         context = ValidationContext()
 
         with pytest.raises(ValidationError, match="requires reference text"):
             validator.check("some text", context)
 
-    def test_bleu_rejects_invalid_score(self) -> None:
+    def test_bleu_validator_raises_on_invalid_min_score(self) -> None:
+        """Test that invalid min_score raises error."""
         with pytest.raises(InvalidThresholdError, match=r"between 0\.0 and 1\.0"):
             BleuValidator(min_score=1.5)
 
         with pytest.raises(InvalidThresholdError, match=r"between 0\.0 and 1\.0"):
             BleuValidator(min_score=-0.1)
 
-    def test_bleu_rejects_invalid_variant(self) -> None:
+    def test_bleu_validator_raises_on_invalid_variant(self) -> None:
+        """Test that invalid variant raises error."""
         with pytest.raises(InvalidThresholdError, match="variant must be"):
             BleuValidator(min_score=0.5, variant=5)  # type: ignore[arg-type]
 
-    def test_bleu_factory(self) -> None:
+    def test_bleu_factory_function(self) -> None:
+        """Test the bleu() factory function."""
         validator = bleu(min_score=0.6, variant=2)
         assert isinstance(validator, BleuValidator)
         assert validator.name == "bleu-2"
 
 
 class TestRougeValidator:
-    def test_rouge_passes_above_threshold(self) -> None:
+    """Tests for RougeValidator."""
+
+    def test_rouge_validator_passes_when_score_meets_threshold(self) -> None:
+        """Test that validator passes when ROUGE score meets threshold."""
         validator = RougeValidator(min_score=0.5, variant="l")
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("the cat sat on the mat", context)
@@ -72,7 +84,8 @@ class TestRougeValidator:
         assert result.actual == 1.0  # Identical text
         assert result.threshold == 0.5
 
-    def test_rouge_fails_below_threshold(self) -> None:
+    def test_rouge_validator_fails_when_score_below_threshold(self) -> None:
+        """Test that validator fails when ROUGE score is below threshold."""
         validator = RougeValidator(min_score=0.9, variant="l")
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("a dog ran through the park", context)
@@ -81,7 +94,8 @@ class TestRougeValidator:
         assert result.actual < 0.9
         assert "below minimum" in result.message
 
-    def test_rouge_variant_selection(self) -> None:
+    def test_rouge_validator_variant_selection(self) -> None:
+        """Test different ROUGE variants."""
         context = ValidationContext(reference="the quick brown fox jumps")
 
         for variant in ("1", "2", "l"):
@@ -89,29 +103,36 @@ class TestRougeValidator:
             result = validator.check("the quick brown fox", context)
             assert result.name == f"rouge-{variant}"
 
-    def test_rouge_requires_reference(self) -> None:
+    def test_rouge_validator_raises_on_missing_reference(self) -> None:
+        """Test that validator raises when reference is missing."""
         validator = RougeValidator(min_score=0.5)
         context = ValidationContext()
 
         with pytest.raises(ValidationError, match="requires reference text"):
             validator.check("some text", context)
 
-    def test_rouge_rejects_invalid_score(self) -> None:
+    def test_rouge_validator_raises_on_invalid_min_score(self) -> None:
+        """Test that invalid min_score raises error."""
         with pytest.raises(InvalidThresholdError, match=r"between 0\.0 and 1\.0"):
             RougeValidator(min_score=1.5)
 
-    def test_rouge_rejects_invalid_variant(self) -> None:
+    def test_rouge_validator_raises_on_invalid_variant(self) -> None:
+        """Test that invalid variant raises error."""
         with pytest.raises(InvalidThresholdError, match="variant must be"):
             RougeValidator(min_score=0.5, variant="3")  # type: ignore[arg-type]
 
-    def test_rouge_factory(self) -> None:
+    def test_rouge_factory_function(self) -> None:
+        """Test the rouge() factory function."""
         validator = rouge(min_score=0.6, variant="2")
         assert isinstance(validator, RougeValidator)
         assert validator.name == "rouge-2"
 
 
 class TestLexicalValidator:
-    def test_lexical_passes_jaccard(self) -> None:
+    """Tests for LexicalValidator."""
+
+    def test_lexical_validator_passes_on_jaccard(self) -> None:
+        """Test that validator passes when Jaccard similarity meets threshold."""
         validator = LexicalValidator(min_jaccard=0.5)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("the cat sat on the mat", context)
@@ -120,7 +141,8 @@ class TestLexicalValidator:
         assert result.name == "lexical"
         assert result.actual["jaccard"] == 1.0
 
-    def test_lexical_fails_jaccard(self) -> None:
+    def test_lexical_validator_fails_on_jaccard(self) -> None:
+        """Test that validator fails when Jaccard is below threshold."""
         validator = LexicalValidator(min_jaccard=0.9)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("a dog ran through the park", context)
@@ -129,7 +151,8 @@ class TestLexicalValidator:
         assert "Jaccard" in result.message
         assert "below minimum" in result.message
 
-    def test_lexical_passes_overlap(self) -> None:
+    def test_lexical_validator_passes_on_overlap(self) -> None:
+        """Test that validator passes when token overlap meets threshold."""
         validator = LexicalValidator(min_overlap=0.5)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("the cat sat on the mat", context)
@@ -137,7 +160,8 @@ class TestLexicalValidator:
         assert result.passed is True
         assert result.actual["token_overlap"] == 1.0
 
-    def test_lexical_fails_overlap(self) -> None:
+    def test_lexical_validator_fails_on_overlap(self) -> None:
+        """Test that validator fails when overlap is below threshold."""
         validator = LexicalValidator(min_overlap=0.9)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("a dog ran through", context)
@@ -145,7 +169,8 @@ class TestLexicalValidator:
         assert result.passed is False
         assert "overlap" in result.message
 
-    def test_lexical_both_thresholds(self) -> None:
+    def test_lexical_validator_with_both_thresholds(self) -> None:
+        """Test validator with both Jaccard and overlap thresholds."""
         validator = LexicalValidator(min_jaccard=0.3, min_overlap=0.5)
         context = ValidationContext(reference="the cat sat on the mat")
         result = validator.check("the cat sat", context)
@@ -154,26 +179,31 @@ class TestLexicalValidator:
         assert "min_jaccard" in result.threshold
         assert "min_overlap" in result.threshold
 
-    def test_lexical_needs_threshold(self) -> None:
+    def test_lexical_validator_raises_when_no_threshold(self) -> None:
+        """Test that validator raises when no threshold is provided."""
         with pytest.raises(InvalidThresholdError, match="At least one"):
             LexicalValidator()
 
-    def test_lexical_rejects_bad_jaccard(self) -> None:
+    def test_lexical_validator_raises_on_invalid_jaccard(self) -> None:
+        """Test that invalid Jaccard threshold raises error."""
         with pytest.raises(InvalidThresholdError, match="min_jaccard"):
             LexicalValidator(min_jaccard=1.5)
 
-    def test_lexical_rejects_bad_overlap(self) -> None:
+    def test_lexical_validator_raises_on_invalid_overlap(self) -> None:
+        """Test that invalid overlap threshold raises error."""
         with pytest.raises(InvalidThresholdError, match="min_overlap"):
             LexicalValidator(min_overlap=-0.1)
 
-    def test_lexical_requires_reference(self) -> None:
+    def test_lexical_validator_raises_on_missing_reference(self) -> None:
+        """Test that validator raises when reference is missing."""
         validator = LexicalValidator(min_jaccard=0.5)
         context = ValidationContext()
 
         with pytest.raises(ValidationError, match="requires reference text"):
             validator.check("some text", context)
 
-    def test_lexical_factory(self) -> None:
+    def test_lexical_factory_function(self) -> None:
+        """Test the lexical() factory function."""
         validator = lexical(min_jaccard=0.5, min_overlap=0.6)
         assert isinstance(validator, LexicalValidator)
         assert validator.name == "lexical"
@@ -181,11 +211,15 @@ class TestLexicalValidator:
 
 # SemanticValidator tests - conditionally run if sentence-transformers is installed
 class TestSemanticValidator:
+    """Tests for SemanticValidator."""
+
     @staticmethod
     def _skip_if_no_transformers() -> None:
+        """Skip test if sentence-transformers is not installed."""
         pytest.importorskip("sentence_transformers")
 
-    def test_semantic_passes_above_threshold(self) -> None:
+    def test_semantic_validator_passes_when_score_meets_threshold(self) -> None:
+        """Test that validator passes when semantic similarity meets threshold."""
         self._skip_if_no_transformers()
         from veritext.validators.metric import SemanticValidator
 
@@ -198,7 +232,8 @@ class TestSemanticValidator:
         assert result.actual >= 0.99  # Identical text
         assert result.threshold == 0.5
 
-    def test_semantic_fails_below_threshold(self) -> None:
+    def test_semantic_validator_fails_when_score_below_threshold(self) -> None:
+        """Test that validator fails when semantic similarity is below threshold."""
         self._skip_if_no_transformers()
         from veritext.validators.metric import SemanticValidator
 
@@ -213,7 +248,8 @@ class TestSemanticValidator:
         assert result.actual < 0.99
         assert "below minimum" in result.message
 
-    def test_semantic_requires_reference(self) -> None:
+    def test_semantic_validator_raises_on_missing_reference(self) -> None:
+        """Test that validator raises when reference is missing."""
         self._skip_if_no_transformers()
         from veritext.validators.metric import SemanticValidator
 
@@ -223,7 +259,9 @@ class TestSemanticValidator:
         with pytest.raises(ValidationError, match="requires reference text"):
             validator.check("some text", context)
 
-    def test_semantic_rejects_invalid_score(self) -> None:
+    def test_semantic_validator_raises_on_invalid_min_score(self) -> None:
+        """Test that invalid min_score raises error without loading model."""
+        # This test doesn't need sentence-transformers since validation happens first
         with pytest.raises(InvalidThresholdError, match=r"between 0\.0 and 1\.0"):
             from veritext.validators.metric import SemanticValidator
 
@@ -234,7 +272,8 @@ class TestSemanticValidator:
 
             SemanticValidator(min_score=-0.1)
 
-    def test_semantic_factory(self) -> None:
+    def test_semantic_factory_function(self) -> None:
+        """Test the semantic() factory function."""
         self._skip_if_no_transformers()
         from veritext.validators import semantic
         from veritext.validators.metric import SemanticValidator