Files
g3/prompts/schemas/rulespec.schema.md
Dhanji R. Prasanna edbae60ff3 Add rulespec extensions: new predicate rules, when conditions, null handling, solon agent
Features:
- New predicate rules: NotContains, AnyOf, NoneOf
- Conditional predicates via when clauses (WhenCondition/CompiledWhenCondition)
- Null handling: YAML null treated as absent for exists/not_exists
- Solon agent for rulespec authoring (agents/solon.md)
- Rulespec schema documentation (prompts/schemas/rulespec.schema.md)

Bugfix:
- Fixed when condition evaluation in datalog path: catch-all branch did
  naive string contains instead of delegating to evaluate_predicate_datalog().
  Rules like matches (regex) were silently ignored, causing vacuous pass
  and letting violations through. Now delegates to evaluate_predicate_datalog()
  which handles all 12 rule types correctly.

Tests: 34 new tests covering all new rules, null handling, when conditions,
and the when+matches bugfix (butler rulespec pattern).
2026-02-07 16:38:27 +11:00

403 lines
10 KiB
Markdown

# Rulespec YAML Schema
> Canonical reference for `analysis/rulespec.yaml` — the machine-readable invariant specification.
## Overview
A rulespec defines **claims** (selectors into the action envelope) and **predicates** (rules that evaluate those claims). When an agent completes work, it writes an **action envelope** via `write_envelope`, and the rulespec is evaluated against it using datalog verification.
## File Location
```
analysis/rulespec.yaml # checked into the repo
```
## Top-Level Structure
```yaml
claims:
- name: <string> # unique identifier for this claim
selector: <string> # path into the action envelope
predicates:
- claim: <string> # references a claim by name
rule: <rule_type> # one of the 12 rule types below
value: <any> # required for most rules (see table)
source: <source> # task_prompt | memory
notes: <string> # optional human-readable explanation
when: # optional conditional trigger
claim: <string> # references a claim by name
rule: <rule_type> # condition rule
value: <any> # condition value (if needed)
```
## Claims
A claim is a named selector that extracts values from the action envelope.
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | ✅ | Unique identifier (referenced by predicates) |
| `selector` | string | ✅ | Dot-notation path into the envelope |
### Selector Syntax
| Syntax | Meaning | Example |
|--------|---------|--------|
| `foo.bar` | Nested field access | `csv_importer.capabilities` |
| `foo[0]` | Array index (0-based) | `tests[0].name` |
| `foo[*]` | All array elements (wildcard) | `items[*].id` |
| `foo.bar.baz` | Deep nesting | `api.endpoints.count` |
**Important**: Selectors operate on the envelope's `facts` content directly. Do NOT include a `facts.` prefix in selectors — the system handles this automatically.
```yaml
# ✅ Correct
claims:
- name: caps
selector: csv_importer.capabilities
# ❌ Wrong — don't prefix with "facts."
claims:
- name: caps
selector: facts.csv_importer.capabilities
```
## Predicate Rules
### Rule Types Reference
| Rule | Value Required | Value Type | Description |
|------|---------------|------------|-------------|
| `exists` | ❌ | — | Value exists and is not null |
| `not_exists` | ❌ | — | Value is null or missing |
| `equals` | ✅ | any | Value equals exactly |
| `contains` | ✅ | any | Array contains element, or string contains substring |
| `not_contains` | ✅ | any | Negation of `contains` |
| `any_of` | ✅ | array | Value is one of the specified set |
| `none_of` | ✅ | array | Value is none of the specified set |
| `greater_than` | ✅ | number | Numeric comparison |
| `less_than` | ✅ | number | Numeric comparison |
| `min_length` | ✅ | number | Array has at least N elements |
| `max_length` | ✅ | number | Array has at most N elements |
| `matches` | ✅ | string | Value matches a regex pattern |
### Existence Rules
```yaml
# Value must exist (not null, not missing)
predicates:
- claim: feature_file
rule: exists
source: task_prompt
# Value must NOT exist (null or missing)
predicates:
- claim: breaking_changes
rule: not_exists
source: task_prompt
notes: "No breaking changes allowed"
```
### Equality
```yaml
predicates:
- claim: api_breaking
rule: equals
value: false
source: task_prompt
```
### Containment
```yaml
# Array contains an element
predicates:
- claim: capabilities
rule: contains
value: "handle_csv"
source: task_prompt
# Array must NOT contain an element
predicates:
- claim: capabilities
rule: not_contains
value: "deprecated_feature"
source: task_prompt
```
### Set Membership
```yaml
# Value must be one of these
predicates:
- claim: output_format
rule: any_of
value: [json, yaml, toml]
source: task_prompt
# Value must NOT be any of these
predicates:
- claim: output_format
rule: none_of
value: [xml, csv]
source: task_prompt
```
### Numeric Comparisons
```yaml
predicates:
- claim: test_count
rule: greater_than
value: 0
source: task_prompt
notes: "Must have at least one test"
- claim: error_rate
rule: less_than
value: 5
source: memory
```
### Array Length
```yaml
predicates:
- claim: capabilities
rule: min_length
value: 2
source: task_prompt
- claim: dependencies
rule: max_length
value: 10
source: memory
notes: "Keep dependency count manageable"
```
### Regex Matching
```yaml
predicates:
- claim: file_path
rule: matches
value: "^src/.*\\.rs$"
source: task_prompt
notes: "File must be a Rust source file in src/"
```
## Conditional Predicates (`when`)
Predicates can have an optional `when` condition. If the condition is **not met**, the predicate is **skipped** (vacuous pass) — it does not fail.
This is useful for rules that only apply in certain contexts.
### When Condition Structure
```yaml
when:
claim: <string> # references a defined claim
rule: <rule_type> # any predicate rule type
value: <any> # optional, depends on rule
```
### Examples
```yaml
# Only enforce endpoint count when there are breaking changes
predicates:
- claim: api_endpoints
rule: min_length
value: 3
source: task_prompt
when:
claim: is_breaking
rule: equals
value: true
notes: "Breaking changes must document all endpoints"
# Only check test coverage when tests exist
predicates:
- claim: coverage_percent
rule: greater_than
value: 80
source: memory
when:
claim: has_tests
rule: exists
# Only enforce format when feature is present
predicates:
- claim: output_format
rule: any_of
value: [json, yaml]
source: task_prompt
when:
claim: has_output
rule: exists
```
### When with Regex Matching
```yaml
# Only require reply_to_message_id when subject starts with "Re: "
predicates:
- claim: reply_to_id
rule: exists
source: task_prompt
when:
claim: subject_line
rule: matches
value: "^Re: "
notes: Reply emails must have reply_to_message_id set
```
## Null Handling
Null values in the action envelope have specific semantics:
- **`null` is treated as absent** — `exists` returns false, `not_exists` returns true
- This applies to both the invariants evaluator and the datalog compiler
- A fact with value `null` produces **no datalog facts** (it is skipped entirely)
### Common Pattern: Asserting Absence
```yaml
# In the envelope:
facts:
breaking_changes: null # explicitly absent
# In the rulespec:
claims:
- name: breaking
selector: breaking_changes
predicates:
- claim: breaking
rule: not_exists
source: task_prompt
notes: "No breaking changes" # ✅ This passes
```
### Edge Cases
| Envelope Value | `exists` | `not_exists` | `contains "x"` | `equals "y"` |
|---------------|----------|-------------|----------------|-------------|
| `null` | ❌ fail | ✅ pass | ❌ fail | ❌ fail |
| missing key | ❌ fail | ✅ pass | ❌ fail | ❌ fail |
| `""` (empty string) | ✅ pass | ❌ fail | ❌ fail | ❌ fail |
| `[]` (empty array) | ✅ pass | ❌ fail | ❌ fail | ❌ fail |
| `0` | ✅ pass | ❌ fail | ❌ fail | depends |
## Action Envelope Format
The action envelope is written via the `write_envelope` tool. It must have a top-level `facts:` key.
```yaml
facts:
feature_name:
capabilities: [cap_a, cap_b]
file: "src/feature.rs"
tests: ["test_a", "test_b"]
api_changes:
breaking: false
breaking_changes: null # Use null to assert absence
```
### Rules for Envelope Facts
1. **Must have `facts:` top-level key** — without it, the envelope is empty
2. **Use file paths as evidence**`"src/foo.rs"`, `"src/foo.rs:42"`
3. **Use `null` for explicit absence** — triggers `not_exists` predicates
4. **Arrays for lists** — capabilities, tests, endpoints
5. **Nested objects for grouping**`feature.capabilities`, `feature.file`
## Complete Example
```yaml
# analysis/rulespec.yaml
claims:
- name: caps
selector: csv_importer.capabilities
- name: file
selector: csv_importer.file
- name: tests
selector: csv_importer.tests
- name: breaking
selector: api_changes.breaking
- name: no_breaking
selector: breaking_changes
predicates:
# Must have capabilities
- claim: caps
rule: exists
source: task_prompt
# Must include handle_csv
- claim: caps
rule: contains
value: "handle_csv"
source: task_prompt
# Must NOT include deprecated features
- claim: caps
rule: not_contains
value: "legacy_parser"
source: memory
# At least 2 capabilities
- claim: caps
rule: min_length
value: 2
source: task_prompt
# File must be a Rust source
- claim: file
rule: matches
value: "^src/.*\\.rs$"
source: task_prompt
# Must have tests
- claim: tests
rule: min_length
value: 1
source: task_prompt
# No breaking changes
- claim: no_breaking
rule: not_exists
source: task_prompt
# If breaking, must document it
- claim: caps
rule: contains
value: "migration_guide"
source: task_prompt
when:
claim: breaking
rule: equals
value: true
notes: "Breaking changes require a migration guide capability"
```
## Verification Pipeline
1. Agent calls `write_envelope` with facts YAML
2. System writes `envelope.yaml` to session directory
3. System reads `analysis/rulespec.yaml` from working directory
4. Rulespec is compiled into datalog relations
5. Facts are extracted from envelope using claim selectors
6. Datalog rules are executed to fixed point
7. Results are written to `rulespec.compiled.dl` and `datalog_evaluation.txt`
8. Summary is returned to the agent
9. On plan completion, `plan_verify()` checks that the envelope exists
## Source Types
| Source | Meaning |
|--------|---------|
| `task_prompt` | Invariant derived from the user's task description |
| `memory` | Invariant derived from workspace memory (AGENTS.md, memory.md) |