honest-test

Predicate Classification

Before generating test cases, honest-test classifies each predicate by AST analysis:

External predicates that cannot be analyzed emit a warning and skip generation unless test values are supplied in honest-test.yaml.

Set Enumeration Algorithm

FUNCTION enumerate_sets(vocabulary):
    set_types ← { name: list(members)
                  FOR (name, recog) IN vocabulary
                  IF recog IS a Set }
    RETURN cartesian_product(set_types.values())

For a vocabulary with format_name(5) × currency_code(150) × style_name(4): 3,000 test cases. All run. No sampling. Maybe slots add one case: Nothing.

Adversarial Input Generation

For every Set member, honest-test generates edit-distance-1 neighbors:

FUNCTION adversarial_neighbors(value):
    results ← []
    # Deletions: remove each character
    # Insertions: insert each alphanumeric at each position
    # Substitutions: replace each character with each alphanumeric
    # Case variations: lower, upper, title
    # Whitespace variations: leading space, trailing space, internal space
    RETURN deduplicate(results) - {value}

Every adversarial neighbor must produce a rejection. Any that are accepted expose vocabulary overlap or case-sensitivity bugs.

Purity Verification Algorithm

FUNCTION verify_purity(link, test_manifest):
    result_1 ← link(deep_copy(test_manifest))
    result_2 ← link(deep_copy(test_manifest))
    IF result_1 ≠ result_2:
        EMIT failure("non_deterministic", link.name)

Non-boundary links only. Boundary links are exempt.

Mutation Detection Algorithm

FUNCTION detect_mutation(link, test_manifest):
    snapshot_before ← deep_copy(test_manifest)
    result          ← link(test_manifest)
    IF snapshot_before ≠ test_manifest:
        diff ← diff(snapshot_before, test_manifest)
        EMIT failure("manifest_mutated", link.name, diff)

Any mutation of the input manifest is a honesty violation. There are no exceptions.

Chain Contract Testing

FUNCTION test_chain_contracts(chain, vocabulary, binding):
    FOR EACH adjacent pair (link_n, link_n1):
        FOR EACH test_manifest IN enumerate_test_cases(link_n.accepts):
            result ← link_n(test_manifest)
            IF "ok" IN result:
                result2 ← link_n1(result["ok"])
                IF "err" IN result2 AND result2["err"].category = "server":
                    EMIT failure("chain_contract", link_n.name, link_n1.name)

Client faults from link N+1 are not contract failures — they indicate the upstream link produced a value the downstream link correctly rejected. Only server faults indicate a broken interface.

State Machine Testing

# Valid transitions
FOR EACH (state, event) → next_state IN machine.transitions:
    result ← transition(machine, state, event)
    ASSERT result = ok({ state: next_state })

# Invalid transitions (all undeclared combinations)
FOR EACH state IN machine.states:
    FOR EACH event IN machine.events:
        IF (state, event) NOT IN machine.transitions:
            result ← transition(machine, state, event)
            ASSERT result = err({ code: "no_transition" })

# Adversarial inputs
FOR EACH state IN machine.states:
    FOR EACH adversarial IN adversarial_neighbors(state):
        result ← transition(machine, adversarial, first_valid_event)
        ASSERT result = err({ code: "invalid_state" })

honest-test.yaml

Programmer-supplied test values for external predicates:

predicates:
  customer_id:
    valid:   ["CUST-00001", "CUST-99999"]
    invalid: ["CUST-0", "cust-00001"]
    strategy: supplied_only

  order_amount:
    strategy: fibonacci
    limit: 1_000_000_000
    negative: false

coverage:
  minimum_vocabulary: 100   # fail if any Set member untested
  minimum_chain: 80         # warn if fault paths < 80%
  minimum_honesty: 100      # fail if any link fails honesty test

Coverage Model

Exhaustive Set enumeration drives vocabulary coverage to 100% automatically. Boundary links are reported separately, not as coverage failures.

Conformance Requirements