Test method library32 methods
Baseline and outcome test
Compare the proposed system with the current workflow using an agreed business outcome and a counter-metric.
Process fit and handoff test
Verify that the system fits queues, roles, service levels, exceptions, and handoffs.
Capacity latency and cost test
Measure throughput, peak-load behavior, latency, and unit economics.
Counter-metric and gaming test
Check whether optimizing the target metric creates undesirable behavior or hidden work.
Completeness and accuracy test
Measure missingness, errors, duplicates, label quality, and reconciliation to authoritative sources.
Representativeness and coverage test
Assess whether evaluation data covers relevant products, jurisdictions, groups, channels, and edge cases.
Leakage and provenance test
Detect target leakage and verify origin, rights, transformations, and permitted use.
Freshness and drift test
Measure changes between development, evaluation, and production data.
Discrimination and error test
Measure ranking or classification quality together with false positives and false negatives.
Calibration test
Confirm predicted probabilities or amounts correspond to observed outcomes.
Stability and sensitivity test
Challenge sensitivity to input perturbations, time, sampling, and plausible operating changes.
Threshold and policy test
Translate scores into decisions and test the business policy surrounding thresholds.
Group outcome test
Compare relevant outcome rates across legally and operationally appropriate groups.
Proxy and feature test
Assess whether inputs or representations act as problematic proxies or lack business necessity.
Error disparity test
Compare false positives, false negatives, calibration, and error severity across groups.
Adverse decision and appeal simulation
Verify notices, reasons, review rights, accessibility, and correction paths.
Factuality and grounding test
Measure whether material statements are supported by authoritative content.
Retrieval recall and citation fidelity test
Verify retrieval finds the needed source and citations support the generated statement.
Instruction and format test
Check that outputs follow required content, structure, tone, and prohibited-content rules.
Hallucination abstention and uncertainty test
Evaluate unsupported answers, appropriate refusal, and escalation under ambiguity.
Task success test
Measure complete goal achievement, not only intermediate response quality.
Tool and action correctness test
Verify tool choice, parameters, sequencing, records changed, and communication sent.
Authority and approval compliance test
Attempt actions above authority and verify approval gates cannot be bypassed.
Loop recovery and rollback test
Test timeouts, repeated actions, partial failure, recovery, and reversibility.
Prompt injection and adversarial test
Test direct and indirect manipulation through user content, retrieved documents, and tools.
Data exfiltration and access test
Verify least privilege, tenant separation, secrets protection, and resistance to extraction.
Dependency failure and fallback test
Test unavailable models, retrieval, tools, networks, and degraded upstream data.
Misuse and red-team test
Explore foreseeable abuse, unsafe objectives, and combinations not covered by normal testing.
Online quality and drift test
Sample live work and monitor task quality, data drift, and outcome drift.
Human override and escalation test
Evaluate disagreement, override quality, escalation rate, and automation bias.
Incident and counterfactual sampling test
Search for rare harms and compare actual decisions with safe alternatives.
Change regression and revalidation test
Confirm model, prompt, data, tool, vendor, or policy changes preserve approved behavior.