Research Roadmap

Aligned with the proposed manuscript and prospective evaluation protocol.

Phase 1 — Research artifact (complete / in progress)

  • Publish GitHub repository with verified LaTeX manuscript source
  • Provide simulator, schemas, synthetic data, tests, and examples
  • Separate verified evidence from vendor claims and forecasts
  • Create a numbered release and archive with Zenodo

Phase 2 — Synthetic evaluation scaffold

  • Generate 1,000+ synthetic scenarios
  • Compare ASBOS against rule-based and monolithic-agent baselines
  • Measure resolution rate, invalid assignments, escalation rate, event count, and audit completeness
  • Note: These are simulator results supporting the evaluation protocol, not field pilot evidence

Phase 3 — Prospective pilot protocol

Suggested 90-day pilot: two baseline weeks, four staged deployment phases, and a final observation period.

Initial cohort may include 20–100 guards across 5–15 sites; final sample based on operational feasibility and statistical power.

Primary outcomes

  • Administrative minutes per scheduled guard-hour
  • Median and 90th percentile time to fill a coverage gap
  • Percentage of scheduled shifts filled before start
  • Payroll exception rate per 1,000 worked hours
  • Report delivery compliance
  • Credential-related assignment blocks
  • Human escalation rate
  • False-positive exception rate

Secondary outcomes

  • Guard acceptance
  • Supervisor workload
  • Client satisfaction
  • System availability

Study design should preserve raw event logs, versioned policies, agent prompts, model/tool versions, human overrides, and outcome definitions. A stepped-wedge or matched-site design is preferable to a simple before-after comparison.

Phase 4 — Paper submission and field evidence

  • Publish preprint of manuscript
  • Submit to agent-systems, HCI, or applied AI venues
  • Report pilot results with confidence intervals
  • Distinguish statistical significance from operational significance

Phase 5 — Economic transparency

  • Report gross benefit, implementation cost, recurring cost, net benefit, benefit-cost ratio, and uncertainty separately
  • Do not present sensitivity-model outputs as validated ROI