Work in Progress: Multi-LLM Dialectic Blackboard (MLDB)
Current Status
Complete working prototype operational with multiple LLM (API) providers:
- Anthropic
- Groq
- DeepSeek
- Mistral
- Ollama (local)
Tested against complex multi-stage cyber attack scenarios in a controlled environment.
What Is MLDB?
An operational implementation of Tim Bass’s 1999 Multisensor Data Fusion (MSDF) architecture, rebuilt for LLM-era reasoning.
This is a Rails 7.1 system where large language models act as Knowledge Sources (KSs) in a structured blackboard architecture.
Raw newline-delimited log streams flow through a disciplined fusion pipeline:
Raw Log
→ Observable
→ Normalizer (L1)
→ EvidenceWindow
→ Proposer (L2)
→ HypothesisGate (non-LLM)
→ Critic (L2)
→ Verifier (L3)
→ Correlator
→ Alert
This is not an agent swarm.
It is structured, ordered deliberation.
Architectural Position
Rejected: Opportunistic Blackboard Dispatch
The classical blackboard model allows any KS to fire when its preconditions are met.
In practice that introduces:
- Complex dispatch logic
- Expensive uncontrolled LLM invocation
- Opaque execution order
- Structural bias toward high-volume sensors
MLDB instead uses structured dialectic sequencing:
- Each KS runs in a defined order.
- No KS is starved.
- Cost and latency are bounded.
- Reasoning remains replayable.
The blackboard remains PostgreSQL.
Dispatch is disciplined job-chaining.
Key Design Elements
1. LLMs Replace L0 Data Refinement
Bass (1999) required rigid parsers for Level 0.
Here, the Normalizer LLM:
- Reads heterogeneous logs directly
- Produces structured JSON
- Extracts entities
- Generates human-readable summaries
No brittle regex farms.
2. EvidenceWindow (Structural Fairness)
The Proposer does not consume “most recent N events.”
Instead:
- A non-LLM
WindowBuilderJob - Time-bounded selection
- Per-sensor quotas
- Stratified sampling
- Jaccard novelty suppression
- SHA256 window hash for replay integrity
Each window persists:
event_ids[]sensor_mixselection_policywindow_hash
Hypotheses must cite specific event_ids.
Replay is deterministic.
3. HypothesisGate (Cost Control)
Before invoking Critic and Verifier:
A non-LLM gate evaluates:
evidence_countsensor_diversitynovelty_vs_recent_hypotheses- Proposer confidence (weak signal)
Failures become:
discarded_low_confidence
Persisted and auditable.
This eliminates near-duplicate hypotheses before additional LLM cost is incurred.
4. Dialectic Roles
| Role | Function |
|---|---|
| Normalizer | Interpret raw logs |
| Proposer | Generate evidence-cited hypotheses |
| Critic | Challenge hypotheses |
| Verifier | Assess credibility |
| Correlator | Link multi-stage patterns |
| Alert | Escalate confirmed cases |
Each role is configurable per provider and model.
Multi-Provider LLM Routing
Per-KS routing across providers with automatic fallback.
Features:
- Retry on provider failure
- Primary cooldown to avoid repeated timeouts
- Per-call audit trail (
LlmCall) - Token tracking
- Cost accounting (
cost_usd) - Cache hit tracking (Anthropic / DeepSeek)
Free-tier calls display $0.00 distinctly from “no data.”
State Discipline
Forward-only hypothesis lifecycle:
proposed → critiqued → corroborated → alerted
Or dismissed at any stage.
Transitions occur under PostgreSQL row locks.
No backward mutation.
Observability
- Per-KS latency (24h + all-time)
- Success rates
- Fallback usage
- Token in/out counts
- Cost today
- Provider attribution on every artifact
- CLI health monitor (
bin/blackboard status) - Pipeline idle/busy detection (Sidekiq depth)
This is an operational system, not just a conceptual model.
Current Status (2026-02)
Implemented
- Structured job-chaining pipeline
- EvidenceWindow with novelty suppression
- HypothesisGate (4 structural signals)
- Correlator linking for multi-stage chains
- Campaign (L3) view
- Cost tracking per call
- Multi-provider routing + fallback
- CLI service orchestration
- Replay integrity
- Timezone-correct UI
- Attack scenario test harness
- Sidekiq concurrency tuned (3 workers)
Next Engineering Steps
- Correlator deduplication by
event_id - Weighted per-sensor quotas
- WindowBuilder backfill logic
- Per-sensor backlog cap
- Convert Control Shell into L4 health monitor
- Real production ingestion from
www.unix.com
The core MSDF engine is stable.
Remaining work improves structural fairness and robustness.
Scope
Currently applied to cybersecurity.
Architecturally domain-agnostic:
- Financial anomaly detection
- Clinical monitoring
- Supply chain signals
- IoT telemetry
- Any newline-delimited signal stream
Only some UI labels are cyber-specific.
Closing
If anyone here is experimenting with:
- Multi-LLM systems
- Structured reasoning pipelines
- Blackboard architectures
- MSDF implementations
I’m interested in comparative discussion.
This project has moved beyond prototype (below) with a newly designed HLA and is functioning as a real multi-sensor fusion engine.