Multi-LLM Dialectic Blackboard (MLDB)

Work in Progress: Multi-LLM Dialectic Blackboard (MLDB)

Current Status

Complete working prototype operational with multiple LLM (API) providers:

  • Anthropic
  • Groq
  • DeepSeek
  • Mistral
  • Ollama (local)

Tested against complex multi-stage cyber attack scenarios in a controlled environment.


What Is MLDB?

An operational implementation of Tim Bass’s 1999 Multisensor Data Fusion (MSDF) architecture, rebuilt for LLM-era reasoning.

This is a Rails 7.1 system where large language models act as Knowledge Sources (KSs) in a structured blackboard architecture.

Raw newline-delimited log streams flow through a disciplined fusion pipeline:

Raw Log
  → Observable
    → Normalizer (L1)
      → EvidenceWindow
        → Proposer (L2)
          → HypothesisGate (non-LLM)
            → Critic (L2)
              → Verifier (L3)
                → Correlator
                  → Alert

This is not an agent swarm.
It is structured, ordered deliberation.


Architectural Position

Rejected: Opportunistic Blackboard Dispatch

The classical blackboard model allows any KS to fire when its preconditions are met.

In practice that introduces:

  • Complex dispatch logic
  • Expensive uncontrolled LLM invocation
  • Opaque execution order
  • Structural bias toward high-volume sensors

MLDB instead uses structured dialectic sequencing:

  • Each KS runs in a defined order.
  • No KS is starved.
  • Cost and latency are bounded.
  • Reasoning remains replayable.

The blackboard remains PostgreSQL.
Dispatch is disciplined job-chaining.


Key Design Elements

1. LLMs Replace L0 Data Refinement

Bass (1999) required rigid parsers for Level 0.

Here, the Normalizer LLM:

  • Reads heterogeneous logs directly
  • Produces structured JSON
  • Extracts entities
  • Generates human-readable summaries

No brittle regex farms.


2. EvidenceWindow (Structural Fairness)

The Proposer does not consume “most recent N events.”

Instead:

  • A non-LLM WindowBuilderJob
  • Time-bounded selection
  • Per-sensor quotas
  • Stratified sampling
  • Jaccard novelty suppression
  • SHA256 window hash for replay integrity

Each window persists:

  • event_ids[]
  • sensor_mix
  • selection_policy
  • window_hash

Hypotheses must cite specific event_ids.

Replay is deterministic.


3. HypothesisGate (Cost Control)

Before invoking Critic and Verifier:

A non-LLM gate evaluates:

  • evidence_count
  • sensor_diversity
  • novelty_vs_recent_hypotheses
  • Proposer confidence (weak signal)

Failures become:

discarded_low_confidence

Persisted and auditable.

This eliminates near-duplicate hypotheses before additional LLM cost is incurred.


4. Dialectic Roles

Role Function
Normalizer Interpret raw logs
Proposer Generate evidence-cited hypotheses
Critic Challenge hypotheses
Verifier Assess credibility
Correlator Link multi-stage patterns
Alert Escalate confirmed cases

Each role is configurable per provider and model.


Multi-Provider LLM Routing

Per-KS routing across providers with automatic fallback.

Features:

  • Retry on provider failure
  • Primary cooldown to avoid repeated timeouts
  • Per-call audit trail (LlmCall)
  • Token tracking
  • Cost accounting (cost_usd)
  • Cache hit tracking (Anthropic / DeepSeek)

Free-tier calls display $0.00 distinctly from “no data.”


State Discipline

Forward-only hypothesis lifecycle:

proposed → critiqued → corroborated → alerted

Or dismissed at any stage.

Transitions occur under PostgreSQL row locks.
No backward mutation.


Observability

  • Per-KS latency (24h + all-time)
  • Success rates
  • Fallback usage
  • Token in/out counts
  • Cost today
  • Provider attribution on every artifact
  • CLI health monitor (bin/blackboard status)
  • Pipeline idle/busy detection (Sidekiq depth)

This is an operational system, not just a conceptual model.


Current Status (2026-02)

Implemented

  • Structured job-chaining pipeline
  • EvidenceWindow with novelty suppression
  • HypothesisGate (4 structural signals)
  • Correlator linking for multi-stage chains
  • Campaign (L3) view
  • Cost tracking per call
  • Multi-provider routing + fallback
  • CLI service orchestration
  • Replay integrity
  • Timezone-correct UI
  • Attack scenario test harness
  • Sidekiq concurrency tuned (3 workers)

Next Engineering Steps

  1. Correlator deduplication by event_id
  2. Weighted per-sensor quotas
  3. WindowBuilder backfill logic
  4. Per-sensor backlog cap
  5. Convert Control Shell into L4 health monitor
  6. Real production ingestion from www.unix.com

The core MSDF engine is stable.
Remaining work improves structural fairness and robustness.


Scope

Currently applied to cybersecurity.

Architecturally domain-agnostic:

  • Financial anomaly detection
  • Clinical monitoring
  • Supply chain signals
  • IoT telemetry
  • Any newline-delimited signal stream

Only some UI labels are cyber-specific.


Closing

If anyone here is experimenting with:

  • Multi-LLM systems
  • Structured reasoning pipelines
  • Blackboard architectures
  • MSDF implementations

I’m interested in comparative discussion.

This project has moved beyond prototype (below) with a newly designed HLA and is functioning as a real multi-sensor fusion engine.

Early Summary with Screenshots