The Architecture At a glance
Stage V has evolved into a multi-process setup on Linux CAN: the Gateway holds a Digital Twin (Brain, assembly actors, ROB, FSM, Observation Tee); a separate TUI Dashboard only observes; runs land on disk as JSONL. Control stays on vcan0; live telemetry uses UDS or Zenoh.
[ Emulator ] ────> (vcan0) <───────> Wiper Actuator
│ ^ │
│ │ │
v │ └ <───────> Headlamp Actuator
┌─ Gateway ──────────────────────┐
│ ┌─ Digital Twin ─────────────┐ │
│ │ Brain Actor │ │
│ │ Assembly actors │ │ ── (UDS / Zenoh) ──> [ Dashboard (TUI) ]
│ │ ROB · FSM │ │
│ │ Observation Tee │ │
│ └──────────────┬─────────────┘ │
└────────────────┼───────────────┘
│
v
[ Disk: JSONL Logs (replayable, schema-versioned)]
Quick snap

Motivation
Modern Software-Defined Vehicles require a delicate balance: handling unpredictable, high-frequency events while maintaining deterministic, thread-safe system states across simulated actuators and components. To explore how modern systems programming can solve this, I have built a working Digital Car Twin prototype from scratch. By leveraging Rust’s memory safety guarantees and an asynchronous, actor-based architecture driven by #ractor, this project moves from a static vehicle description to a fully reactive runtime environment capable of managing component state transitions — and of publishing that runtime as observation streams, which a separate Dashboard can watch live. Here is a retrospective on how that architecture came together, the critical design choices made along the way, and what it takes to plumbing a modern automotive software, from a software programmer's PoV.
### 📂 Dive Into the Journey & Source Code
This capping post summarizes the engineering sprint through Stage V. If you want to track how this system evolved piece-by-piece, use the links below:
- The Latest Iteration: Read Stage V to see the Gateway/Dashboard split, observation file tee, live UDS or Zenoh delivery, and the #ratatui observation console.
- The Predecessor Iterations:
Source for the current baseline: sdv_simulation_5 on GitHub (builds on sdv_simulation_4).
Core Architecture
To make this reactive model work without physical hardware, the architecture relies on a clean separation of concerns between a simulated environment, a protocol gateway, an isolated actor runtime, and an observation consumer:
-
The CAN Bus & Simulator Layer: A simulator injects operational events (RPM, ambient light, rain) into a virtual CAN bus. This bus remains the sole vehicle-side communication medium; simulated actuators receive their target instructions through it, and the Digital Twin reads from it. By Stage V the emulator also brackets a session with
PowerOn/PowerOffso the twin’s lifecycle on the wire is explicit. -
The Gateway Guard: To protect the core logic (of the Digital Twin) from raw I/O blocking, a specialized Gateway acts as the boundary. It handles the low-level mechanics of reading incoming frames from the CAN bus and serializing commands back onto it, translates CAN payloads into the Digital Twin's own vocabulary, and — from Stage V — owns the twin exclusively while teeing diagnostics and ledger rows to disk and optionally publishing them live.
-
The Actor Brain & State Ledger: At the heart sits the Digital Twin, implemented as a hierarchical actor system using #ractor. A parent actor manages dedicated child actors that implement individual component — called Assembly — behaviour (Headlamp and Wiper by Stage IV). This "brain" processes the stream of events forwarded by the Gateway, maintains the authoritative runtime state (
VehicleContext), is responsible for deterministic state management of the Twin, and maintains a precise ledger of exactly how every single event is handled. -
The Observation Path & Dashboard: Stage V makes observation a first-class product. The Gateway always writes
manifest.json,diagnostic.jsonl, andledger.jsonlunder./observations/<run-id>/, and may also stream the same schema (v3) envelopes over a Unix Domain Socket or Zenoh (peer mode) — exactly one live mode at a time, or--no-livefor headless capture. A separate #ratatui Dashboard process is observation-only: it never owns the twin; it formats facts at the edge into driver, engineer, and ledger-tail panes. The Gateway waits for that consumer to attach before installing the twin, so the live session starts with an observer already hooked up.
Solving the Causal Ordering Challenge at the Edge
While an actor-based architecture provides an elegant, single-threaded processing guarantee per mailbox, a major architectural challenge emerged when integrating the asynchronous emulator: Causal Inversion.
The Problem: Out-of-Order 'Tell-Backs'
The simulator injects events randomly and these events reach the core Digital Twin Brain Actor's mailbox in any order. The Brain Actor often issues asynchronous instructions (Tell) to multiple sub-assemblies (children Actors). However, response events (Tell-Back) from these assemblies can arrive back at the brain actor's mailbox out of order or interleaved.
If processed naively, overlapping event processing breaks the state machine's determinism. It introduces interleaving anomalies into the transition ledger, destroying our ability to perform an authentic, offline replay of the vehicle's history.
The Solution: Implementing a Reorder Buffer (ROB)
To enforce deterministic state transitions and ensure the ledger narrates the absolute truth of event causality, I implemented a Reorder Buffer data structure directly within the single-threaded actor loop (Stage IV; still the coordination core in Stage V).
-
Correlation & Tracking: Every outbound
Tellis tagged with a unique sequence ID. When interleavedTell-Backresponses arrive out of order, the FSM uses the buffer to pair them correctly. -
Enforcing Causal Order: The brain actor defers processing a later response if a causally prior event is still pending. The ROB holds these out-of-order responses in a parked state, releasing them to the FSM only when the missing sequence gaps are filled.
-
A True Ledger: Because the single-threaded actor only commits transitions to the ledger after the ROB restores the correct sequence, the resulting transition log is truthful about what the Digital Twin has gone through so far — and those same ledger rows are what the Dashboard tails live and what lands on disk for future replay.
From Coordination to Observation (Stage V)
Stage IV proved multi-assembly coordination. Stage V asks a different question: once the twin’s decisions are truthful, how do we publish them without coupling presentation to the Brain?
The answer is emit-facts / format-at-the-edge. The twin and Gateway publish rich diagnostic and ledger streams; the Dashboard’s panes filter and glyph what a driver or an engineer needs to see. CAN stays first for the vehicle bus; Zenoh in this milestone is an observation carrier, not a bus replacement. Five processes share the stage: Gateway, Dashboard, Emulator, Headlamp actuator, and Wiper actuator.
What This Milestone Achieves & The Horizon Ahead
The Baseline
What this effort through Stage V establishes is a robust, software-only capture of how a vehicle behaves from start to stop — plus a live and on-disk observation surface of that behaviour. By using only #rustlang (actors via #ractor, TUI via #ratatui), and a lean automation layer of #bash scripts to spin up executables and collect telemetry, the prototype validates that real-time digital twin tracking can be achieved deterministically without relying on heavy proprietary suites. The current runnable baseline is sdv_simulation_5.
The Roadmap
There is plenty of territory left to map. Moving forward, the natural progression deepens standards compliance and closes the loop from capture to offline reconstruction:
-
Ecosystem Middleware: Abstracting more of the custom I/O layer with Eclipse SDV technologies such as uProtocol (transport-agnostic service communication). Zenoh is already in use for live observation in peer mode; evolving it toward a proper router topology (and broader pub/sub scaling) remains ahead.
-
Deterministic Replay Tooling: Stage V already tees versioned observation files (
manifest+diagnostic.jsonl+ledger.jsonl) as the capture substrate. The next step is a dedicated offline tool (or Dashboard mode) that plays those runs back without a live Gateway and reconstructs the Digital Twin’s behaviour step-by-step. -
Advanced FSM & Safety Refinements: Transitioning bare-minimum diagnosis mechanics into structured safety verification functions rather than loose conditional expressions; replacing match-expression guard patterns with dedicated Entry/Exit lifecycle functions in the state transition matrix; completing twin shutdown / disband so quit waits for
Offbefore tearing down actors. -
Richer Live Surfaces: The #ratatui Dashboard is in place (driver / engineer / ledger panes). Ahead: visibility swatches, notice styling by diagnostic level, optional ASCII glyph fallbacks, and surfacing active ROB-turn counts from the Brain rather than ledger-hop stamps alone.