Run event model

VoidBox emits structured run events and guest telemetry during execution. OTLP traces and metrics are available when observable execution is enabled with the opentelemetry feature.

Identity fields

Run events share a common envelope, but fields are populated selectively depending on the event type.

Common envelope fields: ts_ms, level, event_type, message
Often present: run_id, environment_id, mode, seq
Event-specific fields: box_name, skill_id, skill_type, stream, stage_name, group_id, payload

Core event types

Events are grouped by lifecycle. Every event carries the common envelope above; the grouping reflects when each event fires, not a field in the payload.

Run lifecycle

Event	Description
`run.started`	Pipeline run has begun execution.
`run.spec.loaded`	Spec has been parsed and loaded into run state.
`run.finished`	Pipeline run completed successfully.
`run.failed`	Pipeline run failed with an error.
`run.cancelled`	Pipeline run was cancelled by the user.
`run.output_ready`	Service-mode run published its output report. Unlike task mode, a service run does not terminalize on this event.
`spec.parse_failed`	Spec failed to parse or validate before execution could start.

Environment and stages

Event	Description
`env.provisioned`	Guest environment has been provisioned (skills, config, mounts).
`stage.queued`	Stage is queued awaiting execution.
`stage.started`	Stage has started executing (attempt number in `payload`).
`stage.completed`	Stage succeeded; `message` includes duration in ms.
`stage.failed`	Stage failed; `message` includes the error.
`stage.skipped`	Stage was skipped; `message` includes the reason.

Box, skill, workflow, logs

Event	Description
`box.started`	A VoidBox has started execution within a stage.
`skill.mounted`	A skill has been written to the guest filesystem.
`workflow.planned`	A workflow planner has generated a pipeline plan.
`log.chunk`	A chunk of streaming output from the guest (stdout/stderr).
`log.closed`	The output stream for a box has closed.

Trace structure

VoidBox emits OpenTelemetry-compatible traces that capture the full execution hierarchy. The box name is encoded in the span name, not as an attribute.

pipeline:<pipeline_name>
  └─ stage:data_analyst
       └─ claude.exec
            ├─ claude.tool.Read
            ├─ claude.tool.Bash
            └─ attributes:
                gen_ai.usage.input_tokens
                gen_ai.usage.output_tokens
                gen_ai.request.model
                claude.total_cost_usd
                claude.tools_count
  └─ stage:quant_analyst
       └─ ...

Tool calls are child spans (claude.tool.<tool_name>) nested under a claude.exec span, not span events. Each stage span carries token, cost, and model attributes using the OpenTelemetry gen_ai.* semantic conventions. Fan-out stages create parallel child spans under the same pipeline parent.

Instrumentation

OTLP traces

Full distributed traces exported via OTLP gRPC. Pipeline, stage, and tool-call spans with rich attributes for token usage, cost, model, and timing.

Metrics

Token counts (input/output), cost in USD, execution duration, and VM lifecycle timing. Exported as OTLP metrics alongside traces.

Structured logs

All log output is prefixed with [vm:NAME] for easy filtering. Runtime JSONL output is parsed with the provider-specific observer: Claude-compatible stream-json for Claude-shaped providers, exec --json for Codex.

Guest telemetry

The guest-agent reads /proc/stat and /proc/meminfo periodically, sending TelemetryBatch messages over vsock. The host-side TelemetryAggregator ingests these and exports as OTLP metrics.

Configuration

Env var	Description
`VOIDBOX_OTLP_ENDPOINT`	OTLP gRPC endpoint (e.g. `http://localhost:4317`)
`OTEL_EXPORTER_OTLP_ENDPOINT`	Standard OpenTelemetry endpoint fallback
`VOIDBOX_SERVICE_NAME`	Service name for exported telemetry (default: `void-box`)
`OTEL_EXPORTER_OTLP_HEADERS`	Reliable way to pass OTLP auth headers today
`VOIDBOX_OTLP_HEADERS`	Parsed internally, but not yet wired into the exporter path

OpenTelemetry support is enabled at compile time:

cargo build --features opentelemetry

For a full OTLP setup walkthrough with Jaeger or Grafana, see the Observability Setup guide.

Guest telemetry pipeline

The guest-side telemetry pipeline works independently from the host tracing system:

The guest-agent periodically reads /proc/stat (CPU usage) and /proc/meminfo (memory usage).
Readings are batched into a TelemetryBatch message and sent to the host over the vsock channel.
The host-side TelemetryAggregator receives batches, computes deltas, and exports them as OTLP metrics.

This gives visibility into guest resource consumption without any instrumentation inside the workload itself.

Persistence providers

Daemon run and session state uses a provider abstraction, allowing different storage backends:

Disk (default)

File-based persistence. Run state and events are stored as JSON files on the local filesystem. No external dependencies.

SQLite / Valkey

sqlite and valkey are example adapter names today. They currently fall back to the disk provider rather than shipping separate production backends.