Skip to content

caliper.metrics.v1

Service id caliper.metrics.v1 — TensorBoard vocabulary with ImPlot immediacy (PLATFORM.md §7.6). This page embeds the header verbatim; the docs build fails if the file moves.

#pragma once
/* caliper.metrics.v1 — TensorBoard vocabulary (experiment/run/tag/step),
 * ImPlot immediacy (PLATFORM.md §7.6). IMMUTABLE once published. Callable
 * from applet job threads; the host serializes internally. image() accepts
 * CPU-resident HWC u8 tensors in v1 (GPU-resident paths arrive with the
 * tensor bridge). */
#include <stdint.h>
#include <caliper/tensor.h>

#define CALIPER_METRICS_V1 "caliper.metrics.v1"

#ifdef __cplusplus
extern "C" {
#endif

typedef struct CaliperMetricsV1 {
    uint32_t struct_size;
    uint64_t (*begin_run)(const char* experiment, const char* run_name); /* 0 = error */
    void     (*end_run)(uint64_t run);
    void     (*scalar)(uint64_t run, const char* tag, int64_t step, double value);
    void     (*histogram)(uint64_t run, const char* tag, int64_t step,
                          const float* values, int64_t count);
    void     (*image)(uint64_t run, const char* tag, int64_t step,
                      const CaliperTensor* hwc_u8);
    void     (*hparams_json)(uint64_t run, const char* json_utf8);
} CaliperMetricsV1;

#ifdef __cplusplus
}
#endif

Semantics

The vocabulary is TensorBoard's, deliberately: experiment / run / tag / step.

  • begin_run(experiment, run_name) opens a run and returns its id. 0 means error — check for it and skip streaming (a 0 id is never a valid run). Every scalar, histogram, image, and hyperparameter you log is scoped to that id.
  • end_run(run) closes it. Call it on every exit path of your training job — completion and cancellation — so a run never dangles as "still running". Partial curves logged before the end are preserved.
  • scalar(run, tag, step, value) appends one point. The tag is the series name ("train/loss", "test/accuracy"); the / groups tags into panes in the dashboard. The step is the x-axis: a global batch index for per-batch loss, and the same global step for accuracy sampled every N batches (per-epoch cadence hides the learning transient on fast-converging datasets) — you choose the axis by choosing the step. Sharing the step domain lets loss and accuracy line up. The store keeps points ordered by step and queries them back ordered (the §16 contract: 10k scalars written and read back in order).
  • hparams_json(run, json_utf8) attaches a flat JSON blob of hyperparameters to the run ({"lr":0.001,"batch":256,...}) so runs are comparable.
  • histogram / image log a distribution or a picture at a step.

Thread-callability

Every entry point is callable from an applet job thread — which is where training lives (never the frame thread). The host serializes writes internally (a mutex over one DuckDB connection in v1), so concurrent jobs are safe. The host also destroys the metrics store after it joins job threads, so a scalar logged in the last instant before a cancel lands cannot fault.

v1 image limitation

image() accepts CPU-resident, contiguous, HWC u8 tensors only. The host gate enforces this: a tensor that is non-contiguous, not on the CPU, or not u8-HWC is logged and dropped, never misinterpreted. GPU-resident image paths (no CPU staging) arrive with caliper.tensor_bridge.v1 in Phase 2C.

The payoff

Metrics is an optional service: probe it, and stream only when present. Every applet that logs a scalar this way inherits the Runs dashboard for free — run list, per-tag plots, EMA smoothing — with no dashboard code of its own. See MLScope for the exemplar: MNIST training that streams train/loss per batch and test/accuracy every 50 batches (from a step-0 baseline) to this service.