Skip to content

Your first ML applet

The ladder from an empty window to a live machine-learning demonstration, one capability per stage. We build SineScope: a tiny MLP that learns y = sin(x) in front of you — live loss curve, the model's prediction bending toward the target in real time, and its weight matrix rendered as a heatmap. Synthetic data keeps it self-contained (no downloads); the last two stages show where real datasets and the remaining services attach.

SineScope is built in this repo — examples/sine_scope/, the SineScope card in your launcher. Every code block below is embedded verbatim from that source file; the docs build fails if they drift. Run the finished thing first if you like:

cmake --build build --target sine_scope
CALIPER_AUTOLAUNCH=dev.example.sine-scope ./build/caliper

Prereqs: Development basics (the mental model), Your first applet (the hello walkthrough). The finished staircase — same patterns, full scale — is applets/embed_scope/, with the cookbook as its field guide.

Stage 0 — the build and the manifest

The manifest requires what training needs and marks visualization optional, so the applet still runs without the bridge:

examples/sine_scope/sine_scope.caliper.toml
[applet]
id      = "dev.example.sine-scope"
name    = "SineScope"
version = "0.1.0"
summary = "Tutorial applet: a tiny MLP learns sin(x) live — loss curve, prediction vs target, first-layer weights as a heatmap."
tag     = "Demo"

[compat]
abi_epoch = 2
min_host  = "0.6.0"

[services]
required = ["caliper.ui.v1", "caliper.log.v1",
            "caliper.jobs.v1", "caliper.device.v1"]
optional = ["caliper.tensor_bridge.v1", "caliper.metrics.v1"]

The CMake file is hello's plus the torch lines — this is the entire ML build delta:

examples/sine_scope/CMakeLists.txt
add_library(sine_scope SHARED sine_scope.cpp)
target_link_libraries(sine_scope PRIVATE
    caliper::sdk caliper::ui_stack
    "${TORCH_LIBRARIES}")                  # vendored third_party/libtorch
target_compile_definitions(sine_scope PRIVATE CALIPER_APPLET_EXPORT)
set_target_properties(sine_scope PROPERTIES
    LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/applets"
    RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/applets"   # .dll on Windows
    CXX_STANDARD 20 CXX_STANDARD_REQUIRED ON)
if(APPLE)   # let the dylib find libtorch at runtime
    set_target_properties(sine_scope PROPERTIES
        BUILD_RPATH "${CMAKE_SOURCE_DIR}/third_party/libtorch/lib")
endif()
add_custom_command(TARGET sine_scope POST_BUILD
    COMMAND ${CMAKE_COMMAND} -E copy_if_different
        ${CMAKE_CURRENT_SOURCE_DIR}/sine_scope.caliper.toml
        ${CMAKE_BINARY_DIR}/applets/sine_scope.caliper.toml)

Stage 1 — the model and the state spine

Two layers are enough to bend a line into a sine:

// Two layers are enough to bend a line into a sine.
struct SineNetImpl : torch::nn::Module {
    torch::nn::Linear fc1{nullptr}, fc2{nullptr};
    SineNetImpl() {
        fc1 = register_module("fc1", torch::nn::Linear(1, 32));
        fc2 = register_module("fc2", torch::nn::Linear(32, 1));
    }
    torch::Tensor forward(torch::Tensor x) {
        return fc2->forward(torch::tanh(fc1->forward(x)));
    }
};
TORCH_MODULE(SineNet);

Every live applet shares the same skeleton state — service wrappers, one mutex, published copies, a generation counter (cookbook §1):

// The spine every live applet shares: service wrappers, one mutex,
// published copies, a generation counter (cookbook §1).
struct SineState {
    caliper::Host*   host = nullptr;
    caliper::Jobs    jobs;        // required (manifest-enforced)
    caliper::Bridge  bridge;      // optional — falsy-inert when absent

    uint64_t job_id = 0;

    // -- cross-thread, under mtx --
    std::mutex mtx;
    std::vector<float> loss_hist;
    std::vector<float> pred_ys;   // model(x) on the fixed 256-point grid
    torch::Tensor      disp_w;    // weight display tensor (handle swap)
    float              w_max = 1e-6f;
    uint64_t           gen = 0;   // bumped per publish (0 = none yet)

    // -- frame-thread only --
    uint64_t seen_gen = 0, tex_gen = 0;
    CaliperTextureId w_tex = 0;
    bool follow = true;
};

on_init probes services. The manifest already guaranteed the required ones exist — no null checks needed for jobs. The optional bridge is falsy-inert: callable unconditionally, no-ops when absent — but good demos show the degradation (Stage 4):

    bool on_init(caliper::Host& host) override {
        auto* st = s_.get();
        st->host   = &host;
        st->jobs   = caliper::Jobs(host);     // required — manifest enforced
        st->bridge = caliper::Bridge(host);   // optional — may be falsy
        host.log_info("sine_scope: on_init");
        return true;
    }

Stage 2 — background compute: the training job

Never compute on the frame thread. A job is a plain function the host runs on a worker thread — note the per-step cancel check (the ≤100 ms contract) and the progress reports that light up the host's jobs tray:

// The training job: a plain function the host runs on a worker thread.
void train_job(void* user, const CaliperJobControl* ctl) {
    auto* st = static_cast<SineState*>(user);

    // device.v1 -> torch device: METAL means MPS on Apple, CUDA on Windows/Linux.
    auto d = caliper::Device::query(*st->host);
    torch::Device dev =
        (d.kind == CALIPER_DEV_CUDA && torch::cuda::is_available())
            ? torch::Device(torch::kCUDA)
        : (d.kind == CALIPER_DEV_METAL) ? torch::Device(torch::kMPS)
                                        : torch::Device(torch::kCPU);

    // Synthetic dataset: 256 points of y = sin(x) on [-pi, pi].
    auto X = torch::linspace(-M_PI, M_PI, 256,
                             torch::TensorOptions().device(dev))
                 .unsqueeze(1);
    auto Y = torch::sin(X);

    SineNet model;
    model->to(dev);
    torch::optim::Adam opt(model->parameters(),
                           torch::optim::AdamOptions(1e-2));

    for (int step = 0; step < 2000; step++) {
        if (ctl->cancelled(ctl)) return;             // <=100 ms contract
        opt.zero_grad();
        auto loss = torch::mse_loss(model->forward(X), Y);
        loss.backward();
        opt.step();
        publish(st, model, X, loss.item<float>());
        ctl->progress(ctl, step / 2000.f, "fitting sin(x)");
    }
    ctl->progress(ctl, 1.f, "done — the line is a sine");
}

The frame side submits on click, shows progress, offers cancel:

        if (!running) {
            if (ImGui::Button("Train"))
                st->job_id = st->jobs.submit("sine_scope: fit",
                                             &train_job, st);
        } else {
            if (ImGui::Button("Cancel")) st->jobs.request_cancel(st->job_id);
            ImGui::SameLine();
            ImGui::ProgressBar(st->jobs.progress_of(st->job_id), {-1, 0});
        }

Stage 3 — publish and plot: watching it learn

The worker publishes owned copies (plot data) and a tensor handle (the weight display) under the mutex, bumping the generation:

// WORKER side: compute outside the lock, swap inside it, bump the gen.
void publish(SineState* st, SineNet& model, const torch::Tensor& X,
             float loss) {
    torch::NoGradGuard ng;
    auto pred = model->forward(X).to(torch::kCPU).contiguous();   // (256,1)
    // First-layer weights (32,1) -> 8x4 grid -> x16 hard blocks (cookbook
    // §4), staged to CPU for tutorial simplicity — the exemplar shows the
    // zero-copy device pull (cookbook §3).
    auto w = model->fc1->weight.detach()
                 .reshape({8, 4})
                 .repeat_interleave(16, 0).repeat_interleave(16, 1)
                 .to(torch::kCPU).contiguous();                   // (128,64)
    const float wmax = w.abs().max().item<float>();

    std::lock_guard<std::mutex> lk(st->mtx);
    st->loss_hist.push_back(loss);
    st->pred_ys.assign(pred.data_ptr<float>(),
                       pred.data_ptr<float>() + 256);
    st->disp_w = w;                       // handle swap — no data copy
    st->w_max  = std::max(wmax, 1e-6f);
    st->gen++;
}

The frame consumes when the generation moves, then draws. The prediction curve bending toward the target is the "it's alive" moment — and it's just two PlotLines, plus the follow-toggle idiom on the loss curve (cookbook §6):

        // These plots are for *viewing*, not editing: ImPlot is interactive by
        // default (drag-pan, scroll-zoom, box-select, right-click menu), so
        // lock every input off. Read-only plots always want these four flags.
        constexpr ImPlotFlags kLockedPlot =
            ImPlotFlags_NoInputs | ImPlotFlags_NoMenus |
            ImPlotFlags_NoBoxSelect | ImPlotFlags_NoMouseText;

        // The fixed grid + target are frame-side constants.
        static std::vector<float> xs, target;
        if (xs.empty()) {
            xs.resize(256); target.resize(256);
            for (int i = 0; i < 256; i++) {
                xs[i] = -3.14159265f + i / 255.0f * 6.2831853f;
                target[i] = std::sin(xs[i]);
            }
        }
        if (ImPlot::BeginPlot("fit", {-1, 240}, kLockedPlot)) {
            ImPlot::SetupAxes("x", "y", 0, 0);
            ImPlot::PlotLine("target sin(x)", xs.data(), target.data(), 256);
            if (!pred.empty())
                ImPlot::PlotLine("model(x)", xs.data(), pred.data(), 256);
            ImPlot::EndPlot();
        }
        ImGui::Checkbox("follow", &st->follow);   // viewport policy, §6
        const ImPlotAxisFlags f =
            st->follow ? ImPlotAxisFlags_AutoFit : 0;
        if (ImPlot::BeginPlot("loss", {-1, 160}, kLockedPlot)) {
            ImPlot::SetupAxes("step", "MSE", f, f);
            if (!loss.empty())
                ImPlot::PlotLine("mse", loss.data(), (int)loss.size());
            ImPlot::EndPlot();
        }

Stage 4 — the bridge: a tensor as pixels

The weight matrix as a colormapped texture. Frame thread only, gen-gated, released in cleanup — and when the bridge is absent, the panel says so politely instead of failing:

        // The weight matrix as pixels — gen-gated rebuild, frame thread only.
        if (st->bridge && gen != 0 && gen != st->tex_gen &&
            disp_w.defined()) {
            if (st->w_tex) st->bridge.release_texture(st->w_tex);
            auto ct = caliper::adapters::to_tensor(disp_w);
            st->w_tex = ct ? st->bridge.texture_from_tensor_mapped(
                                 &*ct, CALIPER_CMAP_RDBU, -w_max, w_max)
                           : 0;
            st->tex_gen = gen;
        }
        if (st->w_tex) {
            ImGui::TextDisabled("fc1 weights (32x1 as 8x4 blocks, RdBu)");
            ImGui::Image(caliper::Bridge::imtex(st->w_tex), ImVec2(128, 256));
        } else {
            ImGui::TextDisabled(
                "tensor_bridge.v1 absent (ok) — no weight heatmap");
        }

(The tutorial stages the weights to CPU for simplicity; the exemplar's Tensors panel shows the full zero-copy device pull — weights that never leave the GPU — in cookbook §3.)

Cleanup grows its symmetric duties — cancel, bounded wait, release:

    void on_cleanup() override {
        auto* st = s_.get();
        if (st->job_id) {
            st->jobs.request_cancel(st->job_id);
            for (int i = 0; i < 1000 && st->jobs.is_running(st->job_id); i++)
                std::this_thread::sleep_for(std::chrono::milliseconds(1));
        }
        if (st->w_tex) { st->bridge.release_texture(st->w_tex);
                         st->w_tex = 0; }
        if (st->host) st->host->log_info("sine_scope: on_cleanup");
    }

Run it (CALIPER_AUTOLAUNCH=dev.example.sine-scope ./build/caliper): you should see the flat line snap into a sine within seconds while the heatmap's blocks reorganize.

Stage 5 — real data instead of synthetic

Everything above holds; only acquisition changes. The rules (cookbook §8): fetch inside the job, cache in host.data_dir(), write atomically (.tmp + rename), self-heal corrupt caches, make the transfer cancellable via curl's progress callback, and add CURL::libcurl/ZLIB::ZLIB to the CMake links. The exemplar's ensure_dataset + mnist_path are the copy-paste source — including the sibling-cache trick (reuse another applet's MNIST download rather than duplicating 11 MB).

Stage 6 — the rest of the framework, one line each

Each remaining service is a small delta from here, and the exemplar shows all of them finished:

  • metrics.v1 — persistence + the Runs dashboard for two lines: run = metrics.begin_run("sine", "mlp32") once, then metrics.scalar(run, "train/loss", step, loss) in the loop. Your run now survives restarts and plots in the host's Runs window.
  • artifacts.v1 — Save/Load buttons so a trained model outlives the process (cookbook §9). Load-then-eval without retraining is the demo magic.
  • data.v1 — when your published state is genuinely tabular, register it and ask SQL questions (cookbook §10).

When all of these feel natural, read applets/embed_scope/ end to end — it is exactly this tutorial's patterns at full scale: a real dataset, a 3-D learned embedding, per-step device pulls, and all eight services in ~900 annotated lines.