How We Measure
We take concepts from early-stage idea to a working, field-credible system. Every performance claim is backed by measurements taken on your target hardware, not estimated from paper. This page explains the protocol and what you receive.
We measure inference performance on the physical device under deployment conditions: locked clocks, minimum 200 runs, full latency distributions (p50/p95/p99/max), thermal throttling detection, and power from sustained-window sampling. Every claim is backed by a reproducible evidence package you can verify independently.
The Estimate-vs-Reality Problem
Why measured numbers are the only numbers that matter for deployment decisions.
Why We Measure
Operation counts, FLOPs, and spec-sheet TOPS are computed on paper. Real hardware routinely disagrees with them, because memory bandwidth, driver overhead, thermal state, and quantization behavior don't appear in a datasheet. Models that fly in a notebook miss their deadline in the field because they were sized on estimates. We close that gap by measuring on the actual device, first.
Our Measurement Protocol
Clocks locked to their performance governor. Device warmed to steady-state thermal before measurement begins. Initial warm-up runs discarded. A statistically sufficient sample collected. We report p50, p95, p99, and max. Not the mean. A thermal guard aborts and flags any run where throttling is detected. Energy is sampled over a sustained window, not from a single-frame peak. The protocol is documented and delivered with every engagement.
What You Receive
An evidence package: a claim-to-file manifest linking every stated number to the raw log that produced it, the raw timing artifacts themselves, a verifier script you can run independently to reproduce the result, and an honest limitations section: what the measurement captures, what it doesn't, and what we would still want to test if the engagement continued.
What We Report That Others Don't
Where our method has known gaps: batch-1 on-device latency is not the same as throughput at scale; driver and runtime version pinning matters and can shift numbers significantly; thermal behavior varies by ambient temperature and enclosure. We say all of this. A measurement that hides its assumptions is not a measurement. It is a claim.
Decisions, Not Just Outputs
Every deliverable is designed to answer a decision: can we meet the latency budget, should we target INT8 or FP16, does this architecture fit the power envelope, is this hardware the right choice? A number without a decision it informs is noise. We frame every measurement around the choice it allows you to make.
IP & Collaboration
Flexible IP arrangements: client-owned, shared, or licensed, agreed before work begins. We welcome collaborative and grant-funded research, and partnerships with integrators, academic groups, enterprise organizations, and innovation programs.
Client Data Policy
Client models, data, and results remain under client control. We work on client-provided hardware or our own bench as the engagement requires, we do not retain client data beyond the engagement without agreement, and confidentiality agreements are routine. Where work must happen on your premises or on an isolated network, that is a normal condition for us.
How We Run the Measurement
Six steps, each designed so a technical buyer can recognize — and independently verify — real rigor.
Environment Setup
CPU and GPU clocks locked to their performance governor. Thermal headroom verified. All background processes that would compete for memory bandwidth or thermal budget are stopped. The goal is a reproducible baseline, not a favorable one.
Warm-Up & Discard
The device runs the inference pipeline until it reaches steady-state thermal. Initial warm-up runs are discarded. Cold-start and ramp-up behavior are recorded separately but excluded from the steady-state distribution. They represent a different operational regime.
Timed Collection
A statistically sufficient sample collected per configuration, sized for stable tail-latency estimates. For safety-critical or high-stakes decisions, a larger sample is used. Every run is timestamped and logged to a raw artifact file.
Full Distribution
p50, p95, p99, and max are reported. The mean is not reported as a headline figure, as it hides tail behavior that matters in production. If a system must respond within a deadline 99% of the time, p99 is the number that matters.
Thermal Guard
A thermal guard monitors the device temperature throughout the run. Any sample collected while thermal throttling is active is flagged and excluded from the primary distribution. Throttled behavior is recorded separately and reported honestly.
Energy Measurement
Power draw is sampled over a sustained window at steady-state load, not from a single-frame peak or a manufacturer datasheet. Energy per inference is computed from the sustained window average, giving a realistic figure for battery or power-budget planning.