Provider Performance

Measured latency, time-to-first-token and uptime for every provider serving a model, from real traffic the marketplace actually served.

curl https://api.surplusintelligence.ai/api/analytics/performance/gpt-5.6-luna

No authentication. Cacheable for five minutes.

{
  "model": "gpt-5.6-luna",
  "window_days": 1,
  "complete": true,
  "computed_at": 1788860286699,
  "window_start": 1788773886699,
  "window_end": 1788860286699,
  "providers": [
    {
      "provider": "openai",
      "requests": 41230,
      "failures": 512,
      "sellers": 7,
      "uptime_pct": 98.8,
      "avg_latency_ms": 1388,
      "p50_latency_ms": 1237,
      "p90_latency_ms": 2254,
      "p99_latency_ms": 7514,
      "ttft_samples": 26104,
      "ttft_p50_ms": 276,
      "ttft_p90_ms": 372,
      "route_overhead_p50_ms": 6.05,
      "tps_samples": 25980,
      "tps_p50": 84.3
    }
  ]
}

How to read it

Per provider, never per offer. Every figure aggregates across every seller offering that model on that provider. A per-offer number would be one seller's private performance data, so it is not computed and not published.

null means "not enough data", never "zero". A percentile is refused rather than approximated when the window holds too few observations to place it — a p99 needs on the order of five hundred samples, which on a 24-hour window means a pair serving roughly 500 requests a day. Expect p99_latency_ms to be null for most pairs and populated for the busy ones. Render a null as unknown; treating it as 0 would read as the fastest possible provider, the exact opposite of what it means.

ttft_samples is deliberately lower than requests. Time-to-first-token only exists for a streamed response, so it ships with its own denominator rather than implying it covers every request.

sellers is a count, never identities. A cell is published only when at least two distinct sellers serve that provider — below that, "per-provider" would be one seller's data wearing an aggregate's label, so the cell is omitted entirely rather than published. This means a provider you can see in the market may be absent here.

uptime_pct is served requests over attempts — requests / (requests + failures). The failure count comes from the health log, not from usage records: a request that never completes is never billed and so leaves no usage row, and uptime computed from usage alone would read ~100% by construction.

tps_p50 is the generation rate, not the whole request. Output tokens per second between the upstream's first byte and its last, so time-to-first-token is excluded and the figure describes how fast the provider streams once it has started. Stream-only, like TTFT, with tps_samples as its own denominator. It is deliberately pessimistic: the histogram rounds throughput down to a bin edge, never up, so a published tps_p50 is never higher than what was actually delivered.

route_overhead_p50_ms is ours, not the provider's. It is the time we spend before opening the upstream socket. It is published so you can see what the marketplace costs you in latency.

window_days is the trailing window every figure covers, and it is now a rolling 24 hours — a deliberate choice of currency over breadth, because a multi-day average does not describe how a provider is behaving today. The window is folded incrementally every 15 minutes rather than recomputed from scratch, and a published row expires six hours after the last good tick — so if the rollup stops running the endpoint starts returning 404 no_performance_data rather than serving stale numbers. A model with no recent traffic 404s for the same reason.

complete says whether the pass read the whole window. When it is false the rollup ran out of its time budget: the latency and TTFT percentiles are still valid (the portion it read is an unbiased sample of them), but requests, failures and uptime_pct are null, because a count taken from part of a window is simply wrong and a ratio built from one is wrong in a predictable direction.

A cell also needs a minimum volume. Below roughly 20 observations in the window a (model, provider) pair is omitted entirely rather than published as noise — so a provider can be absent for thin traffic as well as for thin seller diversity. On a 24-hour window that bar is ~20 requests/day, which is materially higher than it was on a weekly one: expect fewer pairs here than the market page lists.

uptime_pct is null until the failure history covers the window. The uptime denominator comes from a failure log that is younger than the request history, so rather than publishing an optimistic 100% from a source that has not been recording long enough, the field is withheld until it can be computed honestly.

What to do with it

This is the "probe a model, then choose" surface. Read it, sort by whatever you care about, and pass the result back as a routing preference:

"provider_order": ["venice", "openai"]

Sorting is deliberately left to you: a sort_by parameter here could not express the compound orderings people actually want (fastest above some uptime floor, cheapest among the reliable, and so on).

The marketplace shows the same figures without the API: expand a model on the Markets page and the Current offers table carries sortable Latency (ttft_p50_ms) and Uptime (uptime_pct) columns, one value per provider, shared by every offer listed through that provider. A - in either column is a null here, and offers without a figure always sort last.

If you want the router to weigh latency or reliability for you instead, set si_route.objective — see Routing Controls. Note that those objectives rank on the router's own live, per-process observations, not on the figures published here: this page is a trailing 24-hour view for choosing a provider, while routing needs to see the offer that started failing ten minutes ago. The two are complementary and will not always agree.