ALI LIN
Skip to content
ALI LINOpen to full-time roles
02 / 04←→×

Teardown №1

A measurement study of a hosted image generation API, built to turn “it feels fast” into numbers.

subject
fal-ai/flux/dev
work
Measurement harness & report
scope
47 calls · 5 experiments
status
Shipped · 2026
teardown.alilinlab.com ↗
Fig. 01 · The measurement harness · recorded demonstration
Latency series p50 · n = 20
533 msinference at p50

The inference segment reported in the existing latency study.

Standard deviation: 9.9 ms

Question

01

A benchmark answers which model is better. What happens inside the one you already chose?

Teardown is a measurement study of a hosted inference API (fal.ai's flux endpoint) instrumented from a browser the way an integrator would hit it.

Anyone who has waited for a generation and wondered what the wait was made of. Anyone building on one of these APIs, who has to pick a default and cannot pick it from the documentation.

Instrumentation

02

47 structured calls across five experiments, with measurements logged to disk. Select an experiment to see the question it asks.

what does another step cost, and what does it buy?

Study method / reading order
sweep → diff → time → log → publish

step sweep       / image change + inference time
guidance sweep   / adjacent image differences
seed repeats     / hashes + pixel differences
latency series   / segment timing + variability
friction log     / documentation versus behavior

Method summary from the existing case study, not a source-code excerpt.

Measurements

03

Segment timings and image differences are separate measurements. The figures below preserve the recorded values and sample sizes from the existing report.

Segment medians cannot be added to obtain end-to-end p50. End-to-end p95/p99 and the sampling window are not documented in the local case-study data.

Latency anatomy

inference533 msσ 9.9queue250.5 msσ 277.0network0 msσ 0.5dashed = not returned by the API, and where the variance livesqueue σ 277.0 ms against inference σ 9.9 ms
Fig. 02 · Measured segment p50 and standard deviation · n = 20 at S28Source cited in the existing report: e4-latency/stats.md

Inference against steps

03006009001248121620283645stepsstepsinference_msΔ msΔ % of measured131.9+17.4+54.5224.5−9.5−38.9462.4−10.7−17.18153.8+2.6+1.712239.4+10.2+4.216318+10.7+3.420368.8−16.6−4.528520.2−21.4−4.136711.6+13.9+2.045877+3.6+0.4y = 19.52x − 5.0 · R² 0.9978 · N=10
Fig. 03 · Ten measured rungs and the published linear fitSource cited in the existing report: e4-latency/stats.md

Adjacent step differences

0%6%12%18%0.066%S1→20.657%S2→416.088%S4→86.694%S8→125.91%S12→1611.92%S16→207.655%S20→281.488%S28→363.98%S36→45pixels changed, Δ>32 · N=1 per pair · the peak is early, at S4→S8past S28 the curve flattens: 7.655% then 1.488% for the same eight steps
Fig. 04 · Changed pixels, Δ > 32 · one image pair per comparisonSource cited in the existing report: _tools/adjacent-diffs.txt

Seed determinism

run 1545.1 msrun 2546.8 msrun 3549.6 mssha256 8dadd968e921aca2…: all three identical0 / 262,144 pixels differthree distinct durations: real recomputation, not a cache · 512×512 = 262,144 px
Fig. 05 · Three runs · one SHA-256 · 512 × 512 pixelsSource cited in the existing report: e3-seed/report.txt

Key decisions

04
No.DecisionWhyTrade-off
D1

Measure the integration path

WhyThe harness reaches a hosted API from the browser, as an integrator would.

Trade-offThese results describe this integration and model, not every deployment.

D2

Separate timing from image change

WhyLatency, step sweeps, guidance sweeps and seed comparisons answer different questions.

Trade-offA faster response is not itself a measure of image quality.

D3

Publish the measurement scope

WhyThe latency series, step comparisons and repeated seed runs have different sample sizes.

Trade-offA small study supports specific observations, not universal guarantees.

Findings

05
277 msQueue standard deviation
9.9 msInference standard deviation
0Different pixels in the seed repeats

F1 / The hidden segment

Queue sd 277 ms against inference sd 9.9. The segment carrying nearly all the variance is the one the response body never returns.

F2 / Identical bytes

Three runs, three different inference times, one sha256, zero of 262,144 pixels different. Determinism holds and the documentation never says so.

F3 / Past 28

S20 to S28 moves 7.655 percent of pixels. The next eight steps move 1.488.

F4 / No resting point

No adjacent guidance pair falls below 17.167 percent, and the last rung still moves 57.591. It never converges.

Designed / Built

06

Designed

The interactive report and its reading order: questions, measurement methods, figures and findings.

Built

A browser-based measurement harness using TypeScript, React and the fal API, with five experiments and measurements logged to disk.

Next teardown

07

TEARDOWN № 1 is the first in a series. The method, sweep and diff and time and log and publish with sources, ports to any inference API. The findings do not: they are one model, measured.

The next subject has not been announced in this case study.

Next project · 03Preserve
→