Begin with a task and a pass condition
Choose the workload before choosing the benchmark prompt. For document extraction, define the fields and compare them with known answers. For code, run an appropriate test or compiler. For an agent that selects tools, check the tool name and arguments against the intended action.
Use a saved set of prompts and record how you selected them. Include difficult cases and failures you care about. Keep the same set when comparing models. A fast completion that fails the task should remain visible as a failure.
Record model identity and environment
Save the platform model ID, repository name and revision if available, quantization, runtime version, machine memory and chip. State the date and whether the model was already loaded. If the service does not expose a field, mark it unknown rather than guessing from a tier name.
The PROXIES.SX catalog exposes supported models, immutable revisions and configuration requirements. Those fields are not measured customer performance. Record the actual node and model when evaluating a rental.
Separate admission from workload performance
Supplier admission requires a valid passing benchmark no more than 24 hours old, the measured model currently ready, a current countersigned agreement and explicit admin approval. Already admitted free nodes use a seven-day maintenance interval. A pending benchmark is neither a passing result nor approval. Admission reference
The platform benchmark measures delivered output for admission; it is not hardware attestation, a paid-rental result or a long-term availability test. Its passing state does not establish accuracy on your prompts or capacity at your expected concurrency. Use the worksheet below for that evidence. Keep observed measurements distinct from target thresholds and hypothetical examples.
Separate waiting from generation
Measure time to first token from sending the request to receiving the first generated content token. State whether that includes network travel, queueing and prompt processing. A timestamp taken only after response headers arrive misses some of the user's waiting time.
For output speed, record output tokens and the generation interval you used as the denominator. A client receiving streamed text chunks should not count chunks as tokens. Use returned token usage when available, or document the tokenizer and counting method. Measure whole-request duration as well so a long prompt cannot disappear from the comparison.
Repeat under the concurrency you need
Test one request first, then run the simultaneous workload the application actually produces. Keep a separate record for each concurrency level. Report the number of attempts, successful responses, timeouts and other errors.
For latency, include a median and a high percentile only with enough observations to make it meaningful. A p95 calculated from a handful of requests is unstable. Preserve failed attempts in the dataset rather than quietly removing them to improve the average.
A separate 2025 comparative study used a 192 GB M2 Ultra Mac Studio and Qwen-2.5 models to examine five local runtimes. It reported different leaders for sustained generation and first-token latency under its settings. Those results explain why both measurements matter; they do not rank the Qwen3.8 rental configurations here. Apple Silicon runtime study
Use consistent latency and throughput definitions
NVIDIA's GenAI-Perf documents distinct measurements for first response, request duration and token throughput. That separation is useful when comparing any inference endpoints; its example results are not Apple Silicon measurements. GenAI-Perf metric definitions
| Measurement | Record in your worksheet |
|---|---|
| Time to first token (TTFT) | Request start to first generated content. State if your tool instead uses the first server event. |
| Completion latency | Request start to final response. |
| Decode rate | Output tokens divided by the declared generation interval. State whether the first token is excluded. |
| Aggregate throughput | Completed useful work across all concurrent requests per wall-clock interval. |
| Task pass rate | Accepted results divided by all scored attempts. Keep errors visible separately. |
If a stream starts with metadata, timing that first event can understate the wait for readable output. Match definitions before comparing numbers from different tools.
Use several prompt lengths, with the same output cap and generation settings for each candidate. Run a separate cold-load case, then a warm sequence. Finish with expected peak concurrency and report how many cases you ran. A single fast request cannot establish sustained capacity or a stable high-percentile latency.
Download the recording worksheet
The CSV worksheet is an empty recording format. It contains no measured PROXIES.SX performance. Store a prompt-set identifier instead of confidential prompt text when sharing results.
Record at least request start, first content token and completion times; input and output token counts; concurrency; status; and the task's quality result. Add power readings if you are evaluating a provider machine. Mark requests made while loading the model separately from requests made after it is ready.
Turn the result into a capacity decision
Estimate whether the successful throughput can finish the expected work inside its deadline. Allow for bursts and interruption. An overnight classification job may tolerate slower responses that would make an interactive assistant frustrating.
Then use the pricing calculation with measured useful output. For providers, use wall power and observed rental periods in the earnings calculator. Publish the test conditions with any future speed claim, including the cases that did not pass.
Use this method and recording format to produce your own evidence. No model speed, sustained capacity, demand or performance ranking is promised by a catalog entry or an admission badge.
Sources and references
Product reviewed September 16, 2026 against public APIs, provider documentation and application code. External technical references explain the evaluation methods and retain their original publication dates. This guide review did not run a paid rental or payout.
- Current compute product reference. PROXIES.SX.
- Compute operation and pricing reference. PROXIES.SX.
- Production-Grade Local LLM Inference on Apple Silicon (2025). Varun Rajesh et al..
- Compute provider documentation. PROXIES.SX.
- GenAI-Perf metrics. NVIDIA.