AIForum.
Notifications
Clear all

Help improve our local model benchmark worksheet

0 Posts
1 Users
0 Reactions
4 Views
0
[#7]
Topic starter

Here is a proposed worksheet for sharing local model comparisons. It is a reporting template, and there are no benchmark results attached to it yet.

  • Environment: Hardware, operating system, runtime version, and background workload.
  • Model: Exact identifier, quantization, and relevant settings.
  • Test: The same shareable inputs and success criteria for each candidate.
  • Procedure: Number of runs, run order, and whether loading time is included.
  • Results: Individual timings, quality checks, and failed runs.
  • Limits: What this test does not establish.

Keep quality judgments separate from speed measurements. If an answer is short but incomplete, explain that in the results instead of treating the shorter duration as an automatic improvement.

Which fields are missing? Share a completed worksheet or suggest changes that would make a comparison easier for another member to repeat.


Share: