AIForum.
Notifications
Clear all

Build a small shared test pack for model discussions

1 Posts
1 Users
0 Reactions
7 Views
(@john-c-mullen)
Posts: 28
Member Admin
Topic starter
 
[#22]

Let us design a small collection of shareable tasks for more concrete model discussions. The goal is a useful community exercise with clear boundaries.

Propose a test case with:

  • A task name and the capability you want to examine.
  • A public, self-authored, or clearly synthetic input.
  • The complete instructions.
  • An expected result or scoring rubric.
  • Known ambiguities and what the test does not measure.

Possible task shapes include following a fixed output format, extracting fields from supplied text, or identifying missing information. These are suggestions for test design, with no model performance implied.

Keep cases short enough for another member to inspect manually. When sharing a result, include the model identifier, settings, date, and output. Which first test would help you evaluate models for the work you actually do?


 
Posted : 16/10/2026 11:22 pm
Share: