Search results

[bug] issue with transcript mismatch and grading

Status: Open.

#365 In pinchbench/skill;

· changshivek opened on Apr 30, 2026

task_session_chain_analysis asset paths fail before agent execution

Status: Open.

#364 In pinchbench/skill;

· kxfan2002 opened on Apr 30, 2026

task frontmatter category diverges from canonical manifest categories

Status: Open.

#363 In pinchbench/skill;

· innocenceman opened on Apr 30, 2026

Inconsistent results for qwen3.5-9b compared with official reported outputs

Status: Open.

#358 In pinchbench/skill;

· Starlight2004 opened on Apr 24, 2026

Cache judge responses for reruns

Status: Open.

#353 In pinchbench/skill;

· ScuttleBot opened on Apr 22, 2026

Convert hybrid tasks to pure automated grading

Status: Open.

#352 In pinchbench/skill;

· ScuttleBot opened on Apr 22, 2026

Implement batch judge calls

Status: Open.

#350 In pinchbench/skill;

· ScuttleBot opened on Apr 22, 2026

Switch judge model from Opus to Haiku

Status: Open.

#349 In pinchbench/skill;

· ScuttleBot opened on Apr 22, 2026

[BUG] OpenClaw CLI causes task instructions to be overwritten by "Continue where you left off" message

Status: Open.

#345 In pinchbench/skill;

· GruchyC opened on Apr 22, 2026

Could the Official Runs Be Updated?

Status: Open.

#344 In pinchbench/skill;

· jijivski opened on Apr 21, 2026

Cost of Using Claude as a Judge Model per Evaluation

Status: Open.

#342 In pinchbench/skill;

· AmberXu98 opened on Apr 21, 2026

open weight models that don't display when open-weight only is checked

Status: Open.

#341 In pinchbench/skill;

· yieme opened on Apr 15, 2026