Datasets:
Add TypeScript/FastBrain-9B (9.4B) decision model results
Hi! This PR submits the evaluation results and verdict files for TypeScript/FastBrain-9B (9.4B parameters, ClefModel + JointSchemaHead architecture, Apache-2.0) on safeagents-judge-bench (test split, 562 calls across 47 tasks).
The .eval_results/safeagents-judge-bench.yaml file and full verdict traces are also published in TypeScript/FastBrain-9B.
Summary Results (test split — 562 calls)
| Variant | Balanced Acc ↑ | AUROC ↑ | Recall ↑ | False Block Rate ↓ | Binary F1 ↑ | Trajectories Caught | Cat. Macro-F1 ↑ | Cat. Acc ↑ | p50 Latency |
|---|---|---|---|---|---|---|---|---|---|
TypeScript/FastBrain-9B (Calibrated systemone()) |
0.974 | 0.983 | 1.000 | 0.051 | 0.998 | 78/78 | 0.850 | 0.923 | 0.104 s |
TypeScript/FastBrain-9B (Distilled Raw Head) |
0.949 | 0.968 | 0.987 | 0.089 | 0.992 | 77/78 | 0.624 | 0.815 | 0.104 s |
TypeScript/FastBrain-9B (Base Raw Head) |
0.710 | 0.926 | 0.985 | 0.564 | 0.972 | 77/78 | 0.353 | 0.552 | 0.104 s |
Files Included in This PR
results/TypeScript__FastBrain-9B.jsonl(binary track verdicts formatted viabench/decide.py)results/TypeScript__FastBrain-9B.category.jsonl(category track verdicts formatted viabench/decide.py)results/leaderboard.json&README.md(scored and formatted viabench/run.pyandbench/publish.py)bench/judges.toml([[decision]]entry usingadapter = "clef", compatible withjoint_schema_model.py)
Thanks ! Merging it
Feel free to add the eval results also directly into your model card so that it will appear on the leaderboard. Example PRs: https://huggingface.co/Cloudflare/clef-flash/discussions/8 / https://huggingface.co/vllm-sr/d3-ultra/discussions/3
Thanks so much for merging! We have also synced .eval_results/safeagents-judge-bench.yaml directly in the TypeScript/FastBrain-9B model repo so it links directly to the Hugging Face dataset leaderboard widget.
