What Xiaomi did
On September 22, 2026, Xiaomi released MiMo-V2.6 Pro and Flash and open-sourced the weights and technical report. Here's the recipe:
- 7,000+ task environments across software engineering, vulnerability reproduction, knowledge-intensive work, and web design and development.
- About 750,000 trajectories (attempts) over 30 RL steps, in under six days.
- Group-relative rewards: each attempt is scored by "relative comparison within the Group."
- Reward-hacking defense: reward design, adversarial evaluation, anomaly detection and "cross-verification of validators."
The idea: the model learns "through trial and error in iterative feedback loops," not just from static text.
The vocabulary
| Term | Meaning |
|---|---|
| Task | One test with defined inputs and success criteria |
| Trial | One attempt at a task |
| Grader | Logic that scores some part of the agent's performance |
| Transcript | The full record of a trial: outputs, tool calls, reasoning |
| Outcome | The final state of the environment at the end |
The checklist for your own agent
- Start small, from real failures. Anthropic's advice: "20-50 simple tasks drawn from real failures is a great start."
- Give every task a clear pass check. Where you can, grade what the agent produced, not the path it took.
- Pick the right grader:
- Code-based: fast, objective, reproducible. Use it wherever there's a checkable answer.
- Model-based: flexible, and it picks up nuance.
- Human: the gold standard, but expensive. Use it to spot-check the other two.
- Run multiple trials. Model outputs vary between runs, so one pass proves little.
- Choose your metric:
- pass@k: did at least one of k attempts succeed?
- pass^k: did all k attempts succeed? Use this when users need it right every time.
- Read the transcripts. You won't know whether your graders work until you read transcripts and grades from many trials. That's also how you catch an agent gaming the grader.
- Cross-check your graders. Like Xiaomi, don't trust a single validator. Compare it against a second grader or a human sample.
Get the model
MiMo-V2.6 weights and training resources: https://huggingface.co/collections/XiaomiMiMo/mimo-v26
Sources: Xiaomi MiMo-V2.6 release notes (mimo.mi.com), TechNode, eWeek, Anthropic's "Demystifying evals for AI agents".