Solo agent, multi-model orchestration — 70.4% on SWE-bench Verified.
Run this team?
Claim it to edit the profile, add proof and be notified when someone asks about it.
Claim this profileThe claim covers the owner profile Refact.ai, with its teams.
We review claims within 2 business days. The decision appears on your account page.
Reference design — not an operated team.
Documented from a vendor benchmark post or product marketing; no one is known to run this team in production.
The source: Commercial product whose evidence here is vendor benchmark marketing.
Documented by AgentCV from public sources — not submitted by this team.
Not claimed by or endorsed by the team or its organization. Metrics are cited only as the sources state them; absent metrics render as [unknown].
Sources: Refact.ai blog — Open-source SOTA on SWE-bench Verified (refact.ai)
Is this your team? Claim it · Ask us to remove it
Recent activity
Version cuts and proof, newest first — the living track record.
Spec sheet
The benchmark fields — designed for comparison across teams.
- Topology
- Solo + Tools
- Agent count
- 1
- Platform
- Refact.ai
- Runs on
- Refact.ai
- Industries
- software-delivery
- Task kinds
- bug-fixingcode-editingsoftware-development
- Trust tier
- From public sources · Self-Reported
- Proof entries
- 1
Topology & roster
Solo-plus-tools. A single agent with an explicit tool loop (debug_script, strategic_planning, etc.) and orchestration-level model routing across a strong "thinking" model, a debug sub-model, and a planning tool.
System wiring
Node details
Typical Solo + Tools layout — schematic, not verified wiring
HumanHuman operatorHuman gate
- Tool
- Human operator
- Autonomy
- Human-gated
- directs → Agent
BuilderAgent
- Tool
- Agent
- Autonomy
- Runs autonomously
- writes to → Tool A
- writes to → Tool B
- directs ← Human operator
ResourceTool A
- Tool
- Tool A
- Autonomy
- Runs autonomously
- writes to ← Agent
ResourceTool B
- Tool
- Tool B
- Autonomy
- Runs autonomously
- writes to ← Agent
How a typical Solo + Tools team handles a task
Typical Solo + Tools layout — schematic, not verified wiring
Task arrives
Human operator directs Agent.
Agent builds the work
Agent builds the work.
The artifact lands
The artifact lands in Tool A: Agent contributes via "writes to". The artifact lands in Tool B: Agent contributes via "writes to".
Human holds the last word
Human operator holds final approval.
Replicate a typical Solo + Tools setup
Typical Solo + Tools layout — schematic, not verified wiring
Ingredients
- HumanHuman operator
- BuilderAgent
- ResourceTool A
- ResourceTool B
Setup order
- 1.Provision the substrate: Tool A and Tool B.
- 2.Wire Agent: it receives "directs" from Human operator.
- 3.Declare the human gate: Human operator holds final approval.
Performance metrics
Windowed metrics with provenance. [unknown] means it was not tracked — an honest hole beats an invented figure.
59.7%, described by Refact.ai as top-ranked on SWE-bench Lite at the time. Exact publish date of the Lite run not stated in the cited post; asOf uses the conservative start of the 2025-05 to 2025-06 window. Source: Refact.ai blog. [evidence_linked]
184/517, #1 at publication, Claude 4 Sonnet. Source: Refact.ai blog, 2025-06-25. [evidence_linked]
352/500, '#1 open-source AI Agent on SWE-bench' at publication. Orchestration: Claude 3.7 Sonnet; debug_script() sub-agent: Claude 3.7 + o4-mini; strategic_planning() tool: o3. Source: Refact.ai blog, 2025-05-15. [evidence_linked]
372/500 with Claude 4 Sonnet. The vendor's own post calls this figure both '#2 on SWE-bench Verified under pass@1' and, later in the same post, 'the new #1 SWE-bench result' — an internal inconsistency in the vendor's own framing, not resolved here. Source: Refact.ai blog, 2025-06-25. [evidence_linked]
Token economics
Cost transparency is part of the honesty architecture. [unknown] means it was not tracked — not that it is zero.
Blueprint
Operational DNA — why it works, how it was built, and how it is overseen. Not files for sale; knowledge of the design.
Routing distinct sub-tasks (planning, debugging, primary orchestration) to distinct models lets each step use the model best-suited to it, rather than forcing one model to do everything — the same multi-model principle behind Aider's architect/editor split, applied within a single agent's tool loop.
Open-source agent (smallcloudai/refact) and open-source SWE-bench evaluation pipeline (smallcloudai/refact-bench); both repositories are now archived (read-only) as of 2026-07-05.
None in benchmark evaluation — fully autonomous per-issue runs, leaderboard-submitted.
Proof (1)
The team's shared track record — tasks, incidents, lessons, milestones. Per-entry provenance tags are always visible.
- ArtifactMay 15, 2025evidence-linked
Refact.ai reaches #1 open-source on SWE-bench Verified
70.4% (352/500) with Claude 3.7 Sonnet orchestration; top-ranked 59.7% on SWE-bench Lite in an earlier run (per the same post).
https://refact.ai/blog/2025/open-source-sota-on-swe-bench-verified-refact-ai/
Sign in to add a proof entry.
Sign inAttestations (0)
Named third-party statements from people with first-hand experience. Attestations are what separates Peer-Attested from Evidence-Linked.
No attestations yet. Worked with this configuration or agent? Attest to it using the form below — attestations are named third-party statements and are what separates Peer-Attested from Evidence-Linked.
Sign in to attest to this team.
Sign in