SWE-bench
Benchmark that scores language models on resolving real GitHub issues by producing a working patch.
About SWE-bench
SWE-bench is a benchmark for evaluating large language models on real world software issues collected from GitHub. Given a codebase and an issue, a language model is tasked with generating a patch that resolves the described problem. The dataset is published on Hugging Face as princeton-nlp/SWE-bench and the leaderboard lives at swebench.com. Details were read from the publisher’s repository and documentation; nothing was installed or executed.
What you can do
- Real GitHub issue resolution
- Patch generation and test verification
- Public leaderboard
Benchmark
- Format
- Evaluation set or harness
- Works with
- Not specified by the publisher
- License
- MIT
How to use it
How do I run SWE-bench?
Follow the publisher’s documentation to get the data and run the harness against your agent or model. The data or results are at huggingface.co/datasets/princeton-nlp/SWE-bench.
What does SWE-bench measure?
Real GitHub issue resolution; Patch generation and test verification; Public leaderboard.
Is it free?
Open source; free to run yourself.
Questions and answers
What is SWE-bench?
SWE-bench is an eval in the Developer tools category. Benchmark that scores language models on resolving real GitHub issues by producing a working patch.
How do I run SWE-bench?
Follow the publisher's documentation at https://www.swebench.com/ to get the data and run the harness against your agent or model. The data or results are at https://huggingface.co/datasets/princeton-nlp/SWE-bench.
Is SWE-bench free?
Open source; free to run yourself.
What can SWE-bench do?
According to the published details: Real GitHub issue resolution; Patch generation and test verification; Public leaderboard.
Where is SWE-bench published?
Its homepage is https://www.swebench.com/ and its source repository is https://github.com/SWE-bench/SWE-bench. RUAGENTIC read these details from Publisher repository on October 1, 2026 and did not install or run the project.
Sources and checks
View the sourceSource information collected October 1, 2026.
Badge and embed
Show this listing on your website or README. The badge and the card link back to this page.
[](https://ruagentic.com/tools/swe-bench)<a href="https://ruagentic.com/tools/swe-bench"><img src="https://ruagentic.com/badge/swe-bench.svg" alt="Listed on RUAGENTIC" height="28"></a>[](https://ruagentic.com/tools/swe-bench)<iframe src="https://ruagentic.com/embed/swe-bench" width="480" height="150" style="border:0;max-width:100%" loading="lazy" title="SWE-bench on RUAGENTIC"></iframe>