SWE 任务和轨迹进度看板

收集 PR 总数

557,059

1h 0 / 24h 0

有效 SWE 总数

40,282

1h +52 / 24h +1,254

整体处理成功率

11.7%

Valid SWE / 已处理 345,614

difficulty_score 均值

5.86

median 5.8，count 40,231

语言进度

语言	收集 PR	有效 SWE	过去 1h	过去 24h	已处理	处理成功率
Cc	29,469	8,559	+3	+182	29,457	29.1%
C++cpp	45,612	3,104	0	+16	37,183	8.3%
Gogo	126,439	6,858	+15	+333	49,115	14.0%
Javajava	84,868	3,060	+3	+26	37,023	8.3%
JavaScriptjs	36,409	5,193	+9	+184	48,428	10.7%
Pythonpy	98,883	4,233	+3	+180	46,893	9.0%
Rustrust	68,650	4,390	+10	+175	44,220	9.9%
TypeScriptts	66,729	4,885	+9	+158	53,295	9.2%

运行参数

语言	评估模型 (OPENAI)	填充模型 (ANTHROPIC)	并发数	min_source_files	max_source_files
C	`gpt-5.4`	`claude-sonnet-4-6`	12	2	15
C++	`glm-5`	`claude-sonnet-4-6`	8	2	15
Go	`Qwen3.6-35B-A3B`	`Qwen3.6-35B-A3B`	12	2	10
Java	`claude-haiku-4-5-20251001`	`claude-sonnet-4-6`	8	2	10
JavaScript	`Qwen3.6-35B-A3B`	`Qwen3.6-35B-A3B`	12	2	10
Python	`Qwen3.6-35B-A3B`	`Qwen3.6-35B-A3B`	12	3	15
Rust	`Qwen3.6-35B-A3B`	`Qwen3.6-35B-A3B`	8	2	10
TypeScript	`Qwen3.6-35B-A3B`	`Qwen3.6-35B-A3B`	12	2	10

失败原因统计

语言	已处理	有效 SWE	失败	trivial_pr	validation	infra_error	timeout	workflow_error	其他
C	29,457	8,559	20,898	14,724	1,443	4,403	152	428	2
C++	37,183	3,104	34,079	8,332	2,673	23,668	395	607	267
Go	49,115	6,858	42,257	20,114	6,081	12,570	1,614	892	942
Java	37,023	3,060	33,963	14,403	6,143	10,380	1,088	651	1,602
JavaScript	48,428	5,193	43,235	19,637	1,910	21,263	281	144	0
Python	46,893	4,233	42,660	23,374	5,204	14,512	770	299	121
Rust	44,220	4,390	39,830	17,923	5,719	13,405	1,234	832	1,236
TypeScript	53,295	4,885	48,410	11,647	2,959	32,469	559	765	11

trivial_pr：PR 被 LLM 评估为过于简单（如仅修改配置、文档、依赖版本等），不适合作为 SWE 任务。

validation：任务生成后验证失败（NOP agent 未返回 reward=0 或 ORACLE agent 未返回 reward=1）。

infra_error：基础设施错误（Docker 构建失败、网络超时、磁盘空间不足等）。

timeout：处理超时（单个 PR 总超时或 Claude Code session 超时）。

workflow_error：工作流程错误（PR 元数据获取失败、worktree 创建失败、patch 生成失败等）。

fix.patch 复杂度

语言	Valid SWE Count	Avg fix.patch lines	Avg fix.patch hunks	Avg fix.patch files
C	8,559	285.70	15.88	5.25
C++	3,104	286.60	12.42	4.79
Go	6,858	230.69	13.37	4.54
Java	3,060	171.82	11.06	4.46
JavaScript	5,193	75.49	6.24	2.78
Python	4,233	153.14	11.08	3.83
Rust	4,390	233.44	13.22	4.13
TypeScript	4,885	165.57	9.58	4.21

统计方法说明

难度打分 difficulty_score

读取每个有效任务目录的 solution/fix.patch、tests/ 和 instruction.md，由 src/swegen/scoring.py 使用零 API 静态评分。

当前公式采用 log-scale 连续评分，避免中等规模 patch 过早变成 hard。权重为：patch_scope 38%、logic_complexity 32%、context_breadth 15%、test_complexity 10%、instruction_complexity 5%。

label 阈值：easy <= 4.0，medium <= 7.0，hard > 7.0。

Tags 生成与展示

tags 不是看板现场计算的，而是在 swegen 构建任务时由 LLM 根据 PR 信息生成，并写入 task.toml 的 [metadata].tags。

prompt 要求 tags 按三段式生成：编程语言、项目层级/领域、框架/库名或具体主题。看板只读取已有 task.toml 并统计每个语言的 tag 出现次数和占比。

fix.patch 统计

patch 统计来自每个有效任务的 solution/fix.patch，并按语言扩展名过滤代码文件，口径与 upload_march_swe_to_hf.py 的 code-only 统计保持一致。

Avg fix.patch lines 统计代码文件 diff 中新增/删除行数；Avg fix.patch hunks 统计 @@ hunk 数；Avg fix.patch files 统计涉及的代码文件数。

difficulty_label 分布

语言	easy / medium / hard	easy	medium	hard
C	783 / 5710 / 2058	783	5,710	2,058
C++	391 / 1919 / 790	391	1,919	790
Go	514 / 4977 / 1361	514	4,977	1,361
Java	377 / 1957 / 721	377	1,957	721
JavaScript	817 / 3841 / 533	817	3,841	533
Python	240 / 2735 / 1235	240	2,735	1,235
Rust	340 / 2598 / 1450	340	2,598	1,450
TypeScript	450 / 3643 / 791	450	3,643	791

difficulty_score 概览

语言	count	min	p25	median	mean	p75	max
C	8,551	2.4	4.9	6.0	5.95	7.0	9.2
C++	3,100	2.5	4.7	5.9	5.88	7.1	9.1
Go	6,852	2.6	4.9	5.8	5.87	6.8	9.1
Java	3,055	2.8	4.7	5.9	5.85	7.0	9.2
JavaScript	5,191	2.6	4.4	5.2	5.32	6.2	9.2
Python	4,210	2.6	5.2	6.2	6.18	7.3	9.1
Rust	4,388	2.7	5.1	6.3	6.23	7.4	9.0
TypeScript	4,884	2.7	4.7	5.6	5.68	6.6	9.1

全局 Top Tags

library18,339 (45.6%)

backend12,681 (31.5%)

cli5,586 (13.9%)

frontend2,241 (5.6%)

testing1,693 (4.2%)

http1,267 (3.1%)

react1,152 (2.9%)

framework996 (2.5%)

kubernetes615 (1.5%)

embedded611 (1.5%)

async559 (1.4%)

networking477 (1.2%)

cpp451 (1.1%)

graphql296 (0.7%)

parsing280 (0.7%)

eslint271 (0.7%)

postgresql269 (0.7%)

git249 (0.6%)

aws241 (0.6%)

compiler229 (0.6%)

security223 (0.6%)

json218 (0.5%)

typescript214 (0.5%)

database212 (0.5%)

fastapi199 (0.5%)

angular194 (0.5%)

redis190 (0.5%)

quic187 (0.5%)

cryptography182 (0.5%)

kernel180 (0.4%)

每语言 Tags 分布

C c

library4,507 (52.7%)

backend2,142 (25.0%)

cli1,024 (12.0%)

embedded594 (6.9%)

cpp450 (5.3%)

networking320 (3.7%)

testing253 (3.0%)

framework197 (2.3%)

postgresql194 (2.3%)

firmware178 (2.1%)

kernel177 (2.1%)

http170 (2.0%)

quic169 (2.0%)

ruby150 (1.8%)

rust135 (1.6%)

tls127 (1.5%)

bluetooth123 (1.4%)

scheduler110 (1.3%)

cryptography107 (1.3%)

python95 (1.1%)

C++ cpp

library2,245 (72.4%)

backend476 (15.4%)

testing402 (13.0%)

cli229 (7.4%)

framework162 (5.2%)

http113 (3.6%)

boost112 (3.6%)

async81 (2.6%)

parsing52 (1.7%)

serialization47 (1.5%)

qt46 (1.5%)

compiler43 (1.4%)

geometry40 (1.3%)

networking35 (1.1%)

frontend34 (1.1%)

formatting31 (1.0%)

templates31 (1.0%)

json30 (1.0%)

logging30 (1.0%)

actor-framework28 (0.9%)

Go go

backend3,680 (53.7%)

cli1,701 (24.8%)

library1,591 (23.2%)

kubernetes567 (8.3%)

http457 (6.7%)

testing190 (2.8%)

docker123 (1.8%)

aws112 (1.6%)

grpc102 (1.5%)

terraform94 (1.4%)

prometheus81 (1.2%)

framework80 (1.2%)

database75 (1.1%)

api70 (1.0%)

git70 (1.0%)

networking70 (1.0%)

aws-sdk63 (0.9%)

security59 (0.9%)

blockchain52 (0.8%)

dns50 (0.7%)

Java java

backend1,483 (48.5%)

library1,398 (45.7%)

aem147 (4.8%)

testing139 (4.5%)

spring117 (3.8%)

framework104 (3.4%)

http90 (2.9%)

android85 (2.8%)

json65 (2.1%)

sling57 (1.9%)

cli50 (1.6%)

flink47 (1.5%)

concurrency42 (1.4%)

grpc40 (1.3%)

kafka35 (1.1%)

maven35 (1.1%)

nacos34 (1.1%)

mybatis33 (1.1%)

websocket29 (0.9%)

security28 (0.9%)

JavaScript js

library2,692 (51.8%)

backend999 (19.2%)

frontend689 (13.3%)

cli618 (11.9%)

react254 (4.9%)

testing248 (4.8%)

eslint209 (4.0%)

framework187 (3.6%)

typescript157 (3.0%)

http145 (2.8%)

fastify128 (2.5%)

webpack105 (2.0%)

mongoose99 (1.9%)

nodejs85 (1.6%)

express82 (1.6%)

svelte69 (1.3%)

lighthouse62 (1.2%)

async58 (1.1%)

aframe54 (1.0%)

vue54 (1.0%)

Python py

backend1,803 (42.8%)

library1,708 (40.6%)

cli604 (14.3%)

fastapi178 (4.2%)

django115 (2.7%)

testing101 (2.4%)

ansible99 (2.4%)

pytorch93 (2.2%)

async88 (2.1%)

framework78 (1.9%)

http78 (1.9%)

aws68 (1.6%)

aiohttp65 (1.5%)

openai52 (1.2%)

click51 (1.2%)

litellm49 (1.2%)

beets48 (1.1%)

pydantic48 (1.1%)

flask44 (1.0%)

black42 (1.0%)

Rust rust

library2,379 (54.2%)

backend970 (22.1%)

cli930 (21.2%)

testing259 (5.9%)

async188 (4.3%)

http162 (3.7%)

git110 (2.5%)

compiler93 (2.1%)

parsing85 (1.9%)

macros79 (1.8%)

graphql71 (1.6%)

substrate66 (1.5%)

sql61 (1.4%)

blockchain60 (1.4%)

framework58 (1.3%)

serde58 (1.3%)

datafusion56 (1.3%)

cryptography53 (1.2%)

actix-web45 (1.0%)

lsp45 (1.0%)

TypeScript ts

library1,819 (37.2%)

frontend1,344 (27.5%)

backend1,128 (23.1%)

react890 (18.2%)

cli430 (8.8%)

angular188 (3.8%)

graphql151 (3.1%)

framework130 (2.7%)

fullstack115 (2.4%)

electron109 (2.2%)

testing101 (2.1%)

javascript83 (1.7%)

vue75 (1.5%)

express67 (1.4%)

github-actions66 (1.4%)

eslint62 (1.3%)

nextjs62 (1.3%)

vscode54 (1.1%)

http52 (1.1%)

mcp52 (1.1%)

轨迹文件数

轨迹总条数

65,965

平均消息轮数

61.5

平均 composite_score

0.7449

基于 29,596 条有分数的轨迹

轨迹文件总览

数据集	脚手架	模型	Owner	轨迹数	文件大小	平均轮数	平均 Token	平均 Tool Calls	平均 Score
swegen	Claude Code	`glm5`	chaofan	1,096	446 MB	72.7	39,648	37.9	0.7218
swegen	OpenCode	`glm5`	chaofan	860	197 MB	64.8	30,577	34.0	0.7337
swegen	OpenHands SDK	`glm5`	chaofan	1,140	280 MB	119.6	49,811	59.8	0.7533
swegen	Terminus-2	`glm5`	chaofan	1,331	142 MB	53.2	26,968	41.2	0.7048
swerebench_oraclesolved	Claude Code	`glm5`	chaofan	4,053	601 MB	44.1	22,117	22.7	—
swerebench_oraclesolved	OpenCode	`glm5`	chaofan	3,431	405 MB	41.3	19,704	21.5	—
swerebench_oraclesolved	OpenHands-AI	`glm5`	chaofan	2,808	638 MB	135.1	36,434	67.2	—
swerebench_oraclesolved	Terminus-2	`glm5`	chaofan	2,920	291 MB	55.7	25,473	36.9	—
swerebench_oraclesolved	Claude Code	`glm5`	jierun	5,013	779 MB	44.7	24,123	23.3	—
swerebench_oraclesolved	OpenCode	`glm5`	jierun	3,521	464 MB	51.8	22,447	27.0	—
swerebench_oraclesolved	OpenHands SDK	`glm5`	jierun	2,733	507 MB	95.8	35,567	47.5	—
swerebench_oraclesolved	Terminus-2	`glm5`	jierun	2,998	296 MB	55.5	25,257	37.0	—
swerebench_others	Claude Code	`glm5`	jierun	2,885	457 MB	43.8	25,194	23.1	0.7178
swerebench_others	OpenCode	`glm5`	jierun	2,637	360 MB	49.1	23,637	25.8	0.7210
swerebench_others	OpenHands SDK	`glm5`	jierun	1,843	340 MB	90.2	35,602	44.8	0.7868
swerebench_others	Terminus-2	`glm5`	jierun	2,270	225 MB	50.6	24,990	35.1	0.7844
swerebenchv2_python_oraclesolved	Claude Code	`glm5`	jierun	2,728	445 MB	47.5	26,064	25.1	—
swerebenchv2_python_oraclesolved	OpenCode	`glm5`	jierun	1,654	240 MB	60.7	23,898	31.3	—
swerebenchv2_python_oraclesolved	OpenHands SDK	`glm5`	jierun	1,521	313 MB	103.0	40,232	51.0	—
swerebenchv2_python_oraclesolved	Terminus-2	`glm5`	jierun	1,633	174 MB	55.7	27,013	37.6	—
v2nopy_full	Claude Code	`glm5`	jierun	5,128	818 MB	42.7	26,941	22.8	0.7281
v2nopy_full	OpenCode	`glm5`	jierun	3,465	455 MB	47.0	23,172	24.8	0.7353
v2nopy_full	OpenHands SDK	`glm5`	jierun	3,266	620 MB	92.7	38,252	46.0	0.7580
v2nopy_full	Terminus-2	`glm5`	jierun	3,675	411 MB	64.1	28,365	45.6	0.7799

质量评分统计

数据集	脚手架	composite	efficiency	style	tool_mastery	completion	precision
swegen	Claude Code	0.7218	0.9125	0.3414	0.8678	0.6213	0.7722
swegen	OpenCode	0.7337	0.9322	0.3272	0.8886	0.6288	0.7923
swegen	OpenHands SDK	0.7533	0.8975	0.3500	0.8537	0.7737	0.7632
swegen	Terminus-2	0.7048	0.9241	0.4228	0.8983	0.3960	0.8866
swerebench_others	Claude Code	0.7178	0.8864	0.3197	0.7942	0.6405	0.8924
swerebench_others	OpenCode	0.7210	0.8952	0.3171	0.8093	0.6509	0.8621
swerebench_others	OpenHands SDK	0.7868	0.8886	0.3693	0.7698	0.9333	0.8525
swerebench_others	Terminus-2	0.7844	0.8539	0.4292	0.8172	0.8203	0.9323
v2nopy_full	Claude Code	0.7281	0.9186	0.3213	0.8886	0.5565	0.8996
v2nopy_full	OpenCode	0.7353	0.9278	0.3254	0.9008	0.5766	0.8772
v2nopy_full	OpenHands SDK	0.7580	0.9002	0.3555	0.8440	0.7523	0.8370
v2nopy_full	Terminus-2	0.7799	0.9222	0.4310	0.9016	0.6930	0.8808

按数据集对比

数据集	总轨迹数	平均轮数	平均 Token	平均 Tool Calls	平均 Score
swegen	4,427	77.4	36,691	43.8	0.7271
swerebench_oraclesolved	27,477	61.8	25,724	33.3	—
swerebench_others	9,635	55.7	26,711	30.8	0.7475
swerebenchv2_python_oraclesolved	7,536	63.4	28,654	34.4	—
v2nopy_full	15,534	59.2	28,815	33.5	0.7483

源数据路径目录

数据集	脚手架	Owner	文件路径	大小	条数
swegen	Claude Code	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swegen_cc_2135.jsonl`	446 MB	2,135
swegen	OpenCode	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swegen_oc_1177.jsonl`	197 MB	1,177
swegen	OpenHands SDK	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swegen_ohsdk_1140.jsonl`	280 MB	1,140
swegen	Terminus-2	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swegen_t2_1331.jsonl`	142 MB	1,331
swerebench_oraclesolved	Claude Code	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swerebench_oraclesolved_cc_4053.jsonl`	601 MB	4,053
swerebench_oraclesolved	Claude Code	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_oraclesolved_cc_5013.jsonl`	779 MB	5,013
swerebench_oraclesolved	OpenCode	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swerebench_oraclesolved_oc_3431.jsonl`	405 MB	3,431
swerebench_oraclesolved	OpenCode	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_oraclesolved_oc_3521.jsonl`	464 MB	3,521
swerebench_oraclesolved	OpenHands-AI	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swerebench_oraclesolved_oh_2808.jsonl`	638 MB	2,808
swerebench_oraclesolved	OpenHands SDK	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_oraclesolved_oh_sdk_2733.jsonl`	507 MB	2,733
swerebench_oraclesolved	Terminus-2	chaofan	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/chaofan_glm5_swerebench_oraclesolved_t2_2920.jsonl`	291 MB	2,920
swerebench_oraclesolved	Terminus-2	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_oraclesolved_t2_2998.jsonl`	296 MB	2,998
swerebench_others	Claude Code	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_others_cc_2885.jsonl`	457 MB	2,885
swerebench_others	OpenCode	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_others_oc_2637.jsonl`	360 MB	2,637
swerebench_others	OpenHands SDK	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_others_oh_sdk_1843.jsonl`	340 MB	1,843
swerebench_others	Terminus-2	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebench_others_t2_2270.jsonl`	225 MB	2,270
swerebenchv2_python_oraclesolved	Claude Code	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebenchv2_python_oraclesolved_cc_2728.jsonl`	445 MB	2,728
swerebenchv2_python_oraclesolved	OpenCode	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebenchv2_python_oraclesolved_oc_1654.jsonl`	240 MB	1,654
swerebenchv2_python_oraclesolved	OpenHands SDK	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebenchv2_python_oraclesolved_oh_sdk_1521.jsonl`	313 MB	1,521
swerebenchv2_python_oraclesolved	Terminus-2	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_swerebenchv2_python_oraclesolved_t2_1633.jsonl`	174 MB	1,633
v2nopy_full	Claude Code	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_v2nopy_full_cc_5128.jsonl`	818 MB	5,128
v2nopy_full	OpenCode	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_v2nopy_full_oc_3465.jsonl`	455 MB	3,465
v2nopy_full	OpenHands SDK	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_v2nopy_full_oh_sdk_3266.jsonl`	620 MB	3,266
v2nopy_full	Terminus-2	jierun	`/home/ywxzml3j/ywxzml3juser57/LLaMA-Factory/data/chaofan_jierun_traj/jierun_glm5_v2nopy_full_t2_3675.jsonl`	411 MB	3,675

统计方法说明

平均轮数 / Token / Tool Calls

平均轮数：每条轨迹的 messages 数组长度的平均值。

平均 Token：使用 tiktoken cl100k_base tokenizer 对所有 message 的 content + reasoning_content 精确编码计数的平均值。

平均 Tool Calls：assistant 消息中 tool_calls 数组长度之和的平均值。对 Terminus-2 脚手架，统计 assistant 消息 JSON content 中 commands 数组的长度。

质量评分

composite_score（0-1）由五个维度加权：efficiency（效率）、style（风格）、tool_mastery（工具掌握）、completion（完成度）、precision（精确度）。

仅部分文件包含 _score 字段，无分数的文件显示 "—"。