CULTURE-MT Leaderboard

CULTURE-MT is a benchmark for evaluating culturally-aware social media translation.

Participants submit English translations for the benchmark inputs. Submissions are evaluated by a private Judger API, and the aggregated scores are displayed on this leaderboard.


1	GPT-5	--	GPT-5	API	8.08	91.92	0.9	7.19	51.56	40.35	2.31	2026-05-27 22:38:47
2	Gemini 3 pro	--	Gemini 3 pro	API	8.95	91.05	0.2	8.75	51.97	39.09	2.3	2026-05-27 22:46:10
3	CULTURE-MT-baseline-32B	ZJU & FDU & Xiaohongshu	Qwen3-32B	SFT	12.08	87.92	0.73	11.35	56.56	31.37	2.19	2026-05-27 18:26:06
4	CULTURE-MT-baseline-8B	ZJU & FDU & Xiaohongshu	Qwen3-8B	SFT	14.34	85.66	1.27	13.07	57.42	28.24	2.13	2026-05-27 18:33:54
5	GLM4.6	--	GLM4.6	API	15.94	84.06	3.12	12.81	57.39	26.68	2.08	2026-05-27 22:32:00
6	Deepseek V3.2	--	Deepseek V3.2	API	17.73	82.27	2.59	15.14	56.82	25.45	2.05	2026-05-27 22:25:50
7	Qwen3-235B-A22B	--	Qwen3	API	29.27	70.73	12.31	16.97	52.86	17.87	1.76	2026-05-27 23:18:53
8	Seed-X-PPO	--	Mistral-7B	RL	30.58	69.42	3.33	27.25	62.23	7.19	1.73	2026-05-27 23:45:14
9	Qwen3-32B	--	Qwen3	Base	31.67	68.33	5.19	26.48	58.68	9.65	1.73	2026-05-27 23:25:55
10	Qwen3-14B	--	Qwen3	Base	38.95	61.05	12.77	26.18	54.59	6.46	1.55	2026-05-27 23:12:48
11	Qwen3-8B	--	Qwen3	Base	48.64	51.36	15	33.63	45.98	5.39	1.42	2026-05-27 23:37:09
12	Qwen3-4B	--	Qwen3	Base	60.41	39.59	21.33	39.09	37.92	1.67	1.2	2026-05-27 23:31:31
13	Qwen3-1.7B	--	Qwen3	Base	83.79	16.21	45.84	37.95	16.11	0.1	0.7	2026-05-27 23:05:14
14	Qwen3-0.6B	--	Qwen3	Base	94.84	5.16	74.65	20.19	5.09	0.07	0.31	2026-05-27 22:55:41