PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning

Code

• github.com/njunlp/peg-rl
  OfficialIn paper★ 0

Abstract

Reinforcement learning (RL) has shown strong promise for LLM-based machine translation, with recent methods such as GRPO demonstrating notable gains; nevertheless, translation-oriented RL remains challenged by noisy learning signals arising from Monte Carlo return estimation, as well as a large trajectory space that favors global exploration over fine-grained local optimization. We introduce PEGRL, a two-stage RL framework that uses post-editing as an auxiliary task to stabilize training and guide overall optimization. At each iteration, translation outputs are sampled to construct post-editing inputs, allowing return estimation in the post-editing stage to benefit from conditioning on the current translation behavior, while jointly supporting both global exploration and fine-grained local optimization. A task-specific weighting scheme further balances the contributions of translation and post-editing objectives, yielding a biased yet more sample-efficient estimator. Experiments on EnglishFinnish, EnglishTurkish, and EnglishChinese show consistent gains over RL baselines, and for EnglishTurkish, performance on COMET-KIWI is comparable to advanced LLM-based systems (DeepSeek-V3.2). Our code and a set of representative pretrained models are publicly available at https://github.com/NJUNLP/peg-rl and https://huggingface.co/collections/DGME/pegrl

Reproductions