NeurIPS 2026

Concise Reasoning Through the Lens of Lagrangian Optimization

Chengqian Gao¹, Haonan Li¹, Taylor W. Killian², Jianshu She¹, Renxi Wang¹, Liqun Ma¹, Zhoujun Cheng³, Shibo Hao³, Zhiqiang Xu¹

¹MBZUAI ²Brigham Young University ³UC San Diego

Reasoning models overthink, but how short should a response be?

Abstract

Concise reasoning in large language models (LLMs) seeks to generate only essential steps needed to arrive at a final answer, thereby alleviating issues of “overthinking”. Most proposed approaches reduce conciseness to a hand-tuned coefficient or threshold that must be re-tuned across domains and model scales. We address this brittleness by treating concise reasoning as a constrained problem that minimizes length subject to an accuracy floor, and develop Performance-Aware Length Update (PALU), an algorithm guided by a Lagrangian view of this problem. PALU maintains a per-prompt length budget and updates it using two data-driven signals: the prompt's pass rate, which determines whether to compress, and the dispersion of its correct responses, which determines how much. On DeepSeek-R1-Distill-Qwen-1.5B, PALU reduces generation length by 64% while improving accuracy from 44.2% to 51.6% across six benchmarks, matching GRPO accuracy under a 2.5× tighter length budget. The same hyperparameters transfer across math, logic and STEM, and across 1.5B to 14B models, suggesting that constrained optimization remains a productive design scaffold for training reasoning LLMs even where its classical guarantees no longer apply.

Idea

Balance accuracy and length with a Lagrangian, not a length target

Box plots of response length for DeepSeek-R1-Distill-Qwen-1.5B, base model versus PALU, on AIME 24, Math, Logic and STEM. Base medians range from about 16k tokens on AIME 24 to under 4k on STEM; PALU is shorter on every domain. — **Accuracy and length are two objectives, and a Lagrangian can weigh them without a length target.** RL with verifiable rewards, such as GRPO, optimizes accuracy alone; concise-reasoning methods push length toward a fixed target. A fixed target cannot fit every setting: the same 1.5B model writes a median of about 16k tokens on AIME 24 but under 4k on STEM (left), and on AIME 24 the median falls from about 17k tokens at 1.5B to 8k at 32B (right). PALU instead minimizes length subject to an accuracy floor C, which means the same thing in every setting.

Box plots of response length on AIME 2024 for DeepSeek-R1-Distill-Qwen at 1.5B, 7B, 14B and 32B. The median falls from about 17k tokens at 1.5B to about 8k at 32B. — **Accuracy and length are two objectives, and a Lagrangian can weigh them without a length target.** RL with verifiable rewards, such as GRPO, optimizes accuracy alone; concise-reasoning methods push length toward a fixed target. A fixed target cannot fit every setting: the same 1.5B model writes a median of about 16k tokens on AIME 24 but under 4k on STEM (left), and on AIME 24 the median falls from about 17k tokens at 1.5B to 8k at 32B (right). PALU instead minimizes length subject to an accuracy floor C, which means the same thing in every setting.

min_{θ, L > 0} L s.t. R (θ, L, q) \geq C

min_{θ, L > 0} max_{λ \geq 0} ℒ (θ, L, λ) = L + λ (C - R (θ, L, q))

L: the length budget of prompt q, i.e. the max-token limit (max_tokens in vLLM or SGLang) used when sampling its rollouts. R: the expected reward, here accuracy, of rollouts under that budget. C: the accuracy floor. λ: the Lagrange multiplier, which grows while the floor is violated.

Method

PALU reads performance, decides whether to act, and sets how far to move

PALU keeps a length budget for every prompt and updates it once per epoch from the rollouts GRPO has already collected. In this example, 13 of 16 rollouts are correct, so the prompt meets the accuracy floor. Its correct responses spread from 2.8k to 15.4k tokens, so the budget can shrink by a large step, α = 8.6k, from 16k to 7.4k. Each step below is a tractable stand-in for one update of the Lagrangian, which cannot be run directly in LLM training.

1
Read performanceEstimate the pass rate from the previous epoch's rollouts, which replaces a fresh estimate of the residual C − R and costs no extra generation: $R (θ, L, q) \approx 𝔼_{o \sim π_{θ_{old}} (\cdot | q, L_{old})} [r (o, a)]$
2
Decide whether to actShrink the budget when the floor is met; otherwise reset it so the model can recover. This two-regime rule replaces gradient ascent on λ, which cannot grow large enough within 5–20 epochs: $L \leftarrow {\begin{cases} L - α_{τ}^{(q)} & if R (θ, L, q) \geq C \\ L_{max} & otherwise \end{cases}$
3
Set how far to moveUse the quantile gap of correct-response lengths as the step, a stand-in for ∇_LR. Correct responses packed close together give a small step; a long tail gives a large one: $α_{τ}^{(q)} := Q_{1.0}^{(q)} - Q_{1.0 - τ}^{(q)}$ where Q_τ^(q) is the τ-quantile of the lengths of correct responses to q in the previous epoch.

At inference time only a fixed maximum length applies; the shorter reasoning lives in the trained weights.

Finding 1

PALU cuts length by 64% and raises accuracy

Method	Pass@1 (%)	Length (tokens)	AE score ↑
R1-Distill-Qwen-1.5B	44.2	11,297	0.000
GRPO	51.4	10,438	0.563
ThinkPrune-4k	46.4	5,390	0.659
AutoThink-Stage3	50.6	5,563	1.077
ALP	48.9	4,742	0.958
L1-Max	44.6	3,311	0.551
ShorterBetter	37.7	2,585	0.030
PALU	51.6	4,093	1.185

DeepSeek-R1-Distill-Qwen-1.5B trained on DeepScaleR, macro average over MATH-500, AIME 2024, AIME 2025, AMC 2023, OlympiadBench and Minerva-Math. AE score: accuracy-efficiency score (Luo et al., 2025). Bold is best, underlined second best. Per-task numbers and more baselines are in the paper.

PALU reaches GRPO's accuracy with 2.5× fewer tokens. Starting from DeepSeek-R1-Distill-Qwen-1.5B, PALU shortens responses from 11.3k to 4.1k tokens while raising accuracy from 44.2% to 51.6%, the best accuracy-efficiency score in the comparison. Prior methods land on one side of the trade-off: ShorterBetter and L1-Max compress harder but give up accuracy, while ThinkPrune and AutoThink keep accuracy but compress less.

What does a concise answer look like? Below are three questions answered by the base model and by PALU. Text in grey italics is the model's thinking.

Question

Which one is greater, 8.11 or 8.9?

R1-Distill-Qwen-1.5B

First, I need to compare the whole number parts of both numbers. Both 8.11 and 8.9 have the same whole number part, which is 8. Since the whole numbers are equal, I'll move on to comparing the decimal parts. Looking at 8.11, the decimal part is 0.11, while the decimal part of 8.9 is 0.9. To make the …

1. Compare the Whole Numbers: both numbers have the same whole number part: 8. 2. Compare the Decimal Parts: convert 8.9 to have two decimal places: 8.90. Now, compare 0.11 (from 8.11) and 0.90 (from 8.90). 3. Determine Which Decimal is Larger: 0.90 is greater than 0.11. 4. Conclusion: since 0.90 > 0.11, 8.90 is greater than 8.11. Therefore, 8.9 is larger than 8.11. 8.9

PALU

To determine which number is larger between 8.11 and 8.9, I'll compare their decimal places. Both numbers have the same whole number part, which is 8. Next, I'll look at the tenths place. In 8.11, the tenths digit is 1, while in 8.9, it's 9. Since 9 is greater than 1, 8.9 is larger than 8.11.

1. Compare the Whole Number Part: both numbers have the same whole number part, which is 8. 2. Compare the Tenths Place: 8.11 has a tenths digit of 1; 8.9 has a tenths digit of 9. Since 9 is greater than 1, 8.9 is larger than 8.11. 8.9

“Same answer, one step fewer: PALU compares the tenths digits directly instead of padding 8.9 to 8.90.”

Finding 2

PALU keeps compressing after shorter starts to cost accuracy

Accuracy (orange) and response length (blue) on AIME 24 over PALU training. Length falls from about 16k to about 8k tokens while accuracy rises from 0.29 to about 0.37. The background turns from green (negative rolling Spearman correlation) to red (positive correlation) after about 25% of training. — We track accuracy and length on AIME 24 during PALU training, shading the background by the rolling Spearman correlation ρ between the two. Early on (green, ρ < 0), length falls while accuracy rises: this is easy redundancy that any reasonable method removes. After about a quarter of training (red, ρ > 0), further compression genuinely costs accuracy, which is where length-penalty methods stall or start losing accuracy. PALU keeps going: within the red regime alone, length drops from 10.2k to 8.5k tokens while accuracy edges up from 35.0% to 37.4%. It can, because its decisions are made per prompt. Even when the trade-off holds on average, many prompts still meet the accuracy floor and keep shrinking at their own pace.

Finding 3

One setting transfers across domains and model scales

Nine panels, Math, Logic and STEM by 1.5B, 7B and 14B. Each plots accuracy against length reduction during training for PALU, overlong punishment and stage-based budgeting. PALU moves right (shorter) and up in every panel; the baselines barely move or move left (longer) on STEM and on Logic at 14B. — We train DeepSeek-R1-Distill-Qwen at 1.5B, 7B and 14B on a mix of math, logic and STEM data, with the same hyperparameters (C = 0.8) in all nine settings. Each panel plots accuracy against length reduction over training; moving right means shorter. Starting lengths range from 4.2k to 15.4k tokens. The two baselines aim at an 8k-token target, so they compress only where the model already writes more than that, and on STEM they drift toward longer responses. PALU has no length target to miss. It shortens responses in all nine settings, while the baselines do so in four.

Looking ahead

Concise reasoning is a useful feature in its own right. Shorter reasoning traces make it easier for people to work alongside a model, and, I hope, lower the energy cost of deployment.

There is another angle. After nearly thirty years of using natural language, I find that when two texts carry the same information, the shorter one usually comes from the clearer thinker. This makes me wonder whether concise reasoning, beyond improving the human experience and saving energy, can also raise a model's performance ceiling. If this question interests you, feel free to reach out to me, Chengqian Gao. I welcome collaboration.

BibTeX

@article{gao2025concise,
  title={Concise reasoning in the lens of lagrangian optimization},
  author={Gao, Chengqian and Li, Haonan and Killian, Taylor W and She, Jianshu and Wang, Renxi and Ma, Liqun and Cheng, Zhoujun and Hao, Shibo and Xu, Zhiqiang},
  journal={arXiv preprint arXiv:2510.10168},
  year={2025}
}