Zakaj se modeli za sklepanje vrtijo v zankah?

Vir, odlomek za odlomkom

1. u1 Abstract
2. u2 Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature plays. With open reasoning models, we find that looping is common at low temperature. Larger models tend to loop less, and distilled students loop significantly even when their teachers rarely do. This points to mismatches between the training distribution and the learned model, which we refer to as errors in learning, as a key cause. To understand how such errors cause loops, we introduce a synthetic graph reasoning task and demonstrate two mechanisms. First, risk aversion caused by hardness of learning: when the correct progress-making action is hard to learn but an easy cyclic action is available, the model puts relatively more probability on the cyclic action and gets stuck. Second, even when there is no hardness, Transformers show an inductive bias toward temporally correlated errors, so the same few actions keep being chosen and loops appear. Higher temperature reduces looping by promoting exploration, but it does not fix the errors in learning, so generations remain much longer than necessary at high temperature; in this sense, temperature is a stopgap rather than a holistic solution. We end with a discussion of training-time interventions aimed at directly reducing errors in learning.1
3. u3 1 Introduction
4. u4 Reasoning models (Jaech et al., 2024; DeepSeek-AI et al., 2025; Abdin et al., 2025; Guha et al., 2025) use extra inference time compute, generating long chains of thought, to solve harder problems. This has opened a complementary scaling axis of inference-time compute, alongside training compute, resulting in striking gains on challenging tasks such as competitive math and coding. Yet these models often get stuck in loops: endlessly repeating the same text in their chain of thought, especially under greedy decoding and low temperatures (see Section 2.1 for an example). As a result, most model providers recommend running them at a sufficiently high temperature to avoid looping (e.g., see the Hugging Face pages for DeepSeek-R1 and QwQ-32B).
5. u5 This raises several questions: why do these models loop, and how does temperature help? In particular, does temperature address the root cause or mostly act as a stopgap? Ideally, temperature would be a knob we can use to control how much exploration a chain of thought performs, rather than something we must turn up just to avoid looping. More fundamentally, is randomness a necessary resource for good reasoning models? This is reminiscent of classical questions in algorithms about whether randomized algorithms are more powerful than deterministic ones (Motwani and Raghavan, 2013; Vadhan, 2012). In this work, we take a step towards understanding these questions. Our contributions are as follows.
6. u6 Figure 1: Looping and response length for OpenThinker models. Looping fraction and response length for OpenThinker3-1.5B/7B (students) and QwQ-32B (teacher), averaged over AIME 2024/2025. For each (problem, model, temperature) we sample 20 chains; a response is looping if some 30-gram appears times. (Left) Students loop much more than the teacher at low temperature. (Right) While increasing temperature largely removes looping, the student models still produce substantially longer responses than the teacher at high temperatures.
7. u7 Observations with open reasoning models (Section 2). We evaluate several open reasoning models (e.g., DeepSeek–distilled Qwen, Openthinker-3, Phi-4 reasoning) for looping on problems from the American Invitational Mathematics Examination (AIME), a high-school math contest. We make several observations: (i) all models loop at low temperatures; (ii) within a family, smaller models loop more; (iii) for models trained via distillation, students loop far more than their teachers; and (iv) for most models, harder AIME problems elicit more looping. These observations point to imperfect learning, that is, systematic errors in learning of the training distribution, as a key cause. If a student perfectly learned the teacher, then the amount of looping of the student could not be significantly higher than that of the teacher. For instance (Fig. 1), how can it be that OpenThinker3-1.5B loops in 30% of its responses with greedy decoding, while its teacher barely loops?
8. u8 Modeling and understanding looping. Next, building on Bachmann and Nagarajan (2024), we introduce a synthetic graph reasoning task with star graphs to isolate how errors in learning cause looping in a controlled setting. We train Transformers from scratch on random-walk traces that start at a designated start node and aim to reach a leaf goal, mimicking a chain of thought that sometimes makes progress and sometimes backtracks. In this setup we demonstrate two mechanisms through which learning errors cause looping.
9. u9 Risk aversion due to hardness of learning (Section 3). We show that hardness of learning can induce a form of risk aversion. When the correct progress-making action (e.g., the next step in a proof) is hard for the model to learn, but an easy cyclic action is available (e.g., restating a previously stated fact), the model puts relatively more probability on the easy cyclic action, causing looping at low temperatures. We formalize this in Proposition 3.1, which shows that indistinguishability of the hard action diffuses its probability across many alternatives, while the easy action retains its mass. In our graph reasoning task, this leads to low-temperature loops. Temperature reduces looping and improves accuracy, but the model still assigns too little probability to the progress-making action, so chains at high temperature remain much longer than those of a perfect learner.
10. u10 Inductive bias for temporally correlated errors (Section 4). We also show that even in the absence of hardness of learning, Transformers can have an inductive bias toward looping. When the training distribution places (nearly) equal mass on several progress-making actions at a decision point, small estimation errors tilt the model toward a few options, and these errors are correlated over time. When a similar decision point reappears later in the chain, the model tends to reselect the previously favored actions. Under greedy or low-temperature decoding, these small, temporally correlated errors are amplified and produce loops. We again demonstrate this mechanism in the graph reasoning task, and we observe qualitatively similar patterns in example traces from real reasoning models.
11. u11 A catalyst for looping (Section 5). On top of these mechanisms, we discuss another force: as the model repeats the same text, its confidence in continuing the repetition grows. This acts like a catalyst for any looping mechanism.
12. u12 Role of randomness. Across these mechanisms, while temperature reduces looping by promoting exploration, it does not fix the underlying errors in learning. As a result, generations at higher temperatures can still be longer than necessary. How effective temperature is depends on the size of these errors. When they are small, as in the temporally correlated errors setting, response lengths at high temperature closely match those of the teacher. When they are large, as in the risk-aversion mechanism, response lengths can be much larger than those of a perfect learner. We see a related limitation in real reasoning models: at higher temperatures, student models such as OpenThinker-3 still produce longer chains on average than their teacher (Fig. 1). Thus, while randomness can mitigate looping, it need not recover the efficiency of a better-learned model and is not a holistic solution.
13. u13 Finally, while our synthetic reasoning task provides an effective testbed to cleanly isolate several mechanisms, it does not capture the full complexity of real reasoning models, and more work is needed for a thorough understanding of looping in real models. We discuss the limitations of our synthetic setup in Section 6, along with several promising directions for future work, including more holistic training-time interventions that directly reduce errors in learning.
14. u14 To summarize, our main contributions are: (i) pointing to learning errors as a key cause of looping via experiments on open models, (ii) demonstrating mechanisms by which learning errors cause looping, via a novel graph reasoning setup, and (iii) evidence that temperature reduces looping but is not a holistic solution.
15. u15 1.1 Related work
16. u16 While looping has been especially prevalent in reasoning models, it has been observed and studied since the early days of large language models (Fan et al., 2018). Holtzman et al. (2020) brought broad attention to this “neural text degeneration,” showing that low-temperature sampling or beam search can yield generic and repetitive text. In response, several mitigations were explored. Unlikelihood training explicitly down-weights repeated or undesirable continuations (Welleck et al., 2020), and contrastive methods encourage more isotropic token representations, which reduces repetition (Su et al., 2022).
17. u17 A key data-centric insight was that model repetitions were correlated with repetitions in the training corpus; as instruction-tuning data improved and models scaled, looping became less severe (Li et al., 2023). This aligns with our evaluations as well: we find several instruction-tuned models exhibiting little looping.
18. u18 With the rise of reasoning models, however, severe looping has re-emerged. The nature of chain-of-thought data, which includes cyclic actions like backtracking and reflection (Li et al., 2025; DeepSeek-AI et al., 2025; Cuadron et al., 2025; Gandhi et al., 2025) provides fertile ground for models to fall into loops. Moreover, scaling alone is not a satisfactory solution for reasoning models: a core promise of this paradigm is to leverage inference-time compute so that even small models can perform well via longer chains. Understanding and holistically mitigating looping in this setting is therefore important, and our work takes a step towards this.
19. u19 2 Observations on open models
20. u20 We conduct a large-scale study of looping on open reasoning models, spanning a range of model sizes and training paradigms such as distillation from a teacher and RL post-training. The models we test are as follows. Qwen: DeepSeek-R1 Distilled Qwen 1.5B, 7B, 32B (DeepSeek-AI et al., 2025); OpenThinker3: OpenThinker3 1.5B, 7B (Guha et al., 2025) and QwQ-32B (Team, 2025) (the teacher for OpenThinker-3); Phi-4: Phi-4-reasoning, Phi-4-reasoning-plus (Abdin et al., 2025); Llama: DeepSeek-R1 Distilled Llama 8B (DeepSeek-AI et al., 2025).
21. u21 We mark a response as looping if it contains some $n$ -gram appearing at least $k$ times. We use $n=30$ and $k=20$ for all reasoning models (see Appendix A.3 for ablations). All plots report averages over AIME 2024 and 2025 (see Appendix A.4 for ablations with GPQA). For each triple $(\text{problem},\text{model},\text{temperature})$ with $\text{temperature}\in\{0,\,0.2,\,0.4,\,0.6,\,0.8,\,1.0\}$ , we sample 20 responses and compute accuracy, looping percentage, and response length, and then average across problems. We show looping percentages and response lengths as a function of temperature for the OpenThinker3 family in Figure 1. Plots for the other model families, including additional results for instruct models, are in Appendix A. Our main observations are:
22. u22 Looping decreases with temperature. All open reasoning models we tested loop at low temperature, and looping decreases as temperature increases. This is consistent with prior work (Holtzman et al., 2020) that reported similar trends for earlier models such as GPT-2.
23. u23 Higher-capacity models loop less. At low temperature, looping decreases with model capacity (e.g., $76\%$ for Qwen-1.5B vs. $37\%$ for Qwen-32B). Moreover, when distilling from a high-capacity teacher to a lower-capacity student, the student can loop much more than the teacher (e.g., $30\%$ for OpenThinker-3 1.5B vs. $4\%$ for QwQ-32B).
24. u24 Higher-capacity models produce shorter responses. While looping largely disappears at higher temperatures, higher-capacity models still produce shorter responses (Fig. 1). As we argue later, this relates to temperature reducing looping without holistically fixing the underlying causes.
25. u25 Harder problems elicit more looping. For most models, problems that are harder at high temperature tend to loop more at low temperature. In each 15-problem AIME split (24-I/II, 25-I/II), we rank problems by accuracy at temperature $0.8$ , take the easiest 5 and hardest 5, and pool across splits to form 20 easy and 20 hard problems. Looping at temperature $0$ is then significantly higher on hard than easy problems (e.g., $51\%$ vs. $13\%$ for OpenThinker-3 1.5B). The only exception is Phi-4 reasoning which we discuss in Appendix A.2. This suggests that no matter how large a model is, there may always exist problems that are hard enough to induce looping, though we leave a deeper study of this conjecture to future work.
26. u26 What causes looping? Two patterns stand out: low-temperature looping decreases with model capacity, and distilled students can loop far more than their (larger) teachers. If a student perfectly learned its teacher distribution, we would not expect it to loop substantially more than the teacher. This gap points to imperfect learning, that is, systematic errors in the learned distribution, as a key cause. The fact that most models loop more on harder problems also aligns with this view.
27. u27 2.1 Example Traces
28. u28 To qualitatively understand looping, we analyze looping traces generated by OpenThinker-3 1.5B (for detailed analysis including model outputs see Appendix B).
29. u29 Example 1.
30. u30 We prompt OpenThinker-3 1.5B to solve AIME 2025 Problem I-7 with greedy decoding. The task is: pair $A$ – $L$ into six two-letter words (each pair sorted alphabetically), sort the six words, and compute the probability that the last word contains $G$ . The model begins sensibly, correctly counting the total number of pairings, but then makes little progress. It first loops semantically by repeatedly re-deriving the fact that each word is the sorted pair. Later, when it turns to pairs involving $G$ , it collapses into a hard loop that repeats the same statement (e.g., “the word is the pair sorted”) and never exits.
31. u31 This trace illustrates a kind of risk-aversion: under greedy decoding, the model repeatedly takes easy cyclic actions (e.g. repeating the same fact about how a two-letter word is formed), instead of progress-making actions, such as exploring the structural insights needed to solve the problem. In further analysis, we take the model prefix immediately before the final loop and make the teacher model (QwQ-32B) continue under greedy decoding. The teacher does not get stuck in a loop and makes substantial progress. Moreover, when sampling high-temperature continuations from the same prefix, teacher generations are far more likely to contain key progress-making insights than student generations. Overall, the student model demonstrates risk-aversion under greedy decoding, preferring easy cyclic actions even when the teacher can make progress. We provide a detailed quantitative analysis of this prefix, including probability mass on progress-making actions for both student and teacher, in Appendix B.1.
32. u32 Example 2.
33. u33 We observe a different looping pattern on AIME 2025 Problem I-5. Here the task is to count the $8!$ permutations of digits $\{1,\dots,8\}$ that form an eight-digit integer divisible by $22$ . The model correctly reduces divisibility by $22$ into divisibility by $2$ and $11$ and notes the last digit must be even. However, when applying the $11$ -divisibility condition, it makes a mistake that eventually yields a contradiction. At that point the model restarts the analysis, but makes the same mistake, reaches the same contradiction again, and repeats this series of steps several times. Unlike risk-aversion loops, where the model is averse to making forward progress, here it does move forward but repeatedly makes the same mistake, leading to a loop.
34. u34 The analysis in the last subsection pointed to errors in learning as a key cause of looping. In the above examples, we saw qualitative looping patterns such as risk aversion and repeatedly making the same mistake. How do errors in learning actually lead to these patterns? Understanding this directly is challenging: it would require characterizing the teacher’s data distribution, which can be very complex for real reasoning models, and then analyzing how training changes this distribution in the student, which adds further complexity. To make progress, we therefore consider a synthetic graph reasoning task and use it to demonstrate looping mechanisms in a controlled setting. In particular, Section 3 shows how errors in learning can produce risk-aversion similar to Example 1, while Section 4 shows looping due to temporally correlated errors similar to Example 2.
35. u35 3 Risk-aversion due to Hardness of learning
36. u36 We first explain how hardness of learning can lead to risk aversion and looping, and then describe the graph reasoning task, which we use to instantiate this looping mechanism. It is useful to keep in mind the distillation scenario, where a student model is trained on traces generated by a teacher (we later discuss how these ideas extend beyond distillation).
37. u37 As an example, suppose that at some step in the reasoning chain, the teacher’s data distribution has support over two actions: a progress-making action (e.g., the next logical proof step) and a cyclic action (e.g., backtracking or repeating a fact). Assume the progress-making action is hard for the student to learn, while the cyclic action is easy. We show that even if the teacher distribution assigns high probability to the progress-making action, the student can still place relatively more mass on the cyclic action. Thus, while greedy decoding under the teacher distribution would lead to progress, greedy decoding with the student tends to pick the cyclic action repeatedly and get stuck.
38. u38 The mechanism has two components: (i) a hard-to-learn action co-occurring with an easy action, which increases the easy action’s relative probability and leads greedy decoding to prefer it, and (ii) the easy action being cyclic, which, together with its relatively high probability, causes the model to repeatedly select it. We formalize the hardness of an action below and defer the formalization of cyclic actions to the next subsection, after we introduce the graph task.
39. u39 Formalizing hardness. We say an action is hard if the model cannot distinguish it from $n$ other actions (e.g., the correct next proof step is confusable with $n$ alternatives). A larger $n$ corresponds to a harder action. In this case, even if the training distribution assigns high probability to the hard action, the model, trained to maximize log-likelihood, diffuses that mass across the indistinguishable options. The easy action then ends up with relatively higher mass. One way to formalize this is as follows:
40. u40 Proposition 3.1.
41. u41 Consider the following task: there exist $n$ sets of contexts $C_{1},\dots,C_{n}$ which are equi-likely under the training distribution. And there are $n$ distinct “hard” actions $a_{1},\dots,a_{n}$ , and an “easy” action $a_{0}$ . For every context $c_{i}\in C_{i}$ , the training distribution picks action $a_{i}$ with probability $(1-p)$ and $a_{0}$ with probability $p$ . Now consider a learner that cannot distinguish between the $n$ hard actions or, in other words, it is constrained to ignore the context when deciding on the best action. Then the maximum log-likelihood solution for such a learner assigns probability $p$ to the easy action $a_{0}$ and probability $(1-p)/n$ to the hard indistinguishable actions $a_{i},\forall i\in\{1,\dots,n\}$ .
42. u42 We provide the proof in Appendix C, and also discuss there how the proposition applies to language models. To appreciate the implications, note that as the action becomes harder (i.e., as $n$ increases), the probability assigned to it decreases as $(1-p)/n$ , while the probability of the easy action remains $p$ . Thus, for sufficiently large $n$ , greedy decoding picks the easy action.
43. u43 More generally, whenever the student cannot reliably distinguish an action from several plausible alternatives, maximum-likelihood training encourages it to hedge by spreading probability mass across these candidates. Proposition 3.1 captures this effect in an extreme case where the hard actions are completely indistinguishable, but the same qualitative behavior can arise whenever the student’s representation collapses many distinct contexts, forcing the model to diffuse probability mass across the actions that are plausible in those contexts. This collapse can arise for several reasons, including limited capacity of the student model or optimization difficulties during training.
44. u44 3.1 Demonstration with graph reasoning
45. u45 We demonstrate the looping mechanism discussed above in a synthetic graph reasoning task, which is a convenient testbed for studying looping: it lets us precisely define notions such as progress-making and cyclic actions, as we elaborate later in this section.
46. u46 Figure 2: Illustration of star graph. Figure from Bachmann and Nagarajan (2024).
47. u47 Figure 2: Illustration of star graph. Figure from Bachmann and Nagarajan (2024). (photo)

    

48. u48 The star graph. Our graph reasoning task builds on the hardness result of Bachmann and Nagarajan (2024), who train Transformers to find paths in a star graph. A star graph $G(n,\ell)$ is a directed graph with a root $r$ and $n$ simple “spokes,” each a path of length $\ell-1$ ending at a distinct leaf (Figure 2). Each training example is a sequence containing the edge list, a start node (the root $r$ ), a goal node $g$ (a leaf chosen uniformly at random), and a path from $r$ to $g$ . Distinct instances are formed by randomly permuting node labels. Bachmann and Nagarajan (2024) show that Transformers trained with next-token prediction fail to learn the correct path.
49. u49 Here, aside from the root, all nodes have a single outgoing edge. Thus, for path-finding, learning the first edge on the path is harder than learning the remaining edges. Bachmann and Nagarajan (2024) show that models learn the later edges early in training; once those are learned, the first edge becomes the bottleneck. As a result, the learned solution places roughly $1/n$ probability on each outgoing edge from the root while predicting subsequent edges correctly, yielding chance accuracy (see Hu et al. (2025) for a proof of hardness). Note that here the source of hardness is not model capacity (the models have enough capacity to represent the right solution), but the inability of optimization.
50. u50 Figure 3: Hard-to-learn actions induce low-temperature loops. We train small Transformers on random-walk traces on star graphs (hard) and (easier). In the training distribution, the progress-making action is taken with probability and the reset action with ; dashed curves use an exploration variant at the root ( correct edge, other children, reset). The looping count is the average number of transitions per trace; a perfect learner would have looping count at . On , looping and response length are large at low temperature and much smaller at higher temperatures; without exploration, accuracy is below chance at and near chance at the positive temperatures tested, while with exploration it is near at and substantially higher at higher temperatures. On the easier instances, looping remains close to zero and accuracy is near-perfect across temperatures. Overall, a hard progress decision at the root, paired with an easy cyclic reset action, drives low-temperature loops, and making the problem easier or increasing temperature reduces the severe looping on the harder task.
51. u51 The graph reasoning task. Our graph reasoning task is obtained by making two modifications to the above setup.
52. u52 First: we introduce an explicit start node $s$ with a single outgoing edge to the root $r$ . The rest of the graph remains as in the standard star graph, and the goal $g$ is still a leaf. This modification is not crucial for our results, but it helps illustrate the mechanism better (as discussed below).
53. u53 Second (more crucial): we train on a random-walk trace rather than a single path. The walk starts at $s$ . At any node, it moves one step toward $g$ with probability $1-p$ and resets to $s$ with probability $p$ . For simplicity, this reset applies at every node, including $s$ . When $p=0$ , this reduces to the original single-path setting (with the added start node). We use $p=0.3$ .
54. u54 We abuse notation and use $G(n,\ell)$ to denote this modified star-graph where $n$ paths of length $\ell-1$ emanate from the root, and there exists a separate start node with a single outgoing edge to the root. Finally, in some experiments, we also add a small exploration probability, described later. An example training instance is shown below:
55. u55 Each instance is a sequence containing an edge list, a start node, a goal node, followed by a random walk from start to goal. Each node is a separate token, and $\{\texttt{|},\texttt{/},\texttt{=}\}$ are separator tokens. As in the original star-graph setting, distinct instances are formed by randomly permuting node labels.
56. u56 The notions of actions, cyclic actions, and progress-making actions can be precisely stated in this graph reasoning task. An action is simply the choice of the next node to visit from the current node. A cyclic action is an action that takes the walk to an already visited node, and a progress-making action is an action that moves the walk closer to the goal node. In the above task, the teacher distribution takes the progress-making action of moving forward to the next node toward the goal with probability $0.7$ and takes a cyclic action (reset to the start) with probability $0.3$ .
57. u57 For a language model, an action can be viewed as a short chunk of tokens implementing a logical step. Thus if one views the source-to-goal path in the original star-graph setting as a model for the chain of thought of earlier language models that moves toward the goal in a step-by-step manner, then the random-walk variant can be seen as a model for reasoning language models that explore multiple strategies, backtrack, and restart. Our aim, however, is not to capture the full complexity of an actual reasoning model, but to isolate the phenomenon in the simplest setting possible.
58. u58 Intuition via a Markovian student. The random walk defined above only visits nodes along the path from the start to the goal. At each visited node, the training distribution places probability $0.7$ on the progress-making action and $0.3$ on the cyclic action (reset to the start). A perfect student that exactly recovers this distribution would, under greedy decoding, always take the progress-making action and reach the goal without looping. However, hardness at the root breaks this behavior and induces looping.
59. u59 The reset action is easy to learn: it only requires reading the start node from context and jumping to it. The progress-making action is also easy at all nodes except the root, where the student must choose among $n$ outgoing edges. If it cannot distinguish these $n$ options, we expect it to spread the $0.7$ mass on the progress-making action uniformly, assigning $0.7/n$ to each outgoing edge, while still assigning $0.3$ to the easy reset action.
60. u60 Now suppose that the student is Markovian, so that its next-action distribution depends only on the current node and not on the full history of the walk. Under greedy decoding, the distributions above imply that the walk will loop between the start node and the root: from the start, the student moves to the root, and from the root, whenever $0.3>0.7/n$ , it resets back to the start. Note that the student does not get trapped at the start node: both “move to the root” and “stay/reset” are easy to learn there and are assigned probabilities close to $0.7$ and $0.3$ . Loops arise when a hard action at the root coexists with an easy cyclic reset, and we introduce the explicit start node to make this contrast clear.
61. u61 3.2 Observations with Transformers
62. u62 The discussion above assumes a Markovian student, but Transformers can condition on the full prefix. We therefore train a decoder-only Transformer with $\approx 85$ M parameters from scratch and ask whether similar looping emerges (see Appendix D for training details).
63. u63 Low-temperature looping. On $G(5,5)$ , we do observe looping similar to the Markovian student discussed above. Figure 3 reports accuracy, response length, and the looping count (the number of $\text{root}\!\to\!\text{start}$ transitions per test instance) as a function of temperature. Both looping count and response length are large at low temperature and much smaller at higher temperatures; at $T{=}0$ , the looping count is roughly half the response length since traces often alternate $\text{start}\!\to\!\text{root}\!\to\!\text{start}\!\to\!\text{root}\ldots$ . Some traces eventually escape to an outgoing edge from the root, but many fail to reach the goal within the token budget; without exploration, greedy accuracy is only $5.48\%$ , below the $20\%$ chance baseline.
64. u64 Accuracy improves with temperature. While both looping count and response length are much smaller at higher temperatures, accuracy without exploration reaches only approximately chance at the positive temperatures tested. This contrasts with many reasoning models, where accuracy can improve substantially beyond chance as temperature increases. The difference stems from the training distribution: the walk never explores off-path routes, so the model learns to revisit the same path after each reset. Since it cannot reliably pick the correct path at the root, accuracy remains near chance even when loops shorten.
65. u65 To test exploration, we modify the walk at the root: on each root visit, the walk resets with probability $0.3$ , takes the correct outgoing edge with probability $0.5$ , and takes one of the other edges uniformly with probability $0.2$ ; all other internal transitions remain the same (and reaching a non-goal leaf triggers a reset with probability $1$ ). Training on these traces makes accuracy substantially higher at higher temperatures than at $T{=}0$ on $G(5,5)$ (w/exp in Figure 3). As in the non-exploration setting, looping count and response length are much smaller at higher temperatures. As a side note, at $T{=}0$ the model often bounces between start and root and eventually terminates rather than committing to a path (so accuracy can be near $0$ ). At very high temperatures, accuracy degrades slightly because the model occasionally samples tokens corresponding to invalid edges in the graph, and more walks run out of the token budget without reaching the goal. Also, response lengths are somewhat higher than in the non-exploration setting overall, likely because exploration produces longer training traces and the model mirrors this at test time.
66. u66 Less looping on easier problems. We also evaluate $G(5,3)$ (with and without exploration). Since the paths from the root to the leaves are shorter, the progress-making action at the root is less hard to learn. Indeed, the learned probabilities place higher mass on the correct outgoing edge from the root than on the others. As a result, looping is essentially absent on these easier instances: greedy decoding produces no looping without exploration and almost none with exploration, and looping remains close to zero across temperatures. Accuracy is near perfect across temperatures. Response lengths also remain short; their upward trend at higher temperatures reflects occasional sampling of the reset action. This mirrors our earlier finding from Section 2: models loop less on easier problems.
67. u67 Reset probabilities. For the experiments in this section, looping arises because the model spreads progress-making mass across the root children, making the easy reset action more likely. We also observe another force that can contribute to this risk-aversion behavior in some settings: overestimation of the easy reset action itself. This occurs, for example, early in training or when model capacity is limited. We examine this phenomenon in more detail in Appendix H.
68. u68 3.3 Other implications
69. u69 Sources of hardness. In the illustration above, hardness comes from the inability of the optimization to find the right solution. More generally, hardness can arise due to other factors too such as limited model capacity or limited training compute (under-training). The hardness due to model capacity is a plausible explanation for why smaller models loop more within a family trained on the same data. In Appendix G, we demonstrate hardness due to model capacity causing risk-aversion looping in our graph reasoning setup.
70. u70 Temperature as a mitigation. We began by asking whether temperature is a stopgap or a holistic fix. The experiments above suggest an answer. Increasing temperature reduces looping, but it does not remove the underlying learning errors: the model still assigns too little probability to the correct progress-making action. One simple way to see this is response length. On $G(5,5)$ with exploration, at temperature $1$ the learned model has average response length $67.4$ , versus $24.6$ for a perfect learner (a $\approx 2.74\times$ blow-up). Even when looping is reduced, the model still takes many extra steps because it has not learned the right action probabilities. We saw a similar trend with open reasoning models, where smaller models produced longer chains than larger models or their teachers. The same issue also shows up in accuracy: with the same token budget, the learned model reaches only about $72.25\%$ at temperature $1$ , while the teacher achieves near-perfect accuracy. Thus, temperature can reduce looping at the surface level, but the underlying cause remains. We discuss more holistic training-time interventions in Section 6.
71. u71 Finally, Appendix D discusses further implications of these experiments, including comparisons between reasoning vs. instruct models and distilled vs. RL-trained models.
72. u72 4 A Bias for Temporally Correlated Errors
73. u73 In this section, we show that Transformers have an inductive bias for looping, causing its errors to correlate over time, similar to Example 2. The mechanism is easiest to describe in the graph reasoning setting, so we directly jump in.
74. u74 Setup. We use the same star graph and training setup as in Section 3, but change the teacher walk transitions to remove hardness at the root and remove per-step resets. The walk starts at the start node, moves to the root, chooses a root child uniformly at random, and then follows the unique outgoing edges down that spoke to a leaf. If the leaf is the goal, the walk stops; otherwise it resets to the start and repeats. Training and evaluation match the previous section.
75. u75 Figure 4: Temporally correlated errors induce low-temperature loops. We train on star-graph random-walk traces that choose a child at the root, take the unique outgoing edge at other internal nodes, and reset to the start at non-goal leaves. The looping count is the maximum number of visits to any root child in a trace, averaged over test instances. With no margin (), the training walk chooses root children uniformly, but temporally correlated errors cause the model to repeatedly select the same children under greedy decoding. Greedy accuracy is ; at positive temperatures, looping count and response length fall sharply and accuracy rises to approximately . With margin , the training walk favors children that have not yet been visited. This eliminates greedy looping: the looping count is , so no child is revisited, and accuracy is near-perfect throughout.
76. u76 4.1 Observations and implications
77. u77 Low temperature loops. It helps to first consider a perfect learner. On $G(n,\ell)$ , such a model would learn: probability $1$ to the unique outgoing edge at non-root internal nodes; probability $1$ to reset at non-goal leaves; and probability $1/n$ to each child at the root. In practice, the trained model deviates slightly at the root. Instead of learning an exact $1/n$ split, it makes small errors (e.g., $0.2\pm 0.05$ for $n{=}5$ ). More importantly, these errors are correlated across time: the root children that are slightly preferred early in the trace tend to remain preferred on later visits to the root. Under greedy decoding or low temperature, the model therefore keeps revisiting the same one or two paths, producing loops. This is qualitatively similar to Example 2, where the model repeatedly restarts after a contradiction but re-enters the same mistaken reasoning path.
78. u78 We quantify this with a looping count: for each generated trace, we record how many times each root child is visited and report the maximum over children; we then average this value over $4000$ test instances. Under greedy decoding, the average looping count is $11.84$ and accuracy is $44.43\%$ (Fig. 4, margin $=0$ ). At the positive temperatures tested, the looping count falls to about $2.1$ and accuracy reaches approximately $99\%$ . The average response length also falls sharply.
79. u79 Temporal correlation. In addition, we directly probe temporal correlation in the model’s predictions at the root. Under greedy decoding, we record the two highest-probability next-token predictions at each visit to the root, including terminal decisions, and compare every pair of consecutive root visits within a trace. We then compute (i) the fraction of such consecutive-visit pairs where the highest-probability prediction is the same at both visits, and (ii) the analogous fraction for the second-highest prediction. These fractions are $80.46\%$ and $60.29\%$ , respectively, indicating substantial persistence across visits. Thus the Transformer’s errors at the root are temporally correlated, inducing an inductive bias for looping.
80. u80 One might attribute top-child persistence to self-conditioning: after selecting a child, the model observes that choice in its generated history and may prefer it again at the next root visit. However, under greedy decoding the second-highest-probability prediction is never selected, yet its identity is also strongly correlated across visits. Thus, the effect is not merely reinforcement of the sampled action, but reflects persistent structure in the model’s learned conditional distributions. At the same time, conditioning can play a role in certain circumstances, as we discuss in Section 5.
81. u81 A variant with margins. The training walk above samples root children uniformly at every visit. What if the training distribution itself discourages revisiting already-seen children? We study a margin variant: the first time the root is visited, a child is sampled uniformly; on later visits, each child that has ever been visited at the root has its sampling probability reduced by a fixed margin $m$ , and the removed mass is redistributed uniformly over the as-yet-unvisited children. This reduction is applied once per child (its probability does not decrease further with repeated visits). With $m{=}0$ we recover the random walk considered above.
82. u82 Training with margin $m{=}0.1$ removes greedy looping in this experiment: accuracy is $100\%$ and the looping count is $1$ , so no root child is revisited (Fig. 4). The consecutive-visit agreement for the top and second predictions drops to $0\%$ and $23.98\%$ , respectively. A perfect student under the margin-modified training distribution would also never revisit an already-visited child under greedy decoding. At positive temperatures, sampling reintroduces some repeated visits, while accuracy remains near perfect.
83. u83 These results suggest a training-time mitigation: teacher or distillation traces can be biased toward exploring new actions at repeated decision points. In this setting, the intervention is sufficient to eliminate greedy repetition. Although this intervention modifies the training distribution, inference-time repetition penalties pursue a related goal by directly downweighting previously generated tokens (Keskar et al., 2019).
84. u84 Comparing the mechanisms. Both mechanisms we discuss stem from errors in learning, but they differ in nature. The first arises from hardness of learning: probability mass on a hard progress-making action is spread across many indistinguishable alternatives. This gap can be large relative to the training distribution and does not rely on Transformer-specific inductive biases—it can appear for any maximum-likelihood learner when the correct action is indistinguishable from many others. The second relies on an inductive bias toward temporally correlated errors: at repeated decision points, the learned probabilities are slightly skewed toward a few actions. Here the deviations are small, yet sufficient for low-temperature looping.
85. u85 Temperature mitigates looping in both settings by promoting exploration, but it does not fix the underlying learning errors. Its effectiveness therefore depends on the magnitude of those errors. In the hardness setting (Section 3), the errors are large, and at temperature $1$ the learned model’s response length remains $\approx 2.74\times$ that of the teacher. In the present setting, the errors are smaller: the model’s distribution at the root is already close to the ideal with only mild skew. On $G(5,5)$ with $\text{margin}=0$ at temperature $1$ , the learned model and teacher have nearly identical response lengths ( $29.56$ and $29.92$ , respectively). This suggests that temperature can be an effective fix when learning errors are small, while larger errors may require more direct training-time interventions.
86. u86 These mechanisms can also be viewed as two atomic “forces” driving looping. In Appendix E, we present a variant where they act together.
87. u87 5 A Catalyst for Looping
88. u88 In real model traces, we also see another interesting phenomenon. At the beginning of a loop, the model’s probability distribution over next tokens looks relatively normal, but as it repeats the same text, it becomes increasingly confident (Figure 16). This acts as a catalyst for any looping mechanism, making it harder for the model to escape once it has been looping for a while. This behavior has also been observed in prior work for earlier generations of language models (Holtzman et al., 2020; Chiang and Chen, 2021; Xu et al., 2022), and can also be viewed as an instance of error accumulation or self-conditioning (Sinha et al., 2026).
89. u89 Using our synthetic graph reasoning setup, we show that a tiny fraction of looping traces during training can produce such a buildup. Specifically, in the risk-aversion setup (Sec. 3), we modify the training distribution: with probability $0.001$ the example is a trace that deterministically loops between the start and root, otherwise sampled as before. This probability is small enough not to noticeably change the learned distribution early in a generation, but once the model has already looped, conditioning on this history makes continuing the loop much more likely (Fig. 5). Analogous results for the other mechanism are in App. F.
90. u90 Figure 5: Tiny bias drives reset probability toward 1. In the risk-aversion setting on with exploration, we record at every root visit the model’s probability of taking the reset-to-start action, across 4000 greedy-decoded traces (thin lines; thick line is the average). When training includes a tiny fraction () of deterministic looping traces (orange), the reset probability rises toward 1 as the loop persists, making escape increasingly unlikely. Without this bias (blue), the probability stays roughly constant.
91. u91 Figure 5: Tiny bias drives reset probability toward 1. In the risk-aversion setting on with exploration, we record at every root visit the model’s probability of taking the reset-to-start action, acro (photo)

    

92. u92 In reasoning models, the underlying bias can arise from various sources. For example, during pretraining the model may see a few sequences where some text is repeated many times. As a result, during inference, once the model has already repeated itself, the conditional probability of continuing the repetition can become larger than its unconditional value.
93. u93 6 Discussion
94. u94 We began with two questions: why do reasoning models loop, and is temperature a holistic fix or a stopgap? Our evaluation of open reasoning models points to errors in learning as a key cause. Using a synthetic graph reasoning task, we identified two mechanisms by which learning errors cause looping: risk aversion caused by hardness of learning, and an inductive bias toward temporally correlated errors.
95. u95 On the role of randomness, temperature is reasonably effective at reducing looping, but does not fix the underlying learning errors. In the hardness setting, the model still assigns too little probability to progress-making actions, so response lengths at high temperature remain longer than necessary. In the correlated-errors setting, high-temperature response lengths closely match those of the teacher. Thus temperature is a reasonable fix when errors are small, but a stopgap when they are large.
96. u96 6.1 Limitations and Future Work
97. u97 Synthetic-to-real mapping. Our synthetic graph reasoning task is intentionally simple, designed to isolate mechanisms behind looping. The examples discussed in Section 2.1 show glimpses of these mechanisms in real reasoning model traces. For example, in Example 1 we observe that at the looping point, the teacher model places substantially more mass on the progress-making action than the student, consistent with the risk-aversion mechanism (see Appendix B.1 for the full quantitative analysis). While these examples are consistent with the proposed mechanisms, they admittedly do not capture all the details. For instance, beyond showing the teacher puts more mass on progress-making actions, we do not precisely characterize all possible actions and the mass on each. Such precision is challenging for two reasons. First, actions in real models live in semantic space, making it hard to cluster continuations into discrete actions with reliable probability estimates. Second, multiple forces act simultaneously and are hard to disentangle. Appendices E and F give examples of such interactions. These challenges motivate our use of synthetic tasks; developing a tighter mapping between real models and our setup is an important future direction.
98. u98 Other forces. Further, given the complexity of real models, other forces likely also contribute to looping. One example is error accumulation: errors made during generation can compound over time. Our proposed mechanisms show that looping can arise even without such accumulation, but it can act as a catalyst—the self-conditioning effect of Section 5 is one instance. We also observe more general variants in real model traces (Appendix B). Pinning down such forces and their interactions is an important future direction.
99. u99 Beyond temperature. For mitigation, a natural next step is to seek holistic fixes that directly reduce learning errors. Some errors are inevitable due to capacity, but it is unlikely that all of today’s errors are necessary. One concrete direction is to modify how we distill teacher traces, making them easier for the student to learn. For example, a targeted data augmentation approach could identify points in teacher traces that the student finds hard (e.g., high-loss positions) and augment them with brief hints.
100. u100 Correction to the previous version
101. u101 We thank Taehyun Kim for bringing a padding issue in our synthetic experiments to our attention. The padding token ID was not passed to the model, leaving padding tokens unmasked during training. We have retrained the affected models and updated the corresponding figures and discussion. The correction changes some observations, including the overestimation of reset probabilities and the strength of the looping catalyst in the correlated-errors regime, but the main conclusions from the synthetic experiments remain unchanged. Experiments on pretrained language models were unaffected.
102. u102 Acknowledgements
103. u103 We would like to thank the anonymous reviewers for their constructive feedback that helped improve our paper. Additionally, we are deeply grateful to Vaishnavh Nagarajan for his sharp and detailed comments on our work. Part of this work was conducted while Charilaos Pipis was an intern at Microsoft Research.
104. u104 Appendix A Additional experiments and details for Section 2
105. u105 Evaluation details. The reasoning models we test are as follows. Qwen: DeepSeek-R1 Distilled Qwen 1.5B, 7B, 32B [DeepSeek-AI et al., 2025]; OpenThinker3: OpenThinker3 1.5B, 7B [Guha et al., 2025] and QwQ-32B [Team, 2025] (the teacher for OpenThinker-3); Phi-4: Phi-4-reasoning, Phi-4-reasoning-plus [Abdin et al., 2025]; Llama: DeepSeek-R1 Distilled Llama 8B [DeepSeek-AI et al., 2025]. While we mostly focus on reasoning models, we also test a few instruction tuned variants (non-reasoning models). These include Qwen2.5-Math-1.5B-Instruct [Yang et al., 2024b], Qwen2.5-1.5B-Instruct [Yang et al., 2024a], Phi-4 [Abdin et al., 2024], Llama-3.1 8B Instruct [Grattafiori et al., 2024].
106. u106 We consider a text response to contain looping if it contains any $n$ -gram at least $k$ times. We choose $n=30$ and $k=20$ for all reasoning models. Since instruct models produce shorter responses, we relax the looping definition to have $k=10$ for them.
107. u107 We used the Eureka ML Insights Framework [Balachandran et al., 2024, Balachandran et al., 2025] to conduct the evaluations on all models. For the inference parameters, we kept the default top_k and top_p for each model, taken from its HuggingFace repository. We do not use any repetition penalty as we focus on the role of temperature. We ran all reasoning models with a 30K max_tokens budget and all instruct models, we set it based on the max allowed number of tokens for each model (14k for Phi-4, 3k for Qwen2.5-Math-1.5B-Instruct, 30k for Llama-3.1-8B-Instruct and Qwen2.5-1.5B-Instruct).
108. u108 Figure 6: Looping with greedy decoding. Bars show the looping fractions at temperature , averaged over AIME 2024 and 2025. All reasoning models exhibit looping, and within each family larger models loop less (e.g., Qwen 1.5B 7B 32B; OpenThinker3 1.5B 7B QwQ-32B). Distilled students can loop significantly more than their teacher (OpenThinker3 vs. QwQ-32B). Reasoning models can also loop heavily even when their instruction-tuned counterparts barely loop (e.g., Qwen2.5 and Phi-4 families). Finally, RL post-training has limited effect on looping in the Phi-4 family (Reasoning vs. Reasoning-Plus).
109. u109 A.1 Other Observations
110. u110 Reasoning models can loop even when their instruct counterparts barely loop. For the Qwen and Phi-4 model families, we see that the base instruction-tuned models loop very little, while their reasoning counterparts loop a lot. It is not always the case that the instruct models do not loop, as we see with Llama-3.1-8B-Instruct and Qwen2.5-1.5B-Instruct, but even there we observe an increase in the looping percentage for the corresponding reasoning models. The amount a model loops is likely a function of cyclic actions like restatement or backtracking in the training data. This view is supported by past work [Li et al., 2023]. We discuss this further in Appendix D.
111. u111 Effect of RL training. Phi-4 reasoning is one model family where we have access to both a distilled model (Phi-4-Reasoning) and a version that has been lightly tuned with RL (Phi-4-Reasoning-Plus). In this setting, we observe that the looping counts remain roughly the same across the two models.
112. u112 (a) Accuracy
113. u113 (b) Looping Percentage
114. u114 (c) Average CoT Length
115. u115 Figure 7: Openthinker metrics as a function of temperature.
116. u116 (a) Accuracy
117. u117 (b) Looping Percentage
118. u118 (c) Average CoT Length
119. u119 Figure 8: Qwen metrics as a function of temperature.
120. u120 (a) Accuracy
121. u121 (b) Looping Percentage
122. u122 (c) Average CoT Length
123. u123 Figure 9: Phi-4 metrics as a function of temperature
124. u124 (a) Accuracy
125. u125 (b) Looping Percentage
126. u126 (c) Average CoT Length
127. u127 Figure 10: Llama metrics as a function of temperature.
128. u128 A.2 Hard vs Eay Problems
129. u129 Figure 11: Looping with greedy decoding for a hard-easy split of AIME problems. For each model, we use accuracies at temperature 0.8 to create a split of the 5 easiest problems and 5 hardest problems. Then we evaluate the looping percentage at temperature 0 separately on these two sets of problems. The bars on the left show the looping percentage on hard problems and bars on the right show the percentage on easy problems. We can see that for all model families except Phi-4, models looping significantly more in hard problems than in easy ones.
130. u130 Phi-4 reasoning models on hard and easy problems. The Phi-4 reasoning family of models was the only exception to the observation that hard problems induce more looping than easy ones. As we can see in Figure 11, both phi-4-reasoning and phi-4-reasoning-plus consistently exhibit more looping in easier problems than in hard ones. By manually inspecting its responses, we realized that it very often demonstrates a peculiar form of looping; it finds the answer during the CoT, then proceeds to present it, and then it gets stuck indefinitely repeating things like “We’ll produce answer in plain text.” This is one of the main ways in which it loops, which means that for easy problems, it will reach the solution more frequently and, thus, get stuck in this situation more often than in harder problems. Note also that another key way in which phi-4-reasoning models differ from the other models we tested is that it has been fine-tuned on OpenAI o3-mini data [Abdin et al., 2025], as opposed to DeepSeek-R1 or Qwen. Nevertheless, this is only a preliminary attempt at explaining this discrepancy and a more thorough study of the exact underlying factors would make a great direction for future research.
131. u131 A.3 Ablations on the definition of looping
132. u132 In this section we report two ablations of the $(n,k)$ definition of looping. Table 1 varies one parameter around our default $(30,20)$ setting; looping percentages remain close and the relative ordering of the models is unchanged. Table 2 studies the less strict $10$ -gram setting. Our default is already strict: a response is considered looping only if it contains a $30$ -gram at least $20$ times.
133. u133 Table 1: Looping percentages at temperature 0 for different (n, k) combinations
134. u134 To further test whether the $(n,k)$ definition needs to be that strict, Table 2 reports looping percentages at temperature 0 for the $10$ -gram setting in the Qwen and OpenThinker families, with different thresholds $k$ .
135. u135 Table 2: Looping percentages at temperature 0 for different (n, k) combinations
136. u136 Even though looping numbers remain very close even with $n=10,k=40$ , we obseve that with $n=10$ and $k=10$ or $k=20$ the absolute looping increases significantly. Manual inspection of responses flagged by $(10,10)$ but not $(30,20)$ showed nearly all were false positives (e.g., benign reuse of expression $(33/(2\sqrt{14}))y$ ), motivating our use of a somewhat strict definition for looping.
137. u137 A.4 Looping percentages on GPQA
138. u138 To investigate whether our observations from the AIME dataset transfer to different domains, we ran an ablation with Openthinker3 and Qwen2.5 models on the GPQA Diamond dataset [Rein et al., 2024], which contains a set of $198$ highly challenging questions from biology, physics, and chemistry. For each (model, problem, temperature) tuple we generate $4$ responses , for a total of $\approx 800$ responses at each (model, temperature). We evaluate on temperatures 0.0, 0.4, 0.8. We see that our main observations also hold here: lower-capacity models loop more and student models loop significantly more than teacher.
139. u139 Figure 12: Looping with greedy decoding on GPQA.
140. u140 (a) Accuracy
141. u141 (b) Looping Percentage
142. u142 (c) Average CoT Length
143. u143 Figure 13: Qwen metrics on GPQA as a function of temperature.
144. u144 (a) Accuracy
145. u145 (b) Looping Percentage
146. u146 (c) Average CoT Length
147. u147 Figure 14: Openthinker metrics on GPQA as a function of temperature.
148. u148 Appendix B Example Traces of Openthinker-3 1.5B on AIME 2025 problems
149. u149 B.1 AIME 2025 Problem I-7 (Example 1)
150. u150 The model is prompted to solve Problem I-7 from AIME 2025, with greedy decoding:
151. u151 The model begins sensibly. It paraphrases the problem and correctly derives the total number of ways to pair up the letters.
152. u152 Next, it tries to reason about how the two-letter words are formed and loops (semantically) for a bit, repeatedly going over how a word is arranged in alphabetical order.
153. u153 After this, it briefly comes out of the loop but then slips into another one, now restating the goal of the problem without adding new structure.
154. u154 Finally, it starts to argue about how pairs including G can be arranged, but again falls into a loop, repeating that words are formed by sorting the pairs, and never gets out of it.
155. u155 This trace illustrates a kind of risk-aversion: the model repeatedly takes easy cyclic actions (for example, repeating the same fact about how a two-letter word is formed, or restating the problem), instead of progress-making actions, such as exploring the structural insights needed to solve the problem. An example of structural insight is that for the last word to contain G, no pair should be formed with both letters coming from H-L.
156. u156 Further analysis. To probe this behavior further, we truncate the trace at the point where the final looping begins (right after it says “Wait, so actually, the word is always the two letters in alphabetical order, so the word is the pair sorted.”) and let the teacher model (QwQ-32B) continue from that prefix under greedy decoding. To account for non-determinism, we generate 5 continuations. We get 2 distinct traces: one reaches the correct answer, while the other misses a case and therefore undercounts. However, both traces avoid looping and make significant progress, capturing most of the structural insights. We also regenerate 5 completions from the student model from the same prefix and all of them get stuck in a loop.
157. u157 To understand how much probability mass the student and teacher put on progress-making actions, we also sample $50$ continuations of length $2000$ tokens from each model at a high temperature. We then feed these continuations to GPT-5, provide it with a list of key structural insights, and ask whether each continuation contains at least one of them. At temperature $0.7$ , GPT-5 marks $1/50$ student continuations and $34/50$ teacher continuations as containing an insight (at temperature $1$ , these numbers are $8/50$ and $32/50$ ). Thus, the student puts much less mass on progress-making actions than the teacher.
158. u158 Overall, in this example, we see that under greedy decoding the student demonstrates risk-aversion, preferring easy cyclic actions even when the teacher can make progress, and the teacher assigns substantially more probability mass to progress-making actions than the student.
159. u159 B.2 AIME 2025 Problem I-5 (Example 2)
160. u160 Here, we prompt Openthinker-3 1.5B to solve Problem I-5 from AIME 2025, with greedy decoding:
161. u161 The model starts well. It recalls that being divisible by $22$ means being divisible by both $2$ and $11$ , notes that the last digit must be even, and sets up the standard $11$ -divisibility condition in terms of the sums of digits in odd and even positions.
162. u162 From here it makes correct algebraic progress on the $11$ -rule: it rewrites the condition in terms of $S_{\text{odd}}$ alone, reduces modulo $11$ , and finds that $S_{\text{odd}}\equiv 7\pmod{11}$ . It then uses the fact that $S_{\text{odd}}$ is the sum of four distinct digits from $\{1,\dots,8\}$ and correctly narrows down to the only feasible value.
163. u163 At this point the model makes its first key mistake. When it fixes the last digit $E$ to be even and looks at the remaining $7$ digits, it continues to insist that the four digits in the odd positions must sum to $18$ and the three digits in the remaining even positions must also sum to $18$ , as if the last digit had disappeared.
164. u164 This leads to a contradiction ( $E$ would have to be $0$ ), and the model notices that something is wrong.
165. u165 However, instead of changing its approach, it falls into a loop: it repeatedly restates the divisibility-by- $11$ condition, re-derives $S_{\text{odd}}\equiv 7\pmod{11}$ , concludes again that $S_{\text{odd}}=18$ , and runs back into the same contradiction with $E$ .
166. u166 This behavior bears similarity to the temporally correlated errors we saw in our graph reasoning setting: once the model makes a high-level mistake, it keeps returning to the same wrong reasoning whenever it tries to restart. In the graph setting, this corresponded to repeatedly picking the same wrong child at the root, leading to the same leaf, and restarting.
167. u167 After several rounds of semantic looping, the model makes a new mistake! It now claims that $S_{\text{odd}}+S_{\text{even}}=36-E$ instead of $36$ , which it had correctly used earlier. Building on this, it mistakenly concludes $2S_{\text{odd}}\equiv 36-E\pmod{11}$ (instead of $2S_{\text{odd}}\equiv 36\pmod{11}$ , which it correctly found earlier). It follows up with a case analysis over $E\in\{2,4,6,8\}$ .
168. u168 From here, the model switches to a more combinatorial counting approach: for each fixed even last digit $E$ , it wants to count the number of $4$ -element subsets of the remaining $7$ digits whose sum is the required $S_{\text{odd}}$ , and then multiply by $4!\cdot 3!$ to account for permutations in the odd and even positions.
169. u169 When it actually tries to carry this out (starting with $E=2$ ), it gets stuck in a second loop. Instead of systematically listing and counting the subsets, it repeatedly writes down the same few candidate triples and partial checks, without moving on to finish the enumeration or to the other values of $E$ .
170. u170 This second loop demonstrates a risk-aversion pattern. By this point, the model has already identified the three valid triples in the $E=2$ case. The natural progress-making step would be to conclude that there are three such triples, convert this into a count of permutations for $E=2$ , and then move on to the next value of $E$ . Instead, it keeps revisiting the same triples again and again, rechecking them rather than advancing the argument.
171. u171 Further analysis (first loop). To probe this behavior further, we truncate the trace at the point where the model first reaches a contradiction (right after it concludes that $E$ would have to be $0$ ). From this prefix, we let the teacher model (QwQ-32B) continue under greedy decoding and generate $5$ continuations to account for non-determinism. In all traces, the teacher immediately identifies the mistake, repairs the reasoning, and eventually reaches the correct answer. We also generate $5$ new continuations from the student (OpenThinker-3 1.5B) from the same prefix. This time we see two distinct student traces: both loop 2-3 more times, repeatedly re-deriving the same wrong condition and contradiction, but eventually notice the mistake. Once they exit this first loop, they start a case-by-case analysis in $E$ and then get stuck again in a loop, similar to the second loop in the above trace.
172. u172 To get a quantitative sense of how much probability mass the two models place on the progress-making action of correctly escaping the contradiction, we sample $50$ continuations of length $1000$ tokens from each model at temperatures $0.7$ and $1.0$ from the same truncated prefix. We then ask GPT-5 to check whether each continuation correctly fixes the mistaken equation and moves on with valid reasoning. At temperature $0.7$ , GPT-5 marks $19/50$ student continuations and $50/50$ teacher continuations as successfully escaping the contradiction; at temperature $1.0$ , these numbers are $21/50$ and $48/50$ , respectively. Thus, the teacher puts much more mass on the correct progress-making action than the student.
173. u173 Overall, this first loop shows glimpses of the temporally correlated errors mechanism from Section 4: once the student falls into a wrong pattern, it tends to revisit the same mistaken step several times. Here, the gap between the teacher and the student is relatively large, for example in terms of how much probability mass they put on the correct progress-making action. In this sense, the behavior is closer in spirit to the hardness+correlated-errors variant in Appendix E, where we show that looping resembling the temporally correlated errors mechanism can arise even when the teacher distribution puts high mass on the correct progress-making action.
174. u174 Second loop. We also probe the second looping point by truncating the trace right when the model gets stuck in the final infinite loop (at the point after it says “ $7+6+4=17,7+5+5=\text{no}$ ”). From this prefix, we again generate $5$ greedy continuations from both the teacher (QwQ-32B) and the student (OpenThinker-3 1.5B). Interestingly, in this case, along with the student, all teacher continuations gets stuck in the same loop.
175. u175 Further, we sample $50$ continuations of length $2000$ tokens from each model at temperatures $0.7$ and $1.0$ , starting from the same truncated prefix. We then ask GPT-5 whether each continuation makes progress, where we define progress as continuing the enumeration and correctly counting the valid tuples for the $E=2$ case. At temperature $0.7$ , GPT-5 marks $10/50$ student continuations and $34/50$ teacher continuations as making progress; at temperature $1.0$ , these numbers are $27/50$ and $42/50$ , respectively. Thus, the teacher still puts more mass on the correct progress-making actions than the student, but the gap is less substantial, especially at temperature 1.
176. u176 This second loop is interesting because here we see risk-aversion–style looping in both the student and the teacher. There are several possible reasons for why a teacher might also loop.
177. u177 First, the teacher itself may carry errors in learning from its own training process. Even if it was trained with reinforcement learning, hardness of learning (as discussed in Section 3.3) can still cause it to loop.
178. u178 Second, note that the teacher did not loop when we started it from the earlier looping point: from that earlier prefix, it was able to identify the mistake and reach the correct answer. In contrast, when we start it from the later point, it is conditioned on a student trace that has already semantically looped several times (repeating the same mistaken reasoning and contradiction). This suggests that error accumulation in the prefix may also play a role: for example, in-context learning from a “confused” history could cause the teacher itself to behave in a more confused, looping way. Understanding the precise mechanisms by which this kind of error accumulation drives loops is an interesting direction for future work. Past work has also noted that looping sequences are often preceded by less natural text, which is consistent with error accumulation playing a role [Chiang and Chen, 2021] .
179. u179 Appendix C Proof of Proposition 3.1 and mapping to language models
180. u180 We restate the proposition below and provide its proof. See 3.1
181. u181 Proof.
182. u182 Let $q_{1},\dots,q_{n}$ be the probabilities that the model assigns to the $n$ actions, and $q_{0}$ be the probability of the easy action. The Cross-Entropy loss for a context $c_{i}\in C_{i}$ with correct action $a_{i}$ is
183. u183 Since all $n$ contexts are equi-probable, the average Cross Entropy over the dataset will be
184. u184 Note that $\sum_{i=0}^{n}q_{i}=1$ and $q_{i}\geq 0$ for all $i$ , by definition. We can further assume that $q_{i}>0$ , otherwise the loss would be infinite. This is a convex minimization problem under linear constraints, so to find the minimizer it suffices to look at the points satisfying the KKT conditions. In particular,
185. u185 for some $\mu\in\mathbb{R}$ . Using the equality constraint, we conclude that $\mu=1$ and $q_{0}=p$ , $q_{i}=(1-p)/n$ for $i=1,\ldots,n$ is the unique minimizer of the problem, as desired. ∎
186. u186 Mapping to language models. In a language model, an action can be viewed as a short chunk of tokens implementing a logical step (e.g., the next step in a proof). For a span $x_{t:t+k-1}$ , the model’s probability of that action given the prefix is $P_{\theta}(x_{t:t+k-1}\mid x_{<t})=\prod_{i=t}^{t+k-1}P_{\theta}(x_{i}\mid x_{<i}),$ so the log-probability of the chunk is the sum of the per-token log-probabilities. Because next-token training maximizes this sum over tokens, it also maximizes the log-probability of any such chunk. Consequently, if a progress-making step is hard (confusable with $n$ alternative chunks), the model spreads its mass across those chunks, reducing the learned probability on the intended chunk by a $1/n$ factor, while an easy cyclic span retains its mass. The proposition therefore applies directly to these chunks.
187. u187 Appendix D Risk-aversion due to Hardness of Learning: Further details and implications
188. u188 D.1 Training Details
189. u189 We train a decoder-only Transformer from scratch, with $12$ layers, $8$ attention heads, and $768$ embedding dimension ( $\approx 85$ M parameters). We use Adam for $100$ k steps with a learning rate of $10^{-4}$ (cosine decay) and batch size $64$ . We use $2$ M training sequences and train with cross-entropy loss for next-token prediction, applying the loss only to the random-walk portion of the sequence. At test time, the model receives new randomly generated instances and, given the edge list, start, and goal, is expected to generate a walk from the start to the goal. We mark a generated walk as accurate if it takes only valid transitions and eventually reaches the goal and stops.
190. u190 D.2 More implications
191. u191 Instruct vs Reasoning Models. Two ingredients are needed in the mechanism above: (i) hard-to-learn actions, along with (ii) easy-to-learn cyclic actions present in the training distribution. The presence of hard actions amplifies the frequency of easy cyclic actions in model generations. However, if cyclic actions are rare in the training traces, extensive looping is less likely. This is a plausible explanation for why many instruction-tuned models loop less than reasoning models as reasoning traces include more cyclic actions such as re-statement and backtracking.
192. u192 RL vs distillation. The mechanism discussed in this section can affect models trained via RL too, and not just distillation. A guiding principle here is that whenever there is a capacity difference between the teacher model and the student model, hardness of learning for the student can amplify looping behavior. One can approximately view the RL training process as sampling multiple trajectories from the model and training on the correct ones. While there is no explicit teacher model for a RL trained model, this can be thought of as training the model on best-of- $k$ version of itself where $k$ is the number of trajectories sampled. In that sense, there is still a gap between the data generator and the learner, which can possibly cause this mechanism.
193. u193 Appendix E Looping via a combination of Hardness + Correlated Errors
194. u194 In Section 3, we saw how hardness of learning can cause looping and in Section 4 we saw how an inductive bias for correlated errors can lead to looping. In this section, we present yet another variant showing how both hardness and correlated errors can act together and cause looping. In particular, we show that Transformers can have temporally correlated errors even when the teacher puts substantial mass on the correct progress-making action, as opposed to the random uniform distribution we considered in Section 4.
195. u195 Setup. We use the same star graph and random walk as in Section 4, with a modification at the root. On $G(10,5)$ , the teacher places probability $0.5$ on the correct child and spreads the remaining $0.5$ uniformly over four of the nine incorrect children, giving probability $0.5/4$ to each. The teacher thus explores only five root children in total. We train the same 12-layer Transformer for $300$ k steps on $4$ M training sequences, using a node-label vocabulary of size $50$ and a total token budget of $250$ . The remaining training settings match the previous synthetic experiments.
196. u196 For our looping metric, as in Section 4, we use the maximum number of visits to any root child, averaged over $4000$ generated traces. Figure 15 shows substantial looping under greedy decoding: the average looping count is $16.89$ , while a perfect learner would directly choose the correct child and have a looping count of $1$ . Increasing temperature reduces looping and response length, and accuracy rises from $15.0\%$ under greedy decoding to $79.8\%$ at temperature $1$ .
197. u197 (a) Accuracy
198. u198 (b) Looping Count
199. u199 (c) Response Length
200. u200 Figure 15: Looping from combined hardness and temporally correlated errors. We work on in a variant of the correlated-errors setting from Section 4. The teacher places probability on the correct root child and spreads the remaining over four of the nine wrong children. Despite this preference for the correct action, the learned model exhibits strong looping and long responses under greedy decoding. Increasing temperature reduces looping and response length and improves accuracy.
201. u201 Interpretation. Hardness of learning constrains which root children the student can distinguish. Initially, the student may be unable to identify the correct child despite the teacher’s preference for it. After a training prefix visits a wrong child, reaches its non-goal leaf, and returns to the root, the history makes that particular child distinguishable as the previously visited wrong action. The student may therefore treat it separately, while still being unable to distinguish among the other nine children.
202. u202 Under this information constraint, the best distribution the student can learn assigns the previously visited wrong child its probability in the training distribution, $0.5/4=0.125$ . The remaining nine children have total probability $1-0.125=0.875$ , which the student can only spread uniformly, assigning roughly $0.097$ to each. The previously visited wrong child therefore has higher probability than any individual alternative, so greedy decoding selects it again. In effect, hardness induces a negative margin: rather than downweighting the previously selected action, the constrained learned distribution favors it over each unvisited action. This provides a possible explanation for the strong looping in Figure 15. Consistent with this persistent preference, the top and second predictions agree across consecutive root visits in $94.58\%$ and $65.24\%$ of pairs, respectively.
203. u203 Overall, this experiment shows that looping similar to Section 4 can occur even when the teacher distribution places significant mass on the correct progress-making action. It also illustrates that the basic principles in this paper can combine in different ways to cause looping; a problem need not match exactly the setups in Sections 3 and 4 for these mechanisms to appear.
204. u204 Appendix F Further Details on the Catalyst for Looping
205. u205 This appendix provides further details on the catalyst discussed in Section 5. We first show confidence buildup in real model traces (Figure 16), then give a more detailed overview of what happens under the risk-aversion mechanism in Figure 17 including panel (b) for when there is no exploration, and finally demonstrate the catalyst effect in the correlated errors mechanism of Section 4.
206. u206 Figure 16 illustrates this effect. We take the two looping traces discussed in Section 2.1 (AIME 2025 Problem I-7 and Problem I-5, generated with greedy decoding using the OpenThinker3-1.5B model; also discussed in Examples B.1 and B.2). For each trace, we plot the highest probability assigned to any next token at each decoding step. The red line marks the step where the final infinite loop begins. As the loop progresses, the maximum probability rises, showing that the model becomes increasingly confident in continuing the loop.
207. u207 (a) Problem I-7
208. u208 (a) Problem I-7 (photo)

     

209. u209 (b) Problem I-5
210. u210 (b) Problem I-5 (photo)

     

211. u211 Figure 16: Confidence buildup during looping. We plot the maximum next-token probability over time for two looping traces from OpenThinker3-1.5B on AIME 2025 Problem I-7 (a) and Problem I-5 (b), generated with greedy decoding. Each blue dot shows the top-1 probability at a single decoding step; the apparent vertical bands are due to plotting 30k time steps at this resolution. The green curve is a running average (window size 200). The vertical dashed line marks the step where the final infinite loop begins. In both cases, once the loop starts, the top-1 probability rises toward , indicating that the model becomes increasingly confident in continuing the loop.
212. u212 F.1 Further details: risk-aversion mechanism.
213. u213 (a) With exploration
214. u214 (a) With exploration (photo)
215. u215 (b) Without exploration
216. u216 (b) Without exploration (photo)

     

217. u217 Figure 17: Explicit looping bias drives reset probability toward 1. We work in the risk-aversion setting on (Section 3), with and without exploration. For each of 4000 greedy-decoded traces, we record at every visit to the root the model’s probability of taking the reset-to-start action (“source prob”) and plot it against the root-visit index (thin lines). The thicker lines show the average over all traces that reach that visit index (traces that have already ended are not included in the average; the average is shown only at visit indices reached by at least 5% of traces). In the biased models (orange), the training distribution is modified so that with probability the example is a trace that deterministically loops between start and root, and with the remaining probability it is sampled as before. Under this bias, the reset probability rises toward for most traces as the number of visits grows, so once a loop has persisted for a while the model becomes almost certain to continue it. In the unbiased models (blue), the reset probability stays roughly constant in the exploration variant (a) and generally drifts downward in the non-exploration variant (b). Note that panel (a) also appears in the main text in Figure 5.
218. u218 We now reproduce the catalyst effect in the risk-aversion setup of Section 3. As also discussed in Section 5, we modify the training distribution so that with probability $0.001$ the training example is a trace that deterministically loops between the start and root, and otherwise it is sampled as before. We report results for $G(5,5)$ both with exploration (panel (a) of Figure 17, also shown in the main text) and without (panel (b)). For $4000$ model-generated traces with greedy decoding, we plot the probability assigned to the reset (start) action each time the model visits the root. Without the bias, this probability stays around its learned value near the teacher’s $0.3$ . When we add the bias, the reset probability rises toward $1$ for most traces over successive visits, showing that the model becomes nearly certain it should keep looping once the loop has gone on for a while.
219. u219 Figure 18 reports accuracy, looping counts, and response lengths versus temperature for $G(5,5)$ with and without exploration (similar to Section 3), comparing models trained with and without the added bias. With the bias, looping is amplified. For example, on $G(5,5)$ without exploration at $T{=}0.2$ , adding the bias increases the average looping count from $6.16$ to $33.52$ , while accuracy falls from $19.83\%$ to $9.95\%$ . This is consistent with the reinforced reset probability making escape from the loop less likely.
220. u220 (a) Accuracy
221. u221 (b) Looping Count
222. u222 (c) Response Length
223. u223 Figure 18: Confidence bias amplifies looping and degrades accuracy. In the risk-aversion setting on (Section 3), we plot accuracy, looping count, and response length versus temperature, with and without exploration, comparing models trained on the original distribution (solid) and with an added looping bias (dashed). The bias increases low-temperature looping counts and response lengths and reduces accuracy, with especially large effects at .
224. u224 F.2 Correlated errors mechanism.
225. u225 Next we revisit the correlated errors mechanism from Section 4, at margin $0$ . Here, a slightly higher looping bias of $1\%$ is needed to produce the confidence-buildup effect. Specifically, with probability $0.01$ , the training distribution picks a random non-goal leading child at the root (fixed once at the start of the trace) and generates a loop: from the root it goes to that child, walks forward to the leaf, resets to the start, returns to the root, takes the same child again, and so on. With the remaining probability, traces are drawn as before.
226. u226 Figure 19: Explicit looping bias reinforces a single child at the root. For the correlated-errors setting on at margin , we compare models trained with (blue) and (orange) deterministic looping traces. For each of greedy-decoded traces, we identify the most visited child at the root and record the probability assigned to that child at each root visit. Thin lines show individual traces; thicker lines show the average over traces that reach each visit index, shown only where at least of traces contribute. With the bias, confidence in the most visited child grows over successive visits; without it, the probability stays near .
227. u227 Figure 19: Explicit looping bias reinforces a single child at the root. For the correlated-errors setting on at margin , we compare models trained with (blue) and (orange) deterministic looping traces (photo)

     

228. u228 (a) Accuracy
229. u229 (b) Looping Count
230. u230 (c) Response Length
231. u231 Figure 20: Effect of looping bias on accuracy and looping. In the correlated-errors setting on (Section 4), we plot accuracy, looping count, and response length versus temperature at margin . Solid blue lines use the original training distribution; dashed orange lines use deterministic looping traces. The effect is most visible at temperature , where looping count and response length increase slightly while accuracy decreases slightly. Because confidence buildup requires repeated visits to the same child, the aggregate effect is limited when sampling breaks the underlying loop.
232. u232 This change produces confidence buildup in Figure 19. With the bias, the mean probability assigned to the most visited child rises from $0.209$ at the first root visit to $0.797$ at visit $10$ . The $2532$ traces reaching visit $10$ also had an initial mean of $0.210$ , so the rise is not simply due to changes in which traces remain. Overall, $50.3\%$ of traces reach probability at least $0.9$ , compared with none without the bias.
233. u233 Figure 20 shows a small amplification of looping with the $1\%$ bias. At temperature $0.2$ , looping count and response length increase slightly while accuracy decreases slightly, but the effect is less dramatic than in the risk-aversion setting. The catalyst relies on conditioning on trajectories that continue to loop: confidence can build only over repeated visits to the same root child. Here, the model’s root distribution is already close to uniform. At positive temperatures, temperature smooths the small differences between children, and sampling often breaks the repetition before confidence can build, leaving little opportunity for the bias to act.
234. u234 A similar explanation applies to the margin- $0.1$ variant. Because the margin model has no greedy looping to begin with, there is no underlying loop for the bias to amplify: accuracy remains $100\%$ and the looping count remains $1$ , so no root child is revisited. Overall, the looping bias can make an existing loop substantially stronger, but its effect depends on whether the model already has a sufficiently persistent tendency to loop.
235. u235 Appendix G Model capacity as a source of hardness
236. u236 To complement Section 3 and isolate the effect of model capacity on looping, we train models of three different sizes on the $G(5,3)$ task (no exploration variant): a 1-layer model (7.1M parameters), a 2-layer model (14.2M parameters), and a 12-layer model (85M parameters; the same architecture used throughout the rest of the paper). All other hyperparameters are identical to those in Appendix D. To further increase the difficulty for the smaller models, we also increase max_nodes, the size of the node-label vocabulary that each random walk is drawn from, from $30$ (its value in Section 3) to $50$ (we do this increase for all 3 models discussed in this section). Results across temperatures are shown in Figure 21.
237. u237 (a) Accuracy
238. u238 (b) Looping Count
239. u239 (c) Response Length
240. u240 Figure 21: Model capacity affects looping on . We train 1-layer, 2-layer, and 12-layer models on the task (no exploration variant) from Section 3, and plot accuracy, looping count, and response length against temperature. The 12-layer model achieves near-perfect accuracy across temperatures and has no greedy looping; the 2-layer model loops more at low temperature and achieves accuracy above chance but well below the 12-layer model; the 1-layer model loops even more and remains below chance, with invalid edges limiting accuracy at higher temperatures.
241. u241 The 12-layer model reaches near-perfect accuracy at all temperatures and has no greedy looping, matching its behavior on $G(5,3)$ in Section 3. The 2-layer model is affected by reduced capacity: at $T{=}0$ it loops on average about $10.35$ times between source and root, compared to $0$ for the 12-layer model. Its accuracy ranges from $27.6\%$ to $35.3\%$ across temperatures, above the $20\%$ chance baseline but well below the 12-layer model. The 1-layer model fares worse. At $T{=}0$ it loops on average about $52.96$ times between source and root, far more than either the 2-layer or 12-layer model. Its accuracy at $T{=}0$ is $5.55\%$ and reaches $15.2\%$ at $T{=}1$ , remaining below the $20\%$ chance baseline at all tested temperatures. Inspection of its traces shows that at higher temperatures it frequently samples tokens corresponding to invalid edges in the graph, which limits its achievable accuracy. In both small models, the reset action receives substantial probability at generated root visits.
242. u242 We chose $G(5,3)$ for this ablation deliberately. On $G(5,5)$ the source of hardness is optimization: even a large 12-layer model cannot reliably learn the correct first edge from the root, as the problem reduces to learning a parity-like instance [Bachmann and Nagarajan, 2024, Hu et al., 2025], and model capacity is not the bottleneck. On $G(5,3)$ the optimization hardness goes away (our 12-layer model solves the task near-perfectly), so $G(5,3)$ is the natural setting to isolate model capacity as an independent source of hardness. The fact that shrinking the model from 85M to 14.2M and 7.1M parameters reintroduces severe looping confirms that capacity-driven hardness produces risk-aversion behavior similar to that of optimization-driven hardness, discussed in Section 3.
243. u243 Appendix H Overestimation of reset probabilities
244. u244 Model size. The smaller models in Appendix G overestimate reset on their generated histories (Figure 22). Across root positions in greedy traces, the mean reset probabilities are $0.745$ , $0.472$ , and $0.286$ for the 1-, 2-, and 12-layer models, respectively; the fractions above the teacher’s $0.3$ are $93.9\%$ , $76.9\%$ , and $28.9\%$ . These pooled statistics give more weight to traces with many root visits. Giving each usable trace equal weight yields means $0.532$ , $0.356$ , and $0.286$ , so the smaller models still overestimate on average.
245. u245 Figure 22: Smaller models overestimate reset on generated histories. For the capacity models on , we measure the mean reset probability (left) and the fraction above (right) on held-out graphs. Solid curves pool root positions in the valid prefixes of greedy traces, excluding a final end-of-sequence decision. Dashed curves use the fixed context consisting of the prompt followed by start and root, once per graph. The dotted line marks the teacher’s reset probability, .
246. u246 Early training. Overestimation can also occur early in training. In separate $G(5,5)$ runs with and without exploration, using the settings of Section 3, we evaluate the first saved checkpoint ( $1000$ of $100{,}000$ training steps) on $4000$ held-out graphs. Across root positions in valid prefixes of greedy traces, the mean reset probabilities are $0.372$ without exploration and $0.430$ with exploration, with $63.8\%$ and $83.1\%$ of these positions above the teacher’s $0.3$ , respectively.
247. u247 We hypothesize that the overestimation observed in smaller models and early in training arises from the prevalence and ease of learning of the reset action, consistent with neural networks’ preference for simpler functions [Kalimeris et al., 2019, Rahaman et al., 2019], but leave a thorough analysis to future work.