StepFun Open Platform

Agentic workloads place a different set of demands on model design and training. Long, tool-heavy trajectories increase the importance of reasoning over large amounts of prefilling context, make long-context computation increasingly expensive, and introduce new challenges in data construction and reinforcement learning.

Step 5 Preview was designed with these constraints in mind. Architecturally, it follows a narrow-but-deep design with 92 transformer blocks. It interleaves Gated Sliding Window Attention with sparse full-attention layers, with Sparse GQA used in the full-attention path to support efficient long-context training and inference. Scaling this design also surfaced stability issues in deep training, leading us to revisit numerical precision and gradient handling.

On the training side, we built an agent-native data construction system in which agents coordinate parts of the synthesis process under expert-defined principles and quality controls. For reinforcement learning, we focus on the systems and learning problems that become dominant at long horizons: training-inference consistency, rollout efficiency, and credit assignment over very long trajectories.

The sections below describe these design choices and the observations that motivated them.

Architectural Design Philosophy

Depth matters in multi-hop reasoning [1]. In agentic tasks, multi-hop dependencies exist everywhere, while due to the long prefilling requirements by the frequent tool calls, models do not have much chance to perform thorough explicit CoT reasoning, especially taking token efficiency into account. Therefore, implicit reasoning during the model prefilling is preferred [2]. However, we know that the information flow in transformer's prefilling is unidirectional along both temporal and depth axes, which means the senentic embeddings tracing the multi-hop dependency chain have to move monotonically towards deeper layers in the future tokens. As a consequence, if it is not deep enough, the model is easy to run out of layers in long prefilling scenarios, making such latent multi-hop reasoning impossible [3].

Step 5 Preview follows a "narrow but deep" architecture. Although there are several research lines such as looped or recurrent transformers to deal with the depth curse, we adopt the simplest approach -- scaling up the depth directly. Step 5 Preview is composed of 92 transformer blocks, which is as deep as huge open source models like Qwen 3.8 max and Kimi K3, while the total and the active parameters are less than 1/4 and 1/3 of the huge counterparts, respectively. The "narrow but deep" architecture attribudtes Step 5 Preview as strong long-horzon agentic capability as those trillion-scale counterparts, while maintaining a much lower cost. Nevertheless, we find such design introduces some training instabilities. We discuss the topic in Sec 2.

Step 5 Preview utilizes 3:1 interleaved efficient and full (sparse) attention layers, following Step 3.5 [4] as well as the community's common practice. For the efficient attention part, after studying a variety of architectures (e.g. various linear attention alternatives) and configurations, we choose Gated Sliding Window Attention (Gated SWA) -- we find the performance gap between Gated SWA and other fancy choices (if better) is rather marginal, while Gated SWA is much simpler for efficient implementation, especially considering Speculative Decoding.

The full attention part of Step 5 Preview is based on Gated GQA. To support efficient long context training and inference (up to 1M context length), we propose Sparse GQA, introducing sparse indexing mechanism into standard GQA inspired by DSA [5]. Different from the original DSA, we further introduce block-wise token merging to reduce the indexer's footprint. We discuss our methodology in Sec 3.

Training Stability

Similar to [6, 8], we adopt mixed precision FP8 strategy to train Step 5 Preview. Although "narrow but deep" architectures are well motivated as mentioned above, we have observed several instability phenomena such as numerical explosion or gradient spike in scaling up the training, yet we find such instability seldom occurs in training smaller, shallower or wider networks. We come up with the following methodology to stablize the training:

FP32 hidden state. Massive activations are commonly observed in large language models [7], which are believed to act as the bias terms of the network. Typically those massive values do not cause training instability. However, when the network goes deeper, we find a few massive activations in the hidden states tend to blow up very quickly during the training, if the corresponding float-point precision is set to BF16 as in [6]. Although the underlying reason is still unclear, we find applying FP32 hidden states [8] as well as the corresponding gradients works well, in addition, further reducing the peak values by nearly an order of magnitude. Compared with the conventional BF16 precision, FP32 hidden state introduces only marginal training overhead.
FP32 RMSNorm scale. Step 5 Preview adopt Zero-centered RMSNorm [9] as the normalization layers, whose formulation is as follows:

y_i = \frac{x_i}{\mathrm{RMS}(x)} \times (1 + \gamma_i),

where \gamma_i is the learnable scaling factor, which is typically stored in BF16 precision. However, if x contains massive values, we find the design may cause instability: it is known that the scaling weigh (1 + \gamma_i) corresponding to the massive dimension is usually very small [10], i.e. \gamma_i is very close to -1; yet BF16 is not very precise near -1. During training, we find \gamma_i is exactly equal to -1 occasionally, entirely masking out the corresponding channel; then in the next few iterations, \gamma_i is updated to a nearby value by perturbations such as weight decay, making the channel open again. This cycle repeats, leading to the output oscillating back and forth. To address the issue, we suggest applying FP32 \gamma_i instead of BF16, making the behavior of RMSNorm much smoother where \gamma_i is near -1.

Finegrained gradient clamp. During training such a deep model, we observe gradient spikes usually occur more and more frequently in the mid-stage of training, which slows down convergence. We find the gradient spikes typically do not originate from data noises nor LLM head, instead, rise in some middle layers and are amplified by several orders of magnitude within a few layers. Data repetition also seems to contribute to the spikes. Thus we hypothesize the phenomenon may be resulted from layer redundancy [11, 12], which tends to amplify a certain gradient signal again and again during backpropagation. Fortunately, the abnormal gradients are very sparse. We therefore propose fingrained gradient clamp, clipping the top massive gradient vectors throughout the gradient flow. We find clamping only 0.00004% of gradient values per layer leads to excellent gradient spike suppression and smoother convergence.

Figure 1: LLM loss comparison before (blue) and after (orange) finegrained gradient clamp.

Sparse GQA

Step 5 Preview is born for long-horizon agentic tasks, in addition, strong long context reasoning capability requires native long-term long context training. When the context length reaches 1M tokens, the full attention layers become the major efficiency bottleneck in both training and inference. Inspired by [6, 13], we introduce a carefully optimized sparse attention mechanism to accelerate the full attention part in Step 5 Preview. To reduce the runtime cost of sparse attention, three bottlenecks must be addressed: the cost of the indexer used for selection, insufficient GPU utilization when computing over the selected content, and the trade-off between selection granularity and the amount of data read. In Step 5 Preview, we choose GQA for better long context accuracy (validated in our experiment) and supporting high-speed inference with multi-GPU parallelism. However, compared with MQA and MLA, GQA amplifies all the above three issues. Thus we come up with the following methodology, named Sparse GQA, that largely overcomes those bottlenecks.

Reducing indexer's cost. We find in some existing implementations of sparse attention such as DSA [5], the indexer's overhead is non-negligible, which could even exceed the cost of the sparse Flash Attention operator itself under extremely long contexts. We consider two possible solutions: 1) having multiple GQA groups share a single indexer, or 2) compressing the indexers for multiple adjacent historical positions into a single block representation. After some ablations, we find the first option compromises GQA's suitability for parallelism and has a relatively noticeable impact on model quality. In Step 5 Preview, we thus adopt the second solution with a block size of 8, reducing the cost of the indexer and top-k selection by 8×.
Efficient implementation of sparse attention. Sparse attention makes computation fragmented. During prefilling, we need to organize these fragmented, small computations so that the GPU can be utilized more effectively and deliver actual speedups. For each token, once the K/V selected by the indexer and top-k operation enters the computation, each individual chunk contains only a small amount of work. Processing tokens one by one makes it difficult to fully utilize the GPU. Our optimization is to take the union of the top-k results for adjacent tokens. Since adjacent tokens are semantically similar, the K/V selected by their top-k operations often overlap substantially. Our implementation therefore groups these overlapping computations together to improve GPU utilization.
Trade-off between top-k and block size. In Sparse GQA, there are two important hyper-paramters: the number of selected top tokens k and block size B. We compare model quality, indexing cost, and main-path latency under different top-k and block-size configurations to explain the rationale behind the final configuration. Our experiment supports the choice of B = 8 and k = 4096 (i.e. the number of selected blocks is k / B = 512). Increasing B reduces indexing cost, while increasing k compensates for the coarser selection. As sequences grow longer, the estimated indexing savings outweigh the additional main-path cost, while token coverage remains close to B = 1, k = 2048 reference (78.06% vs. 79.09%).

Figure 3: Illustration to Sparse GQA. The indexer compresses shared index keys into block representations, and each KV group independently selects relevant history. The main path reads the original K/V according to the selected historical blocks as well as the causally visible positions within the current block.

Agentic Data Synthesizing

During the development of Step 5 Preview we observe a rapid increase in frontier models' ability to perform long-horizon, agentic work [14, 15, 16]. Those emerging capabilities motivate us to rethink data construction as an agent-native process.

Unified context layer. Domain experts naturally develop priors, intuitions, and practices for constructing high-quality data, which are often embedded in their own context for task designs, pipelines, and decisions [17]. We therefore build a unified data synthesizing pipeline that captures task specifications, pipeline states, environment configurations, intermediate results, and prior feedback across different domains, and makes them accessible throughout the construction process. This allows agents to reuse experience across tasks while researchers can continue to provide human priors and audit critical points. As construction proceeds, the resulting feedback and experience are accumulated back into the shared context for loop development [18]. Moreover, this substantially reduces the effort of adapting data construction to new domains, as agents can build on accumulated design principles, quality criteria, and failure modes rather than developing them from scratch.

Agent-driven knowledge sourcing. On the other hand, knowledge stands at the core of model capability, yet the knowledge covered by constructed tasks is ultimately constrained by the breadth of the available sources. Broad and systematic sourcing is therefore crucial for uncovering diverse and long-tail knowledge. Within the unified context, we organize knowledge from pretraining data and broad Internet sources, e.g, Github, into a structured knowledge space, allowing agents to systematically traverse it to identify knowledge that is long-tail, underrepresented, or missing from the model, and use it to synthesize more challenging tasks. In this way, knowledge sourcing becomes an integrated part of the construction process, turning broad knowledge coverage into a systematic process for discovering and training on the model's knowledge gaps.

Quality steering. Autonomous synthesis does not automatically produce the data we want. We find that agents exhibit systematic biases in both difficulty and diversity: they tend to construct easier tasks, often by simplifying the underlying problem or providing excessive information, and repeatedly converge on familiar task patterns. We therefore introduce explicit signals and interventions to steer agents along these dimensions. For difficulty, we use observable signals, e.g., task constraints, ground-truth patch length, number of test cases, and performance across different frontier models to calibrate task difficulty. For diversity, we track the distribution and balance the sourcing for each direction to steer agents away from overrepresented repositories, topics, and workflows. These mechanisms turn difficulty and diversity from passive evaluation criteria into active controls over agent behavior. Beyond these construction biases, we also address model hacking [19,20]: frontier models excel at exploiting weaknesses in task specifications or evaluations without genuinely solving the underlying task, typically in more realistic and challenging agentic environments [21]. We therefore incorporate explicit anti-hacking checks into task construction, auditing both the task specification and evaluation procedure for exploitable shortcuts and iteratively refining checks against discovered failure modes. This ensures that task success reflects the intended capability rather than artifacts of the task or evaluator.

Division of roles. Our agentic approach involves a deliberate division of labor between expert judgment, agent orchestration, and deterministic execution. Experts provide high-level construction principles and priors, and audit critical points of the pipeline where domain judgment is essential. Agents operate within these expert-defined principles to dynamically coordinate the construction process, make cross-step decisions, and adapt to intermediate results. We deliberately delegate deterministic and repeatable operations to reusable code and tools, rather than asking agents to execute them. In practice, routine operations such as building Docker images, checking task verification, measuring test coverage, and validating output formats are encapsulated as reusable procedures that agents can invoke when needed. This reduces repeated agent or human intervention on routine operations, removes intermediate bottlenecks, and allows the pipeline to run continuously at a high throughput. By amortizing these procedures across tasks, we substantially increase the amount of high-quality data that can be constructed while reducing agent and human effort per example.

Unified context layer retains experience, agentic knowledge sourcing broadens exploration, Quality Steering controls quality, and Division of Roles enables efficient scaling. Together, these components form an agent-native and continuously improving data construction system.

Supported by this infrastructure, we ultimately construct million-level challenging and verifiable environments for frontier models, spanning scientific reasoning, software engineering, machine learning research & development, and professional work. We also collect the construction trajectories, decisions, and feedback generated throughout this process, which will provide training data for improving the next generation of models in automated post-training data research.

Agentic RL

What Makes a Task Worth Training On

Three axes define a trainable task. We profile every candidate data along three axes: duration, how long the agent must sustain coherent progress; depth, how many state transitions accumulate before reaching a passing state, measured in practice by files touched and lines edited; and discriminativeness, how well the task separates one attempt from another, measured as the reward variance across a group of rollouts. The first two are designed into a task, and the third is measured against the policy. We build tasks high on both duration and depth, such as long-running terminal work, repository-scale software engineering, and multi-stage program construction.

Zero Training-Inference Mismatch

Online RL system runs the same model on a training and an inference side, built from distinct operators with different performance objectives. Tiny numerical differences amplify through MoE dispatch and depth increase. These system-intrinsic errors then couple with off-policy staleness. Thus, We addressed the two sources separately: aligning the systems exactly where alignment is possible, and replaying what staleness makes impossible to align. (1) Bitwise alignment at under 10% overhead. At staleness = 0 we fully aligned the operators between training and inference: the logprob difference is exactly zero. Beyond the previous deterministic kernels [22], this required resolving differences introduced by SWA, MTP, top-k selection, and others. An optimized tree-reduction computation makes the result TP-size-invariant: training at TP = 1 and inference at TP = 8 stay bitwise identical, with total overhead below 10%. After alignment, an FP8 policy reproduces the BF16 behavior, with stable tokens-per-turn, steadily growing turns, and stable entropy, where the unaligned version drifted into bloated, unstructured single turns. (2) RL trace replay. At staleness > 0 the principle is what you see is what you get: the training side replays the inference computational state. We replay routed experts with frozen MoE router [23], including expert indexes and coefficients for both the actor and the reference policy. We replay the top-p candidate set [5], recomputing logprob only over the candidate tokens. And we add a zero-center projection to the logit gradient [24], removing the common-mode drift; With these techniques, entropy stops decaying and holds stable. Together these keep the long-horizon logprob difference stably below 1e-2.

High-Throughput and Low-Redundancy Infrastructure

As agents take on longer-horizon tasks, turns and tokens grow at a startling rate, and inference already accounts for the majority of total compute. Efficiency here is two different problems. One is extracting more throughput; the other is making sure the samples that were expensively produced reach the trainer. (1) Every step delivered at scale. Our session_admission scheduler [25] keeps a session affine to one instance, and lets requests wait and retry in place under high concurrency. On top of stable scheduling: a MTP-3 head trained after SFT provides roughly 30% end-to-end speedup; Native FP8 MoE gives another ~20% while matching BF16 quality; and coordinated EP, CPU offload of KV, MXFP4 weight storage, and FP8 KV QAT expand effective KV-cache capacity by 8–12x. At training side, we lowers peak memory by chunking MoE computation, sparse-attention index selection, and LM-head projection with cross-entropy, trading extra sub-batch scheduling for improved training scalability. (2) Every rollout accounted for. A rollout passes through a long chain before training, and every boundary is a place a sample can quietly drop out. Such drops are not uniform: they concentrate in the longest, hardest rollouts, shifting the training distribution. We implement a lifecycle ledger that tracks every expected rollout to exactly one named terminal state, and a turn-level event record inside the scaffold that captures every intervention within a rollout. Together these brought the end-to-end sample drop rate below 1%.

Towards Infinite-Horizon Policy Improvement

An infinite-horizon agent strains RL in two distinct ways: the trajectory outgrows the context window before the task is finished, and the single episodic outcome reward has to explain everything that happened along the way. (1) Context management decouples the horizon from the context window. Test-time scaling [26] converts inference compute into performance, but the maximum model length caps how far a rollout can scale. Longer-context architectures only raise that ceiling. The scaling law we want is one where the horizon is unbounded while the working context stays finite. So when the budget is nearly spent, the agent reorganize its work into a resumable state and continues. Following the bitter lesson [27], we design compaction as the first-class citizen of policy training, trained end-to-end under the same task reward. (2) Credit is assigned at the scale on which behaviour operates. An outcome says something went wrong, never which decision was responsible [28], and infinite-horizon agents stretch that one binary verdict over hundreds of turns and millions of tokens. At the token level, a learnable critic converts the outcome into a reasonable per-token baseline. We pretrain the critic to reduce the actor loss variance. At the turn level, failures like malformed tool calls neither follow earlier turns nor affect later ones. We penalize the offending turn's returns and leave the rest untouched. Reward hacking rides the same channel: a detector refuses hacking actions in-flight and penalizes the turn, telling the agent to try another approach and continuing the rollout.

Where We Go From Here

For Step 5 Preview, much of the remaining work is about pushing this boundary further: enabling agents to keep making useful progress over longer trajectories without letting context and training cost grow at the same pace. We expect both the model and the surrounding training system to continue evolving toward that goal.

The systems described here are still evolving. As agents take on longer tasks and interact with increasingly complex environments, the limits of context, training infrastructure, and feedback become harder to separate. Improvements in one part of the system often expose bottlenecks elsewhere.

Acknowledgments

We thank everyone who contributed to Step 5 Preview for their work and dedication. This effort was made possible by the collective contributions of our colleagues and interns across the team.

View all contributors ↗