Weekly Overview
Recommendation systems research this week splits along a clear line: generative recommendation is shifting from "can it generate?" to "is generating worth the cost?", while improvements to conventional ranking architectures converge on granularity alignment for heterogeneous signals.
Thread 1: Generative retrieval is starting to dismantle its own cost structure. The standard autoregressive interface pays for SID length twice — history enters the backbone as T×K code tokens, and every beam search step runs through the backbone again. Pinterest's PinGER uses an asymmetric attention mask to compress input to one token per item, delivering +3.60% fresh Pins saves and -0.7% end-to-end latency in serving. On the academic side, Deco delegates the entire beam search to a lightweight KV-cached decoder, achieving 3.2–8.6× throughput gains within a 50 ms p95 budget. Octen-WebGR packs 201 million web pages into 4B model parameters, runs on a single GPU with 10 GB memory, and hits 59.6% top-1 on navigational queries versus 46.5% for dense retrieval. All three point to the same conclusion — the bottleneck in generative retrieval has moved from model capacity to serving cost.
Thread 2: Multi-objective and fine-ranking are shifting from "compute more steps" to "compute in the right places." Kuaishou's Refiner introduces latent reasoning into ID-based ranking via cumulative additive correction. Alibaba's KL-MOPD finds that teacher difficulty in multi-objective on-policy distillation varies across tokens, allocates weights by initial KL divergence, and proves strict superiority over vanilla MOPD under an additive joint objective. Both point to the same thing: the extra compute in generative ranking must be precisely targeted where it's needed.
Thread 3: Granularity alignment in ranking architectures. Alibaba's HiRank shows that sequential and non-sequential signals have different effective ranks, and that homogeneous self-attention is inefficient on such heterogeneous inputs — online orders/user +2.27%. Meituan's HGE addresses the structural mismatch between container-level ranking and item-level transactions — online RPS +1.42%. Criteo's COMPOTE demonstrates that the gains from multi-hypothesis structure come from the annealing objective, not from capacity. The common thread: the mismatch isn't in model capacity, it's in the unit of modeling.
Scaling and Efficiency Optimization in Generative Retrieval
Generative retrieval is being deployed in industry faster than its cost structure is understood. This week's papers dissect the same bill from different angles: how many tokens history input consumes, how many backbone forward passes beam search requires.
PailitaoMMSearch (Alibaba) — Taobao Pailitao's unified end-to-end multimodal retrieval system generates product Semantic IDs directly from raw query images and free-form instructions. The system scales along three dimensions. Data: a reusable toolset composes intent- and lifecycle-specific workflows covering cold-start synthesis, quality control, real-traffic refinement, and continuous data production. Intent: a three-stage pipeline of continual pre-training (CPT), mixed-budget SFT, and unified RL progressively equips a single model with e-commerce product knowledge, multiple retrieval relations, and adjustable reasoning budgets; the RL stage combines intent-balanced optimization with Sample–Beam rollouts aligned to multi-candidate serving. Scenario: the model exposes quality–cost operating points between capacity and inference budget. Online A/B results: item page views +17.90%, transactions +27.04%, GMV +8.94%.
This design contrasts with Gryphon's approach. Gryphon jointly trains an item-level scoring component within encoder-decoder generative recommendation, resolving the mismatch between SID sequence likelihood and relevance objectives — Recall@1000 improves 3.7% on an industrial music service, replacing 15+ candidate generators and the coarse-ranking stage. Gryphon's approach adds a discriminative "corrector" inside the generative framework; PailitaoMMSearch instead expands intent coverage and reasoning budget within the generative framework. Amap's Gwhere offers a third reference path: a contrastive residual quantization tokenizer aligns text, visual, spatial, and collaborative signals, paired with the EAKTO RL objective — post-launch P-CTR +5.83%, U-CTR +6.20%.
PinGER (Pinterest) — The core of this paper is an asymmetric attention mask. SID tokens and dense tokens are interleaved in the input sequence, and the mask restricts subsequent items to attend only to dense tokens. The effect: every SID position is supervised during training, while each item consumes only one token at serving time — and both compute the same function. On top of this, target-conditioned generation, RL, and prefix diversity penalties steer retrieval toward surface-specific goals and more diverse candidates. The serving stack is customized for wide-beam generation, with decoding constrained to valid identifiers and candidates resolved by host-local indexes. Post-launch on Related Pins: fresh Pins saves +3.60%, overall saves +0.62%, end-to-end latency -0.7%, while eliminating the cost of the replaced nearest-neighbor retrieval.
This mask design addresses the efficiency problem left by long/short-term interest modeling approaches like DualGR. DualGR uses dual-branch long/short-term routers, search-based SID decoding (S2D), and an exposure-aware next-token prediction loss (ENTP-Loss) — online video views +0.527%, watch time +0.432%. The gain magnitude is comparable to PinGER's overall saves, but PinGER additionally achieves latency reduction and retrieval cost elimination.
Octen-WebGR (Octen) — Stores 201 million web pages in the parameters of a 4B language model, generating two-token semantic identifiers for a given query, constrained to a real URL table. Evaluation uses real navigational traffic from one month after training. The key finding: generative retrieval has the biggest advantage on "new query, old page" — for unseen queries where the target page has sufficient interaction history, top-1 hit rate is 30.9% versus 17.2% for dense retrieval trained on the same click volume (21.4% with click prior). Conversely, embedding retrieval retains an edge on obscure pages — the two methods generalize on the query side and page side respectively, and are complementary. Overall navigational traffic top-1 is 59.6% versus 46.5% for dense retrieval; frontier LLMs answering directly from memory reach at most 37.5%, but at least 40× the cost per query. The 4B model runs on a single GPU with 10 GB memory; exact dense retrieval requires 412 GB.
Scaling experiments (up to 32 million URLs) yield two constraints: at equal exposures per query–page pair, the required model size grows sublinearly with index size; retrieval effectiveness is limited more by page exposure than by model capacity. This complements EGR's embedding-native approach — EGR uses a single shared LLM to jointly learn item and user representations, indexed directly as dense vectors, achieving +2.91% conversion rate on Snap DPA. The two approaches draw the boundary between "generation" and "retrieval" differently.
Deco — Separates "reading history" from "writing identifiers." The backbone reads each history item as one fused token, compressed in one pass into a compact next-item concept z; a lightweight KV-cached decoder conditions on z and the generated prefix to run the entire beam search. The identifier, trie, and autoregressive decomposition all remain unchanged. The decoder approaches full-backbone accuracy with cross-entropy alone, and distillation improves it further. Across seven e-commerce, movie, and music-streaming datasets, accuracy matches or exceeds standard generative recommendation and the fused-input full-backbone teacher, with 3.2–8.6× throughput gains within a 50 ms p95 latency budget under a production-grade serving pipeline. The derived conclusion is more noteworthy: generating 1000 candidates is faster than the teacher generating 100; a 1.5B-parameter LLM backbone matches the throughput of a 69M teacher while achieving higher accuracy.
This approach belongs to the same class as RelayGR's cross-stage relay inference — both redistribute compute at the serving-stack level rather than changing the model. RelayGR achieves 1.5× sequence length and 3.6× throughput under a fixed P99 SLO, via sequence-aware triggers, affinity-aware routers, and memory-aware expanders. Deco's difference is that it fully decouples candidate budget from backbone size.
SLIDE — Targets a specific pathology in list-level generation: directly trained list-wise models with greedy decoding fall substantially short of beam search. Two causes — beam search explores a wider search space; sparse positive signals introduce position bias, leaving later positions under-supervised. SLIDE decouples offline search from online list generation: a teacher model retrieves high-quality candidates via offline item-level beam search, concatenates their SIDs by beam score into list-level distillation targets, and the student generates Top-K in a single greedy forward pass. This converts candidates from different search paths into sequence targets within a single trajectory, and decouples teacher size from online deployment. On FORGE and Taobao Search, single-pass greedy decoding exceeds the beam search reference; swapping in a stronger offline teacher yields further gains. Supplementary RL experiments support the effectiveness of list-level generation.
A related problem has a different solution in PROMISE: a lightweight process reward model performs step-by-step verification, paired with PRM-guided beam search that dynamically prunes erroneous branches, enabling a small model to match or exceed a large model with additional test-time compute. SLIDE moves search offline and reduces online to a single forward pass; PROMISE keeps search online but uses a reward model to narrow the branches.
CRIM — Fully separates item-level semantic encoding from request-level history modeling. A small set of learnable summary tokens compresses each item into a compact continuous memory, and segment-level readout tokens (REC tokens) provide predictive supervision within the interaction history. These memories are independent of user and query, and reusable across requests. At inference, cached memories are composed in interaction order, optionally injected with the query, producing a personalized retrieval representation — repeated item text encoding is eliminated while request-relevant context modeling is preserved. Training uses a joint mechanism: cached memory plus sampled live re-encoding, re-encoding only a few history items per request, allowing the retrieval objective to optimize both the item encoder and request encoder with limited compute, with periodic cache refresh for consistency. State-of-the-art retrieval performance on MovieLens-1M, Qilin, and the in-house LazSeq, with substantially lower inference cost at the same backbone scale.
This line intersects with HyMiRec's hybrid multi-interest approach — HyMiRec uses a lightweight recommender to extract coarse-grained interest embeddings and an LLM recommender to capture fine-grained interest embeddings, compressing user history via a cosine-similarity residual codebook. CRIM's difference is that the reuse unit is the item rather than the user history segment.
Ranking Refinement and Multi-Objective Optimization in Generative Recommendation
Generative ranking has long relied on one-shot prediction, whereas ranking requires fine-grained scoring across multiple candidates. This week's two papers introduce additional inference-time compute from different angles, but both emphasize precise placement.
Refiner (Kuaishou) — A latent residual reasoning framework for generative ranking, paired with parameter-efficient fine-tuning. Three components: a residual reasoning mechanism that progressively refines ranking scores through cumulative additive correction; history queries that provide compressed user behavior representations, giving reasoning candidate-specific context; and LoRA fine-tuning that adapts a pre-trained model to latent reasoning. Consistent improvement over direct prediction across two public benchmarks, one offline industrial dataset, and online A/B.
The method choice is worth noting. Many works try to port LLM chain-of-thought reasoning to recommendation, but this explicit text-space reasoning doesn't apply to the broader deployment of non-text recommendation models. Refiner pursues latent reasoning inside ID-based models — in the same direction, xGR addresses the serving side: unified handling of prefill and decode via staged computation and separated KV caches, enabling early ranking termination and mask-based item filtering, achieving at least 3.49× throughput over SOTA baselines under strict latency constraints. Refiner adds reasoning steps; xGR compresses serving overhead. Together they form the complete answer to "affordable inference-time compute."
KL-MOPD (Alibaba) — In generative recommendation, a positive sample is only recalled if every decoding step selects the correct token, so conventional item-wise methods lack token-level intermediate supervision. This paper introduces multi-objective on-policy distillation (MOPD) to generative recommendation, distilling two single-objective teachers into one student at each token. The problem: vanilla MOPD has two defects — token-level imbalance, where teacher distillation difficulty varies across tokens; and cross-token inconsistency, where the hard-to-distill teacher changes across tokens. The result is that some objectives learn poorly at certain steps, impeding item recall for both objectives.
KL-MOPD's solution is direct: allocate each token's teacher weight by initial student–teacher token-level KL divergence, strengthening supervision of the hard-to-distill teacher at every step. The authors prove, under mild conditions, that KL-MOPD strictly outperforms vanilla MOPD under an additive joint recommendation objective. It's plug-and-play in engineering, requiring only one-time pre-trained weight calibration with negligible additional training overhead. Individual and joint metrics improve across three public benchmarks, a billion-scale industrial dataset, and online A/B on an e-commerce advertising platform.
Another path to multi-objective coordination in generative recommendation is OneLoc: Kuaishou's local life services uses geo-aware semantic IDs, geo-aware self-attention, and neighbor-aware prompting to integrate geographic information, then uses RL to balance multiple objectives — GMV +21.016%, order count +17.891%. KL-MOPD does distillation weighting at token granularity; OneLoc does objective trade-offs via RL at sequence granularity.
Ranking Architectures and Sequential Multi-Intent Modeling
Architectural improvements on the ranking side this week converge on the same class of problem: signals from different sources have different statistical properties, and cramming them into the same attention block or modeling unit loses information.
HiRank (Alibaba) — A specific observation about large-scale industrial ranking models: sequential and non-sequential features have different effective ranks (eRank), and applying homogeneous self-attention directly to all semantic tokens is inefficient; shared representations also ignore scenario-specific patterns, introducing negative transfer across scenarios. HiRank fuses gated sequential attention with heterogeneous feature interaction within a unified backbone, enabling bidirectional fusion of sequential and non-sequential representations; scenario-aware aggregation then compresses heterogeneous low-order tokens into a compact representation; different learned projections transform this compact representation into scenario-aware composite tokens — complementary global feature views, more uniform information density — on which higher-order self-attention is more effective. Performance scales favorably with parameter count on industrial-scale datasets; online A/B orders/user +2.27%.
This directly engages with HyFormer. HyFormer integrates long-sequence modeling and feature interaction into a single backbone, surpassing LONGER and RankMixer on industrial datasets with online A/B CTR gains. HiRank's increment is explaining via the eRank lens why such integration requires asymmetric treatment, and providing a scenario-dimension solution. For the multi-task side, see AdaTT: multi-level deep fusion networks with residual and gating for adaptive inter-task fusion, a strong baseline on Meta's industrial dataset.
HGE (Meituan) — Live streaming and O2O e-commerce are inherently "container-style" displays: the platform ranks and estimates at the container level (live rooms, stores), while actual transactions occur at the item level within containers. This granularity mismatch creates three problems — high label variance, severe label sparsity, and dynamically heterogeneous item composition. HGE addresses each with three modules: HSAE (hybrid set-aware encoder) models dynamic item composition and feature interaction; PLL (proxy label learning) decomposes post-click GTV into purchase probability × quantity × unit price, reducing label variance; RCEF (residual correction and ensemble fusion) combines item-level aggregation with container-level inference to alleviate label sparsity. Offline XAUC +0.63%, online RPS +1.42%, fully deployed on main traffic.
Container-level and item-level decomposition has a counterpart in causal inference. Probabilistic Residual Learning uses probabilistic user grouping, domain-level confounder modeling, and do-calculus aggregation, achieving +2.3% CTR on Meta's online system. Both acknowledge that the observational unit and the decision unit don't coincide, and bridge them with residual structure.
COMPOTE — Next-interaction prediction is inherently ambiguous: a single history often corresponds to multiple plausible continuations in different catalog regions, and standard sequential recommenders compress these alternatives into a single prediction, conflating user intent. COMPOTE is a multi-hypothesis Transformer trained with an annealing objective — transitioning from uniform supervision to winner-takes-all assignment. This mechanism forces hypotheses to specialize to different catalog regions while preventing head collapse common in conventional multi-head architectures. Inference uses an unnormalized max-similarity rule.
The theory provides two supports: in the population case, the optimal solution partitions the conditional next-item distribution according to the maximum-entropy quantization principle, while pooled scores preserve Bayes-optimal ranking; the generalization bound grows logarithmically with the number of hypotheses. On three Amazon Reviews 2023 benchmarks, item ID alone surpasses strong ID baselines, generative baselines, and multi-interest baselines. Capacity-matched ablations confirm the gains come directly from the annealing objective, not from extra capacity, ensembles, or routing.
This line traces back to DSIN: splitting behavior sequences into sessions at 30-minute intervals, modeling intra-session interest with biased-encoding self-attention and inter-session evolution with Bi-LSTM, surpassing DIN and DIEN on CTR tasks. COMPOTE changes "multi-segment interest" from temporal splitting to hypothesis splitting, and enforces differentiation via annealing rather than architecture. On the retrieval-augmentation side, the contrast is RAT, which enhances candidate item representations by retrieving similar samples, improving AUC by roughly 0.5–1.0% on public datasets, with gains concentrated in cold-start and long-tail.
Exploration-Exploitation Optimization for Ad Creative and Budget
The exploration problem on the advertising side this week lands on two decisions at different granularities: creative-level selection and budget weight allocation. Both are constrained by sparse and delayed feedback.
Hierarchical Empirical Bayes Creative Pre-ranking (Roblox) — Dynamic creative optimization decides which creative to render per request. Prior work places this selection before ad ranking but relies on deterministic point predictions without calibrated uncertainty. AIGC makes creatives nearly free, while per-creative feedback remains sparse, and selection is constrained by strict latency budgets.
The proposed creative pre-ranker scores each recalled creative, shortlisting each campaign's pool before the per-user ranker runs. Creatives sit in a four-level hierarchy — audience segment, ad item, campaign, creative — and cold creatives fall back to information accumulated at parent levels, with borrowing strength determined in closed form from historical logs, one parameter per level, no tuning required. To stay within the latency budget, the pre-ranker predicts by audience segment rather than individual user, using attribute-aggregated counters. Selection performs one Thompson sample per creative, giving new creatives a chance to be learned.
Replaying Roblox sponsored-ad traffic for campaigns with competing creatives: shortlisting improves predicted play rate by 8.14% versus 2.80% for random shortlisting at the same scale; forwarded creative count drops by over 40% versus the full pool; serving latency increases by only 0.2%. Deployed in Roblox production after online A/B. For neighboring work on Thompson sampling in cold-start scenarios, see Dynamic Prior Thompson Sampling: closed-form quadratic solutions set prior means, ensuring new arms beat the current winner with probability ε, improving Qualified Play-Through Rate and reducing regretful impressions in online experiments on a thumbnail personalization system. Delightful Exploration offers another gating approach — expected improvement multiplied by surprise as an exploration gate, requiring no hyperparameter tuning across Bernoulli bandits, linear bandits, and tabular MDPs, with regret growth far below Thompson Sampling and ε-greedy.
GP-UCB Constrained Black-Box Lever Optimization (Booking.com) — Optimizing discrete allocation weights across competing segments sharing a budget. Practical constraints are latency, noise, and aggregated feedback, plus strict operational limits on weight adjustment frequency. The combinatorial action space plus extreme data sparsity make classical MAB with independent arm evaluation inefficient. This paper uses GP-UCB: a Gaussian process prior on the reward function shares information across adjacent configurations, substantially compressing the exploration horizon.
The engineering path has two steps. First, a high-fidelity synthetic simulator evaluates kernel configurations across stationary, drifting, and reactive environments, identifying a Matérn + Linear composite kernel as most robust to local noise and global drift. Then a large-scale RCT in the real production environment achieves statistically significant efficiency gains over an ε-greedy baseline. This "simulator for kernel selection + online RCT validation" combination directly informs black-box tuning under operational constraints.
A neighboring work on causal bandits under budget constraints is BCCB: unified learning, exploration, and budget control, operating effectively with just 1 user on the Criteo Uplift dataset, whereas offline methods require roughly 10,000 historical observations. Experimental design for method comparison also matters — Artificial Replay proves that under unbiased estimation, experimental cost drops from 2T to T+o(T) with sublinear variance growth, validated on UCB, Thompson Sampling, and ε-greedy. The synthetic simulator design in the GP-UCB paper can be seen as the continuous-action-space counterpart of the same problem.
Directions to Watch
The cost bottleneck in generative retrieval has moved from the model side to the serving side. At least four papers this week are working different facets of the same problem: PinGER compresses input tokens (1 per item), Deco strips beam search from the backbone (3.2–8.6× throughput), CRIM reuses item encodings across requests, SLIDE moves search offline (single greedy forward pass generates Top-K). Add xGR's serving-stack optimization and RelayGR's cross-stage relay, and a plausible conclusion emerges — progress in generative retrieval over the next year will come more from "how to organize compute" than "how to design models." Multiple production lines at Pinterest and Alibaba are pushing in this direction, with the practical payoff directly tied to lower per-request cost and larger candidate pools.
Token-level supervision is becoming the main battleground for multi-objective coordination in generative recommendation. KL-MOPD's token-level imbalance and cross-token inconsistency are structural problems in generative recommendation — a positive sample is only recalled if all tokens are correct, so any item-level loss design loses intermediate-step signal. Adjacent evidence on this line includes PROMISE's process reward model step-by-step verification and Refiner's cumulative residual correction. Worth noting: KL-MOPD's theoretical guarantee holds under an additive joint objective, an assumption that may not be strictly satisfied in real-world multi-objective trade-offs — follow-up work may start here.
Granularity alignment for heterogeneous signals is worth more investment than model capacity. HiRank's eRank lens, HGE's container–item granularity decomposition, and COMPOTE's hypothesis count and generalization bound all point to the same conclusion from different angles: cramming signals with different statistical properties into the same modeling unit loses information, and that loss can't be recovered by adding parameters. Meituan, Alibaba, and Criteo all have corresponding production lines. The gain magnitudes (orders/user +2.27%, RPS +1.42%) aren't as eye-catching as generative retrieval's headline numbers, but these are increments on steady-state traffic — higher marginal value.
Paper Roundup
Scaling and Efficiency Optimization in Generative Retrieval
PailitaoMMSearch — Alibaba proposes a unified end-to-end multimodal generative retrieval system that generates product Semantic IDs directly from query images and free-form instructions, scaling along data, intent, and scenario dimensions; Taobao Pailitao online item PV +17.90%, transactions +27.04%, GMV +8.94%.
PinGER — Pinterest uses an asymmetric attention mask to interleave SIDs with dense tokens, supervising every SID position during training and consuming one token per item at serving; Related Pins online fresh Pins saves +3.60%, overall saves +0.62%, end-to-end latency -0.7%.
Octen-WebGR — Octen stores 201 million web pages in 4B model parameters, generating two-token semantic identifiers constrained to a real URL table; navigational traffic top-1 59.6% versus 46.5% for dense retrieval, running on a single GPU with 10 GB memory.
Deco — Separates history reading from SID writing; the backbone reads one fused token per item and compresses it into a next-item concept, and a lightweight KV-cached decoder runs the beam search; 3.2–8.6× throughput gains within 50 ms p95 across seven datasets.
SLIDE — A search-to-list distillation framework with Taobao Search as the experimental setting; concatenates offline item-level beam search candidates by score into list-level distillation targets, and the student generates Top-K in a single greedy decode; exceeds the beam search reference on FORGE and Taobao Search.
CRIM — Uses learnable summary tokens to compress items into cross-request reusable continuous memory, paired with REC readout tokens and joint cached + sampled re-encoding training; achieves state-of-the-art retrieval performance at lower inference cost on MovieLens-1M, Qilin, and LazSeq.
Ranking Refinement and Multi-Objective Optimization in Generative Recommendation
Refiner — Kuaishou proposes a latent residual reasoning framework that progressively refines ranking scores through cumulative additive correction, combined with history queries and LoRA fine-tuning; outperforms direct prediction across two public benchmarks, offline industrial data, and online A/B.
KL-MOPD — Alibaba proposes multi-objective on-policy distillation weighted by token-level KL divergence, addressing token-level imbalance and cross-token inconsistency, with theoretical proof of superiority over vanilla MOPD; improvements across three public benchmarks, a billion-scale industrial dataset, and online A/B.
Ranking Architectures and Sequential Multi-Intent Modeling
HiRank — Alibaba proposes a unified multi-scenario ranking framework that fuses gated sequential attention with heterogeneous feature interaction via the effective-rank (eRank) lens, with scenario-aware aggregation generating composite tokens; online orders/user +2.27%.
HGE — Meituan addresses the granularity mismatch between container-level ranking and item-level transactions with the HSAE set-aware encoder, PLL proxy label decomposition, and RCEF residual correction fusion; offline XAUC +0.63%, online RPS +1.42%, fully deployed.
COMPOTE — Proposes an annealed multi-hypothesis Transformer whose objective transitions from uniform supervision to winner-takes-all, forcing hypothesis specialization and preventing head collapse; surpasses strong baselines on three Amazon Reviews 2023 benchmarks using item ID alone, with capacity-matched ablations confirming gains come from the annealing objective.
Exploration-Exploitation Optimization for Ad Creative and Budget
Hierarchical Empirical Bayes Creative Pre-ranking — Roblox proposes a four-level empirical Bayes creative pre-ranker that aggregates by audience segment and shortlists via a single Thompson sample; replay shows predicted play rate +8.14% (versus 2.80% for random shortlisting), forwarded creatives reduced by over 40%, serving latency up only 0.2%, deployed in production.
GP-UCB Constrained Black-Box Lever Optimization — Booking.com uses GP-UCB to share information across adjacent configurations for optimizing budget allocation weights, with a synthetic simulator identifying a Matérn + Linear composite kernel as most robust; large-scale online RCT achieves statistically significant efficiency gains over ε-greedy.