Issues

• Muse Spark token embedding gets no gradient under TP

  pytorch/torchtitan#5915 · Filter by author anijain2305 opened on Nov 7, 2026

• Question about TitanRL blog

  pytorch/torchtitan#5914 · Filter by author yifanjiang-goaly opened on Nov 6, 2026

• [Feature Request] Independent replication degrees for dense parameters and routed experts

  pytorch/torchtitan#5901 · Filter by author johnjunjun7 opened on Nov 6, 2026
  3

  

• FLUX: tensor_parallel_degree / expert_parallel_degree > 1 are silently ignored and fail later as OOM

  pytorch/torchtitan#5854 · Filter by author wangdefeng1 opened on Nov 5, 2026

• DeepSeek-V3 16B fails at init with TP=8 and EP=8 and dp_shard ≥ 16:FSDP does not support uneven sharding on dim 1

  pytorch/torchtitan#5739 · Filter by author jenniew opened on Nov 2, 2026
  9

  

• [RL][ROCm] Support vLLM graph replay with batch-invariant FlexAttention

  module: rocm
  pytorch/torchtitan#5725 · Filter by author rishisinhanj opened on Nov 1, 2026

• [RFC] DeepSeek-V4: align the sparse-attention stack with torchtitan's components

  pytorch/torchtitan#5720 · Filter by author sdmyzlp opened on Oct 31, 2026
  2

  

• [RFC] Support additional training diagnostics in TorchTitan

  pytorch/torchtitan#5692 · Filter by author NaDongsfd opened on Oct 29, 2026
  7

• RFC: Consistent last-words checkpoints for TorchFT training

  pytorch/torchtitan#5637 · Filter by author Skuld7451 opened on Oct 27, 2026
  

• [graph_trainer] sac_and_offload reloads stale data when a forward in-place write goes through a view

  pytorch/torchtitan#5628 · Filter by author andrewliu2001 opened on Oct 26, 2026

• [loss] Confident targets get exactly 0 loss and target gradient in _LossParallelCrossEntropy

  pytorch/torchtitan#5623 · Filter by author catherinelee274 opened on Oct 25, 2026

• TorchFT-based training suspension and resource yielding for higher-priority workloads

  pytorch/torchtitan#5620 · Filter by author chango9543 opened on Oct 25, 2026
  1

  

• # [RFC] Decoupling TorchTitan Test Cases from Hardware

  pytorch/torchtitan#5613 · Filter by author lixing-he opened on Oct 24, 2026
  4

• [RFC][MultiModal] Model-generic vision encoder parallelism for multimodal training

  pytorch/torchtitan#5595 · Filter by author QIU023 opened on Oct 23, 2026

• [RFC] Let custom routed-expert layouts declare their FSDP shard dimension

  pytorch/torchtitan#5545 · Filter by author samansg-aws opened on Oct 19, 2026
  5

• [qwen3] No local_compile region covers the block's norm/residual seams

  pytorch/torchtitan#5537 · Filter by author yuxinwan-amd opened on Oct 19, 2026
  3

  

• [RFC] Optional dependencies and cross-device testing

  pytorch/torchtitan#5502 · Filter by author JavaZeroo opened on Oct 18, 2026
  1

• [RFC][rl] GRPO on Intel XPU: torchtitan + Monarch + TorchStore

  pytorch/torchtitan#5401 · Filter by author songhappy opened on Oct 15, 2026
  3

• [CI] GraphTrainer EP recipes retain pre-EP sharding after config construction change (#5280)

  pytorch/torchtitan#5361 · Filter by author sanketpurandare opened on Oct 13, 2026
  2

  

• [CI] GPT-OSS and Qwen3 model goldens fail after batch-wise MoE balancing (#4772)

  pytorch/torchtitan#5360 · Filter by author sanketpurandare opened on Oct 13, 2026
  1

  

• [DistMuon] Performance and memory optimization

  pytorch/torchtitan#5278 · Filter by author JavaZeroo opened on Oct 11, 2026

• [remat] SelectiveAC replay exposes attention projection placeholders to RoPE

  pytorch/torchtitan#5173 · Filter by author sanketpurandare opened on Oct 8, 2026
  

• [RFC] Integrate NCCL RMA into TorchTitan for zero-SM pipeline parallel communication

  pytorch/torchtitan#5150 · Filter by author DAMI211 opened on Oct 8, 2026
  

• CUDAGraph + Copy-on-write tensor lazy copy produce NaN output

  pytorch/torchtitan#5125 · Filter by author wwwjn opened on Oct 7, 2026
  

• [graph_trainer] Preserve reduce-dtype accumulation for singleton FSDP gradients

  pytorch/torchtitan#5043 · Filter by author sanketpurandare opened on Oct 3, 2026
  2