method · first seen 4 Oct 2026
tensor parallelism
Also: TP, tensor-parallel, tensor model parallelism
Splitting individual weight matrices across multiple GPUs so each computes part of a layer's operations.
Mentions per week
PapersCodeDiscussion
Rank 199 on 16 Nov 2026 · 8 mentions in 28 days · 8 independent groups · papers → code · first seen 6 weeks ago · acceleration +0.00
Earliest sources
- Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offsPapers · 4 Oct 2026 · Javad Mirzaei, Jeebak Mitra
- christopherowen/dgx-spark-networkingCode · 4 Oct 2026 · christopherowen · 44 stars
- Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and InferencePapers · 9 Oct 2026 · Xingyu Xie, Xiaoyong Xie, Zhouchen Lin, Kim-Chuan Toh
- Scaling LLM Inference via Parallelism: Throughput-Latency Trade-offPapers · 14 Oct 2026 · Javad Mirzaei, Jeebak Mitra
- Lowaltitudeflight/local-llm-tuning-guideCode · 16 Oct 2026 · Lowaltitudeflight · 118 stars
- Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-SparkCode · 18 Oct 2026 · Elevatormusic · 84 stars
Latest sources
- Benchmarking LLM Inference on Desktop-Scale NVIDIA Grace Blackwell ClustersPapers · 15 Nov 2026 · Kevin Bhimani, Christian Escobar, Daniel Mihovch, Rudolf Eigenmann, Sunita Chand…
- Starscream: Reimagining LLM Sharding on AMD Instinct GPUs with GPU PartitioningPapers · 11 Nov 2026 · Sumanth Gudaparthi, Smeet Chheda, Ganesh Dasika
- Decode Token Parallelism: Memory-Efficient Distributed Attention Parallelism for Scalable LLM DecodingPapers · 9 Nov 2026 · Susav Shrestha, Brad Settlemyer, Narasimha Reddy
- ProprietaryLegal/Volta-sPLIt-DeepSeek-V4.1-Flash-1017-8xV100-SXM2-PP2xTP4Code · 3 Nov 2026 · ProprietaryLegal · 33 stars
- BasisKV: Value Latents for Communication and Key RoutingPapers · 2 Nov 2026 · Lixun Zhang, Zishan Shao, Kangning Cui, Yu Liang, Fan Yang, Yiran Chen
- Huage001/Ascend-H3Code · 30 Oct 2026 · Huage001 · 36 stars
- InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area NetworkPapers · 23 Oct 2026 · Juan Parra-Ullauri, Anthony Chiu, Hari Madhukumar, Zhaozhou Wu, Anderson Bravalh…
- jiayuqi7813/crack-open-models-dgx-sparkCode · 22 Oct 2026 · jiayuqi7813 · 73 stars
- Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-SparkCode · 18 Oct 2026 · Elevatormusic · 84 stars
- Lowaltitudeflight/local-llm-tuning-guideCode · 16 Oct 2026 · Lowaltitudeflight · 118 stars
- Scaling LLM Inference via Parallelism: Throughput-Latency Trade-offPapers · 14 Oct 2026 · Javad Mirzaei, Jeebak Mitra
- Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and InferencePapers · 9 Oct 2026 · Xingyu Xie, Xiaoyong Xie, Zhouchen Lin, Kim-Chuan Toh