Definition: A fabric is a high-speed, interconnected network or infrastructure layer that links compute, storage and services so they behave as one system. In AI, it usually refers to the network connecting GPU servers and storage within a cluster or data center.Why It Matters: AI training and large-scale inference depend on moving huge volumes of data between nodes quickly. If the fabric is slow or congested, expensive GPUs sit idle waiting for data. The fabric therefore sets the ceiling on cluster scale and performance. For enterprises, fabric design affects cost, reliability and how quickly models can be trained or served. However, high-performance fabrics are expensive and need specialized skills to tune and monitor.Key Characteristics: AI fabrics emphasize high bandwidth, low latency and low packet loss. They use tiered or mesh topologies so that any node can reach any other with predictable performance. Technologies include InfiniBand, RDMA over Converged Ethernet (RoCE) and high-speed Ethernet. Congestion control, traffic engineering and monitoring are essential, and the fabric is usually separated into networks for compute traffic, storage traffic and management.
A fabric connects servers through layers of switches in a topology designed so that traffic has many equal paths between any two points. During distributed training, GPUs exchange gradients and activations constantly, and collective operations require all participants to synchronize. The fabric moves that data using protocols that bypass much of the operating system, such as RDMA, to reduce latency and CPU load. Congestion control and load balancing spread traffic across links and prevent hot spots. Storage traffic, which feeds training data to GPUs, often runs on a separate network or class of service so that it does not interfere with compute traffic. Telemetry tracks link utilization, errors and latency, so that operators can detect and fix problems before they slow jobs.ProsA well-designed fabric keeps GPUs busy and lets clusters scale to many nodes with predictable performance. It also provides flexibility to add capacity without redesigning the network.ConsHigh-performance switches, cables and optics are costly, and tuning is specialized. Faults such as congestion or a failing link can slow an entire distributed job.
Distributed Model Training: A research team trains a large model across hundreds of GPUs. A low-latency fabric allows gradient exchange to complete quickly, keeping utilization high.Shared Storage for Datasets: An enterprise provides high-throughput access to a shared dataset over a dedicated storage network so that every training node reads data at full speed.Multi-Node Inference: A provider serves a very large model sharded across several servers. The fabric carries intermediate results between them with minimal delay, keeping response times low.
Switched Fabrics in Storage (1990s): Fibre Channel fabrics connected servers and storage arrays through switches, popularizing the term.High-Performance Computing Interconnects (2000s): InfiniBand and similar technologies linked supercomputer nodes with very low latency.Data Center Fabrics (2010s): Leaf-spine architectures and large Ethernet fabrics scaled cloud data centers.GPU Communication Libraries (2016-2020): Software for collective communication between GPUs made fabric performance central to training speed.RoCE and Ethernet for AI (2020-2023): Ethernet-based approaches with RDMA and improved congestion control provided alternatives to InfiniBand.Current Practice and Trends (2023-Present): AI-specific fabrics are designed for very large GPU clusters, with emphasis on congestion management, reliability and observability.
When to Use: Invest in fabric design when training or serving models across multiple servers, or when storage throughput limits GPU utilization. For single-server workloads, standard networking is usually sufficient.Designing for Reliability: Design for redundancy so that a single link or switch failure does not stop a job. Test the fabric under realistic collective operations before putting it into production. Keep firmware and configuration consistent across devices.Operating at Scale: Monitor link utilization, errors and latency continuously. Use topology-aware scheduling so jobs are placed close together. Plan for growth by choosing designs that can add nodes without major rework.Governance and Risk: Segment management, storage and compute networks, and restrict administrative access to fabric devices. Keep configuration under version control, and patch network firmware on a schedule.
