Should you thought Nvidia’s GB200 rack techniques had been massive, CEO Jensen Huang is simply getting began. At GTC final month, the world’s most dear firm revealed plans to make use of photonic interconnects to pack greater than a thousand GPUs right into a single mammoth system by 2028.
The corporate is not ready to safe provide chains both. Over the previous month, the GPU large has invested billions in firms specializing in optics and interconnects, like Marvell, Coherent, and Lumentum, in preparation for the widespread deployment of those techniques.
“For everybody who’s in our ecosystem, we want much more capability,” Huang mentioned throughout his GTC keynote speech. “We’d like much more capability for copper; we want much more capability for optics; we want much more capability for CPO; and that is why we have been working with all of you to put the muse for this degree of progress.”
Nonetheless, Nvidia’s journey so far started a lot earlier. In actual fact, by the point OpenAI revealed ChatGPT to the world in late 2022, Nvidia already knew it had an issue.
On the time, the GPU large’s most potent techniques solely featured eight GPUs, and the fashions driving the AI increase required hundreds to coach. Nvidia wanted a much bigger field, or not less than a sooner community that would successfully distribute work throughout dozens of chips.
We caught our first glimpse of this with Nvidia’s Grace Hopper superchips in 2023, however it wasn’t till early 2024 that the total image got here into view. Unveiled at GTC that yr, the Grace Blackwell NVL72, a monstrous 120 kilowatt machine, makes use of a copper backplane containing miles of cables to make 36 nodes and 72 GPUs behave like one huge AI accelerator.
Copper was the pure alternative for this, Gilad Shainer, senior VP of networking at Nvidia, advised El Reg.
“Copper is the most effective connectivity, if you need to use it,” he mentioned. “It’s extremely price efficient, very low-cost, and consumes zero energy. It’s extremely dependable. There aren’t any energetic parts.”
However copper is not excellent. At 1.8 TB/s, the cables may solely stretch just a few ft earlier than the sign degraded as GPUs communicated with each other. Should you ever questioned why the NVL72’s NVSwitches are all within the middle of the rack, it is as a result of the runs had been that brief. Copper’s restricted attain additionally meant Nvidia needed to cram as many GPUs right into a single rack as potential.
Two years later, Nvidia is quickly approaching the boundaries of copper and might want to embrace optics if it desires to assemble an excellent greater GPU system.
The pluggable drawback
When Huang first confirmed off the NVL72 rack, codenamed Oberon, the one commercially viable option to join two accelerators optically would have been to make use of pluggable optics.
These modules are concerning the dimension of a pack of gum and include all of the lasers, retimers, and digital sign processing required to show electrical alerts into gentle and again once more.
Pluggables are nothing new in datacenter networks, however utilizing them for scale-up compute materials, like Nvidia’s NVLink, presents sure issues.
To succeed in the 1.8 TB/s of bandwidth, every Blackwell GPU would have required eighteen 800 Gbps pluggables: 9 for the accelerator, and one other 9 for the change. On their very own, these pluggables do not use that a lot energy – round 10-15 watts – however multiplied throughout 72 GPUs, that provides up fairly shortly.
As Huang famous in his 2024 GTC keynote speech, optics would have required a further 20,000 watts of energy.
Nonetheless, so much has modified for the reason that Oberon rack was first revealed. Developments in co-packaged optics (CPO), which integrates optical engines instantly alongside the change ASIC, have helped drive down energy consumption.
In 2025, Nvidia turned one of many first AI infrastructure suppliers to embrace CPO by integrating it instantly into its Spectrum Ethernet and Quantum InfiniBand switches. (Broadcom-based Micas Networks was making comparable strikes.)
This dramatically decreased the variety of pluggables required to construct an AI coaching cluster. Nonetheless, it was solely extra not too long ago that the corporate started discussing using optics and CPO for its NVSwitch materials.
NVLink goes optical
After pooh-poohing optical interconnects as too power-hungry two years earlier, Huang revisited the subject at GTC this spring by unveiling the Vera Rubin NVL576 and Rosa Feynman NVL1152, two multi-rack techniques that will use photonics to broaden their compute domains by an element of eight.
If NVL576 sounds acquainted, that is as a result of the quantity has come up earlier than. In actual fact, alongside the unique NVL72 rack, Nvidia teased a configuration with precisely that many GPUs, although to our information no such system was ever deployed within the wild.
Nvidia additionally briefly marketed its Vera Rubin Extremely Kyber racks below the NVL576 branding earlier than deciding that it did not really wish to depend every particular person GPU die as a standalone accelerator.
Until Nvidia’s advertising or roadmap adjustments once more, the precise Vera Rubin NVL576 will use a mixture of copper and optical interconnects.
“There’s quite a lot of dialog about ‘Is Nvidia going to copper scale up or optical scale up?’ We’ll do each,” Huang mentioned throughout this GTC keynote.
In line with Ian Buck, VP of Hyperscale and HPC at Nvidia, the primary layer of the community will use copper interconnects within the rack, which implies no adjustments to the GPUs. The second backbone layer will use pluggable modules.
We do not know precisely what topology Nvidia plans to make use of for this, however a two-tier fats tree will surely match the invoice, and would solely require a single rack’s value of switches (72 ASICs in whole) for the backbone layer.
For the modules themselves, pluggables could be the best possibility, however Nvidia may additionally go for near-packaged optics (NPO), like what Lightmatter confirmed off final month.
For Vera Rubin, Nvidia is simply speaking about optical scale for its Oberon NVL72 racks and never its NVL144 Kyber techniques.
We’re not precisely certain why Nvidia made the choice, however it’s value noting that for those who can scale up optically, you need not pack the whole lot into one rack. So, it could have merely made extra sense to help optical scale up throughout eight racks from a thermal and energy standpoint.
Nvidia Feynman goes co-packaged
The place issues actually begin to get fascinating is with Nvidia’s Feynman technology, which is meant to begin transport in mid-to-late 2028. We’re advised these techniques can be obtainable with both copper or co-packaged optical NVLink interconnects.
Nvidia is being considerably tight-lipped about how it will all work, however there are a few potential avenues.
The only possibility could be to combine CPO into the NVLink change ASIC and proceed utilizing copper interconnects within the rack.
This may require a two-tier NVSwitch material and two or three completely different change ASICs: one which’s half optical, one which’s solely optical, and sure one with out CPO.
Going this route would allow Nvidia to help a number of configurations just by swapping the NVLink change trays or wheeling in a backbone rack as wanted.
The extra fascinating risk could be to combine the CPO into each the change and the GPU package deal. This may nearly actually lead to a number of Feynman GPU SKUs – one with and one with out optics – however it will scale back the material to a single tier.
Talking with El Reg at GTC final month, Shainer declined to touch upon which method the corporate deliberate to maneuver ahead with, however highlighted some great benefits of a single-tiered compute material.
“Scale-up is one thing that you do not wish to construct a number of tiers if you do not have to, since you wish to reduce latency between the compute engines,” he mentioned.
Whereas potential to bake CPO into the GPU, a single tier NVL1152 system would require one helluva high-radix change. However with Feynman unlikely to ship till mid to late 2028, we suppose it is potential.
Securing the technique of manufacturing
Both possibility goes to wish a wholesome provide of laser modules. Whereas CPO strikes a lot of the optics and sign processing onto the package deal, lasers are normally stored separate for the aim of serviceability. This helps to elucidate the $4 billion ($2 billion every) Nvidia plowed into Coherent and Lumentum, each firms specializing in optical lasers, final month. If it will embrace CPO in a significant approach, the availability chain must be prepared.
Additional proof to recommend Nvidia is shifting to on-accelerator CPO is the corporate’s $2 billion tie up with Marvell introduced earlier this week.
As a part of that funding, Nvidia will work with Marvell to combine NVLink Fusion, a licensed model of its high-speed interconnect tech, into customized XPUs to be used with the GPU large’s Vera CPUs. The work may also lengthen to the event of optical I/O applied sciences, although to what extent the businesses did not elaborate.
As our sibling web site The Subsequent Platform mentioned earlier this week, Marvell’s $3.25 billion acquisition of Celestial AI may come into play right here.
The startup’s photonic interconnect tech may very well be used to construct a coherent reminiscence community spanning a number of racks, which may very well be simply as engaging to Nvidia as it will be to certainly one of Marvell’s greatest prospects, together with AWS. As you might recall, AWS is amongst Nvidia’s greatest NVLink Fusion prospects, with plans to make use of the tech in its next-gen Trainium4 compute clusters.
In any case, Nvidia has clearly seen the sunshine on optical scale-up, and we are able to count on CPO to play a a lot greater position in its system design shifting ahead. ®
Should you thought Nvidia’s GB200 rack techniques had been massive, CEO Jensen Huang is simply getting began. At GTC final month, the world’s most dear firm revealed plans to make use of photonic interconnects to pack greater than a thousand GPUs right into a single mammoth system by 2028.
The corporate is not ready to safe provide chains both. Over the previous month, the GPU large has invested billions in firms specializing in optics and interconnects, like Marvell, Coherent, and Lumentum, in preparation for the widespread deployment of those techniques.
“For everybody who’s in our ecosystem, we want much more capability,” Huang mentioned throughout his GTC keynote speech. “We’d like much more capability for copper; we want much more capability for optics; we want much more capability for CPO; and that is why we have been working with all of you to put the muse for this degree of progress.”
Nonetheless, Nvidia’s journey so far started a lot earlier. In actual fact, by the point OpenAI revealed ChatGPT to the world in late 2022, Nvidia already knew it had an issue.
On the time, the GPU large’s most potent techniques solely featured eight GPUs, and the fashions driving the AI increase required hundreds to coach. Nvidia wanted a much bigger field, or not less than a sooner community that would successfully distribute work throughout dozens of chips.
We caught our first glimpse of this with Nvidia’s Grace Hopper superchips in 2023, however it wasn’t till early 2024 that the total image got here into view. Unveiled at GTC that yr, the Grace Blackwell NVL72, a monstrous 120 kilowatt machine, makes use of a copper backplane containing miles of cables to make 36 nodes and 72 GPUs behave like one huge AI accelerator.
Copper was the pure alternative for this, Gilad Shainer, senior VP of networking at Nvidia, advised El Reg.
“Copper is the most effective connectivity, if you need to use it,” he mentioned. “It’s extremely price efficient, very low-cost, and consumes zero energy. It’s extremely dependable. There aren’t any energetic parts.”
However copper is not excellent. At 1.8 TB/s, the cables may solely stretch just a few ft earlier than the sign degraded as GPUs communicated with each other. Should you ever questioned why the NVL72’s NVSwitches are all within the middle of the rack, it is as a result of the runs had been that brief. Copper’s restricted attain additionally meant Nvidia needed to cram as many GPUs right into a single rack as potential.
Two years later, Nvidia is quickly approaching the boundaries of copper and might want to embrace optics if it desires to assemble an excellent greater GPU system.
The pluggable drawback
When Huang first confirmed off the NVL72 rack, codenamed Oberon, the one commercially viable option to join two accelerators optically would have been to make use of pluggable optics.
These modules are concerning the dimension of a pack of gum and include all of the lasers, retimers, and digital sign processing required to show electrical alerts into gentle and again once more.
Pluggables are nothing new in datacenter networks, however utilizing them for scale-up compute materials, like Nvidia’s NVLink, presents sure issues.
To succeed in the 1.8 TB/s of bandwidth, every Blackwell GPU would have required eighteen 800 Gbps pluggables: 9 for the accelerator, and one other 9 for the change. On their very own, these pluggables do not use that a lot energy – round 10-15 watts – however multiplied throughout 72 GPUs, that provides up fairly shortly.
As Huang famous in his 2024 GTC keynote speech, optics would have required a further 20,000 watts of energy.
Nonetheless, so much has modified for the reason that Oberon rack was first revealed. Developments in co-packaged optics (CPO), which integrates optical engines instantly alongside the change ASIC, have helped drive down energy consumption.
In 2025, Nvidia turned one of many first AI infrastructure suppliers to embrace CPO by integrating it instantly into its Spectrum Ethernet and Quantum InfiniBand switches. (Broadcom-based Micas Networks was making comparable strikes.)
This dramatically decreased the variety of pluggables required to construct an AI coaching cluster. Nonetheless, it was solely extra not too long ago that the corporate started discussing using optics and CPO for its NVSwitch materials.
NVLink goes optical
After pooh-poohing optical interconnects as too power-hungry two years earlier, Huang revisited the subject at GTC this spring by unveiling the Vera Rubin NVL576 and Rosa Feynman NVL1152, two multi-rack techniques that will use photonics to broaden their compute domains by an element of eight.
If NVL576 sounds acquainted, that is as a result of the quantity has come up earlier than. In actual fact, alongside the unique NVL72 rack, Nvidia teased a configuration with precisely that many GPUs, although to our information no such system was ever deployed within the wild.
Nvidia additionally briefly marketed its Vera Rubin Extremely Kyber racks below the NVL576 branding earlier than deciding that it did not really wish to depend every particular person GPU die as a standalone accelerator.
Until Nvidia’s advertising or roadmap adjustments once more, the precise Vera Rubin NVL576 will use a mixture of copper and optical interconnects.
“There’s quite a lot of dialog about ‘Is Nvidia going to copper scale up or optical scale up?’ We’ll do each,” Huang mentioned throughout this GTC keynote.
In line with Ian Buck, VP of Hyperscale and HPC at Nvidia, the primary layer of the community will use copper interconnects within the rack, which implies no adjustments to the GPUs. The second backbone layer will use pluggable modules.
We do not know precisely what topology Nvidia plans to make use of for this, however a two-tier fats tree will surely match the invoice, and would solely require a single rack’s value of switches (72 ASICs in whole) for the backbone layer.
For the modules themselves, pluggables could be the best possibility, however Nvidia may additionally go for near-packaged optics (NPO), like what Lightmatter confirmed off final month.
For Vera Rubin, Nvidia is simply speaking about optical scale for its Oberon NVL72 racks and never its NVL144 Kyber techniques.
We’re not precisely certain why Nvidia made the choice, however it’s value noting that for those who can scale up optically, you need not pack the whole lot into one rack. So, it could have merely made extra sense to help optical scale up throughout eight racks from a thermal and energy standpoint.
Nvidia Feynman goes co-packaged
The place issues actually begin to get fascinating is with Nvidia’s Feynman technology, which is meant to begin transport in mid-to-late 2028. We’re advised these techniques can be obtainable with both copper or co-packaged optical NVLink interconnects.
Nvidia is being considerably tight-lipped about how it will all work, however there are a few potential avenues.
The only possibility could be to combine CPO into the NVLink change ASIC and proceed utilizing copper interconnects within the rack.
This may require a two-tier NVSwitch material and two or three completely different change ASICs: one which’s half optical, one which’s solely optical, and sure one with out CPO.
Going this route would allow Nvidia to help a number of configurations just by swapping the NVLink change trays or wheeling in a backbone rack as wanted.
The extra fascinating risk could be to combine the CPO into each the change and the GPU package deal. This may nearly actually lead to a number of Feynman GPU SKUs – one with and one with out optics – however it will scale back the material to a single tier.
Talking with El Reg at GTC final month, Shainer declined to touch upon which method the corporate deliberate to maneuver ahead with, however highlighted some great benefits of a single-tiered compute material.
“Scale-up is one thing that you do not wish to construct a number of tiers if you do not have to, since you wish to reduce latency between the compute engines,” he mentioned.
Whereas potential to bake CPO into the GPU, a single tier NVL1152 system would require one helluva high-radix change. However with Feynman unlikely to ship till mid to late 2028, we suppose it is potential.
Securing the technique of manufacturing
Both possibility goes to wish a wholesome provide of laser modules. Whereas CPO strikes a lot of the optics and sign processing onto the package deal, lasers are normally stored separate for the aim of serviceability. This helps to elucidate the $4 billion ($2 billion every) Nvidia plowed into Coherent and Lumentum, each firms specializing in optical lasers, final month. If it will embrace CPO in a significant approach, the availability chain must be prepared.
Additional proof to recommend Nvidia is shifting to on-accelerator CPO is the corporate’s $2 billion tie up with Marvell introduced earlier this week.
As a part of that funding, Nvidia will work with Marvell to combine NVLink Fusion, a licensed model of its high-speed interconnect tech, into customized XPUs to be used with the GPU large’s Vera CPUs. The work may also lengthen to the event of optical I/O applied sciences, although to what extent the businesses did not elaborate.
As our sibling web site The Subsequent Platform mentioned earlier this week, Marvell’s $3.25 billion acquisition of Celestial AI may come into play right here.
The startup’s photonic interconnect tech may very well be used to construct a coherent reminiscence community spanning a number of racks, which may very well be simply as engaging to Nvidia as it will be to certainly one of Marvell’s greatest prospects, together with AWS. As you might recall, AWS is amongst Nvidia’s greatest NVLink Fusion prospects, with plans to make use of the tech in its next-gen Trainium4 compute clusters.
In any case, Nvidia has clearly seen the sunshine on optical scale-up, and we are able to count on CPO to play a a lot greater position in its system design shifting ahead. ®
