NVIDIA explained that in the booming AI industry, some models still take months to train. And the main problem on the way to progress is this – the performance gain starts to decrease as the number of GPUs in the data center increases.
NVIDIA claims that its new Hopper architecture will help solve this problem. It can speed up the training of Transformer models on H100 GPUs up to six times faster than previous generation chips. Now training can take place in days, not weeks.
The fastest supercomputer will be built on the Hopper architecture and will include 4600 H100 processors. The system will only be used for NVIDIA internal research and will be operational in a few months.