Nvidia CEO Jensen Huang believes we should build more data centers in locations that are not used today to train AI models.
During Computex, it was actually Nvidia that took the cake with Nvidia CEO Jensen Huang being received like a rock star. During question period, a journalist shouts at him, “We love you,” which is greeted with laughter.
Whenever a question is asked about sustainability, we notice that Huang wants to provide clarity. He knows very well that his new chips require more and more energy. The latest Blackwell B200 chip requires a whopping 1,200 watts of power. However, he comes up with a clear answer.
“AI doesn’t care where you go to school. The world doesn’t have enough energy available near the population. He points to growing energy surpluses in big cities like Amsterdam or Taipei, where new businesses can no longer get power connections.
Meanwhile, the world has a lot of surplus energy. The sun can provide a huge amount of energy. The only drawback: this happens in places where people don’t live. We need to build data centers and power plants to train AI in those places.
Accelerated action
Training an AI model via deep learning requires a lot of energy. As this training progresses, you will not need any external connection and response time is not a concern. Once the data is available, training can begin. This is repeated continuously until the model becomes intelligent enough.
The end result is an LLM (Large Language Model) such as GPT-4, LLaMA 2, Claude or Gemini.
“Today, this training is accelerated through GPUs, which are more power efficient than a traditional CPU. A GPU can perform accelerated acceleration up to 99 percent more efficiently than a CPU. This in itself represents Significant savings in energy costs and that’s why you have tools like ChatGPT today that cannot train those models in a timely and efficient manner.
Heuristics must be improved
In addition to new data center locations, specifically for training AI models, Nvidia’s CEO sees a lot of room for progress in… Deduction-side. This is the aspect where the model is used in practice, after the training is over. For example, every question you ask on ChatGPT requires inference.
“Generative AI is not about training, it’s about inference. Many things today are not optimized enough. Take our weather simulation tool developed in Taiwan. Thanks to our hardware on the inference side, it works 3,000 times more efficiently than before. Accelerating the power of Computing is the key word here.”
“Don’t think about training, think about heuristics. This is where big energy gains can be made, making the whole thing more sustainable.”
And he didn’t say a word about the fact that that extra computing power means you can speed up more, thus pushing the limits again, resulting in higher power consumption.
