The project saw the commitment of all available resources and the best of technology took to the field
Parameterizing all the facets of human language was the challenge posed by this interesting project, born from the partnership between companies Microsoft e Nvidia.
The result is a machine with an artificial neural network that has been given the name of Megatron-Turing Natural Language Generation (MT-NLG). This particular artificial intelligence has seen a lot of time and resources go into the field by big tech companies. However, it can be said that they have largely paid off given the development of a unique, innovative product with out-of-the-ordinary specifications.
READ ALSO: Do you love the vegetable garden and high tech? here is the vertical one of Living Farming Tree
MT-NLG exceeded by 3 times what was considered the best artificial neural network, OpenAI’s GPT-3, thanks to 530 billion parameters. It therefore ranks well above what was the range of machines known to date and for this very reason we look at it with great curiosity, especially in view of its practical applications.
The Megatron-Turing Natural Language Generation project and its technical characteristics
The project was trained on Microsoft Azure NDv4 and Nvidia’s Selene machine learning supercomputer. It also consists of 560 DGX A100-type servers, consisting of eight 80GB A100 GPUs.
The data storage aspect is the hot topic of this product. His is a truly significant assest, on an exponential scale, which has been given the name of The Pile. It contains 825 GB of text data. These were selected through the contents of the network, such as Wikipedia but also scientific publications.
READ ALSO: The new robot that flies on the skateboard: here is Leonardo
AI is therefore able to process texts and develop what could be considered embryonic critical thinking. Furthermore, by doing so, important foundations have been laid on what is robotic training and production on the consumer market, thus re-measuring the production costs that are currently prohibitive.