Meta Unveils the new version of its open-source large language model, Llama 3

Meta has launched Meta Llama, the latest version of its open-source large language models (LLMs), which follows Llama 2. The initial Llama 3 models have 8 billion and 70 billion parameters, but Meta plans to roll out models with more than 400 billion parameters in the future.

Training for Llama 3 utilized a dataset that is over ten times larger than those used for its predecessors, encompassing diverse data types like code and multilingual text. The architecture of Llama 3 has been improved, featuring an enhanced tokenizer for more efficient user input processing and a grouped query attention feature for better comprehension of contextual information, thereby increasing text processing accuracy. To facilitate the training of Llama 3, Meta developed a custom training infrastructure using server clusters with 24,000 GPUs each. This infrastructure includes software that identifies and resolves technical issues, boosting efficiency.

Llama 3 models will soon be available on Amazon Web Services, Databricks, Google Cloud Platform, Hugging Face, Kaggle, IBM WatsonX, Microsoft Azure, NVIDIA NIM, Snowflake, and more importanly, the recently launched Meta AI assistant integrated into Messenger, Instagram, WhatsApp, and Facebook Feed, to improve user interaction and accessibility.