AMD's Love for Llamas
AMD is really into llamas lately, and it's not just a quirky fascination. Let's break down why AMD is so interested in llamas and what it means for their latest projects.
Llama in the Open Source Community
- The open-source community is head over heels for Llama, Meta's open-source large language model.
- Many new models in the generative AI world are built on top of Llama 2, and developers are using it for all sorts of things.
- AMD doesn't want to miss out on the llama love!
AMD's Impressive Showcase
- Lisa Su, the CEO of AMD, showcased the new AMD Instinct MI300X Accelerator.
- MI300X performs 1.2 times better than NVIDIA H100 on a single kernel when running Meta's Llama 2 70B.
- For inferencing with Llama 2, AMD's MI300X performs 1.4 times faster than an H100 server with eight MI300X units.
Microsoft and AMD Collaboration
- Kevin Scott, the CTO of Microsoft, expressed excitement about using MI300X on Azure.
- He eagerly awaits the performance of GPT-4 and Llama 2 on MI300X in production.
Emphasis on Software Innovation
- Lisa Su stressed the importance of software as the real driver of innovation.
- ROCm 6, the latest version of AMD's open-source parallel computing offering, is introduced as an alternative to NVIDIA's CUDA.
AMD's Ecosystem Strength
- Victor Peng, AMD's President, showcased how a strong ecosystem helped create a successful open-source framework in ROCm.
- MI300X with ROCm 6 is eight times faster than MI250X with ROCm 5 for inferring Llama 2 70B.
Groundbreaking Meta Partnership
- Meta is partnering with AMD to use MI300X to build its data centers.
- MI300X is trained to be the fastest design-to-deployment solution in Meta's history.
Shift to AMD at Meta
- Meta is shifting from using 150k H100 GPUs for research and training to AMD MI300X.
- This move is part of Meta's investment in building new experiences and advancing open technologies.
Open Approach Advocacy
- Ajit Matthews, Meta AI's senior director of engineering, emphasized the importance of an open ecosystem for better and safer technology.
- Matthews highlighted AMD's focus on open software like ROCm and open hardware with MI300X.
Broad Partnership History
- Meta and AMD have been working together on EPYC CPUs since 2019.
- Meta recently deployed AMD's Genoa and Bergamo based servers at scale.
AMD's Edge Computing Focus
- AMD is stepping into edge computing with Ryzen AI PCs and the Ryzen AI software.
- Lisa Su demonstrated that the newest version of AMD's on-device neural processing unit, Ryzen 8040, makes Llama 2 7B perform 1.4 times faster than previous versions.
In conclusion, AMD's love for llamas isn't just a quirky interest; it's deeply connected to their advancements in AI and open-source contributions. The llama journey continues, and AMD is at the forefront of this unique intersection of technology and llamas.