Unlocking AMD's Llama Adventure

AMD's Love for Llamas

AMD is really into llamas lately, and it's not just a quirky fascination. Let's break down why AMD is so interested in llamas and what it means for their latest projects.

Llama in the Open Source Community

• The open-source community is head over heels for Llama, Meta's open-source large language model.
• Many new models in the generative AI world are built on top of Llama 2, and developers are using it for all sorts of things.
• AMD doesn't want to miss out on the llama love!

AMD's Impressive Showcase

• Lisa Su, the CEO of AMD, showcased the new AMD Instinct MI300X Accelerator.
• MI300X performs 1.2 times better than NVIDIA H100 on a single kernel when running Meta's Llama 2 70B.
• For inferencing with Llama 2, AMD's MI300X performs 1.4 times faster than an H100 server with eight MI300X units.

Microsoft and AMD Collaboration

• Kevin Scott, the CTO of Microsoft, expressed excitement about using MI300X on Azure.
• He eagerly awaits the performance of GPT-4 and Llama 2 on MI300X in production.

Emphasis on Software Innovation

• Lisa Su stressed the importance of software as the real driver of innovation.
• ROCm 6, the latest version of AMD's open-source parallel computing offering, is introduced as an alternative to NVIDIA's CUDA.

AMD's Ecosystem Strength

• Victor Peng, AMD's President, showcased how a strong ecosystem helped create a successful open-source framework in ROCm.
• MI300X with ROCm 6 is eight times faster than MI250X with ROCm 5 for inferring Llama 2 70B.

Groundbreaking Meta Partnership

• Meta is partnering with AMD to use MI300X to build its data centers.
• MI300X is trained to be the fastest design-to-deployment solution in Meta's history.

Shift to AMD at Meta

• Meta is shifting from using 150k H100 GPUs for research and training to AMD MI300X.
• This move is part of Meta's investment in building new experiences and advancing open technologies.

Open Approach Advocacy

• Ajit Matthews, Meta AI's senior director of engineering, emphasized the importance of an open ecosystem for better and safer technology.
• Matthews highlighted AMD's focus on open software like ROCm and open hardware with MI300X.

Broad Partnership History

• Meta and AMD have been working together on EPYC CPUs since 2019.
• Meta recently deployed AMD's Genoa and Bergamo based servers at scale.

AMD's Edge Computing Focus

• AMD is stepping into edge computing with Ryzen AI PCs and the Ryzen AI software.
• Lisa Su demonstrated that the newest version of AMD's on-device neural processing unit, Ryzen 8040, makes Llama 2 7B perform 1.4 times faster than previous versions.

Conclusion

In conclusion, AMD's love for llamas isn't just a quirky interest; it's deeply connected to their advancements in AI and open-source contributions. The llama journey continues, and AMD is at the forefront of this unique intersection of technology and llamas.