Unlocking AMD's Llama Adventure

Unlocking AMD's Llama Adventure

AMD's Love for Llamas

AMD is really into llamas lately, and it's not just a quirky fascination. Let's break down why AMD is so interested in llamas and what it means for their latest projects.

Llama in the Open Source Community

  • The open-source community is head over heels for Llama, Meta's open-source large language model.
  • Many new models in the generative AI world are built on top of Llama 2, and developers are using it for all sorts of things.
  • AMD doesn't want to miss out on the llama love!

AMD's Impressive Showcase

  • Lisa Su, the CEO of AMD, showcased the new AMD Instinct MI300X Accelerator.
  • MI300X performs 1.2 times better than NVIDIA H100 on a single kernel when running Meta's Llama 2 70B.
  • For inferencing with Llama 2, AMD's MI300X performs 1.4 times faster than an H100 server with eight MI300X units.

Microsoft and AMD Collaboration

  • Kevin Scott, the CTO of Microsoft, expressed excitement about using MI300X on Azure.
  • He eagerly awaits the performance of GPT-4 and Llama 2 on MI300X in production.

Emphasis on Software Innovation

  • Lisa Su stressed the importance of software as the real driver of innovation.
  • ROCm 6, the latest version of AMD's open-source parallel computing offering, is introduced as an alternative to NVIDIA's CUDA.

AMD's Ecosystem Strength

  • Victor Peng, AMD's President, showcased how a strong ecosystem helped create a successful open-source framework in ROCm.
  • MI300X with ROCm 6 is eight times faster than MI250X with ROCm 5 for inferring Llama 2 70B.

Groundbreaking Meta Partnership

  • Meta is partnering with AMD to use MI300X to build its data centers.
  • MI300X is trained to be the fastest design-to-deployment solution in Meta's history.

Shift to AMD at Meta

  • Meta is shifting from using 150k H100 GPUs for research and training to AMD MI300X.
  • This move is part of Meta's investment in building new experiences and advancing open technologies.

Open Approach Advocacy

  • Ajit Matthews, Meta AI's senior director of engineering, emphasized the importance of an open ecosystem for better and safer technology.
  • Matthews highlighted AMD's focus on open software like ROCm and open hardware with MI300X.

Broad Partnership History

  • Meta and AMD have been working together on EPYC CPUs since 2019.
  • Meta recently deployed AMD's Genoa and Bergamo based servers at scale.

AMD's Edge Computing Focus

  • AMD is stepping into edge computing with Ryzen AI PCs and the Ryzen AI software.
  • Lisa Su demonstrated that the newest version of AMD's on-device neural processing unit, Ryzen 8040, makes Llama 2 7B perform 1.4 times faster than previous versions.


In conclusion, AMD's love for llamas isn't just a quirky interest; it's deeply connected to their advancements in AI and open-source contributions. The llama journey continues, and AMD is at the forefront of this unique intersection of technology and llamas.

Read more