Article Source

• Large Concept Models (LCMs)

Large Concept Models (LCMs)

Abstract

In this video, we dive into Large Concept Models (LCMs), an innovative architecture from a recent Meta paper titled: “Large Concept Models: Language Modeling in a Sentence Representation Space”.

Unlike Large Language Models (LLMs), LCMs operate on higher-level semantics, processing concepts instead of tokens, making them more akin to human reasoning, and possibly positioning them as a future rival to the current token-based LLM architecture.

Throughout the video, we explain what reasoning in the abstract concepts space means, and explore the Large Concept Models architecture, diving into few architecture options.

Specifically, two LCMs are based on Diffusion Models, so we provide a brief reminder about Diffusion Models and then explain the Diffusion-based LCMs.

• Paper - https://arxiv.org/abs/2412.08821
• Code - large_concept_model
• Written review - Large Concept Models (LCMs) by Meta: The Era of AI After LLMs?