Tag: Scaling Laws

All the talks with the tag "Scaling Laws".

• The Era of 1-bit LLMs - A Brief Overview

  Adhilsha Ansad
  Published:Jul 22, 2024 at  at 02:00 PM

  This overview discusses the concept of 1-bit Large Language Models (LLMs) based on the paper "The Era of 1-bit LLMs - All Large Language Models are in 1.58 Bits". It presents BitNet b1.58, a 1-bit LLM variant that achieves competitive performance with full-precision Transformer LLMs while being more cost-effective in terms of latency, memory, throughput, and energy consumption. The overview highlights the potential of 1-bit LLMs in defining new scaling laws, training models, and designing hardware optimized for 1-bit LLMs.