Document Library

Reference architectures, white papers, and solutions briefs to help build and enhance your network infrastructure, at any level of deployment.

Optimizing Large Language Models with the OpenVINO™ Toolkit

Last Updated: Mar 12, 2025

Large language models (LLMs) have enabled breakthroughs in natural language understanding, conversational AI, and diverse applications such as text generation and language translation. Additionally, large language models are massive, often over 100 billion parameters and growing. A recent study published in Scientific American1 by Lauren Leffer articulates the challenges with large AI models, including scaling to small devices, accessibility when disconnected from the internet, as well as power consumption and environmental concerns.

Download PDF