LM Studio

Discover, download, and run open-weights LLMs entirely offline on your local machine with a modern visual UI and local API server

⚠️ CRITICAL VRAM & HARDWARE REQUIREMENT

VRAM is king for local LLMs: Large Language Models are heavily memory-bandwidth bound. Unlike video games that generate frames on the fly, an AI model must store all active weights, layer parameters, and working conversation memory (KV context cache) directly inside fast GPU Video RAM (VRAM) to produce usable generation speeds.

If a model's memory footprint exceeds your dedicated GPU VRAM, remaining model layers spill over into system RAM across the system PCIe bus (the "RAM cliff"). This causes token output to plummet from 45–80+ tokens/second down to an agonizing 1–3 tokens/second. Always pick model quantizations sized to fit fully inside your GPU's VRAM with headroom left for your active context window.

💡 TIP

LM Studio features a built-in Hugging Face model search engine that auto-detects your system GPU VRAM and visually indicates whether a model fits in full GPU offload, partial offload, or exceeds your memory limit.

Hardware Tiers & Recommended GPUs

To help plan your homelab or workstation build, here is a breakdown of GPU hardware setups ranging from a single budget-friendly graphics card up to high-end multi-GPU arrays and enterprise server clusters, along with the class of local models each tier can comfortably run.

Budget / Entry Single GPU

Tier 1: Budget

Example Hardware NVIDIA GeForce RTX 3060 12GB (approx. $200–$280 used) or RTX 4060 Ti 16GB

Dedicated VRAM Pool 12 GB – 16 GB GDDR6

Supported Model Tiers Lightweight Models (3B to 8B parameters) quantized to 4-bit or 5-bit (Q4_K_M / Q5_K_M) with standard 8k–16k context window buffers.

Performance & Role Generates 40–70 tok/sec. Perfect for personal writing assistants, terminal helpers, daily productivity tasks, and rapid summarization on a tight budget.

Mid-Range Workstation GPU

Tier 2: Mid-Range

Example Hardware NVIDIA GeForce RTX 3090 24GB (approx. $650–$750 used, community sweet spot) or RTX 4070 Ti Super 16GB / RTX 4080 16GB

Dedicated VRAM Pool 16 GB – 24 GB GDDR6X (up to 936 GB/s memory bandwidth on 3090)

Supported Model Tiers Mid-Tier Reasoning & Coding Models (14B to 32B parameters) at Q4_K_M, or dense 8B models at full uncompressed 8-bit precision (Q8_0) with 32k+ expanded context windows.

Performance & Role Generates 30–60 tok/sec. Ideal for heavy programming workflows, in-depth technical analysis, and complex multi-turn logic without cloud latency or monthly fees.

High-End Prosumer Dual-GPU Array

Tier 3: High-End

Example Hardware Dual (2x) NVIDIA RTX 3090 24GB or Dual (2x) RTX 4090 24GB (~$1,500–$3,800), or Apple Silicon Mac Studio (64GB–128GB unified memory)

Dedicated VRAM Pool 48 GB GDDR6X split across two PCIe x16 slots

Supported Model Tiers Flagship Heavyweight Models (70B parameters) quantized to 4-bit (Q4_K_M requires ~40–42GB VRAM), or mid-tier 32B models at full uncompressed precision with 64k+ context.

Performance & Role Generates 15–28 tok/sec. Enables true near-frontier logical reasoning, nuanced autonomous agent execution, and enterprise-grade software architecture fully offline.

Extreme Multi-GPU Cluster

Tier 4: Enterprise

Example Hardware Quad (4x) RTX 3090/4090 (96GB VRAM), Dual/Quad NVIDIA RTX 6000 Ada (96GB–192GB ECC VRAM), or 8x NVIDIA H100 / A100 80GB SXM5 Nodes (640GB VRAM) ($10,000–$300,000+)

Dedicated VRAM Pool 96 GB to 640+ GB ultra-high-speed memory with NVLink / SXM interconnects (up to 3.2 TB/s)

Supported Model Tiers Massive Frontier Models & Mixture-of-Experts (MoE) Clusters (120B to 400B+ parameters), uncompressed FP16 70B models, and simultaneous multi-tenant serving across organizations.

Performance & Role Production enterprise throughput serving hundreds of concurrent user requests simultaneously with massive 128k+ token context windows and zero precision loss.

•
The VRAM Math Breakdown: Model weights alone require approximately (Parameters × Bits) / 8 bytes. An 8B parameter model at 4-bit quantization consumes ~4.5 GB of VRAM for weights, plus 2–4 GB for runtime activations and context cache (~8 GB total). A 70B parameter model at 4-bit consumes ~38 GB for weights alone, requiring at least 44–48 GB VRAM for safe operation.
•
Context Window (KV Cache) Overhead: Every additional 1,000 tokens of conversation history kept in active context allocates additional GPU VRAM for key-value tensors. On large 70B models, pushing from 8k to 32k context can demand an additional 8 GB to 14 GB of pure VRAM.
•
Why VRAM Beats Raw Compute: An older card with high VRAM (like an RTX 3060 12GB or RTX 3090 24GB) will vastly outperform a newer gaming card with only 8GB VRAM (like an RTX 4060 8GB), because running out of VRAM halts GPU acceleration entirely.

LM Studio Core Features

•
In-App Hugging Face Hub: Search for any model file published on Hugging Face directly within the app. Filter by quantization (GGUF), architecture, downloads, and compatibility.
•
Visual GPU Layer Offloading: Real-time hardware slider controls how many transformer layers are offloaded into your GPU versus system RAM, with instant VRAM bar visualizers.
•
Local OpenAI-Compatible Server: Start an embedded HTTP server on http://localhost:1234/v1 that mirrors OpenAI's API format (/v1/chat/completions, /v1/models, /v1/embeddings). Instantly point tools like Cursor, Cline, Open WebUI, and Python scripts to your local machine.
•
Cross-Platform Hardware Backends: Fully supports NVIDIA CUDA on Windows and Linux, Apple Metal on macOS (leveraging unified memory), and AMD ROCm for Radeon hardware.
•
Chat Playground & System Prompts: Create customizable presets with custom system prompts, temperature dials, top-p, frequency penalties, and structured JSON generation.

Performance & Offloading Best Practices

•
Aim for 100% GPU Offload: Slide GPU offload to the maximum number of layers. Allowing even 2 or 3 layers to spill into CPU RAM forces the GPU to synchronize with the PCIe bus on every single token step, severely crippling inference speed.
•
Select the Q4_K_M Quantization Sweet Spot: Medium 4-bit quantization (Q4_K_M) preserves over 98% of full uncompressed model intelligence while slashing memory requirements by over 60%. If VRAM is tight, this is the gold standard format.
•
Enable Flash Attention: In LM Studio's model configuration panel, toggle Flash Attention on to reduce context memory usage by 20–30% and boost processing speed on long multi-turn chats.
•
Close Background GPU Consumers: Ensure heavy hardware-accelerated browsers, games, or video editors are closed to free up the extra 1–2 GB of VRAM that can make the difference between fitting a model or spilling to RAM.