Beyond Chat: How to Set Up Local LLM Code Completion in VS Code (Without Copilot)

Local LLM code completion in VS Code has matured to the point where you can replicate the core Copilot experience entirely on your own hardware, with zero data leaving your machine. This tutorial walks you through the full setup, from model selection to a production-ready configuration file you can copy and paste today.

Table of Contents

• Why Companies Are Banning Copilot and What You Can Do About It
• Architecture Overview: How Local Code Completion Works
• Prerequisites and Hardware Requirements
• Step 1: Installing and Configuring Ollama
• Step 2: Setting Up Continue in VS Code
• Step 3: Optimizing for Real-World Code Completion
• Practical Demo: Local Code Completion in a React Project
• How Does It Compare to GitHub Copilot?
• Troubleshooting Common Issues
• The Future of Local AI-Assisted Development

Why Companies Are Banning Copilot and What You Can Do About It

The Data Privacy Problem with Cloud-Based AI Code Assistants

In 2023, Samsung banned generative AI tools including ChatGPT for employees after engineers inadvertently pasted proprietary source code into the service, resulting in internal data leaks. Around the same time, Apple restricted internal use of ChatGPT and GitHub Copilot, citing concerns about confidential data exposure. JPMorgan Chase was reported in the same wave of restrictions.

These are not edge cases. Cloud-based code completion tools work by sending your code context to remote servers. That means file contents, function signatures, variable names, and sometimes file paths travel outside your network. For organizations subject to GDPR, HIPAA, or strict NDA requirements, this creates a compliance problem that no terms-of-service clause fully resolves.

The Local Alternative Is Now Viable

Two shifts made local code completion practical. First, open-weight code models shrank. A 7B parameter model can run on a laptop with 16 GB of RAM and produce genuinely useful completions. Second, the tooling caught up. Ollama makes serving models locally trivial, and extensions like Continue bridge the gap between a local model and the inline completion experience developers expect. If you have an Apple Silicon Mac, a modern NVIDIA GPU, or even a capable CPU, you can set this up in under 30 minutes.

Cloud-based code completion tools work by sending your code context to remote servers. That means file contents, function signatures, variable names, and sometimes file paths travel outside your network.

Architecture Overview: How Local Code Completion Works

The Three-Layer Stack

The local code completion pipeline has three components:

1. Model runtime: Ollama serves models through a local HTTP API on your machine.
2. Code model: A specialized model like Qwen2.5-Coder, DeepSeek Coder, or Code Llama that understands code syntax and patterns.
3. VS Code integration: The Continue extension connects to the local API and provides inline tab-autocomplete and chat functionality.

How It Differs from Copilot

With Copilot, every keystroke that triggers a suggestion sends context to GitHub's cloud infrastructure, where a model generates a response and streams it back. With the local stack, the VS Code extension sends a request to localhost:11434, Ollama runs inference on your hardware, and the response never touches any external network. The difference is architectural, not cosmetic. Your code stays on your machine, period.

Prerequisites and Hardware Requirements

Minimum and Recommended Specs

• Minimum: 16 GB RAM, modern CPU (Apple M1 or newer, Intel i7 10th gen or newer). No dedicated GPU required for 7B models.
• Recommended: 32 GB RAM or a GPU with 8+ GB VRAM (NVIDIA RTX 3060 or better) for running 13B+ models with comfortable headroom.
• Sweet spot: Apple M2 or M3 with 16 to 32 GB of unified memory handles 7B to 13B models well, since the GPU and CPU share the same memory pool.

Keep in mind that the context window size (num_ctx) heavily affects memory consumption, sometimes more than the parameter count itself.

Software Requirements

• VS Code (recent version)
• Ollama (latest release)
• Continue extension
• Terminal access for installation and testing

Step 1: Installing and Configuring Ollama

Installation

Install Ollama using the method appropriate for your operating system:

# macOS (using the official installer)
# Note: Ollama is available as a direct download from https://ollama.com/download
# Homebrew cask install:
brew install --cask ollama
Linux (official install script)
curl -fsSL https://ollama.com/install.sh | sh
Windows
Download the installer from https://ollama.com/download
Verify installation on any OS
ollama --version

After installation, start the Ollama service. On macOS and Windows, launching the Ollama application starts the background service automatically. On Linux, you may need to run ollama serve in a separate terminal session, or enable and start the systemd service if one was created by the installer.

Pulling a Code Completion Model

Not all models are equal for code completion. Here is a quick guide:

• Qwen2.5-Coder 7B: Strong balance of quality and speed. Supports fill-in-the-middle (FIM) prompting, which is critical for inline completions. This is the model this tutorial recommends for most setups.
• DeepSeek Coder V2: Higher quality ceiling, especially in larger variants. Reports competitive benchmark results against closed-source models on code intelligence tasks. Note that this is a mixture-of-experts model, so its memory requirements may differ from what you'd expect based on total parameter count alone.
• Code Llama 7B: Lighter weight, decent for basic completions on constrained hardware.

An important distinction: "instruct" model variants are fine-tuned for chat-style interactions, while "base" or standard variants are better suited for code completion where the model continues or fills in code. For tab-autocomplete, prefer a model with FIM support.

Pull and test your chosen model:

# Pull the model (this downloads several GB on first run)
ollama pull qwen2.5-coder:7b
Run an interactive test to verify it loads and responds
ollama run qwen2.5-coder:7b "Write a Python function that reverses a string"

Testing the Local API

Ollama exposes a local HTTP API on port 11434. Confirm it is working with a direct curl request:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "def fibonacci(n):\n  ",
  "stream": false,
  "options": {
    "temperature": 0.2,
    "num_ctx": 2048
  }
}'

You should receive a JSON response containing the model's completion. If you see a connection error, verify Ollama is running with curl http://localhost:11434 (which should return a simple "Ollama is running" message).

Step 2: Setting Up Continue in VS Code

Why Continue?

Continue is an open-source extension that supports both chat and tab-autocomplete with local models. It works with Ollama out of the box and has a large active user base on the VS Code Marketplace. Alternatives exist: Tabby takes a self-hosted server approach with its own model management, and Llama Coder offers a simpler but more limited experience. Continue strikes the best balance of features and flexibility for most developers.

Installation and Initial Setup

1. Open VS Code, go to the Extensions panel, and search for "Continue".
2. Install the Continue extension (published by Continue).
3. After installation, Continue creates a configuration file. Open it through the Continue sidebar panel or navigate to it directly. The location is typically ~/.continue/config.json on macOS and Linux, or %USERPROFILE%\.continue\config.json on Windows. Note that Continue has been migrating to a config.yaml format in newer versions, so check the Continue documentation for your version's specific configuration approach.

Here is the complete configuration file for Ollama integration. This is the config you can copy, paste, and start using immediately:

{
  "models": [
    {
      "title": "Qwen2.5 Coder 7B (Local)",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder Autocomplete",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b",
    "apiBase": "http://localhost:11434"
  },
  "tabAutocompleteOptions": {
    "debounceDelay": 500,
    "maxPromptTokens": 2048,
    "disable": false
  }
}

Save this file, and Continue will connect to your local Ollama instance.

Verifying the Connection

Open any code file and start typing. After a brief pause (controlled by the debounce delay), you should see inline ghost text suggestions. You can also open the Continue chat panel from the sidebar and send a message to confirm the chat model is responding.

If nothing appears, check the Continue output panel in VS Code (View > Output > Continue) for error messages.

The difference is architectural, not cosmetic. Your code stays on your machine, period.

Step 3: Optimizing for Real-World Code Completion

Tuning Autocomplete Performance

The default settings work, but tuning them to your hardware and workflow makes a noticeable difference:

{
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder Autocomplete",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b",
    "apiBase": "http://localhost:11434"
  },
  "tabAutocompleteOptions": {
    "debounceDelay": 350,
    "maxPromptTokens": 4096,
    "multilineCompletions": "always",
    "disable": false
  }
}

• debounceDelay: Controls how long (in milliseconds) Continue waits after you stop typing before requesting a completion. Lower values (300ms) feel snappier but increase load. Higher values (700ms) reduce unnecessary requests. Start at 350 to 500 and adjust.
• maxPromptTokens: Determines how much surrounding code context gets sent to the model. More context means better suggestions but slower responses. 2048 is conservative; 4096 is a good middle ground if your hardware keeps up.
• multilineCompletions: When set to "always", the model can suggest entire blocks of code rather than single lines.

For the model itself, keep temperature low (0.1 to 0.3) for completions. Higher temperatures introduce creativity, which is useful in chat but counterproductive when you want predictable, syntactically correct code.

Context Awareness: Helping the Model Understand Your Project

Create a .continueignore file in your project root to prevent noisy or irrelevant files from polluting the model's context:

node_modules/
dist/
build/
.git/
*.lock
*.min.js

In chat mode, Continue supports context providers like @file and @codebase to pull in specific files or search your project. Note that full codebase indexing requires an embeddings model to be configured separately. Consult the Continue documentation for your version's specific requirements around codebase indexing.

Choosing the Right Model Size for Your Hardware

• 1.5B models (e.g., Qwen2.5-Coder 1.5B): Fastest inference, lowest memory usage, but noticeably less accurate. Good for very constrained hardware or when you prioritize speed above all.
• 7B models: The sweet spot. Useful completions across most languages, manageable memory footprint (roughly 4 to 8 GB depending on quantization).
• 13B to 33B models: Approaching Copilot-level quality for many tasks. Requires 8 to 16+ GB of VRAM or substantial unified memory.

Quantization also matters. Models in Ollama are typically distributed as quantized GGUF files (served via the llama.cpp backend). Lower-bit quantizations (like Q4) reduce memory usage and increase speed at some cost to quality. Higher-bit quantizations (like Q8) preserve more model fidelity but use more resources. Ollama's default quantization for most models is a reasonable starting point.

Practical Demo: Local Code Completion in a React Project

If you are getting started with React or want a refresher, SitePoint's guide to learning React covers the fundamentals.

Autocompleting a React Component

With Ollama running and Continue configured, open a new .tsx file and start writing a component. Here is what the experience looks like:

// You type this:
import { useState } from 'react';
interface Todo {
id: number;
text: string;
completed: boolean;
}
export function TodoList() {
const [todos, setTodos] = useState([]);
const [input, setInput] = useState('');
// LLM suggests the following when you start typing "const add":
const addTodo = () => {
if (input.trim() === '') return;
setTodos([
...todos,
{ id: Date.now(), text: input, completed: false }
]);
setInput('');
};
// LLM suggests toggle and delete functions as you continue:
const toggleTodo = (id: number) => {
setTodos(
todos.map(todo =>
todo.id === id
? { ...todo, completed: !todo.completed }
: todo
)
);
};
}

The model picks up on the Todo interface, the state variables, and the naming patterns you established. Completions appear as ghost text that you accept with Tab.

Using Chat for Refactoring

Select a block of code in VS Code, open the Continue chat panel, and describe what you want. For example, selecting a class component and typing "Convert this to a functional component with hooks" produces a refactored version:

import React, { useState } from 'react';
// Before: Class component
class Counter extends React.Component {
state = { count: 0 };
render() {
return (
 this.setState({ count: this.state.count + 1 })}>
Count: {this.state.count}

);
}
}
// After: Continue's suggested refactor
function Counter() {
const [count, setCount] = useState(0);
return (
 setCount(count + 1)}>
Count: {count}

);
}

The quality of these refactors depends on the model size and context provided. A 7B model handles straightforward transformations well. More complex refactors involving multiple files benefit from larger models.

AI code assistance is shifting from a cloud-only luxury to a local-first commodity.

How Does It Compare to GitHub Copilot?

Honest Assessment

Where local wins: privacy, cost, offline capability, and the freedom to customize models and configuration without vendor lock-in. Where Copilot wins: raw suggestion quality (particularly for complex multi-file completions), speed on lower-end hardware (since inference happens on powerful cloud GPUs), and workspace-level context understanding. The gap is real but narrowing.

Troubleshooting Common Issues

Model Not Loading or Slow Responses

Verify Ollama can see your model and check resource usage. Run ollama list to confirm the model is downloaded. If responses are slow, try a smaller model or a more aggressively quantized variant. Close other memory-heavy applications to free up RAM or VRAM.

No Autocomplete Suggestions Appearing

First, confirm Ollama is running:

curl http://localhost:11434

If that returns a response, check the Continue output panel in VS Code for errors. The most common culprits are a mismatched model name in config.json (the name must exactly match what ollama list shows), a wrong or missing apiBase URL, or tabAutocompleteOptions.disable accidentally set to true.

Suggestions Are Low Quality

Switch to a code-specialized model if you are using a general-purpose one. Increase maxPromptTokens to give the model more surrounding context. And verify you are using a model variant that supports fill-in-the-middle prompting, since standard causal models can only complete forward from a prefix, which limits their usefulness for mid-line edits.

If you set up the stack described in this tutorial today, you will be well positioned as models improve and tooling matures. Your completions will get better with every model upgrade, and your code will never leave your machine.

The Future of Local AI-Assisted Development

Code models are getting smaller and better simultaneously. The DeepSeek-Coder-V2 family demonstrated that open models can approach closed-source performance on code intelligence benchmarks. Inference optimization techniques like speculative decoding are closing the speed gap between local and cloud-based completion. And Microsoft's ongoing work on Language Model APIs within VS Code suggests that local model integration may eventually become a first-class, standardized feature of the editor itself.

The trajectory is clear: AI code assistance is shifting from a cloud-only luxury to a local-first commodity. If you set up the stack described in this tutorial today, you will be well positioned as models improve and tooling matures. Your completions will get better with every model upgrade, and your code will never leave your machine.

Related: VS Code Extensions for Productive Development