AIForum.

How to Use Ollama for Code Completion in VS Code with Continue

On this page

THE SHORT ANSWER

Continue is an open-source AI code assistant for VS Code and JetBrains, and Ollama can serve the models that power its inline completions entirely on your own machine. This guide walks through installing both, configuring the autocomplete role with a local Qwen2.5-Coder model, verifying that suggestions appear, and diagnosing the failures that most often stop completions from working.
Quick facts

Continue is an open-source AI code assistant that runs inside VS Code and JetBrains editors, and Ollama can serve the models that power it. Used together, they provide inline code completion that runs on your own machine: you install the Continue extension, point its autocomplete role at a local Ollama model such as qwen2.5-coder:1.5b, and ghost-text suggestions appear as you type. This guide covers the setup, the configuration format that currently works, and the failure modes that most often stop completions from appearing.

What you are actually configuring

Continue separates model usage into roles. Autocomplete is its own role, distinct from the models you use for chat or for editing existing code. That separation matters because the two jobs have different requirements: autocomplete needs to return a short suggestion quickly on nearly every keystroke, while chat can afford a slower, larger model. Continue’s documentation recommends small, fast models specifically for the autocomplete role, and notes that thinking-type models are typically not recommended there because the extra reasoning latency works against the interaction.

Ollama plays a different part. It is the local runtime and API server that downloads, stores, and executes the model. Continue talks to it over HTTP, so the model does not need to be loaded inside the editor process. Ollama’s own blog describes the pairing as a way to run an AI code assistant entirely on your laptop, or to have Ollama deployed on a server that powers completion and chat remotely.

A practical consequence: you can mix local and hosted models. A small local model can handle autocomplete while a larger hosted model handles chat, or you can keep everything local if privacy or offline use is the priority.

Prerequisites

  • VS Code installed and working.
  • Ollama installed on the same machine, or reachable on your network. The Ollama blog links to the official download page for the latest version.
  • Enough memory for the model you choose. Continue’s documentation recommends the 1.5B Qwen2.5-Coder variant for local autocomplete, which is the smallest option it lists for the role. Model weights are only part of the picture; context and KV cache plus runtime overhead also consume memory, so do not size your machine from parameter count alone.
  • The Continue extension installed in VS Code.
  • Optional: a Mistral API key if you later want to try Codestral instead of a local model.

Step 1: Install Ollama and pull an autocomplete model

Continue’s autocomplete guide recommends Ollama for local use and names Qwen2.5-Coder 1.5B and 7B as the best open models for the autocomplete role. The documented command to fetch the smaller one is:

ollama run qwen2.5-coder:1.5b

This downloads the model and drops you into an interactive session in the terminal. You can type a prompt to confirm it responds, then exit. The download is the part that matters: once the model is present locally, Continue can request completions from it through Ollama’s local API, which listens on the default Ollama port.

Expected result: the command completes without a download error, and the model appears in the list of locally available models. Running ollama list in a terminal is a quick way to confirm it is there.

If you prefer to keep the terminal free, the same model can be pulled without opening a chat session, but the ollama run form is what the Continue documentation shows and is the simplest thing to verify first.

Step 2: Install the Continue extension in VS Code

Install Continue from the VS Code Marketplace, then reload the window if VS Code asks. After installation, Continue adds a panel to the editor where you can see model status and error messages. That panel is the first place to look whenever completions do not appear, because it surfaces configuration and connection problems that are otherwise invisible.

Continue also exposes a configuration file that you will edit in the next step. The exact menu path and file location are shown in the extension itself; the important thing is that you edit Continue’s own config, not VS Code’s settings.json.

Step 3: Point the autocomplete role at Ollama

Current Continue configurations use YAML with schema: v1. The documented local-autocomplete configuration looks like this:

name: My Config
version: 0.0.1
schema: v1
models:
  - name: Qwen 1.5b Autocomplete Model
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete

Three details carry most of the weight here:

  • provider: ollama tells Continue to talk to a local Ollama server rather than a hosted API.
  • model: qwen2.5-coder:1.5b must match the tag you actually downloaded. A mismatch produces a model-not-found error rather than a completion.
  • roles: - autocomplete is what makes this model the one used for inline suggestions. A model entry without the autocomplete role will not produce ghost text, even if it works fine in chat.

If Ollama is running on a different machine than VS Code, you will need to point Continue at that host’s API address rather than the local default. Continue’s documentation demonstrates an apiBase setting in its Mistral/Codestral example, so check the current Ollama provider reference for the field name your version expects before assuming the same key applies.

Step 4: Verify that completions work

  1. Open any source file in VS Code and start typing a function or a familiar pattern.
  2. Pause for a moment. Continue should render a grey inline suggestion after the cursor.
  3. Accept the suggestion with the Tab key, or dismiss it and keep typing.
  4. If nothing appears, trigger a completion manually using Continue’s keyboard shortcut (the default manual trigger in VS Code is Alt+). A manual trigger that works while automatic suggestions do not usually points to a debounce or timing issue rather than a broken model connection.

When this works, the loop is: keystroke → Continue builds a prompt from surrounding code → Ollama runs the model → suggestion returns → ghost text renders. Each stage can fail independently, which is why the next section separates them.

Local versus hosted autocomplete models

Continue’s documentation publishes a recommendation table for the autocomplete role. It is a provider recommendation, not a benchmark, and it should be read that way:

Model roleBest open modelsBest closed modelsNotes
AutocompleteQwen2.5-Coder (1.5B), Qwen2.5-Coder (7B)Codestral, Mercury CoderClosed models are slightly better than open models

The tradeoff is not only quality. A local Ollama model keeps your code on your machine and works without a network connection, but it competes with the rest of your system for memory and produces suggestions at whatever speed your hardware allows. A hosted model such as Codestral avoids local resource pressure but sends your code to a third party and requires an API key.

If you do try Codestral, note two documented details. First, the API keys for Codestral and Mistral’s general APIs are different, so a general Mistral key may not work for Codestral. Second, if you are sharing a key across a team or otherwise want to use the general endpoint, the documentation shows setting apiBase to https://api.mistral.ai/v1.

If you are following an older tutorial

Older Continue guides, including the May 2024 guest post on the Ollama blog, show a JSON configuration with a top-level tabAutocompleteModel key and a models array using title, provider, and model fields. That format is visibly different from the current YAML-with-roles format shown above.

This is the single most common reason a copied configuration appears to do nothing. If your config file is JSON and your extension version expects YAML with schema: v1, the autocomplete role may never be registered. Check which format your installed version reads before debugging anything else, and prefer the current documentation over older blog snippets.

Troubleshooting: when no completions appear

Work through these in order. Each check is cheap and eliminates a whole category of cause.

  1. Is the model actually downloaded? Run ollama list. If qwen2.5-coder:1.5b is missing, the config points at something that does not exist.
  2. Is Ollama running? If the server is not up, Continue cannot reach it. Starting Ollama and retrying is the fastest test.
  3. Does the model tag match exactly? qwen2.5-coder:1.5b and qwen2.5-coder:7b are different downloads. A typo yields a not-found error rather than a suggestion.
  4. Is the autocomplete role present? A model entry without roles: - autocomplete will not drive inline completions.
  5. Is the config format correct for your version? See the previous section on YAML versus JSON.
  6. Are you using a thinking-style model? Continue’s documentation advises against these for autocomplete, since the extra reasoning step adds latency to an interaction that needs to feel immediate. Switch to a small completion-oriented model.
  7. Are completions slow rather than absent? This is usually a memory or model-size problem. Dropping from the 7B to the 1.5B variant is the documented local recommendation and the simplest thing to try.
  8. Is it a Codestral key problem? If you switched to Codestral, confirm you are using a Codestral API key rather than a general Mistral key, and check the apiBase setting if you are pointing at the general endpoint.

After each change, re-run the verification in Step 4 rather than assuming the fix worked.

Hardware, privacy, and licensing tradeoffs

Memory. The 1.5B model is recommended for local autocomplete precisely because it is small. Larger models may produce better suggestions but need correspondingly more memory, and memory use includes context and KV cache on top of the weights.

Concurrency. Ollama can run multiple models and handle concurrent requests, which is what makes the split between a local autocomplete model and a separate chat model practical. On a machine with limited memory, running both at once may not be viable; the Ollama blog suggests trying each configuration and deciding whether you prefer local autocomplete or local chat.

Privacy. A purely local Ollama setup keeps code on your machine, which is the main reason teams adopt this pattern. Moving any role to a hosted provider changes that property for the code sent to it.

Licensing. Do not treat local availability as permission to use a model commercially. The Ollama blog notes that Codestral 22B carries a non-production license and, as of that May 2024 post, was described as suitable for research and testing rather than daily local use. Always check the license that ships with the specific model you download.

Next steps

Once autocomplete is stable, the natural extensions are adding a separate chat model, testing whether the 7B variant is worth the extra memory on your hardware, and comparing a local model against a hosted one on your own codebase rather than on general impressions. Because Continue’s configuration format has changed over time, keep a copy of the working config so you can diff it after an extension update.

If you hit something this guide does not cover, share your editor version, the model tag you downloaded, and the exact error text from the Continue panel in the community — those three details resolve most configuration problems quickly.

FAQ

Can I get code completion in VS Code fully offline with Ollama and Continue?

Yes. Ollama's blog describes running Continue with Ollama entirely on your own laptop, or with Ollama deployed on a server you control. Keeping the autocomplete model local means the code used for suggestions stays on your machine.

Which model should I use for local autocomplete?

Continue's documentation recommends Ollama for local use and names Qwen2.5-Coder 1.5B and 7B as the best open models for the autocomplete role. The 1.5B variant is the smaller and more forgiving option for limited hardware.

Why does my autocomplete model not produce any suggestions?

Check in this order: the model is downloaded (ollama list), Ollama is running, the model tag in your config matches the download exactly, and the model entry includes roles: - autocomplete. A model configured without that role will work in chat but not for inline completion.

Why does a configuration copied from an older tutorial do nothing?

Older Continue guides use a JSON config with a top-level tabAutocompleteModel key, while current configurations use YAML with schema: v1 and model roles. If the format does not match what your installed version reads, the autocomplete role may never be registered.

Can I use Codestral instead of a local model?

Yes. Continue's documentation recommends Codestral through the Mistral API for the best autocomplete experience, but notes that Codestral API keys differ from Mistral's general API keys, and that you set apiBase to https://api.mistral.ai/v1 if you use the general endpoint. The Ollama blog also notes that Codestral 22B carries a non-production license.

References

  1. https://docs.continue.dev/customize/deep-dives/autocomplete/
  2. https://www.ollama.com/blog/continue-code-assistant

Related terms

Autocomplete role
A role assigned to a model entry in Continue's configuration that designates it for inline ghost-text suggestions, separate from chat or edit roles.
Ollama
A local runtime and API server that downloads, stores, and executes language models on your own machine or a server you control.
Qwen2.5-Coder
An open model family listed by Continue's documentation as the best open option for the autocomplete role, in 1.5B and 7B sizes.
tabAutocompleteModel
A key used in older Continue JSON configurations to select the autocomplete model; current configurations use YAML with schema v1 and model roles instead.
Codestral
A Mistral model that Continue's documentation lists among the best closed models for autocomplete; the Ollama blog notes its 22B version carries a non-production license.

Leave a Reply

Your email address will not be published. Required fields are marked *