Chat quota reached
Well, the month is still young, but I already ran out of free tokens for my favorite LLM service. So I decided to run a local instance of an open-source LLM on my machine.
What could possibly go wrong?
With a few free tokens left, I let AI guide me through the process.
First try: Ollama
Getting Started with Ollama
Wrap-up and check if "Ollama is running":
curl http://localhost:11434
Connect with Continue
Continue - Ship faster with Continuous AI
Ok, so let's follow: Using Ollama with Continue: A Developer's Guide
npm install -g @continuedev/cli
Check Recommended Models for best performance.
Let's tune the selection a bit as suggested in What Are the Best Practices for Ollama / Ollama (Local Models):
For code generations:
ollama pull qwen3-coder:30b
This results in the following config .continue/config.yaml:
name: Local Config
version: 1.0.0
schema: v1
models:
# https://hub.continue.dev/ollama/gpt-oss-20b?view=config
- name: gpt-oss-20b
provider: ollama
model: gpt-oss:20b
roles:
- apply
- chat
- edit
capabilities:
- tool_use
# https://hub.continue.dev/ollama/qwen3-coder-30b?view=config
- name: qwen3-coder-30b
provider: ollama
model: qwen3-coder:30b
roles:
- apply
- chat
- edit
capabilities:
- tool_use
# https://hub.continue.dev/ollama/llama3.1-8b?view=config
- name: llama3.1-8b
provider: ollama
model: llama3.1:8b
roles:
- apply
- autocomplete
- chat
- edit
# https://hub.continue.dev/ollama/nomic-embed-text-latest?view=config
- name: nomic-embed-text-latest
provider: ollama
model: nomic-embed-text:latest
roles:
- embed
continue config set model llama3.1-8b
How To Run an Open-Source LLM on Your Personal Computer
Sample project setup
Once Ollama and continue are installed, you can kick off the project with the continue CLI:
Please create both the AGENTS.md file and the .continue/rules/review.md file using the Write tool after analyzing the repository.
Focus on providing actionable information that would help both AI agents and human developers understand and work effectively with this codebase.
Keep the files concise but informational.
While not 100% right on the first run:
...the generated AGENTS.md contains a reference to a non-existing file CONTRIBUTING.md.
Despite this small issue, working with Continue and Ollama was a great start to get experience and a feeling for how local models can enrich the development.
While inside the editor of my choice I could