Europe/London
BlogMay 9, 2026

AnythingLLM, MCP, and the Quiet Arrival of Local Context for AI Agents

Dubz
A few weeks after the MCP post went up, Anthropic published an open-source harness for code-aware agents that does something we have been quietly waiting for: it turns local code and docs into context an agent can actually use, without sending the source to a third-party model. Combined with AnythingLLM and the Model Context Protocol, it is the first clean path to "local context" that does not feel like a science project. This post is about why that matters and how to think about it. The repository anthropics/defending-code-reference-harness is a reference implementation for an agent that can scan a codebase, identify a security issue or a refactor opportunity, and propose a patch that the developer reviews. The interesting part is not the model. It is the plumbing: how the agent decides what to read, how it grounds its answer in the local files, and how it produces a diff that points at the exact lines it claims to fix. For indie builders, the takeaway is simple. The reference architecture is in the open. You do not have to design the context layer yourself, and you do not have to ship your codebase to a hosted service to get this behavior. Most of the past two years of agent work has been about cramming more text into the prompt window. That works up to a point, and then it stops working for a simple reason: a 200,000-token context window is not the same thing as a 200,000-token understanding of your codebase. The model has read the words. It has not built a map. Local context is different. The agent indexes your repository, your docs, your tickets. It retrieves the relevant chunks at query time. The chunks can be small. The model gets to focus. That is the part that matters for any team whose codebase is large, proprietary, or regulated. AnythingLLM has been moving in this direction for a while, with a desktop and self-hosted runtime that lets you point a vector store at your own files. MCP is the protocol that lets any client talk to that store without bespoke glue code. The Anthropic harness is the third piece: a working agent that uses this stack to do something concrete. If you want to try the workflow today, the rough shape is:
  1. Run AnythingLLM locally, or on a small VPS, and point it at the repositories and docs you want the agent to know about.
  2. Expose that AnythingLLM workspace as an MCP server. The community has writeups on this; treat the specifics as version-dependent.
  3. Connect your agent client (Claude Code, an IDE plugin, or a custom CLI) to that MCP server.
  4. Run the Anthropic reference harness, or fork it, against the same workspace.
You now have an agent whose context is your local files, whose actions go through a protocol you can audit, and whose model weights you can choose. None of those four steps are magic. Each of them is a separate project, which is itself the point: you can swap any of them. Three things make this stack worth your time, and one thing does not. The thing that is worth your time:
  • Privacy by construction. Your code does not leave your machine. That is a feature for any indie builder working on something they plan to sell.
  • Swappable parts. AnythingLLM is not the only option. The Anthropic harness is not the only agent. MCP is the contract that lets you swap parts without rewriting your workflow.
  • A real reference implementation. You can read the harness source, run it, and learn from it without depending on a vendor blog post.
The thing that is not worth your time:
  • Treating this as production-ready on day one. It is a reference. Read the security notes, audit the network calls, and do not point it at your production database until you have done that work.
The interesting thing about this moment is that none of the three pieces (AnythingLLM, MCP, the Anthropic harness) is owned by the same company. They are independent projects that happen to compose. That is the version of the AI ecosystem indie builders should want to live in. The moment one of these pieces locks you in, the open-source version of this stack stops being open. For now, it is.
  • Anthropic, defending-code-reference-harness on GitHub: https://github.com/anthropics/defending-code-reference-harness
  • AnythingLLM: https://github.com/Mintplex-Labs/anything-llm
Share this post: