Choosing local inference is not choosing a worse product out of stubbornness. It is a deliberate architectural decision with genuine costs on both sides.
There is a fifth arrangement worth naming as a hybrid: the remote development environment, where your code and commands live on a machine you control but that is not the laptop in front of you. It behaves like local execution from the agent’s point of view and like remote execution from your network’s.
For most cloud-backed tools, the model runs server-side, on the vendor’s cloud or behind a third-party model provider. Your machine sends a request, a remote server runs the inference, meaning the actual computation that produces a response, and sends back an answer. You are not running a local model. You are making a network call.
So what do you mean by “topology”?
Combine those three layers and you get four arrangements that cover most of what developers actually use. They are not interchangeable.
Reading an implementation noun as a category label is how people end up believing they know more about a vendor’s infrastructure than the documentation actually says.
- Model inference. The computation that turns your request into a response.
- Command execution. Shell commands, file edits, test runs, anything the agent actually does.
- Code storage and indexing. Where a working copy of your repository lives, and where any search index over it is built and kept.
One is a legal commitment about deletion timelines. The other is a statement about topology. Both can be true at once, or one can be true without the other. An ownership boundary and a physical boundary are not the same guarantee, even when the underlying hardware looks identical.
The four topologies
Three questions, and no product name answers them: where the model runs, where commands execute and what leaves your machine. Check the mode and configuration, not the brand.
- Local interface with remote inference.Code suggestions or chat responses are generated remotely, but no commands execute. The editor is a client. Autocomplete and non-agentic chat both sit here, which is why treating them as automatically separate topologies adds names without necessarily adding architectural distinctions. Once chat can invoke tools or execute commands, it has moved into another category.
- Local agent with remote inference.The model runs remotely, while file changes and commands execute on your machine. Anthropic documents that Claude Code runs locally and sends prompts and model outputs over the network to reach the model. Cursor’s CLI agent has tools for file operations, searching, running shell commands and web access: the file operations and shell commands run against the host environment where the CLI was launched, while inference and some connected services remain remote.
- Cloud agent with remote execution.Model inference remains remote, while the repository working copy and command execution operate inside a vendor-managed environment. The inference service and the execution environment may themselves be separate systems: GitHub bills cloud agent sessions in Actions minutes and AI credits separately, and lets you change the model for a task, which only makes sense if the model is not part of the environment running your tests. Your machine is not the execution surface and does not need to stay online.
- Fully local or fully self-hosted agent.Inference and command execution both remain on local or organisation-controlled infrastructure. The word “fully” is carrying weight there, because a vendor may describe execution as self-hosted while retaining the control plane or the agent loop in its own cloud. This is the only arrangement where source code need not cross an external boundary, and it is the hardest to achieve completely.
Self-hosted arrangements complicate this further, and Cursor’s own documentation is unusually clear about the split: on self-hosted paths, tool calls run on hardware you control while the agent loop still runs in Cursor’s cloud. Partly local is not fully local, and that is the vendor saying so rather than a critic inferring it.
Where is the model actually running?
Think of it like a phone call routed through an unknown number of relay stations. You speak, something listens, and an answer comes back, but the route in between is invisible unless you go looking. The analogy starts to break down when you realise that each relay station in an AI coding setup may have different permissions to copy, store, or act on your data. A phone call does not do that. But the basic shape of the comparison holds: the experience at each end can look identical even when the path is completely different.
Topology here means the physical and organisational shape of a running system: where code gets processed, where commands get executed, and whose machines are involved.
The mode determines the topology, not the brand.
Where inference runs is not where commands execute
This is the distinction that catches people off guard. Even if you are clear on whether your model is local or remote, there is a second question: when the agent actually does something, runs a shell command, modifies a file, calls an API, where does that happen?
Tools like Ollama, LM Studio, and Continue pointed at a local backend work differently: they run the model on your own hardware. No data leaves your machine during inference itself. That sounds like a straightforward win for privacy, and in many respects it is, but it comes with a real tradeoff. Local inference generally involves some combination of reduced capability, slower speed, or a narrower context window compared with hosted frontier models, and that gap tends to widen on long, complex agentic tasks, the exact kind where you would most want the model to be good.
Even in autocomplete mode, where the interaction feels lightest, something is being transmitted. Depending on the product, configuration and request, transmitted context may include portions of the active file, nearby functions, referenced files, type information or retrieved repository content. Results come back and are written locally. You never see the upload happen, and the experience feels local because the output appears locally.
If you need persistent, parallel throughput, cloud agents may justify their additional risks. Local agents can also run in parallel through separate processes or worktrees, but they consume local resources and require the machine to remain available. Cursor documents that its cloud agents can run many tasks concurrently and do not require the developer’s machine to remain online.
The same tool, in a different mode, can behave like a fundamentally different machine. Not a little different. Different in where computation happens, whose infrastructure it runs on, and what it can touch on your system. GitHub Copilot autocomplete and a Copilot cloud agent may share a product name, but they occupy very different positions in the execution stack. GitHub itself draws the line: its documentation distinguishes the autonomous cloud agent, which works in a GitHub Actions-powered environment, from agent mode in the IDE, which makes edits directly in your local development environment.
What happens to your code while all this is running?
If code cannot leave your network, only a fully local or fully self-hosted arrangement works, and switching on a local model is not enough. Continue with Ollama can keep inference local, but keeping the overall session inside your boundary also requires controlling embeddings, search, telemetry and external tool calls. These connections are easy to miss. Anthropic documents that Claude Code sends only the requested hostname to its API for a safety check before WebFetch retrieves a URL, and that the check occurs regardless of the model provider
An AI coding system can span three distinct layers, and each may operate in a different location:
The contractual layer and the physical layer are not the same layer
Cloud agents go further. Because they need to act on a codebase rather than a few surrounding lines, they generally create a remote working copy of the assigned repository or branch. GitHub scopes this narrowly by default: its documentation states that Copilot can only make changes in the repository specified when the task starts, and by default can only access context in that same repository. For the duration of the task, a working copy of your code exists on infrastructure you do not own.
This is where procurement conversations go wrong. A Zero Data Retention agreement is a meaningful contractual control, but its coverage can depend on the account, model and feature being used. It is still fundamentally different from saying that the data never left the machine.
What does this mean in practice?
Teams can combine these modes, but local execution should never be mistaken for a network boundary. With remote inference, relevant code context still leaves the machine. What remains local is the command execution, not necessarily the data path.
These tools are built to feel seamless, and seamlessness hides topology. Read the documentation for the specific mode you are using, and check its date.
A cloud agent works differently, and the vendors do not implement it the same way. GitHub’s Copilot cloud agent works inside an ephemeral, GitHub Actions-powered development environment, with a documented ceiling of 59 minutes per session. Cursor’s cloud agents run in isolated VMs with full development environments, described as isolated Ubuntu machines. Other products use containers or comparable isolated environments. The useful category is “vendor-managed execution environment.” The implementation underneath is a per-product fact you have to look up, not something you can infer from the category.
When people talk about “their AI coding agent,” they usually picture a single product doing a single job. A helpful assistant living in the editor, suggesting code, running tasks. One thing, in one place. That picture is often wrong, and the gap between the picture and the truth matters more than many developers realise.
A local agent runs commands on your machine, against your filesystem and your environment. Depending on its sandbox and permission settings, the agent may be able to access files, credentials and services available to your user account. Approval prompts, allowlists and directory restrictions all narrow that surface, and they vary by tool and by configuration. What does not vary is that the blast radius is yours, so it is worth knowing which restrictions are actually switched on rather than assuming the defaults are conservative.
