How to Run AI Agents Locally on Linux

Last updated: September 2026

Quick answer. "Running an agent locally" almost always means the agent process runs on your machine while the model runs in someone else's datacenter, and the two have completely different privacy and failure properties. On Linux the practical setup is: install the agent, turn its sandbox on because it ships off, deny the credential paths because the default read scope is your whole computer, and decide separately whether you also want the weights local via Ollama or vLLM. Start with claude --version and a settings.json that sets sandbox.enabled: true, then verify what actually happened with yeet run github:yeet-src/exectop --comm claude rather than trusting the agent's own transcript.

I build kernel-side tooling and I watch engineers set these things up, which is a narrow vantage point: I see the moment after somebody has installed an agent and before they have any idea what it is doing. I do not run your fleet, I have not audited the model you are about to point at your source tree, and I am not going to tell you which agent to pick, because on Linux they are more alike than the comparison posts suggest. What I keep seeing is that the word "locally" is carrying two meanings in the same sentence, and the setup people end up with is not the one they thought they were choosing.

What does "running an AI agent locally" actually mean on Linux?

It means the agent process is on your machine, and it usually does not mean the model is. Claude Code, Codex CLI, Cursor's CLI, Aider and Cline all install as local programs: they run as your user, read your filesystem directly, spawn subprocesses on your box, and hold your session state on local disk. The inference that decides what those programs do happens over an API call to a hosted model.

That split is the whole story, and it is worth being precise about because the two halves fail differently. The local half determines what the agent can reach: your files, your sockets, your credentials, your other processes. The remote half determines where your code goes: into a request body, over TLS, to a vendor. Someone worried about the first is asking a Linux access-control question. Someone worried about the second is asking a data-handling question. Most "how do I run agents locally" threads have both people in them, arguing past each other.

There is a third arrangement, genuinely local in both halves, where the weights run on your own GPU through Ollama, llama.cpp or vLLM and nothing leaves the machine. That is a real option in 2026 and it is a different tradeoff rather than a strictly better one, which is covered further down.

Which AI agent should I install on Linux, and does the choice matter?

Less than the setup does. Claude Code, Codex CLI, Aider, Cline and the self-hosted forks all know Linux administration well, all shell out through a Bash-equivalent tool, all read files directly, and all inherit your user's permissions when they do. The differences that show up in daily use are context handling, cost, and how the harness recovers when a command fails, and none of those change what the agent can touch on your host.

What does change materially is the sandbox each one ships and whether it is on. That is a per-agent fact and it is the first thing to check after installing:

claude --version
cat ~/.claude/settings.json 2>/dev/null || echo "no settings file yet"

The second command matters more than the first. An agent with no settings file is running whatever its defaults are, and defaults are chosen so the install works everywhere rather than for your threat model. That is a reasonable choice by the vendor and a bad one to inherit silently, because the permissive default is invisible: nothing prompts you, nothing warns you, and the agent behaves identically whether you configured it or not.

Does an AI coding agent run with my full user permissions on Linux?

Yes, unless you configured something. An agent launched from your shell is a process with your uid, which means every file you can read it can read, every socket you can open it can open, and every binary on your $PATH it can execute. The Linux permission model has no way to express "this process is an agent" because it was designed for humans, and it grants on identity rather than on intent.

Anthropic documents where that leaves Claude Code specifically. From the sandboxing page:

"Default read behavior: read access to the entire computer, except certain denied directories. Note that this default still allows reading credential files such as ~/.aws/credentials and ~/.ssh/."

The same page states why both isolation layers matter: "Without network isolation, a compromised agent could exfiltrate sensitive files like SSH keys. Without filesystem isolation, whether from a permissive policy or from disabling the filesystem layer, a compromised agent could backdoor system resources to gain network access."

That is a vendor documenting the sharp edge of its own default, which is more than most do. It is also the single most useful fact for someone setting up on Linux this week, and it sits two clicks deep in a docs site nobody reads before installing.

How do I turn on the sandbox before I let an agent touch my repo?

Set it in ~/.claude/settings.json and confirm it applied. The minimum useful configuration turns on both layers and denies the credential paths by name:

{
  "sandbox": {
    "enabled": true,
    "filesystem": { "enabled": true },
    "network": { "allowedDomains": ["github.com", "*.npmjs.org"] },
    "credentials": {
      "files": [
        { "path": "~/.aws/credentials", "mode": "deny" },
        { "path": "~/.ssh", "mode": "deny" }
      ],
      "envVars": [
        { "name": "GITHUB_TOKEN", "mode": "deny" },
        { "name": "NPM_TOKEN", "mode": "deny" }
      ]
    }
  }
}

filesystem.enabled is the layer that is off by default, and with it off the docs are explicit that "sandboxed commands get unrestricted read and write access to the host filesystem". credentials.files denies the reads that the default read scope would otherwise permit. envVars unsets the named variables before each sandboxed command, which closes the path where a token never touches disk but sits in the environment anyway.

Two limits worth knowing before you rely on this. The sandbox governs the agent's Bash subprocesses, and the file protection is part of the filesystem layer, so disabling that layer takes the credential file denials with it. And the network layer decides by hostname supplied by the client, which the docs note can be worked around: "code running inside the sandbox can potentially use domain fronting or similar techniques to reach hosts outside the allowlist."

Can I run an AI agent on Linux without sending my code to an API?

Only by running the weights locally, and that is a real change in what you get. Ollama, llama.cpp and vLLM all serve an OpenAI-compatible endpoint on localhost, and agents that accept a custom base URL will talk to them. Aider and Cline do this natively; Claude Code and Codex are wired to their vendors' APIs.

The honest version of this tradeoff is that a local model is a different tool rather than the same tool run privately. A quantized model on a single consumer GPU is competitive at short, well-specified edits and falls off on exactly the thing agentic work is made of: long chains of interdependent actions where an early wrong step compounds. That is not a vendor talking point: the Terminal-Bench 2.0 results put smaller models around 15% where frontier models now clear 90%, and the gap is widest on precisely these long chains.

The setup itself is short:

ollama serve
ollama pull qwen2.5-coder:32b

Then point the agent at http://localhost:11434/v1. What to check first is VRAM headroom against the model's quantized size, because a model that spills into system RAM will run, slowly enough that you will assume something else is broken.

What can I actually see my local agent doing on my machine?

Less than you would expect from its transcript, and the gap has a documented cause. An agent's log records the tool calls its framework mediated. Anything a spawned process does afterwards is outside that record, and Anthropic states the specific mechanism: OTEL_* environment variables are not passed to the subprocesses the Bash tool spawns.

So one Bash call that runs npm test is one event with a duration and an exit status, whatever the process tree underneath it did. If the question is "what ran on my machine", the transcript is testimony rather than evidence, and the kernel is where the evidence is:

curl -fsSL https://yeet.cx | sh
yeet login
yeet run gh:yeet-src/exectop -- --pid $(pgrep -n claude)

exectop attaches three sched tracepoints and prints a row per kind of command, folded by count, following the agent's process tree through fork. Nothing is configured in the agent and the agent is not asked to cooperate. Attaching to a running pid cannot see children forked before you attached, which the tool states rather than papering over; launching the agent under it instead closes that gap. The full treatment of that boundary covers what auditd and strace each catch and where they stop.

Which parts of a local agent setup still make network calls?

More than the model endpoint, and the list is worth walking before you assume an air-gapped setup is air-gapped. Even with weights on your own GPU:

• Package installs. Any npm install, pip install or cargo fetch the agent decides to run reaches a registry, and postinstall scripts execute as your user during that.
• MCP servers. A server in .mcp.json is a separate local process with its own syscalls and its own network access, outside the agent's permission prompts. Auditing one is a separate job from auditing the agent.
• Language servers and tooling the agent launches, which fetch indexes and check versions on their own schedule.
• Telemetry, which is per-agent and per-version and worth reading the settings for rather than assuming.

The practical check is to stop reasoning about it and look. A default-deny egress policy will tell you what breaks, which is the fastest inventory of what your setup actually reaches, and the egress-attribution problem covers why a hostname allowlist is a weaker control than it looks.

Should I run a local agent in a container or a VM instead?

Use one when the agent is running code you did not write, and do not mistake it for access control. A container gives you a reproducible environment and a filesystem boundary, and it does nothing about credentials, network routes or mounted source trees you deliberately handed in. An agent inside a container with your ~/.aws mounted and a route to your internal network has exactly the access you gave it.

The layer comparison, for a host you own:

Route	What it does better	What it does not cover
The agent's own sandbox	Already installed, zero setup, denies credential paths by name	Governs Bash subprocesses, not the agent's own reads
Landlock / agent-jail	Unprivileged, in mainline, enforces on the inode	Confines without reporting what it refused
agent-lock and exectop on yeet	Refusal and record in one place, editable live	Needs BPF LSM and a privileged install
Container or VM	Reproducible, correct for untrusted code	Does not constrain access you deliberately granted
Local weights (Ollama, vLLM)	Nothing leaves the machine	Weaker on long agentic chains; changes capability, not permissions

The layer-by-layer confinement comparison goes through each of these properly.

The bottom line: local means two things, and only one of them is about privacy

Decide which half of "local" you actually care about before you configure anything, because the two halves have different answers. If you care what the agent can reach on your box, the work is Linux access control: turn on the sandbox that shipped with your agent, set filesystem.enabled because it defaults off, deny ~/.ssh and ~/.aws/credentials by name, and know that the default read scope without those settings is your entire computer. If you care where your code goes, the sandbox does nothing for you and the answer is local weights through Ollama or vLLM, at a real cost in capability on long tasks. If you need a reproducible environment or you are running untrusted code, reach for a container and pay that setup cost deliberately. If you want to know what the agent actually did rather than what it reported, that record has to come from below the agent, which is auditd if compliance already reads it and exectop on yeet if you want it now on one box.

The trap here is that all of this feels handled because the agent is running on hardware you own. Local is not a security property. It describes where a process runs, and a process running on your own machine with your own permissions is precisely the thing that can read your keys.

Frequently asked questions

Can I run an AI coding agent fully offline on Linux?

Yes, by serving the weights yourself with Ollama, llama.cpp or vLLM and pointing an agent that accepts a custom base URL at localhost. Aider and Cline support this directly. The agent still needs network access for anything else it does, including package installs and MCP servers, so an offline model is not the same thing as an offline setup.

Does running an AI agent locally keep my source code private?

Only if the model is also local. An agent installed on your machine still sends the file contents it reads to whichever inference endpoint it is configured against, which for Claude Code, Codex and Cursor is a hosted API. The agent being a local process controls what it can reach, not where the context it gathers is sent.

What permissions does an AI agent have when I run it on Linux?

Everything your user has, unless you configured a sandbox. The agent runs with your uid, so it can read any file you can read and open any socket you can open. Claude Code's documented default read behavior is read access to the entire computer except certain denied directories, and that default still permits reading credential files.

How much VRAM do I need to run a coding model locally?

It depends on parameter count and quantization more than on the agent you pair it with, and the number to check is whether the quantized weights fit with headroom for context. A model that spills into system RAM still runs, just slowly enough that people assume something else is wrong. Check nvidia-smi against the model's file size before concluding a model is too slow to be useful.

Is Ollama enough to run an AI agent locally?

It is enough to serve the model, which is one of the two halves. Ollama exposes an OpenAI-compatible endpoint that agents accepting a custom base URL will talk to. You still need an agent harness on top of it, and you still need to decide what that harness is allowed to touch on the machine.

Can an AI agent running locally read my SSH keys?

By default, on most setups, yes. Claude Code's sandboxing documentation states its default read behavior is read access to the entire computer and notes this still allows reading credential files such as ~/.aws/credentials and ~/.ssh/. The sandbox ships a credentials setting that denies those paths specifically, and setting it before the first session costs nothing.

How do I see which files a local AI agent opened?

Not from the agent's transcript, which records the tool calls its framework mediated rather than the syscalls underneath them. A kernel-side probe on the file-open path attributes each open to the process that made it, including processes the agent spawned. agent-lock does this while also refusing opens outside a directory you name.

Do local AI agents work without root on Linux?

The agents themselves do, and they should run without it. Some of the confinement options differ: Landlock is explicitly unprivileged and designed for a process to restrict itself, while BPF LSM enforcement needs a privileged install. Running the agent as root removes the only permission boundary you had, so it is the one configuration to avoid.

What is the difference between running an agent locally and using a cloud agent?

Where the process lives and therefore what it can reach. A local agent has your filesystem, your sockets and your credentials, and its blast radius is your machine. A hosted agent runs in the vendor's sandbox and reaches your systems only through what you connected to it. The local one is more capable and more dangerous for the same reason.

Does a container make a local AI agent safe?

It makes it reproducible and bounds the filesystem, which is not the same as safe. Credentials, network routes and mounted source trees pass straight through a container boundary because you put them there. Containers answer what the agent can touch on this machine and not what it can do with access you deliberately granted.

Sources

• Claude Code sandboxing documentation. Anthropic's own reference for the sandbox this post configures. Documents the default read behavior as "read access to the entire computer, except certain denied directories", notes it "still allows reading credential files such as ~/.aws/credentials and ~/.ssh/", specifies the filesystem, network and credentials settings used above, and states the domain-fronting limitation on hostname-based allowlists.
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. ICLR 2026, Merrill, Shaw, Carlini et al. 89 containerized terminal tasks, each with a human-written reference solution and tests. The source for the claim that agentic capability falls off on long chains of interdependent actions rather than on individual command knowledge.
• Ollama. The shortest path to serving weights locally with an OpenAI-compatible endpoint, and the setup referenced in the offline section.
• Landlock. The unprivileged access-control LSM in mainline Linux, and the confinement route that needs no privileged install.
• agent-jail. Landlock-based confinement for a coding agent, enforcing on the inode so symlink and hardlink traversal resolve to the same refusal.
• exectop. The process-launch monitor used above to see what an agent actually executed, scoped to a process tree by --comm.
• yeet documentation. The runtime the kernel-side scripts here are written in, for when the ready-made shape is wrong and you need to write your own probe.

Related resources

• How to Audit What an AI Coding Agent Actually Ran on Linux. Where the transcript stops and what auditd and strace each catch
• How to Sandbox an AI Coding Agent on Linux. The confinement options compared layer by layer
• Can an AI Coding Agent Exfiltrate My Source Code?. Why a domain allowlist is a weaker control than it looks
• How to Audit a Local MCP Server on Linux. The other local process in your agent setup
• AI Agents on Linux: Two Threat Models. Which problem your stack is actually solving

Built with yeet, a JS runtime for writing eBPF programs on Linux machines. Join us on discord.