How to Run AI Agents Locally on Linux

Last updated: October 2026

Quick answer. Install the agent, turn on the sandbox it ships with, deny your credential paths by name, and then check what the agent ran from the kernel rather than from its own transcript; the three steps below are the whole setup. "Running an AI agent locally" almost always means the agent process runs on your Linux machine while the model runs in a vendor's datacenter, and the two halves need different setup: the local half decides what the agent can reach on your box, the remote half decides where your code goes. If you want the second half local too, add Ollama and point an agent that accepts a custom base URL at it.

I build kernel-side tooling and I watch engineers set these things up, which is a narrow vantage point: I see the moment after somebody has installed an agent and before they have any idea what it is doing. I do not run your fleet, I have not audited the model you are about to point at your source tree, and I am not going to tell you which agent to pick, because on Linux they are more alike than the comparison posts suggest. What I keep seeing is that the word "locally" carries two meanings in the same sentence, and the setup people end up with is not the one they thought they were choosing.

How do I run an AI agent locally on Linux in three steps?

1. 1

   Install the agent and check its settings file

   Claude Code's native installer for Linux is one line, and Codex CLI and Aider install the same way. All three run as programs under your user: they read your filesystem directly, spawn subprocesses on your box, and keep their session state on local disk. The model that decides what they do is reached over an API call to a vendor, which is the half of "locally" the word leaves out. The agents differ in context handling, cost and how the harness recovers when a command fails, and none of that changes what the process can touch on your host; what does is the sandbox each one ships and whether it is on.

   $ curl -fsSL https://claude.ai/install.sh | bash

   Copy

   Codex CLI: npm install -g @openai/codex. Aider: curl -LsSf https://aider.chat/install.sh | sh

   $ claude auth login

   Copy

   Signs you in to your Anthropic account. On a remote box it prints a link; open it in any browser and paste the code back.

   $ cat ~/.claude/settings.json 2>/dev/null || echo 'no settings file yet'

   Copy

   No settings file means the agent is running its defaults.

   Defaults are chosen so the install works everywhere rather than for your threat model: nothing prompts you, nothing warns you, and the agent behaves identically whether you configured it or not.

2. 2

   Turn the sandbox on and deny your credential paths

   Install the two packages the sandbox runs on:

   $ sudo apt-get install -y bubblewrap socat

   Copy

   The docs list both as required on Linux. Fedora: sudo dnf install bubblewrap socat

   Once bubblewrap is installed, start claude from a project directory and paste the prompt below. The sandbox ships off; this turns it on, keeps whatever is already in your settings file, and denies your AWS and SSH credential paths and two token variables by name, because by default the agent can read your entire computer. Filesystem isolation comes on with the sandbox. It also turns off the retry that lets Claude rerun a blocked command outside the sandbox.

   Configure the sandbox:

   >Turn on the Claude Code sandbox on this machine. First check that bubblewrap and socat are installed with command -v bwrap socat, and if either is missing, stop and tell me how to install it. Then merge this into ~/.claude/settings.json, keeping any settings already there, and show me the finished file:

   {"sandbox": { "enabled": true, "allowUnsandboxedCommands": false, "network": { "allowedDomains": ["github.com", "registry.npmjs.org"] }, "credentials": { "files": [ { "path": "~/.aws/credentials", "mode": "deny" }, { "path": "~/.ssh", "mode": "deny" } ], "envVars": [ { "name": "GITHUB_TOKEN", "mode": "deny" }, { "name": "NPM_TOKEN", "mode": "deny" } ] }}}

   See fullCopy

   Claude asks before it writes the file. Leave this session open for step 3: Claude Code reloads its settings when the file changes, so the sandbox applies without a restart.

   Codex ships its own sandbox built on Landlock and seccomp, with read-only and workspace-write modes, which the Codex sandbox post walks through knob by knob.

3. 3

   Test the sandbox

   sandbox-watch follows the agent's process tree through fork in the kernel, stamps every command it runs with the net namespace it ran in, and compares that to the agent's own, so it can tell which commands ran inside the sandbox and which ran outside it.

   In a second terminal on the same Linux host, start the watcher:

   $ curl -fsSL https://yeet.cx | sh && yeet run gh:yeet-src/sandbox-watch --tty -- --pid $(pgrep -n claude)

   Copy

   Paste this into Claude on your first terminal:

   >Check whether this machine can push to GitHub over SSH. Test the connection, tell me which key in ~/.ssh it would use, and show me that key's public half so I can add it to GitHub if it is missing.

   See fullCopy

   sandbox-watch splits what it watches into two sides: commands that ran inside the sandbox, where the credential path or the network is walled off, and commands that ran outside it, in the agent's own namespace, where nothing is. The headline turns red the moment anything lands on that second side.

   The replay below is the run you do not want: the sandbox left at its defaults, /sandbox on and nothing else, without the two settings from step 2. Watch what happens.

   sandbox-watch

   ● sandbox-watch ▏ scope claude, pid 26422 ▏ session over ↻ replay

   57 execs in 0s · 6 things that don't fit

   ✗ 2 got past the sandbox ✓ 6 blocked by the sandbox

   got past = ran in the agent's own namespace · blocked = ran inside the sandbox

   ── got past the sandbox ──────────────────────────────────────────────────────────────────

   ✗ ssh -T -o -o ⟨3 args⟩ ran outside the sandbox · opened your r… [verified]

   ✗ cat ran outside the sandbox · its command reaches crede…

   ── blocked by the sandbox ────────────────────────────────────────────────────────────────

   ✓ cat ran inside the sandbox · credential paths walled off

   ✓ 3 -c ⟨2 args⟩ ran inside the sandbox · credential paths walled off

   ✓ bash -c exec … ran inside the sandbox · credential paths walled off

   ✓ ssh -G ⟨1 arg⟩ ran inside the sandbox · the network walled off

   ✓ ls -la ⟨1 arg⟩ ran inside the sandbox · credential paths walled off

   ✓ ssh -T -o -o ⟨3 args⟩ ran inside the sandbox · the network walled off

   ── attempts exectop saw ──────────────────────────────────────────────────────────────────

   ssh -T -o -o ⟨3 args⟩ ×2 fetches from the network

   cat ×6 touches credential paths

   3 -c ⟨2 args⟩ ×2 touches credential paths

   bash -c exec … ×2 touches credential paths

   ssh -G ⟨1 arg⟩ fetches from the network

   ls -la ⟨1 arg⟩ touches credential paths

   The ssh -T connection test failed inside the sandbox, because egress there goes nowhere but the proxy, so the agent ran it a second time outside the sandbox, in its own namespace, where ~/.ssh is the real directory. sandbox-watch caught that second run opening the real ~/.ssh/known_hosts and tagged it verified, because the file-open probe watched the open land on the real file rather than inferring it from the command. The agent said so itself in its reply: "I re-ran the test outside the sandbox." A third panel lists every command that reached for a credential or the network, whichever side it ran on, so what the agent tried and what the sandbox did read at once.

   This is what a wrong configuration looks like, and it is why the test matters. The defaults leave reads open to your whole machine and leave the unsandboxed retry on, so a blocked command does not stay blocked. Set credentials.files and "allowUnsandboxedCommands": false as in step 2 and the same run holds: every sensitive command stays on the blocked side, the got-past panel stays empty, and the headline stays green. Same agent, same prompt; the only difference between a row that got past and a row that stayed put is the two lines you set in step 2. Run the test so you find out which one you have.

Does an AI agent run with my full user permissions on Linux?

Yes, unless you configured something. An agent launched from your shell is a process with your uid, so every file you can read it can read, every socket you can open it can open, and every binary on your $PATH it can execute. The Linux permission model has no way to express "this process is an agent" because it was designed for humans, and it grants on identity rather than on intent.

Anthropic documents where that leaves Claude Code. From the sandboxing page, on what a sandboxed command may read by default:

"Default read behavior: read access to the entire computer, except certain denied directories. This default still allows reading credential files, so protect credentials you don't want commands to read."

The same page's access table puts reads at "Most of the machine, including credential files such as ~/.ssh and ~/.aws/credentials", and states why both isolation layers matter: "Effective sandboxing requires both filesystem and network isolation. Without network isolation, a compromised agent could exfiltrate sensitive files like SSH keys." That is a vendor documenting the sharp edge of its own default, which is more than most do, and it sits two clicks deep in a docs site nobody reads before installing.

Two scope facts from the same page change how much a sandbox buys you. The sandbox wraps shell commands, and the page is explicit that the built-in file and web tools "follow permission rules instead. A denyRead entry doesn't stop the Read tool, and allowedDomains doesn't limit WebFetch." The other processes Claude Code starts, "command hooks, local MCP servers, plugin monitors, LSP servers, and helper commands such as your status line command and apiKeyHelper run with your full access." So a sandbox that is on and configured still leaves the agent's own file reads under a different rule set, and leaves every MCP server you added outside the boundary entirely.

How do I turn on the Claude Code sandbox on Linux before the agent touches my repo?

Run /sandbox in a session or set it in ~/.claude/settings.json, then add the credential denials, because the docs state plainly that "the sandbox is off by default." Once it is on, filesystem isolation is on with it (sandbox.filesystem.disabled defaults to false), writes are confined to the working directory, directories you added and a per-user temp directory, and network egress goes through a local proxy that checks each host against your allowed domains, "which start empty." Reads stay wide open until you deny paths. The settings the step 2 prompt writes are the docs' own credentials example with a network allowlist and allowUnsandboxedCommands added, and two of their entries do the work.

credentials.files denies the reads the default scope would otherwise permit, and credentials.envVars removes the named variables from the environment of sandboxed commands, which closes the path where a token never touches disk but sits in the environment anyway. Both kinds of entry also accept "mode": "mask", which hands the command a placeholder instead of the secret itself.

Three limits before you rely on this. The file denials are part of the filesystem layer, so "with the filesystem layer off, sandboxed commands get unrestricted read and write access to the host filesystem" and the credential file rules go with it, while the environment variable scrubbing is independent and keeps applying. The network layer decides by the hostname the client supplies, which the docs call out: "code running inside the sandbox can potentially use domain fronting or similar techniques to reach hosts outside the allowlist." And there is an unsandboxed retry path for commands that fail inside the boundary: Claude may rerun one with the dangerouslyDisableSandbox parameter, which you approve in manual mode and a classifier model approves in auto mode. The step 2 settings set "allowUnsandboxedCommands": false, and with it off the docs say Claude Code ignores that parameter. Codex ships its own sandbox built on Landlock and seccomp, with read-only and workspace-write modes, which the Codex sandbox post walks through knob by knob.

Can I run an AI agent on Linux without sending my code to an API?

Only by running the weights locally, and that changes what you get. Ollama, llama.cpp and vLLM serve an OpenAI-compatible endpoint on localhost, and agents that accept a custom base URL will talk to it. Aider and Cline do this natively; Claude Code and Codex are wired to their vendors' APIs. The setup is two lines:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5-coder:32b

Then point the agent at http://localhost:11434/v1. Check VRAM headroom against the model's quantized size first, because a model that spills into system RAM still runs, slowly enough that you will assume something else is broken, and nvidia-smi against the model's file size settles that before you blame the agent.

A local model is a different tool rather than the same tool run privately. The Terminal-Bench 2.0 paper benchmarked 89 real terminal tasks and found "that frontier models and agents resolve less than 65% of tasks, with smaller models scoring around 15%", and its error analysis puts the gap on long chains of interdependent actions where an early wrong step compounds, which is what agentic work is made of. A quantized model on one consumer GPU is competitive at short, well-specified edits and falls off on the long chains. Pick it for privacy with that cost in view, not as a free swap.

How do I see what a local AI agent actually ran on my Linux machine?

Not from its transcript, and the gap has a documented cause. An agent's log records the tool calls its framework mediated, so one Bash call that runs npm test is one event with a duration and an exit status, whatever the process tree underneath it did. Anthropic's monitoring docs state the mechanism: "Claude Code doesn't pass OTEL_* environment variables to the subprocesses it spawns, including the Bash tool, hooks, MCP servers, and language servers." The audit post takes that boundary apart against auditd and strace. If the question is "what ran on my machine", the transcript is testimony and the kernel is where the evidence is:

curl -fsSL https://yeet.cx | sh
yeet run gh:yeet-src/exectop -- --pid $(pgrep -n claude)

exectop attaches three sched tracepoints (sched_process_exec, sched_process_fork and sched_process_exit), follows the agent's process tree through fork in the kernel, folds repetition into one row per kind of command, and ranks anything that does not look like ordinary work above the rest. Nothing is configured in the agent and the agent is not asked to cooperate. Running the script needs no account: the daemon installs as root once and the script runs as your user. The limit its README states rather than hides: attaching to a running pid sees "a running process and everything it starts from now on", and the status bar says pre-existing children not tracked. If you can start the agent yourself, the README's launch mode holds the command stopped until the probe is attached, so nothing forks unobserved. When you want refusal as well as a record, agent-lock confines the agent and everything it spawns to one directory with BPF LSM and logs each decision, on kernels built with CONFIG_BPF_LSM.

Which parts of a local AI agent setup still reach the network?

More than the model endpoint, even with the weights on your own GPU, and the list is short enough to walk before you call a setup air-gapped:

• Package installs. Any npm install, pip install or cargo fetch the agent decides to run reaches a registry, and postinstall scripts execute as your user while it does.
• MCP servers. A server in .mcp.json is a separate local process with its own syscalls and its own network access, outside the agent's permission prompts and, per the sandboxing docs, outside its sandbox. Auditing one is a separate job from auditing the agent.
• Language servers, hooks and helper commands the agent launches, which the same docs say "run with your full access", and which fetch indexes and check versions on their own schedule.
• Telemetry, per agent and per version; read the settings rather than assuming.

The practical check is to stop reasoning about it and look at the connections. dialout catches every outbound TCP connection at fexit/tcp_connect, groups it by the process that dialed it, and flags the ones that leave the private network, so an "offline" agent setup that dials a registry or a telemetry endpoint shows up as a row with the process name next to it:

yeet run github:yeet-src/dialout -- --once --secs 30 | less -R

It watches TCP connects, so a lookup that never opens a TCP connection is outside it, and the post on what an agent connects to covers the DNS and SNI side. A default-deny egress policy is the other fast inventory, and the exfiltration post covers why a hostname allowlist is a weaker control than it looks.

Should I run a local AI agent in a container or a VM on Linux instead?

Use one when the agent is running code you did not write, and do not mistake it for access control. A container gives you a reproducible environment and a filesystem boundary, and it does nothing about credentials, network routes or mounted source trees you deliberately handed in. An agent inside a container with your ~/.aws mounted and a route to your internal network has exactly the access you gave it.

The layer comparison, for a host you own:

Route	What it does better	What it does not cover
Claude Code's own sandbox	Already installed, no setup, denies credential paths by name	Wraps shell commands only; the Read tool, hooks and MCP servers run outside it
agent-jail on Landlock	Unprivileged, in mainline, enforces on the inode	Confines without reporting what it refused
agent-lock and exectop on yeet	Refusal and record in one place, editable live, no change to the agent	Needs BPF LSM for refusal and a root daemon; Linux only
Container or VM	Reproducible, correct for untrusted code	Does not constrain access you deliberately granted
Local weights (Ollama, vLLM)	Nothing leaves the machine for inference	Weaker on long agentic chains; changes capability, not permissions

The layer-by-layer confinement comparison goes through each of these properly.

When is the Claude Code sandbox enough for an AI agent on Linux, and when does yeet add nothing?

For one developer on one laptop whose worry is keys, the vendor's sandbox with the credential block above does the job, and it does it with nothing to install. It is maintained by the people who ship the agent, it follows the agent across updates, it needs no root daemon and no kernel with BTF, and it covers the case most people searching this page are in: a coding agent on a workstation that holds SSH keys and a cloud credential file. If that is you, turn it on, add the two denials, and stop reading here.

A kernel-side watcher adds nothing when the agent lives in a throwaway VM you will discard, when the host cannot load BPF (yeet needs a BTF-capable Linux kernel and installs a root daemon, so it is out on macOS and inside most unprivileged containers), or when a policy is all you need and a record of what happened is not. It earns its place in three cases: when you need an answer after the fact that does not depend on the agent reporting on itself, when the agent is not Claude Code and ships a weaker sandbox or none, and when the things the sandbox leaves outside, MCP servers, hooks, language servers, are the things you are worried about. It is not a replacement for a container, an EDR, or an audit pipeline that compliance already reads. It is a look at what one application started, scoped to that application, on one box.

The bottom line: local means two things, and only one of them is about privacy

Decide which half of "local" you care about before you configure anything, because the two halves have different answers. If you care what the agent can reach on your box, the work is Linux access control: turn on the sandbox that shipped with your agent, deny ~/.ssh and ~/.aws/credentials by name, allowlist the domains it needs, and know that without those settings the default read scope is your entire computer. If you care where your code goes, the sandbox does nothing for you and the answer is local weights through Ollama or vLLM, at a cost in capability on long tasks that the Terminal-Bench numbers put in plain sight. If you need a reproducible environment or you are running untrusted code, reach for a container and pay that setup cost deliberately. If you want to know what the agent actually did rather than what it reported, that record has to come from below the agent, which is auditd if compliance already reads it and exectop on yeet if you want it now on one box, with no account.

The trap is that all of this feels handled because the agent is running on hardware you own. Local is not a security property. It describes where a process runs, and a process running on your own machine with your own permissions is precisely the thing that can read your keys.

Frequently asked questions

Can I run an AI coding agent fully offline on Linux?

Yes, by serving the weights yourself with Ollama, llama.cpp or vLLM and pointing an agent that accepts a custom base URL at localhost. Aider and Cline support this directly. The agent still needs network access for anything else it does, including package installs and MCP servers, so an offline model is not the same thing as an offline setup. A thirty-second dialout snapshot while the agent works is the quickest way to see which one you have.

Does running an AI agent locally on Linux keep my source code private?

Only if the model is also local. An agent installed on your machine still sends the file contents it reads to whichever inference endpoint it is configured against, which for Claude Code, Codex and Cursor is a hosted API. The agent being a local process controls what it can reach, not where the context it gathers is sent.

Does the Claude Code sandbox stop the agent's own Read tool from reading ~/.ssh?

No. The sandbox wraps shell commands, and the docs state that a denyRead entry does not stop the Read tool, which follows permission rules instead. A credentials deny keeps a sandboxed cat ~/.ssh/id_ed25519 from working; the file tool needs its own deny in your permission rules. A kernel-side probe on the file-open path, which is what agent-lock attaches, sees both kinds of read because it sits below both rule sets.

How much VRAM do I need to run a coding model locally on Linux?

It depends on parameter count and quantization more than on the agent you pair it with, and the number to check is whether the quantized weights fit with headroom for context. A model that spills into system RAM still runs, just slowly enough that people assume something else is wrong. Check nvidia-smi against the model's file size before concluding a model is too slow to be useful.

Is Ollama enough to run an AI agent locally on Linux?

It is enough to serve the model, which is one of the two halves. Ollama exposes an OpenAI-compatible endpoint that agents accepting a custom base URL will talk to. You still need an agent harness on top of it, and you still need to decide what that harness is allowed to touch on the machine.

Can an AI agent running locally on Linux read my SSH keys?

By default, on most setups, yes. Claude Code's sandboxing documentation states its default read behavior is read access to the entire computer except certain denied directories, and that this default still allows reading credential files. The sandbox ships a credentials setting that denies those paths specifically, and setting it before the first session costs nothing.

How do I see which files a local AI agent opened?

Not from the agent's transcript, which records the tool calls its framework mediated rather than the syscalls underneath them. A kernel-side probe on the file-open path attributes each open to the process that made it, including processes the agent spawned. agent-lock does this while also refusing opens outside a directory you name, and its audit flag watches without refusing.

Do local AI agents work without root on Linux?

The agents themselves do, and they should run without it. Some of the confinement options differ: Landlock is explicitly unprivileged and designed for a process to restrict itself, while BPF LSM enforcement and the yeet daemon need a privileged install, after which the scripts run as your user. Running the agent as root removes the only permission boundary you had, so it is the one configuration to avoid.

Does yeet run gh:yeet-src/exectop need an account, and does it send anything off the machine?

No account. Scripts run without signing in; an account is only needed for the hosted parts, such as Slack alerts and model calls. Two things do leave the box unless you turn them off: the installer reports whether it succeeded along with the contents of /etc/os-release, which the --no-phone-home flag disables, and the CLI sends usage events, which setting YEET_NO_ANALYTICS=1 in the environment disables.

What is the difference between running an agent locally and using a cloud agent?

Where the process lives and therefore what it can reach. A local agent has your filesystem, your sockets and your credentials, and its blast radius is your machine. A hosted agent runs in the vendor's sandbox and reaches your systems only through what you connected to it. The local one is more capable and more dangerous for the same reason.

Does a container make a local AI agent safe?

It makes it reproducible and bounds the filesystem, which is not the same as safe. Credentials, network routes and mounted source trees pass straight through a container boundary because you put them there. Containers answer what the agent can touch on this machine and not what it can do with access you deliberately granted.

Sources

• Claude Code sandboxing documentation (Anthropic, 2026). states "the sandbox is off by default" and that /sandbox or sandbox.enabled turns it on; documents the default read behavior as "read access to the entire computer, except certain denied directories" that "still allows reading credential files", with its access table putting reads at "most of the machine, including credential files such as ~/.ssh and ~/.aws/credentials"; says "effective sandboxing requires both filesystem and network isolation" and that without network isolation "a compromised agent could exfiltrate sensitive files like SSH keys"; is the source for the credentials.files and credentials.envVars example and the mask mode, for allowed domains that "start empty", for "a denyRead entry doesn't stop the Read tool", for the processes that "run with your full access", for the unsandboxed retry and allowUnsandboxedCommands, and for the domain-fronting limitation.
• Claude Code quickstart (Anthropic, 2026). the native Linux install line curl -fsSL https://claude.ai/install.sh | bash; apt, dnf and apk packages exist for Debian, Fedora, RHEL and Alpine; native installs update themselves in the background.
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces (ICLR 2026, Merrill et al.). 89 tasks, each with a containerized environment, a human-written solution and tests; finds "that frontier models and agents resolve less than 65% of tasks, with smaller models scoring around 15%"; its error taxonomy places the residual failures on long chains of interdependent actions.
• Ollama for Linux (Ollama, 2026). the install line curl -fsSL https://ollama.com/install.sh | sh, serving the OpenAI-compatible endpoint the agents above are pointed at.
• Codex CLI (OpenAI, 2026). npm install -g @openai/codex, the curl -fsSL https://chatgpt.com/codex/install.sh | sh installer, and prebuilt Linux x86_64 and arm64 binaries in its releases.
• Aider installation (Aider, 2026). curl -LsSf https://aider.chat/install.sh | sh for Mac and Linux, plus the pip, pipx and uv routes.
• agent-jail (yeet-src). Landlock-based confinement for a coding agent from a C launcher that calls landlock_create_ruleset, landlock_add_rule and landlock_restrict_self, with an eBPF watcher on the open and fork tracepoints showing what happens; marked experimental and verified on one kernel.
• exectop (yeet-src). three sched tracepoints, folding of repeated execs into one row per kind, four modes (launch, container, pid, host), the launch wrapper that holds the target stopped until the probe attaches, and the attach-mode limit stated as "pre-existing children not tracked".
• dialout (yeet-src). every outbound TCP connection captured at fexit/tcp_connect, grouped by the dialing process, with public egress flagged first; --once --secs N for a snapshot and dump.js for newline-delimited JSON.
• agent-lock (yeet-src). BPF LSM confinement of an agent and everything it spawns to one directory, refusing with EPERM and logging each decision, --audit to watch only; needs CONFIG_BPF_LSM and bpf in the active LSM list.
• Monitoring usage, Claude Code documentation (Anthropic, 2026). "Claude Code doesn't pass OTEL_* environment variables to the subprocesses it spawns, including the Bash tool, hooks, MCP servers, and language servers", which is why the agent's own telemetry stops at the shell.

Related resources

• How to Audit What an AI Coding Agent Actually Ran on Linux. Where the transcript stops and what auditd and strace each catch
• How to Sandbox an AI Coding Agent on Linux. The confinement options compared layer by layer
• How Do I Sandbox Codex on Linux?. Codex's Landlock and seccomp modes, knob by knob
• What Does My AI Agent Connect To?. The DNS and SNI side of egress, per process
• Can an AI Coding Agent Exfiltrate My Source Code?. Why a domain allowlist is a weaker control than it looks
• How to Audit a Local MCP Server on Linux. The other local process in your agent setup
• AI Agents on Linux: Two Threat Models. Which problem your stack is actually solving

Built with yeet, a JS runtime for writing eBPF programs on Linux machines. Join us on discord.