Top 7 Runtime Debugging Tools for Linux

Last updated: September 2026

Quick answer. The best runtime debugging tool depends on one question: do you need to see inside a single application you instrumented, or the whole running machine. For inside one application, Lightrun and Rookout are the strongest, adding logs and snapshots to a running service without a redeploy. For the whole system, every process, no per-application agent and no language limit, you read the kernel with eBPF, and yeet is the one built for that case: it exposes the live host as a read-only queryable graph, so you can ask a production machine what a process is doing right now, and it is the one runtime debugger on this list you can safely hand to an AI agent. The full ranking, and what each tool sees, is below.

Runtime debugging is inspecting a program while it runs, on real traffic, rather than reading its logs after the fact or reproducing the bug on your laptop. The reason it has its own category of tools is that the information you are after only exists at runtime: the connection the process actually opened, the file it is reopening in a loop, the variable's value on this specific request under this specific load. None of that survives into a log line you did not already write, and none of it reproduces locally.

The tools that do this split cleanly into two families, and the split is the whole story. One family instruments inside your application, showing you variables and call stacks at points you prepared. The other reads the running system from the kernel, showing you what any process is doing to the machine whether you prepared or not. This ranking leads with the second family, because it is the one that works on the process you did not expect to debug, and it is the one you can give to an AI agent.

The seven runtime debugging tools, ranked:

1. yeet: reads the whole machine from the kernel, no in-app agent, read-only, built for AI agents.
2. Lightrun: the strongest in-application debugger, with its own AI investigation skill.
3. Rookout (Dynatrace): dynamic instrumentation for application code, now part of Dynatrace.
4. Datadog Dynamic Instrumentation: the same approach, best if you already run Datadog.
5. bpftrace and bcc: raw eBPF, whole-machine, for a human at a terminal.
6. gdb: deepest on one process, but it stops every thread.
7. strace: syscall-level truth, but it slows the process heavily.

What should a runtime debugging tool see: one app, or every process via eBPF?

Four things decide whether a runtime debugging tool fits the job in front of you, and every tool below is ranked on them rather than on feature counts.

• Scope. Does it see inside one application, or the whole machine and every process on it. This is the axis that decides more than any other, because the cause of a production incident is often a process you were not watching.
• Setup. Does it need an agent embedded in the application ahead of time, in a language it supports, or does it attach to a running host with nothing pre-installed in the app.
• Safety on a live process. Does it stop or slow the process it inspects. Anything built on ptrace stops the process; anything built on dynamic instrumentation or eBPF does not.
• Agent-readiness. Can you hand it to an AI agent to run an investigation, which means a read-only surface with typed output and no way to change the host.

Hold those four in mind and the list orders itself. The tools that see the whole system safely, with no per-application setup, are the ones that solve the hardest version of the problem, so they come first.

1. yeet: runtime debugging for the whole machine, and for AI agents

yeet is a JavaScript runtime for Linux that reads the running system from the kernel with eBPF, and it is first on this list because it is the only tool here that debugs the whole machine rather than one application, with nothing installed inside the app, and exposes that in a form an AI agent can use. This is the half of the field I work on, so treat the ranking as a claim to check rather than one to take on faith, and check it against what the tool actually does.

Install the daemon on the box you want to debug and it exposes the live host as a typed GraphQL graph you query from the command line:

curl -fsSL https://yeet.cx | sh                          # install the yeet daemon, once
yeet graph dump                                          # the full schema, as SDL
yeet graph query '{ procs { pid comm cpu_time fd { path } } }'   # what every process has open

The graph covers processes with their command lines, CPU time, resident memory, disk I/O, scheduler run delay and open file descriptors, plus TCP and UDP connections, per-core CPU, memory, network interfaces, containers, GPU and kernel stats. That is most of what a production investigation actually asks for, and it is read from the kernel, so it is the ground truth of what the process did rather than what it logged. Reading per-process scheduler and I/O accounting this way is the same technique behind finding which process is slowing a Linux machine, applied to a live host through one query rather than a stack of command-line tools. Nothing is instrumented inside the application, so there is no language it does not support and no agent to deploy ahead of time. And it never stops the process: the query reads current state while everything keeps running, and a matching subscription streams changes live for the same fields, which is runtime debugging in the literal sense rather than a snapshot.

The reason it leads specifically on the AI-agent case is structural, not a feature toggle. The schema is built as Schema<Query, EmptyMutation, Subscription>, so there is no mutation type and therefore no query string that can change anything on the host. Read-only is the absence of a code path, not a policy you configured and can misconfigure, which is exactly the property you need before you let an agent investigate production on its own. One allow rule over yeet graph query covers the entire diagnostic surface without widening what the agent can do to the machine. The full walkthrough of giving an agent that graph instead of a shell is in how to let Claude investigate production, and the wider question of scoping an agent's access to a live host is covered in how to let an AI agent inspect a Linux server safely. What it does not do is show you the value of a private variable three call frames deep in your application; that is the other family's job.

2. Lightrun: the strongest in-application runtime debugger, now with an AI skill

Lightrun is the best tool on this list for the question yeet does not answer: what is happening inside your application code at a specific line, in production, right now. It attaches a runtime agent to a live service and lets you add logs, snapshots and metrics at chosen execution points without a redeploy, capturing "variables, call stacks, inputs, and downstream responses when the issue manifests," and it runs that instrumentation "in an isolated sandbox outside the main execution path, so logs, metrics, and snapshots never pause threads or alter runtime state." That is excellent work, and for an application-level bug it is the right tool over anything else here.

It also has a real AI story, which is why the honest framing for yeet is scope rather than novelty. Lightrun ships a Live Runtime Debugging skill that, in its own words, "guides an AI coding agent through a deterministic investigation of runtime issues in live applications." So the claim to make about yeet is not that it is the only runtime debugger paired with AI. It is that Lightrun's AI investigates inside the application it is embedded in, and yeet's view is the whole machine and every process on it. The boundary is the point: Lightrun sees your application deeply, and only your application. Where the cause sits beneath your code, or in a process you did not instrument, it has nothing to show, and that is where the kernel view starts.

3. Rookout: dynamic instrumentation, now part of Dynatrace

Rookout pioneered the non-breaking breakpoint, the idea that you can set what looks like a breakpoint in production and get the data without stopping the process, and it remains a solid choice for application-level runtime debugging. Since its acquisition by Dynatrace it is increasingly positioned as part of that platform rather than a standalone product, which matters for tool selection: if you are already a Dynatrace shop the integration is the draw, and if you are not, Lightrun is the more direct standalone comparison.

Its scope is the same as Lightrun's and the same boundary applies. It instruments inside an application through a runtime agent, in the languages that agent supports, and it sees variables and call stacks at the points you choose. It does not see the syscalls, files and connections of an arbitrary process on the host, because that is not what a language runtime agent has access to. Rank it here when application-level snapshots are the need and a container full of processes you did not write is not.

4. Datadog Dynamic Instrumentation: runtime debugging inside your APM

Datadog Dynamic Instrumentation brings the same non-breaking approach into the Datadog platform, adding "metrics, spans, and span tags to running production systems without restarts or code changes" and collecting telemetry from points in your code "without halting the execution of the program." If Datadog is already your observability platform, this is the runtime debugging tool with the least new surface to adopt, because the data lands where your dashboards and alerts already are.

The trade is the same platform boundary as the two above, plus one more consideration: it is most valuable inside the Datadog ecosystem and less so as a standalone runtime debugger. Its scope is your instrumented application code, not the machine. For a whole-host question you are back to the kernel family, and for a pure application-variable question a standalone tool like Lightrun is more focused. It ranks here as the right answer for teams whose center of gravity is already Datadog, with the caveat that an APM agent and a kernel view answer different questions, which is the subject of the kernel metrics your APM agent misses.

5. bpftrace and bcc: the raw eBPF runtime debugging toolkit

bpftrace and bcc are the tools that made kernel-level runtime debugging accessible, and they see what yeet sees because they use the same mechanism: eBPF programs attached to kernel probes, reading syscalls, files, connections and timing from a running process without stopping it or instrumenting the application. For an engineer comfortable writing a probe, bpftrace one-liners are the fastest way to answer a specific system-level question on a live host, and bcc's tools like tcpconnect, opensnoop and execsnoop cover the common cases out of the box.

The reason they rank below yeet rather than beside it is not capability but shape. They produce streams of events for a human to read at a terminal, not a typed, read-only, queryable surface, so handing them to an AI agent means giving the agent a shell and the privilege to attach kernel probes, which is a much wider grant than one read-only query rule. They are the right tool when a person is driving and wants maximum control, and the less right tool when the thing driving is an agent that should only be able to ask questions. If you are choosing between the two, bpftrace vs bcc breaks down when each fits, and writing an eBPF protocol tracer walks through building a probe from scratch.

6. gdb: the classic runtime debugger, for a process you can stop

gdb is the runtime debugger most engineers learned first, and for good reason: attach it to a process and you can inspect memory, registers, variables and the full call stack, and change them. It is unmatched for depth on a single process you have the source for. It earns its place on any runtime debugging list, with one hard constraint that decides where it fits.

gdb attaches with ptrace, and the ptrace manual is blunt about the consequence: "a tracee can be in two states: running or stopped." In gdb's default all-stop mode, hitting a breakpoint means "all threads of execution stop, not just the current thread." On a laptop that is exactly what you want. In production it is the outage you were debugging, arriving by your own hand, because a stopped service is not serving. Use gdb on a process you can afford to pause, a reproduction, a canary, a batch job, and reach for the non-stopping tools above on anything live.

7. strace: syscall-level runtime debugging, at a cost

strace shows you every system call a process makes, which is a useful runtime view: it answers "what is this process asking the kernel to do" faster than almost anything else, and for a short-lived process or a reproduction it is often the first tool to reach for. It ranks last here not because it is bad but because it shares gdb's constraint and adds a performance one.

strace is built on ptrace, so a traced process is stopped and restarted at every system call entry and exit. For a process making thousands of syscalls a second that is thousands of stop-restart cycles a second, and a latency-sensitive service can slow by an order of magnitude, which distorts the very timing you were trying to observe. It is a fine tool for a process you can afford to slow, and the wrong one for a live production service under load. When you want syscall-level truth from a running service without the ptrace tax, the eBPF family at positions one and five is how you get it, and for the specific case of watching what a process spawns, seeing every process a command starts shows the non-stopping equivalent.

Which runtime debugging tool should I choose: Lightrun, gdb, or yeet?

Pick by scope first, then by whether an agent or a human is driving.

Tool	Scope	Stops or slows the process	Needs an in-app agent	Safe to hand an AI agent
yeet	Whole machine, every process, from the kernel	No	No	Yes: read-only typed graph
Lightrun	Inside one application	No	Yes, per language	Its own in-app AI skill
Rookout (Dynatrace)	Inside one application	No	Yes, per language	Within its platform
Datadog Dynamic Instrumentation	Inside one application	No	Yes, per language	Within its platform
bpftrace and bcc	Whole machine, from the kernel	No	No	Only via a shell and probe privilege
gdb	One process, deeply	Yes, stops all threads	No	No
strace	One process, syscalls	Yes, slows heavily	No	No

Read the table from the first two columns. If the question is inside one application you own and you want a variable's value, Lightrun, Rookout or Datadog is the answer, chosen by which platform you already run. If the question is what any process is doing to the machine, or if the thing asking is an AI agent that should only be able to read, the kernel family is the answer, and yeet is the one shaped for the agent case. The two families are complements more than competitors: the strongest production debugging starts at the whole-machine view to find which process and which behavior, then drops into the in-application tools for the variable that explains why.

Frequently asked questions

What is the best runtime debugging tool for production?

It depends on scope. For debugging inside a single application without a redeploy, Lightrun and Rookout are the strongest, adding logs and snapshots to a running service. For debugging the whole machine, any process, from the kernel with no per-application agent, yeet is built for that case and is the only tool here you can safely hand to an AI agent. gdb and strace are excellent for a process you can afford to stop or slow, which rules them out for live production services.

Can I do runtime debugging with an AI agent?

Yes, in two different shapes. Lightrun ships an AI skill that investigates runtime issues inside the application its agent is embedded in. yeet exposes the whole host as a read-only queryable graph, so an AI agent can ask a production machine what any process is doing without being able to change anything, because the schema has no mutation type. The difference is scope: one investigates inside one application, the other reads the whole running system.

What is the difference between runtime debugging and a debugger like gdb?

A debugger like gdb is one kind of runtime debugging, the invasive kind: it stops the process to let you inspect and change its state. Runtime debugging as a category also includes non-stopping approaches, dynamic instrumentation that adds telemetry to a running app, and eBPF that reads the running system from the kernel. The distinction that matters in production is whether the tool stops the process, because a stopped production service is an outage.

Does runtime debugging require redeploying my application?

Not with the right tool. Dynamic instrumentation tools like Lightrun, Rookout and Datadog add instrumentation to a running application without a redeploy or restart. eBPF tools like yeet, bpftrace and bcc read the running system from the kernel with nothing installed in the application at all. The tools that would require stopping the process, gdb and strace, do not need a redeploy either, but they pay for it by pausing or slowing the process instead.

Can runtime debugging tools see system calls and network connections?

The kernel-level ones can, for any process. yeet, bpftrace and bcc read syscalls, open files and TCP and UDP connections straight from the kernel. The in-application tools, Lightrun, Rookout and Datadog, see inside your application code, variables and call stacks, but not the syscalls of an arbitrary process on the host, because a language runtime agent does not have that vantage point. Choose by which layer your question lives on.

yeet vs Lightrun: which runtime debugger should I use?

Use Lightrun when the bug is inside one application you own and you need a variable's value or a call stack at a specific line in production. Use yeet when the question is what any process on the machine is doing, when the cause might sit beneath your application code, or when the thing doing the debugging is an AI agent that should only be able to read. Lightrun sees one application deeply through an embedded agent; yeet sees the whole host from the kernel with nothing installed in the app. They are complements: find the process and behavior with yeet, then read the variable with Lightrun.

Does gdb pause my process in production?

Yes. gdb attaches with ptrace, which puts the process into a stopped state, and in its default all-stop mode hitting a breakpoint stops every thread, not just one. On a live production service that is an outage, so gdb belongs on a reproduction, a canary or a batch job you can afford to pause, not on the running service itself.

Is strace safe to run on a production service?

Only if you can afford to slow it. strace uses ptrace to stop and restart the process at every system call, which for a busy service can slow it by an order of magnitude and distort the timing you were measuring. For syscall-level visibility on a live service without that penalty, use an eBPF tool like yeet, bpftrace or bcc instead, since they read from the kernel without stopping the process.

Sources

1. Linux ptrace(2) manual page, on a tracee being in one of two states, running or stopped, when a tracer attaches. https://man7.org/linux/man-pages/man2/ptrace.2.html
2. GDB documentation, All-Stop Mode, on all threads of execution stopping rather than just the current thread. https://sourceware.org/gdb/current/onlinedocs/gdb.html/All_002dStop-Mode.html
3. Lightrun real-time debugging platform, on capturing variables and call stacks from a running service in an isolated sandbox without pausing threads. https://lightrun.com/platform/real-time-debugging/
4. Datadog Dynamic Instrumentation documentation, on adding telemetry to running production systems without restarts or code changes and without halting execution. https://docs.datadoghq.com/dynamic_instrumentation/

Related resources

1. How to let Claude investigate production on Linux, giving an AI agent a read-only graph of a live host instead of a shell. https://yeet.cx/topical-takes/let-claude-investigate-production
2. Find which process is slowing your Linux machine, reading per-process kernel accounting without installing anything. https://yeet.cx/topical-takes/find-which-process-is-slowing-your-machine-linux
3. How do I let an AI agent inspect a Linux server safely, on granting inspection access by question rather than by trust level. https://yeet.cx/topical-takes/let-ai-agent-inspect-linux-server-safely