Find an N+1 Query From API Traffic

Last updated: October 2026

Quick answer. An N+1 is a correctness-shaped performance bug: nothing errors, every call succeeds, and one request quietly becomes two hundred. It hides because each individual call is fast and logged as a success, so the only symptom is an endpoint that is slower than it should be for reasons the code review did not catch. The way to find it is to count calls rather than time them: look for a burst, a short window in which one caller made far more requests to one dependency than a single request should need. On Linux you can read that from the traffic itself with eBPF, which catches N+1s through HTTP dependencies and third-party APIs that an ORM's query log never sees. The threshold that works in practice is simple: more than roughly twenty calls to the same place inside one second, tied to one inbound request, is almost always a loop that should have been a join or a batch.

The N+1 is the most common performance bug in applications that talk to anything, and the least likely to be caught by the tools watching for failures. A list endpoint fetches fifty records, then fetches each record's author one at a time. Fifty-one round trips where two would do. Every one of them returns 200 in four milliseconds, so there is nothing to alert on and nothing in the error budget.

What makes it survive is that the evidence is distributed. The ORM logs fifty successful queries, which looks like a busy service rather than a broken one. The APM reports the endpoint's p95 climbing, which tells you where without telling you why. And if the N+1 goes out over HTTP to another service or a third-party API, the ORM's query log never sees it at all.

What is an N+1 query, and why doesn't it show up as an error?

An N+1 is one query to fetch a collection followed by one more query per item in it, where a single query could have fetched everything. The name is literal: N items produce N additional calls, plus the original one. The pattern generalises beyond databases to any dependency, so an endpoint that loops over results calling a REST API per record has exactly the same bug with a longer round trip.

It does not show up as an error because nothing about it is erroneous. Each call is well formed, each answer is correct, and the aggregate is merely wasteful. Error-rate monitoring is blind to it by construction, since every call returns 200, and latency monitoring sees the symptom without the cause: a p95 that drifts upward as data grows, because the number of round trips scales with the size of the collection rather than staying constant.

The scaling property is what makes it dangerous rather than merely untidy. A list that held three items in development holds three hundred in production, so a bug that was invisible in testing becomes the dominant cost of the endpoint. That is also why N+1s tend to appear suddenly, long after the code shipped: nothing changed except how much data there is.

How do I detect an N+1 without an APM?

Count calls per unit of time instead of measuring their duration. The signature of an N+1 is not slowness, it is density: a burst of requests from one caller to one dependency, packed into the time it takes to serve one inbound request. Measured that way the bug is obvious even though every individual call looks healthy.

The useful metric is the densest second. For each dependency an endpoint calls, ask what the maximum number of calls in any one-second window was. A handler that makes one or two calls per request has a burst count in the low single digits no matter how busy the service gets, because the calls spread out across requests. A handler with an N+1 produces a tight cluster, because the loop runs as fast as the dependency answers.

Twenty calls in one second to the same destination is the practical threshold, and it is deliberately blunt. It is high enough that normal fan-out, a handler that legitimately calls three services, does not trip it, and low enough that a loop over even a modest collection does. When you see it, the question is no longer whether there is a loop but which collection it is iterating.

Why doesn't my ORM's query log show the N+1?

It usually does show it, in the sense that all N+1 queries are in there, and that is exactly the problem: it shows fifty lines that each look like a normal query, with nothing marking them as a group. Reading a query log to find an N+1 means noticing that fifty near-identical statements differ only in a parameter, which is a pattern humans miss at exactly the volume where it matters.

Two cases escape the query log entirely. The first is an N+1 over HTTP: when the loop calls another service or a third-party API rather than the database, the ORM has no involvement and logs nothing. These are the expensive ones, because an HTTP round trip costs a hundred times what a local query does, and they are increasingly common as applications decompose into services.

The second is a loop in code the query log cannot attribute to one request. Without a request identifier threaded through, the log is an interleaved stream from every concurrent request, so the fifty calls belonging to one slow request sit scattered among the calls of twenty others. Reconstructing the group is the work, and it is the work that counting bursts at the traffic layer skips.

What else does counting API calls reveal: tail latency, retries?

The same view answers several questions that are usually investigated separately, because they are all properties of traffic rather than of code. Once you can count and time what crossed the wire per endpoint, a short list of comparisons covers most of what goes wrong:

What you look for	The signal	What it usually means
A burst of calls in one second	More than ~20 to one dependency	An N+1 loop that should be a join or a batch
A heavy tail	p99 more than ten times p50	A slow path taken by a minority of requests, often a cache miss or a retry
Credentials in the clear	An authorization header on a plaintext transport	A service calling an internal peer over HTTP instead of HTTPS
Keys that come and go	A response key present in under 90% of samples	API drift, or an optional field a consumer treats as required
A retry storm	One client repeating the same call at a high rate	A client whose backoff is broken, amplifying an upstream problem

The tail-ratio row is the one that pairs best with N+1 hunting, because the two often appear together. An endpoint whose p99 is ten times its p50 has a path that is rare and expensive, and a loop that only runs when a cache misses is a textbook producer of exactly that distribution.

Can an AI agent find N+1 API calls automatically?

This is a good use of an agent precisely because it is mechanical. Finding an N+1 by hand means knowing which endpoint to suspect, then reading enough of its traffic to notice a repetition. An agent can ask the question of every endpoint at once, which is the part people do not do because it is tedious rather than hard.

What it needs is the ability to express the criteria itself rather than read someone's dashboard. With yeet, and tools like httpscope that model endpoints from captured traffic, the agent reads one page describing the available metrics and then composes the comparison it wants: endpoints whose densest second exceeds twenty calls, or whose p99 is more than ten times their p50, with the sample counts attached so it can tell a finding from a fluke. Because nothing in that surface is pre-judged, "chatty" is a number the agent picks rather than a threshold baked into a product.

The follow-up is where it earns its place. Having found the burst, the agent can pull the actual calls in that second and show you that they are the same request path with a different id, which is the evidence that turns "this endpoint is slow" into "this endpoint calls /users/{id} once per row". That is a complete finding rather than a lead, and it took one investigation rather than an afternoon.

How do I start counting API calls on a Linux box?

Get the inventory first, because you cannot count calls on endpoints you have not enumerated. apiwatch does that in one command, from traffic rather than from configuration:

curl -fsSL https://yeet.cx | sh    # install yeet, once
yeet run gh:yeet-src/apiwatch      # list this box's APIs from 30s of live traffic

What comes back is every HTTP API the machine serves and calls, with the process behind each one, the endpoints they serve, the status codes they return, and which outside APIs the machine calls in turn. For N+1 hunting the outbound half is the interesting one, because a service making two hundred calls a minute to one third-party host is visible there immediately, and that list is usually the first time anyone has seen it written down.

One honest limit for this particular question: apiwatch alerts on failures and availability, so it will show you the calls and their volume but will not flag an N+1 as such. Counting bursts per endpoint and tying them to the inbound request that caused them is the layer above it. Knowing which tool answers which question saves you from concluding that a quiet alert channel means a healthy service.

The bottom line: N+1s are a counting problem, not a timing one

Every tool that watches for failures misses this bug, and every tool that watches latency reports it without explaining it. The reason is that an N+1 is not slow in any individual way. It is slow in aggregate, because it does the same correct thing far too many times.

So count. The densest one-second window per dependency separates a handler that fans out normally from one running a loop, and it does so without a request identifier, without instrumenting the ORM, and without catching only the database half of the problem. Then read the calls in that second, confirm they are the same path with a different parameter, and you have the line of code.

Frequently asked questions

What is an N+1 query problem?

It is a pattern where fetching a collection of N items triggers one additional query per item, so N items cost N+1 round trips instead of one or two. It appears in ORMs that lazily load relations, and equally in code that loops over results calling an HTTP API per record. Nothing fails; the cost is entirely in the number of round trips, which grows with the size of the data.

How do I detect an N+1 in production?

Count calls per second per dependency rather than measuring their latency. An N+1 produces a dense burst, typically more than twenty calls to the same destination inside one second tied to a single inbound request, while a handler with normal fan-out stays in the low single digits regardless of load. That density is the signature, and it shows up without a request identifier.

Why doesn't my APM show the N+1?

Most APMs will show you that an endpoint is slow and, if your ORM is instrumented, that it spent its time in database calls. What an APM often will not do is make the repetition obvious, especially when the loop calls another service over HTTP instead of the database, which falls outside the ORM instrumentation entirely. You see the symptom clearly and the cause only if the right layer happened to be instrumented.

Can I find N+1 calls without instrumenting my application?

Yes, by reading the calls at the kernel rather than inside the process. Every HTTP request a service makes crosses the socket layer, so counting them per destination and per second with eBPF requires no library, no agent in the runtime and no code change, and it covers services you did not write and runtimes with no instrumentation available.

How many calls per second means an N+1?

More than about twenty to the same destination inside one second, tied to one inbound request, is a reliable practical threshold. It is deliberately conservative: a handler legitimately calling three or four services never reaches it, while a loop over even a few dozen rows passes it immediately. Tune it down if your handlers are naturally quiet, and check the sample count before treating a single burst as a finding.

What is the difference between an N+1 and a retry storm?

An N+1 is one caller making many distinct calls, usually to the same path with different parameters, as part of serving one request. A retry storm is one caller repeating the same call because it keeps failing or timing out. Both show as a burst, and the request paths tell them apart: varying identifiers point to a loop, identical requests point to retries.

Does this work for gRPC or GraphQL N+1s?

It depends on the decoder. GraphQL resolvers are a notorious source of N+1s and the resulting calls are ordinary HTTP to the backend, so they count like anything else. gRPC rides on HTTP/2 and needs a decoder that handles its framing, such as grpcsnoop, and it reports errors in trailers rather than status codes, so a tool reading statuses alone will undercount gRPC failures even while counting the calls correctly.

Can an AI agent find chatty API calls automatically?

Yes, and it suits an agent better than a person because it is a comparison run across every endpoint rather than a hunch about one. Given a queryable surface over the traffic, the agent asks for endpoints whose densest second exceeds a threshold it chooses, then pulls the calls in that window to show the repetition. The second step is what turns the finding into evidence.

Will fixing an N+1 always make the endpoint faster?

Usually and not always, which is worth checking before you rewrite anything. Collapsing fifty round trips into one join removes the round-trip cost, which dominates when the dependency is remote. If the calls were local, already cached and individually sub-millisecond, the win can be smaller than the complexity you add. Measure the endpoint's latency distribution before and after rather than assuming.

How do I tie the calls back to the request that caused them?

By the time window and the calling process, which is enough in practice when the burst is dense. The calls belonging to one inbound request cluster tightly, usually inside the same second, from the same process, to the same destination. That is why burst density rather than total volume is the right metric: it survives concurrency that would make a simple per-minute count ambiguous.

Sources

1. yeet API debugging use case, which names the slow query and the one running 200 times among what an agent finds in dependency traffic. https://yeet.cx/use-cases/api-debugging
2. apiwatch, the published eBPF API watchdog that lists every API a machine serves and calls from live traffic. https://github.com/yeet-src/apiwatch
3. RFC 9110, HTTP Semantics, for the request semantics each counted call is read against. https://www.rfc-editor.org/rfc/rfc9110.html
4. Linux kernel BPF program types, for the attach points a capture uses to count calls without entering the path. https://docs.kernel.org/bpf/libbpf/program_types.html
5. yeet documentation, the runtime the capture runs on. https://yeet.cx/docs/

Related resources

1. How do I get an AI agent to debug my API, on giving an agent the traffic instead of the code. https://yeet.cx/topical-takes/ai-agent-debug-api
2. How do I detect API drift after a deploy, for the other thing a shape model catches that status codes cannot. https://yeet.cx/topical-takes/detect-api-drift
3. Find which process is slowing your Linux machine, for the per-process accounting beneath a slow endpoint. https://yeet.cx/topical-takes/find-which-process-is-slowing-your-machine-linux