Debugging APIs

With yeet, your agent can follow the failure into the network and see what actually happened.

curl -fsSL https://yeet.cx | sh

Linux only  |  Manual install  |  Read the docs

When logs aren’t enough

yeet comes in when your dashboards are green and the host isn’t. We sit alongside tools like Datadog and Grafana, replacing the custom exporter, the sidecar and the SSH-and-tcpdump session.

Your agent without yeet

502 on POST /checkout

  • what did the API return?curl gets 502 Bad Gateway. The log says “upstream connect error or disconnect/reset before headers.”
  • what did checkout send to payments?TLS records tcpdump can’t read.
  • who sent the reset?Not in any log.
  • drops, DNS, connection pool?ssh: Permission denied (publickey).

A guess: “payments is flaky, add a retry.” The rest waits for someone with access to the host.

Your agent with yeet

502 on POST /checkout

  • what did the API return?41 of 230 POST /checkout returned 502 in the last two minutes, all from checkout on host-03 and host-07.
  • what did checkout send to payments?POST /v1/charges went out on a reused connection and got no response. Postgres and Redis calls returned normally.
  • who sent the reset?Not payments. The resets reached checkout from the load balancer, on connections idle for more than 350 seconds.
  • drops, DNS, connection pool?No kernel drops. DNS answers unchanged. checkout’s pool keeps idle connections for up to 600 seconds.

A verdict: checkout’s connection pool keeps idle connections for 600 seconds, and the load balancer closes them at 350. Requests sent on those dead connections get reset and come back as 502s. It’s the infrastructure, not the code: set the pool’s idle timeout under 350 seconds.

Runtime test fixtures

The probe your agent writes for one incident doesn’t have to go away. yeet can run it as a service on the host, all day.

Point yeet at your API and it becomes part of how you ship. Now your agent can read traffic continuously and serve it to your CI. Now your test suite checks what production actually does.

  • An API drift watcher that runs on every deploy
  • An endpoint inventory that flags new routes
  • A Prometheus metric for something your app never exported
  • A Slack alert when a client starts retrying
Runtime test fixtures, as a loopOn your host, a yeet service reads every request and response your API sends and serves them as fixtures over an HTTP route. Your CI test suite pulls them and runs real traffic as test cases. The deploy that passes ships back to your API, and a response that drifts fails the build.your APIrequests and responsesyeet serviceruns all dayfixtures routeserved over HTTPCI test suitereal traffic as test casesdeploydrift fails the buildreadspulls fixturesshipsevery deploy is tested against what production actually did

How your agent works with yeet

Your agent starts with a question, uses live system data to form a hypothesis, and keeps investigating until it has an answer.

  1. 1

    Symptoms

    What the API actually did, read off the wire.

    HTTP, inside TLS

    • Which endpoint is slow, and how often it fails

      Rate, errors and latency per endpoint and per container, with no SDK in the app.

    • The response bodies of the 500s

      What the API returned when the logs only say “internal error”, with the request that caused it.

    • What your app sends over HTTPS

      Every request and response, decrypted at the TLS library. No proxy, no certificate.

    • Which client is in a retry storm

      Retries per original request, per client. The storm your logs record as ordinary traffic.

    • Which client keeps getting 401s

      Auth failures per client and token issuer. The token itself is never stored or shown.

    • API drift after a deploy

      The agent that made the change compares live responses before and after the deploy.

    • Where the live API drifts from its spec

      Endpoints, fields and status codes the running service returns that your OpenAPI spec doesn’t describe, including changes that never touched the code.

  2. 2

    Dependencies

    What the service asked of everything it depends on, and what came back.

    SQL, Redis, gRPC, third-party APIs

    • The slow query, and the one running 200 times

      Postgres and MySQL statements by shape, latency and count, tied to the process that sent them.

    • What your app does to Redis

      Commands and latency per client, including the KEYS call nobody meant to ship.

    • What the third-party API actually sent

      The response at your end of the connection, when the vendor’s docs and its behavior disagree.

    • The conversation between two services

      gRPC fields, SQL statements and WebSocket frames between containers, as they cross the wire.

    • What the webhook actually sent

      The payload and headers your process received, to settle why the signature check failed.

    • Rate-limit headroom with third-party APIs

      How much of each provider’s limit is left, by the process spending it.

  3. 3

    What is underneath

    The connections, packets and lookups the application never sees.

    TCP, TLS, DNS, the kernel

    • Who is resetting the connections

      Which side sent the RST, the process that owned the socket, and the peer it was talking to.

    • Where the kernel dropped the packets

      Every drop with the kernel’s own reason: a netfilter rule, a full buffer, a bad checksum.

    • Who timed out first

      One timeline for client and server, and which side closed the connection.

    • Is the connection pool exhausted

      Open connections per process and peer, against requests waiting for one.

    • Is the client reusing connections

      New connections against requests, per peer. Find the TLS handshake paid on every call.

    • Large payloads fail, small ones work

      Retransmits of full-size segments and the kernel’s “fragmentation needed” messages point at an MTU problem.

    • DNS failures and stale answers

      What each process resolved, what failed, and where it connected anyway.

    • TLS handshake failures

      The handshake error, the server name and the process that made the call.

Make your machines readable

See what is actually happening with your APIs in < 60 secs with yeet

curl -fsSL https://yeet.cx | sh

Linux only  |  Manual install  |  Read the docs