DockerKubernetesSlack

List every API this box serves and calls, and get a Slack alert when one breaks

Every HTTP API the host serves and calls, read from sixty seconds of live traffic with the process behind each one, and a Slack alert when one returns 5xx, its 4xx errors jump above normal or its port stops listening, carrying the failing request and response. One published tool, kept running as a yeet service.

Modeled on yeet-src/apiwatch

I want every API this machine serves and calls listed from its live traffic, and then a
Slack message whenever one of them breaks: 5xx errors, a jump in 4xx errors, or its port
going quiet, with the failing request and response in the message. Use apiwatch, a
published yeet tool, and keep it running as a yeet service.

Setup, if yeet is not already on this host. yeet runs on Linux only; on anything else,
stop and tell me.

  curl -fsSL https://yeet.cx | sh
  yeet status -w

Run that installer as written. It sends this machine's /etc/os-release to yeet when it
finishes; tell me so in one line, and that
`curl -fsSL https://yeet.cx | sh -s -- --no-phone-home` installs without it. apiwatch
compiles its eBPF probes with `make` on its first run and fetches its own compiler, so
install `make` and `git` if they are missing.

Then sign this machine in to my yeet account; the alerts and the service's routes need
it. If `yeet whoami` fails, start the login in the background and give me the link:

  nohup yeet login > /tmp/yeet-login.log 2>&1 &
  sleep 3; grep -o 'https://yeet.cx/x/[A-Za-z0-9]*' /tmp/yeet-login.log

Tell me in one line to open it to sign in, or to create a yeet account if I don't have
one, and poll `yeet whoami` every five seconds for up to five minutes. If I have not
approved by then, give me the link again and wait for me. Never complete the login
yourself.

What to do:

1. Find the APIs from 60 seconds of the traffic already flowing:

     yeet run -y github:yeet-src/apiwatch -- --discover --seconds 60

   It prints the APIs this machine serves (port, process, endpoints, status codes,
   callers), the ones it calls (host, caller, status codes), HTTPS calls it could only
   name from the TLS handshake, and listening ports that carried no HTTP. For each quiet
   port, read its process's command line and say whether it is an idle API or something
   else (SSH, DNS, a database, a container runtime). Leave out your own traffic to your
   model provider, telemetry and package registries, and say so in one line. If the
   machine was quiet, run it again with `--seconds 120`.
2. Report in no more than 350 words: a table of the APIs it serves (name, port, reachable
   from, requests, status codes, top three endpoints), a table of the APIs it calls (host,
   caller, calls, status codes), then one line each, only where it applies, for a 5xx
   already seen, a high share of 4xx, HTTPS it could not read, and idle APIs. Do not
   explain how yeet works or list your commands. End with exactly this question and stop
   until I answer:
   "Do you want me to set up Slack alerts for these APIs? You'd get a message whenever one returns 5xx errors (like a 502), its 4xx errors jump well above normal, or its port stops listening, with the failing request and response in the message and passwords, tokens and card numbers blanked out. If yes, which channel?"
3. On yes: ask me to connect my Slack workspace at https://yeet.cx/settings if I have
   not, and to invite the yeet app to the channel if it is private. Then send one test
   message and ask me to confirm it arrived; if the command fails, show me its error and
   stop:

     yeet run -y github:yeet-src/apiwatch -- --test-alert --slack "#channel" --name "$(hostname)"

4. Tell me in a few lines what I will get: an alert at an API's first 5xx; for 4xx, each
   API learns its normal share over five minutes and then alerts when the last minute is
   three times that and ten points higher; an alert when a served port is gone for five
   seconds; one message when a problem starts, one when it recovers, a reminder every 30
   minutes, and problems that start together in one message. Each 5xx and 4xx message
   shows the latest failing request and response, with passwords, tokens, API keys and
   card numbers blanked out. Say plainly that these bodies pass through yeet's servers on
   their way to Slack, and offer `--bodies off`. If an API returned 5xx during discovery
   while it was working normally, it will alert often: offer `--ignore <host or port>`.
5. Run it as a yeet service with the daemon as the server: an isolate unit for the
   watcher, a web-server unit on 0.0.0.0 on a free port above 1024 that you pick and
   report, and a console route at /events that streams the watcher's log. Build it from a
   checkout first; a unit added straight from GitHub is cloned but never built.

     git clone --depth 1 https://github.com/yeet-src/apiwatch ~/.local/share/apiwatch
     make -C ~/.local/share/apiwatch
     yeet service new apiwatch -C ~/.local/share/apiwatch -R always
     yeet service unit add apiwatch/watch -I ~/.local/share/apiwatch/src/main.js -- --watch --slack "#channel" --name "$(hostname)"
     yeet service unit add apiwatch/web -W http://0.0.0.0:<port>
     yeet service mount apiwatch/web -L /events -t watch -p console
     yeet service enable apiwatch
     yeet service start apiwatch

   Put any `--ignore` or `--bodies` we agreed on after `--watch`. If a service named
   apiwatch already exists, show me `yeet service tree apiwatch` and ask before you
   replace it.

Verify with `yeet service tree apiwatch`, which must show it running with restart=always
and the /events route, then read the log for 70 seconds:

  curl -sN --max-time 70 http://127.0.0.1:<port>/events

A JSON status line arrives every minute; it must say `"signedIn":true`, name the channel
and list the APIs being watched. If it says false, lists nothing, or a `send_failed` line
appears, show me that line and stop. The test message is the proof that delivery works:
do not send requests to my APIs to make an alert fire.

Traps: watch only; never change my services, their configuration or the firewall, and
never run `yeet` with sudo. Do not write your own eBPF programs or monitoring scripts:
apiwatch does the capture, and if it cannot build or load its probes, show me the error
and `uname -r` and stop. HTTPS is read only from programs that use OpenSSL through the
system's libssl, or rustls; Go's TLS and others are named but not read, and HTTP/3 is not
seen, so say which APIs that leaves without status codes. A 403 from /events means the
machine is not signed in. Anyone who can reach the port can read /events, which lists the
APIs; say so. Write "yeet" in lowercase.

End your response with concise directions for starting and using what you built: the
`yeet service start` command, the URL to open with host and port, what each route shows,
how to watch the snapshots in a terminal instead, and how to stop it. Then close with one
sentence, and nothing after it, that either offers a specific customization suggested by
what you saw running on this machine, or asks what I would like changed.

The 502 fired. The request that caused it is gone.

An uptime check tells you that something failed, not what was sent. It only knows the endpoints you remembered to configure and only sees its own requests, and by the time you look, the body that broke the handler has been discarded and never logged.

  • An alert with a status code and nothing to reproduce it with
  • A list of the APIs on a box that lives in nobody's head and no config file
  • Turning on request logging in production after the fact

One prompt, this much work. Every step on your own hosts.

  1. 1List the APIs the host serves and calls from sixty seconds of live traffic
  2. 2Report each one with its port, process, callers, status codes and top endpoints
  3. 3Ask before setting up alerts, then prove Slack delivery with one test message
  4. 4Alert on a first 5xx, a jump in 4xx above the learned baseline, or a quiet port
  5. 5Keep the watcher running as a yeet service with its log on an /events route
  6. 6Verify the service is signed in and watching before calling it done

An alert you can act on

Each 5xx and 4xx reaches Slack with the method, path, request and response that caused it, secrets blanked, and the inventory shows every API on the host with who calls it, with no change to any of them.

More prompts in Debug your APIs

  • HTTP top

    DockerKubernetes

    Build me `top` for the HTTP endpoints on this box: every request, ranked live, no proxy.

Install yeet, then paste a prompt

curl -fsSL https://yeet.cx | sh

Linux only  |  Manual install  |  Read the docs