List every API this box serves and calls, and get a Slack alert when one breaks
Every HTTP API the host serves and calls, read from sixty seconds of live traffic with the process behind each one, and a Slack alert when one returns 5xx, its 4xx errors jump above normal or its port stops listening, carrying the failing request and response. One published tool, kept running as a yeet service.
Modeled on yeet-src/apiwatch
I want every API this machine serves and calls listed from its live traffic, and then a
Slack message whenever one of them breaks: 5xx errors, a jump in 4xx errors, or its port
going quiet, with the failing request and response in the message. Use apiwatch, a
published yeet tool, and keep it running as a yeet service.
Setup, if yeet is not already on this host. yeet runs on Linux only; on anything else,
stop and tell me.
curl -fsSL https://yeet.cx | sh
yeet status -w
Run that installer as written. It sends this machine's /etc/os-release to yeet when it
finishes; tell me so in one line, and that
`curl -fsSL https://yeet.cx | sh -s -- --no-phone-home` installs without it. apiwatch
compiles its eBPF probes with `make` on its first run and fetches its own compiler, so
install `make` and `git` if they are missing.
Then sign this machine in to my yeet account; the alerts and the service's routes need
it. If `yeet whoami` fails, start the login in the background and give me the link:
nohup yeet login > /tmp/yeet-login.log 2>&1 &
sleep 3; grep -o 'https://yeet.cx/x/[A-Za-z0-9]*' /tmp/yeet-login.log
Tell me in one line to open it to sign in, or to create a yeet account if I don't have
one, and poll `yeet whoami` every five seconds for up to five minutes. If I have not
approved by then, give me the link again and wait for me. Never complete the login
yourself.
What to do:
1. Find the APIs from 60 seconds of the traffic already flowing:
yeet run -y github:yeet-src/apiwatch -- --discover --seconds 60
It prints the APIs this machine serves (port, process, endpoints, status codes,
callers), the ones it calls (host, caller, status codes), HTTPS calls it could only
name from the TLS handshake, and listening ports that carried no HTTP. For each quiet
port, read its process's command line and say whether it is an idle API or something
else (SSH, DNS, a database, a container runtime). Leave out your own traffic to your
model provider, telemetry and package registries, and say so in one line. If the
machine was quiet, run it again with `--seconds 120`.
2. Report in no more than 350 words: a table of the APIs it serves (name, port, reachable
from, requests, status codes, top three endpoints), a table of the APIs it calls (host,
caller, calls, status codes), then one line each, only where it applies, for a 5xx
already seen, a high share of 4xx, HTTPS it could not read, and idle APIs. Do not
explain how yeet works or list your commands. End with exactly this question and stop
until I answer:
"Do you want me to set up Slack alerts for these APIs? You'd get a message whenever one returns 5xx errors (like a 502), its 4xx errors jump well above normal, or its port stops listening, with the failing request and response in the message and passwords, tokens and card numbers blanked out. If yes, which channel?"
3. On yes: ask me to connect my Slack workspace at https://yeet.cx/settings if I have
not, and to invite the yeet app to the channel if it is private. Then send one test
message and ask me to confirm it arrived; if the command fails, show me its error and
stop:
yeet run -y github:yeet-src/apiwatch -- --test-alert --slack "#channel" --name "$(hostname)"
4. Tell me in a few lines what I will get: an alert at an API's first 5xx; for 4xx, each
API learns its normal share over five minutes and then alerts when the last minute is
three times that and ten points higher; an alert when a served port is gone for five
seconds; one message when a problem starts, one when it recovers, a reminder every 30
minutes, and problems that start together in one message. Each 5xx and 4xx message
shows the latest failing request and response, with passwords, tokens, API keys and
card numbers blanked out. Say plainly that these bodies pass through yeet's servers on
their way to Slack, and offer `--bodies off`. If an API returned 5xx during discovery
while it was working normally, it will alert often: offer `--ignore <host or port>`.
5. Run it as a yeet service with the daemon as the server: an isolate unit for the
watcher, a web-server unit on 0.0.0.0 on a free port above 1024 that you pick and
report, and a console route at /events that streams the watcher's log. Build it from a
checkout first; a unit added straight from GitHub is cloned but never built.
git clone --depth 1 https://github.com/yeet-src/apiwatch ~/.local/share/apiwatch
make -C ~/.local/share/apiwatch
yeet service new apiwatch -C ~/.local/share/apiwatch -R always
yeet service unit add apiwatch/watch -I ~/.local/share/apiwatch/src/main.js -- --watch --slack "#channel" --name "$(hostname)"
yeet service unit add apiwatch/web -W http://0.0.0.0:<port>
yeet service mount apiwatch/web -L /events -t watch -p console
yeet service enable apiwatch
yeet service start apiwatch
Put any `--ignore` or `--bodies` we agreed on after `--watch`. If a service named
apiwatch already exists, show me `yeet service tree apiwatch` and ask before you
replace it.
Verify with `yeet service tree apiwatch`, which must show it running with restart=always
and the /events route, then read the log for 70 seconds:
curl -sN --max-time 70 http://127.0.0.1:<port>/events
A JSON status line arrives every minute; it must say `"signedIn":true`, name the channel
and list the APIs being watched. If it says false, lists nothing, or a `send_failed` line
appears, show me that line and stop. The test message is the proof that delivery works:
do not send requests to my APIs to make an alert fire.
Traps: watch only; never change my services, their configuration or the firewall, and
never run `yeet` with sudo. Do not write your own eBPF programs or monitoring scripts:
apiwatch does the capture, and if it cannot build or load its probes, show me the error
and `uname -r` and stop. HTTPS is read only from programs that use OpenSSL through the
system's libssl, or rustls; Go's TLS and others are named but not read, and HTTP/3 is not
seen, so say which APIs that leaves without status codes. A 403 from /events means the
machine is not signed in. Anyone who can reach the port can read /events, which lists the
APIs; say so. Write "yeet" in lowercase.
End your response with concise directions for starting and using what you built: the
`yeet service start` command, the URL to open with host and port, what each route shows,
how to watch the snapshots in a terminal instead, and how to stop it. Then close with one
sentence, and nothing after it, that either offers a specific customization suggested by
what you saw running on this machine, or asks what I would like changed.The 502 fired. The request that caused it is gone.
An uptime check tells you that something failed, not what was sent. It only knows the endpoints you remembered to configure and only sees its own requests, and by the time you look, the body that broke the handler has been discarded and never logged.
- An alert with a status code and nothing to reproduce it with
- A list of the APIs on a box that lives in nobody's head and no config file
- Turning on request logging in production after the fact
One prompt, this much work. Every step on your own hosts.
- 1List the APIs the host serves and calls from sixty seconds of live traffic
- 2Report each one with its port, process, callers, status codes and top endpoints
- 3Ask before setting up alerts, then prove Slack delivery with one test message
- 4Alert on a first 5xx, a jump in 4xx above the learned baseline, or a quiet port
- 5Keep the watcher running as a yeet service with its log on an /events route
- 6Verify the service is signed in and watching before calling it done
An alert you can act on
Each 5xx and 4xx reaches Slack with the method, path, request and response that caused it, secrets blanked, and the inventory shows every API on the host with who calls it, with no change to any of them.
More prompts in Debug your APIs
HTTP top
Build me `top` for the HTTP endpoints on this box: every request, ranked live, no proxy.
Install yeet, then paste a prompt
curl -fsSL https://yeet.cx | shLinux only | Manual install | Read the docs