Build top for the HTTP endpoints on this host
A live web page of every plaintext HTTP endpoint crossing the host, ranked by traffic, with rate, latency and status mix, served by a yeet service. Reads request lines off the wire at the TC layer, so nothing in the apps changes.
Modelled on yeet-src/httpinspect
I want `top`, but for the HTTP endpoints on this host, as a web page I can open from my laptop. Build it as a yeet script served by a yeet service. Setup, if yeet is not already on this host: curl -fsSL https://yeet.cx | sh yeet status -w Read https://yeet.cx/docs/raw/scripts/index.md, scripts/ebpf.md and cli/services.md, and skim these working examples for the API before writing code: https://github.com/yeet-src/pktscope https://github.com/yeet-src/httpscope https://github.com/yeet-src/wssnoop A yeet script is one JavaScript file running in a V8 isolate: it imports a compiled .bpf.o, attaches the programs, reads the ring buffer, aggregates, and writes one JSON snapshot per second to its console. There is no Node, no npm, no exec or spawn. Scaffold with `yeet new <name>`, and use `yeet toolchain install` if `make` cannot find clang and bpftool. No terminal UI: the only front end is the web page below, and the console snapshots are what you read in the terminal while you build. Deliver the finished tool as a web page served by a yeet service, with the daemon as the server: `yeet service new`, an isolate unit for the script, a web-server unit on 0.0.0.0 on a free port above 1024 that you pick and report, a static route at / serving a public/ directory (one index.html with its script and styles inline is fine: plain DOM, no framework, no CDN), and a console route at /events that the page opens over WebSocket to receive each snapshot and draw the dashboard from it. Do not write a Python, Node or any other server process; if you find yourself starting one, stop and use a route. While you work, `yeet run .` prints the snapshots to the terminal, which is how you check the probe and the numbers before the page exists. If you need to read TLS traffic or traffic inside a network namespace such as a container, see httpscope and wssnoop above. What to build: 1. A BPF program attached at the TC layer (tcx) on every up interface, loopback included. For each TCP payload that starts with an HTTP/1.x request line, send the method, path, Host header, source and destination, and a timestamp to a ring buffer. Do the parsing of the first bytes in BPF and nothing else there; the isolate does the rest. 2. In the isolate, key endpoints by `METHOD host path`, collapsing query strings. Keep a running count, requests in the last second, and last-seen time. Pair responses to requests by socket to get on-the-wire latency and status codes bucketed as 2xx, 3xx, 4xx and 5xx. 3. Once a second, write one JSON snapshot of the table to the console: every endpoint with its count, req/s, last seen, p50 and p95 latency, status mix, and its last ten requests (time, full path with query string, status, latency, client address), plus total requests and distinct endpoints. 4. The web page, in public/: one table sorted by count with rank, method, host, path, count, req/s, p50, p95, the status mix and last seen, with the totals above it. Click a row to expand it in place and show that endpoint's last ten requests; click again to collapse. The expanded row must stay expanded, and keep its place, as snapshots redraw the table. No charts. Served by the yeet service described above. Verify with a local server and a loop of curl requests to a few paths, including one that returns 500. Snapshots must show the endpoints within a second or two with counts climbing and the 500 in the status mix. Then start the service, open the page, confirm the same rows appear there and keep updating without a reload, and expand the failing endpoint to see its recent requests with the 500s in them. Traps: this reads plaintext only, so HTTPS is invisible at this layer; say so on the page rather than silently showing nothing. The web-server unit answers only once the host is signed in with `yeet login`; if the page returns 403, that is why. Build it bottom up and verify each layer before the next: the probe loads and attaches, events arrive in the isolate, the aggregation is right, then the UI. "It compiled" is not "it works": the only proof is rows on screen with a count that climbs while you drive traffic at it. Tell me what you verified and how, and what it cannot see. End your response with concise directions for starting and using what you built: the `yeet service start` command, the URL to open with host and port, what each route shows, how to watch the snapshots in a terminal instead, and how to stop it. Then close with one sentence, and nothing after it, that either offers a specific customization suggested by what you saw running on this machine, or asks what I would like changed.
Your access log is what the app claims it served.
Per-endpoint traffic exists where someone added middleware or a proxy. The service that is hot this afternoon is usually the one nobody instrumented, and local services talking over loopback never appear anywhere.
- Adding an SDK to find out whether you need an SDK
- Loopback chatter between local services that no proxy sees
- A retry storm that looks like load until you can see the path
One prompt, this much work. Every step on your own hosts.
- 1Attach at the TC layer on every up interface and parse request lines in BPF
- 2Stream method, path, host and addresses to the isolate over the ring buffer
- 3Key by METHOD host path and keep counts, rates and last seen
- 4Pair responses by socket for latency and status mix
- 5Write a JSON snapshot of the table to the console every second
- 6Serve one table as a page from a yeet service, redrawn live over WebSocket
- 7Expand any row in place to inspect its recent requests
top, for HTTP
One screen that answers which endpoint is hot, how fast, and how often it fails, for every plaintext service on the host, with no change to any of them, in a browser tab from anywhere on your network.
Install yeet, then paste a prompt
curl -fsSL https://yeet.cx | shLinux only | Manual install | Read the docs