Installing the yeet Terraform Provider
A complete working example of everything in this guide is available at github.com/yeet-src/terraform-aws-yeet-ec2-termination-handler.
This guide takes you from an empty directory to a working yeet deployment on AWS, managed entirely with Terraform. It covers:
- Installing and configuring the yeet Terraform provider
- Registering hosts that Terraform manages directly
- Registering autoscaled hosts (e.g. ECS cluster nodes) that AWS creates and destroys outside of Terraform
- The EC2 termination handler — a small Lambda function that automatically deregisters autoscaled hosts from yeet when AWS terminates them
How yeet host registration works
Every machine running the yeet agent is a host. A host registers itself by logging in with two values:
- API key — your account credential. The same key is shared by all your hosts.
- Prune key — a per-host secret (any unique string, typically a UUID) chosen at registration time. Whoever holds the prune key can later prune (deregister) that host via the yeet API.
The prune key exists to solve a lifecycle problem: when a machine is destroyed, something needs to tell yeet it is gone, and that something needs a credential scoped to just that host. How you manage prune keys depends on who owns the machine's lifecycle:
| Who creates/destroys the host | Pattern |
|---|---|
| Terraform (VMs, bare metal) | Terraform generates the prune key, creates a yeet_host resource with it, and destroys that resource — which prunes the host — when the machine is destroyed. |
| An AWS Auto Scaling group | The instance generates its own prune key at boot, registers, and stores the key as an EC2 tag on itself. A Lambda fires on instance termination, reads the tag, and calls the prune API. |
Prerequisites
You will need:
- Terraform ≥ 1.5 — installation instructions
- An AWS account with credentials configured locally (
aws configureor environment variables). The examples assume your credentials can create Lambda functions, IAM roles, and EventBridge rules. - A yeet API key — go to yeet.cx/settings, then click New under Platform Access Tokens
- Node.js 20+ — the Lambda is built automatically during
terraform apply
Verify your tooling:
terraform version
aws sts get-caller-identity
node --version
Install the provider
Create a working directory and a terraform.tf declaring the providers you need. The yeet provider is published on the public Terraform Registry as yeet-src/yeet, so Terraform downloads it automatically — no manual installation steps.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.0"
}
yeet = {
source = "yeet-src/yeet"
version = ">= 0.12.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.0"
}
}
}
The provider takes a single argument: your API key. Never hardcode it — pass it in as a sensitive variable, or better, read it from a secret store.
Option A — sensitive variable (providers.tf):
variable "yeet_api_key" {
type = string
sensitive = true
}
provider "yeet" {
api_key = var.yeet_api_key
}
Supply it via the environment so it never lands in a file:
export TF_VAR_yeet_api_key="<your api key>"
Option B — AWS Secrets Manager (recommended for teams; the key is fetched at plan/apply time and shared with the Lambda later without ever being committed):
aws secretsmanager create-secret --name yeet_api_key --secret-string "<your api key>"
data "aws_secretsmanager_secret_version" "yeet_api_key" {
secret_id = "yeet_api_key"
}
provider "yeet" {
api_key = data.aws_secretsmanager_secret_version.yeet_api_key.secret_string
}
# Used by later examples in this guide:
locals {
yeet_api_key = data.aws_secretsmanager_secret_version.yeet_api_key.secret_string
}
Then initialize:
terraform init
This downloads the providers and writes .terraform.lock.hcl, which pins exact provider versions and checksums. Commit the lock file to version control so everyone on the team — and CI — uses identical provider builds.
Register a Terraform-managed host
For machines whose lifecycle Terraform owns end to end, declare one yeet_host resource per machine. Generate the prune key with the random provider so Terraform remembers it in state:
resource "random_uuid" "prune_key" {}
resource "yeet_host" "host" {
prune_key = random_uuid.prune_key.result
}
Then have the machine itself install the agent and log in at first boot. With cloud-init, pass the two values into your user data and run:
runcmd:
- ["sh", "-c", "curl -fsSL https://yeet.cx | sh"]
- ["yeet", "login", "--key", "${yeet_api_key}", "--prune-key", "${yeet_host_prune_key}"]
where ${yeet_api_key} is your API key and ${yeet_host_prune_key} is yeet_host.host.prune_key, interpolated via templatefile().
terraform apply registers the host and boots the machine; terraform destroy destroys the yeet_host resource, which prunes the host from yeet. If this is your only use case, you can stop reading here.
Register autoscaled hosts (ECS cluster nodes)
Instances in an Auto Scaling group are created and terminated by AWS, not Terraform, so there is no per-instance yeet_host resource to create or destroy. Instead, each instance registers itself at boot:
- Generate a fresh prune key locally (
uuidgen) - Install the agent and log in with it
- Tag the instance with the prune key, so it can be recovered at termination time
Add this to the Auto Scaling group's user data:
#!/usr/bin/env bash
set -euo pipefail
INSTANCE_ID=$(ec2-metadata --instance-id | cut -d ' ' -f 2)
curl -fsSL https://yeet.cx | sh
PRUNE_KEY=$(uuidgen)
yeet login --key "${yeet_api_key}" --prune-key "$PRUNE_KEY"
aws ec2 create-tags --region ${aws_region} --resources "$INSTANCE_ID" \
--tags Key=YeetHostPruneKey,Value="$PRUNE_KEY"
${yeet_api_key} and ${aws_region} are template variables filled in by Terraform's templatefile() when building the launch template.
The aws ec2 create-tags call requires the instance's IAM role to allow tagging. Attach a policy like this to the instance profile used by your Auto Scaling group:
resource "aws_iam_policy" "ec2_tagging" {
name = "ec2-self-tagging"
policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Action = ["ec2:CreateTags"]
Resource = "*"
}]
})
}
At this point hosts appear in yeet when instances boot — but nothing removes them when instances terminate. That is the termination handler's job.
The EC2 termination handler
Architecture
When any EC2 instance in the account reaches the terminated state, AWS emits an event onto the default EventBridge bus. The termination handler is three pieces wired together:
EC2 instance terminates
│
▼
EventBridge rule matches source=aws.ec2, state=terminated
│
▼
Lambda (Node.js 20) reads the YeetHostPruneKey tag off the
│ terminated instance via ec2:DescribeTags
▼
POST https://api.yeet.cx/hosts/prune
authenticated with your API key,
body contains the prune key
EC2 tags outlive the instance. After termination, the instance is gone but its tags remain queryable via DescribeTags for a while. That window is what lets the Lambda recover the prune key from an instance that no longer exists — no per-host state needs to live anywhere else.
The Lambda source
Project layout for the function:
ec2_termination_handler/
├── main.tf # all Terraform resources
├── variables.tf
├── outputs.tf
└── lambda/
├── index.js # the handler
├── package.json
└── Makefile # builds ecs_lifecycle_handler.zip
The complete lambda/index.js:
const {
EC2Client,
DescribeTagsCommand
} = require("@aws-sdk/client-ec2");
const ec2Client = new EC2Client();
async function getHostPruneKeyFromEC2Tags(instanceId) {
const command = new DescribeTagsCommand({
Filters: [
{
Name: "resource-id",
Values: [instanceId]
},
{
Name: "key",
Values: ["YeetHostPruneKey"]
}
]
});
const resp = await ec2Client.send(command);
if (resp.Tags && resp.Tags.length > 0) {
return resp.Tags[0].Value;
}
throw new Error(`No YeetHostPruneKey tag found for instance ${instanceId}`);
}
async function pruneHost(instanceId, pruneKey) {
const url = "https://api.yeet.cx/hosts/prune";
const apiKey = process.env.YEET_API_KEY;
if (!apiKey) {
throw new Error("YEET_API_KEY environment variable not set");
}
const resp = await fetch(url, {
method: "POST",
headers: {
"Content-Type": "application/json",
"Authorization": `Bearer ${apiKey}`
},
body: JSON.stringify({
prune_key: pruneKey
})
});
if (!resp.ok) {
const errorText = await resp.text();
throw new Error(`yeet API error: ${resp.status} - ${errorText}`);
}
}
exports.handler = async (event) => {
const instanceId = event.detail["instance-id"];
try {
const pruneKey = await getHostPruneKeyFromEC2Tags(instanceId);
await pruneHost(instanceId, pruneKey);
return { statusCode: 200, body: "Success" };
} catch (error) {
console.error("Error handling instance termination:", error);
throw error;
}
};
The handler re-throws on failure after logging. This marks the invocation as failed, which triggers Lambda's automatic retry behavior: for async invocations, Lambda retries a failed invocation twice (roughly 1 minute and 2 minutes after the failure), three attempts total. If the handler swallowed errors instead, every invocation would look successful and failed prunes would silently leak hosts.
Idempotency. Pruning an already-pruned host is a no-op, so retried events are safe.
Dependencies and the build
lambda/package.json:
{
"name": "ec2-termination-handler",
"version": "1.0.0",
"description": "EC2 instance termination handler for yeet",
"main": "index.js",
"dependencies": {
"@aws-sdk/client-ec2": "^3.700.0"
}
}
lambda/Makefile:
.PHONY: build deps dist clean
.DEFAULT_GOAL := build
build: dist
deps:
npm install
dist: deps
zip -r ecs_lifecycle_handler.zip index.js node_modules package.json package-lock.json
The Terraform resources
The complete main.tf. Every resource carries count = var.enabled ? 1 : 0, so the whole handler can be switched on or off per environment with one boolean. The null_resource runs npm install and the archive_file data source zips the result — no manual build step needed.
resource "null_resource" "lambda_deps" {
triggers = {
package_json = filemd5("${path.module}/lambda/package.json")
}
provisioner "local-exec" {
command = "cd ${path.module}/lambda && npm install"
}
}
data "archive_file" "lambda" {
type = "zip"
source_dir = "${path.module}/lambda"
output_path = "${path.module}/lambda/ecs_lifecycle_handler.zip"
excludes = ["ecs_lifecycle_handler.zip"]
depends_on = [null_resource.lambda_deps]
}
resource "aws_lambda_function" "ec2_termination_handler" {
count = var.enabled ? 1 : 0
filename = data.archive_file.lambda.output_path
source_code_hash = data.archive_file.lambda.output_base64sha256
function_name = "ec2-termination-handler-${var.env}"
role = aws_iam_role.ec2_termination_lambda_role[0].arn
handler = "index.handler"
runtime = "nodejs20.x"
timeout = 60
environment {
variables = {
YEET_API_KEY = var.yeet_api_key
}
}
}
resource "aws_iam_role" "ec2_termination_lambda_role" {
count = var.enabled ? 1 : 0
name = "ec2-termination-lambda-role-${var.env}"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "lambda.amazonaws.com"
}
}]
})
}
resource "aws_iam_role_policy" "ec2_termination_lambda_policy" {
count = var.enabled ? 1 : 0
name = "ec2-termination-lambda-policy"
role = aws_iam_role.ec2_termination_lambda_role[0].id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"ec2:DescribeTags"
]
Resource = "*"
},
{
Effect = "Allow"
Action = [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
]
Resource = "arn:aws:logs:*:*:*"
}
]
})
}
resource "aws_cloudwatch_event_rule" "ec2_terminated" {
count = var.enabled ? 1 : 0
name = "ec2-terminated-${var.env}"
description = "Capture EC2 instance terminated events"
event_pattern = jsonencode({
source = ["aws.ec2"]
detail-type = ["EC2 Instance State-change Notification"]
detail = {
state = ["terminated"]
}
})
}
resource "aws_cloudwatch_event_target" "ec2_termination_lambda" {
count = var.enabled ? 1 : 0
rule = aws_cloudwatch_event_rule.ec2_terminated[0].name
target_id = "EC2TerminationLambda"
arn = aws_lambda_function.ec2_termination_handler[0].arn
}
resource "aws_lambda_permission" "allow_eventbridge" {
count = var.enabled ? 1 : 0
statement_id = "AllowExecutionFromEventBridge"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.ec2_termination_handler[0].function_name
principal = "events.amazonaws.com"
source_arn = aws_cloudwatch_event_rule.ec2_terminated[0].arn
}
variables.tf:
variable "env" {
type = string
description = "Environment name"
}
variable "yeet_api_key" {
type = string
description = "yeet API key for authentication"
sensitive = true
}
variable "enabled" {
type = bool
description = "Enable EC2 termination handler"
default = true
}
outputs.tf:
output "lambda_function_arn" {
description = "ARN of the EC2 termination handler Lambda function"
value = var.enabled ? aws_lambda_function.ec2_termination_handler[0].arn : null
}
output "lambda_function_name" {
description = "Name of the EC2 termination handler Lambda function"
value = var.enabled ? aws_lambda_function.ec2_termination_handler[0].function_name : null
}
output "lambda_role_arn" {
description = "ARN of the Lambda IAM role"
value = var.enabled ? aws_iam_role.ec2_termination_lambda_role[0].arn : null
}
output "eventbridge_rule_name" {
description = "Name of the EventBridge rule"
value = var.enabled ? aws_cloudwatch_event_rule.ec2_terminated[0].name : null
}
Wiring it into your stack
Call it as a module from your root configuration, passing the same API key the hosts register with:
module "ec2_termination_handler" {
source = "./ec2_termination_handler"
env = "prod"
yeet_api_key = local.yeet_api_key
enabled = true
}
Then:
terraform init # picks up the new module
terraform plan
terraform apply
The boot-time self-registration and the termination handler must be enabled together. If hosts register but the handler is disabled, terminated instances linger in your yeet host list forever. If the handler runs but hosts don't tag themselves, every termination produces a No YeetHostPruneKey tag found error in the logs.
Operations and edge cases
Testing end to end. Terminate one instance in your Auto Scaling group (the ASG will replace it):
aws autoscaling terminate-instance-in-auto-scaling-group \
--instance-id i-0123456789abcdef0 \
--no-should-decrement-desired-capacity
Then check the function's log group:
aws logs tail /aws/lambda/ec2-termination-handler-prod --since 10m
A successful run logs the invocation with no error lines, and the host disappears from your yeet host list.
Scoping. The EventBridge rule matches every EC2 termination in the account and region, not just your yeet hosts. For unrelated instances the Lambda throws No YeetHostPruneKey tag found, EventBridge retries twice, and the event is dropped. This is harmless but noisy in accounts with lots of non-yeet churn. If that matters, either narrow the event pattern upstream or change the handler to treat a missing tag as a clean no-op return instead of an error — the trade-off is that a no-op return also silences genuinely broken registrations.
Dead-letter queue. To make failed prunes inspectable rather than dropped after the third retry, add an SQS queue as the function's async dead-letter target (dead_letter_config on the Lambda, plus sqs:SendMessage on the role).
Monitoring. The two CloudWatch metrics worth alarming on are Errors and Throttles on the function. Both are emitted automatically.
API key rotation. The key lives in the Lambda's environment. If you rotate it in Secrets Manager, re-run terraform apply — the data source picks up the new value and Terraform updates the function's environment in place.
Quick reference
| Step | Command / artifact |
|---|---|
| Install providers | terraform init |
| Store API key | aws secretsmanager create-secret --name yeet_api_key --secret-string "<key>" |
| Terraform-managed host | yeet_host resource + yeet login in cloud-init |
| Autoscaled host, boot | yeet login --prune-key $(uuidgen) + YeetHostPruneKey EC2 tag |
| Autoscaled host, termination | EventBridge → Lambda → POST /hosts/prune |
| Build the Lambda | Automatic on terraform apply (requires Node.js 20+) |
| Verify | aws logs tail /aws/lambda/ec2-termination-handler-<env> |