Amazon EC2
Using Spot Instances for the Workloads That Can Actually Tolerate Them
- Whether the workload can checkpoint or resume from an arbitrary interruption point
- Whether it's stateless enough to simply restart elsewhere
- Sensitivity to a hard two-minute (or less) shutdown deadline
- Whether Capacity Rebalancing is configured, and whether the workload tolerates overlap/gap during a rebalance event
Production premise
Spot Instances trade guaranteed availability for savings up to 90% off On-Demand — while running, a Spot Instance is identical to On-Demand in every way except one: AWS can reclaim it back with a two-minute warning whenever it needs the capacity.
Failure mode & inflection point
Teams that treat Spot as "just cheaper EC2" for a workload that isn't actually interruption-tolerant learn the difference the first time capacity is reclaimed mid-task with no checkpoint to resume from. The two-minute interruption notice is a best-effort warning, not a guarantee — EC2's own docs are explicit that an instance can be interrupted before the warning becomes available. A separate, earlier "rebalance recommendation" signal exists specifically to give more lead time, but it's also best-effort and can arrive alongside the two-minute notice rather than meaningfully before it.
Production guardrail
Only put genuinely interruption-tolerant, checkpoint-able, or stateless workloads on Spot — batch jobs, CI runners, fault-tolerant distributed compute, not anything that can't gracefully lose an in-progress unit of work. If using Auto Scaling Group Capacity Rebalancing to replace at-risk instances proactively, plan around both the documented default and its real edge case rather than picking one: AWS's documented default sequence waits for the replacement instance to pass health checks before terminating the old one — that's the behavior to design for. But two independent race conditions can produce early draining anyway even though neither contradicts the documented sequence: the rebalance recommendation and the two-minute Spot interruption notice can collide closely enough to fire together, and a load balancer's target-group deregistration delay (300 seconds by default) can outlast the 120-second Spot countdown, pulling the old instance out of rotation before the new one is confirmed healthy. Design the workload to tolerate a gap regardless of which path plays out — and if the 120-second window is tight, shortening the deregistration delay below it is the concrete lever, not just hoping the race doesn't happen.