The Ping-Pong: How to Save Config to Flash Without Losing It to a Power Cut
A pattern every embedded engineer should have in their toolbox — and why it applies far beyond firmware.
- The problem sounds trivial
- What flash actually gives you (and doesn’t)
- The insight: atomic-looking commits from non-atomic hardware
- The ping-pong design
- The commit “trick” — magic word programmed last
- The tricky bit: when a sector fills up
- Wear leveling — a lovely side effect
- What if you need more space?
- When this pattern generalizes
- When you should NOT reach for this
- Takeaways
The problem sounds trivial
Your device needs to remember a few things across reboots. Some configuration, a pairing key, a session counter, a device ID. A few hundred bytes at most.
You reach for the obvious answer: write it to flash.
You put together a naïve implementation. It works on the bench. You ship. And then, weeks later, a customer emails you:
“The device forgot all its settings after the power blip yesterday.”
You investigate. You reproduce it. And you realize the bug isn’t a bug in the traditional sense — it’s a fundamental property of how flash memory works, and your storage code was never actually safe.
This article is about the pattern that fixes it. It’s called ping-pong sector storage, and once you understand why it works, you’ll see the same shape of solution in filesystems, databases, distributed consensus, and blockchain commit protocols.
Let’s build it up from first principles.
What flash actually gives you (and doesn't)
If you’ve never worked at the flash bit level, three properties will surprise you.
1. Writes are one-way. When you write to flash, you can only flip bits from 1 to 0. To change a 0 back to 1 requires an erase operation. This is a physical property of the underlying NOR/NAND storage cells — floating gates that trap charge, and the erase is the process that pulls that charge out.
2. Erase is coarse. The smallest thing you can erase is a sector — typically 4 KB or larger. You cannot erase a single byte. Even changing one bit that’s currently 0 back to 1 requires erasing the entire enclosing 4 KB block.
3. Erase takes real time. A sector erase on typical QSPI flash takes 30–400 milliseconds. That’s an eternity in CPU cycles. During that window, if power drops, the sector is in an unpredictable, partially-erased state — some bytes may have flipped, others may still hold their old values, and the sector may not even be reliably readable.
Now consider what your naïve implementation looked like:
save_settings():
erase_sector(settings_sector) ← 200 ms window of vulnerability
write_bytes(settings_sector, buf, len)
That first line is your enemy. Any power cut inside that 200 ms window destroys your settings entirely. And the second line has its own vulnerability window — power cuts between the erase and the write leave you with a sector full of 0xFF, which your read routine interprets as “factory fresh.”
If your device has any chance of being in the field where operators can unplug it, where batteries can droop, where brown-outs happen — this design is a time bomb.
The insight: atomic-looking commits from non-atomic hardware
We can’t make individual flash operations atomic — that’s a hardware constraint we don’t control. But we can build a scheme on top of the raw flash where every possible failure state leaves the system in a known-good position.
The trick has two parts:
Part 1 — Never overwrite in place. When we want to save new settings, we write them to a different location. The old settings stay intact until the new ones are fully committed.
Part 2 — Version everything. Every saved snapshot carries a monotonically-increasing sequence number. On boot, we scan our storage area and pick the highest-versioned entry that also passes integrity checks (magic word + CRC). Corrupt / partial entries are silently skipped.
If a power cut happens between saves, the previous good copy is what gets loaded next boot. If it happens during the save of a new copy, that new copy fails its integrity check on boot and gets ignored — and again the previous copy is loaded.
The system never lands in “everything is gone.” That’s the invariant we’re defending.
The ping-pong design
Take two flash sectors, side by side. Call them A and B. Each holds a ring of fixed-size records — say 21 records of 192 bytes each, fitting neatly into a 4 KB sector.
Sector A (4 KB, 21 slots) Sector B (4 KB, 21 slots)
┌────────────────────────────────┐ ┌────────────────────────────────┐ │ slot0 seq=1 (initial) │ │ slot0 <empty, 0xFF> │ │ slot1 seq=2 (config change)│ │ slot1 <empty> │ │ slot2 seq=3 (adopted pan) │ │ slot2 <empty> │ │ slot3 seq=4 (key rotation) │ │ ... │ │ ... │ │ │ │ slot20 seq=21 (last slot) │ │ │ └────────────────────────────────┘ └────────────────────────────────┘ ACTIVE IDLE
An update never touches an existing slot. It writes to the next empty slot within the active sector. That’s just a program operation — no erase, no danger window. The seq counter increments.
Boot scans both sectors, finds every slot that has a valid magic word and correct CRC over its payload, and picks the winner: the one with the highest seq. That becomes the loaded state.
The commit "trick" — magic word programmed last
Here’s the elegant part. Each record has a small header, and the first thing in the header is a 4-byte magic word (e.g. 0x3153564E for the ASCII “NVS1”). The rule is: write the payload first, write the magic word last.
Why? Because if power dies between writing the payload and writing the magic word, the slot on flash will have the payload bytes but the magic word will still read as 0xFFFFFFFF (the erased state). Your boot-time scanner treats “magic mismatch” as “this slot is empty, skip it.” So the partially-written slot behaves exactly like a never-touched one, and your previous good record still wins.
This is the moral equivalent of a commit point in a database transaction — the moment when a bunch of separate operations suddenly become observable as a single atomic thing. The magic word is your commit fence. Everything before it is speculative; after it, the change is real.
Consequence: every update either fully succeeds or fully appears not to have happened. There is no half-committed state that survives to be read.
The tricky bit: when a sector fills up
Eventually you use up all 21 slots in sector A. What now?
Your next update needs to land somewhere. And you eventually need to reclaim the space in A. The temptation is to erase A right away — but if you erase A, you lose your only good copy while writing the new one to B. Any power cut here is catastrophic.
The correct sequence:
Step 1 — before: Step 2 — write new record into B[0]
┌───────────────────────┐ ┌───────────────────────┐
│ A: slot0..20 (seq=21) │ ACTIVE │ B: slot0 (seq=22) NEW │ (freshly committed)
│ B: all 0xFF │ IDLE │ A: slot0..20 (seq=21) │ (still present)
└───────────────────────┘ └───────────────────────┘
Step 3 — erase A Step 4 — steady state
┌───────────────────────┐ ┌───────────────────────┐
│ A: all 0xFF (erased) │ IDLE │ B is now ACTIVE │
│ B: slot0 (seq=22) │ ACTIVE │ A is IDLE, awaits next│
└───────────────────────┘ └───────────────────────┘
rollover
Every step is crash-safe. Let’s walk through each failure point:
Cut between step 1 and 2 (before B[0] is fully written): A has the latest record. B is empty or partial. Boot picks A.
Cut between step 2 and 3 (B[0] fully written, A not yet erased): Both sectors have valid records now. Boot picks the higher seq which is in B. On the next update we notice A still has stale records and can proceed to erase it (or just erase it eagerly on boot).
Cut mid-erase (step 3): A is partially-erased garbage. Its records will fail their CRC or magic-word check. B[0] is intact. Boot picks B[0].
At no point in the sequence is there zero valid records anywhere on the chip. That’s the invariant. The commit is the magic-word write in step 2, and everything after that is idempotent cleanup — if it gets interrupted, the next boot can safely retry the same cleanup.
Wear leveling — a lovely side effect
Flash cells wear out. Each sector has an endurance rating — typically 100,000 program/erase cycles for consumer QSPI flash. If you erase a sector 30 times per second, you’ll burn it out in a day. So write-heavy applications need to spread writes across many sectors to extend life.
In the ping-pong scheme, we only erase a sector when it fills up — which happens every 21 commits (in our example). Between those, all we do is program new slots, which doesn’t touch the erase counter at all. And the two sectors take turns getting erased, so wear is naturally balanced across them.
The endurance math:
21 commits/erase × 100,000 erases × 2 sectors ≈ 4,200,000 commits
Assuming a very generous 100 commits per day (which is a lot for any config workload), you’d need over a hundred years to hit the endurance limit. On any realistic device this is a non-concern.
What if you need more space?
A single record in our example is 192 bytes. Some applications need more — cryptographic key material, cached tables, logs. Two options:
Bigger records — each still fits within one sector, but with fewer slots per sector. A 512-byte record leaves 8 slots per 4 KB sector. Endurance and crash-safety semantics are unchanged.
Wider deployment — use more than two sectors. Instead of A/B, use A/B/C/D. Same rotation, same semantics, more headroom.
The invariants — magic-word commit fence, monotonic seq counter, boot scanner picks the winner, at-least-one-valid-record-always — remain identical. The pattern scales.
When this pattern generalizes
Ping-pong storage is a specific instance of a much more general technique that shows up everywhere in systems engineering:
Filesystems: ext4’s journaling, ZFS’s copy-on-write, and NTFS’s transactional overwrites all use “write to new location, atomically update a pointer” to survive crashes.
Databases: PostgreSQL’s MVCC and SQLite’s WAL mode use the same pattern — new versions get new slots, commit is a small atomic operation that changes which version is current.
Distributed systems: Two-phase commit uses a “prepare then commit” fence exactly like our magic-word write.
Bootloaders: A/B partition schemes for OTA updates ship a new image to slot B, and only change the “which slot to boot” pointer when the new image is fully written and verified.
Notice the shared shape:
Never destroy the old thing before the new thing is completely in place.
Have one small atomic operation that flips “the old thing is current” to “the new thing is current.”
Anything interrupted before that atomic flip is silently discarded on recovery.
Anything interrupted after is followed by idempotent cleanup on next boot.
If you can identify a “commit fence” in your system — one small write that’s the last thing to happen and that gates whether the change becomes visible — you can build almost any kind of durable update on top of it. The magic word in flash is just one implementation. A directory rename on a filesystem is another. A single-row commit in a database is a third.
When you should NOT reach for this
This pattern isn’t free — it costs disk space (two sectors instead of one), adds boot-scan latency (small, but real), and complicates your storage code. If you have any of these situations, simpler approaches may serve better:
You don’t care if the data is lost. Cached data, telemetry logs you’re OK dropping — write in place, don’t bother.
You have a filesystem underneath. If your platform already gives you a real filesystem with atomic rename, use that. Don’t reimplement crash safety at a lower layer.
You’re writing enormous records rarely. If you’re only saving 100 KB once a week, the endurance-driven wear-leveling motivation disappears; a single-sector “erase-then-write with a backup copy elsewhere” scheme might be enough.
Your device has battery-backed RAM. If you can survive power cuts by staying alive long enough to finish writes, you don’t need this level of protection at the storage layer.
But for the common embedded case — a device with no filesystem, no RTOS storage layer, no battery backup, holding a few KB of state that MUST survive arbitrary power loss — ping-pong is the classic answer, and it’s remarkable how many people reinvent it (badly) without realizing there’s a well-understood pattern already.
Takeaways
Five principles that generalize beyond flash storage:
1. Understand your medium’s failure modes before designing on top of it. Flash’s one-way writes and coarse erase aren’t inconvenient details — they’re the design constraints that dictate what patterns are viable.
2. Never destroy the old before the new is complete. This is the copy-on-write principle. It costs space but buys correctness.
3. Find or engineer a commit fence. One small operation that’s atomic at the hardware level, and that decides whether the change becomes visible.
4. Design so that any interrupted state resolves cleanly on recovery. No half-committed states that survive to be read as though they’re valid.
5. Verify the invariant holds at every failure point. Walk through every place a power cut could hit your sequence and prove that at least one valid state remains. If any point fails, redesign.
Ping-pong storage is a small pattern. But the discipline it embodies — being paranoid about the medium’s failure modes, deferring destructive operations, making commits atomic even when the underlying hardware isn’t — is the foundation of every reliable storage system I’ve ever seen, from a $2 microcontroller with an external flash chip to a distributed database serving billions of requests a second.
Learn the small pattern well, and you’ll recognize the shape of the bigger ones wherever you find them.
If you build embedded systems and hit similar challenges around durable state, crash safety, or unusual storage media, I'd love to hear how you solved them.
Hardware/PCB design & addon services
