Encrypting a Small LoRa Network From Scratch and the Races We Hit Along the Way

A retrospective on the cryptographic design of a small wireless system: one battery-powered field device, one mains-powered hub, a backend behind MQTT. The crypto is the easy part. The state machine around it is where the learning is.

Table of Contents

The setup
What we pick, and why we keep it small
A from-scratch primer on the three primitives
What is AES, really?
The problem: messages are not 16 bytes long
Counter mode: turn AES into a stream cipher
CCM: counter mode plus integrity
X25519: agreeing on a secret without ever sending one
The Diffie-Hellman idea
Why X25519 specifically
Why a hardware RNG matters more than you’d think
Three layers of state
The nonce, in detail
What goes into the AAD
Pairing
The races we hit
The boring decisions that matter
What we deliberately did not do
What we would do differently if we shipped this again

The setup

The system is small enough to draw on the back of a napkin:

Figure 1. Three components, two channels. The LoRa link is the part this article is about.

A target is a sensor running on a primary cell, deployed somewhere inconvenient. It talks to a hub over a LoRa link in a Sub-GHz ISM band. The hub aggregates events from a small number of paired targets and forwards them to a backend over Wi-Fi and MQTT. Each hub owns up to N targets that have been explicitly paired to it during installation.

The LoRa link itself is broadcast. Anybody with a software-defined radio costing less than a meal can demodulate the chirps. That fixes the security shape of the problem before any code is written. The wireless link has to be:

Confidential. Observers cannot read what targets send. The payload bytes are application-sensitive — battery, sensor counts, health, alerts — and we do not want them in cleartext on the air.
Authenticated. The hub must be sure a packet came from a specific paired target, not from a stranger holding the same modulation parameters.
Replay-resistant. A captured packet cannot be re-transmitted later and accepted. Including across target reboots, because embedded devices reboot.
Pair-bounded. Only targets that have completed an explicit, physically-observable pairing can transmit operational data. No silent self-enrolment.
Operator-bounded. Sensitive operational state — what radio network we’re on, factory reset, key rotation — cannot be triggered remotely through the cloud control plane.

The constraints are equally fixed. LoRa payloads are tiny (a handful of bytes after framing). The MCU is small. Cryptographic operations must be cheap, predictable, and survive frequent reboots and reflashes. We don’t get to assume a long-running TLS session; we don’t get to assume an internet connection at the target; we don’t get to assume the device has been gracefully shut down before its last operation.

What follows is the design as it shipped, with the corners we hit along the way. The intended reader is an embedded developer who has some security background — comfortable with “AES-CCM is an AEAD,” but who wants to see what an actual deployment looks like and where the non-obvious failure modes are.

What we pick, and why we keep it small

We deliberately picked a tiny cryptographic palette:

X25519 for the long-term identity keypair on each device, and for the initial key agreement during pairing.
AES-128 in CCM mode for every operational LoRa packet — authenticated encryption with associated data, in one primitive.
A hardware RNG on the MCU for X25519 keypair generation, session IDs, and nonce-domain bootstrapping.

That’s it. No additional hashes, no custom MAC constructions, no in-house block ciphers. Anything that looks like “we’ll write a small helper because the library is too heavy” is the start of a vulnerability.

The justifications are worth spelling out, because each is a defence against a common LoRa-project failure mode.

AES-CCM rather than GCM

CCM is the NIST AEAD that small MCUs already accelerate in hardware, doesn’t require a GF(2¹²⁸) multiplier, and tolerates short tags well. Every byte of LoRa payload is precious; a CCM construction with an 8-byte tag costs us 8 bytes, which is acceptable. The AAD slot lets us bind the cleartext header (addresses, packet ID, PAN ID) into the tag without paying to encrypt it.

X25519 rather than classical DH or RSA

A Curve25519 public key fits in 32 bytes — broadcastable in a single LoRa frame. Key generation is fast enough to do on first boot. There are no parameter choices to mess up; you can’t pick a bad curve, you can’t fumble the generator, you can’t accidentally use a 512-bit modulus. mbedtls’s implementation rejects the all-zero shared secret per RFC 7748 §6.1, which is the one validation step you must not skip.

No bespoke primitives

No custom hashes, no hand-rolled CBC-MAC, no “we’ll just XOR the packet ID into the key, it’s good enough.” This is the single most common cryptographic mistake in IoT firmware. The work in this article is entirely about how to compose well-understood primitives correctly under embedded constraints.

The session key is a 128-bit AES key, derived once at pairing time from the X25519 shared secret via HKDF-SHA-256. The HKDF info field is a canonical string built from the PAN ID and both endpoints’ long addresses. That binding means the same shared secret cannot be reused across a different PAN or with the addresses swapped — useful when you start thinking about hub re-deployments and address-recycling edge cases.

A from-scratch primer on the three primitives

If you already know what AES-CCM is and roughly how Diffie-Hellman works, skip to the next section. The rest of this article will make sense without rereading anything.

If those names are unfamiliar, this section is for you. Cryptography suffers from a vocabulary problem: there are a handful of ideas that take ten minutes to grasp and a thousand pages of formal treatment to prove rigorously. We are going to spend the ten minutes here. The goal is not mathematical rigour; the goal is that by the end of this section you should understand what each primitive does, what shape of input and output it has, and what it would mean for it to fail.

What is AES, really?

AES stands for the Advanced Encryption Standard. The “advanced” is mostly historical — it was the winner of a competition in the late 1990s to replace an older standard called DES. Today it is the single most widely deployed symmetric cipher in the world; every TLS session, every encrypted Wi-Fi frame, every iPhone disk encryption key is going through some flavour of AES.

Strip away the standardisation paperwork and AES is shockingly simple to describe: it is a function that takes exactly 16 bytes of input plus a secret key, and produces exactly 16 bytes of output. That’s it.

Figure A. AES is a function from 16 bytes to 16 bytes, parameterised by a secret key. Everything else is a recipe for stretching this primitive into something useful for messages of arbitrary length.

The mapping has two important properties:

It is a bijection. For a fixed key, every 16-byte input maps to exactly one 16-byte output, and every 16-byte output came from exactly one 16-byte input. There is no information loss; the function is perfectly reversible if you have the key.
Without the key, the mapping looks random. Given a million (input, output) pairs, the best known attack still cannot recover the key faster than trying ≈ 2¹²⁸ candidates. That number is bigger than the number of atoms in a million Earths. Nobody is doing it.

The “128” in AES-128 refers to the key length: 128 bits, which is 16 bytes. There are also AES-192 and AES-256 variants with larger keys, but for our purposes 128 bits is the sweet spot — enough to be unbreakable in any realistic threat model, small enough to fit comfortably in a LoRa payload.

Internally AES is a sequence of ten very fast, very simple operations — byte substitutions, row shifts, column mixes, key XORs — applied to a 4×4 grid of bytes. The details do not matter for this article. What matters is the shape: 16 bytes in, 16 bytes out, key makes it invertible.

Deeper dive: If you want to know exactly what AES is doing inside that black box — every operation traced through real bytes, with nothing more than arithmetic as a prerequisite — see the companion article AES, Explained from Scratch. The rest of this post does not depend on it.

The problem: messages are not 16 bytes long

A LoRa packet might be 30 bytes. A health report might be 8. AES on its own only knows how to encrypt exactly 16 bytes at a time. So you need a mode of operation — a recipe for how to apply AES to messages of arbitrary length.

The naive approach is to split the message into 16-byte chunks and encrypt each one separately. This is called ECB (Electronic Code Book). It has a famous and fatal problem: if two chunks of plaintext happen to be identical, their ciphertexts are also identical. An attacker who sees a long ciphertext can read off the structure of the underlying plaintext just from the pattern of repeats. The canonical demonstration is what happens if you encrypt a bitmap of the Linux penguin under ECB — the penguin is still clearly visible in the ciphertext, because the large regions of the same colour become large regions of the same ciphertext.

ECB is the answer to the question “what is the simplest thing that could possibly work” and the answer is “this.” Don’t use it for anything but very specific cryptographic building blocks.

Counter mode: turn AES into a stream cipher

A much better idea is CTR (counter) mode. The trick: instead of encrypting the plaintext directly, you encrypt a sequence of counter values (0, 1, 2, 3, …) and XOR the resulting keystream into the plaintext.

Figure B. AES-CTR turns the block cipher into a stream cipher. Encrypt successive counter values, XOR the resulting keystream into the plaintext. Decryption is the same operation — XOR is symmetric. Critical: never reuse a counter under the same key.

Three things to notice. First, decryption is identical to encryption — XOR is symmetric, so the receiver generates the same keystream from the same counter and XORs it into the ciphertext to recover the plaintext. Second, you can encrypt messages of any length, because you just generate as many keystream blocks as you need. Third, identical plaintext blocks no longer produce identical ciphertext blocks, because they get XORed against different keystream blocks.

But CTR mode has a knife-edge requirement: the counter must never repeat for the same key. If it does, the two ciphertexts are the XOR of the two plaintexts plus the same keystream — and the keystream cancels out:

   C₁ = P₁ ⊕ KS
   C₂ = P₂ ⊕ KS
   C₁ ⊕ C₂ = P₁ ⊕ P₂          ← attacker can compute this

If the attacker knows or can guess any part of one plaintext, they recover the corresponding part of the other plaintext for free. This is why the nonce — the starting value of the counter — has to be unique per encryption. That is the entire reason §4 of this article exists.

CCM: counter mode plus integrity

CTR mode gives you confidentiality but not integrity. An attacker who knows where a particular byte sits in the plaintext can flip the corresponding bit in the ciphertext, and the receiver, applying the same XOR on decrypt, gets back a plaintext with that bit flipped. No alarm goes off; the message just silently turns into a different message. For a control system this is a catastrophe.

The fix is to add an authentication tag: a short fixed-size value the sender computes over the message (and any associated context), which the receiver re-computes and checks. If the tag does not match, the receiver throws the message away.

AES-CCM (Counter with CBC-MAC) is the standard recipe for doing both jobs at once with one block cipher. The construction has two halves:

The CTR half encrypts the plaintext, exactly as described above.
The CBC-MAC half computes an authentication tag by chaining AES encryptions over the associated data, the plaintext, and a formatted length field. The chaining means any single bit difference anywhere in the inputs avalanches into a completely different tag.

The output is ciphertext ‖ tag. The receiver runs both halves on the way back: decrypt with CTR, re-compute the tag and compare. A single mismatched bit makes the comparison fail. The whole thing is called an AEAD — Authenticated Encryption with Associated Data — because in one primitive you get confidentiality of the plaintext, integrity of both the plaintext and any extra “associated data” you fed in alongside, and authenticity (only someone with the key could have produced a valid tag).

This is the modern default for any new symmetric protocol. The two other big AEADs in the wild are AES-GCM (faster on hardware with a polynomial multiplier) and ChaCha20-Poly1305 (faster in software without an AES accelerator). We chose CCM because the small MCUs in this design already accelerate AES in hardware and CCM needs nothing beyond AES itself.

X25519: agreeing on a secret without ever sending one

AES is symmetric cryptography — both sides need to know the same key. That raises an obvious question: how do they agree on the key in the first place, without an attacker who is listening to every exchange learning it too?

This is the problem that public-key cryptography solves, and the specific scheme we use is X25519 — the Diffie-Hellman key agreement protocol carried out on a particular elliptic curve called Curve25519.

Let’s build it up.

The Diffie-Hellman idea

Imagine you and a friend want to agree on a colour, but you can only communicate by shouting across a noisy room. Anyone in the room can hear everything you say.

The trick: you both start with the same publicly-known base colour — say, yellow. Each of you secretly chooses your own colour to mix in (you pick red, your friend picks blue). You each mix your secret colour with the public yellow, get a new colour (you get orange, your friend gets green), and shout the result to each other.

Now you both perform a second mix. You take your friend’s green and add your own secret red. Your friend takes your orange and adds their own secret blue. Both of you end up with the same brown — the combination of yellow + red + blue.

Figure C. The Diffie-Hellman paint analogy. Both sides arrive at the same secret colour. The eavesdropper hears yellow, orange, and green — but cannot work back to red or blue.

The eavesdropper heard yellow, orange, and green — but separating orange back into yellow and red is hard if you don’t know which red was used. (Real paint is a lousy analogy here because paint mixing is easy to reverse; the cryptographic equivalent uses operations that are easy to do but practically impossible to undo.)

The cryptographic version replaces colours with numbers and “mixing” with a mathematical operation that is easy in one direction and hard in the other. In classical Diffie-Hellman the operation is modular exponentiation; in elliptic-curve Diffie-Hellman (ECDH) it is scalar multiplication of a point on an elliptic curve. Don’t worry about what that means geometrically — the only fact you need is that the operation has the same property as the paint mix: combining your secret with a public starting point is easy; recovering your secret from the result is computationally infeasible.

Why X25519 specifically

Plain elliptic-curve cryptography has historically been a minefield of footguns. You have to choose a curve, choose a base point, check that the curve is not weak, check that the implementation doesn’t leak the private key through timing differences. Every one of these choices has had at least one famous standard ruined by a bad decision.

X25519 is the result of one specific design philosophy: build a curve where the bad decisions are impossible. Curve25519 was proposed by Dan Bernstein in 2005 with the explicit goal of eliminating choices. There is one curve. There is one base point. The implementation is uniform — every operation runs in the same amount of time regardless of the inputs, so side-channel attacks through timing don’t get a foothold. The keys are 32 bytes (256 bits) and the practical security is around 128 bits — comparable to AES-128.

In practice, X25519 from your point of view as the implementer is:

Generate a keypair: ask the library for a private key (32 random bytes, clamped according to RFC 7748) and the corresponding public key (32 bytes you can broadcast). Costs roughly one millisecond on a modern small MCU.
Compute the shared secret: feed your private key and your peer’s public key into one library call. Out comes a 32-byte value. Your peer feeds their private key and your public key into the same operation and gets the same 32-byte value. Costs another millisecond or so.

That 32-byte shared value is your raw input keying material. You don’t use it directly as an AES key for one thing it’s 32 bytes and AES-128 wants 16, for another it’s the same value every time the same two devices pair, which limits domain separation. So you run it through a key derivation function like HKDF-SHA-256 with a label that includes the PAN ID and both long addresses. Out comes a 16-byte AES key bound to those specific endpoints in that specific network.

Figure D. From an X25519 exchange to a usable AES-128 session key. The hub does the mirror operation with its private key and the target's public key, and arrives at the exact same 16 bytes.

The hub, meanwhile, has done the same operation with its own private key and the target’s public key, and computed the same 32-byte shared value, and run it through the same HKDF with the same inputs, and arrived at the same AES key.

One detail worth knowing: there is exactly one validation step you must not skip. RFC 7748 §6.1 specifies that if the shared secret comes out as all zeros, you must reject it — that is a sign of either a buggy implementation or a malicious peer sending a specially-crafted public key. Every serious library does this check for you; if you ever roll your own, this is the one line of code that absolutely must be there.

Why a hardware RNG matters more than you'd think

The whole edifice — the secrecy of the X25519 private key, the uniqueness of every AES-CCM nonce, the unpredictability of session IDs — rests on one quietly demanding assumption: you can produce random numbers that an attacker cannot guess.

This is harder than it looks. Computers are by design deterministic: given the same inputs they produce the same outputs. So where does randomness come from?

Two kinds of "random"

Software random number generators are pseudo-random: they take a small seed and stretch it deterministically into a long sequence of output that looks random. If you know the seed, you know the entire sequence. C’s rand(), Python’s random module, and most language-standard generators fall in this category. They are fine for shuffling cards in a game; they are not fine for cryptography.

The classic embedded failure mode is seeding a software PRNG from the boot clock or the uptime counter. Both are easily predictable to anyone who can guess when the device powered on. There have been real shipping IoT products that generated “cryptographic” keys this way and were trivially broken.

What we need is true randomness — bits whose value an attacker cannot predict no matter how much they know about the device, the firmware, the timing, or the history.

Where physical randomness comes from

Physics provides several reliable sources of unpredictability at the chip level:

Thermal noise across a resistor or a junction.
Jitter in the relative phases of free-running oscillators.
Metastability in latches that are deliberately driven into an undefined region.

Modern MCUs include a small dedicated peripheral — the hardware RNG, also called a TRNG (True Random Number Generator) — that samples one of these physical phenomena, runs a small amount of whitening over it to flatten any residual bias, and presents the result as a stream of bytes you can read out of a register.

These are not free — they take many microseconds to produce a single 32-bit word, because they wait for enough physical entropy to accumulate. They are also not infinitely fast under load. So practical embedded cryptographic stacks layer them:

Figure E. The RNG layered design. Physical entropy from the hardware seeds a cryptographically secure pseudo-random generator, which feeds every cryptographic consumer in the system.

The CSPRNG — cryptographically secure pseudo-random number generator — is itself deterministic, but it is seeded with real entropy from the hardware. Once seeded, it can produce gigabytes of output that looks indistinguishable from true random, far faster than the hardware source alone could manage. Common embedded choices are CTR_DRBG (built on AES) and HMAC_DRBG (built on a hash).

You typically also feed in a personalisation string when seeding the CSPRNG — something device-specific, like the chip’s factory-fused unique ID. Its job is to make sure two devices that happen to be in identical states at the same wall-clock instant still produce different output streams. Belt-and-braces, but cheap.

The practical pattern in our code is therefore:

At first use, call mbedtls_ctr_drbg_seed() with the hardware RNG as the entropy source and a short personalisation string.
From then on, every “give me random bytes” call goes through mbedtls_ctr_drbg_random(), which is fast and produces cryptographically strong output.
Sensitive intermediates — private keys, shared secrets — are zeroed from memory after use so they cannot be lifted from RAM by a later compromise.

If any of those steps is wrong — the hardware RNG was never enabled, the seeding was done with a constant string, the CSPRNG was reseeded with a guessable value, the keys were left in memory after use — every other security property in this article quietly collapses. This is why the unglamorous plumbing in the RNG layer gets as much care as the cipher itself.

Three layers of state

This is where the design actually lives. Cryptography on an embedded device is not one big “secure context” — it’s three independent layers of state, each with its own lifetime, its own loss-detection story, and its own commit-and-rollback policy.

Figure 2. Three layers of state. Each layer has its own loss-detection story and its own commit policy.

Identity

On first boot, each device generates an X25519 keypair using the MCU’s hardware RNG (via CTR_DRBG seeded from the entropy peripheral). The private key is stored in a non-volatile area that survives firmware updates; the public key is broadcast during pairing.

Each device also has a long address — a 64-bit unique identifier derived from the MCU’s factory-fused chip ID — and a short address, a 16-bit identifier derived from the long. The long is canonical; the short is a courtesy shortcut so LoRa headers can stay small. Shorts can collide across devices; the protocol handles that in the obvious way (long address is the tiebreaker during pairing).

The identity layer is stable for the device’s lifetime. Reflashing firmware does not regenerate the keypair. The only operation that rotates identity is an explicit factory reset, which we will return to in a moment.

Pair

When a hub and a target successfully complete pairing, both sides derive a per-pair 128-bit AES key from their X25519 shared secret, and both sides persist:

the peer’s long and short addresses
the derived AES key
the PAN ID they agreed on

The hub additionally persists a “paired DB” — the set of all targets it has paired with. The target persists a single record for its hub.

The AES key never leaves either device. It is not transmitted, not echoed to a serial console, not written to a log. This sounds obvious; it is worth checking in a real codebase, because several shipping LoRa products print pairing keys to a debug UART at provisioning time and then forget to disable that path.

Session

A session is a short-lived nonce domain. On boot each device:

Reads a 32-bit value from the HW RNG.
Mixes it through xorshift32 to whiten any peripheral bias.
Takes the low 16 bits as mySid. Value zero is reserved as “not yet known”; if the random draw gives zero, retry.

mySid is the device’s view of “what session am I in.” Its peer’s session — peerSid — is whatever the peer most recently announced.

These are needed to construct the nonce. They have to be rolled freshly per boot, because if a device crashes mid-burst and replays old packets after reboot, the nonce must not repeat. That is the whole point: a session ID is a domain separator that turns “this is my 42nd packet of all time” into “this is my 42nd packet of this boot’s session.” The receiver tracks one counter per session, not one counter forever.

The asymmetry is the part that catches people. It is not “session = a pair of random numbers agreed once during pairing.” It is “each side independently rolls a fresh value at boot, and they tell each other about it on the first message after they come up.” If only one side has rebooted, the other already knows its own session, so the returning side just announces its new value and we continue.

If both sides reboot at once — say a power outage — neither knows the other’s session. The protocol’s first encrypted message they exchange is a session resync: each side announces its own mySid and echoes what it thinks the other’s mySid is. After one round trip both sides know both values. No out-of-band coordination needed, no operator standing in front of the device with a pairing button.

This design is the thing that makes the system robust to power loss in ways that LoRaWAN’s frame-counter discipline is not. LoRaWAN devices that hard-code session keys and reset their counter to zero on reboot are a famous footgun. Here, every reboot lands us in a fresh nonce space by construction.

The nonce, in detail

The nonce is the heart of the cryptosystem and deserves a careful walk-through. AES-CCM requires a nonce that is never reused with the same key. Reuse breaks confidentiality entirely; for any pair of frames encrypted under the same (key, nonce) the XOR of the ciphertexts is the XOR of the plaintexts, which is usually game over.

We construct a 12-byte CCM nonce out of fields that are guaranteed by construction never to repeat:

Figure 3. The 12-byte AES-CCM nonce. Every field is doing real work: identity, sender-side freshness, receiver-side freshness, and a monotone counter inside a session.

Each field is doing real work:

srcShort — domain separation between targets. Two targets in the same PAN that happen to draw the same mySid value cannot collide in nonce space because their short addresses differ.
mySid — domain separation between reboots of the same target. Reboot → fresh mySid → fresh nonce space → all old packets become invalid replays because their nonces fall in the wrong half.
peerSid — binds each packet to a specific receiver session. A captured packet replayed at the hub after the hub has rebooted will have an obsolete peerSid and fail the MIC check.
packetId — monotone counter within a session. Each (src, mySid, peerSid) tuple has its own packetId space starting at 0.

The math is worth doing aloud. With the HW RNG providing a uniform 16-bit mySid per boot, the probability that two consecutive boots of the same device happen to pick the same session is 1 in 65 536. Combined with the receiver also being in a fresh session — another 1 in 65 536 of accidentally matching — the chance that a captured packet’s full nonce repeats is small enough to ignore inside our threat model. And even if it did, the MIC check would catch it.

What goes into the AAD

The cleartext LoRa header — addresses, packet ID, PAN ID — is fed in as associated data rather than encrypted. Encrypting the header would be pointless (the receiver needs to read it to route the packet) but it absolutely needs to be authenticated. Binding it into the AAD means an attacker who flips a destination address byte in a captured packet invalidates the MIC; the receiver drops the frame before AES-decrypt even runs.

The PAN ID is in the AAD. A target paired into PAN A literally cannot talk to a hub on PAN B even if the AES key somehow happened to be the same — the AAD mismatch kills the MIC. PAN is a coarse domain separator, useful when you start running multiple networks side-by-side and want the protocol layer itself to enforce isolation.

Replay protection

Each side maintains a per-peer last-seen packetId. Packets with packetId <= last_seen are dropped before AES even gets called. This catches replays inside an active session, cheaply. Cross-session replays — captured-yesterday, replayed-today — die at MIC verify because the nonce contains stale mySid or peerSid values.

Here is the generic encrypt-side flow, condensed:

/* Build the 12-byte CCM nonce. */
nonce[0] = srcShort & 0xFF;  nonce[1] = srcShort >> 8;
nonce[2] = mySid    & 0xFF;  nonce[3] = mySid    >> 8;
nonce[4] = peerSid  & 0xFF;  nonce[5] = peerSid  >> 8;
nonce[6]  = (packetId >>  0) & 0xFF;
nonce[7]  = (packetId >>  8) & 0xFF;
nonce[8]  = (packetId >> 16) & 0xFF;
nonce[9]  = (packetId >> 24) & 0xFF;
nonce[10] = (packetId >> 32) & 0xFF;
nonce[11] = (packetId >> 40) & 0xFF;

/* The cleartext header is the AAD; payload is encrypted in place. */
rc = aes_ccm_encrypt_and_tag(
        key,
        nonce,    NONCE_LEN,
        header,   HEADER_LEN,        /* AAD */
        payload, ciphertext, len,    /* PT in, CT out, same buffer */
        tag,      TAG_LEN);

The receive side mirrors this. Both sides drive the same primitive through the same library; there is one place in the codebase where the nonce is built, and there is no other.

Pairing

Pairing is where you give two devices the shared key they will use forever. Most pairing bugs are not crypto bugs — they are state machine bugs that put one side into “we are paired” and the other into “we are not.” We will get to those.

The high-level walk:

Figure 4. Pairing sequence. Plaintext frames carry no secrets; encrypted frames use the freshly derived per-pair AES key. Durable commits are deferred until the on-air TX is confirmed.

A few decisions in this flow deserve explicit defence.

The pairing window is operator-gated. The hub does not accept any DISCOVERY-ACK at any time. The operator presses a button, or the backend issues a “begin pairing” command, and the hub opens a seconds-long window. Outside that window, even otherwise-valid candidates are rejected. This is what makes the system “pair-bounded” in the threat-model sense: drive-by pairing is impossible.

DISCOVERY-ACK is plaintext, on purpose. It contains no secret material. Its only job is to say “yes, I’m here, send me the next step.” Requiring a pre-shared key to acknowledge discovery would defeat the point of having a pairing flow. The next message, PAIRING, is encrypted with the freshly derived per-pair key; a stranger who saw the plaintext DISCOVERY-ACK still cannot forge a PAIRING reply or read its contents.

The AES key is derived from the X25519 shared secret with HKDF-SHA-256. The HKDF info string is built from the PAN ID and both endpoints’ long addresses, in canonical order. That binding prevents an attacker who captures a pairing exchange from replaying it on a different PAN, and it prevents the rare case of two pair attempts producing colliding keys when only one of the long addresses differs.

Durable state changes are deferred until TX-acknowledgement. This is the rule that catches half the pairing race conditions. The hub does not commit its paired_db row until the PAIRING-ACK frame has been confirmed on the air. The target does not commit its paired record to NVS until the CONNECTION frame it sends out has been acknowledged. If a TX fails after retries, RAM state rolls back cleanly. The AES key lives in RAM only until the durable commit boundary.

That last rule sounds straightforward when written down. It is not straightforward to implement. The story is in the next section.

The races we hit

Documenting failure modes is more interesting than documenting successes. Here are three race windows we shipped and then fixed.

Race 1 — split-brain pairing: the hub committed but the target didn't

What we shipped first. When the hub decided to accept a pairing attempt, it queued the PAIRING-ACK frame, immediately queued the follow-up CONNECTION frame, and committed the paired_db entry. The optimistic ordering get both frames in the radio queue while we have the keys hot was meant to minimise pairing latency.

What went wrong. The PAIRING-ACK transmission sometimes failed after retries (the target had moved, or the link was momentarily congested). The TX-fail callback rolled back the in-RAM paired_db entry. But the CONNECTION frame was already in the queue, or in the air, or even already failed on its own. The hub then fired a connection_failed event for a pairing relationship that, as far as either side was concerned, did not exist. The backend dashboard lit up with a confusing trail of events; field engineers thought there was a bug in the radio stack.

The fix. Reorder the commit semantics:

WRONG:

queue PAIRING-ACK
queue CONNECTION
commit paired_db

on TX-fail of either:
  roll back, but
  second one is in flight

RIGHT:

queue PAIRING-ACK
wait for PAIRING-ACK TX-confirmed
  ↓ on confirmed:
    queue CONNECTION
    commit paired_db
    on TX-fail of CONNECTION:
      FSM-level retry, bounded
  ↓ on TX-fail of PAIRING-ACK:
    roll back; no CONNECTION ever
    on the wire

The new rule is: CONNECTION is queued after PAIRING-ACK has left the air, not after PAIRING-ACK has been queued. The TX-confirmed callback for PAIRING-ACK is what triggers the CONNECTION queue push and the durable commit; the TX-fail callback for PAIRING-ACK rolls back cleanly because CONNECTION has not been touched yet.

The lesson is general: when an embedded protocol commits state at multiple stages, the commit order has to follow the wire order, not the convenience of having all the data in scope at once.

Race 2 — the millisecond between "we sent the reply" and "we promoted the session"

This one is subtle and worth the page.

The setup. On the target side, the freshly derived per-pair AES key lives in a “staged” session slot during the brief interval between receiving PAIRING and being sure we have replied to it. It gets promoted to the “live” session table only when our reply (the CONNECTION frame) is TX-confirmed.

The hub, meanwhile, sends its first encrypted operational frame as soon as it sees our CONNECTION TX. There is a window — roughly one millisecond on this stack — between the radio reporting “your reply left the air” and our own software actually getting to the “promote staged session to live” call.

If the hub’s first encrypted frame arrives in that millisecond, our RX dispatcher looks up the session by short address, finds nothing in the live table, and drops the frame. The hub retries, the target finally promotes, retry succeeds, no permanent damage — but every single pairing produced one or two spurious MIC failures in the logs. Some installations correlated the spurious failures with “pairing is unreliable” and we got tickets.

The fix. Add a defensive fallback in the RX dispatcher: if the live session lookup fails, peek at the staged-session slot. If the staged key successfully verifies the MIC, promote the staged slot to live as a side-effect and continue with the decrypted frame. Now the millisecond window is closed at both ends — the on-air queue’s TX-ack callback still drives the canonical promotion path, but an opportunistic RX-side promotion is also legal.

The deeper lesson is that “the durable commit happens at TX-confirmed” is a clean rule on the sending side, but the receiving side cannot assume the same timing. If you have a window where state is correct in one place and stale in another, document the fallback path explicitly.

Race 3 — the target's NVS got wiped and nobody told the hub

The setup. Targets persist their paired record to non-volatile storage. Occasionally that storage is corrupted — flash wear-out on a field unit, a botched firmware update that erased the wrong sector, a development board where the NVS partition is a RAM-only stub. The target then boots into ORPHAN state, sends a plaintext BLINK, and waits to be re-paired.

The hub, meanwhile, still has the target in its paired DB. From its point of view, the target is just being weirdly quiet, then suddenly sending plaintext frames that fail every encrypted-frame check.

What we wanted. A way for the target to tell the hub “I have lost my state, you should realign your expectations of me,” without needing to re-pair from scratch every time.

The fix. Add a small persistent boot epoch value to the target’s identity layer. Every operational packet carries the target’s current epoch in its (authenticated) header. When NVS is healthy, the epoch is loaded from NVS on boot and is stable. When NVS is missing, the epoch is seeded fresh from the HW RNG so it is guaranteed to differ from anything the hub has seen before.

The hub treats a changed epoch as “this target’s persistent state has been reset; tear down the pairing and run the normal orphan-detection path.” On the development board, where NVS is a RAM-only stub, the epoch changes every reboot and the realignment path fires every boot, which is exactly the same protocol path as the production NVS-corruption case, just exercised more often. That kind of accidental test coverage is its own reward.

The general pattern is worth naming: if a peer can lose persistent state in a way the other peer cannot detect, give it a small explicit “freshness” field and make it part of the authenticated header. Without this, you end up trying to debug ghost pairings by power-cycling things, which is not a strategy.

The boring decisions that matter

These are the policy decisions outside the cryptography itself — boring on the surface, load-bearing in practice.

The AES key never leaves NVS. Not over MQTT, not over UART, not in a debug log, not even in a diagnostic dump. Some pairing-key exfiltration is unavoidable if the device is physically captured and the MCU is opened, but everything short of that is closed. This rules out an entire family of “we just need to grab the key for support purposes” features.

Channel-by-operation policy table. Three control planes — LoRa, MQTT, and the physical UART used at install and service time — are not interchangeable.

Operation	LoRa	MQTT	UART
Pair / unpair targets	yes	yes	yes
View paired DB	n/a	yes	yes
View integrity events	n/a	yes	yes
Set PAN ID	no	no	yes only
Factory reset the hub	no	no	yes only
Rotate identity keypair	no	no	yes only

The two emphasised nos — PAN change and factory reset over MQTT — are deliberate. These are the operations whose abuse compromises the device’s whole identity. They require physical presence on the box. A compromised backend, a stolen MQTT credential, or a malicious backend administrator cannot move the hub to a different network or rotate its keys. They can issue normal pair/unpair commands; they cannot cause an identity-level event.

The cost of this policy is zero — these are not operations you perform remotely in normal life. The benefit is that an entire class of cloud-side attacker capability is structurally absent. It is, in the literal sense, a defence-in-depth choice that costs nothing.

Pairing requires an explicit, time-bounded window. Not a flag set once at install and forgotten. Not a “first-come-first-served” acceptance of any BLINK ever heard. The operator opens a window of a few seconds; the hub accepts pairings during that window only. A flood of fake DISCOVERY-ACKs during a window can occupy the slot, but the backend sees the unauthorised-pairing event surface in real time and an operator can react.

Crash-during-pairing is engineered for, not hoped against. The whole “durable state changes deferred until TX-acknowledgement” discipline is what makes this work. The session and AES key are in RAM only until the commit boundary; if the radio TX times out after retries, RAM rolls back and the device returns to its prior durable state without ambiguity.

What we deliberately did not do

A short, honest list. Pretending the design is universally bulletproof would be the opposite of credible.

No side-channel hardening. The AES engine on the MCU has some constant-time properties by virtue of being hardware-accelerated, but we have not done any DPA-grade analysis. If an attacker can get a target onto a bench with EM probes and time, they will eventually extract the key. Our threat tier doesn’t pay for the countermeasures, but we’d note this explicitly to anyone evaluating us for a different deployment.

No forward secrecy at the pair-key level. The AES key derived at pairing time is the AES key used until unpair. We do not ratchet. An attacker who compromises a target today, recovers the AES key, and then turns up with all the captured ciphertext from last year can decrypt all of it. Pairwise ratcheting is on the wishlist but adds state machine complexity we have not paid for yet.

No quantum resistance. X25519 is not quantum-resistant. When post-quantum KEMs stabilise in standards and library support catches up to the embedded world, we will revisit. Today the additional bytes-on-the-wire for a Kyber768 key exchange would not survive a LoRa frame, and the threat is not in scope for our deployment lifetime.

No defence against sustained jamming. A determined RF adversary can make our band useless inside their broadcast range. The protocol doesn’t pretend otherwise; we lean on watchdog logic at the backend to notice when a target has gone silent.

What we would do differently if we shipped this again

Two things.

First, we would design the staged-session-promotion path on the target side with the dual-callback shape from the start, rather than discovering the ~1 ms RX-side window after deployment. The fix was easy; the diagnosis was not, because the symptom looked like a flaky radio rather than a state-machine race.

Second, we would put the boot-epoch field into the protocol from day one rather than retrofitting it. The original protocol assumed NVS was reliable; the realignment path was bolted on once we had field units coming back with corrupted persistence. It works, but adding fields to an authenticated header after the fact means a generation of devices need a firmware update before the field is universally trustworthy.

The bigger meta-lesson: in an embedded protocol, every persistent state on either side needs an explicit “freshness” signal in the authenticated payload, and every wire commit needs to follow on-air order, not in-RAM convenience. The cryptography is the easy part. The state machine around it is where the production-grade work is.

This post describes the cryptographic design of a small wireless system as it actually shipped. The MCU, the LoRa module, and the specific radio band are not named because none of that affects the shape of the design pick any modern Sub-GHz part and the same patterns apply.