krishworkstech.com

Share at:
grok-1
chatgpt-6
gemini-icon-logo
perplexity-color-1-

The setup

The system is small enough to draw on the back of a napkin:

Target battery, field-deployed Hub mains, N paired targets Backend cloud, MQTT LoRa Wi-Fi/MQTT
Figure 1. Three components, two channels. The LoRa link is the part this article is about.

Why a hardware RNG matters more than you'd think

The whole edifice — the secrecy of the X25519 private key, the uniqueness of every AES-CCM nonce, the unpredictability of session IDs — rests on one quietly demanding assumption: you can produce random numbers that an attacker cannot guess.

This is harder than it looks. Computers are by design deterministic: given the same inputs they produce the same outputs. So where does randomness come from?

Two kinds of "random"

The classic embedded failure mode is seeding a software PRNG from the boot clock or the uptime counter. Both are easily predictable to anyone who can guess when the device powered on. There have been real shipping IoT products that generated “cryptographic” keys this way and were trivially broken.

What we need is true randomness — bits whose value an attacker cannot predict no matter how much they know about the device, the firmware, the timing, or the history.

Where physical randomness comes from

Physics provides several reliable sources of unpredictability at the chip level:

  • Thermal noise across a resistor or a junction.
  • Jitter in the relative phases of free-running oscillators.
  • Metastability in latches that are deliberately driven into an undefined region.

Modern MCUs include a small dedicated peripheral — the hardware RNG, also called a TRNG (True Random Number Generator) — that samples one of these physical phenomena, runs a small amount of whitening over it to flatten any residual bias, and presents the result as a stream of bytes you can read out of a register.

These are not free — they take many microseconds to produce a single 32-bit word, because they wait for enough physical entropy to accumulate. They are also not infinitely fast under load. So practical embedded cryptographic stacks layer them:

Physical entropy source thermal noise / oscillator jitter / metastability — inside the MCU's HW RNG block slow but truly unpredictable CSPRNG (e.g. CTR_DRBG, HMAC_DRBG) stretches a small entropy seed into gigabytes of strong output fast, suitable for all callers Consumers X25519 private key session IDs (mySid) nonces, salts padding, blinding
Figure E. The RNG layered design. Physical entropy from the hardware seeds a cryptographically secure pseudo-random generator, which feeds every cryptographic consumer in the system.

The CSPRNG — cryptographically secure pseudo-random number generator — is itself deterministic, but it is seeded with real entropy from the hardware. Once seeded, it can produce gigabytes of output that looks indistinguishable from true random, far faster than the hardware source alone could manage. Common embedded choices are CTR_DRBG (built on AES) and HMAC_DRBG (built on a hash).

You typically also feed in a personalisation string when seeding the CSPRNG — something device-specific, like the chip’s factory-fused unique ID. Its job is to make sure two devices that happen to be in identical states at the same wall-clock instant still produce different output streams. Belt-and-braces, but cheap.

The practical pattern in our code is therefore:

  1. At first use, call mbedtls_ctr_drbg_seed() with the hardware RNG as the entropy source and a short personalisation string.
  2. From then on, every “give me random bytes” call goes through mbedtls_ctr_drbg_random(), which is fast and produces cryptographically strong output.
  3. Sensitive intermediates — private keys, shared secrets — are zeroed from memory after use so they cannot be lifted from RAM by a later compromise.

If any of those steps is wrong — the hardware RNG was never enabled, the seeding was done with a constant string, the CSPRNG was reseeded with a guessable value, the keys were left in memory after use — every other security property in this article quietly collapses. This is why the unglamorous plumbing in the RNG layer gets as much care as the cipher itself.

Three layers of state

This is where the design actually lives. Cryptography on an embedded device is not one big “secure context” — it’s three independent layers of state, each with its own lifetime, its own loss-detection story, and its own commit-and-rollback policy.

IDENTITY survives reboots, reflashes, factory reset "who am I" — long address, X25519 private & public keys lifetime: device lifetime PAIR survives reboots, lost on unpair / factory reset "who do I trust" — peer addresses, derived AES key, PAN ID lifetime: until unpair SESSION fresh every boot, exchanged on first message after restart "what nonce space am I in" — 16-bit mySid, peer's most-recent peerSid lifetime: until reboot
Figure 2. Three layers of state. Each layer has its own loss-detection story and its own commit policy.

Identity

Pair

When a hub and a target successfully complete pairing, both sides derive a per-pair 128-bit AES key from their X25519 shared secret, and both sides persist:

  • the peer’s long and short addresses
  • the derived AES key
  • the PAN ID they agreed on

Session

A session is a short-lived nonce domain. On boot each device:

  1. Reads a 32-bit value from the HW RNG.
  2. Mixes it through xorshift32 to whiten any peripheral bias.
  3. Takes the low 16 bits as mySid. Value zero is reserved as “not yet known”; if the random draw gives zero, retry.

mySid is the device’s view of “what session am I in.” Its peer’s session — peerSid — is whatever the peer most recently announced.

These are needed to construct the nonce. They have to be rolled freshly per boot, because if a device crashes mid-burst and replays old packets after reboot, the nonce must not repeat. That is the whole point: a session ID is a domain separator that turns “this is my 42nd packet of all time” into “this is my 42nd packet of this boot’s session.” The receiver tracks one counter per session, not one counter forever.

The asymmetry is the part that catches people. It is not “session = a pair of random numbers agreed once during pairing.” It is “each side independently rolls a fresh value at boot, and they tell each other about it on the first message after they come up.” If only one side has rebooted, the other already knows its own session, so the returning side just announces its new value and we continue.

If both sides reboot at once — say a power outage — neither knows the other’s session. The protocol’s first encrypted message they exchange is a session resync: each side announces its own mySid and echoes what it thinks the other’s mySid is. After one round trip both sides know both values. No out-of-band coordination needed, no operator standing in front of the device with a pairing button.

This design is the thing that makes the system robust to power loss in ways that LoRaWAN’s frame-counter discipline is not. LoRaWAN devices that hard-code session keys and reset their counter to zero on reboot are a famous footgun. Here, every reboot lands us in a fresh nonce space by construction.

The nonce, in detail

The nonce is the heart of the cryptosystem and deserves a careful walk-through. AES-CCM requires a nonce that is never reused with the same key. Reuse breaks confidentiality entirely; for any pair of frames encrypted under the same (key, nonce) the XOR of the ciphertexts is the XOR of the plaintexts, which is usually game over.

We construct a 12-byte CCM nonce out of fields that are guaranteed by construction never to repeat:

srcShort 2 B mySid 2 B peerSid 2 B packetId 6 B (48-bit monotone counter) 12 bytes total — fed straight into AES-CCM as the nonce who I am 16-bit short addr my own fresh session at boot receiver's claimed session, fresh on receiver's reboot monotone counter, persistent within session, reset to 0 on new session The cleartext header (addresses, PAN ID, packetId) is fed as AAD — authenticated but not encrypted. Any tampering invalidates the MIC.
Figure 3. The 12-byte AES-CCM nonce. Every field is doing real work: identity, sender-side freshness, receiver-side freshness, and a monotone counter inside a session.

Each field is doing real work:

  • srcShort — domain separation between targets. Two targets in the same PAN that happen to draw the same mySid value cannot collide in nonce space because their short addresses differ.
  • mySid — domain separation between reboots of the same target. Reboot → fresh mySid → fresh nonce space → all old packets become invalid replays because their nonces fall in the wrong half.
  • peerSid — binds each packet to a specific receiver session. A captured packet replayed at the hub after the hub has rebooted will have an obsolete peerSid and fail the MIC check.
  • packetId — monotone counter within a session. Each (src, mySid, peerSid) tuple has its own packetId space starting at 0.

The math is worth doing aloud. With the HW RNG providing a uniform 16-bit mySid per boot, the probability that two consecutive boots of the same device happen to pick the same session is 1 in 65 536. Combined with the receiver also being in a fresh session — another 1 in 65 536 of accidentally matching — the chance that a captured packet’s full nonce repeats is small enough to ignore inside our threat model. And even if it did, the MIC check would catch it.

What goes into the AAD

Replay protection

Each side maintains a per-peer last-seen packetId. Packets with packetId <= last_seen are dropped before AES even gets called. This catches replays inside an active session, cheaply. Cross-session replays — captured-yesterday, replayed-today — die at MIC verify because the nonce contains stale mySid or peerSid values.

Here is the generic encrypt-side flow, condensed:

/* Build the 12-byte CCM nonce. */
nonce[0] = srcShort & 0xFF;  nonce[1] = srcShort >> 8;
nonce[2] = mySid    & 0xFF;  nonce[3] = mySid    >> 8;
nonce[4] = peerSid  & 0xFF;  nonce[5] = peerSid  >> 8;
nonce[6]  = (packetId >>  0) & 0xFF;
nonce[7]  = (packetId >>  8) & 0xFF;
nonce[8]  = (packetId >> 16) & 0xFF;
nonce[9]  = (packetId >> 24) & 0xFF;
nonce[10] = (packetId >> 32) & 0xFF;
nonce[11] = (packetId >> 40) & 0xFF;

/* The cleartext header is the AAD; payload is encrypted in place. */
rc = aes_ccm_encrypt_and_tag(
        key,
        nonce,    NONCE_LEN,
        header,   HEADER_LEN,        /* AAD */
        payload, ciphertext, len,    /* PT in, CT out, same buffer */
        tag,      TAG_LEN);

The receive side mirrors this. Both sides drive the same primitive through the same library; there is one place in the codebase where the nonce is built, and there is no other.

Pairing

Pairing is where you give two devices the shared key they will use forever. Most pairing bugs are not crypto bugs — they are state machine bugs that put one side into “we are paired” and the other into “we are not.” We will get to those.

The high-level walk:

Target Hub ORPHAN BLINK (plaintext, "I am here") operator opens PAIRING window (seconds-long, explicit) DISCOVERY (unicast, plaintext) DISCOVERY-ACK (plaintext, no secrets) derive per-pair AES key PAIRING (encrypted) CONNECTION (encrypted) target commits to NVS hub commits paired_db OPERATION: encrypted HEALTH / events
Figure 4. Pairing sequence. Plaintext frames carry no secrets; encrypted frames use the freshly derived per-pair AES key. Durable commits are deferred until the on-air TX is confirmed.

A few decisions in this flow deserve explicit defence.

The pairing window is operator-gated. The hub does not accept any DISCOVERY-ACK at any time. The operator presses a button, or the backend issues a “begin pairing” command, and the hub opens a seconds-long window. Outside that window, even otherwise-valid candidates are rejected. This is what makes the system “pair-bounded” in the threat-model sense: drive-by pairing is impossible.

DISCOVERY-ACK is plaintext, on purpose. It contains no secret material. Its only job is to say “yes, I’m here, send me the next step.” Requiring a pre-shared key to acknowledge discovery would defeat the point of having a pairing flow. The next message, PAIRING, is encrypted with the freshly derived per-pair key; a stranger who saw the plaintext DISCOVERY-ACK still cannot forge a PAIRING reply or read its contents.

The AES key is derived from the X25519 shared secret with HKDF-SHA-256. The HKDF info string is built from the PAN ID and both endpoints’ long addresses, in canonical order. That binding prevents an attacker who captures a pairing exchange from replaying it on a different PAN, and it prevents the rare case of two pair attempts producing colliding keys when only one of the long addresses differs.

Durable state changes are deferred until TX-acknowledgement. This is the rule that catches half the pairing race conditions. The hub does not commit its paired_db row until the PAIRING-ACK frame has been confirmed on the air. The target does not commit its paired record to NVS until the CONNECTION frame it sends out has been acknowledged. If a TX fails after retries, RAM state rolls back cleanly. The AES key lives in RAM only until the durable commit boundary.

That last rule sounds straightforward when written down. It is not straightforward to implement. The story is in the next section.

The races we hit

Documenting failure modes is more interesting than documenting successes. Here are three race windows we shipped and then fixed.

Race 1 — split-brain pairing: the hub committed but the target didn't

What we shipped first. When the hub decided to accept a pairing attempt, it queued the PAIRING-ACK frame, immediately queued the follow-up CONNECTION frame, and committed the paired_db entry. The optimistic ordering get both frames in the radio queue while we have the keys hot  was meant to minimise pairing latency.

What went wrong. The PAIRING-ACK transmission sometimes failed after retries (the target had moved, or the link was momentarily congested). The TX-fail callback rolled back the in-RAM paired_db entry. But the CONNECTION frame was already in the queue, or in the air, or even already failed on its own. The hub then fired a connection_failed event for a pairing relationship that, as far as either side was concerned, did not exist. The backend dashboard lit up with a confusing trail of events; field engineers thought there was a bug in the radio stack.

The fix. Reorder the commit semantics:

WRONG:
queue PAIRING-ACK
queue CONNECTION
commit paired_db

on TX-fail of either:
  roll back, but
  second one is in flight
RIGHT:
queue PAIRING-ACK
wait for PAIRING-ACK TX-confirmed
  ↓ on confirmed:
    queue CONNECTION
    commit paired_db
    on TX-fail of CONNECTION:
      FSM-level retry, bounded
  ↓ on TX-fail of PAIRING-ACK:
    roll back; no CONNECTION ever
    on the wire

The new rule is: CONNECTION is queued after PAIRING-ACK has left the air, not after PAIRING-ACK has been queued. The TX-confirmed callback for PAIRING-ACK is what triggers the CONNECTION queue push and the durable commit; the TX-fail callback for PAIRING-ACK rolls back cleanly because CONNECTION has not been touched yet.

The lesson is general: when an embedded protocol commits state at multiple stages, the commit order has to follow the wire order, not the convenience of having all the data in scope at once.

Race 2 — the millisecond between "we sent the reply" and "we promoted the session"

This one is subtle and worth the page.

The setup. On the target side, the freshly derived per-pair AES key lives in a “staged” session slot during the brief interval between receiving PAIRING and being sure we have replied to it. It gets promoted to the “live” session table only when our reply (the CONNECTION frame) is TX-confirmed.

The hub, meanwhile, sends its first encrypted operational frame as soon as it sees our CONNECTION TX. There is a window — roughly one millisecond on this stack — between the radio reporting “your reply left the air” and our own software actually getting to the “promote staged session to live” call.

If the hub’s first encrypted frame arrives in that millisecond, our RX dispatcher looks up the session by short address, finds nothing in the live table, and drops the frame. The hub retries, the target finally promotes, retry succeeds, no permanent damage — but every single pairing produced one or two spurious MIC failures in the logs. Some installations correlated the spurious failures with “pairing is unreliable” and we got tickets.

The fix. Add a defensive fallback in the RX dispatcher: if the live session lookup fails, peek at the staged-session slot. If the staged key successfully verifies the MIC, promote the staged slot to live as a side-effect and continue with the decrypted frame. Now the millisecond window is closed at both ends — the on-air queue’s TX-ack callback still drives the canonical promotion path, but an opportunistic RX-side promotion is also legal.

The deeper lesson is that “the durable commit happens at TX-confirmed” is a clean rule on the sending side, but the receiving side cannot assume the same timing. If you have a window where state is correct in one place and stale in another, document the fallback path explicitly.

Race 3 — the target's NVS got wiped and nobody told the hub

The setup. Targets persist their paired record to non-volatile storage. Occasionally that storage is corrupted — flash wear-out on a field unit, a botched firmware update that erased the wrong sector, a development board where the NVS partition is a RAM-only stub. The target then boots into ORPHAN state, sends a plaintext BLINK, and waits to be re-paired.

The hub, meanwhile, still has the target in its paired DB. From its point of view, the target is just being weirdly quiet, then suddenly sending plaintext frames that fail every encrypted-frame check.

What we wanted. A way for the target to tell the hub “I have lost my state, you should realign your expectations of me,” without needing to re-pair from scratch every time.

The fix. Add a small persistent boot epoch value to the target’s identity layer. Every operational packet carries the target’s current epoch in its (authenticated) header. When NVS is healthy, the epoch is loaded from NVS on boot and is stable. When NVS is missing, the epoch is seeded fresh from the HW RNG so it is guaranteed to differ from anything the hub has seen before.

The hub treats a changed epoch as “this target’s persistent state has been reset; tear down the pairing and run the normal orphan-detection path.” On the development board, where NVS is a RAM-only stub, the epoch changes every reboot and the realignment path fires every boot, which is exactly the same protocol path as the production NVS-corruption case, just exercised more often. That kind of accidental test coverage is its own reward.

The general pattern is worth naming: if a peer can lose persistent state in a way the other peer cannot detect, give it a small explicit “freshness” field and make it part of the authenticated header. Without this, you end up trying to debug ghost pairings by power-cycling things, which is not a strategy.

The boring decisions that matter

These are the policy decisions outside the cryptography itself — boring on the surface, load-bearing in practice.

Operation LoRa MQTT UART
Pair / unpair targets yes yes yes
View paired DB n/a yes yes
View integrity events n/a yes yes
Set PAN ID no no yes only
Factory reset the hub no no yes only
Rotate identity keypair no no yes only

The two emphasised nos — PAN change and factory reset over MQTT — are deliberate. These are the operations whose abuse compromises the device’s whole identity. They require physical presence on the box. A compromised backend, a stolen MQTT credential, or a malicious backend administrator cannot move the hub to a different network or rotate its keys. They can issue normal pair/unpair commands; they cannot cause an identity-level event.

The cost of this policy is zero — these are not operations you perform remotely in normal life. The benefit is that an entire class of cloud-side attacker capability is structurally absent. It is, in the literal sense, a defence-in-depth choice that costs nothing.

Pairing requires an explicit, time-bounded window. Not a flag set once at install and forgotten. Not a “first-come-first-served” acceptance of any BLINK ever heard. The operator opens a window of a few seconds; the hub accepts pairings during that window only. A flood of fake DISCOVERY-ACKs during a window can occupy the slot, but the backend sees the unauthorised-pairing event surface in real time and an operator can react.

Crash-during-pairing is engineered for, not hoped against. The whole “durable state changes deferred until TX-acknowledgement” discipline is what makes this work. The session and AES key are in RAM only until the commit boundary; if the radio TX times out after retries, RAM rolls back and the device returns to its prior durable state without ambiguity.

What we would do differently if we shipped this again

Two things.

First, we would design the staged-session-promotion path on the target side with the dual-callback shape from the start, rather than discovering the ~1 ms RX-side window after deployment. The fix was easy; the diagnosis was not, because the symptom looked like a flaky radio rather than a state-machine race.

Second, we would put the boot-epoch field into the protocol from day one rather than retrofitting it. The original protocol assumed NVS was reliable; the realignment path was bolted on once we had field units coming back with corrupted persistence. It works, but adding fields to an authenticated header after the fact means a generation of devices need a firmware update before the field is universally trustworthy.

The bigger meta-lesson: in an embedded protocol, every persistent state on either side needs an explicit “freshness” signal in the authenticated payload, and every wire commit needs to follow on-air order, not in-RAM convenience. The cryptography is the easy part. The state machine around it is where the production-grade work is.

Scroll to Top
  • Schematic design
  • PCB and schematic source files
  • Assembling drawing files
  • Providing prototype/sample and production PCB service
  • Testing and validation of designed hardware
  • HIPAA
  • Azure Key
  • Management
  • ES, Checksum,
  • MD5sum
  • AWS
  • Azure
  • GCP
  • DigitalOcean
  • Kotlin
  • Python
  • Tensorflow
  • Computer Vision
  • ECG
  • SPO2
  • Heart Rate
  • Glucometer
  • Blood Pressure
  • UX UI Process
  • Figma and FigJam
  • Adobe Suite
  • Selenium Java
  • Postman
  • Swagger
  • Jmeter
  • SQL
  • Java Scripter
  • Test ng
  • Extents Reports
  • Flutter
  • Java
  • Kotlin
  • Swift
  • Dart
  • React JS
  • Python
  • NodeJS
  • Django
  • HTML, CSS, JS
RDBMS
  • PostgreSQL
  • Oracle
  • MySQL
  • MariaDB
No SQL Based
  • MongoDB
  • GCP
  • FirestoreDB
  • DynamoDB
  • Azure
  • CosmosDB
  • AWS