← Course Home Module 12 · IoT Security Machinery: Anomaly Detection, Q-Learning & Crypto
Module 12 · Core Machinery

IoT Security Machinery: Anomaly Detection, Q-Learning & Crypto

Paper 3 defends billions of small connected devices with four pieces of machinery welded into one pipeline: an LSTM that treats attacks as forecasting errors, a salted hash that makes tampering visible, a Q-learning agent that decides how to fight back, and lattice-based encryption built to survive quantum computers. This module teaches each technology from scratch — the paper itself is Module 13's walkthrough.

▶
Audio recap
A ~2-minute spoken summary of this module — great for revision on the go.

12.1 The IoT security problem

The Internet of Things is billions of small networked devices — sensors, actuators, cameras, medical monitors, smart meters — most of them cheap, always on, and rarely updated. Every one of them is a door into a network, so connectivity growth is literally attack-surface growth. The classics set the tone: WannaCry (2017) — ransomware that swept hospitals and corporations worldwide; SolarWinds (2020) — a supply-chain breach that rode trusted software updates into government agencies; the Mirai botnet — hundreds of thousands of hacked IoT devices (cameras, routers) conscripted into a giant attack army.

Traditional defenses are rule-based and signature-based: they recognize attacks by matching them against a catalogue of known patterns. Three structural failures follow. They only catch known attacks; the catalogue needs constant manual updating; and they are far too slow for zero-day threats — attacks exploiting vulnerabilities no one has catalogued yet. In a dynamic IoT network, a defense that must be told what an attack looks like is always one attack behind.

Paper 3's idea Instead of patching one weakness, integrate four AI/crypto components — each solving a distinct security job — into a single unified pipeline: detect anomalies (LSTM), verify data integrity (homomorphic hashing), respond to threats automatically (Q-learning), and encrypt against present and future adversaries (lattice-based LWE). The claim is that the whole is an adaptive, quantum-proof defense that none of the parts is alone.
network traffic→ LSTM anomaly detection→ hash integrity check→ Q-learning response→ LWE-encrypted communication

12.2 Anomaly detection = forecasting + surprise

Here is the elegant reuse: Paper 3's detector is the same machine you mastered in Module 5 — an LSTM doing next-step time-series forecasting. Network traffic (packet sizes, inter-arrival times, protocol and source/destination identifiers, normalized and one-hot encoded) becomes a time series X = {x1, …, xT}, and the LSTM is trained with an MSE loss on 30-second lookback windows to predict what normal traffic does next:

x̂t+1 = f(xt, xt−1, …, xt−k) k = lookback window

At run time, the trick is to measure surprise. Compare what actually arrived with what the model expected:

Et = |xt − x̂t| anomaly score
anomaly if Et > τ τ = threshold

Small error: traffic is behaving as normal traffic behaves. Large error: something the model has never seen is happening — flag it. The paper trains and evaluates on a simulated IoT network of ~150,000 labeled events, 60% normal and 40% attacks (DoS floods, data injection, spoofing).

Why this catches zero-days The model learns only what normal behaviour looks like — so anything it cannot predict is suspicious, whether or not that attack has ever been seen before. No signatures, no catalogue, no manual updates. A brand-new attack is, by definition, not normal traffic, so it produces exactly the prediction error the detector watches for. That inversion — model the defender's own network instead of the attacker's tricks — is the core conceptual move.

12.3 Thresholds: where to draw the line

Everything above hinges on one number: τ. Paper 3 sets it at the 95th percentile of observed prediction errors (MSE values) — i.e. the top 5% most surprising moments get flagged. The choice is a trade-off dial, and you already have the vocabulary for it from Module 7: set τ low and you flag constantly — high recall (sensitivity), terrible precision, an operator drowning in false alarms; set τ high and alarms are trustworthy but real attacks slip underneath — high specificity, missed detections. A 95th-percentile cut is a reasonable default: it adapts to the error distribution of this network rather than a hard-coded magic number.

A fixed threshold is brittle Paper 3 found the limitation in its own results: at timestamp 810 the actual value was 1.25 against a predicted 0.72 — the anomaly was detected, but the model badly underestimated the magnitude of the sharp, rare spike, because an LSTM trained on historical patterns generalizes poorly to extremes it never saw. A single static τ also cannot follow a network whose "normal" drifts. The paper's own future-work answer: adaptive/dynamic thresholding that tracks the error distribution, and hybrid statistical + learning approaches. If an examiner asks "what would you improve?", start here.

12.4 Reinforcement learning in one sitting

This course has now shown you three ways a machine can learn. Supervised: learn from labelled examples (Paper 1's forecasters). Self-supervised: generate your own labels from the data (Paper 2's robot). The third paradigm is reinforcement learning (RL): an agent acts in an environment, receives rewards for good outcomes, and learns a policy — a rule for choosing actions — by trial and error. Nobody shows it correct answers; it discovers them through consequences.

The formal frame is a Markov Decision Process (S, A, P, R, γ): a set of states S the environment can be in, actions A the agent can take, transition probabilities P for where each action leads, a reward function R scoring outcomes, and a discount factor γ between 0 and 1. Discounting is just impatience made precise: a reward n steps in the future is worth γn of the same reward now — the future counts, but a bit less, so the agent prefers sooner payoffs and its sums stay finite.

Q-learning is the workhorse algorithm: learn a table Q(s, a) = the expected long-term (discounted) reward of taking action a in state s. Every experience nudges the table:

Q(s, a) ← Q(s, a) + α [R + γ·maxa′ Q(s′, a′) − Q(s, a)]

Read it symbol by symbol: α is the learning rate (how big a nudge); R is the reward just received; s′ is the state you landed in; maxa′ Q(s′, a′) is the best you believe you can do from there; and the bracket is the surprise — the gap between what this action turned out to be worth (immediate reward plus discounted best future) and what the table currently claims. Positive surprise nudges Q(s, a) up; negative nudges it down. Along the way the agent must balance exploration vs exploitation: sometimes try actions that look worse, or you never discover they were better.

One update, by hand Say Q(mild attack, block) = 4, α = 0.5, γ = 0.9. The agent blocks a mild attack, earns R = 10, and lands in a calm state whose best Q-value is 0. Target = 10 + 0.9×0 = 10; surprise = 10 − 4 = 6; new Q = 4 + 0.5×6 = 7. The table just learned that blocking mild attacks is more valuable than it thought — no human wrote that rule anywhere.

12.5 Q-learning as a security guard

Paper 3 keeps the setup deliberately small. States = threat levels reported by the detector: no attack, mild attack, severe attack. Actions = block (passive defense: drop/quarantine the traffic) or counter-attack (active defense against the source). Rewards are calibrated to threat severity, so the agent's incentives mirror real damage. The learned Q-table (the paper's Table 3):

StateBlockCounter-attack
No attack0.000.00
Mild attack10.005.00
Severe attack5.0015.00

Read the policy off the table by taking the best action per row, and notice it is sensible: when nothing is happening, do nothing — defensive maneuvers cost resources and there is no threat to justify them. For a mild attack, blocking (10) beats counter-attacking (5): neutralize the low-level threat cheaply and avoid escalation or collateral damage. For a severe attack, counter-attacking (15) beats blocking (5): against a high-severity threat, passive defense is not enough — decisive active measures stop the damage before it lands.

The point is adaptivity Nothing in that table was hard-coded. The policy was learned from interaction with attack scenarios, and it keeps updating as new experience arrives — including when attack signatures are obfuscated or the threat landscape shifts. A rule-based responder is frozen at deployment; a Q-learning responder is still learning in production. That is the same static-vs-adaptive argument as 12.1, now applied to response instead of detection.

12.6 Hashing and integrity

Detection asks "is this traffic behaving normally?" Integrity asks a different question: "is this data the same data that was sent?" The tool is a cryptographic hash function: a one-way function mapping any input to a fixed-size fingerprint (SHA-256: 256 bits), designed so that any change to the input — a single flipped bit — produces a completely different hash. Recompute the hash on arrival; if it no longer matches, the data was tampered with in transit. The salt s is a secret extra input mixed in before hashing, so an attacker who intercepts data cannot precompute or forge matching hashes without knowing s:

H(m, s) = SHA-256(m ⊕ s) m = data, s = secret salt
verify: H(m′, s) = H(m, s) fails ⇒ tampering detected

The "homomorphic" property Paper 3 leans on: integrity can be verified on encrypted or distributed data without revealing its content — the check runs on fingerprints, never on the plaintext. That enables two things IoT badly needs. It is decentralized: any node can verify locally, with no round-trip to a central validation server that would be a bottleneck and a single point of failure. And it is lightweight: hashing is cheap enough for real-time, packet-level checks on constrained devices.

Fingerprint intuition Mailing a document with its hash is like mailing it with a wax seal keyed to every letter on the page: change one comma and the seal shatters visibly. The salt is the private seal-stamp — without it, a forger can rewrite the document but cannot re-seal it.

12.7 The quantum threat and lattice crypto

Today's public-key encryption (RSA, elliptic curves) rests on problems like factoring huge numbers — hard for ordinary computers, but Shor's algorithm running on a large quantum computer would solve them efficiently, breaking RSA and ECC outright. Post-quantum cryptography means schemes built on problems believed hard even for quantum computers. The leading family is lattice-based cryptography, grounded in the Learning With Errors (LWE) problem:

y = Ax + e mod q A random matrix, x secret key, e small noise

The intuition: given A and y with exact equations (e = 0), recovering the secret x is basic linear algebra — easy. But add a little random noise e to every equation and the system becomes computationally intractable to unscramble: you can no longer tell which tiny errors were added where, and every candidate x is thrown off by noise you cannot separate out. Noisy linear equations are hard to solve — for classical and, as far as anyone knows, quantum computers alike. As a bonus, LWE-based schemes support homomorphic operations — computing on data while it stays encrypted — which pairs naturally with 12.6's verify-without-revealing theme. For constrained IoT hardware, lightweight variants such as RLWE (ring-structured, smaller keys) and the Foxtail+ protocol trade parameters for memory and energy efficiency; Paper 3 additionally precomputes keys so encryption adds minimal latency between devices.

Harvest now, decrypt later "Quantum computers don't exist yet, so why hurry?" Because adversaries are already recording today's encrypted traffic to decrypt in the quantum era. Any data whose secrecy must outlive the arrival of large quantum machines — medical records, state secrets, infrastructure credentials — is effectively already exposed if it travels under RSA/ECC today. That is why migrating to post-quantum encryption early matters, and why Paper 3 bakes it in rather than bolting it on.
Exam warm-up — say it out loud
  1. Why can prediction-based anomaly detection catch zero-day attacks when signature-based systems cannot?
  2. Explain the Q-learning update rule symbol by symbol, then justify Paper 3's learned policy row by row.
  3. What makes LWE hard, in plain words — and why doesn't Shor's algorithm help?
  4. Why is a 95th-percentile threshold both a reasonable choice and a brittle one?

Module 12 Quiz

10 questions on the four technologies. Master these before Module 13's paper walkthrough.

← Previous
Module 8: The Optimization Toolbox