The three layers of anti-bot detection

If you’ve ever built a scraper or an AI agent that touches the open web, you know the shape of the failure. Everything works locally. The code probably vibecoded but sort of works (maybe). Then the real site gives you a 403, a blank page, a redirect loop, or some cheerful “just checking your browser” screen that never clears.

The usual first thought is CAPTCHA. Maybe you need a solver. Maybe you need to click the challenge. Maybe you need to patch navigator.webdriver. Sometimes, sure. But that mental model is too small.

Modern anti-bot systems don’t sit in one place. They stack checks across the connection, the browser environment, and the user’s behavior. A lot of tools only deal with the middle layer, which is why they feel like they work on toy examples and then fall apart on real sites.

Here is the three-layer version I wish I had started with.

┌─────────────────────────────────────────────────────────┐
│  Layer 3 — Behavioral                                   │
│  Mouse movement · Typing cadence · Scroll patterns      │
│  Session history · IP reputation · Return patterns      │
├─────────────────────────────────────────────────────────┤
│  Layer 2 — Browser Environment                          │
│  Canvas · WebGL · AudioContext · Plugins · Fonts         │
│  navigator.webdriver · Automation artifacts              │
├─────────────────────────────────────────────────────────┤
│  Layer 1 — Network / TLS                                │
│  JA3/JA4 fingerprint · HTTP/2 SETTINGS · Cipher order   │
│  GREASE values · ALPS · Header ordering                 │
└─────────────────────────────────────────────────────────┘
  ▲ Detection happens bottom-up: network → browser → behavior

Layer 1: The request is suspicious before HTML exists

The first decision often happens before the server sends you a single byte of page content. Cloudflare, Akamai, AWS WAF, and similar systems inspect the TLS handshake itself. When a browser connects over HTTPS, it sends a ClientHello: cipher suites, extensions, ordering, supported features, all the little negotiation details. Chrome has a recognizable shape here. So does Safari. So does Firefox.

Most HTTP clients do not.

In Rust, the default path usually means reqwest plus rustls. That is excellent for normal networking, but it does not look like Chrome on the wire. The JA3/JA4 fingerprint is different. GREASE values are missing. Chrome-specific extensions such as ALPS, compressed certificates, and signed certificate timestamps are absent or ordered differently. The cipher suite list is not quite right.

And then HTTP/2 gives you another fingerprint. Chrome’s initial settings are not arbitrary. The values and order are part of the browser’s shape. Akamai even has a name for this: the “Akamai fingerprint.” If your TLS and HTTP/2 signatures say “generic library” while your User-Agent says “Chrome,” the site does not need to run JavaScript to know something is off.

This is why a plain HTTP client can get blocked before the DOM exists. No canvas. No WebGL. No automation flag. Just a network stack that does not match the identity it is claiming.

For Rust, the practical answer right now is wreq, a reqwest fork built around BoringSSL, the same TLS library Chrome uses. With the right profile, it can emit a Chrome-shaped handshake instead of a Rust-library-shaped one. That solves exactly one but pretty important problem.

Layer 2: The browser has to survive the challenge

If the network layer passes, the site may still serve a JavaScript challenge instead of the actual page. These challenges are usually small enough to look harmless, though some are large enough to be their own little application. They run in the browser, probe the environment, compute a fingerprint, and set a pass cookie if the result looks plausible.

This is where most people start, because this is the part you can see. The checks are not mysterious once you read the scripts:

• Can the canvas draw like a real machine? The script creates an invisible canvas, draws text or shapes, and reads pixels back with getImageData(). Real machines differ slightly because of fonts, GPU drivers, antialiasing, and OS rendering. Empty stubs, fixed constants, or all-zero buffers stand out.
• Does WebGL match the claimed device? A Mac-shaped User-Agent with a Linux software renderer is a bad story. The renderer string, supported extensions, and GPU behavior all have to agree with the rest of the fingerprint.
• Does AudioContext behave like a browser with an actual audio stack? Some challenges create an oscillator, pass it through a compressor, and inspect the resulting data. It is a strange test until you remember that real systems produce stable but device-specific output.
• Does the browser have normal browser stuff? navigator.plugins, screen dimensions, deviceMemory, hardwareConcurrency, visibility state, timing resolution, crypto.subtle, and a dozen other APIs all contribute little bits of evidence.
• Is automation obvious? navigator.webdriver = true is still the loudest possible signal. Playwright, Puppeteer, and Selenium have spent years teaching detection vendors exactly what automated browsers look like.

Layer 3: The session has to behave like a person

Suppose you pass the network fingerprint and the JavaScript challenge. You are still not done. Systems like DataDome, HUMAN/PerimeterX, and reCAPTCHA v3 look at behavior. Not in the vague “did the user seem human?” sense, but in a large pile of small measurements that become very hard to fake at scale.

Mouse movement is one example. Humans do not move pointers in perfect straight lines at constant speed. They accelerate, overshoot, correct, pause, and click slightly off-center. Typing has the same texture: uneven delays, occasional hesitation, key events in the right order. Scrolling comes in bursts. Sessions have history. Returning visitors have cookies, cached assets, and patterns from earlier visits.

Automation is often too clean. page.fill("input", "text") changes a value instantly without the messy trail of keyboard events. A click lands dead center. A fresh browser profile appears from a data center IP with no history, no cookies, no prior relationship to the site, and somehow claims to be a normal returning user.

reCAPTCHA v3 score model (approximate):

  0.0 ─────────────── 0.5 ─────────────── 1.0
  │                    │                    │
  Bot-like             Ambiguous            Human-like
  │                    │                    │
  ├─ DC IP             ├─ Residential IP    ├─ Residential IP
  ├─ No history        ├─ Some cookies      ├─ Returning visitor
  ├─ Instant typing    ├─ Basic mouse       ├─ Natural behavior
  └─ Fresh profile     └─ Partial session   └─ Full session history

Individually, these signals are small. Together, they are hard to ignore. DataDome tracks dozens of behavioral signals per session and trains models per site. reCAPTCHA v3 turns the whole mess into a 0.0–1.0 score. A brand-new automated session from a data center IP might score 0.1. Getting closer to 0.9 usually means residential IPs, coherent browser identity, realistic interaction timing, and a session history that does not reset every run.

This is also where one-off demos become misleading. It is possible to make one session look decent. It is much harder to make hundreds of sessions look like a believable population of real users.

The part most tools miss

The common failure mode is solving the visible layer and ignoring the other two. You patch navigator.webdriver. You return something from canvas. You add a few stealth scripts. Meanwhile, the TLS handshake already said “not Chrome” before JavaScript ran, and the behavioral model is watching a machine type an entire form in zero milliseconds.

The durable systems either cover all three layers or avoid some of the detection surface entirely. That is the real distinction. Not “does this bypass Cloudflare today,” but “does the whole session tell one consistent story?”

If the network says Chrome, the JavaScript environment says Chrome, the GPU says the same machine, the timezone matches the IP, and the user behavior looks like a low-volume human, blocking becomes more expensive and less certain.