What Is a CAPTCHA Challenge? Cost, bypass and detection

Most teams deploy a CAPTCHA challenge believing it separates humans from bots. The published measurements say it does something narrower and stranger than that. This piece walks the request flow, the solver economics, and the false-positive population, using other people's numbers because they are better than ours.

What Is a CAPTCHA Challenge?

A CAPTCHA challenge is a test a website serves when its risk scoring cannot decide whether a request came from a person. It works by asking for a response that is cheap for humans and expensive to automate. Defenders detect abuse through the telemetry collected around the challenge, not the answer itself. Start by measuring your escalation rate.

Where these numbers come from

I want to be blunt about provenance before the first claim lands.

Blockdash did not run original telemetry for this article. Every figure below belongs to somebody else, and every one of them is attributed to a named researcher, a dated paper, or a vendor's own published documentation.

That is a limitation, and it is also the point. The CAPTCHA literature is unusually good. Two large academic user studies, one solver-market survey, one attack paper, and one agent benchmark cover more ground than any single operator's logs would.

Where a source is an interested party, such as a solver service's price list or a CAPTCHA vendor's own blog, I say so in the sentence.

Where two sources disagree, I say who and pick a side.

Sidenote. If you want to reproduce any of this against your own traffic, the two UC Irvine studies both publish their data and analysis code. The 2023 USENIX dataset is at github.com/sprout-uci/captcha-study.

A CAPTCHA challenge is a cost-imposition control, not a humanity test

Start with the thing the name gets wrong.

CAPTCHA stands for Completely Automated Public Turing Test to Tell Computers and Humans Apart, a term Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford formalized at EUROCRYPT 2003. Their framing was explicit and it was a security-economics framing, not an identity one.

The argument went like this. Build the test on an AI problem nobody has solved. Either the test holds, or somebody solves the AI problem and everyone benefits.

That was a bet on a gap staying open. It closed.

By 2014, Ian Goodfellow and colleagues at Google published a convolutional network reaching 99.8% on the distorted-digit recognition task that the reCAPTCHA of the era was built on. The paper was about reading Street View house numbers. It happened to read the CAPTCHAs too.

So the test no longer tells computers and humans apart. What it still does is impose a cost.

That distinction is the whole article, so here it is stated flatly: a modern CAPTCHA challenge does not ask "are you human." It asks "is completing this worth more to you than it costs."

Look at what the request flow actually does and the reframe becomes obvious.

A third-party CAPTCHA deploys as an iframe holding a public site key. Rui Jin and colleagues at the University of Science and Technology of China documented the full sequence in June 2023.

The client loads the widget. The widget collects environment and behavior data. That data goes to the provider, not to you.

The provider decides. If it decides favorably, it mints a response token and hands it to the client.

The client posts that token to your server. Your server calls the provider's verification endpoint with the token and your private secret key, and gets back a verdict.

Notice what your application never sees.

You never see the images. You never see the mouse trace. You never see the fingerprint. You see a token and a boolean, and on scoring products, a float.

Which means the challenge your user solves is not the control. The control is the provider's classifier, and the visible puzzle is what happens when that classifier abstains.

Andreas Plesner, Tobias Vontobel and Roger Wattenhofer at ETH Zurich confirmed this from the attacker side in their COMPSAC 2024 paper. Looking under the hood of reCAPTCHAv2, they found the system leans heavily on cookie and browser history data when deciding whether a session is human.

Their sharper finding: they measured no significant difference in the number of challenges humans and bots had to solve. The puzzle difficulty was not the discriminator. The session history was.

Now go one level further down and ask why the architecture is built this way.

It is built this way because of a business constraint, not a security one. A CAPTCHA provider serving millions of sites has a cross-site view that no individual site has. Google's own reCAPTCHA v3 documentation states the mechanism plainly: the system learns by seeing real traffic on your site, and scores in a staging environment or shortly after implementation will differ from production.

That is a network-effect asset. The provider's edge is aggregate history, which is exactly the asset that cannot be given to the customer without destroying it.

So the visible puzzle survives partly as the fallback for sessions the aggregate has no opinion on. Users with no cookie, no history, a fresh browser profile, or a privacy configuration that strips all three.

Which produces the operational consequence most teams miss. The population that gets challenged is not the bot population. It is the low-history population, and bots are only one part of that.

Here is the cost side in real numbers.

Searles, Prapty and Tsudik at UC Irvine ran a black-box network analysis of reCAPTCHAv2 as part of their November 2023 study. A page with reCAPTCHA on it pulled 408.5 KB on first load against 0.631 KB for the same page without it.

Subsequent loads dropped to 29.3 KB once assets cached.

Page load latency, measured three ways: 374.65 ms added via Chrome DevTools, 471 ms via Pingdom, 1,260 ms via WebPageTest. The spread is a function of network distance from the host, and the largest number is the most realistic for a global user base.

That is the price before anybody clicks anything.

The counter-case, and it is a strong one. Reid Tatoris and Ben Wolters of Cloudflare, writing in April 2022, stated directly that CAPTCHA remains an effective tool for differentiating real human users from bots despite the existence of solving services. Their argument was that the alternative to a challenge is a binary allow or block, and a binary decision inflates either false positives or false negatives.

They are right about the binary problem and I think they are wrong about the framing. A challenge is a third option, but its value comes from the cost it imposes and the telemetry it generates, not from any residual ability to tell a person from a program.

Cloudflare's own subsequent behavior supports the cost reading more than the humanity reading. They kept the challenge and threw away the puzzle.

What to instrument this week: log the ratio of sessions that receive a visible puzzle to sessions that pass silently, segmented by whether the session arrived with a prior cookie. If those two segments look nothing alike, your challenge is measuring history, not humanity.

The checkbox is not the test. The 400 KB of JavaScript around it is.

Here is the single most useful measurement in the CAPTCHA literature, and almost nobody cites it.

In the UC Irvine 13-month deployment, a Playwright-driven headless Chrome browser was served an image challenge on its first request. Every time.

A Jitbit mouse macro replaying recorded cursor movement and clicking the same checkbox was not. It took roughly 14 rapid attempts before an image challenge appeared, which is the same behavior the researchers observed from manual human clicking.

Sit with that for a second.

The macro is an automation tool. It produces no human intent, no human decision, no human anything. It sailed through.

The headless browser produced a perfectly plausible click and got stopped cold.

The discriminator was not the interaction. It was the automation surface underneath it.

That result comes from Appendix B.6 of the Searles, Prapty and Tsudik paper published November 2023, and it is the clearest published demonstration I know of that behavior-based CAPTCHA is a browser-integrity check wearing a behavior costume.

Now the mechanism, three levels down.

Level one, the immediate cause. Playwright and similar frameworks leave detectable artifacts in the JavaScript environment. Automation flags on the navigator object, driver-injected properties, timing signatures in the event loop, differences in how the rendering stack responds to probing. A macro driving a real Chrome profile leaves none of those, because it is a real Chrome profile.

Level two, the property of the system that makes this possible. The reCAPTCHA widget executes roughly 167 KB of minified JavaScript inside its own iframe with full access to the browser's introspection APIs. That file, recaptcha__en.js, is fetched on every uncached page load and appears three times in the UC Irvine network trace.

That code is not scoring the puzzle. The puzzle is 2 KB of PNG. The code is enumerating the client.

Level three, why the browser is built to permit it. The web platform exposes an enormous amount of client state to same-origin script for backwards-compatibility reasons that predate any of this. Canvas, WebGL, font metrics, audio context, timing precision, and dozens of navigator properties were all shipped to solve real developer problems and were never designed to be non-discriminating.

Nobody decided browsers should be fingerprintable. It fell out of twenty years of shipping capability.

Which is why the same code path is used by every commercial bot-detection product regardless of whether it draws a puzzle. The puzzle is the part you can see.

Worked example one, from the same study's dashboard export.

Across 9,538 reCAPTCHAv2 sessions on a live university account service, Google's own admin console reported 7,629 sessions passing on the checkbox alone. 1,890 sessions escalated to an image challenge and passed. 143 escalated and failed the image.

19 sessions failed outright.

So on a population that we know was human, because every participant had authenticated to a campus VPN before reaching the form, 20% got escalated to a picture puzzle and 0.2% were ultimately refused.

Read those two numbers as a pair. The system's hard-block rate on legitimate humans was very low. Its friction rate was one in five.

Worked example two, the solve times behind that escalation.

Checkbox-only sessions: mean 1.85 seconds across 7,334 records, median 1.67, standard deviation 0.71.

Sessions that escalated to images: mean 10.3 seconds across 1,807 records, median 8.20, standard deviation 6.54.

That is a 557% increase in time-on-challenge, and the variance goes from 0.50 to 42.8. The escalated path is not just slower. It is wildly less predictable, which is what a long tail of frustrated users looks like in a distribution.

The economics of suppressing the automation signal are worth stating, because they are not symmetrical with the economics of the puzzle.

Solving one image challenge through a human farm costs, per the published price lists Jin and colleagues collected in 2023, between $0.60 and $2.99 per thousand. Call it a third of a cent.

Suppressing the automation surface so the checkbox passes silently costs nothing per attempt. It is a fixed engineering cost paid once. That asymmetry is why the checkbox path is the one attackers optimize and the image path is the one they buy their way past when optimization fails.

Named victim one: the QA engineer whose own Playwright integration suite hits the signup form in CI. That suite is indistinguishable from the thing the control is built to stop, and it will eat an image challenge on every run. Teams routinely respond by allowlisting the CI egress IP, which creates an unauthenticated bypass on a static address.

Named victim two: the user running Firefox with Resist Fingerprinting enabled, or a hardened Brave profile. Those configurations exist specifically to flatten the introspection surface, which produces a client that looks unusual in exactly the direction automation looks unusual.

Counter-case, adjudicated. Jin and colleagues found that hCaptcha's operator-configurable difficulty setting had little impact on either the response time or the success rate of the solving services they tested. Searles and colleagues, testing humans at USENIX Security 2023, found difficulty mattered a great deal to people, with hCaptcha's human pass rate dropping from 81.4% on easy to 70.6% on difficult.

Both are correct and together they are damning. Turning up difficulty degrades your users and does not degrade the attacker. If you have a difficulty dial, that is the direction the evidence points.

What to instrument: split your challenge outcomes by whether the session produced a visible puzzle, and alert on the escalation rate rather than the failure rate. A rising escalation rate against flat failure is your control quietly taxing real customers.

Humans take 1.85 seconds on the checkbox. Bots take 1.4.

The comparison table in the USENIX Security 2023 paper is the most quietly devastating thing in the field, so let me just read it out.

Searles, Nakatsuka, Ozturk, Paverd, Tsudik and Enkoji recruited 1,400 participants who collectively solved 14,000 unmodified, currently-deployed CAPTCHAs. Then they set the human results next to the fastest published automated attacks on each type.

Behavior-based reCAPTCHA click: humans 3.1 to 4.9 seconds at a 71 to 85% pass rate. The automated attack, from Suphannee Sivakorn, Iasonas Polakis and Angelos Keromytis at IEEE EuroS&P 2016, ran 1.4 seconds at a 100% pass rate.

Distorted text: humans 9 to 15.3 seconds at 50 to 84% agreement. Automated, under one second at 99.8%.

GeeTest slider: humans 28 to 30 seconds. Automated, 5.3 seconds at 96%.

hCaptcha: humans 18 to 32 seconds at 71 to 81%. Automated, 14.9 seconds at 98%.

Image-based reCAPTCHA is the only row where humans are competitive. Humans 15 to 26 seconds at 81%, automation 17.5 seconds at 85%.

And in the contextualized condition, where participants believed they were creating an account rather than taking part in a study, human time on that row rose to 22 seconds. Which puts people behind the machines on the last task where they were ahead.

A precision note, because the papers use a word I will not.

Those percentages are pass rates against the challenge, not precision against a population. The 100% figure for the Sivakorn attack means the attack passed every challenge it attempted, under its own experimental conditions. It does not mean the control has zero discriminating power over all traffic.

The distinction matters and the field is sloppy about it.

Now the mechanism behind that 100%, because it is the most instructive attack in the literature.

Sivakorn and colleagues did not defeat the puzzle. They defeated the history.

Their 2016 result: at least 63,000 valid tracking cookies could be automatically created per day from a single IP address. After nine days of aging, checkbox attempts using those cookies succeeded. That yielded 52,000 to 59,000 checkbox solves per day per IP, at a 1.4 second average.

The attack is a cookie farm with a nine-day lead time.

Level two on that mechanism. It works because the risk signal that dominates the checkbox decision is longitudinal, and longitudinal signals can be manufactured by anyone willing to wait. Cost per aged identity approaches zero at volume. Patience is not a scarce resource for an automated system.

Level three, why the design tolerates that. A cross-site history signal is the only thing that lets a provider silently pass the large majority of legitimate returning users, which is the entire commercial promise of the product. Making history harder to accumulate would break the good path before it broke the bad one.

That tradeoff is baked in. It is not a bug anyone is going to fix.

The 2024 escalation is worth its own paragraph.

Plesner, Vontobel and Wattenhofer trained YOLOv8 segmentation and classification models on the object categories reCAPTCHAv2 draws from. They report solving 100% of the challenges presented, against 68 to 71% for prior published work.

The mechanism behind the jump to 100% is not model quality alone. It is retry. Their testing showed the model did not need to be right on the first image set, because reCAPTCHAv2 permits multiple attempts. A model that fails the first puzzle passes the second.

Worked example: assume a per-challenge success probability of 70% and three permitted attempts. The probability of eventual success is one minus 0.3 cubed, which is 97.3%. At four attempts it is 99.2%.

Retry allowance converts a mediocre solver into a reliable one, and retry allowance exists because humans need it too.

Named victim one of getting this wrong: the older user. Searles and colleagues measured an average increase of 0.09 seconds of solve time per year of participant age across all CAPTCHA types, roughly triple the 0.03 seconds per year that Elie Bursztein and colleagues reported at IEEE S&P in 2010. Whatever you set as a timeout, it lands on them first.

Named victim two: the first-time visitor. In the UC Irvine deployment, first-attempt checkbox solving averaged 2.02 seconds against 1.56 by the tenth attempt, a 35% improvement from familiarity alone. Freshmen were 80% slower on total solve time than seniors. Your slowest users are the ones who have never seen your form.

Named victim three: the phone user. Searles and colleagues found statistically significant differences between participants on computers and participants on phones for distorted text (p < 0.02) and for the harder click-based reCAPTCHA setting (p < 0.01), with computer users faster in both. They found the same split between physical keyboard and touch input across five of the twelve challenge types tested.

Mobile is not a slightly worse version of desktop on this control. It is a measurably different population, and the puzzle types were designed on a desktop.

Counter-case, adjudicated. Bursztein and colleagues in 2010 found participants with PhDs solved faster than other educational groups. Searles and colleagues in 2023 found no correlation between self-reported education and solving time. The UC Irvine 2023 deployment found a clean freshman-to-senior gradient, which they attribute to exposure rather than education.

I side with exposure. The 2023 attempt-count data shows the same person getting faster with repetition, which is a learning curve, not a credential.

What would change my mind: a study that controls for prior CAPTCHA exposure directly rather than using academic year as a proxy. Nobody has run it.

What to instrument: record time-to-solve per challenge and alert on the 95th percentile, not the mean. The mean is dominated by the silent checkbox path and will look healthy while your tail is on fire.

A solved CAPTCHA costs an attacker about a third of a cent

This is the section competitors skip, and it is the one that should decide your deployment.

Jin, Huang, Duan, Zhao, Liao and Zhou at USTC surveyed five solving services in June 2023 and published the price list. Per thousand solved challenges:

Those are the services' own published rates, which makes them an interested party's numbers. I use them anyway because a price list is a commitment, and because the 2captcha and Anti-Captcha figures I checked in July 2026 sit in the same band.

Compare against Marti Motoyama, Kirill Levchenko, Chris Kanich, Damon McCoy, Geoffrey Voelker and Stefan Savage, who measured this market at USENIX Security 2010 and found retail prices reaching roughly $1 per thousand.

Sixteen years. The floor has not moved.

That is the finding. Not that solving is cheap, which everyone knows, but that a decade and a half of CAPTCHA evolution produced no durable price increase for the attacker.

Now the labor side, which explains why.

2captcha publishes its worker rates. Per the USTC survey, workers earn $0.26 per thousand correctly recognized text challenges and $1.00 per thousand image challenges. The retail markup on text solving runs roughly 2x to 5x.

Level two on that mechanism. The margin is thin because the labor supply is elastic and the skill floor is zero. CAPTCHAs are designed to be solvable by any human, which by construction makes the labor market for solving them global and undifferentiated.

The design goal and the exploit are the same property.

Level three, why nothing fixes this. Raising difficulty raises the solver's cost and your user's cost at close to the same rate, because both parties are humans performing the identical task. The USTC team's hCaptcha difficulty result, where harder settings barely moved solver performance while Searles measured a 10-point drop in human pass rate, suggests the ratio is actually worse than one-to-one.

You cannot win a race where the other runner is your own customer.

Worked example, and this is the calculation I would put in front of a product owner.

Suppose you are defending account creation and you believe an abusive account is worth $4 to whoever creates it. Assume a solving service at $2.99 per thousand and a 90% success rate, which is roughly what the USTC team measured across most provider-solver pairs.

Cost per successfully solved challenge: $2.99 divided by 1,000, divided by 0.9. That is $0.0033.

As a share of the $4 target: 0.08%.

Add residential proxy cost and phone verification and the total acquisition cost still lands well under a dollar. The challenge contributes a rounding error.

Second worked example, at the other end of the value scale.

Now suppose you are defending a comment form where an abusive post is worth $0.005 to a spammer, roughly the value of a low-quality backlink placement. Same $0.0033 per solve.

That is 66% of the target value, and the attack stops being worth running.

There is your deployment rule, and it does not depend on anyone's product.

A CAPTCHA challenge is worth deploying when the cost to solve it is a meaningful fraction of what the protected action is worth to an attacker. Below roughly a dollar of attacker value, it works. Above ten dollars, it is decoration.

Everything between those numbers is where you actually have to think.

The FunCaptcha row deserves a note. At $2.00 to $3.99 per thousand it is the most expensive type on the list, and the USTC team recorded "Unsolvable" responses from three of five services when they tested it against a live Outlook registration page. Their read was that Arkose's challenges require logical and spatial reasoning that resists automated solving, which pushes attackers onto the more expensive human path.

Higher solve cost is a real and measurable defensive property. It is also, as far as I can tell, the only one that has moved in fifteen years.

Named victim of the economics: the legitimate high-value user. If you deploy a challenge on a $500 transaction because the value is high, you have inverted the logic. High protected value means high attacker willingness to pay, which means the challenge is cheapest to defeat exactly where you needed it most, and the friction lands on customers who are about to spend $500.

Second named victim: the small merchant paying per-challenge on an enterprise CAPTCHA tier. Your cost per challenge served can exceed the attacker's cost per challenge solved, at which point the control is a net transfer of money from you to your vendor.

Counter-case, adjudicated. Cloudflare's position, stated by Tatoris and Wolters in April 2022, is that solving services do not make CAPTCHA ineffective, because most abusive traffic is not sophisticated enough to route through one. That is probably true of commodity scraping and volumetric junk.

It is not true of anything with a monetizable target, which is the traffic fraud teams care about. Both things can hold at once, and the split is the attacker's revenue per attempt.

What to instrument: compute your own version of the ratio. Attacker value per successful action, divided by market solve price for your challenge type. If that number is above about 300, the challenge is not your control.

Agents solve CAPTCHAs at 40%, and that contradiction is the most useful result in the field

Two well-conducted studies published a year apart reached opposite conclusions, and the reason they disagree is the most operationally useful result in the field.

ETH Zurich, September 2024: 100% of reCAPTCHAv2 challenges solved.

MBZUAI and collaborators, presented at NeurIPS 2025: the best multimodal agent tested reached 40.0%, against 93.3% for humans.

Both numbers are real. They are measuring different things, and the gap between them is where defenders still have an advantage.

Here is the reconciliation.

The Open CaptchaWorld benchmark spans 20 modern CAPTCHA types, 225 challenges at first release and 463 after the July 2025 expansion. Every puzzle runs in a real browser loop where the agent must perceive a screenshot and issue clicks or keystrokes until the task completes.

They also introduced a metric called CAPTCHA Reasoning Depth, counting the cognitive and motor steps a puzzle requires. Some of their tasks need four to six atomic steps involving path planning, sequential clicks, position verification, and submission.

The results by model: OpenAI's o3 driving Browser-Use at 40.0%. GPT-4.1 and Gemini 2.5 Pro at 25.0%. Claude and GPT-4o variants between 5% and 20%, with some near random on the deeper tasks.

The failure modes they catalogued were overthinking and interface misunderstanding. Not perception failure.

Compare the ETH setup. Plesner and colleagues trained purpose-built YOLOv8 models on the specific object categories reCAPTCHAv2 draws from, driven through Selenium, attacking one known challenge type with unlimited retries.

That is a specialist solver against a fixed target. The benchmark is a generalist agent against twenty moving ones.

Level two on why the gap exists. Single-shot visual classification is solved. Closed-loop interaction under a novel interface is not. The bottleneck for the agent is not seeing the image. It is maintaining state across a multi-step motor sequence in a UI it has never encountered.

Level three, why that gap is structural rather than temporary. Purpose-built solvers require per-target engineering, which is exactly the cost the CAPTCHA-as-a-service model was designed to impose. When a provider rotates challenge types, the specialist model needs retraining and the generalist agent needs nothing.

So the two curves are moving toward each other from opposite directions. Specialist solvers get cheaper to build. Generalist agents get better at interaction.

They have not met yet. That is the window.

Worked example of what this means in practice. If your challenge type is stable, well-known, and image-classification-based, assume a solve rate near 100% at commodity prices and stop counting it as a control.

If your challenge requires multi-step interaction with a rotating interface, the published generalist agent performance sits between 5% and 40%, and the human-farm path costs you between $0.60 and $3.99 per thousand instead.

That second number is still cheap. But it is the difference between free and metered, and metered is a control.

Now the part that complicates the defensive read, and it complicates it badly.

The same interaction gap that stops abusive agents stops legitimate ones. Open CaptchaWorld's authors describe CAPTCHAs as a critical bottleneck for deploying web agents, and they built the benchmark precisely because other agent benchmarks filter CAPTCHAs out.

Your customers are increasingly arriving through agents you would want to serve. A booking assistant, a price comparison tool the user authorized, an accessibility agent operating on someone's behalf.

Which is the whole reason the industry is building an identity layer instead.

Thibault Meunier and Mari Galicer at Cloudflare proposed Web Bot Auth at the IETF in May 2025, built as a profile on top of RFC 9421 HTTP Message Signatures. Agents sign their requests with a private key and publish the public key at a well-known directory path. The origin verifies the signature.

Cloudflare's verified-bots team put the problem with the status quo in one line in July 2025: existing identification relies on IP ranges that get shared or rotated and user-agent headers that are trivially spoofed.

By August 2025 Cloudflare had launched a Signed Agents classification and was publishing signed-agent traffic patterns on Radar. Registry work continued through October 2025 with a Signature Agent Card draft co-authored with Amazon.

That is a real standards effort with real deployment behind it. It is also incomplete, and the incomplete part matters: a signature proves which agent sent the request, not whether the human it claims to act for exists.

Named victim one of getting this wrong: the blind user whose screen-reader-adjacent automation trips the interaction check. Multi-step motor puzzles are the hardest possible case for assistive technology.

Named victim two: the merchant whose authorized comparison-shopping integration gets challenged into failure, generating a support ticket that reads as a site outage.

Counter-case, adjudicated. You could argue Open CaptchaWorld understates agent capability because it tests zero-shot through a generic Browser-Use agent rather than a tuned adversarial pipeline. That is a fair objection and I think it is correct.

Which means 40% is a floor, not a ceiling, and the window I described above is closing at an unknown rate. Plan accordingly.

What would change my mind: a published agent result above 80% on Open CaptchaWorld's expanded 463-challenge set. At that point interaction depth stops being a defensive property and the identity layer is the only remaining answer.

What to instrument: start logging whether inbound requests carry an RFC 9421 signature, before you have any policy for it. You want the baseline now.

Your challenge's false-positive cost is a conversion number, and it is bigger than you think

Every section above concerns what the control catches. This one concerns what it breaks, which is where the published evidence is strongest and where operator instrumentation is weakest.

Start with abandonment.

Searles and colleagues ran a dedicated abandonment study across four groups of 100 participants in 2023. Of 574 who started, 174 quit before finishing. A 30% abandonment rate.

Broken out by condition, abandonment ranged from 18% to 45%.

The critical split: participants in the contextualized setting, who believed they were creating an account rather than participating in a CAPTCHA study, were 120% more likely to abandon than those in the direct setting.

In that contextualized group, nearly 50% of everyone who eventually quit did so before solving the first challenge.

Read that again. Half the losses happened at first sight of the puzzle, before any solving difficulty could have caused them.

That is not a usability problem. That is a bounce.

Level two on why context drives abandonment. The direct-setting participants had been told solving CAPTCHAs was the task, so friction was the job. The contextualized participants had a goal, and the challenge was an obstacle between them and it. Same puzzle, different cost basis.

Your users are always in the contextualized condition. Every study that recruits people to solve CAPTCHAs is measuring your best case.

Level three, why this is invisible in most stacks. Abandonment at the challenge produces no server-side event. The user never submits. Your funnel shows a drop between page view and form submission, and the challenge sits inside that gap unattributed.

If you have never instrumented challenge-render-to-token-received as its own step, your CAPTCHA's cost does not appear anywhere in your analytics. It appears in your conversion rate, undifferentiated.

Worked example, using the UC Irvine escalation rate against a plausible funnel.

Take 100,000 monthly signup attempts. Apply the observed 20% image-escalation rate and you get 20,000 users routed to a puzzle averaging 10.3 seconds.

Now apply Cloudflare's measured abandonment differential. Their April 2022 telemetry found visitors with human properties were 31% less likely to abandon a Managed Challenge than a traditional CAPTCHA action. Whatever your baseline abandonment on the puzzle path, roughly a third of it is attributable to the puzzle specifically rather than to the form.

If puzzle-path abandonment runs at 10%, that is 2,000 lost signups a month. The 143 image failures and 19 hard blocks in the UC Irvine data are the part you can see. The abandonment is roughly an order of magnitude larger and completely silent.

Second worked example, the accessibility population.

The W3C Accessible Platform Architectures Working Group has maintained a document titled "Inaccessibility of CAPTCHA: Alternatives to Visual Turing Tests on the Web" since 2005, most recently updated as a Working Draft in December 2021. Its central finding is that every type of CAPTCHA will be unsolvable by users with certain disabilities.

Not difficult. Unsolvable.

WCAG 2.1 Success Criterion 1.1.1 handles this by requiring a text alternative describing the purpose plus a second CAPTCHA in a different sensory modality. The WCAG Working Group's own published rationale is that they structured it this way because forbidding CAPTCHA outright would cause sites to abandon WCAG conformance rather than abandon CAPTCHA.

That is a standards body openly documenting a compromise it does not like.

And the standard modality alternative is broken. Kevin Bock, Daven Patel, George Hughey and Dave Levin published unCaptcha at USENIX WOOT in 2017, defeating reCAPTCHA's audio challenge at 85.15% with a 5.42 second average, using off-the-shelf speech recognition.

So the accommodation path is both the least secure and the most necessary path in the system.

Named victim one: the blind user routed to audio. They receive the weakest control in your stack, and everyone knows it.

Named victim two: the user on a shared corporate NAT or a carrier-grade NAT mobile connection. Shared egress inflates apparent request volume from a single address, which pushes exactly the risk signals that trigger escalation.

Named victim three: the Tor user. Searles and colleagues visited all 200 inspected sites twice, once in Chrome incognito and once over Tor, specifically because anecdotal reports suggested Tor users face more frequent and harder challenges. Privacy tooling and automation produce the same signature.

Named victim four: the non-English speaker. Bursztein and colleagues showed in 2010 that participants outside the US took longer on English-language schemes. Text challenges have never stopped being language tests.

Counter-case, adjudicated. Google's position, given to TechRadar in February 2025 in response to the UC Irvine cost analysis, is that reCAPTCHA is a security product, that user data is used only to improve the service, and that the majority of their user base has moved to reCAPTCHA v3 where no visual challenge is served.

The v3 point is the substantive one and it is fair. If most deployments are scoring-only, the visible-puzzle costs I have described apply to a shrinking share of traffic.

I would still push back on two grounds. BuiltWith's crawl of 673 million sites, cited in the 2023 USENIX paper, found 15.2 million using CAPTCHAs with reCAPTCHA at 97.3% of those, and the UC Irvine inspection of the top 200 sites in 2023 still found visible challenges on 60% of the sites that had them. And scoring-only deployment does not remove the cost, it relocates it, because a low score still has to route somewhere.

What would change my mind: published deployment data on the v2-to-v3 split. Neither Google nor BuiltWith breaks it out, and without it both my read and Google's are inference.

What to instrument: add a client-side timer from challenge render to token receipt, and fire an event on unload if no token arrived. That single event is the difference between knowing your false-positive cost and guessing at it.

What doesn't work

Five approaches that look reasonable, have been measured, and fail. Each of these still ships in production somewhere.

Distorted text challenges. Defeated at 99.8% by Goodfellow and colleagues in 2014 using a convolutional network built for Street View house numbers. Yang Zi, Haichang Gao and colleagues published an end-to-end attack in IEEE TIFS in 2020 reaching over 97% in fractions of a second.

Human performance on the same task, from the 2023 USENIX study: 50% to 84% case-sensitive agreement, rising to 73% to 93% case-insensitive.

Your users are worse at this than the machines by a wide margin, and have been for over a decade.

It looked plausible because the failure is invisible from the inside. A site running text CAPTCHA that sees no spam concludes the control works, when what it actually has is no attacker. Absence of abuse is not evidence of defence, and text CAPTCHA is the purest example of that confusion in the field.

Text challenges still appeared on 14 of the top 200 websites in the 2023 inspection, and the retail price to solve one sits at the bottom of the market at $0.40 to $1.39 per thousand.

That is the cheapest row on the price list defending, in most cases, the same actions as the expensive rows.

Audio fallback as an accessibility accommodation. Broken at 85.15% by Bock and colleagues at WOOT 2017, in 5.42 seconds, using Google's own speech recognition against Google's own challenge.

It looked plausible because it satisfies WCAG's two-modality requirement. It fails because speech recognition improved faster than audio obfuscation could, and because obfuscating audio past machine recognition also obfuscates it past human recognition. Jonathan Bigham and Anna Cavender measured human audio CAPTCHA success at 39% to 43% in their CHI 2009 study.

You are shipping a control that machines pass roughly twice as often as the people it exists to serve.

And you cannot simply remove it. WCAG 2.1 SC 1.1.1 requires a second modality, so deleting the audio path costs you conformance while keeping it costs you the weakest link in the stack.

This is the one dead end with no good exit inside the CAPTCHA paradigm, which is a large part of why the W3C working group's own recommendation is to prefer non-interactive approaches wherever they are feasible.

Raising the difficulty setting. Covered above and worth restating as a dead end in its own right. The USTC 2023 experiment found hCaptcha's difficulty setting had little impact on solver response time or success rate. The USENIX 2023 human study found human pass rates fell from 81.4% to 70.6% across the same two settings.

Worse, the difficulty dial does not even reach the cheapest attackers. The USTC team concluded from response time, variance and price that CapSolver runs fully automated with no human workers, and measured it above 80% against most providers tested and above 90% against text.

Fully automated solving undercuts the human-farm price and is unaffected by how tired or well-paid a worker is.

It looked plausible because difficulty is the one dial the product gives you. It fails because you and the attacker are testing the same species, and increasingly the attacker is not even using humans.

Client-side-only verification. Egor Homakov documented in December 2014 that the g-recaptcha-response token could be harvested through clickjacking, letting an attacker collect valid tokens from trusted users and replay them.

The general failure is structural: a token generated on a client you do not control is a bearer credential. If you are not calling the provider's server-side verification endpoint with your secret key, and checking the returned hostname and action fields, you are checking that a token exists rather than that it is yours.

It looked plausible because the widget renders and turns green. Green is a client-side event.

The verification response carries the fields that make replay detectable: the hostname the token was issued for, the action name it was bound to, and the challenge timestamp. Most integrations check the success boolean and ignore all three.

Treating a low reCAPTCHA v3 score as a block signal. Ismail Akrout, Amal Feriani and Mohamed Akrout published a reinforcement learning attack on reCAPTCHA v3's behavioral scoring in 2019, reaching scores above 0.9 with a 97% success rate and needing only 2,000 data points to train.

Two thousand samples. That is a weekend.

Google's own documentation, worth quoting for how carefully it is worded, recommends taking action in the background rather than blocking traffic, and says to start with a threshold of 0.5 and tune from your own traffic in the admin console.

The vendor is telling you not to use it as a gate. A great many deployments use it as a gate anyway, because a float between zero and one looks like it means something absolute.

It does not. It means something relative to your site's traffic, which is why the same documentation warns that staging scores and scores within seven days of implementation differ from long-run production values.

Allowlisting your own automation by IP or user-agent. Every team that puts a challenge on a form eventually discovers their own integration tests, uptime monitors and partner integrations failing it. The standard fix is an allowlist keyed on egress IP or on a custom user-agent string.

Cloudflare's verified-bots team stated the problem with that in July 2025: identification built on IP ranges and user-agent headers fails because ranges get shared between services or change over time, and headers are trivially spoofed.

An allowlisted static egress IP is an unauthenticated bypass with a long life and no audit trail. A custom user-agent is a password you publish in every request.

It looked plausible because it is the fastest way to unbreak CI on a Friday. It fails because you have created exactly the thing your challenge exists to prevent, and pointed it at the same endpoint.

The replacement is the signature work described earlier, which is why RFC 9421 exists.

Sidenote. On the free tier, reCAPTCHA v3 exposes a reduced set of discrete score levels rather than the full continuous range. If you are threshold-tuning on free-tier scores, you have fewer buckets than you think you have. Verify this against current Google documentation before designing around it.

Where the challenge actually belongs

Here is my verdict, and then the four things to do with it.

A CAPTCHA challenge is a step-up control with a metered cost to the attacker and an unmetered cost to your users. Deploy it as a response to a risk signal, never as a gate on a flow. If it fires on more than a small single-digit percentage of your traffic, it is not a control, it is a tax.

Cloudflare arrived at the same conclusion from a much larger dataset and acted on it harder than anyone else has.

Their April 2022 numbers: deferring the visual-puzzle decision until more browser information was available cut CAPTCHAs served by 91% over one year. Average time in a challenge fell from 32 seconds to roughly one second. Visitors with human properties became 31% less likely to abandon.

By September 2023 they had removed visual puzzles from their own network entirely and made Turnstile's Managed mode free for unlimited use.

The mechanism is worth naming precisely, because it is copyable. They did not build a better puzzle. They moved the decision point later, ran a rotating battery of non-interactive checks first, and reserved interaction for the residual cases where the signal stayed ambiguous.

That is a scoring architecture with a challenge attached to the uncertain tail. Which is what your deployment should be.

Four operational moves, in the order I would do them.

One. Measure your escalation rate before you change anything. In the UC Irvine deployment, 20% of a fully legitimate population was escalated to a visible puzzle. If yours is higher than that, your risk signal is weak and the challenge is absorbing the uncertainty. If it is near zero, ask whether the control is doing anything at all.

Two. Instrument challenge abandonment as a distinct funnel step. Render timestamp, token timestamp, unload without token. Without this event you cannot compute your false-positive cost, and every argument about whether the challenge is worth keeping stays an opinion.

Three. Run the economics ratio. Attacker value per successful action against market solve price for your challenge type. Published 2023 prices run $0.60 to $3.99 per thousand depending on type, with FunCaptcha at the top of that range and text at the bottom. If your protected action is worth more than about a dollar to an attacker, plan for the challenge to be paid past and put your real detection elsewhere.

Four. Start logging RFC 9421 signatures now. The Web Bot Auth work has Cloudflare, OpenAI and Anthropic publishing keys and Cloudflare running a Signed Agents classification since August 2025. You do not need a policy yet. You need a baseline, because the population you are currently challenging is about to split into agents that can identify themselves and agents that cannot, and that split is a far better feature than any puzzle result.

The mental model to keep: the challenge is not asking a question. It is charging a toll, and watching how the payer behaves at the booth.

Re-check this in six months. The number that moves first is the agent benchmark.

Open questions

Things I could not answer from the published record, stated honestly.

What is the actual v2-to-v3 deployment split? Google says most of its base has moved to scoring. BuiltWith does not break the versions out. Every claim about how much visible-puzzle friction still exists on the web, including mine, is inference.

Does escalation rate correlate with fraud rate at all? I found no published study relating a site's image-challenge escalation rate to its measured abuse rate. That is the single most useful missing number in this field.

How fast is the generalist agent curve moving? Open CaptchaWorld reported 40.0% as the ceiling in 2025 under zero-shot conditions. Nobody has published a tuned adversarial pipeline against the expanded 463-challenge set. Until somebody does, the interaction gap is an unmeasured asset.

What does a signed-agent world do to challenge volume? If a meaningful share of automated traffic starts identifying cryptographically, the residual unsigned population changes composition. Nobody has modelled what that does to a challenge threshold tuned on today's traffic.