Testing CAPTCHA-Protected Sign-In Across 7 Auth Providers

To end-to-end test a login or sign-up page with a CAPTCHA switched on, run CI against the provider's own test mode: Cloudflare and Google test keys, Clerk Testing Tokens, the Firebase Auth emulator, or an Auth0 IP AllowList. Keep the real CAPTCHA for one staging job and a low-frequency production login monitor. That is where CaptchaAI fits, provided the widget is a type it solves (Turnstile and reCAPTCHA v2, v3 and Enterprise are GA; Friendly Captcha and CaptchaFox are beta) and the token reaches your server through a field or parameter your test controls. For several providers below, the honest answer is to stay on the test mode.

Scope: your own tenant, synthetic accounts, authorized testing

Scope. This guide is for teams testing or monitoring sign-in and sign-up on their own application, in a tenant, project or realm they administer, with dedicated synthetic test accounts: authorized QA, CI and uptime monitoring. It is not for signing in to accounts you do not own, creating accounts in bulk, or testing credentials against another service.

Mark every synthetic account (a +e2e subaddress or a reserved test number), delete what registration tests create, and scope the CaptchaAI key to the staging and monitor jobs only.

Provider matrix: CAPTCHA options, test paths and CaptchaAI fit

CI first: provider test keys and testing tokens

Most sign-in suites never need a real solve. Cloudflare's dummy Turnstile keys work on any hostname, including localhost, and come in matched pairs: a test secret accepts only the dummy token, and a production secret rejects it, so switch both together.

The always-pass site key fills cf-turnstile-response with the dummy token XXXX.DUMMY.TOKEN.XXXX, so a browser test only waits for that field to be non-empty. API-level tests can send that string directly, since the test secret accepts nothing else.

For reCAPTCHA v2, Google's test pair is site key 6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI and secret 6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe. The widget shows a warning banner, the checkbox still has to be ticked but never opens a challenge, and every verification passes. Google has no v3 test key, so create a separate v3 key for test environments. hCaptcha publishes test keys too (site key 10000000-ffff-ffff-ffff-000000000001, secret 0x0000000000000000000000000000000000000000), which matters because CaptchaAI does not solve hCaptcha.

Drive the switch from the environment, so one build runs in both jobs:

jobs:
  e2e-ci:
    runs-on: ubuntu-latest
    env:
      TURNSTILE_SITE_KEY: 1x00000000000000000000AA
      TURNSTILE_SECRET_KEY: 1x0000000000000000000000000000000AA
      RECAPTCHA_SITE_KEY: 6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI
      RECAPTCHA_SECRET_KEY: 6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npx playwright install --with-deps chromium
      - run: npx playwright test --grep-invert @real-captcha
  e2e-staging:
    runs-on: ubuntu-latest
    environment: staging
    env:
      TURNSTILE_SITE_KEY: ${{ vars.TURNSTILE_SITE_KEY }}
      TURNSTILE_SECRET_KEY: ${{ secrets.TURNSTILE_SECRET_KEY }}
      CAPTCHAAI_API_KEY: ${{ secrets.CAPTCHAAI_API_KEY }}
      SYNTHETIC_USER_EMAIL: ${{ vars.SYNTHETIC_USER_EMAIL }}
      SYNTHETIC_USER_PASSWORD: ${{ secrets.SYNTHETIC_USER_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npx playwright install --with-deps chromium
      - run: npx playwright test --grep @real-captcha

The failure worth guarding is the reverse, a test secret reaching production and waving every request through:

// Refuse to start production with a vendor test key anywhere in the environment.
const VENDOR_TEST_KEYS = [
  "1x0000000000000000000000000000000AA", // Turnstile, always passes
  "6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe", // reCAPTCHA v2 test secret
  "0x0000000000000000000000000000000000000000", // hCaptcha test secret
];
if (process.env.NODE_ENV === "production") {
  const leaked = Object.entries(process.env).filter(([, v]) => v && VENDOR_TEST_KEYS.includes(v));
  if (leaked.length > 0) {
    throw new Error(`test CAPTCHA keys in production: ${leaked.map(([k]) => k).join(", ")}`);
  }
}

The wider pattern is in CAPTCHA handling in continuous integration testing.

A shared Playwright helper: read the sitekey, solve, fill the right field

The staging and monitor jobs share one helper. It reads data-sitekey, submits method=turnstile or method=userrecaptcha to https://ocr.captchaai.com/in.php with json=1, waits 15 seconds, then polls res.php every 5 seconds for up to 120 seconds. CAPCHA_NOT_READY means still working; any other status: 0 is an error. The token goes into cf-turnstile-response, g-recaptcha-response, or a field name you pass.

// captchaai.ts: shared by the provider sections below.
import type { Page } from "@playwright/test";

type Task = { method: "turnstile"; sitekey: string } | { method: "userrecaptcha"; googlekey: string };
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function call(path: string, params: Record<string, string>): Promise<{ status: number; request: string }> {
  const res = await fetch(`https://ocr.captchaai.com/${path}?${new URLSearchParams({ ...params, json: "1" })}`);
  const text = await res.text();
  try {
    return JSON.parse(text);
  } catch {
    throw new Error(`${path} returned non-JSON: ${text.slice(0, 80)}`); // e.g. a plain-text ERROR_ code
  }
}

export async function solveWithCaptchaAI(task: Task & { pageurl: string }): Promise<string> {
  const key = process.env.CAPTCHAAI_API_KEY;
  if (!key) throw new Error("CAPTCHAAI_API_KEY is not set");
  const submit = await call("in.php", { key, ...task });
  if (submit.status !== 1) throw new Error(`in.php: ${submit.request}`);
  const deadline = Date.now() + 120_000;
  await sleep(15_000);
  while (Date.now() < deadline) {
    const poll = await call("res.php", { key, action: "get", id: String(submit.request) });
    if (poll.status === 1) return poll.request;
    if (poll.request !== "CAPCHA_NOT_READY") throw new Error(`res.php: ${poll.request}`);
    await sleep(5_000);
  }
  throw new Error("no CaptchaAI result within 120 s");
}

export async function solveCaptchaOnPage(page: Page, field?: string): Promise<string> {
  const widget = page.locator(".cf-turnstile[data-sitekey], .g-recaptcha[data-sitekey]").first();
  const sitekey = await widget.getAttribute("data-sitekey", { timeout: 10_000 });
  if (!sitekey) throw new Error("no Turnstile or reCAPTCHA v2 widget with data-sitekey on this page");
  const turnstile = await widget.evaluate((el) => el.classList.contains("cf-turnstile"));
  const pageurl = page.url(); // the page that hosts the widget, not the app that redirected here
  const token = await solveWithCaptchaAI(
    turnstile ? { method: "turnstile", sitekey, pageurl } : { method: "userrecaptcha", googlekey: sitekey, pageurl },
  );
  const name = field ?? (turnstile ? "cf-turnstile-response" : "g-recaptcha-response");
  await page.evaluate(({ n, t }) => {
    document.querySelectorAll<HTMLInputElement | HTMLTextAreaElement>(`[name="${n}"]`).forEach((el) => { el.value = t; });
  }, { n: name, t: token });
  return token;
}

Call it after every other field is filled, and submit straight away: a Turnstile token lives 300 seconds, a reCAPTCHA token two minutes, and both are single-use. Turnstile typically solves in under 10 seconds and reCAPTCHA v2 in under 60. Playwright's default 30-second test timeout is shorter than the helper's 120-second cap, so every @real-captcha test calls test.setTimeout(180_000). The submit-and-poll contract is explained in how to solve Cloudflare Turnstile using the API, and the Cypress E2E guide ports it to Cypress.

Three limits apply. The helper only works when the server reads the token from the posted form; single-page apps that keep it in component state from the widget callback need an API-level test instead (Better Auth and Supabase below). An invisible v2 widget (data-size="invisible") also needs invisible=1 on the submit. And reCAPTCHA Enterprise needs enterprise=1 and returns the token in result with a user_agent you must reuse, so extend call() before pointing it at an Enterprise widget.

Auth0 Bot Detection and a pre-user-registration Action that verifies Turnstile

Auth0's protection lives under Dashboard > Security > Attack Protection > Bot Detection. "Require a CAPTCHA" takes Never, When Risky (with a Low, Medium or High Bot Detection Level) or Always, and the CAPTCHA Providers list offers Auth Challenge (the default), Simple CAPTCHA, reCAPTCHA Enterprise, hCaptcha, Friendly Captcha and Arkose.

Do not solve Auth0's own challenge in tests. Set Require a CAPTCHA to Never in a test tenant, or add your runners' egress IPs to the IP AllowList (up to 100 addresses or CIDR ranges), which suits self-hosted or static-egress runners better than GitHub-hosted ones with changing IPs. Auth0 renders and checks Auth Challenge and Simple CAPTCHA itself and documents no token field for them. CaptchaAI does not solve hCaptcha or Arkose, and Auth0 documents no way to hand Universal Login an externally solved reCAPTCHA Enterprise token, so keep Auth0's own challenge on the Never or AllowList path.

The clean CaptchaAI case is a Turnstile widget you add yourself. Prompt partials, which need a Custom Domain and a Custom Page Template, inject HTML into the signup screens, for example at the form-content-end insertion point, and any input whose name starts with ulp- reaches event.request.body in the pre-user-registration Action. Turnstile's data-response-field-name renames its hidden input to match:

<!-- Partial: load https://challenges.cloudflare.com/turnstile/v0/api.js once from the page template -->
<div class="ulp-field">
  <div class="cf-turnstile" data-sitekey="1x00000000000000000000AA" data-response-field-name="ulp-turnstile-token"></div>
</div>

The Action verifies the token with Cloudflare's siteverify and denies the sign-up on failure. Each tenant carries its own pair: the dummy site key above and the always-pass secret in the TURNSTILE_SECRET Action secret for the test tenant, real keys in staging:

exports.onExecutePreUserRegistration = async (event, api) => {
  const token = event.request.body?.["ulp-turnstile-token"];
  if (!token) {
    api.access.deny("turnstile_missing", "Please complete the verification and try again.");
    return;
  }
  const res = await fetch("https://challenges.cloudflare.com/turnstile/v0/siteverify", {
    method: "POST",
    headers: { "Content-Type": "application/x-www-form-urlencoded" },
    body: new URLSearchParams({
      secret: event.secrets.TURNSTILE_SECRET,
      response: token,
      remoteip: event.request.ip,
    }),
  });
  const outcome = await res.json();
  if (!outcome.success) {
    const codes = (outcome["error-codes"] || []).join(",");
    api.access.deny(`turnstile_failed:${codes}`, "Verification failed. Please try again.");
  }
};

An Action runs on Auth0's servers after the form posts, so it can verify a token but never render or solve a widget. In staging, the browser test calls solveCaptchaOnPage(page, "ulp-turnstile-token") right before it clicks Continue, and a missing token shows up in the tenant logs as turnstile_missing rather than as a vague sign-up failure.

Auth.js (NextAuth) Credentials provider: verifying the token in authorize()

Auth.js v5 calls authorize(credentials, request) with the posted fields and the original Request, and returning null fails the sign-in. A Turnstile widget inside your custom sign-in form posts cf-turnstile-response alongside the email and password, so authorize() can verify it first:

// auth.ts (next-auth v5)
import NextAuth, { CredentialsSignin } from "next-auth";
import Credentials from "next-auth/providers/credentials";
import { lookupUser } from "@/lib/users"; // your own: validates the unknown inputs, returns a User or null

class CaptchaFailed extends CredentialsSignin {
  code = "captcha"; // the sign-in redirect carries ?code=captcha
}

async function turnstileOk(token: unknown, ip: string | null): Promise<boolean> {
  if (typeof token !== "string" || token.length === 0) return false;
  const body = new URLSearchParams({ secret: process.env.TURNSTILE_SECRET_KEY ?? "", response: token });
  if (ip) body.set("remoteip", ip);
  const res = await fetch("https://challenges.cloudflare.com/turnstile/v0/siteverify", { method: "POST", body });
  const outcome = (await res.json()) as { success: boolean };
  return outcome.success === true;
}

export const { handlers, signIn, signOut, auth } = NextAuth({
  providers: [
    Credentials({
      credentials: {
        email: { type: "email", label: "Email" },
        password: { type: "password", label: "Password" },
        "cf-turnstile-response": { type: "hidden" },
      },
      async authorize(credentials, request) {
        const ip = request.headers.get("x-forwarded-for")?.split(",")[0]?.trim() ?? null;
        if (!(await turnstileOk(credentials["cf-turnstile-response"], ip))) throw new CaptchaFailed();
        return lookupUser(credentials.email, credentials.password); // null: wrong email or password
      },
    }),
  ],
});

Returning null redirects with code=credentials, so a rejected token would look exactly like a wrong password. The CredentialsSignin subclass gives it its own code, and a staging test can assert on code=captcha versus code=credentials. The token is an ordinary form field, so the shared helper works unchanged; server-side token validation covers hostname and action checks.

Better Auth: the captcha plugin and the x-captcha-response header

Better Auth's captcha plugin verifies tokens as middleware. Its provider is cloudflare-turnstile, google-recaptcha, hcaptcha, captchafox or vercel-botid. It guards /sign-up/email, /sign-in/email and /request-password-reset by default, and clients send the token in an x-captcha-response header.

// auth.ts
import { betterAuth } from "better-auth";
import { captcha } from "better-auth/plugins";

export const auth = betterAuth({
  plugins: [
    captcha({
      provider: "cloudflare-turnstile",
      secretKey: process.env.TURNSTILE_SECRET_KEY!, // CI: 1x0000000000000000000000000000000AA
    }),
  ],
});

Since your client code sets that header, a staging check can skip the widget and call the endpoint under the default /api/auth base path:

// better-auth-signin.spec.ts
import { test, expect } from "@playwright/test";
import { solveWithCaptchaAI } from "./captchaai";

test("sign-in accepts a real Turnstile token @real-captcha", async ({ request, baseURL }) => {
  test.setTimeout(180_000);
  const token = await solveWithCaptchaAI({
    method: "turnstile",
    sitekey: process.env.TURNSTILE_SITE_KEY!,
    pageurl: `${baseURL}/sign-in`,
  });
  const res = await request.post("/api/auth/sign-in/email", {
    headers: { "x-captcha-response": token, origin: baseURL! },
    data: { email: process.env.SYNTHETIC_USER_EMAIL, password: process.env.SYNTHETIC_USER_PASSWORD },
  });
  expect(res.ok()).toBeTruthy();
});

In CI, the same request with the header set to XXXX.DUMMY.TOKEN.XXXX against the always-pass test secret exercises the plugin wiring without any solve. For a google-recaptcha v2 key, send method: "userrecaptcha" with googlekey instead. CaptchaFox (beta) needs your own proxy on the solve (proxy use is off by default on CaptchaAI accounts, so ask support to enable it) and the returned user agent on the request. CaptchaAI does not solve hCaptcha, and BotID inspects the request rather than a token, so both stay on their vendors' test paths.

Clerk: bot protection, Testing Tokens and when CaptchaAI still helps

Clerk's Bot sign-up protection (Protect > Rules in the Dashboard) uses Cloudflare Turnstile, applies to sign-up, and challenges only clients it suspects are bots, which a scripted browser can easily be. Custom flows must render <div id="clerk-captcha" /> before signUp.create() runs, or Clerk falls back to an invisible widget that blocks suspected bots. A Server Action cannot help: it runs on the server, and the widget runs in the browser.

The supported route is Testing Tokens. Install @clerk/testing, set CLERK_PUBLISHABLE_KEY and CLERK_SECRET_KEY, and call clerkSetup() from a setup project that your test projects depend on, because a function-based globalSetup runs in another process and its environment never reaches the workers:

// global.setup.ts
import { clerkSetup } from "@clerk/testing/playwright";
import { test as setup } from "@playwright/test";

setup.describe.configure({ mode: "serial" });

setup("global setup", async ({}) => {
  await clerkSetup();
});

// sign-up.spec.ts
import { setupClerkTestingToken } from "@clerk/testing/playwright";
import { test } from "@playwright/test";

test("synthetic sign-up", async ({ page }) => {
  await setupClerkTestingToken({ page });
  await page.goto("/sign-up");
  // Fill the fields your instance requires; use a +clerk_test address and code 424242.
});

Testing Tokens are short-lived, bound to one instance, and work on development and production instances, though in production the helpers do not support code-based sign-in, so use email and password there. Addresses with the +clerk_test subaddress verify with 424242 in test mode.

That leaves CaptchaAI little room. Clerk's bot protection covers sign-up, not sign-in, and Clerk documents no way to hand its SDK an externally solved token, so treat CaptchaAI as out of scope for Clerk unless your own staging trial shows otherwise.

Firebase Auth: RecaptchaVerifier, test phone numbers and the emulator

Phone sign-in on the web uses RecaptchaVerifier, whose v10+ signature is new RecaptchaVerifier(auth, containerOrId, parameters), with size: 'invisible' binding it to your submit button. On top of that, reCAPTCHA bot protection (reCAPTCHA Enterprise through Identity Platform) can guard email and password as well as phone flows, and in ENFORCE mode it rejects requests without a reCAPTCHA token; AUDIT only scores them. reCAPTCHA SMS defense adds toll-fraud scoring to SMS flows.

The SDK owns the verifier, so use Firebase's test paths. Add up to 10 fictional numbers with fixed codes under Phone numbers for testing, then either set appVerificationDisabledForTesting (a mock reCAPTCHA that accepts only those numbers) or connect to the Auth emulator, which runs no reCAPTCHA:

// phone-sign-in.js: one test switch, off by default.
import { getAuth, connectAuthEmulator, RecaptchaVerifier, signInWithPhoneNumber } from "firebase/auth";

export function createPhoneSignIn(app, { testMode = "off" } = {}) {
  const auth = getAuth(app);
  if (testMode === "emulator") {
    connectAuthEmulator(auth, "http://127.0.0.1:9099"); // the emulator does not run reCAPTCHA
  } else if (testMode === "fictional-numbers") {
    auth.settings.appVerificationDisabledForTesting = true; // set before the verifier renders
  }
  const verifier = new RecaptchaVerifier(auth, "sign-in-button", { size: "invisible" });
  return (phoneNumber) => signInWithPhoneNumber(auth, phoneNumber, verifier);
}

Against the emulator, the test reads the SMS code over its REST API:

// phone-sign-in.spec.ts: runs against `firebase emulators:start --only auth`
import { test, expect } from "@playwright/test";

const PHONE = "+16505553434";
const CODES_URL = `http://127.0.0.1:9099/emulator/v1/projects/${process.env.FIREBASE_PROJECT_ID}/verificationCodes`;

test("phone sign-in against the Auth emulator", async ({ page }) => {
  await page.goto("/login?auth=emulator"); // a test-only build maps this to testMode: "emulator"
  await page.getByLabel("Phone number").fill(PHONE);
  await page.getByRole("button", { name: "Send code" }).click();
  let code = "";
  await expect.poll(async () => {
    const body = (await (await fetch(CODES_URL)).json()) as { verificationCodes?: { phoneNumber: string; code: string }[] };
    code = body.verificationCodes?.filter((v) => v.phoneNumber === PHONE).at(-1)?.code ?? "";
    return code;
  }).not.toBe("");
  await page.getByLabel("Verification code").fill(code);
  await page.getByRole("button", { name: "Verify" }).click();
});

Keep fictional numbers out of production builds and rotate their codes: their ID tokens carry the same signature as a real phone user's.

Supabase Auth: captchaToken, Turnstile vs hCaptcha, and RLS after sign-in

Supabase Auth offers hCaptcha or Cloudflare Turnstile on sign-in, sign-up and password reset (Settings > Authentication > Bot and Abuse Protection), and the client passes options.captchaToken to signUp, signInWithPassword, signInWithOtp and similar calls. CaptchaAI does not solve hCaptcha, so only Turnstile projects have a CaptchaAI path.

Locally, the Supabase CLI reads supabase/config.toml; give CI the Turnstile test secret:

[auth.captcha]
enabled = true
provider = "turnstile"
secret = "env(TURNSTILE_SECRET_KEY)"

With the always-pass secret loaded, CI calls pass captchaToken: "XXXX.DUMMY.TOKEN.XXXX" and never render a widget. Supabase's examples keep the token in component state, so the staging check signs in through supabase-js and then proves row-level security scopes the rows:

// supabase-smoke.ts: staging, synthetic user, Turnstile provider (run with: npx tsx supabase-smoke.ts)
import { createClient } from "@supabase/supabase-js";
import { solveWithCaptchaAI } from "./captchaai";

// Publishable (or legacy anon) key only.
const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_PUBLISHABLE_KEY!);

const captchaToken = await solveWithCaptchaAI({
  method: "turnstile",
  sitekey: process.env.TURNSTILE_SITE_KEY!,
  pageurl: `${process.env.APP_URL}/login`,
});
const { data, error } = await supabase.auth.signInWithPassword({
  email: process.env.SYNTHETIC_USER_EMAIL!,
  password: process.env.SYNTHETIC_USER_PASSWORD!,
  options: { captchaToken },
});
if (error) throw new Error(`sign-in rejected: ${error.message}`);

// Requests now carry the user's JWT, so policies such as (select auth.uid()) = user_id apply.
const { data: rows, error: rlsError } = await supabase.from("projects").select("id, user_id");
if (rlsError) throw rlsError;
if (rows.some((r) => r.user_id !== data.user.id)) throw new Error("RLS leak: rows owned by another user");
console.log(`signed in as ${data.user.id}; RLS returned ${rows.length} own rows`);

Never swap in the secret key (sb_secret_, the successor to service_role) for a user-emulating test: it ignores row-level security, so a broken policy would pass.

Keycloak: the registration-flow reCAPTCHA step, realm headers and E2E tests

Keycloak adds reCAPTCHA to self-registration. Open Authentication > Flows > Registration, set reCAPTCHA to Required, and enter the site key and secret under its gear icon; a reCAPTCHA v3 toggle covers score-based keys, and a separate Enterprise step exists. Google's widget is framed, so Realm Settings > Security Defenses must allow https://www.google.com in the X-Frame-Options and Content-Security-Policy headers, or the widget never renders.

For the CI realm, kcadm.sh swaps in Google's v2 test pair. The step's provider ID is registration-recaptcha-action, and its config keys are site.key, secret.key and recaptcha.v3:

#!/usr/bin/env bash
# CI realm only: switch the registration reCAPTCHA step to Google's v2 test keys.
set -euo pipefail
KCADM="${KCADM:-/opt/keycloak/bin/kcadm.sh}"
REALM="${KC_REALM:-ci}"

"$KCADM" config credentials --server "$KC_URL" --realm master --user "$KC_ADMIN" --password "$KC_ADMIN_PASSWORD"

CONFIG_ID=$("$KCADM" get authentication/flows/registration/executions -r "$REALM" \
  | jq -r '.[] | select(.providerId == "registration-recaptcha-action") | .authenticationConfig // empty')
if [ -z "$CONFIG_ID" ]; then
  echo "No reCAPTCHA config in realm $REALM: save the step's gear-icon form once, then rerun." >&2
  exit 1
fi

TMP=$(mktemp)
"$KCADM" get "authentication/config/$CONFIG_ID" -r "$REALM" \
  | jq '.config["site.key"] = "6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI"
        | .config["secret.key"] = "6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe"
        | .config["recaptcha.v3"] = "false"' > "$TMP"

"$KCADM" update "authentication/config/$CONFIG_ID" -r "$REALM" -f "$TMP"
rm -f "$TMP"

Keycloak is the most direct CaptchaAI fit here: a plain HTML form, with g-recaptcha-response read from the POST and checked against Google's siteverify. The nuance is pageurl. The widget lives on the Keycloak host after your app redirects there, so CaptchaAI needs that URL (the helper reads page.url()), and the staging key's domain list must include the Keycloak hostname.

// keycloak-register.spec.ts: staging realm with a real reCAPTCHA v2 key
import { test, expect } from "@playwright/test";
import { solveCaptchaOnPage } from "./captchaai";

test("synthetic registration through Keycloak @real-captcha", async ({ page }) => {
  test.setTimeout(180_000);
  await page.goto("/"); // the app redirects to the Keycloak login page
  await page.getByRole("link", { name: "Register" }).click();
  const id = `e2e-${Date.now()}`;
  await page.locator("#username").fill(id);
  await page.locator("#email").fill(`${id}@example.test`);
  await page.locator("#firstName").fill("E2E");
  await page.locator("#lastName").fill("Synthetic");
  await page.locator("#password").fill(process.env.SYNTHETIC_USER_PASSWORD!);
  await page.locator("#password-confirm").fill(process.env.SYNTHETIC_USER_PASSWORD!);
  await solveCaptchaOnPage(page); // last, so the token is fresh
  await page.locator('#kc-register-form [type="submit"]').click();
  await expect(page).toHaveURL(new RegExp(process.env.APP_ORIGIN!));
});

Delete the user afterwards with kcadm.sh delete users/<id> -r <realm>, keep it to one registration per run, and see registration flow testing for sign-up test design. A v3 key would need version=v3 and the step's action (register by default), which this helper does not send.

Where each test runs, and what it costs

Keep those two alerts apart. A CaptchaAI error code means the check itself is broken. An accepted solve followed by a rejected sign-in is the signal that real users may be locked out too.

CaptchaAI bills thread-based plans with unlimited solves per thread, not per solve. BASIC ($15/month, 5 threads) covers a production monitor plus a nightly staging suite with room to spare, since each check holds a thread only while its solve runs. Current plans are on the pricing page.

Troubleshooting

FAQ

Can I just turn the CAPTCHA off in tests?

In CI, yes. Keep one staging job with the real widget, because only a real key catches a wrong domain list, a CSP header that blocks the widget, or a secret that was never set.

What about the hCaptcha option in Supabase, Better Auth or Auth0?

CaptchaAI does not currently solve hCaptcha or Arkose (FunCaptcha). Use the vendor's test keys, or Auth0's IP AllowList, in every environment.

Why not call CaptchaAI from an Auth0 Action or a Server Action?

Both run server-side after the browser has posted the form, so a token they fetched would arrive too late to be part of that request, and code that solves its own CAPTCHA switches the check off for every visitor. The solve belongs in the test runner, which fills the field before it submits.

Next step: add one real-CAPTCHA check to staging

Keep CI on test mode, then add a single @real-captcha staging test with the helper above wherever a solve fits: Auth.js, Keycloak, Better Auth and Supabase with Turnstile, and Auth0 with your own Turnstile partial.

Get a CaptchaAI API key for your staging sign-in checks