AI Watermark Detector Probability Calculator

Green-list watermarking biases a language model's sampling so that generated text carries a statistical signature invisible to readers. At each position, a hash of the preceding tokens seeds a pseudorandom split of the vocabulary into a green list of size gamma times the vocabulary and a red remainder, and a constant delta is added to the logits of green tokens before the softmax. Unwatermarked text hits green tokens at rate gamma by chance, so detection reduces to a one-proportion z-test: z equals the observed green count minus n times gamma, divided by the square root of n times gamma times (1 minus gamma). Because the numerator grows with n while the denominator grows with the square root of n, the z-score scales with the square root of text length. Typical published settings are gamma of 0.25 to 0.5 and delta of 1 to 4. Two trade-offs bound the method. A larger delta detects shorter passages but distorts word choice, and the perplexity penalty rises with it, which is why a quality impact figure sits alongside the detection probability here. More fundamentally, low-entropy text cannot be watermarked at all: where only one continuation is plausible, as in a quoted fact or boilerplate code, the sampler has no freedom to steer and no signal accumulates. Editing, paraphrasing, and round-trip translation dilute the green fraction roughly in proportion to the tokens changed, and the temperature term used here is a simplified stand-in for per-step entropy rather than an exact model of it.

Hiding a signal inside a carrier is old practice; the word watermark comes from papermaking, where wire designs in the mould left thinner, translucent marks visible when a sheet is held up to light, a technique recorded in Italian mills from the late thirteenth century. Digital watermarking of images and audio became an active research field in the 1990s, driven by copyright enforcement, and linguistic steganography experiments in the 2000s tried to hide bits in text through synonym substitution, but results were brittle because natural language offers few genuinely lossless substitutions. The modern approach for language models arrived with A Watermark for Large Language Models by John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein at the University of Maryland, posted in January 2023, which introduced the green-list rule and the z-statistic detector modelled here. Scott Aaronson had described a related scheme in 2022 while working at OpenAI, using pseudorandom Gumbel sampling keyed to a secret. Google DeepMind put SynthID-Text into production for Gemini and published its tournament sampling design in Nature in October 2024, the first large-scale live evaluation of the idea. Policy pressure followed the research, with the 2023 United States executive order on artificial intelligence and the EU AI Act both pushing toward machine-readable marking of synthetic content.