The rule catalogue & its limits¶

What String actually does. This page sets out every pattern redact.String applies, what each one replaces, the reasoning behind the conservative opaque-token fallback, and — just as importantly — what the catalogue does not catch.

Every claim here is drawn directly from the patterns in redact.go. All patterns are compiled once at package init and use Go's RE2 engine — no backtracking, so matching is linear in the length of the input.

How String is applied¶

redact.String runs its rules in a fixed order over the input, each pass operating on the output of the last:

1. URL userinfo
2. Credential query parameters
3. Authorization-header tokens
4. AWS secret-key assignments
5. JSON Web Tokens
6. Well-known provider prefixes (each prefix in turn)
7. The long-opaque-token fallback

Because the specific, high-confidence rules run before the broad fallback, a token that a named rule already masked is never re-examined by the fuzzy pass.

The rules¶

URL userinfo¶

Matches scheme://user:pass@ for http/https URLs and replaces the userinfo with <redacted>, preserving the scheme and host so the URL stays recognisable.

https://admin:hunter2@api.example.com/v1  →  https://<redacted>@api.example.com/v1

Only http and https schemes are matched, and only the full user:pass@ form.

Credential query parameters¶

Case-insensitively matches these parameter names and masks the value (everything up to the next & or whitespace) with ***, keeping the parameter name:

apikey, api_key, key, access_token, refresh_token, token, secret, password, auth, authorization, signature.

?api_key=sk-abc123&user=bob  →  ?api_key=***&user=bob

The auth and authorization names are included here so a stray ?authorization=sk-abc123 is caught even when it does not also match the Authorization-header rule below.

Authorization-header tokens¶

Case-insensitively matches an authorization: header written into free text (an error message, a log line) followed by a scheme — Bearer, Basic, Digest, or ApiKey — and masks the credential, keeping the header name and the scheme:

Authorization: Bearer eyJhbGciOi…  →  Authorization: Bearer ***

AWS secret-key assignments¶

AWS secret access keys carry no distinguishing prefix, so they are matched by their assignment form — aws_secret_access_key or secret_access_key followed by = or : — rather than by value. Matching a bare 40-character value would also scrub git SHA-1 hashes. The key name and separator are preserved; the value becomes ***:

aws_secret_access_key = wJalrXUtnFEMI…  →  aws_secret_access_key = ***

JSON Web Tokens¶

Matches the distinctive eyJ… three-segment base64url shape and replaces the whole token with <redacted-token> — there is no meaningful prefix to keep. The eyJ header prefix keeps false positives low, and this rule catches bare Bearer eyJ… tokens that the Authorization-header rule would miss.

eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.dozjgNryP4J  →  <redacted-token>

Well-known provider prefixes¶

These formats are unambiguous: the prefix followed by enough alphanumeric content is almost certainly a real credential. Each match keeps its literal prefix for debug readability and replaces the remainder with ***. Crucially, the number of characters kept is anchored to the known prefix, never discovered from the matched text — so a token body that happens to contain _ or - cannot leak.

The long-opaque-token fallback¶

Anything left that is a single run of ≥ 41 characters from [A-Za-z0-9_-] is replaced whole with <redacted-token>. This is the catch-all for high-entropy secrets that carry no recognisable prefix.

Why 41 characters¶

The fallback threshold is deliberately high to keep false positives near zero. Common non-secret opaque strings are all shorter than 41 characters:

A UUID, an MD5 digest, or a git SHA — the identifiers that most often appear in error strings and log lines — sit at or below 40 characters and pass through untouched. SHA-256 (64 chars) will be redacted; that is an accepted tradeoff, on the grounds that raw hashes rarely appear in the kind of free-form strings this module sanitises, and masking one is harmless where leaking a 41-plus-char secret is not.

The threshold errs toward precision over recall: it would rather miss an unusually short opaque secret than corrupt a legitimate identifier.

Replacement tokens¶

String uses a small, fixed set of replacements, which keeps the output bounded — it never pathologically grows the input:

Guarantees¶

String holds three invariants, enforced by fuzz testing:

• Never panics on any input.
• Idempotent — String(String(s)) == String(s), so redacting an already-redacted string is safe.
• Bounded growth — the output is never more than the input plus a small constant, so redaction cannot blow up the size of a log line.

Limitations¶

A pattern catalogue never reaches 100 % recall. Design around these gaps:

• Non-standard formats slip through. A bespoke or internal credential format the catalogue has never seen will not match. Where you handle such inputs, redact them upstream — do not rely on this module alone.
• Short opaque secrets slip through. A high-entropy secret shorter than 41 characters is only caught if it appears in a recognised assignment (a credential query parameter, an Authorization header, an AWS assignment) or carries a known prefix. A bare short token in free text will not match the fallback.
• ASCII only. Every pattern uses ASCII character classes. Virtually all real-world provider tokens are ASCII, and UTF-8 credentials in the wild are vanishingly rare — but a non-ASCII secret will not be matched.
• URL userinfo is HTTP(S) only. Userinfo in other schemes (e.g. postgres://, redis://) is not matched by the userinfo rule, though a long enough token in one may still trip the fallback.

The takeaway is the same as the threat model: treat redact.String as a boundary safety net that catches the common accidents, and keep stripping known secrets at the source for anything it cannot recognise.

Related¶

• Threat model — why boundary redaction exists.
• Redact at the boundary — applying the rules in practice.