Patterns for reviewing whole projects across languages and frameworks. Each case asks the same production question: can untrusted input or the wrong actor reach a sensitive sink?
Legacy invariant map 21 seeded Node.js cases. The baseline preserves these failure modes and structural tests. Current comparative claims come from the separate v2 evaluation, not from the case count itself.
Legacy baseline 21 invariants One intentionally vulnerable Node.js service. Its tests preserve routes and dangerous sinks; only three cases have exploit-level smoke tests. Browse the fixture.
Controlled v2 suite 5 language pairs Ten vulnerable/safe repositories, matched conditions, held-out fix verifiers, raw transcripts, token accounting, and repeated activation trials. Inspect the evaluation.
01
Authentication Trust
Signal Client-provided identity, demo headers, optional sessions, weak middleware order.
Check Confirm that the server owns identity and role claims before any sensitive handler runs.
Proof Tests reject forged headers, missing sessions, and downgraded middleware paths. 02
Authorization Dominance
Signal Guards exist, but they run after reads, writes, exports, queue jobs, or provider calls.
Check Trace the request until the sensitive sink and verify the guard dominates the sink.
Proof Tests fail when a user reaches the sink without the required role or permission. 03
Tenant Boundary
Signal tenantId is accepted from payloads, params, headers, cookies, or untrusted claims.
Check Verify tenant scope comes from trusted server context and is applied to every object query.
Proof Cross-tenant read, update, delete, export, and job tests return denied results. 04
Owner Scope
Signal Object lookup uses only id, slug, file key, token, or public-looking route names.
Check Verify the object belongs to the authenticated actor or an authorized shared scope.
Proof Another user's object id cannot be read, changed, deleted, previewed, or exported. 05
Mass Assignment
Signal Payloads flow directly into ORM create, update, merge, mapper, DTO, or model calls.
Check Allowlist mutable fields and block policy fields such as role, status, plan, price, quota, ownerId, and tenantId.
Proof Tests prove restricted fields are ignored or rejected across create and update paths. 06
State Transition Invariants
Signal Status changes, approvals, refunds, cancellations, publishes, and deletes lack transition rules.
Check Verify each state change has actor, current state, next state, and business rule validation.
Proof Invalid transitions and repeated side effects are rejected. 07
Route Exposure
Signal Preview, helper, asset, public, test, admin, debug, import, export, or internal routes are reachable.
Check Treat route names as unreliable and confirm exposure by framework routing rules.
Proof Public routes are intentionally public, protected routes deny anonymous access. 08
Side Effect Reachability
Signal A read-looking path sends email, enqueues jobs, charges money, creates records, or calls AI providers.
Check Classify by side effect, not HTTP verb or file name.
Proof Unauthenticated and low-privilege users cannot trigger costly or mutating work. 09
Fail-open Behavior
Signal Redis, cache, provider, queue, feature flag, or rate-limit failure allows privileged actions.
Check Decide which failures must deny, degrade, or retry, then enforce that branch explicitly.
Proof Dependency failure tests confirm protected actions fail closed where required. 10
Visibility Rules
Signal Draft, private, archived, deleted, embargoed, paid, or scoped content shares generic read paths.
Check Verify visibility predicates are attached to every list, detail, export, search, and preview query.
Proof Private and draft objects do not leak through alternate query paths. 11
Secrets and Unsafe Examples
Signal Real keys appear in local config, examples, docs, tests, deployment files, or generated output.
Check Separate placeholders from live values and remove tracked secrets before commit.
Proof Secret scans and manual review show only safe placeholders remain. 12
CORS and CSRF
Signal Broad origins, credentialed CORS, cookie auth, form posts, server actions, or mutating browser endpoints.
Check Verify origin policy, same-site behavior, CSRF tokens, and method constraints match the auth model.
Proof Cross-site mutation attempts fail while intended clients still work. 13
Rate Limits and Abuse Cost
Signal Public forms, login, password reset, quote builders, file conversion, AI calls, PDF rendering, and email sends.
Check Rate-limit by actor, IP, route, tenant, and cost unit where needed.
Proof Burst tests hit limits and provider spend cannot be amplified anonymously. 14
Malformed Input
Signal JSON, multipart, CSV, PDF, XML, markdown, URLs, emails, phone numbers, dates, and nested objects enter business logic.
Check Validate shape, size, type, encoding, ranges, nesting depth, and canonical form before use.
Proof Invalid payload tests fail predictably without crashes or partial writes. 15
Logging and Error Leakage
Signal Stack traces, provider errors, tokens, user PII, SQL, file keys, or prompts are logged or returned.
Check Separate developer diagnostics from user-visible errors and redact sensitive fields.
Proof Error-path tests confirm safe responses and safe logs. 16
Upload and Storage Boundaries
Signal User-controlled filenames, object keys, MIME types, previews, transformations, and delete paths.
Check Verify type checks, size limits, random object keys, tenant prefixes, malware controls, and private bucket defaults.
Proof Tests reject dangerous files and block cross-tenant storage access. 17
Signed URL Scope
Signal Signed reads, writes, previews, downloads, and deletes use predictable keys or weak ownership checks.
Check Bind signed URL generation to actor, object, tenant, operation, expiration, and visibility.
Proof A valid URL cannot be reused for another object, tenant, or operation. 18
AI, PDF, and Document Processing
Signal Prompt input, document upload, OCR, PDF rendering, templates, code execution, or external processors.
Check Bound file size, execution, network access, prompt injection impact, output trust, and provider cost.
Proof Malicious documents and prompts cannot exfiltrate secrets or trigger uncontrolled work. 19
Payments and Finance
Signal Price, discount, currency, tax, refund, subscription, quota, balance, invoice, or webhook state.
Check Trust provider events and server-side price tables, not browser payloads.
Proof Client-side price tampering and replayed finance actions are rejected. 20
Webhook Authenticity
Signal Third-party events update accounts, payments, usage, emails, CRM records, or order state.
Check Verify signatures, timestamps, replay protection, idempotency, event ownership, and state transition rules.
Proof Unsigned, stale, replayed, and cross-account webhook events fail. 21
Naming Bias
Signal Names like public, helper, preview, demo, asset, safe, internal, lead, quote, temporary, or test reduce suspicion.
Check Ignore friendly names and review actual reachability, authority, object scope, and side effects.
Proof Findings are based on source-confirmed behavior, not on labels or filename intuition. Secure. Source-backed production security review, authorized hardening, verification, and scoped coverage reporting.
Back to home