Using Codex as a Security Review Agent

A codex security review uses the OpenAI Codex CLI as a focused reviewer that reads a diff, a PR, or a whole repository and reports concrete vulnerabilities instead of style nits. It is not a replacement for SAST tools or a real pentest, but it closes a gap those tools miss: it understands intent, traces data flow across files, and explains exploitability in plain language. This guide covers setting it up, writing prompts that produce useful findings instead of noise, and wiring it into CI so every pull request gets checked before a human ever opens it.

Why Use Codex for Security Review At All

Traditional static analysis tools are pattern matchers. They flag eval(), string-concatenated SQL, or a hardcoded string that looks like a key, and they do it fast and deterministically. What they cannot do is reason about whether a particular code path is actually reachable by an attacker, whether an authorization check three files away covers this endpoint, or whether a "safe" library call is being used in an unsafe way given the surrounding logic.

Codex, run as an agent rather than a one-shot completion, can open multiple files, follow a function call across modules, check how a parameter is validated upstream, and then explain the finding in terms of impact: what an attacker would need, what they would get, and how bad it is. That reasoning step is the actual value. A codex security review is best thought of as a first-pass human reviewer with infinite patience and no context-switching cost, not a rules engine.

The realistic use cases are:

• Pre-merge review on every PR touching auth, payments, file uploads, or user input handling
• A standing local habit: run a review before you open a PR, not after
• Triage of an unfamiliar codebase you just inherited
• A second opinion after a SAST tool fires, to cut false positives before they hit a human queue

Installing and Configuring Codex CLI

Install the CLI and authenticate once:

npm install -g @openai/codex
codex login

Codex reads project-level instructions from an AGENTS.md file at the repo root, the same convention used by other coding agents. This is where you tell it what "security review" means for your codebase specifically, rather than relying on generic training knowledge.

Create or extend AGENTS.md:

# Security review conventions

When asked to do a security review, focus on:
- Injection: SQL, NoSQL, command, template, and LDAP injection
- Auth: missing or incorrect authorization checks on API routes
- Secrets: hardcoded keys, tokens, or credentials in source or config
- SSRF: any outbound request built from user-controlled input
- Deserialization of untrusted data
- Insecure direct object references (IDOR) in REST or GraphQL resolvers
- Unsafe file uploads (path traversal, missing type/size validation)

Ignore:
- Formatting, naming, and style issues
- Missing tests unless the untested path is the vulnerable one
- Dependency version bumps unless a known CVE is in play

For every finding report: file and line, the vulnerable code snippet,
the exploit scenario in one sentence, and a concrete fix.

This file is loaded automatically on every codex invocation in the repo, so you write the review policy once and every future run, local or in CI, follows it.

Running a One-Off Review Locally

For a quick check before opening a PR, run Codex against your working diff:

git diff main...HEAD > /tmp/review.diff
codex exec "Read /tmp/review.diff and do a security review per AGENTS.md. \
Only report real, exploitable issues. For each one give file:line, \
severity (critical/high/medium/low), and a fix."

codex exec runs Codex non-interactively with a single prompt and prints the result, which makes it scriptable. For an interactive session where you want to ask follow-up questions about a finding, drop the exec and just run codex in the repo, then paste the same prompt.

If you want Codex to look at the whole repository rather than just a diff, point it at the directory instead:

codex exec "Do a full security review of the src/api directory per \
AGENTS.md. Prioritize the payment and auth modules. List findings \
ordered by severity."

Codex CLI runs in a sandbox by default and will ask for approval before running commands or writing files. For a read-only review you do not need write access at all, so run it in the most restrictive mode:

codex exec --sandbox read-only "Security review of src/api per AGENTS.md"

This guarantees the agent cannot modify anything while it reviews, which matters when you are pointing it at a real repo with real state.

Writing Prompts That Produce Signal, Not Noise

The single biggest failure mode with an LLM-based reviewer is over-reporting: forty comments, most of them theoretical, on a fifteen-line diff. You control this with the prompt, not by hoping the model calibrates itself.

Three rules that consistently cut noise:

1. Force a severity bar. Tell Codex to only report issues it would block a merge for, and to explain why lower-severity issues were excluded rather than silently dropping them:

codex exec "Security review this diff. Only report issues you would \
block a PR merge over. For anything you considered and dismissed, \
list it in one line under 'Considered, not blocking' with a reason."

2. Require an exploit path. A finding without a concrete "here is how an attacker triggers this" is usually noise:

For every finding, state the exact HTTP request or user action that \
triggers it. If you cannot describe a concrete trigger, do not report it.

3. Scope the review to what changed. A full-repo review on every PR is slow and re-surfaces pre-existing issues that were already accepted as risk. Scope to the diff, and only widen to full-file context when the diff touches a security-sensitive area:

Review only the changed lines. If a changed function calls into code \
outside the diff, read that code for context but do not report issues \
in it unless the diff itself introduces the problem.

Combine all three into one review prompt template and save it, so every invocation is consistent:

codex exec "$(cat security-review-prompt.txt)" < /tmp/review.diff

Example: Catching a Real IDOR

Here is a concrete pattern Codex catches reliably that most SAST tools miss because it requires cross-file reasoning. Say a route handler looks like this:

router.get("/api/invoices/:id", requireAuth, async (req, res) => {
  const invoice = await db.invoice.findUnique({
    where: { id: req.params.id },
  });
  res.json(invoice);
});

requireAuth confirms the request has a valid session. It does not confirm the session's user owns this invoice. A pattern-matching scanner sees an authenticated route and moves on. Codex, told to check for IDOR, will trace requireAuth into its own definition, see that it only sets req.user and does not check ownership, then flag the missing where: { id, userId: req.user.id } clause and explain that any authenticated user can enumerate id values and read other customers' invoices.

The fix Codex proposes in this case is usually direct:

router.get("/api/invoices/:id", requireAuth, async (req, res) => {
  const invoice = await db.invoice.findUnique({
    where: { id: req.params.id, userId: req.user.id },
  });
  if (!invoice) return res.status(404).end();
  res.json(invoice);
});

That is the class of bug a codex security review earns its keep on: not "you used string concatenation," but "this authorization check exists but checks the wrong thing."

Wiring Codex Security Review into CI

Running the review manually works until people forget to run it. Put it in the pipeline so it runs on every pull request automatically. A minimal GitHub Actions job:

name: codex-security-review
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm install -g @openai/codex

      - name: Generate diff
        run: git diff origin/${{ github.base_ref }}...HEAD > review.diff

      - name: Run Codex security review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          codex exec --sandbox read-only \
            "Security review per AGENTS.md. Read review.diff. \
            Output findings as GitHub-flavored markdown with a \
            table: severity, file:line, description, fix." \
            > findings.md

      - name: Post findings as PR comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const body = fs.readFileSync('findings.md', 'utf8');
            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body,
            });

Two operational details matter here. First, run with --sandbox read-only in CI, since the job only needs to read code and post a comment, not modify the repo. Second, do not fail the build on findings by default. A hard gate on an LLM's judgment produces the same alert fatigue as any other noisy check, and teams turn it off within a month. Post the findings as a comment, let a human triage, and only add a hard gate later for a narrow, high-confidence subset, such as detected hardcoded secrets.

For a stricter gate on that narrow subset, run a second, separately scoped step:

codex exec --sandbox read-only \
  "Check review.diff ONLY for hardcoded secrets, API keys, or \
  credentials. Output PASS or FAIL on the first line, nothing else \
  if PASS." > secrets-check.txt

if ! head -1 secrets-check.txt | grep -q PASS; then
  echo "Hardcoded secret detected, failing build"
  exit 1
fi

Narrow, high-precision checks like a secrets scan are safe to gate on because false positives are rare and the cost of a leaked key is high. Broad "any vulnerability" checks are not, because the false positive rate is too variable to trust as a merge blocker.

What Codex Catches Well vs What It Misses

Codex security review is strong at logic bugs that require reading intent: missing authorization checks, incorrect trust boundaries, unsafe use of a correctly-imported library, and business logic flaws like a discount code that can be applied twice. It is also good at explaining why a finding matters in language a non-security engineer can act on, which is often the actual bottleneck in getting fixes shipped.

It is weaker at things that need runtime data: race conditions that only manifest under load, timing attacks, memory safety issues in compiled languages, and anything that depends on the exact behavior of an infrastructure component it cannot see, like a misconfigured load balancer or a cloud IAM policy. It also cannot verify a fix actually closes the hole; you still need a test or a manual check for that. Do not treat a passing codex security review as a substitute for a dependency vulnerability scanner, a secrets-in-git-history scan, or an annual penetration test. It is one more layer, positioned earliest in the pipeline where fixes are cheapest.

Comparing Codex Review to Traditional SAST

The honest comparison is complementary, not competitive:

• SAST tools are deterministic, fast, and cheap to run on every commit. Use them for known bad patterns, dependency CVEs, and license checks.
• Codex review is slower and non-deterministic between runs, but it understands context a rule engine cannot: whether a check exists elsewhere, whether an input is actually attacker-controlled, whether the "vulnerable" pattern is inside a test fixture that never runs in production.

Running both is the right answer for a serious team. Let the SAST tool block on its high-confidence rules, and let Codex catch the class of bug that requires understanding what the code is trying to do.

Best Practices for a Codex Security Review Agent

A few habits separate a useful setup from a noisy one that gets ignored after a week:

• Keep AGENTS.md specific to your stack. A generic "check for security issues" prompt produces generic, low-value output. Naming your actual auth pattern, your ORM, and your known-sensitive modules focuses the review.
• Review diffs, not entire repositories, for routine PR checks. Full-repo reviews are for periodic audits, not every commit.
• Require an exploit path in every finding, and reject findings that do not have one during prompt iteration.
• Do not hard-gate merges on broad findings. Post as comments, gate only on narrow high-precision checks like secrets detection.
• Re-run the review after a fix is applied, not just once. Confirm the specific finding is resolved rather than assuming the follow-up commit addressed it.
• Version your review prompt in the repo alongside AGENTS.md, so changes to review policy go through the same PR process as code changes.

FAQ

Is a codex security review a replacement for a human security engineer? No. It is a triage layer that catches a meaningful share of common vulnerability classes before a human looks at the diff, which reduces the volume a human reviewer has to work through. Anything security-critical, like auth architecture changes or new payment flows, still needs a human review, and larger organizations still need periodic professional penetration testing.

Will Codex report false positives? Yes, and the rate depends heavily on your prompt. A vague "find security issues" prompt produces more false positives than a scoped prompt that requires a concrete exploit path and a severity bar. Iterate on the prompt the same way you would tune any other check, and expect to spend real time getting it calibrated to your codebase before trusting it in CI.

Can Codex fix the vulnerabilities it finds, not just report them? Yes, Codex can propose and apply patches when run outside read-only sandbox mode, but for a review step you generally want it read-only so it cannot silently rewrite code you have not reviewed. Use a separate, explicit fix step, ideally in a follow-up commit a human approves, rather than letting the review step also make edits.

Does this work on private or on-premises code? Codex CLI sends the code you point it at to the OpenAI API to run the review, so check your data handling and compliance requirements before pointing it at a repository with sensitive customer data or regulated code. For fully air-gapped environments, this workflow is not appropriate without a self-hosted or enterprise agreement that covers data handling.

How is this different from asking ChatGPT to review a diff? The agent loop is the difference. Codex CLI can open additional files, run grep across the repo, and follow a function call to its definition before answering, rather than only seeing the text you paste into a chat window. That multi-file context is what makes it catch cross-file issues like the IDOR example above, which a single-turn chat review would miss entirely.

What is a reasonable cadence for running this? On every pull request that touches API routes, auth, payments, or file handling is the minimum useful cadence. Many teams also run a broader, full-repo version monthly or before a major release to catch drift that individual diffs missed.