Overnight Agents: The AI Agent Merge Gates That Define Done

Picture the queue at 7:50 a.m.: fourteen pull requests that did not exist when you closed the laptop, each one green in the coordinator’s summary, each one described as “done.” Some of them are. One bumped a lockfile to make a test pass. The coordinator is not lying to you. It is reporting its own definition of finished, and its definition is not the one your production branch needs.

This is a runbook for AI agent merge gates: the written definition of “done” that CI enforces while you sleep. By the end you will have six gates in branch protection, where the agent cannot argue with them: tests green, a diff-size ceiling, a secret scan, path rules for lockfiles, infra, CI config and auth code, an eval threshold with margin, and a human approval on anything that touches production. You will also have a morning routine with an order and a timebox.

The one-line version: “coordinator finished” is a status, “safe to merge” is a verdict, and only the gates issue verdicts.

Cursor Projects, Sep 10: the laptop closes and the pull requests keep coming

Cursor shipped Projects in beta on Sep 10, 2026 (changelog; blog). The line that matters here is about where the work runs: “A Project runs on its own computer in the cloud, so closing your laptop doesn’t stop it.” The coordinator delegates to agents that implement, “brings the finished work back to you to check,” and can “follow all your PRs, fixing CI and acting when they open or merge.”

Then the sentence about review, written about migrations: “Early on, you review each PR closely. As the fixes hold up, you review less, and the coordinator keeps working through the migration on its own.” That is a trust ramp. The instrument that measures whether the fixes are holding up is yours to build.

Screenshot: Cursor blog, “Introducing Projects” (Sep 10, 2026), captured Sep 13, 2026.

The pattern is not one vendor’s. Cursor’s Aug 19, 2026 changelog already had cloud agents that “automatically subscribe to PRs they create,” “fixing CI and addressing bot comments.” OpenAI’s Agents API went to public beta on Sep 10, 2026 with cloud agents “fully managed by OpenAI” (announcement), and a cron job around codex exec does the same job with less ceremony on infrastructure you control (non-interactive mode). Same operating problem: an agent works unattended and opens pull requests while nobody reads them. The news stops here.

Why “coordinator finished” is not a merge signal

A chatbot that gets a migration wrong hands you a wrong paragraph. An agent that gets it wrong opens a pull request, and if your branch rules let a green check stand in for a decision, it merges. Agents act, and overnight they act without a witness.

The coordinator’s “done” is an internal state: the subagent returned, the diff exists, the tests it chose to run passed. A merge gate is a property of the change, evaluated by something the agent does not control, with a pass condition written before the run started. Every step below restates that principle.

The test harness for agentic software and the measurement guide for agents already cover how to measure; this piece is about what a measurement is allowed to unlock.

AI agent merge gates, written as six checks

Write the gate table before you touch a workflow file. It is the design-review artifact, the thing you argue about with whoever wants the agent to merge lockfile bumps “because they’re trivial.”

Gate	Pass signal	Fail signal	Who can override
1. Tests green	Suite passes on the merge commit; test count held	Any failure; skipped or deleted tests; coverage under the floor	Nobody; fix the change
2. Diff ceiling	≤ 400 changed lines and ≤ 15 files (illustrative)	Over either limit; three PRs from one agent on the same files in an hour	A human adds ceiling-waived with a reason
3. Secret scan	Zero findings from scanner and push protection	Any finding; a new file matching .env, *.pem, *credentials*	Nobody; rotate first
4. Path rules	No touched path in the always-human classes	Lockfile, infra module, workflow file, or auth path touched	Code owner approval only
5. Eval threshold	Scope judge above threshold with margin; no regression in the agent-behavior suite	Score under threshold; judge disagreement on re-run	A human, after reading the judge’s rationale
6. Human approval	One code-owner approval on the most recent push	No approval; approval older than the last commit	Nobody

Three gates are deterministic, two are policy, and one is probabilistic (the eval), which is why it sits fifth and never alone. The cheap gates run first so the expensive judge only sees survivors.

Wire AI agent merge gates into branch protection before the first overnight run

Gates that live in a workflow the agent can edit are suggestions. Gates that live in branch protection are rules. Put the required checks and the review rule at the repository level, where changing them takes admin rights and shows up in the audit log (docs.github.com).

The shape below is illustrative, written with the current ruleset field names in GitHub’s documentation; adapt the status-check contexts and repository policy before you apply it.

{
  "name": "agent-merge-gates",
  "target": "branch",
  "enforcement": "active",
  "rules": [
    {
      "type": "required_status_checks",
      "parameters": {
        "strict_required_status_checks_policy": true,
        "required_status_checks": [
          { "context": "tests" },
          { "context": "diff-ceiling" },
          { "context": "secret-scan" },
          { "context": "path-rules" },
          { "context": "eval-threshold" }
        ]
      }
    },
    {
      "type": "pull_request",
      "parameters": {
        "required_approving_review_count": 1,
        "require_code_owner_review": true,
        "dismiss_stale_reviews_on_push": true,
        "require_last_push_approval": true
      }
    }
  ],
  "bypass_actors": []
}

Three settings carry the weight. strict_required_status_checks_policy means the checks must pass on the merge commit, not on a commit from before the agent rebased. dismiss_stale_reviews_on_push plus require_last_push_approval mean an agent that pushes “one more fix” after you approved has to wait for you again. And bypass_actors is empty: no bot identity, no coordinator, no “automation” team. Add one and you have built a door and handed the agent the key. I have never regretted an empty bypass list.

Gate 1 is your existing suite with one addition: the tests check compares the test count on the PR to the base branch and fails on a decrease. An agent that makes a red suite green by deleting the red test has done the most natural thing in the world, and only the count notices.

Gate 2: a diff-size ceiling the agent cannot negotiate with

Small diffs are the only kind a human can review at the rate a coordinator produces them. Treat the ceiling as a reviewer-throughput constraint before it is a code-quality opinion, and it becomes easy to defend. Thresholds below are illustrative.

BASE="origin/${GITHUB_BASE_REF:-main}"
LINES=$(git diff --shortstat "$BASE"...HEAD -- . ':(exclude)**/*.snap' ':(exclude)**/*.lock' \
        | awk '{print ($4+0)+($6+0)}')
FILES=$(git diff --name-only "$BASE"...HEAD | wc -l | tr -d ' ')
if [ "${LINES:-0}" -gt 400 ] || [ "$FILES" -gt 15 ]; then
  echo "diff ceiling exceeded: ${LINES} lines across ${FILES} files; split it or ask for ceiling-waived"
  exit 1
fi

Exclude generated files from the count or the ceiling fires on noise and the waiver becomes routine. Keep the waiver a human action with a reason in the label comment. A coordinator that splits one 1,200-line change into four 300-line PRs has learned to route around the gate; see the failure modes below.

Gates 3 and 4: secret scan, then path rules for lockfiles, infra, CI, and auth

Run the secret scan as its own required check, separate from tests, so a green suite never hides a red scanner. Use a scanner in CI (gitleaks and trufflehog are the usual choices) and turn on push protection so a secret is refused at push time, before a PR exists (docs.github.com). Add a path check for files that should never appear in an agent’s diff: .env, private keys, anything named like credentials. The remedy for a hit is rotate first, review second; a secret that sat in a PR overnight is leaked whether or not anyone read it.

The always-human list belongs in CODEOWNERS, because branch protection already knows how to require an owner’s review. An illustrative file:

/infra/                     @org/platform-oncall
/terraform/                 @org/platform-oncall
/.github/workflows/         @org/platform-oncall
/services/auth/             @org/security-review
/packages/*/crypto/         @org/security-review
package-lock.json           @org/platform-oncall
pnpm-lock.yaml              @org/platform-oncall
poetry.lock                 @org/platform-oncall
/db/migrations/             @org/data-oncall

The list is short on purpose: paths where a wrong change is expensive to undo or hard to see. A lockfile bump at 3 a.m. is a supply-chain decision. A workflow edit can rewrite the gates themselves. An infra module change is a production change wearing a code review’s clothes. The path-rules check fails the PR with a readable message when an owner-protected path is touched without owner approval, so the agent stops retrying and the morning queue shows why. The review-agent policy piece takes this list further, into who may merge each class.

Gate 5: an eval threshold with margin, never alone

Two evals belong here, answering different questions. The first is a scope judge on every agent PR: does the diff do what the task card asked, and nothing else. It is an LLM-as-judge call with the task description, the diff, and a rubric, producing a score and a rationale. Set the threshold with margin, because judge scores wobble between runs on an identical diff, and a threshold at the edge of the noise is a coin flip.

The second runs only when the PR touches the agent’s own behavior: prompts, tool definitions, harness config. AWS published the pattern on Sep 8, 2026: on pull request, deploy the agent, invoke a fixed prompt set, score the traces with built-in evaluators (GoalSuccessRate, Correctness, ToolSelectionAccuracy and the trajectory matchers), compare to thresholds, block the PR on regression, tear down (AWS). Their arithmetic is the budget line: “4 evaluators × 5 prompts = 20 judge calls per PR.” The sibling on agent regression gates in CI is the full runbook; here it is one required check, eval-threshold, where a fail blocks and a pass unlocks nothing by itself.

Screenshot: AWS Machine Learning Blog, “Automated agent evaluation with Amazon Bedrock AgentCore and GitHub Actions” (Sep 8, 2026), captured Sep 13, 2026.

Gate 6: the morning review, run as a queue instead of a skim

By 8 a.m. the deterministic gates have sorted the night’s output. The chart below is a modeled night, not measured data; most of the filtering should have happened before you sat down.

Illustrative funnel for one night of a coordinator-run migration. Modeled counts, not vendor data.

Run the morning as a queue, never as a scroll through notifications:

1. Blocked PRs first, by gate. Read the failure reason before the diff. A secret-scan hit gets rotated first; a path-rule hit gets the owner paged; a ceiling hit gets split or waived with a reason.
2. Passing PRs, smallest first. Read the scope judge’s rationale before the diff; it says what the judge thought the change was for, which is the fastest way to notice when it is for something else.
3. Approve means approve the push you read. With stale-review dismissal on, an agent that pushes after your approval resets the clock. That is a feature.
4. Timebox it. Twenty-five minutes, then the rest waits. A queue that always empties by 8:30 is being rubber-stamped, which the sibling on approval queue hygiene treats as its own incident class.

The only path to the protected branch runs through the morning review; the retry loop stays on the blocked side.

When a gate blocks a PR the agent will retry, since cloud agents subscribe to their own PRs and fix CI. Cap that: three attempts per PR, a unique failure signature per attempt, then a needs-human label and silence. The sibling on CI fix loops that do not thrash has the loop guards.

The “review less” ramp Cursor describes is real. Descend it by widening, per path class, which gates may unlock a merge without you, in a written policy that itself sits behind a code-owner review, rather than by reading faster. Docs and test-only PRs first. Small application code after a few weeks without a revert. Never the always-human list.

What breaks overnight, and the signal that tells you

The agent edits the gate. A PR that touches .github/workflows/, deletes a failing test, adds it.skip, or lowers a coverage floor will pass tests by definition. Signals: test count lower than the base branch; a workflow file in the diff; a negative coverage delta. Mitigation: the workflow directory is on the always-human list, and the tests check fails on a count decrease. Novee’s Black Hat 2026 findings (Aug 5–6, 2026) go further: a file an earlier workflow pass wrote can be an attacker’s input, and their mitigations include treating workflow-written files as untrusted and deterministic gates (Novee).

Ceiling gaming by PR splitting. Signals: several PRs from one agent on overlapping files within an hour; a description that says “part 3 of 4.” Mitigation: the ceiling counts an agent’s open PRs on the same files together, and a series is reviewed as one change.

Eval flapping. Signals: the same commit passes the scope judge on re-run after failing, with no diff change; pass rates drifting without a prompt change. Mitigation: margin on the threshold, two judge samples with agreement required, a weekly look at the score distribution. A gate is allowed to be conservative.

The morning queue becomes a rubber stamp. Signals: median time from open to approve under two minutes; one reviewer approving everything; approvals landing before the checks finished. Mitigation: the timebox, smallest-first order, and the rule that approval requires reading the judge’s rationale. If the queue is too long, the fix is fewer PRs per night rather than faster approvals.

Merge gates are operating-layer infrastructure, not a smarter prompt

Nothing in this runbook lives in the prompt. A coordinator asked nicely to “only open safe PRs” will still open the lockfile bump, because it cannot see the blast radius from inside the task. The gates live in the layer that runs whether or not the agent is having a good night: branch protection, required checks, a written path policy, a queue with an order. That is what agentic ops means in practice: the desk needs an operating layer, and the merge gate is the piece that faces the repository.

The evidence matters as much as the verdict. Every blocked PR should leave behind why, which check said so, and what the agent tried next, readable a week later when a merged change turns out to be wrong. The same discipline that lets you replay what a fleet did lets you tighten a gate with a reason instead of a feeling. The agentic CI/CD piece covers the pipeline around this one; the merge gate is where it meets the branch that ships.

FAQ: AI agent merge gates

What are AI agent merge gates?

AI agent merge gates are the checks an agent-written pull request must pass before it can merge, enforced by branch protection rather than by the agent. A set: tests green, a diff-size ceiling, a secret scan, path rules for lockfiles and infrastructure, an eval threshold with margin, and a human approval on the latest push.

Should an AI coding agent be allowed to merge its own pull requests?

Not on a branch that ships. Keep the agent’s identity out of every bypass list, require a code-owner review for protected paths, and dismiss approvals when the agent pushes again. Widen what a signed policy may merge without you per path class, starting with docs and test-only changes, and never for infrastructure or auth code.

Sources

• Cursor changelog: “Cursor Projects” (Sep 10, 2026); “Cloud Agents and Cursor Harness Improvements” (Aug 19, 2026)
• Cursor blog: “Introducing Projects” (Sep 10, 2026)
• OpenAI: “Introducing the Agents API” (Sep 10, 2026)
• OpenAI Codex docs: non-interactive mode, codex exec
• AWS Machine Learning Blog: “Automated agent evaluation with Amazon Bedrock AgentCore and GitHub Actions” (Sep 8, 2026)
• Novee: “Critical flaws in Anthropic, Google and OpenAI’s coding agents” (Black Hat 2026, Aug 5–6, 2026)
• GitHub Docs: rulesets, required status checks, CODEOWNERS, secret scanning and push protection