Lesson 15.815.8 Checks that block

In everyday life. A smoke detector beeps. A circuit breaker cuts the power. Both notice a problem, but only one stops anything. Each is right for its job. The mistake is to believe the beeper is a breaker.

What you will learn. The difference between a check that stops a merge and one that only reports, and how to make the right ones stop it.

Why it matters. A gate is a check that can really stop a merge. A team that mistakes a report for a gate has no gate.

Which checks stop a merge?

Only the ones the repository requires. Every check on a pull request is one of two kinds.

• An advisory check is one that only reports. It may post findings, but the merge button still works.
• A blocking check is one that stops the merge. The pull request cannot join main until it passes.

A check blocks only when the repository says so. Branch protection, or a ruleset, its newer and more flexible form, is a set of rules on a branch such as main. The rules that matter here:¹

What does a workflow do?

It runs the checks. A workflow is a file in .github/workflows/ that tells the host which checks to run, and when. An event, such as a new pull request, starts it. It is made of jobs, and each job is a list of steps that run in order. A job stops at its first failing step, so a later step in the same job never runs. Each job shows on the pull request as one check. Marking a check required in branch protection turns it from advisory into blocking. Running checks like this on every change is called continuous integration, or CI.

What are Brightline's required checks?

One workflow runs everything a machine can check every time, in three jobs, so a failing test cannot hide a weakened test or a printed key:

Each script fails closed: if it cannot do its job, because it cannot find the base branch, the one the pull request merges into, or a folder it should search, it fails instead of passing. A check that passes when it cannot see is worse than none, because it looks like protection.

What do these checks miss?

A lot, on purpose. Each script catches one common mistake, cheaply, on every change. Neither proves more than that. The test check cannot tell whether a new test checks anything useful, or whether the test runner's settings were changed to skip tests. The secret check misses a logging call split over two lines, or a key in a variable whose name looks harmless. Review covers what they miss. A check is only as strong as the files that run it. So the check scripts, the workflow, package.json and the type checkers' settings all have a named owner (15.9). Here is part of the workflow:

permissions:
  contents: read          # the workflow token can read the code, nothing else
jobs:
  checks:                 # the type checks and tests
    steps:
      - run: npm run typecheck
      - run: npm test
  tests-unchanged:        # a separate job, so a failing test cannot hide it
    steps:
      - run: bash scripts/check-tests-unchanged.sh "origin/${{ github.base_ref }}"

Figure 15.13. One job stops at the first failure. One job per check lets every problem report.

Can an AI review block a merge?

Only through the repository. An AI review is advisory by default. It never approves or blocks on its own (15.7). If you want open findings to stop a merge, choose one way to make the repository enforce it. Require conversation resolution, so an unresolved finding's thread blocks the merge. Or add a required job that reads the review's result and fails on an Important finding. Or run a security review inside your own workflow, with your own step that fails when it reports findings. A review in a workflow is the same agent outside the app that you met in 12.8: one task, run headless, with read-only access and nobody watching. The rule is the same in every case: the repository's settings do the blocking, not the reviewer.

What does the pull request template do?

A file, .github/pull_request_template.md, fills every new description with the questions a reviewer needs answered. They ask what and why, how it was tested, with the commands and their output, the level of risk, and how to undo it. It costs the author a minute and saves the reviewer from guessing.

How do I keep a workflow safe?

A workflow runs code from the pull request, so treat it as running untrusted code. Give its token, the key it uses to act on the repository, read-only permissions. Keep keys in repository secrets, which the host stores and gives to workflows.³ Never commit them to a file. The host gives no secrets to workflows triggered by a fork, an outside copy of the repository. Never use a trigger that runs with your secrets, such as pull_request_target, to build or run a pull request's code.⁴ And when a check reads labels, such as tests-changed, make the workflow run on the labeled and unlabeled events too, or adding the label changes nothing until the next push.

In the case study. No pull request merges while a finding from the bug review or the security review is open. Lint, type checks, tests and builds are required. The template asks how a change was tested and how to undo it.⁵

At work. Ana makes all three jobs required on main, with conversation resolution on. The agent-speedup pull request shows three red marks, one per job. The tests fail on the cutoff case. The test comparison fails on the edited expected value. The secret check fails on the debug line that prints the key. The merge button is gray. Nobody had to read the diff to stop it, though someone still will, to learn why an agent did this.

The operating rule: make every check a machine can run a required check, and never mistake an advisory report for a gate.

Figure 15.14. Advisory feedback and merge requirements. Only enforced requirements control the merge gate, and the repository's settings decide what they are.

Practice it. Practice: checks that block, in your own tool, fifteen minutes.