Unsolicited PRs arrive looking ready to merge -- linked to nothing, from accounts that will not stick around for review. A share of that volume is good-faith work that simply used a model to move faster, a share is spray-and-pray, and a share is badge collecting: a contribution aimed at a high-profile organization repository so the org shows up as a badge on the author's GitHub profile.
I do not try to detect AI or filter AI-based code, and I am fine with PRs that disclose responsible use of a model. What I try to do is load-balance attention so projects I lead stay reviewable now and maintainable over the long term. The tooling story starts on OWASP Nest, then moves through two GitHub Actions: check-pr-issue-action(public archive) and its successor, check-contribution-action, which is multi-check maintainer DX.
Nest as the problem spot#
OWASP Nest is a community platform with steady contribution traffic: product work, Dependabot, mentorship-shaped proposals, and drive-by PRs. The heat is highest around Google Summer of Code-- a paid internship, so the volume has a monetary side. Nest is not the only place I have seen this; I have watched the same seasonal spike on other projects for years. The scarce resource is not CI minutes but the limited time that open source volunteering implies: maintainer context switches -- reading, commenting, closing, then reopening the same conversation with the next account.
Unsolicited AI slop thrives when opening a PR is free and owning an agreed problem is optional. Without an issue there is no shared definition of done, and if the author is not on the hook for the issue they have not taken a place in the project's priority queue.
The defined Nest workflow is blunt on purpose:
- Find or open an issue that describes the work.
- Get assigned to that issue by a maintainer.
- Only then start coding and open a PR that references the issue.
Assignment is the pre-approval step. It means a human already agreed this work belongs on the project now, and that you are the person expected to deliver it. The PR template says it in all caps: if you are not assigned, stop -- unassigned PRs get closed automatically. CONTRIBUTING draws the same path in the workflow diagram: get assigned before resolve.
That rule is what makes automation honest. Once "assigned to the issue" is part of the social contract, a workflow can check that the PR author is also an issue assignee. A match means the PR is expected, pre-approved work; a miss means drive-by volume that never cleared the queue. The docs already describe the path, and the automation checks that the author followed it.
PR templates are not new -- they have been a standard way to shape incoming contribution requests for years -- and they still exist, even though drive-by contributors still ignore them. The first indicator of a low-quality PR is often the leftover instruction block: the author did not read the template, and did not remove the all-caps heading that told them to delete it.
check-pr-issue-action: the assignee gate#
arkid15r/check-pr-issue-action(public archive) was the first cut, and Nest used it for a long stretch. Scope was intentionally narrow: one job, done well enough to prove the social contract.
What it actually checked:
- Is the PR linked to an issue?
- Does the issue assignee match the PR author?
- Skip bots / a skip list; warn or close when the bar is missed
It was an assignee gate with an issue-reference prerequisite: no orphan PRs, and no PRs from people who never took the issue. Linking without assignment still lets someone spray against a public backlog. Requiring the author to be an assignee closes that hole -- the bot is not guessing intent, it is checking that the project already put this person on this ticket.
What it did not do: DCO trailers, commit signatures, selective "close this failure but only comment on that one", a composable check menu, or safer pull_request_target packaging as a first-class maintainer DX story. When Nest needed those, the right move was a successor GitHub Action.
Improved version: check-contribution-action#
arkid15r/check-contribution-action keeps the assignee match and adds more checks for maintainer DX.
Compared with the first GitHub Action:
check-pr-issue-action | check-contribution-action | |
|---|---|---|
| Core question | Is this PR's author the issue assignee? | Same question, plus which other bars does this repository need today? |
| Shape | Fixed / narrow gate | Menu of checks via check_for |
| Failure policy | Mostly warn-or-close as a blob | Per-check close_on -- auto-close the worst misses; comment-only on others |
| Authorship trail | Out of scope | Optional DCO Signed-off-by and GPG/SSH commit signatures |
| Trusted actors | Skip list / bots | Skip file for leaders and automation |
| Wiring hygiene | Ad hoc per consumer | The wiring Nest uses: pull_request_target, do not check out PR head, pin a release SHA |
Nest's live settings (GitHub Action pin v0.1.7) look like this:
# From OWASP/Nest .github/workflows/check-contribution.yaml
uses: arkid15r/check-contribution-action@f90721e95e8b5d7f32b8e565552498d51d052ca5 # v0.1.7
with:
check_for: commit_sign_off, commit_signature, issue_assignee, issue_reference
close_on: issue_assignee, issue_reference
github_token: ${{ secrets.GITHUB_TOKEN }}
skip_users_file_path: .github/check-contribution-skip-usernames.txtAgainst slop and load, those settings mean:
| Check | Intent |
|---|---|
issue_reference | PR must point at a real issue (GitHub link or closing keywords) |
issue_assignee | PR author must be an assignee on that issue -- expected, pre-approved work (the original gate, still required) |
commit_sign_off | DCO Signed-off-by trailer -- authorship trail |
commit_signature | GPG or SSH signed commits where the project wants that bar |
Selective close_on | Nest auto-closes on missing issue or assignee; other failures can stay comment-only so maintainers are not babysitting every red X |
| Skip file / bots | Trusted automation and project leaders skip the human gates |
issue_reference and issue_assignee remain a pair -- that is the lineage from the initial GitHub Action. Everything else is maintainer DX on top: teach the next step in the failure comment, stop repeating the same review note, keep authorship and signature policy in CI instead of tribal knowledge, and let each repository pick which bars are merge-blocking.
These checks do not "detect AI"; they detect missing intent and missing ownership. Good-faith AI-assisted work that follows CONTRIBUTING, gets assigned, links the issue, and can defend the patch still passes. Undisclosed spray that skips the path fails fast -- which is the load-balancing part.
A gate that only turns the PR red, with no why and no next step, still leaves the teaching on the maintainer. A gate that names the next step is mentoring at scale: link the issue, get assigned, sign off, resubmit -- cheaper than repeating the same review comment a hundred times.
What comes next#
check-contribution-action is meant to keep growing as a menu of checks, not a single opinionated bot. The next capabilities are not shipped yet; they should follow real-world requirements as a project matures -- more contributors, more privileged paths, a higher bar on authorship -- so the menu can grow with that load:
- Semantic commits -- require Conventional Commits (or a project-configured pattern) so history stays reviewable and changelog automation stays honest. Sloppy commit subjects are a soft signal; enforcing a format is a cheap gate that also helps humans skim PRs.
- Sensitive-path authorization -- treat supply-chain and CI/CD surface area as privileged. Paths like
.github/workflows/, Dependabot config, release tooling, lockfile-only policy files, and other build/provenance glue should not be mergeable from a random assignee by default. The check would fail (or require an explicit maintainer co-author / review) unless the actor is in a maintainer allowlist. Issue assignment is enough for product code; it is not enough to rewrite how the project builds and ships.
These are different kinds of maturity: semantic commits are hygiene so history stays easy to skim, and sensitive-path rules are privilege, because clearing the issue gate is not a license to edit CI/CD.
I am deliberately keeping this as roadmap language. Shipping a path gate wrong is worse than not having one: false confidence around workflows is how you get surprise pull_request_target foot-guns. When those checks land, they should stay opt-in via check_for / close_on, with clear failure comments that name the next step.
Beyond Nest: Nettacker, OWHF#
The same attention problem shows up in different costumes. OWASP Nettacker is an automated penetration testing and information-gathering framework, and unsolicited diffs are not free when the software is something people run against real targets. Trust and review quality matter as much as merge velocity.
The Open World Holidays Framework(OWHF; holidays on PyPI) is a different product lane. A decent country or subdivision PR usually demands calendar research: statutes, gazette notices, regional exceptions, movable feasts. That time investment is natural friction. You cannot produce a credible OWHF patch in thirty seconds -- the calendar research still has to be real.
So the slop problem is less severe on OWHF than on Nest, though it is not zero. We still see auto-generated descriptions, invented rules, and confident wrongness from time to time -- especially in pre-GSoC season, when people hunt for portfolio commits and models make volume cheap.
That seasonal pattern matters: gates are not hostility to newcomers -- they are how the commons survives the weeks when incentive and tooling align toward noise.
Not only custom GitHub Actions#
Contribution gates are one slice of how I try to simplify life for myself and the other maintainers I work with, not the whole stack.
On the AI side, PR-review products such as CodeRabbit and Cubic sit in the review thread and catch a class of "looks fine, is wrong" issues before a human has to. There is a real failure in that loop: an AI-based tool requesting changes on AI-generated or AI-assisted code is two models talking to each other. The thread can look like review while the ownership gap stays open. On the pre-AI side, static analysis platforms like SonarQube still earn their keep for smells, bugs, and security hotspots at repository scale. For open-source projects, those offerings are typically free -- a rare case where the commercial tooling market subsidizes maintainer attention instead of taxing it.
None of that replaces the local and CI/CD baseline we already run hard: pre-commit, Ruff, ESLint, Prettier, Semgrep, and whatever else the project's CI already encodes. The point is layering: formatters and linters keep the diff boring, Semgrep and Sonar catch classes of defect, AI reviewers skim for logic and review nits at PR time, and contribution gates decide whether the PR was expected work in the first place. They are different jobs, and together they are how a small maintainer set survives a large tooling surface without drowning in noise.
Contribution gates encode a social contract: issue first, assignment second, then code. Assignment is how the project says "yes, start", and the GitHub Actions check that the PR author is that assignee so the queue stays intentional. AI changed the cost of producing the third step; the workflows raise the cost of skipping the first two, so maintainers are not the only firewall.